Convert Hyper-V VM vhdx to VMware vSphere vmdk

This turned out to be pretty simple, if pretty slow. I was converting from a Hyper-V 2012 R2 VM into vSphere 5.0. The VM itself was also running Windows Server 2012 R2.

There are a few steps and points to note:

  1. Download and install StarWind Converter
  2. Make sure you have plenty of disk space – of the various vmdk formats that StarWind Converter can output, the only one that is compatible with ESXi hosts is a full fat file – so it’ll take up the full amount of space even if the original vhdx file was thin provisioned and consequently smaller.
  3. Make sure you have fast disks and network connections. I had an 850GB VM to convert and even using 10Gb networks and SSD storage it still took hours and hours.
  4. Log on to the Hyper-V VM
    1. Make a note of the drive letter used by the CD/DVD drive
    2. Make a note of what the other drive letters are too. I suggest creating a file called Drive_<driveletter>.txt in the root of each drive to ensure things have stayed the same once the vSphere VM is up and running.
    3. Power off the Hyper-V VM you want to convert to vSphere.
  5. Fire up StarWind Converter and pick the source vhdx file – I pointed it directly at the vhdx file on the Hyper-V host via a UNC path.
  6. Pick the output file format to convert to – you want to choose “VMware ESX server image”
  7. Chose the filename and location for the vmdk file – put this on a separate disk/place to the source if possible – you want to maximise data throughput and minimise disk & network contention. The naming convention in vSphere datastores is <VM Name>.vmdk for the first disk, <VM Name>_1.vmdk for the second disk etc. so it helps if you use that convention here.
  8. Kick off the conversion and wait for it to finish. This will probably take a while.
  9. Now you’ll have pairs of vmdk files: <VM Name>.vmdk and <VM Name>-flat.vmdk. The former is the descriptor file and will be about 1KB, the latter is the actual data file and will be whatever size your Hyper-V VM thought its disk was.
  10. Now you need to create an empty vSphere VM. Place it on a datastore that has sufficient capacity for your newly converted vmdk files.
    1. It doesn’t matter what size you make the initial harddisk as we’ll remove it later anyway.
    2. Configure the VM with the same amount of RAM and number of CPUs as the Hyper-V VM had.
    3. Configure the NIC type to whatever you want, personally I always use VMXnet3. Set the port group to match the VLAN that the Hyper-V VM was on.
    4. I used the default SAS disk controller.
    5. Go into Options – and change the boot setting from BIOS to EFI. The default for new VMs is BIOS, so this is especially important as otherwise the vSphere VM will not boot.
  11. Once the VM is created, edit the settings and remove the harddisk, choose the option to delete the files from disk.
  12. Open the datastore browser and select the folder where your new VM lives. Upload the converted <VM Name>.vmdk file(s) and the associated <VM Name>-flat.vmdk file(s). This will take a while. Note that the datastore browser hides a lot of files, including the “*-flat.vmdk” files.
  13. Now edit the VM settings, and add a new disk. When prompted, choose the option to add an existing disk and browse to find the <VM Name>.vmdk file. Repeat for any additional _1.vmdk, _2.vmdk etc. disks.
  14. Power the vSphere VM on. Once Windows has started, log on and open Disk Management.
    1. You’ll need to change the CD/DVD drive letter to whatever it was in the Hyper-V VM.
    2. Next bring online any other disks and they should drop in with their correct drive letters.
  15. Now install VMware tools. If you’ve used the VMXnet3 NIC this will appear after the install so you can configure it with the correct IP details before rebooting the server to complete the tools install.
  16. Remove any old Hyper-V hardware from within the VM.
  17. That’s it.

If you wanted to change the vmdk type to thin provisioned afterwards you could migrate the VM to a different datastore and change the disk type as part of the migration.

Obviously don’t power the Hyper-V VM back on again once the vSphere one is up and running.

The key points here are:

  • StarWind Converter is very handy.
  • Change the VM boot type from BIOS to EFI or it won’t boot.
  • The process is very slow due to the amount of data (probably) involved. The 800GB VM took me the best part of a (working) day to convert from powering off the Hyper-V VM and having the vSphere one up and running and back in operation.


Posted in Hyper-V, Storage, vSphere, Windows | Tagged , , , , , , , , , , , , , | Leave a comment

Using Group Policy WMI Filters with examples

WMI Filters exist at the bottom on the Group Policy Management Console and are a way to target Group Policy Objects (GPOs) based on the results of the WMI query.

Other ways of targeting GPOs are by the OU that it is linked to (and the members of that OU), or by Active Directory security group.

WMI filters are useful as they can give you much more granularity, and are also dynamic. For example, if you’re applying a GPO to your PCs but only want it to apply to Windows 7, use a WMI filter. If the PC gets upgraded to Windows 10 the GPO will automatically stop applying. The only other way to do this would be to manually have all your Windows 7 PCs as a member of a security group or an OU, and have a way to ensure they were removed if the OS changed. Computers that are members of a security group have to be rebooted for the group membership change to take effect (this is when the computer account logs on to AD).

So, some filters:

  • VMware VMs only
    SELECT Model FROM Win32_ComputerSystem WHERE Model = “VMWare Virtual Platform”
  • For the inverse just add NOT after WHERE, e.g. everything except VMware VMs
    SELECT Model FROM Win32_ComputerSystem WHERE NOT Model = “VMWare Virtual Platform”
  • Hyper-V VM
    SELECT Model,Manufacturer FROM Win32_ComputerSystem WHERE Model = “Virtual Machine” AND Manufacturer = “Microsoft Corporation”
    I added Manufacturer as it makes it clearer that this is a Hyper-V VM as opposed to the “Model” property potentially just signifying a generic “Windows has detected that it is not running on physical hardware”. It does not do that, as far as I know Model = “Virtual Machine” is unique to Hyper-V. (e.g. for VirtualBox you use “VirtualBox”)
  • Physical servers (which in my environment means not VMware or Hyper-V)
    SELECT Model FROM Win32_ComputerSystem WHERE NOT Model LIKE “%Virtual%”
  • Laptops, or PCs & Servers with a UPS
    SELECT * FROM Win32_Battery
  • Windows 7
    SELECT Caption,Primary FROM Win32_OperatingSystem WHERE Caption LIKE ‘Microsoft Windows 7%’ AND Primary = TRUE
    Note that I’m checking the “Primary” property based on info in the book VBScript, WMI, and ADSI Unleashed by Don Jones (chapter 29, p.482). It is quite possibly not necessary.
  • Windows 7 or Server 2008 (inc. R2)
    SELECT Caption,Primary FROM Win32_OperatingSystem WHERE (Caption LIKE ‘Microsoft Windows Server 2008%’ AND Primary = TRUE) OR (Caption LIKE ‘Microsoft Windows 7%’ AND Primary = TRUE)
  • Computers with names beginning Finance
    SELECT Name FROM Win32_ComputerSystem WHERE Name LIKE ‘Finance%’
Posted in Windows | Tagged , , , , , , , , , , , , , , , , , , , , | Leave a comment

Testing for open ports with PowerShell

Want to find out if a TCP port is open via a firewall and/or if something is listening on that port? From within a script? You can use PowerShell.

A colleague found this, which uses System.Net.Sockets.TcpClient from the .NET framework via PowerShell. That would work, but then I found Test-NetConnection.

Test-NetConnection allows testing of TCP ports, and is a built-in cmdlet, which for me is preferable over custom code wherever possible. I’d previously used Test-Connection, which is simpler and just uses ICMP echo requests (aka ping).

I’m using the cmdlet as follows:

if(Test-NetConnection -ComputerName -Port 443 -InformationLevel Quiet){
    # Something is listening on that port, do some stuff
} else {
    # Nothing listening on that port, do something else

Simple as that.

Posted in Networking, PowerShell | Tagged , , , , , , , , | Leave a comment

Samsung SmartThings first impressions and initial setup

Samsung SmartThings is Samsung’s consumer home automation/Internet of Things (IoT) product range. I had a starter kit of this through the post to test. The kit retails at £199, but only seems to be available either direct from Samsung or from Currys. Here’s my initial thoughts.

Kit contents

The starter kit that I have received consists of:

  • SmartThings Hub – as the name suggests, this is the centre of the system. It contains ZigBee and Z-Wave radios which are used to communicate with the various SmartThings sensor and control devices. The Hub connects to your home broadband router via Ethernet cable (included), not WiFi. Presumably cramming a WiFi radio in there too would be just too many antennas in close proximity and risk interference. The hub is slightly smaller than a stack of three CD jewel cases, but in white and with rounded corners. It is powered by an external mains “plugtop”-type Power Supply Unit, which has about 2 metres of cable. The PSU provides 5V at 2A, but does not use a MicroUSB connector, instead opting for a coaxial-style one. The bottom plate of the hub slides off to reveal the backup battery compartment. This takes 4 x AA cells, which are included.
  • Power Outlet – this allow switching of a 13A UK mains socket outlet. It’s fairly compact, not being much larger that the outline of a regular 13A plug, but about 4cm deep. It does not provide dimming functionality, though it does have a power sensor so can tell the number of Watts being drawn by whatever you’ve plugged into it.
  • Motion Sensor – This is a 180 degree PIR motion sensor, but also contains a temperature sensor, so the name seems a strange choice. It comes with a removable wall mount bracket that it can be clipped in and out of, and also has a keyhole mounting on its rear. You even get a drilling template for the mounting bracket, plus screws and wall plugs. This device is smaller than I expected, being about 4x4x2cm.
  • Multi Sensor – the multi-sensor can do several things: it has a separate magnet that can be used to detect doors/windows opening/closing much like regular burglar alarm door sensors; it can detect temperature; it has a vibration sensor to know if something has been picked up or moved, maybe an earthquake – or even the spin cycle starting on your washing machine; finally it can also detect orientation, so something being knocked over – a heater or baby’s cot or valuable item. A wall bracket is included for the main sensor, plus screws, and a sticky foam pad for the magnet.
  • Presence Sensor – this is a fairly chunky keyfob-style sensor that can trigger alerts when it comes into or goes out of range of the Hub. This could be used to alert you when your kids get home from school, or turn on lights & heating when you get home. When in range of the Hub it can be made to beep too, so could be used if you constantly lose your keys, or want some kind of pager.

The box is roughly cube-shaped, and folds out in half, with the Hub and its PSU in one half, and the sensors in the other. You also get a CD-case sized foldout envelope with a “welcome” code printed on it, and containing quick-start guides for the hub and the found sensors. The envelope is also printed with four numbered instructions:

  1. Download the free SmartThings app for Android, iOS or Windows.
  2. Create an account in the app
  3. Follow the on-screen instructions in the app to connect your hub
  4. Enter the welcome code when prompted

SmartThings App

The Android app (v2.0.3 at time of writing) is rated 3.7/5 stars. Reviews range from things like “2.0 is a big improvement – four stars” to “Good app, bad battery use – three stars” and “Crashes or hangs constantly – one star”. My phone is an HTC One M9 with Android 5.1 so we’ll see how it goes. The app needs quite a few permissions, and I’m not sure why some of them are necessary (e.g. Contacts) but current Android versions don’t let me block stuff I’m not happy with so I’ll just have to hope the app behaves. The app is 47MB.

On running the app you’re presented with the SmartThings logo and two buttons: Sign Up or Log In. Signing up requires you to provide your full name, email address and a password. By signing up you are stating that you’ve agreed to the terms of service and the privacy policy (of course I’ve not read either…). Next you get a “hello” screen, and have to pick your region, the only options are “US or Canada”, United Kingdom or Ireland. Then you are asked for the 5-character welcome code, and are told that this can be only used once, but to contact support if it doesn’t work – mine did work.

Now it starts to walk you through the setting up of the Hub and “Things”. It even gives you a video to show you how to plug the network and power cables into the hub… to be honest, if you struggled with that bit this is probably not the product for you! The hub has an indicator light on the front, and this shows red when initially powered on, then goes off for a few seconds, comes back on blue, then changes to green, followed a while later by purple – this whole process took maybe two or three minutes. At this point I touched the “next” button in the app. The app then says “please wait” and gives you a spinning thing, the app screen confirms when your hub is ready. During this time the hub light went blue again, then green, and then I finally got a green blob in the app and was able to continue. This bit took another two or three minutes – just long enough that I thought it was going to timeout and fail! Next the app asks you for a name for your home, which defaults to “Home”, it also asks for a picture – you can choose from various built-in ones like some outdoor fairy lights, tulips, park railings, a field of green crops etc. but cannot pick your own – at this stage I have no idea what this is used for – perhaps some kind of security/authentication? It also wants to know the location of your hub, apparently so that they know when you’re at home (potential security implications here too, both good and bad).

Add the Things

We now move onto the stage where we connect the “things”. This seems to mostly involve pulling the “Remove to pair” plastic tab that three of the four devices have (motion, multi and presence).

  • For the motion sensor, this levers off the front cover and pulling the tab allows the CR2450 3V cell battery to make contact and power the unit up. I had to pull the tab pretty hard to get it to come out. Once powered up, a blue light starts flashing on the sensor, which soon stops.
  • The power outlet has two indicator lights and two buttons on either side of the casing, these lights do the same things as the motion sensor – flash blue for a bit then go off.
  • Pulling the tab on the multi sensor also causes the front cover to come off, and the light does the same thing as the other devices. It is also powered by a CR2450 button cell. The cover only fits on properly one way up.
  • The presence sensor doesn’t seem to have an indicator light, and is powered by a smaller CR2032 button cell.

None of the indicator lights from the battery-powered things are visible once the covers have been replaced, so you’re not going to see lots of little lights flashing all over your house. Once they were all powered up and had stopped flashing (and in the case of the presence sensor, I gave it a few minutes) I tapped “Next” on the app, and was told that it was looking for my new things. The green light on the Hub started flashing, and two of the things were discovered – the Multipurpose sensor and the SmartSense Presence. A few minutes later the motion sensor appeared, I had picked it up and took the cover off at this point so I wonder if that had something to do with it (the light flashed green – maybe to show that it had detected movement, or maybe by coincidence it had just finished registering with the Hub). More than five minutes after this, the power outlet still wasn’t detected, so I pushed one of the buttons on the side – this caused it to turn the outlet on, the light went solid blue and I heard a mechanical relay operate inside. I pushed the other button and it turned the outlet off again, but still didn’t register with the hub. I pulled it out of the mains wall socket, and plugged it back in again and a few seconds later it appeared in the app.

Next we have to configure each thing. This involves giving it a name, and selecting a room. I put the multipurpose sensor and the power outlet into my living room, the motion sensor into the hall, and I created a room called my name for the presence sensor – I plan to keep that with me so it won’t be fixed in a particular room. You can take a photo for each room or add one from the phone’s gallery.


Finally we get the All Done! screen, and the initial setup is complete, and we’re taken to the Welcome screen. This explains:

  • Dashboard – monitor your smart home for intrusion, leaks and floods, smoke and CO, and more
  • My Home – organise your smart home by creating rooms to control and monitor your devices
  • Routines – Customise different actions to take place throughout the day in your smart home
  • Notifications – See what is happening in your home now and what has happened recently
  • Marketplace – Marketplace is where you can discover new ways to personalise your smart home

Swipe the screen to the left and you move to more information screens, some of which say the same thing as the welcome screen, and other tell you something new/different:

  •  Smart Home Monitor – same as Dashboard
  • Rooms – same as My Home
  • Things – The things category lets you control and monitor all of the connected products in your smart home
  • SmartApps – Once you add SmartApps from the Marketplace, you can manage them from here
  • Family – Family lets you stay connected to those who matter most y seeing when people, pets, and cars have arrived home

The final screen, Family, has a Get Started button at the bottom. Tapping this takes you into the app proper, and to the dashboard.

So that’s it for my first impressions. I’ll write more as I start to configure and use the system over the next few days.

Posted in Android, Hardware, Security | Tagged , , , , , , , , , , , , , , , , | Leave a comment

Cleaning a large Windows\assembly\temp folder

I found that one of my servers had very low free space on the C drive, and by using TreeSize I found that 9GB was being consumed by the folder C:\Windows\assembly\temp:
windows assembly temp large

This isn’t supposed to happen, the temp folder is used for .Net Global Assembly Cache (GAC) uninstalls (whereas the tmp folder is used for installs). It seems as though these folders should be tidied up automatically, but clearly sometimes are not.

The assembly folder is also a bit interesting to work with via Windows Explorer. You can do some fiddling around with a desktop.ini file and delete the folders via Explorer, or use my method, which is via the command line. I am doing this to a remote machine – you don’t want to be on the console of a server unless strictly necessary, and this is not one of those times.

pushd \\problemserver\c$\windows\assembly\temp
for /f %i in ('dir /b') do rd /s /q %i

which takes a little while to go through and delete the folders. Some won’t delete, giving Access is denied errors, but most seem to. Here’s an updated TreeSize view of the same server after running the above commands:
windows assembly temp after cleaning

Clearly, if you run an rd /s /q anywhere inside your Windows folder (or anywhere at all, for that matter) you’d better be sure you are in the right place and are happy about the stuff it’s going to delete prior to pushing the button! You are responsible for what you do to your own systems.

Posted in Windows | Tagged , , , , , , , , , , , , , , | Leave a comment

SysProcs is RDS equivalent of Citrix LogOffCheckSysModules

Now that I’m no longer running Citrix XenApp, I had to research how to tell the Terminal Server service to ignore a process that was stopping a process being killed when the RemoteApp executable had quit.

In XenApp you add the exe name of the “rogue” process(es) to a registry string value called LogOffCheckSysModules, but this key is Citrix-specific and thus won’t have any effect if you’re just using Remote Desktop Session Host.

It turns out (as I’d hoped) that there is an equivalent way to do this. You add a DWORD named with the process you want Windows to ignore to the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\Sysprocs

and set the value to zero. There will already be several processes listed in there by default.

Posted in Windows, XenApp | Tagged , , , , , , , , , , , , , , , , , | 1 Comment

PowerShell script to update RDMS server list

Remote Desktop Services in Windows Server 2012 is great in many ways, but it has some serious flaws. One of these is the pretty terrible so-called Remote Desktop Management Server concept. This is actually just some functionality built into Server Manager.

One of the most annoying things is that RDMS will fail to work if a server has been added to your Remote Desktop Deployment but you’ve not added it as a server to be managed via your own personal instance of Server Manager. This is quite a likely situation if you work in a team and/or have an automated server provisioning process. And it sucks. Instead of seeing your RD deployment, you just get a blank page with some text saying:

The following servers in this deployment are not part of the server pool:
The servers must be added to the server pool.

No “Add now” button, no “continue anyway”, just a flat refusal to do anything at all.

So, Having recently done some work playing around with XML in PowerShell, and figuring that ServerManager probably stores its config in an XML file (though that wasn’t guaranteed – but luckily is true) I wrote a cmdlet that will update Server Manager to ensure that all the servers in your Remote Desktop deployment are added to Server Manager.

You’ll need to modify the code to specify your Connection Broker(s) (it copes with HA Connection Brokers). Save the code into a .psm1 file and then you can load this in your $profile script by using Import-Module <your-modules.psm1>.

function global:Update-RDMS {
        Ensure all RDS deployment servers are added to the Remote Desktop Management Server (Server Manager)

        This cmdlet queries the Connection Broker and updates Server Manager to ensure all the RDS servers are added.
        Server Manager is killed if running, updated and restarted.

        Updates the RDMS

        RCM August 2015

        Write-Debug "Starting Update-RDMS"

        $ConnectionBrokers = "",""
        $ServerManagerXML = "$env:USERPROFILE\AppData\Roaming\Microsoft\Windows\ServerManager\Serverlist.xml"
        Write-Debug "Import RDS cmdlets"
        Import-Module RemoteDesktop
        Write-Debug "Find active Connection Broker"
        $ActiveManagementServer = $null
        foreach($Broker in $ConnectionBrokers){
            $ActiveManagementServer = (Get-ConnectionBrokerHighAvailability -ConnectionBroker $Broker).ActiveManagementServer
            if($ActiveManagementServer -eq $null){
                Write-Host "Unable to contact $Broker" -ForegroundColor Yellow
            } else {
        if($ActiveManagementServer -eq $null){
            Write-Error "Unable to contact any Connection Broker"
            if(Get-Process -Name ServerManager -ErrorAction SilentlyContinue){
                Write-Debug "Kill Server Manager"
                # Have to use tskill as stop-process gives an "Access Denied" with ServerManager
                Start-Process -FilePath "$env:systemroot\System32\tskill.exe" -ArgumentList "ServerManager"
            Write-Debug "Get RD servers"
            $RDServers = Get-RDServer -ConnectionBroker $ActiveManagementServer
            Write-Debug "Get Server Manager XML"
            [XML]$SMXML = Get-Content -Path $ServerManagerXML
            foreach($RDServer in $RDServers){
                $Found = $false
                Write-Host ("Checking "+$RDServer.Server+" ") -NoNewline -ForegroundColor Gray
                foreach($Server in $SMXML.ServerList.ServerInfo){
                    if($RDServer.Server -eq ${
                        $Found = $true
                if($Found -eq $true){
                    Write-Host "OK" -ForegroundColor Green
                    Write-Host "Missing" -ForegroundColor Yellow
                    $NewServer = $SMXML.CreateElement("ServerInfo")
                    $SMXML.ServerList.AppendChild($NewServer) | Out-Null
                    $NewServer.SetAttribute("lastUpdateTime",[string](Get-Date -Format s))
            # Remove xmlns attribute on any newly added servers, this is added automatically by PowerShell but causes Server Manager to reject the new server
            $SMXML = $SMXML.OuterXml.Replace(" xmlns=`"`"","")
            Write-Debug "Save XML file"
            Write-Debug "Start Server Manager"
            Start-Process -FilePath "$env:systemroot\System32\ServerManager.exe"
Posted in PowerShell, Remote Desktop, Windows | Tagged , , , , , , , , , , , , , , , , | Leave a comment

Powershell RegEx Web Scraping

I wrote this as a learning process, and because I’d been wanting something that did this for a while anyway. Checking a website regularly is tedious, but getting an email when it shows new info I’m interested in is perfect.

The local police force publish the locations of safety cameras (speed cameras), including mobiles ones, once a week (though sometimes more towards the end of the week, which is less useful!). I wanted a way to be notified if any new camera sites were added, and also to know which of the possible mobile camera sites were going to be active in a particular week – but only be told of sites that I was interested in (i.e. those on my way to/from work). Not that I zoom around the place or anything, but it’s just nice to know!

The information is published on a web site, with the full list of camera locations split into separate pages for each area/county, plus a page that lists the mobile camera sites for the week.

The URL for the week’s active mobile sites changes when it is updated, so I’m retrieving that and then “following it” to get to the page for the current week. The current list URL ends in an index number which seems to increment when the page is updated, so I store this number in the registry so that I can compare it from one run of the script to the next, and only send notifications when it changes.

The full list of camera sites is exported to an XML file, and then compared with the web site each time the script subsequently runs to allow and changes to sites to be tracked and a notification sent. The XML is then updated and the process repeats. I’ve hard-coded the script to only bother checking certain areas/counties as I don’t need to be notified of new sites in places that I never go to. This would be easy to change though, see the array being filled around line 196, and note the number in the URL for the area you’re interested in.

The mobile sites that I’m interested in are specified in another XML file, along with my email address and SMTP (mail) server details.

The data on the web pages is formatted in tables and lists, so I am using regular expressions to find and extract it. This is a combination of removing matching text and retrieving matching text to get the page contents into a usable format in memory. This was the most interesting bit of the script to code, and trying to get the regex right in each case was “fun”. I found that was helpful as the real-time highlighting allows you to quickly get things working. Note that PowerShell seems to use case-insensitive matching by default, so you’ll want to add an “i” into the “modifier” box to the right of the regular expression builder box.

I run the script every three hours via a scheduled task. I’ve yet to find a way to make PowerShell run hidden, but you can make it disappear quickly and keep running in the background with the -WindowStyle Hidden command line argument, which is good enough for me for the moment. I’ve wondered if the old utility runh.exe would still work, but not got around to testing this yet. Please comment if you have a way to do this. When creating the scheduled task, the program/script is powershell.exe and the argument is:

-WindowStyle Hidden <full path to the script.ps1>

The config XML file looks like this, the formatting is a bit weird because I used Export-Clixml to create it in the first place, deal with it (!):

&lt;Objs Version="" xmlns=""&gt;
 &lt;Obj RefId="0"&gt;
  &lt;TN RefId="0"&gt;
   &lt;S N="EmailAddress"&gt;;/S&gt;
   &lt;S N="SMTPServer"&gt;;/S&gt;
   &lt;S N="CamList"&gt;0109,0164,0160,0061,0057,0156,0195,0054,0021,0065,0066&lt;/S&gt;

You need to change it to your own settings. The script looks for the two XML files in whatever location you choose to run it from, I’ve got it sat in Documents\SafetyCam but you can put it anywhere you have read & write access to. Here’s the script:

# URI to the page containing the link to the page that has the list of this week's mobile camera locations
$NewsPageURI = ""
# URI to the full list of camera sites
$AllLocationsURI = ""
# URI stub of individual council sites list
$LocationsURIStub = ""
# All known camera site list XML file
$SitesFile = Join-Path -Path $PSScriptRoot -ChildPath "SafetyCameraSites.XML"
# Config file
$ConfigFile = Join-Path -Path $PSScriptRoot -ChildPath "SafetyCameraConfig.xml"
# Registry key for tracking active sites and interested site changes
$RegKey = "HKCU:\Software\SafetyCam"
# Set up a CRLF string
$CRLF = "`r`n"

################# Functions #################
function Send-Error($ErrorText){
    Send-MailMessage -From "" -To $MyEmailAddress -SmtpServer $SMTPServer -Subject "Error: $ErrorText" -Body $ErrorText
    Write-Host "Error: $ErrorText" -ForegroundColor Red
function Get-CameraTextFromHTML([string]$CamType){
    # $CamType should be "fixed", "mobile" or "red light"
    $CamText = ""
    if($PageHTML -match "&lt;b&gt;$CamType(.)*?&lt;td class(.)*?absmiddle&gt;(.)*?&lt;\/tbody&gt;"){
        # Found the matching fixed camera text pattern
        $CamText = $Matches[0]
        # Get rid of the table header stuff before the camera list
        $CamText = $CamText -replace '&lt;B&gt;(.)*?valign(.)*?"50%"&gt;',""
        # Get rid of the table footer stuff after the camera list
        $CamText = $CamText -replace '&lt;\/td&gt;&lt;\/tr&gt;&lt;\/tbody&gt;$',""
        # Get rid of the bit separating the two table colums
        $CamText = $CamText -replace '&lt;\/td&gt;(.)*?"50%"&gt;',""
        # Get rid of the HTML at the start of each row of text
        $CamText = $CamText -replace '&lt;p&gt;(.)*?absmiddle&gt;',""
        # Tidy up any extraneous entries
        $CamText = $CamText -replace '&lt;p&gt;&amp;nbsp;&lt;\/p&gt;',""
        $CamText = $CamText -replace '&amp;gt;',"&gt;"
        # Get rid of the HTML at the end of each row of text and replace with a delimiter of our own
        $CamText = $CamText -replace '&lt;\/a&gt;(.)*?&lt;\/p&gt;',"!"
        # But not after the very last line
        $CamText = $CamText -replace '!$',""
    }elseif($PageHTML -match "&lt;b&gt;$CamType(.)*?&lt;p&gt;there are currently no operational $camtype cameras in this council area&lt;\/p&gt;"){
        # No cameras of this type found, tidy up text string
        $CamText = $Matches[0]
        $CamText = $CamText -replace '&lt;b&gt;(.)*?&lt;p&gt;',""
        $CamText = $CamText -replace '&lt;\/p&gt;',""
        $CamText = "Unable to parse HTML"
function Get-CamList([string]$CamURINumber){
    $Page = Invoke-WebRequest -Uri "$LocationsURIStub$CamURINumber"
    # Get the HTML content of the page, and strip out all CRLF to make it easier to use regular expressions
    $PageHTML = $Page.ParsedHtml.body.innerHTML.Replace("`r`n","")
    if($PageHTML -match "&gt;[\w\s]+ camera information&lt;"){
        $CouncilName = $Matches.Item(0)
        $CouncilName = $CouncilName -ireplace "&gt;",""
        $CouncilName = $CouncilName -ireplace " camera information&lt;",""
        $CouncilName = $CouncilName.replace("`r`n","")
    # Get the fixed camera locations
    $Fixed = Get-CameraTextFromHTML -CamType "fixed"
    $Mobile = Get-CameraTextFromHTML -CamType "mobile"
    $RedLight = Get-CameraTextFromHTML -CamType "red light"
    # Split the locations into arrays
    $FixedArray = $Fixed.Split("!")
    $MobileArray = $Mobile.Split("!")
    $RedLightArray = $RedLight.Split("!")
    # Add each camera in each array into a Cam object and add these objects into a master CamList array
    $CamList = @()
    for ($i = 0; $i -lt $FixedArray.Count; $i++)
        $Cam = New-Object -TypeName System.Object
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamURINumber" -Value $CamURINumber
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CouncilName" -Value $CouncilName
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamType" -Value "Fixed"
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamInfo" -Value $FixedArray[$i]
        $CamList += $Cam
    for ($i = 0; $i -lt $MobileArray.Count; $i++)
        $Cam = New-Object -TypeName System.Object
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamURINumber" -Value $CamURINumber
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CouncilName" -Value $CouncilName
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamType" -Value "Mobile"
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamInfo" -Value $MobileArray[$i]
        $CamList += $Cam
    for ($i = 0; $i -lt $RedLightArray.Count; $i++)
        $Cam = New-Object -TypeName System.Object
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamURINumber" -Value $CamURINumber
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CouncilName" -Value $CouncilName
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamType" -Value "RedLight"
        Add-Member -InputObject $Cam -MemberType NoteProperty -Name "CamInfo" -Value $RedLightArray[$i]
        $CamList += $Cam
    # Reurn the CamList object

#################   Main   #################
# Read in config info from file
    $Config = Import-Clixml -Path $ConfigFile
    $MyEmailAddress = $Config.EmailAddress
    $SMTPServer = $Config.SMTPServer
    $MySiteList = $Config.CamList
    $MySites = $MySiteList.Split(",")
    Send-Error "Error: problem with config file"
Write-Host ("Email address   : "+$MyEmailAddress)
Write-Host ("SMTP server     : "+$SMTPServer)
Write-Host ("Monitored sites : "+$Config.CamList)
# Get the current locations page from the main news page
    $NewsPage = Invoke-WebRequest -Uri $NewsPageURI
    Send-Error "Error: Failed to retrieve main news page"
# Get the URL for this week's locations page
$NewsHref = ""
foreach($Link in $NewsPage.Links){
    if($Link.innerText -match "mobile speed camera enforcement schedule for week commencing"){
        $NewsHref = $Link.href
        break # Only take the first URL, multiple URLs occasionally present, most recent seems to be placed first
if($NewsHref -eq ""){
    Send-Error "Error: Unable to get current locations URI from news page"
# Build the URI to the current locations page
$CurrentLocationsURI = $NewsPageURI.Replace("index.aspx",$NewsHref)
$NewsID = $NewsHref -ireplace "viewnews.aspx\?newsid=",""
# Set up a flag to determine whether to send email
$ActiveSiteChanges = $true
# Get previous config data from registry, this allows us to check for changes to NewsID (different active mobile sites list)
# or changes to the number of sites we're interested in reporting on.
if(Test-Path $RegKey){
    # Registry key exists, check NewsID value
    if(((Get-ItemProperty -Path $RegKey -Name "NewsID" -ErrorAction SilentlyContinue)."NewsID" -ne $NewsID) -or ((Get-ItemProperty -Path $RegKey -Name "MySiteList" -ErrorAction SilentlyContinue)."MySiteList" -ne $MySiteList)){
        # NewsID or MySiteList has changed, update them and run full script
        New-ItemProperty -Path $RegKey -Name "NewsID" -Value $NewsID -Force | Out-Null
        New-ItemProperty -Path $RegKey -Name "MySiteList" -Value $MySiteList -Force | Out-Null
        # Get the active locations
            $CurrentLocations = (Invoke-WebRequest -Uri $CurrentLocationsURI).Content
            Send-Error "Error: Failed to retrieve current locations page"
        # Check that the current locations page contains the expected date header text
        if(($CurrentLocations -match "&gt;[\s\w]*week commencing[\s\w]*.&lt;") -eq $false){
            Send-Error "Error: Current Locations page did not contain expected date header text"
        # Check for where mobile cameras are going to be located
        [string]$ActiveLocations = ""
        foreach($Site in $MySites){
            if($CurrentLocations -match $Site){
                $CurrentLocations -match "&gt;[\w\s(),\/]*: "+$Site+"&lt;" | Out-Null # matches location line, including "," and "/"
                $Location = $Matches.Item(0)
                $Location = $Location.Replace("&gt;","")
                $Location = $Location.Replace("&lt;","")
                $ActiveLocations += "Camera at "+$Location+$CRLF
        if($ActiveLocations -eq "" -and $CurrentLocations.Length -gt 0){
             $ActiveLocations = "No active locations found that you are interested in this week"
        # NewsID has not changed since last run of this script, camera list is probably unchanged
        $ActiveSiteChanges = $false
    # Registry key does not exist, create it, then set the NewsID value and run full script
    New-Item -Path $RegKey | Out-Null
    New-ItemProperty -Path $RegKey -Name "NewsID" -Value $NewsID | Out-Null
    New-ItemProperty -Path $RegKey -Name "MySiteList" -Value $MySiteList | Out-Null
# Check for changes to numbers of camera sites
# Read in all known camera sites from HTML pages
$AllCams = @()
$AllCams += Get-CamList -CamURINumber 5 # North Somerset
$AllCams += Get-CamList -CamURINumber 7 # Bristol City Council
$AllCams += Get-CamList -CamURINumber 8 # South Gloucstershire
if(Test-Path $SitesFile){
    # Read in known camera sites from file
        $PreviousCams = Import-Clixml -Path $SitesFile
        Send-Error "Reading sites file"
    # No config file present, can't compare this time. Write sites list to config file for use next time.
        $AllCams | Export-Clixml -Path $SitesFile
        Send-Error "Writing sites file"
# Extract just the camera info (location and site code) so that we can search using -contains
# ... for current list
$CamInfo = @()
foreach($Cam in $AllCams){
    $CamInfo += $Cam.CamInfo
# ... for previous list loaded from file
$PreviousCamInfo = @()
foreach($PCam in $PreviousCams){
    $PreviousCamInfo += $PCam.CamInfo
# Check each current camera site to see if it exists in the previous list
$NewCams = @()
    foreach($Cam in $CamInfo){
    if($PreviousCamInfo -contains $Cam){
        # Cam is already known
        $NewCams += $Cam
# Check each previous camera site to see if still valid
$RemovedCams = @()
foreach($Cam in $PreviousCamInfo){
if($CamInfo -contains $Cam){
        # Cam is still there
        $RemovedCams += $Cam
$CamSiteChanges = $true
$CamSiteUpdates = ""
if($NewCams.Count -ne 0 -or $RemovedCams.Count -ne 0){
    foreach($NewCam in $NewCams){
        $CamSiteUpdates += "New site: $NewCam$CRLF"
    foreach($RemovedCam in $RemovedCams){
        $CamSiteUpdates += "Removed site: $RemovedCam$CRLF"
    # Overwrite the XML file with the latest camera list
    $AllCams | Export-Clixml -Path $SitesFile -Force
    $CamSiteUpdates = "No changes to camera sites"
    $CamSiteChanges = $false
$CamSiteUpdates = $CamSiteUpdates | Sort-Object
if($ActiveSiteChanges -or $CamSiteChanges){
    # Build up email body text
    $DateHeader = ("Current Safety Camera Locations as at "+(Get-Date -Format s))
    $Body = $DateHeader+$CRLF+$CRLF
    $Body += $ActiveLocations+$CRLF
    $Body += [string]$AllCams.Count+" locations searched for "+[string]$MySites.Count+" cameras"+$CRLF+$CRLF
    $Body += $CamSiteUpdates+$CRLF+$CRLF
    $Body += "Locations this week: "+$CurrentLocationsURI+$CRLF
    $Body += "All known locations: "+$AllLocationsURI+$CRLF+$CRLF
    Write-Host $Body -ForegroundColor Cyan
    $BodyHTML = $Body -replace "`r`n","
    Send-MailMessage -BodyAsHtml ('&lt;font face="Calibri"&gt;'+$BodyHTML+'&lt;/font&gt;') -Subject $DateHeader -To $MyEmailAddress -From "" -SmtpServer $SMTPServer
    Write-Host "No changes to active cams or site list" -ForegroundColor Yellow

Obviously, if you use this as-is, and get zapped by a camera, that is your fault for breaking the law, not the fault of me or my script!! If your local police force publishes camera info in a different way you’ll have a nice coding exercise on your hands. I’m hoping that this post will be a good reference for web scraping and regular expressions in PowerShell.

Update 2015-09-15: Only take the first URL from the “news” page, if multiple ones are present.

Posted in PowerShell | Tagged , , , , , , , , , , , , , , , , | Leave a comment

Experts Exchange Windows Server 2012 Master

Been working on achieving this for a little under a month after not doing any Experts Exchange stuff for ages. I am also now a Wizard overall. Hurrah.

Answering questions on EE is a good learning experience – you get to see the problems that people are having with certain technologies, which can greatly assist you if you come to do something similar in the future. It also increases your breadth of knowledge, which can only be a good thing.

Posted in Business, Free training, Windows | Tagged , , , | Leave a comment

Remove Forefront Client Security by force

I was recently trying to upgrade the Antivirus software on some servers from Forefront Client Security to System Center Endpoint Protection 2012 R2. On most servers it worked fine. However on a few I was unable to remove some of the FCS components due to missing .msi files. This was because somebody (not me!) had been deleting the contents of the C:\Windows\Installer folder, probably to save disk space.

This meant that the uninstall command was failing as Windows Installer couldn’t find the right .msi file. This in turn meant I was unable to install SCEP as its installer check to see that all previous have gone before it’ll install.

There is a very faffy way of fixing missing MSIs where you have to track down the correct version of the .msi via the Microsoft Update Catalogue and/or a WSUS server, but it was too fiddly and I didn’t have time. Thus, I wrote a script to manually uninstall the old FCS stuff (or at least, enough of it that SCEP will install and be happy).

The script will continue on errors, and you will get errors as some of the stuff only relates to 32-bit OS, and some only to 64-bit. Plus you might have already been able to remove some of the FCS components properly via Windows Installer.

Here’s the script, save it as a .cmd file and run as administrator.

@echo off
echo Stop services
net stop MOM
net stop FCSAM
net stop FcsSas

echo Delete services
sc delete MOM
sc delete FCSAM
sc delete FcsSas

echo Kill GUI
tskill msascui /a

echo Delete files
rd /s /q "C:\Program Files\Microsoft Forefront"
rd /s /q "C:\Program Files (x86)\Microsoft Forefront"

Echo Remove registry keys
echo ...MOM
rem 64-bit
reg delete "HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{F692770D-0E27-4D3F-8386-F04C6F434040}" /f
reg delete "HKLM\SOFTWARE\Wow6432Node\Microsoft\Microsoft Operations Manager\2.0" /f
rem 32-bit
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{F692770D-0E27-4D3F-8386-F04C6F434040}" /f
rem both
reg delete "HKLM\SOFTWARE\Classes\Installer\Products\D077296F72E0F3D438680FC4F6340404" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\D077296F72E0F3D438680FC4F6340404" /f

echo ...SAS
rem 64-bit
reg delete "HKLM\SOFTWARE\Classes\Installer\Products\E4EB3435742B0D148BD1E4C755649001" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\E4EB3435742B0D148BD1E4C755649001" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{5343BE4E-B247-41D0-B81D-4E7C55460910}" /f
rem 32-bit
reg delete "HKLM\SOFTWARE\Classes\Installer\Products\838A5BA2CAD95F54E82C10D9DD4C4B6F" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\838A5BA2CAD95F54E82C10D9DD4C4B6F" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{2AB5A838-9DAC-45F5-8EC2-019DDDC4B4F6}" /f

echo ...FCS
rem 64-bit
reg delete "HKLM\SOFTWARE\Classes\Installer\Products\EE98922AA7EA8F240A0CC999FC6B44BF" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\EE98922AA7EA8F240A0CC999FC6B44BF" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{A22989EE-AE7A-42F8-A0C0-9C99CFB644FB}" /f
rem 32-bit
reg delete "HKLM\SOFTWARE\Classes\Installer\Products\FF0CF4D4791FF10448E21E811F2D46E7" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\FF0CF4D4791FF10448E21E811F2D46E7" /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{4D4FC0FF-F197-401F-842E-E118F1D2647E}" /f
rem both
reg delete "HKLM\SOFTWARE\Wow6432Node\Microsoft\Microsoft Forefront" /f
reg delete "HKLM\SOFTWARE\Microsoft\Microsoft Forefront" /f

echo Done.

Use with care.

Posted in Scripting, Security, Windows | Tagged , , , , , , , , , , , , , , , , | Leave a comment