PowerShell System Drive Clean-up

I’ve been working through a process of cleaning and optimising the C drives on a lot of VMs recently. Over time they build up with lots of junk that, when put together with hundreds of VMs, requires a significant amount of extra primary VM storage, backup storage, and increases backup times. These files also take up space within the VM that can cause security and anti-malware updates to fail, and lead to application or logging failures. The reduced free space also leads to increased file fragmentation, which then means more IOs to and from your storage.

So the first step in my process is to run this script, which deletes various unnecessary bits and pieces from the Windows folder and user profiles, and empties recycle bins and deletes various other folders that some of my colleagues have a tendency to put onto the C drive (e.g. 5GB of SQL Server install media, which is available for them on the network).

It copes with various flavours of OS, I’ve been using it on 2003 through 2008 R2 so far. You can add paths to the array at the beginning. For paths within the Users (or Documents and Settings) folder it will search through each user profile, excluding the local administrator or any service accounts (which in my organisation all contain “svc” in the name.

I’m toying with the idea of removing the $hf_mig$ folder on 2003 now that there will be no more security updates, likewise SoftwareDistribution, but I’ve not fully looked into the implications of this yet. There’s probably lots of other places where stuff can be safely deleted from too. Another one I’ve just thought of would be *.log from the Windows folder.

The wildcards rely on the handling within the Remove-Item cmdlet, so if you’re not sure what it’ll do, it’s easy to try it manually on a test folder first.

DO NOT run this without understanding what it does, and knowing what data your applications store where, and how your users work. I AM NOT responsible if you run this and trash your server(s)! Look very carefully at the paths that I’m using and see if each one is something you feel happy with or not. That said, it hasn’t caused me any problems yet – I’ve run it on about 30 different servers so far.

Usage is:
TidyCDrive.ps1 <server name>

param([parameter(mandatory = $true)][string]$Target)

$PathsToClean = New-Object -TypeName System.Collections.ArrayList
$PathsToClean.Add('\Windows\System32\Config\SystemProfile\Local Settings\Temp\*') | Out-Null
$PathsToClean.Add('\Windows\Temp\*') | Out-Null
$PathsToClean.Add('\Windows\ie8updates') | Out-Null
$PathsToClean.Add('\Windows\ie8') | Out-Null
$PathsToClean.Add('\Windows\ie7') | Out-Null
$PathsToClean.Add('\Windows\ie7updates') | Out-Null
$PathsToClean.Add('\Windows\ServicePackFiles') | Out-Null
$PathsToClean.Add('\Windows\$NtUninstall*') | Out-Null
$PathsToClean.Add('\Windows\$NtServicePackUninstall*') | Out-Null
$PathsToClean.Add('\Windows\$950099Uinstall*') | Out-Null
$PathsToClean.Add('\Windows\$UninstallRDC$') | Out-Null
$PathsToClean.Add('\Users\xxxxx\AppData\Local\Microsoft\Windows\Temporary Internet Files') | Out-Null
$PathsToClean.Add('\Users\xxxxx\AppData\Local\Microsoft\Windows\Explorer\thumbcache*.db') | Out-Null
$PathsToClean.Add('\Users\xxxxx\AppData\Local\Microsoft\Windows\Terminal Server Client\Cache') | Out-Null
$PathsToClean.Add('\Users\xxxxx\AppData\Local\Temp\*') | Out-Null
$PathsToClean.Add('\Program Files\Veritas\Patch') | Out-Null
$PathsToClean.Add('\wuinstall') | Out-Null
$PathsToClean.Add('\$Recycle.bin\*\*') | Out-Null
$PathsToClean.Add('\Recycler\*') | Out-Null
$PathsToClean.Add('\SQL2008R2Ent') | Out-Null
$PathsToClean.Add('\SQLserver 2008 R2 SP2') | Out-Null
$PathsToClean.Add('\Security Update') | Out-Null

Write-Host ("Disk Clean & optimise for "+$Target) -ForegroundColor Gray
# record initial C drive free space
$WMIDisk = Get-WmiObject -Class Win32_LogicalDisk -ComputerName $Target -Filter "DeviceID='C:'" -ErrorAction Stop | Select-Object -Property Size,FreeSpace
[single]$CFreeSpaceBefore = $WMIDisk.FreeSpace/1GB
Write-Host ("Free space: "+$CFreeSpaceBefore)
# clean C drive based on known paths
foreach($Path in $PathsToClean){
    Write-Host ("Cleaning $Path")
    $UNCPath = ("\\"+$Target+"\C$"+$Path)
    if($UNCPath -like "*\users\xxxxx\*"){
        if(Test-Path -Path ("\\"+$Target+"\C$\Users")){
            $ProfilesFolder = ("\\"+$Target+"\C$\Users")
        }else{
            $ProfilesFolder = ("\\"+$Target+"\C$\Documents and Settings")
            $UNCPath = $UNCPath.Replace("\Users\","\Documents and Settings\")
        }
        $LocalProfiles = Get-ChildItem -Path $ProfilesFolder -Directory | Where-Object -Property Name -Like *admin* | Where-Object -Property Name -notlike "Administrator" | Where-Object -Property Name -notlike "*svc*"
        foreach($LocalProfile in $LocalProfiles){
            $UNCProfilePath = $UNCPath.Replace("xxxxx",$LocalProfile.Name)
            Write-Host $UNCProfilePath -ForegroundColor DarkGray
            Remove-Item -Path $UNCProfilePath -Recurse -Force -ErrorAction SilentlyContinue
        }
    }else{
        Write-Host $UNCPath -ForegroundColor DarkGray
        Remove-Item -Path $UNCPath -Recurse -Force -ErrorAction SilentlyContinue
    }
}
# Check size of SoftwareDistribution folder isn't too large
$SD = (Get-ChildItem -Path "\\$Target\C$\Windows\SoftwareDistribution" -Recurse -Force | Measure-Object -Property Length -Sum).Sum/1MB
if($SD -gt 900){
    Write-Warning "SoftwareDistribution is a little large"
}
$WMIDisk = Get-WmiObject -Class Win32_LogicalDisk -ComputerName $Target -Filter "DeviceID='C:'" -ErrorAction Stop | Select-Object -Property Size,FreeSpace
[single]$CFreeSpaceAfter = $WMIDisk.FreeSpace/1GB
Write-Host ("Free space: "+$CFreeSpaceAfter)
$CFreedUpGB = $CFreeSpaceAfter - $CFreeSpaceBefore
Write-Host ("Freed up {0:N2}GB within C drive" -f $CFreedUpGB) -ForegroundColor Green
Posted in Performance, PowerShell, Windows | Tagged , , , , , , , , , , , , , , | Leave a comment

PowerShell: Monitor changes to a folder

This uses a FileSystemWatcher object and its WaitForChanged method, which is rather nifty way to not use much resource to keep track of file system changes. What we’re NOT doing here is polling the folder contents.

I’m then trying a couple of different methods to try and work out, if possible, the user responsible for the changes. This doesn’t always work, but it’s not too bad. You’ll get better info about the user if the folder is being accessed remotely via an SMB share thanks to the Get-SmbOpenFile cmdlet, otherwise I try and get the file owner, or don’t bother at all. Get-SmbOpenFile needs elevated privileges so you’ll need to run this “as administrator”.

$Folder = ’D:\TD2’
$FileSystemWatcher = New-Object System.IO.FileSystemWatcher $Folder
while ($true) {
  $Change = $FileSystemWatcher.WaitForChanged(‘All’, 1000)
  if ($Change.TimedOut -eq $false){
    [string]$User = ""
    $FilePath = Join-Path -Path $Folder -ChildPath $Change.Name
    if(Test-Path -Path $FilePath){
      $SMBUserArray = Get-SmbOpenFile -IncludeHidden | Where-Object -Property Path -Like $FilePath
      if($SMBUserArray.Count -ge 1){
        $User = "SMB: "+$SMBUserArray[0].ClientUserName+" "
      }else{
        try{
          $User = "Owner: "+(Get-Acl -Path $FilePath -ErrorAction Stop).Owner+" "
        }catch{}
      }
    }
    Write-host ($User+$Result.ChangeType+": "+$Result.Name)
  }
}

Press Ctrl-C to abort when you’re done monitoring.

Posted in PowerShell, Security | Tagged , , , , , , , , , , , , , , | Leave a comment

Run Disk Cleanup – save GB of space

I’m a big fan of the built-in Windows Disk Cleanup utility, cleanmgr.exe. Since the extra features added to the Vista & Server 2008 version it has become a great way to recover sometimes quite substantial amounts of space from your system (OS) drive though its Windows Update Cleanup feature.

Here’s an example run on one of my Server 2008 R2 VMs:
cleanmgr 2008 r2
Nearly 8GB back? Thank you very much! Here’s another:

disk cleanup 12GB
I’ll get nearly 12GB back on this server, installed in 2010.

If the Windows Update Cleanup option doesn’t show its because no space can be freed by that method (or you ran it once with that option and haven’t rebooted yet, see below). Or you didn’t run it “as administrator”.

The downside is that whilst you can automate it, it is a little fiddly. Also whilst it is present on all 2008+ servers, it isn’t made available unless you enable Desktop Experience. That’s overkill just for running this utility though, but luckily you can just copy the required (two) files from winsxs and into the right places. Personally, I have a small .cmd file that copies the two files and then runs cleanmgr.exe.

Note that on a desktop OS such as Windows 7 or 8.x Disk Cleanup is always present. I recovered about 5GB on my Windows 7 laptop.

Disk Cleanup is actually a very powerful utility, hidden behind a fairly basic GUI. You can write your own custom cleanup tasks, either using the built-in DataDrivenCleaner COM object, or by writing you own (go on, you know you want to!). The DataDrivenCleaner can be used to search for (e.g.) files with a specific file extension in a certain folder that are older than a certain number of days.

Note that on Windows 7 and 2008 R2 (and probably Vista & Server 2008 too) you have to manually reboot after doing the Windows Update Cleanup, you won’t be prompted.

As the computer is shutting down, you then get a “Configuring Windows Updates” briefly, then as it starts back up you’ll get it again, and it might sit at 100% for a while:
cleanmgr cleanup 1

Then it’ll reboot again (automatically), and on the way up will sit at “Cleaning up.”:
cleanmgr cleanup 2

This might take a while too. The whole process could take half an hour or more, depending on how fast your CPU and disk are and how much stuff it found to tidy up. Once it finishes you’ll have the space back.

Posted in Windows | Tagged , , , , , , , | Leave a comment

PowerShell: Add users to Active Directory

Quickie script to add a load of users to Active Directory. This was to generate a set of test users, so the account settings are rather scarce.

You have to convert the password text string to data type System.Security.SecureString, which I’m doing inline. If you just use a text string for the password you’ll get a “Cannot convert” error.

for ($i = 1; $i -lt 10; $i++)
{
    Write-Host ("User"+$i)
    New-ADUser -DisplayName "User$i" -AccountPassword (ConvertTo-SecureString -AsPlainText "Pa55w0rd" -Force) -Name "User$i" -Enabled $true
}

Note that if you run the script on a domain controller, you need to be running PowerShell “As Administrator” or you’ll get an “Access is denied” error. If you run the script from elsewhere you just need permission to create the accounts.

 

Posted in PowerShell | Tagged , , , , , , , , , , , , , | Leave a comment

PowerShell: Email on high CPU usage

Another quickie. The requirement was to get an email alert when CPU usage reached over 90%. This is achievable via PerfMon Data Collector Set alerts, EventLog task triggers and a script to send an email (unless you’re on pre-Server 2012 when you could send an email as an event log trigger directly – why was that option removed?).

But that’s a load of faff, whereas this script is tiny and easy to schedule to start via Task Scheduler. Or just run it when you log on (though you never log on to the console of a server do you…?).

It demonstrates how easy it is to get values out of the performance counters without having to resort to WMI.

while($true){
    if((Get-Counter -Counter "\Processor(_Total)\% Processor Time").CounterSamples.CookedValue -ge 90){
        Send-MailMessage -Body "CPU is high, maybe take a look" -From "yourserver@yourdomain.com" -SmtpServer "smtp.yourdomain.com" -Subject "CPU High on yourserver" -To "you@yourdomain.com"
    }
    Start-Sleep -Seconds 5
}

Clearly you could make this nicer, e.g. have a counter that stops it emailing more than once every x number of times within a certain time period.

Posted in Performance, PowerShell | Tagged , , , , , , , , , , , , | Leave a comment

PowerShell: Check if a process is running and if not start it

Had a requirement to do this, and here’s what I came up with:

if((Get-Process -Name notepad -ErrorAction SilentlyContinue) -eq $null){
    ."C:\Windows\notepad.exe"
}

The thing that happens if the process we check for, could obviously be anything. You might want to send an email (use Send-MailMessage) or start a different executable or whatever.

Note that you don’t put .exe on the end of the executable name when using Get-Process.

Posted in PowerShell | Tagged , , , , , | Leave a comment

Active Directory Dynamic DNS broken by Juniper SRX firewall ALG

Last week a Juniper SRX firewall was put in to replace an older internal firewall. This week a load of Windows Servers dropped out of Active Directory DNS. The scavenging period in AD DNS was configured to be 7 days. Coincidence? I think not.

But what was going on, and why? DNS queries were working from the affected servers, and servers not behind the new SRX firewall were unaffected. I ran Wireshark on an affected server and on the primary DNS server it was configured to talk to.

There were indeed UDP packets going back and forth quite happily between the affected server and the DNS server, originating on a high port within the range 49152 – 65535 and going to port 53 on the DNS server. So I knew that it wasn’t a firewall port rule thing – DDNS uses the same ports as any other DNS traffic.

DNS traffic contains a set of flags that tell the DNS server what type of operation it should do with the rest of the data in the communication. For a standard DNS query the flags are 0x0100. These were being sent and received as normal: I could see the DNS transaction IDs matching up on both the affected server and the DNS server.

However, a DDNS update is distinguished by having the flags set to 0x2800, and for these all I could see was the dynamic update data being sent out of the affected server but this was never received at the DNS server. The effected server’s DNS Client service would keep sending the data for a while and then give up and a warning event 8015 in the System Event Log:

Source: DNS Client Events
Event ID: 8015
Level: Warning
User: NETWORK SERVICE
Message: The system failed to register host (A or AAAA) resource records (RRs) for network adapter with settings:
 Adapter Name : {xxxx}
 Host Name : servername
 Primary Domain Suffix : rcmtech.co.uk
 DNS server list :
    192.168.1.10, 192.168.1.11
 Sent update to server : <?>
 IP Address(es) :
  192.168.2.3
The reason the system could not register these RRs was because the update request it sent to the DNS server timed out. etc...

So something was eating the Dynamic DNS update data as it went over the network, but was not eating regular DNS queries. Whatever it was was looking at the flags inside the DNS data and only blocking dynamic update DNS traffic.

So I did a bit of searching and found that this was a known problem with Juniper SRX firewalls due to the ALG (Application Layer Gateway). I spoke to my SRX expert who assured me that ALG was switched off. After a few more hours of testing he discovered that ALG was in fact not switched off after all, and after disabling it for DNS everything is working normally again. It seems that ALG defaults are generally not ideal anyway when it comes to DNS.

Posted in Networking, Windows | Tagged , , , , , , , , , , , , , , , , , , , , | Leave a comment