Ho ho ho, look how I compare myself to Mark Russinovich by the use of a post title.
Anyway, clearly my skills are somewhat more rudimentary. However, I did fix this problem using Process Monitor, and his blog post was a good start – and doing the two day David Solomon Windows OS Internals online course was handy too.
So, here’s the problem: two servers, both Windows Server 2003 SP2 and all updates as far as January 2011. One is a Dell PowerEdge 2850, the other is virtual running with one vCPU. On both these servers, when you right-click a file you get a pause of nearly 30 seconds before the menu opens. If you double-click a file you get a similar pause before it opens. Tedious huh?
Interestingly, double-click an executable – no hang, opens straight away. Open a command prompt and open the file from there, no delay.
So it’s related to explorer.exe. Mark starts off his investigations by running Process Explorer, and so did I, but to be honest the stack for each explorer.exe thread doesn’t mean much unless you know what most of the stuff actually means/does. So I gave up with that and moved to Process Monitor instead. This monitors file and registry access – and consequently gives you a load of information. Too much information usually. Luckily it has a good filtering system.
So I picked my test file, the help file for Process Monitor, procmon.chm. This, as with all files on the server, was taking ages to open. I also chose to work first on the PowerEdge, this would eliminate any SAN/CPU shared resource performance issues that the VM might have been experiencing.
So you fire up Process Monitor and set the following filters (in addition to the default exclude filters):
Process Name - is - Explorer.exe - Include
Process Name - is - hh.exe - Include
Result - is not - SUCCESS - Include
Where hh.exe is the HyperHelp executable that opens the .chm file that I was using for testing. Having set the filters, you click the Clear button, do whatever it is that you want to monitor, then click the Capture button to stop capturing any more stuff.
So I’m scrolling down the list keeping an eye out on the Time of Day column, looking to see if the time jumps at any point. And indeed it does, from 08:32:46 to 08:33:11 from one line to the next. The two lines are as follows:
Operation - Path - Resultand just above the first line was a similar line pointing to the main HKEY_CLASSES_ROOT:
RegOpenKey - HKCU\Software\Classes\Applications\TextPad.exe\shell \edit\command - NAME NOT FOUND
RegOpenKey - HKCU\Software\Classes\Applications\themes.exe - NAME NOT FOUND
RegOpenKey - HKCR\Applications\TextPad.exe\shell\open
This was more interesting – I don’t really care too much what’s wrong in my chunk of the registry (HKCU) but Classes Root could affect the whole server. So I had a look in that registry location, and found this:
(Default) - REG_SZ - "\\Development\c$\Program Files\TextPad 4\TextPad.exe" "%1"
I happen to know that server (called Development) was decommissioned a while ago, i.e. it’s not contactable.
So I exported the TextPad.exe registry key (inlcuding all subkeys and values) and then deleted it. Double clicking my test .chm file now made it open immediately but the right-click menu was still just as slow. I searched for any more instances of the word “Development” in the rest of the resgistry and found several more referring to TextPad and this server plus UNC path. I followed the same procedure, export to file then delete. Now the right-click context menu opens immediately too.
Case closed. ;-)
Lessons to be learnt?
- Don’t run anything on a server other than the actual server application itself. It’s a server not a PC, yes the GUI looks like your PC but a server is there to provide a different function.
- Use remote management tools and remote access whenever possible. As above, you have a PC for running this stuff on, a Remote Desktop or console session uses up a lot of resources that are better allocated to the server application.
- Don’t run software from UNC paths – the Explorer shell doesn’t like it when it can’t find stuff that it thinks it should be able to.