Set Hyper-V 2012 VM NUMA node

Run the following PowerShell command line and you can see what NUMA node your VMs are running on, assuming you a) have NUMA aware host hardware and b) haven’t enabled NUMA spanning:

(Get-Counter -ComputerName "HFR3-UWE12" -Counter "Hyper-V VM Vid Partition(*)\Preferred NUMA Node Index").CounterSamples | Select-Object -Property InstanceName,CookedValue | Where-Object -Property InstanceName -NE "_total"

This will give you output similar to the following:

InstanceName           CookedValue
------------           -----------
VM01                             0
VM02                             1
VM03                             1
VM04                             0

Where the number in the CookedValue column is the NUMA node.

This will be set when the VM is powered on, and will remain the same until the VM is powered off or Live-migrated off the host.

If you need to fit some VMs onto a host and are tight on RAM you might want to override this and specify which NUMA node a VM will be placed onto when you power it on. There are two ways to accomplish this: use WMI or edit the VM configuration XML file.

The following PowerShell will let you modify the setting for a VM. Note that the VM must be powered off for this to work. It was initially based on this, which didn’t work, so I then re-wrote it with help from here.

$VMName = "VM01"
$HostName = "HVHost01"

# Set to 1 to enable NUMA Node preference
$NumaNodesAreRequired = 1
# Set to the preferred NUMA node number (starting at 0)
$NumaNodeList = 1

$VM = Get-WmiObject MSVM_ComputerSystem -Namespace root\virtualization -ComputerName $HostName | where-object -property ElementName -eq $VMName
$VMManagementService = Get-WmiObject -Namespace root\virtualization -Class Msvm_VirtualSystemManagementService -ComputerName HFR3-UWE12
Write-Host "Before:"
foreach($VSSettingData in $VM.GetRelated("MSVM_VirtualSystemSettingData")){
    Write-Host ("VM Name  : "+$VSSettingData.ElementName)
    Write-Host ("NUMA Rqd.: "+$VSSettingData.NumaNodesAreRequired)
    Write-Host ("NUMA Node: "+$VSSettingData.NumaNodeList)
}
foreach($VSSettingData in $VM.GetRelated("MSVM_VirtualSystemSettingData")){
    $VSSettingData.NumaNodeList = @($NumaNodeList)
    $VSSettingData.NumaNodesAreRequired = $NumaNodesAreRequired
    $VMManagementService.ModifyVirtualSystem($VM, $VSSettingData.PSBase.GetText(1)) | Out-Null
}
Write-Host "After:"
$VM = Get-WmiObject MSVM_ComputerSystem -Namespace root\virtualization -ComputerName $HostName | where-object -property ElementName -eq $VMName
foreach($VSSettingData in $VM.GetRelated("MSVM_VirtualSystemSettingData")){
    Write-Host ("VM Name  : "+$VSSettingData.ElementName)
    Write-Host ("NUMA Rqd.: "+$VSSettingData.NumaNodesAreRequired)
    Write-Host ("NUMA Node: "+$VSSettingData.NumaNodeList)
}

So we’re getting a WMI object for the VM specified ($VM), pulling out the VirtualSystemSettingData into $VSSettingData (note that this is actually in XML, even though PowerShell allows us to treat it like an object) and displaying the current NUMA node data. Then we change the NUMA node data and write the XML back via the WMI method ModifyVirtualSystem (hence the need to use the .PSBase.GetText(1) method. You have to set the NumaNodesAreRequired value to 1 (True) if you want to specify the NUMA node for the VM, otherwise the settings you specify in NumaNodeList is just a preference and the VM might actually end up on a different node. We then get the VM WMI object again so that you can check that the values have changed (they won’t if the VM is running).

I’d probably go with the WMI method, but it’s interesting to see the values in the XML so here’s where to look:

<configuration>
  <settings> 
    <numa_node_mask type="integer">1 
    <numa_nodes_required type="bool">True
  </settings>
</configuration>

Note that in the PerfMon counters and the PowerShell script above the NUMA nodes are referenced starting at 0, but in the XML they start at 1. The two settings seem to be inserted between the memory and processors sections when using the PowerShell code above.

This entry was posted in Hyper-V, PowerShell and tagged , , , , , , . Bookmark the permalink.

3 Responses to Set Hyper-V 2012 VM NUMA node

  1. Pingback: Hyper-V 2012 VM RAM and NUMA nodes | Robin CM's IT Blog

  2. Brian H says:

    This doesn’t work in 2012 R2 due to the new v2 namespace, and the fact that it appears that the properties NumaNodeList and NumaNodesAreRequired seem to be no longer supported by either WMI or the xml file.

    Any idea how to accomplish this in R2?

  3. rcmtech says:

    In a word: no. As you say, the stuff I use above just doesn’t seem to be there anymore. I’ve had to work around this limitation by just putting less stuff on each host. The hosts had a relatively low total memory to VM RAM size ratio anyway, which is why I was trying to shoehorn the VMs onto the NUMA nodes. Once I upgraded the hosts to 2012 R2 and this all broke I just had to add some extra hosts (they’re all old hardware anyway – I considered upgrading the RAM but adding extra hosts was a zero cost option as the hardware was lying around having been decommissioned from something else.
    Note that the RAM overhead calculations seem to be total nonsense, I found I was having to ensure that I had plenty of spare RAM, far and away more than I expected, or otherwise VMs would sometimes not power on (but other times would be fine – even on 16 identical hosts with identical VMs on each host).
    Lesson is to not be too tight with your RAM allocation. I was only doing the NUMA stuff because I had previously switched from the same hosts running XenServer 5.something, which wasn’t NUMA-aware, so the VMs just sat across the two banks of RAM. Performance was fine though so I didn’t care. I changed the hosts over to Hyper-V to improve VM build automation and host monitoring and then found I couldn’t get the same size VMs to fit without the fiddling around above.
    This article is somewhat outdated now – I suspect few people are running non-R2 2012 Hyper-V hosts now.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s