VMware vSphere Resource Pools – Resource Allocation Revisited

I have found VMware vSphere Resource Pools to be an often misunderstood element, and incorrectly implemented by even seasoned VMware administrators.  I recently found an example of an incorrect implementation of Resource Pools being used to organize VM’s in vCenter’s Hosts and Clusters view, without considering the potential performance impact of this configuration.  I thought that I might try to explain the resource pool piece a bit better, particularly around Shares.

First, a few technical points to remember:

• Resource Pools can only be used when VMware Distributed Resource Scheduler (DRS) is enabled on a cluster.  DRS is available in vSphere Enterprise and Enterprise Plus editions.
• VMware Shares are like stock/shares in a company –> Shares are proportional.  The more  shares you issue (on your company or in your VMware environment), the less valuable the individual share is.  The more shares you have, the greater your voting power/ability to claim a right to resources is.
• The vSphere Hosts Cluster itself is technically a Resource Pool too.
• Shares only come into play when there is resource contention (Memory or CPU).  If the host/cluster has spare resources available for a guest it allocates availability of those resources based on the configured vRAM size and vCPU count, taking into consideration any limits (Mem/CPU) on the VM.
• Resource Pools can be used to apply resource Shares to VM’s, as well as Reservations and Limits.  Shares, Reservations, and Limits can also be set on a per-VM basis.  The values on Resource Pools and individual VM’s combined can do funny things.

Now, consider the following screen shot of the environment running on my laptop lab:

I have 16 VM’s total, 4 in ResourcePool1, and 1 VM in ResourcePool2.  The resource pools are set to ‘Normal Shares’ with no other configuration changes to the default.  The VM’s are all the same size as far as vRAM and vCPU are concerned.

If we look at the ‘Resource Allocation’ tab of the Cluster, we see that the VM’s in the Cluster resource pool each have 5% of the Shares for CPU, and each ‘Normal’ resource pool has 21% of the total shares of the cluster.  If we divide the 21% shares by the 4 VM’s in ResourcePool1, we find that each VM gets 5.25% of the cluster’s total shares.  That means that the VM’s in this ‘Normal’ pool actually get a bit more CPU time under resource contention than those in the parent cluster pool.  The single VM (MoveVM1) in ResourcePool2 gets 21% of the shares all to itself – much larger than the remaining VM’s in the Cluster, even though it may not be a VM of higher importance in your environment.

Now consider Memory resource allocation:

The ‘Normal’ shares value for memory on a resource pool is 163840.  Compare this to the 640 shares each single VM gets in my current config (VM-* each have 64MB RAM, so a very low share value per VM).  We’ve effectively configured only a fraction of 1% (basically 0% shares) to the VM’s in the parent cluster pool, while the few VM’s in the ResourcePool1 and ResourcePool2 get to share 48% of cluster resources per pool.  This still leads to an imbalance for the VM’s in each pool as compared to VM’s in the sibling pool and to the VM’s in the parent pool.  I.E. – MoveVM1 gets all 48%; while the four VM’s in ResourcePool1 share 48% (48/4=12% each).

Now let’s say that ‘MoveVM1’ was just a VM that a well-intentioned administrator spun up for some testing, putting it in a resource pool for ease of locating it in an otherwise busy vCenter view.  This otherwise inconsequential testing VM now has the right to claim 48% of the memory (vRAM) in the cluster and 21% of CPU time when there is resource contention in the cluster.  Now if this lowly testing VM is the cause of resource contention on an otherwise healthy cluster, it’s getting its cake and eating it too.  When the going get’s tough, the least valuable workload gets the priority in this configuration!  We’ve created a case where the meek (VM) shall inherit the resources.  All your resources are belong to us.  You get the idea – a big fat oops!

Hopefully you can see how using Resource Pools as units of organization can negatively affect the performance of an environment under resource contention, even if those resource pools are configured with ‘default/normal’ values.  The risk becomes more severe if the values of those resource pools were to change (say, move to a ‘High’ value for shares on ResourcePool2 in my lab).

Best Practices

• It is best to use folders the ‘VM’s and Templates View’ in the vSphere Client for organizing VM’s and only (sparingly) use resource pools to achieve specific performance goals under resource contention (shares) and/or resource allocation goals (limits & reservations) under normal operating circumstances with resource availability.
• Use the Resource Allocation tab to review and tune your Resource Pool (and individual VM) Shares settings.  Export the resource allocation list from vCenter and use Excel if you want to experiment with values and do what-if calculations.
• Restrict permissions to create and modify resource allocation on VM’s and on Resource Pools in vCenter to only VMware Administrators who understand the implications of these changes, and who are capable of doing the math to properly design a resource allocation model.  If you don’t have an expert on staff, engage a consultant to help (might I suggest Clearpath Solutions Group ;-)).
• Check out VMware vCenter Operations Manager (vCenter Ops/ vCOPs) for advanced monitoring and capacity planning (the product formerly known as VMware CapIQ is included with vCenter Ops).  Good monitoring and forecasting can help to avoid resource contention in highly consolidated, oversubscribed, or performance-sensitive environments (and what environments are not all three of these today?).
• Review the VMware vSphere Resource Management Guide to better understand the use and function of Resource Pools. The Guide can be found here: https://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-resource-management-guide.pdf.
• Buy  Duncan Epping’s and Frank Denneman’s book, VMware vSphere 5 Clustering Technical Deepdive to really understand how VMware vSphere Distributed Resource Scheduler (DRS) (as well as VMware High Availability (HA) VMware Distributed Power Management (DPM), and VMware Storage DRS) function and are best configured.