VMware vSphere Resource Pools – Resource Allocation Revisited

I have found VMware vSphere Resource Pools to be an often misunderstood element, and incorrectly implemented by even seasoned VMware administrators.  I recently found an example of an incorrect implementation of Resource Pools being used to organize VM’s in vCenter’s Hosts and Clusters view, without considering the potential performance impact of this configuration.  I thought that I might try to explain the resource pool piece a bit better, particularly around Shares.

First, a few technical points to remember:

  • Resource Pools can only be used when VMware Distributed Resource Scheduler (DRS) is enabled on a cluster.  DRS is available in vSphere Enterprise and Enterprise Plus editions.
  • VMware Shares are like stock/shares in a company –> Shares are proportional.  The more  shares you issue (on your company or in your VMware environment), the less valuable the individual share is.  The more shares you have, the greater your voting power/ability to claim a right to resources is.
  • The vSphere Hosts Cluster itself is technically a Resource Pool too.
  • Shares only come into play when there is resource contention (Memory or CPU).  If the host/cluster has spare resources available for a guest it allocates availability of those resources based on the configured vRAM size and vCPU count, taking into consideration any limits (Mem/CPU) on the VM.
  • Resource Pools can be used to apply resource Shares to VM’s, as well as Reservations and Limits.  Shares, Reservations, and Limits can also be set on a per-VM basis.  The values on Resource Pools and individual VM’s combined can do funny things.

Now, consider the following screen shot of the environment running on my laptop lab:

  vSphere Resource Pool Shares CPU Resource Allocation

I have 16 VM’s total, 4 in ResourcePool1, and 1 VM in ResourcePool2.  The resource pools are set to ‘Normal Shares’ with no other configuration changes to the default.  The VM’s are all the same size as far as vRAM and vCPU are concerned.

If we look at the ‘Resource Allocation’ tab of the Cluster, we see that the VM’s in the Cluster resource pool each have 5% of the Shares for CPU, and each ‘Normal’ resource pool has 21% of the total shares of the cluster.  If we divide the 21% shares by the 4 VM’s in ResourcePool1, we find that each VM gets 5.25% of the cluster’s total shares.  That means that the VM’s in this ‘Normal’ pool actually get a bit more CPU time under resource contention than those in the parent cluster pool.  The single VM (MoveVM1) in ResourcePool2 gets 21% of the shares all to itself – much larger than the remaining VM’s in the Cluster, even though it may not be a VM of higher importance in your environment.

Now consider Memory resource allocation:

 vSphere Resource Pool Shares Memory Resource Allocation

The ‘Normal’ shares value for memory on a resource pool is 163840.  Compare this to the 640 shares each single VM gets in my current config (VM-* each have 64MB RAM, so a very low share value per VM).  We’ve effectively configured only a fraction of 1% (basically 0% shares) to the VM’s in the parent cluster pool, while the few VM’s in the ResourcePool1 and ResourcePool2 get to share 48% of cluster resources per pool.  This still leads to an imbalance for the VM’s in each pool as compared to VM’s in the sibling pool and to the VM’s in the parent pool.  I.E. – MoveVM1 gets all 48%; while the four VM’s in ResourcePool1 share 48% (48/4=12% each).

Now let’s say that ‘MoveVM1’ was just a VM that a well-intentioned administrator spun up for some testing, putting it in a resource pool for ease of locating it in an otherwise busy vCenter view.  This otherwise inconsequential testing VM now has the right to claim 48% of the memory (vRAM) in the cluster and 21% of CPU time when there is resource contention in the cluster.  Now if this lowly testing VM is the cause of resource contention on an otherwise healthy cluster, it’s getting its cake and eating it too.  When the going get’s tough, the least valuable workload gets the priority in this configuration!  We’ve created a case where the meek (VM) shall inherit the resources.  All your resources are belong to us.  You get the idea – a big fat oops!

Hopefully you can see how using Resource Pools as units of organization can negatively affect the performance of an environment under resource contention, even if those resource pools are configured with ‘default/normal’ values.  The risk becomes more severe if the values of those resource pools were to change (say, move to a ‘High’ value for shares on ResourcePool2 in my lab).

Best Practices

  • It is best to use folders the ‘VM’s and Templates View’ in the vSphere Client for organizing VM’s and only (sparingly) use resource pools to achieve specific performance goals under resource contention (shares) and/or resource allocation goals (limits & reservations) under normal operating circumstances with resource availability.
  • Use the Resource Allocation tab to review and tune your Resource Pool (and individual VM) Shares settings.  Export the resource allocation list from vCenter and use Excel if you want to experiment with values and do what-if calculations.
  • Restrict permissions to create and modify resource allocation on VM’s and on Resource Pools in vCenter to only VMware Administrators who understand the implications of these changes, and who are capable of doing the math to properly design a resource allocation model.  If you don’t have an expert on staff, engage a consultant to help (might I suggest Clearpath Solutions Group ;-)).
  • Check out VMware vCenter Operations Manager (vCenter Ops/ vCOPs) for advanced monitoring and capacity planning (the product formerly known as VMware CapIQ is included with vCenter Ops).  Good monitoring and forecasting can help to avoid resource contention in highly consolidated, oversubscribed, or performance-sensitive environments (and what environments are not all three of these today?).
  • Review the VMware vSphere Resource Management Guide to better understand the use and function of Resource Pools. The Guide can be found here: http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-resource-management-guide.pdf.
  • Buy  Duncan Epping’s and Frank Denneman’s book, VMware vSphere 5 Clustering Technical Deepdive to really understand how VMware vSphere Distributed Resource Scheduler (DRS) (as well as VMware High Availability (HA) VMware Distributed Power Management (DPM), and VMware Storage DRS) function and are best configured.

Comments

  1. Good stuff, I like the “stock market” example as a good way to explain shares. If you want a PowerCLI script that helps keep shares balanced, I have one over on a post I did on this topic as well: http://wp.me/p1bQqu-mJ

    • Awesome – thanks, Chris. I used your post on PSOD of ESXi 4.1 when joining vCenter 5 for the same client, same day. What a great collaboration community we’ve got going on!

  2. Awesome – thanks, Chris. I used your post on PSOD of ESXi 4.1 when joining vCenter 5 for the same client, same day. What a great collaboration community we’ve got going on!

  3. Remember that Resource Pools can also be used on hosts which are not part of a cluster, even hosts that are not managed by vCenter Server – the “root” resource pool object can either be a host OR a DRS cluster.

    I use Provider-Consumer terminology when explaining Resource Pools to people:

    When you have no Resource Pools, the host/cluster is the Provider and the VMs are the Consumers, both in terms of utilised and reserved resources.

    If you introduce Resource Pools under a host/cluster, the host/cluster is only the Provider to the Resource Pools – the Resource Pools are the Consumers. The Resource Pools themselves are also Providers and it’s their child VMs that are the Consumers.

  4. Remember that Resource Pools can also be used on hosts which are not part of a cluster, even hosts that are not managed by vCenter Server – the “root” resource pool object can either be a host OR a DRS cluster.

    I use Provider-Consumer terminology when explaining Resource Pools to people:

    When you have no Resource Pools, the host/cluster is the Provider and the VMs are the Consumers, both in terms of utilised and reserved resources.

    If you introduce Resource Pools under a host/cluster, the host/cluster is only the Provider to the Resource Pools – the Resource Pools are the Consumers. The Resource Pools themselves are also Providers and it’s their child VMs that are the Consumers.

  5. Nice article Josh. I really wish VMware would just give us a standard folder, like an OU. I know people (who I won’t name) that use RP’s just for vShield. It would be nice if vShield would address those folders created in the VM view.

  6. Nice article Josh. I really wish VMware would just give us a standard folder, like an OU. I know people (who I won’t name) that use RP’s just for vShield. It would be nice if vShield would address those folders created in the VM view.

  7. So I am a little confused. So if I have an application that requires 10 virtual machines to run. It is considered a mission critical application so it needs to have access to the resources it requires to run at all time (during contention/non-contention) should I bother creating a resource pool or just do a per-vm reservation (assuming I am ok with the slot-size impact for HA)? It seems like I am gauranteeing (sp) peak performance with reservation and not necessarily with a resource pool. I read alot of these articles and it seems the big issue is around shares. What about reservations and limits with resource pools?

  8. So I am a little confused. So if I have an application that requires 10 virtual machines to run. It is considered a mission critical application so it needs to have access to the resources it requires to run at all time (during contention/non-contention) should I bother creating a resource pool or just do a per-vm reservation (assuming I am ok with the slot-size impact for HA)? It seems like I am gauranteeing (sp) peak performance with reservation and not necessarily with a resource pool. I read alot of these articles and it seems the big issue is around shares. What about reservations and limits with resource pools?

  9. this is good info! thx for the insight.

  10. this is good info! thx for the insight.

  11. This is an excellent post. Thanks!

Drop a comment below:

%d bloggers like this: