Multipathing Support in vSphere Standard Edition

A customer recently asked me if storage multipathing is supported in vSphere Standard Edition.  The customer had reviewed the vSphere Editions comparison chart at https://www.vmware.com/products/datacenter-virtualization/vsphere/compare-editions.html and saw that Standard Edition did not include the ‘Storage APIs for Array Integration, Multipathing’ feature.  While the answer was quick and easy, I saw a good chance to provide some detailed information to the customer to better equip them to make an informed purchasing decision and to run their vSphere environment in the long run.  So what is the answer?  vSphere Standard Edition supports multipathing for storage.  This means that you can use Round-Robin for your block-based iSCSI, FC, amd FCoE storage connectivity.  What you don’t get with vSphere Standard Edition are 3 of the 4 Storage API’s for vSphere:
vSphere API for Storage Awareness (VASA), vSphere API for Array Integration (VAAI), vSphere Storage APIs for Multipathing.  The fourth, vSphere Storage API for Data Protection, is included in all vSphere editions.

Now the follow-up question, “should we upgrade our vSphere edition to gain access to these additional storage APIs?  Is it worth it?”  The exercise of ascribing worth to anything in this world is tough, and often times takes you on some interesting (terrifying) paths of introspection, unexpected tangents, and a re-evaluation of your core values.  Determining if a vSphere edition upgrade is ‘worth it’ can be tough from a technical and product packaging perspective.  But I digress – I’m not going to get all philosophical on you here, so back to the original question: are the additional storage APIs worth the cost of a higher vSphere edition?  Let’s break it down.

VASA is becoming increasingly important in the software defined datacenter.  VASA allows us to tag our virtual workloads with the desired characteristics of underlying storage, and then let the software place workloads on the right datastores.  I talk more about VASA here: https://vmtoday.com/2013/03/configuring-vmware-vasa-for-emc-vnx/.  While I don’t see many customers asking for VASA today, I see it as a must-have for the next generation virtualized / software-defined data center.

VAAI is an important tool for improving scalability and performance of a virtualized data center.  By offloading certain tasks from the hypervisor to the storage array, we can see greater VM densities (albiet with some limitations) and better performance for certain operations.  VAAI is especially helpful in dynamic vSphere environments where frequent Storage vMotion operations are performed, Linked Clones are used (vCloud Director or Horizon View virtual desktops).  While not essential, VAAI support is included in nearly all new arrays and does improve performance and VM:host and VM:datastore ratios, which could drive down hardware and software costs.

And finally, vSphere Storage APIs for Multipathing.  As I stated earlier, multipathing is supported in all vSphere editions.  To better understand what you get by adding the vSphere Storage APIs for Multipathing, let’s look at how multipathing works in vSphere.  vSphere storage is built on a Pluggable Storage Architecture (PSA) – a modular approach to building the vSphere storage stack.  Within the PSA, we have multipathing plug-ins (MPP).  A native multipathing plug-in (NMP) is included in all editions of vSphere.  MPPs support two sub-types of plug-ins: Path Selection Plug-ins (PSP) and Storage Array Type Plug-In (SATP).  PSPs are responsible for defining which physical path ESXi uses to for each I/O request.  Default PSPs in the NMP include Round-Robin, Most Recently Used, and Fixed.  You can learn more about there here: VMware PSPs.  SATPs define how physical paths are monitored and how fail-over happens should a path become dead, as well as defining optimal paths to storage arrays.  vSphere includes a few generic SATPs for active-active, ALUA, and local arrays, as well as a bunch of array-specific SATPs for different manufacturers arrays.  To help you visualize this, I put together the diagram to the left (click to enlarge) depicting where the NMP, PSPs, and SATPs are situated within a highly-available storage area network.

The NMP and its associated PSPs and SATPs provide a pretty comprehensive solution, but with the vSphere Storage APIs for Multipathing, VMware allows third party vendors to write their own multipathing plug-ins with PSPs and SATPs that are optimized for their own arrays.  A prime example is EMC’s PowerPath/VE.   PowerPath/VE provides dynamic host registration against the storage array, dynamic path testing, and a load-based multipathing PSP for both EMC and many third-party arrays.  How do these features help? Here are some examples:

• As data centers are becoming more dynamic (i.e. we’re building up and tearing down physical and virtual resources more frequently), dynamic host registration against arrays is helpful.  Including PowerPath/VE in a vSphere AutoDeploy repository can reduce host build time by auto-registering the host to the array.
• Dynamic path testing may detect a path failure or an non-optimal path more quickly than a native vSphere plug-in, and orchestrate a graceful path fail-over (and fail-back).
• Load-based multipathing in PowerPath/VE evaluates workloads and tests paths to ensure that I/O is balanced between paths, and the selected path is well suited for the VM workload being sent to the array.  This is in contrast to the NMP’s Round-Robin PSP where a set number of I/O’s are sent down a given path (1000 IOs by default, but often changed to 1 I/O) before the next path is used for I/O.  There is some conflicting information on how much a PSP like PowerPath/VE can improve performance (see Redefining ESXi IO Multipathing in the Flash Era). Anecdotally, in a dense vSphere environment I managed a few years ago (40:1 VMs including SQL, Exchange and other decent sized workloads), I saw a noticable performance improvement (greater throughput and reduced latency) after installing PowerPath/VE.

So are the vSphere Storage APIs for Multipathing ‘worth it’?  I would say yes if you have a highly dynamic environment, are pushing higher densities/consolidation ratios, or need enhanced monitoring of paths (PowerPath/VE can work with EMC’s Virtual Storage Integrator to display path information in vCenter and have an array that is supported with PowerPath/VE or another vendor’s MPP.

While the Storage API’s are valuable, don’t stop there when evaluating whether to upgrade to a higher vSphere edition ; consider the other benefits.  As you move from Standard to Enterprise and Enterprise Plus, you gain the following features:

• Virtual Serial Port Concentrator
• Distributed Resources Scheduler (DRS)
• Distributed Power Management (DPM)
• Storage I/O Control (SIOC)
• Network I/O Control (NIOC)
• vSphere Distributed Virtual Switch
• Host Profiles
• Auto Deploy
• Storage DRS
• Profile-Driven Storage
• Single Root I/O Virtualization (SR-IOV) Support

These additional features all combine to provide a highly automated virtualized data center with a ton of software-defined functionality.  I would make the argument that the Enterprise Plus functionality is essential (‘worth it’) for any virtualized data center.

And if you’ll indulge me while I hop on my soap-box, I would also argue that the three current vSphere editions should be collapsed into a single vSphere edition to provide customers with the complete kit for building a virtualized and/or software-defined data center.  vSphere is vSphere is vSphere – make the choice easy for building a basic virtualized environment.  Then offer a private cloud with automation bundle with orchestrated deployment/self service by vCloud Director, enhanced monitoring with vCenter Operations, and automated disaster recovery with SRM (basically vCloud Suite Advanced, plus SRM).  Finally, offer a hybrid cloud bundle with multi-cloud management capabilities (vCloud Automation Center, vFabric).  This keeps the Good → Better → Best scheme that the sales and marketing people love, but makes it simple for customers to take big steps along their journey from virtualized infrastructure to hybrid cloud <stepping down from soapbox now>.

I hope this clears up some licensing confusion for you, and gives you some good technical information for deciding which vSphere edition / features are right for you.  Questions, comments, heckling: leave a comment below!