VMtoday

VMware News, Views, and How-To's from vExpert Josh Townsend

You are here: Home / Archives for Storage

Storage

EMC Elect 2013 Awardees Announced

by 1 Comment

EMC Elect 2013 LogoEMC announced today the inaugural 2013 class of EMC Elect.  The EMC Elect program, announced in November, recognizes EMC community including customers, partners, EMC employees, analysts and independent voices for going above and beyond to share their knowledge, collaborate, communicate (on social media, the EMC Community Network, at events like EMC World, etc.), and for being generally awesome in their EMC area of expertise.

A panel of highly respected foundering members of the EMC Elect were chosen for their extensive participation with EMC as a brand. Their leadership, commitment and positive attitude set the cadence of the community. These 10 members reviewed the 270 nominations to choose their fellow EMC Elect.  As a founding member, I can say that the review process was rigorous considering each nominee’s contributions to the community as a whole.

I invite you to join me in congratulating these folks – the full list of EMC Elect 2013 members is here: https://community.emc.com/community/connect/emc_elect?view=blog).  I know there are many people deserving of the recognition who were not nominated this year.  I encourage you to think about who in your own local EMC community is influential and most helpful and nominate them for next year’s class of EMC Elect!

Congrats to the new EMC Elect!

Storage Basics – Part IX: Alternate IOPS Formula

by 12 Comments

StorageBasics9It’s been a long time since I published an article in my Storage Basics series – the series has been some of the top content on my site to date.  I sat for a beta version of the VMware Certified Advanced Professional – Desktop Design (VCAP-DTD) test today and was reminded by the test of a post I had in waiting about an alternate way to calculate IOPS (hint hint – memorize these formulas).  I have several more articles sitting in draft form, but a new job, crazy kids, home improvement projects, and a wife with chronic cerebral spinal fluid (CSF) leaks (it sounds bad – and it is – but look on the bright side: not many guys can get away with saying that their wife’s brain leaks!) all take time.  But enough excuses and back to storage….

I wrote in Storage Basics – Part II: IOPS on how to calculate theoretical IOPS for a given disk type and speed, and then followed it up in Storage Basics – Part III: RAID with some information on  how different RAID configurations impact IOPS.  I.E. More disks equals more IOPS, but with a write penalty assigned for different RAID types.  I then moved into a discussion of interconnects in Storage Basics – Part IV: Interface, but didn’t do a good job there of tying the interconnect speed to performance and IOPS.  Finally, in Storage Basics – Part VI: Storage Workload Characterization, I hinted briefly on how I/O request size can impact storage performance.  Let’s dig into this topic a little deeper!

The Original IOPS Forumula

Calculating IOPS at the disk/RAID level is fun and all (and I suggest you brush up on it if you are sitting for the VCAP exams), but does not paint a full picture of the storage subsystem.  The basic formula to get a single disk IOPS (I covered this in Part II) looks like this:

\frac{\text{Seek Latency} + \text{Rotational Latency}}{1000} = IOPS

(The LaTeX formulas keep getting messed up – sorry if they don’t look right. Let me know and I’ll fix them, again).
For simplicity sake, let’s just assume that this formula yields 180 IOPS for a standard 15k RPM SAS disk (the industry standard number of IOPS for this speed and type of disk)

The formula for RAID IOS is in Part III – here’s a sample for RAID5, where:

p = IOPS required, f = write IO penalty factor, r = % Read, w = % Write

{(p \times r)} + {((p \times w) \times f)} = \text{Back-end Spindle IOPS}

Let’s put it to use: let’s say I have a workload that I’ve measured as needing 4000 IOPS from the OS/application perspective, of which 70% are write (not unusual in VMware View Linked Clones): here’s how you figure out how many disks you need from an IOPS perspective for those linked clones:

{(4000 \times 30\%)} + {((4000 \times 70\%) \times 4)} = 12,400\ast

*12,400 Back-end IOPS needed on my spindles in a RAID5 array

Then, convert the number of back-end IOPS needed to a number of disks needed given our RAID5 assuming the 15k SAS disk:

\frac{12,400}{180} = 69

That is, I need 69 disks (rounded up) required on the back-end to meet our 4000 front-end IOPS, ignoreing things like best practices when putting a bunch of disks in a RAID group/storage pool (4+1 RAID5 sets) and hotspares (1 in 30 for SAS)

A quick run at the math suggests that you should probably not be using RAID5 for a write-heavy workload (run the math yourself with a write penalty of 2 for RAID0+1 to  see how many fewer disks you would need).  Also, this particular example leaves out any calculation for cache impact, including EMC FAST Cache or the View Storage Accelerator (CBRC).  If you have those technologies in the mix, then a very rough rule of thumb would be to size your back-end IOPS based on your front-end IOPS calculation (4000 in the example above, instead of 12,400) – I say very rough because this really depends on your cache size and cache hit ratio, and the rule pertains a bit more to read-heavy workloads than write heavy.  If you have CBRC in the mix, you *might* be able to greatly reduce and maybe eliminate the Read IOPS from the equation.  But before you go taking my word as gospel, test, test, and test some more using your particular workloads.

The Alternative IOPS Formula

But this is all back-end spindle stuff and me having fun with \text{LaTeX} (LaTeX is the language I used to make the cool math equations – if the forumlas don’t look right drop a comment so I can fix them. WordPress keeps eating the latex syntax…).  Hopefully your SAN admin already knows this, so go ahead and give her your front-end IOPS count and read:write ratio and you’ll get some LUN’s presented.  But you, the VMware admin, might be responsible for server hardware, including buying the interconnect card (HBA, iSCSI NIC).  How do IOPS inform your selection?  This is where the alternate formula for calculating IOPS comes in.  Here’s the forumla:

\frac{\text{Throughput (MB per second)} \times 1024 \text{(convert KB to MB)}}{\text{Block Size (KB/IO)}} = IOPS

[Read more…] about Storage Basics – Part IX: Alternate IOPS Formula

Announcing EMC Elect

by 3 Comments

EMC ElectEMC announced today a new community recognition program called EMC Elect.  Much like the VMware vExpert program or the Microsoft MVP program, EMC Elect provides individuals recognition for their contributions to the EMC community.  Programs like this are incredibly beneficial – for the honoree as well as for the sponsoring company and the wider technology community.  I’ve certainly benefited from being selected as a VMware vExpert, and I hope that VMware has benefited from my feedback as a result of my participation in the vExpert program.  More importantly, the positive feedback from the vExpert program has encouraged me to contribute more as a VMUG leader, blogger, consultant and advocate – and I hope above all that my contributions benefit the community.  With this in mind, I’m pleased to announce that I’ve been honored  as a Founder in the EMC Elect program.

The ten founding EMC Elect honorees represent community members from customers, partners, community advocates and EMC specialists. The founders are charged with jumpstarting the program, beginning with a community-driven, peer-nomination program to onboard the freshman class of the Elect.  Those honored by being selected for participation in EMC Elect will represent the EMC community at large.  While many of the Elect will be storage rockstars, others (like me) will represent the partner, customer, and greater community of storage, virtualization, and security professionals  – but we’ll all share the same goal of sharing knowledge, improving our craft, and driving the mutual success of the community.

Those selected to be an EMC Elect will earn a few great benefits:

  • Public recognition and status (and the responsibility that comes with being a leader)
  • Exclusive content and opportunities (think beta’s, briefings, a VMAX for the living room)
  • Privileged access and private communication channels (ability to shape and influence the products and community we all know and love)

There are pockets of top influencers hidden all throughout the EMC community – internally and externally; folks who really believe in driving excellence in what they do and sharing what they know with the community.  This is your opportunity to recognize these contributors by filling out a nomination form (don’t be shy about self-nomination either).  Nominate those employees, customers, and partners who you feel contribute significantly to the overall EMC community.  Nominations are open through December 15th.  I hope you’ll join us!

For all the details on the EMC Elect program, visit the EMC Community Network (ECN) site here: https://emc.im/RWbRKF

VMworld Europe News and Announcements

by Leave a Comment

vmworld 2012 right here right nowI wanted to let you all know about some announcements that have come out at VMworld Europe over the past couple days.  These announcements help round out some of the new products and features announced at VMworld 2012 in San Francisco.  These new and updated features are a big deal for customers who want to take a wholistic approach to deploying and managing a hybrid cloud across different service providers.  If you’re thinking about upgrading your vSphere license to a vCloud Suite license (you have until December 15th to claim the free upgrade to the vCloud Suite Standard Edition, along with aggressive discounting on upgrades to vCloud Suite Advanced and Enterprise), these new features may help you decide. I’ll cover the feature sets of the updated suites at the bottom of this post, but first – new and updated products:

End User Computing Updates

View is a pretty awesome product, but it can’t solve all desktop deployment scenarios.  We saw VMware Mirage announced and demoed at VMworld 2012 San Francisco.  Now, new validated designs from Nimble Storage, Cisco (on UCS-E) and other partners for ROBO desktops using View and/or VMware Mirage to deploy and manage remote desktops.  With Mirage, remote offices lacking resources or connectivity can be managed with the ease of a View desktop, without the constraints of connectivity and datacenter technologies. More on the ROBO solution here: https://blogs.vmware.com/euc/2012/10/using-vmware-mirage-in-remote-and-branch-offices.html and here: https://blogs.vmware.com/euc/2012/10/better-management-eucremote-branch.html.

New Products

vCloud Automation Center 5.1 – Based on DynamicOps (VMware acquisition in July 2012), vCloud Automation Center provides a solution for hybrid cloud governance through policy-based provisioning and cross-cloud automation of cloud resource request management.  vCloud Automation Center can manage VMware vSphere & vCloud environments, Microsoft Hyper-V and Azure, and public clouds like Amazon EC2.  The product is not yet available, but will be coming in Q4.

VMware IT Business Management Suite 7.5 – The VMware IT Business Management Suite gives customers transparency into the cost, quality and value of cloud services, providing a business context for IT activities and enabling fact-based decisions aligned with business goals and ultimately transform to become a broker of IT services.  Availability is expected in Q4 2012.

Updated Products

vFabric Application Director 5.0 –    standardizes and accelerates how customers model and deploy multi-tier applications to any cloud through easy-to-create blueprints with standardized, pre-approved OS and middleware components. While optimized for VMware vCloud Suite-based clouds, vFabric Application Director 5.0 will make it possible for customers to use the same blueprints to deploy applications across multiple virtual and hybrid cloud infrastructures, including Amazon EC2.  Availability is expected in Q4 2012.

With this release, VMware will continue to expand vFabric Application Director application support to include all Microsoft packaged applications (Exchange, SQL Server, SharePoint) as well as custom applications (Java,.Net, Ruby on Rails). VMware vFabric Application Director will also help simplify ongoing application maintenance tasks such as code and configuration, and it will support auto-scaling of applications in response to changing business needs. 

vCenter Operations Management Suite 5.6 – Enhancements to the VMware vCenter Operations Suite 5.6 continue to deliver on VMware’s strategy to converge management disciplines, incorporating new compliance views into the operations dashboard to help customers proactively enforce compliance with IT policies, security guidelines and regulatory requirements. New, customizable, group-based views will enable IT operations to more closely align with business priorities. VMware vCenter Operations Management Suite 5.6 will also include VMware vFabric Hyperic application-level monitoring to help customers proactively manage the health of business-critical applications, including Microsoft SQL Server, Exchange, SharePoint, Windows OS as well as Oracle Database and custom Java applications.  vCOps will be able to monitor machines running across a hybrid could, including VM’s on Amazon and Azure.

A new vCOps edition is now available – vCOps Manager Foundation.  This is FREE in all vSphere editions and provides vSphere Health and Performance monitoring, alerting, and trending beyond what vCenter can provide.  The Foundation edition cannot monitor VM’s, and cannot do Capacity Optimization, Chargeback, Configuration Management, etc.

Availability is expected in Q4 2012.

vCloud Connector 2.0 – a bridge that enables customers to transfer VMs and applications across vSphere and vCloud Director private cloud environments as well as public clouds from VMware vCloud Service Providers. vCloud Connector 2.0 will help make it possible to extend both networks and workload catalogs across hybrid cloud environments – in effect, extending the logical boundaries of the datacenter – without the need for reconfiguring networks or settings when moving workloads. Also, distributed teams will be able to coordinate their development efforts by ensuring that everyone has a single, synchronized workload catalog that spans across the cloud environment.

vCloud Connector will come in two editions – Core Edition and Advanced Edition.  Core comes free with any vCloud Director purchase, Advanced is packaged with all editions of the new vCloud Suites.  Availability is expected in Q4 2012.

vCloud Suite Changes

All of the new and updated products above are now included in the new vCloud Suite.

  • All editions of vCloud Suite (Standard, Advanced, and Enterprise) pick up the new vCloud Connector
  • vCloud Suite Advanced picks up new features of vCOps 5.6, including vSphere change and configuration management, Application Awareness, and Chargeback.
  • vCloud Suite Enterprise gets everything: vCloud Automation Center, vFabric, vCOps all features (including Configuration Manager, Storage Adapters for Symmetrix, VMAX, VNX, NetApp).

These new additions to the vCloud Editions do not bring any price changes.  Customers simply get new and enhanced functionality.  Here’s the updated edition breakdown:

vCloud Suite Standard vCloud Suite Advanced vCloud Suite Enterprise
Cloud Management
vCloud Automation Center

  • Provisioning and management of IT services in vSphere environments
 ✔
vFabric Application Director

  • Automated configuration and deployment of multi-tier cloud applications
 ✔
vCenter Operations Management Suite

  • Application Monitoring – OS, middleware, databases
  • Configuration Compliance – vSphere host & VM change and configuration
  • Performance & Capacity Optimization – analytics, dashboards and alerting
  • Application Awareness – discovery and dependency mapping
  • Chargeback – metered utilization reporting and accountability
 ✔✔✔✔

.

 ✔✔✔✔

✔
Cloud Infrastructure
vCenter Site Recovery Manager

  • Automated disaster recovery planning, testing, and execution
 ✔
vCloud Networking and Security

  • Advanced continuous availability, firewall and network
  • Software defined networking, security, and ecosystem integration
 ✔ ✔✔✔ ✔✔✔
vCloud Director & vCloud Connector Advanced Edition

  • Virtualized datacenters with multi-tenancy and public cloud extensibility
 ✔ ✔ ✔
vSphere

  • Virtualized infrastructure with policy-based automation
 ✔ ✔ ✔
Note: vCloud Suite Editions are licensed per CPU.

EMC News

We’re big fans of EMC at Clearpath (and they like us too: we were named their SMB Partner of the Year!).   While there are not a whole lot of new EMC announcements at VMworld Barcelona, there are some demos of using the new VMware vVols technology that virtualizes storage (SAN, NAS, DAS, internal, etc.) to manage and present distributed storage nodes to VM’s.  More explanations and demo videos of this technology preview here: https://virtualgeek.typepad.com/virtual_geek/2012/10/vmworld-2012-psst-want-to-see-the-future-of-storage-with-vmware-and-emc.html#more

ESXi 5 Won’t Boot From USB

by 21 Comments

I’ve been upgrading my home VMware lab to vSphere 5.1 today and ran into an issue that I thought I would share.  I had previously written about my home lab environment – I run HP xw8600 workstations, and because I’m using SATA disks behind a SAS controller they are being seen as ‘remote’ disks to the ESXi installer.  This means that I need another type of disk that will be seen as a ‘local’ disk so ESXi can create a scratch partition.  A USB thumb drive was the answer, but not without issues.

When I installed ESXi 5.0 on the thumb drive, the install completed without problem, but after the post-install reboot ESXi 5 failed to boot.  I received a non-system disk / no bootable device error.

When I installed ESXi 5.1 on the same thumb drive, the install completed without problem, and again, after the post-install reboot ESXi failed to load.  On ESXi 5.1, however, I didn’t receive an error, but just had a blinking cursor in the upper left corner of an otherwise blank screen.  The cursor just blinked endlessly – no ESXi loaded.

The fix was the same for both 5.0 and 5.1, even though the symptoms were different.  Because I’m running on vintage hardware, my BIOS does not support booting from GUID Partition Table (GPT) partitions.  As of vSphere 5.0, VMware changed the partitioning layout of the system disk from Master Boot Record (MBR) to GPT.  My motherboard does not have a UEFI option, so booting from GPT isn’t going to work.  A GPT is not necessary on a 4GB thumb drive (GPT is necessary for partitions over 2.20TB), so the fix was to simply format my system disk as MBR.  There is an install option in the ESXi installation routine to force the system disk to be formatted with MBR instead of GPT, enabling ESXi to boot correctly.  Here’s how to enable ESXi to format the installation disk with MBR:

1.) Boot server from ESXi installation media.

2.) Wait for prompt in lower left-hand corner that says <SHIFT+O: Edit boot options> .  Press SHIFT+O (that’s the letter O, not a zero).

SHIFT+O Edit Boot Options VMware ESXi

3.) In the lower-left hand corner, after the word ‘runweasel’ insert a space then type formatwithmbr.  Press enter to continue the installation process.

runweasel formatwithmbr

ESXi should now install as normal.  After the install, you should find that your server (workstation) boots into ESXi.  If it doesn’t, you may have to play with BIOS options a bit more.  I’ve heard of AHCI mode causing issues with boot from USB, as well as incorrect boot order, and simply enabling the boot from USB option in BIOS.  Aso, check to see if your server/workstation can support an EFI image instead of traditional BIOS.

If you are interested in some other ESXi boot options, see here: https://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-installation-setup-guide.pdf.

When using the formatwithmbr option, the VMFS paritition created on the boot disk will be formatted as MBR.  Any other VMFS partitions that are created on other disks in your system will be formatted with GPT per the vSphere Storage section in the official vSphere 5 documentation set.