January 2, 2013

It’s been a long time since I published an article in my Storage Basics series – the series has been some of the top content on my site to date. I sat for a beta version of the VMware Certified Advanced Professional – Desktop Design (VCAP-DTD) test today and was reminded by the test of a post I had in waiting about an alternate way to calculate IOPS (hint hint – memorize these formulas). I have several more articles sitting in draft form, but a new job, crazy kids, home improvement projects, and a wife with chronic cerebral spinal fluid (CSF) leaks (it sounds bad – and it is – but look on the bright side: not many guys can get away with saying that their wife’s brain leaks!) all take time. But enough excuses and back to storage….

I wrote in Storage Basics – Part II: IOPS on how to calculate theoretical IOPS for a given disk type and speed, and then followed it up in Storage Basics – Part III: RAID with some information on how different RAID configurations impact IOPS. I.E. More disks equals more IOPS, but with a write penalty assigned for different RAID types. I then moved into a discussion of interconnects in Storage Basics – Part IV: Interface, but didn’t do a good job there of tying the interconnect speed to performance and IOPS. Finally, in Storage Basics – Part VI: Storage Workload Characterization, I hinted briefly on how I/O request size can impact storage performance. Let’s dig into this topic a little deeper!

The Original IOPS Forumula

Calculating IOPS at the disk/RAID level is fun and all (and I suggest you brush up on it if you are sitting for the VCAP exams), but does not paint a full picture of the storage subsystem. The basic formula to get a single disk IOPS (I covered this in Part II) looks like this:

$\frac{\text{Seek Latency} + \text{Rotational Latency}}{1000} = IOPS$

(The LaTeX formulas keep getting messed up – sorry if they don’t look right. Let me know and I’ll fix them, again).
For simplicity sake, let’s just assume that this formula yields 180 IOPS for a standard 15k RPM SAS disk (the industry standard number of IOPS for this speed and type of disk)

The formula for RAID IOS is in Part III – here’s a sample for RAID5, where:

p = IOPS required, f = write IO penalty factor, r = % Read, w = % Write

${(p \times r)} + {((p \times w) \times f)} = \text{Back-end Spindle IOPS}$

Let’s put it to use: let’s say I have a workload that I’ve measured as needing 4000 IOPS from the OS/application perspective, of which 70% are write (not unusual in VMware View Linked Clones): here’s how you figure out how many disks you need from an IOPS perspective for those linked clones:

${(4000 \times 30\%)} + {((4000 \times 70\%) \times 4)} = 12,400\ast$

*12,400 Back-end IOPS needed on my spindles in a RAID5 array

Then, convert the number of back-end IOPS needed to a number of disks needed given our RAID5 assuming the 15k SAS disk:

$\frac{12,400}{180} = 69$

That is, I need 69 disks (rounded up) required on the back-end to meet our 4000 front-end IOPS, ignoreing things like best practices when putting a bunch of disks in a RAID group/storage pool (4+1 RAID5 sets) and hotspares (1 in 30 for SAS)

A quick run at the math suggests that you should probably not be using RAID5 for a write-heavy workload (run the math yourself with a write penalty of 2 for RAID0+1 to see how many fewer disks you would need). Also, this particular example leaves out any calculation for cache impact, including EMC FAST Cache or the View Storage Accelerator (CBRC). If you have those technologies in the mix, then a very rough rule of thumb would be to size your back-end IOPS based on your front-end IOPS calculation (4000 in the example above, instead of 12,400) – I say very rough because this really depends on your cache size and cache hit ratio, and the rule pertains a bit more to read-heavy workloads than write heavy. If you have CBRC in the mix, you *might* be able to greatly reduce and maybe eliminate the Read IOPS from the equation. But before you go taking my word as gospel, test, test, and test some more using your particular workloads.

The Alternative IOPS Formula

But this is all back-end spindle stuff and me having fun with $\text{LaTeX}$ (LaTeX is the language I used to make the cool math equations – if the forumlas don’t look right drop a comment so I can fix them. WordPress keeps eating the latex syntax…). Hopefully your SAN admin already knows this, so go ahead and give her your front-end IOPS count and read:write ratio and you’ll get some LUN’s presented. But you, the VMware admin, might be responsible for server hardware, including buying the interconnect card (HBA, iSCSI NIC). How do IOPS inform your selection? This is where the alternate formula for calculating IOPS comes in. Here’s the forumla: