Now that vSphere v4.1 is out Dell EqualLogic have finally released their much awaited Multipath Extension Module (MEM) for their PS series SANs. With SAN performance being key to the performance of a virtualisation infrastructure I was keen to find out just how much more performance could be squeezed out of a Dell EqualLogic SAN.
All my deployments typically use what I shall refer to from here on in as ‘Classic MPIO’ - that is, multiple VMkernel ports on a vSwitch, each VMkernel port tied to a specific physical uplink, VMkernel ports associated with the software iSCSI initiator and using VMware’s round-robin setting on each storage path.
So before I hand over the results of my testing, I guess I should cover what I’m using for the test setup hardware & configuration:
- Dell EqualLogic PS5000XV SAN
- 16x 146.8GB 15Krpm SAS Hard Disks
- Firmware v5.0.1
- RAID 50
- Dell PowerEdge M610 Blade
- 2x Xeon X5570 CPUs
- 24GB RAM
- 2x 1Gb Mezzanine NICs dedicated to iSCSI
- Ethernet pass-through modules
- Nortel 5510-48T Switch Stack
- 2 switches in the stack
- SAN & server connectivity distributed across switches
- Flow control enabled
- Jumbo frames enabled
- VMware vSphere v4.1
- ESXi build 260247
- EqualLogic Multipathing Extension Module for VMware vSphere v1.0.0
I’m not going to go through the setup of MEM here as it already been covered in Jon’s excellent write-up at Virtualization Buster and I see no reason to reinvent the wheel! I will recommend the Remote CLI installation he does though as I couldn’t get the package to deploy to my hosts using Update Manager - the package installs fine, but it fails to show as an applicable update no matter how many times you scan the hosts.
To perform the testing I’ve used Iometer v2006.07.27 on an unpartitioned RDM accessed by a virtual machine sitting on the host with MEM configured. Iometer’s settings were configured as follows;
- Single worker
- Unpartitioned RDM selected as disk target
- Maximum Disk Size set to 8192 sectors
- 8192 sectors is to arbitrarily limit it, so nearly all I/Os will be “out of the cache” after the first set. The array has a cache memory that receives all in-coming Write data. If there is an immediate Read on the data, the array reads the data from the cache. The data is never written to the drives. Therefore, the array will turn I/Os around as quickly as they come in.
- # of Outstanding I/O’s configured as 64 per target
- This is inline with documented Dell EqualLogic recommendations for SAN performance testing
I performed a number of tests; first on the system using a ‘Classic MPIO’ configuration and then repeated the same tests after the host had been configured with MEM. All tests were run for 60 seconds and the results are averages over this time. I settled on a variety of test regimes to see what could be achieved in both sequential and random I/O environments with read, write and read/write loads, as defined below:
- 32K 100% Sequential Read
- 32K 100% Random Read
- 32K 75% Sequential Read (25% Sequential Write)
- 32K 75% Random Read (25% Random Write)
- 32K 100% Sequential Write
- 32K 100% Random Write
The test probably most applicable to virtualisation environments would be the 32K 75% Random Read (25% Random Write) test, as it’s the most taxing test on the list.
First up we have a look at IOPS:
It’s pretty clear from the IOPS results that MEM has made significant improvements to how hard you can drive your SAN - that’s a 30-40% performance improvement on almost all tests with the exception of the random write test!
I attribute the smaller increase in write performance purely down to the RAID 50 configuration of the test SAN - those of you with RAID 10 SANs should see even better performance. Peaks from the sequential read/write test averaged out at over 9300 IOPS! Absolutely stunning!
IOPS is all very well and good, but how many megabytes are we actually throwing around here? Allow me to answer that with another graph:
Total bandwidth on the sequential read/write test peaked at over 290MB/sec. I have never seen a single gigabit connection exceed 125MB/sec, so to get 40MB/sec more over two connections is quite simply an amazing feat.
The wow factor didn’t stop here. Next up came the latency results:
Not only has raw bandwidth been improved, it’s actually at a lower latency when saturated too! This is actually (in my opinion) more important than raw performance values. I’ve already waxed lyrical about this in my Dell EqualLogic SAN review, but suffice to say that any random I/O workload (such as a SQL database) will benefit significantly from this improvement.
Finally, I wanted to see how much extra CPU time a virtual machine would use when dealing with the increased workload capacity that MEM provides, so here’s a graph to illustrate my findings:
Yes, there is an increase, but it’s not much. It’s also much inline with the other graphs, with only the write tests differing. Again, I would put this down primarily to the RAID 50 write performance of the SAN - it' isn't going to be using much CPU if it’s just waiting for the SAN to write data!
In conclusion, Dell EqualLogic have pulled yet another ace from their sleeve with their Multipathing Extension Module for VMware vSphere - more raw performance at a lower latency for every customer hosting a VMware vSphere v4.1 infrastructure and at no cost! Bargain!
My only question is: Why aren’t you using it yet? :)
I hope you find this article helpful - now where did I put that ‘I love EqualLogic’ badge. ;)