Storage Nation

So here , we showed how vSphere 4.1 SIOC and EMC FAST can work together to make DRS for storage possible today.

In my series on data center transformation I started with Server transformation and the closer integration of server and storage virtualization through the use of VAAI or vStorage APIs for storage arrays. These APIs were introduced at VMworld in 2008 when VMware announced their vStorage initiatives. When VMware released these APIs on July 13, 2010, Hitachi jointly released support for these APIs on our AMS 2000 storage arrays . A lot of effort went into this integration as it is a massive technology enhancement for the transformation of the data center. The testing that we have done with Hitachi Dynamic Provisioning volumes on an AMS 2300 with VAAI has shown the following results: Full copy – 18% performance improvement (speed to copy VM’s) Write same – 85% performance improvement (speed to clone VM’s) Hardware Assisted Locking – 25% to 35% performance improvement including the removal of SCSI reserves (powering on 1400 VM’s on 4 x Servers simultaneously) See what VMware CTO Steve Herrod says about these enhancements in his executive blog.

This is a series of post discussing storage array architectures.  Previous posts: Choosing Between Monolithic and Modular Architectures – Part I In the first post, I discussed the shared storage model architectures typified by what we sometimes think of as Enterprise arrays, but I’ve called monolithic.  This term harks back to the mainframe days of large single computers (see Wikipedia definition ), hence it’s use to describe storage arrays with a large single cache.  In the last 10 years we have seen a move away from the single shared cache to a distributed cache architecture built from multiple storage engines or nodes, each with independent processing capability but sharing a fast network interconnect.  Probably the most well known implementations of this technology have come from 3Par (InServ), IBM (XIV) and EMC (VMAX).  Let’s have a look at these architectures in more detail. EMC VMAX The VMAX architecture consists of one to eight VMAX engines (storage nodes) connected together by what is described as the Virtual Matrix Architecture.  Each engine acts as a storage array in its own right, with front-end host port connectivity, back-end disk directors, cache (which presumably is mirrored internally) and processors.  The VMAX engines connect together using the Matrix Interface Board Enclosure (MIBE), which are duplicated for redundancy.  The virtual matrix enables inter-engine memory access, which is required to provide connectivity when the host access port isn’t on the same engine as the data.  There are two diagrams in the gallery at the end of this post, one showing the logical view of the interconnected engines and the second showing how back-end disk enclosures are dedicated to each engine. What’s not clear from the documentation is how the virtual matrix architecture operates, other than being based on the RapidIO.  I’m not sure if VMAX engines have direct access to the cache in other engines or whether the processor of connected engines is required.  In addition, can an engine access cache in another engine purely to manage throughput of the local host and disk connections? I’m not entirely sure.

Thank you to all that participated in the Virtualgeek 2010 Survey (originally at this post).

Thanks to @ianhf for posting a related blog entry berating Netapp for their lackadaisical approach in delivering features in Data ONTAP that customers really want.  My original post is here ; @ianhf’s post is here .  I urge you to read it as it takes what I was saying a whole step further and was probably the post I would have written if I my use of the English language was more eloquent and structured. So, as part of their rebuttal, Netapp employees pointed me to two features; snapmirror migrate and Data Motion .  In the spirit of fairness, I have researched these two features using the only tools available to me – Netapp documentation – and here’s what I found

I’ve just been reading up on Data ONTAP 8.0 as part of some ongoing work I’m doing.  You may be aware that version 8.0 of the filer operating system brings two major features; 64-bit support and the (sort of) integration of the Spinnaker code to create the multi-node version of ONTAP (called cluster mode). The move to 64-bit is an essential feature required to enable Netapp filers to scale.  Currently both aggregates and Flexvols are limited to only 16TB.  Plenty of people have complained this limit is far too low and it is one of the most restrictive scaling issues within ONTAP.  By moving to 64-bit aggregates, scalability is increased dramatically (but perhaps not proportially as expected) to 100TB.  Unfortunately things are not all rosy.  For instance, although aggregates move up in size, Flexvols do not; they stay at a mere 16TB.  There are a number of other things to consider that mean Flexvols are still not as flexible as they should be.  For instance: There is no option to move Flexvols between aggregates without taking an outage.  Volumes can be copied (vol copy command) or cloned (vol clone command).  As volumes get larger, taking extended outages to simply move data is unacceptable.  Netapp needs to add the facility to transparently move the underlying location of a Flexvol without user impact.

So – do these ideas work together?  Do they complement each other or compete?   What’s the source of these questions Chad – and why are you asking them?  (BTW – this topic is one I’m covering at VMworld, along with some of the testing data) If you’re interested, read on… Storage IO Control (SIOC), along with Network IO Control (NIOC) represent VMware’s first forays into expanding the IDEA of DRS (which is in essence distributed resource prioritization) beyond CPU cycles and memory – and into storage and network IO subsystems.

The good news — there's general consensus from the enterprise IT organization I speak with that — yes, they'd like to pursue a private cloud model. The challenging part — as they look around their data centers, all they can see is the legacy of past decisions: incompatible architectures, stove-pipe stacks and the rest.  It's like an archeology project: you can see the layers of different waves that have come through the IT organization. How can you move forward when all you can see is history

This is the third part in my series on data center transformations. My last post was on server transformation and the impact of virtual servers on the data center. In this post I will address the impact of storage transformation on the data center. Data is at the core of the Data center Data is at the core of the data center, and any effort to transform the data center must involve the movement, provisioning, access, and protection of data which is provided by storage systems

We are regularly told that checking our own bodies for signs of change is a good thing.  Early diagnosis of disease gives more of a fighting chance of curing the problem.  So, in the IT world, where we assume all of our backups have been taken successfully, how often should we be checking the results and ensuring the backup will work on the fateful day we need to do a restore?  This question was posed by Federica Monsone on Twitter this week.  Here’s an attempt to provide an answer.