Storage Nation

Last night, IDC released their storage market share analysis for what happened in Q2.  Some things remained the same, other parts seem to be changing. And if you watch this market as closely as I do, it's worth reviewing some of the more interesting bits. What You Need To Know For many years, IDC has published the most authoritative numbers on storage hardware and software sales.  While no methodology is perfect, I've come to view IDC's numbers as absolutely essential to anyone in the storage business. EMC doesn't give IDC access to any privileged information, they have to figure things out from published reports.  Generally speaking, IDC's estimates of EMC's business are usually within a few points (plus or minus) of our own.

As I sit back and contemplate the last few weeks, I keep coming to the same conclusion.  The market has clearly expressed its strong interest in all things cloud-like, and just about every technology vendor on the planet — large and small — is frantically repositioning themselves just as quickly as possible. Whatever "tipping point" we all were waiting for, it's clearly happened.  Game on. And I think this rapid shift has caught many in the IT industry by surprise: technology vendors, system integrators, consultants — few people are really interested in IT as usual.  Everyone wants to talk about the cloud.

There are three new white papers available on the site that may be of interest.  They are: Create a Smarter Storage Strategy http://thestoragearchitect.tradepub.com/free/w_fnet24 Availability and the Cloud http://thestoragearchitect.tradepub.com/free/w_fnet22/ The Economic Impact of File Virtualization: Reducing Costs and Improving Efficiency for File-Based Storage http://thestoragearchitect.tradepub.com/free/w_fnet09/ As usual I welcome any feedback as to whether this part of the site is useful. Disclaimer:  For each subscription I receive a payment which goes to fund the running of this site and www.thevirtualisationarchitect.com.  This includes covering the costs of trial subscriptions to cloud services.

So here , we showed how vSphere 4.1 SIOC and EMC FAST can work together to make DRS for storage possible today.

With VMworld in San Francisco this week, it's only fitting that Michael Haag is on the Hotspot discussing 3PAR's support for server virtualization . If you are at VMworld this year, please stop by our booth, #313, to see what all the commotion is about and why 3PAR storage is so popular for cloud computing .

The ongoing battle for 3Par by HP & Dell tells us much more about the state of the IT Industry than just the desires of two companies to acquire some interesting storage tech.  It signals an acceptance that storage is a key feature in the future direction of the IT industry – more important than networking and almost as important as the virtualisation platform itself. This may seem like a bold statement to make, however we need to look forward to where the industry is headed.  First of all, vendors want us to buy their unified hardware stacks; it represents that move back to a consolidated architecture that kept one vendor dominant in the mainframe days – IBM.  “No-one gets fired for buying IBM” the saying goes (or used to go), demonstrating how IBM was seen as the data centre supplier for all things computing in the 70’s and 80’s.  Of course we know that politics within organisations and the cost of IBM hardware eventually broke the monopoly, but the status quo worked well for many companies for many years.

In my series on data center transformation I started with Server transformation and the closer integration of server and storage virtualization through the use of VAAI or vStorage APIs for storage arrays. These APIs were introduced at VMworld in 2008 when VMware announced their vStorage initiatives. When VMware released these APIs on July 13, 2010, Hitachi jointly released support for these APIs on our AMS 2000 storage arrays . A lot of effort went into this integration as it is a massive technology enhancement for the transformation of the data center. The testing that we have done with Hitachi Dynamic Provisioning volumes on an AMS 2300 with VAAI has shown the following results: Full copy – 18% performance improvement (speed to copy VM’s) Write same – 85% performance improvement (speed to clone VM’s) Hardware Assisted Locking – 25% to 35% performance improvement including the removal of SCSI reserves (powering on 1400 VM’s on 4 x Servers simultaneously) See what VMware CTO Steve Herrod says about these enhancements in his executive blog.

This is a series of post discussing storage array architectures.  Previous posts: Choosing Between Monolithic and Modular Architectures – Part I In the first post, I discussed the shared storage model architectures typified by what we sometimes think of as Enterprise arrays, but I’ve called monolithic.  This term harks back to the mainframe days of large single computers (see Wikipedia definition ), hence it’s use to describe storage arrays with a large single cache.  In the last 10 years we have seen a move away from the single shared cache to a distributed cache architecture built from multiple storage engines or nodes, each with independent processing capability but sharing a fast network interconnect.  Probably the most well known implementations of this technology have come from 3Par (InServ), IBM (XIV) and EMC (VMAX).  Let’s have a look at these architectures in more detail. EMC VMAX The VMAX architecture consists of one to eight VMAX engines (storage nodes) connected together by what is described as the Virtual Matrix Architecture.  Each engine acts as a storage array in its own right, with front-end host port connectivity, back-end disk directors, cache (which presumably is mirrored internally) and processors.  The VMAX engines connect together using the Matrix Interface Board Enclosure (MIBE), which are duplicated for redundancy.  The virtual matrix enables inter-engine memory access, which is required to provide connectivity when the host access port isn’t on the same engine as the data.  There are two diagrams in the gallery at the end of this post, one showing the logical view of the interconnected engines and the second showing how back-end disk enclosures are dedicated to each engine. What’s not clear from the documentation is how the virtual matrix architecture operates, other than being based on the RapidIO.  I’m not sure if VMAX engines have direct access to the cache in other engines or whether the processor of connected engines is required.  In addition, can an engine access cache in another engine purely to manage throughput of the local host and disk connections? I’m not entirely sure.

The recent proposed acquisition of 3Par by Dell and/or HP has made me think a little more about the direction the storage industry is taking in terms of their storage array design architecture.  Since storage arrays became a category of devices in their own right, we’ve seen the growth of the monolithic, sometimes called Enterprise storage array.  Hu Yoshida discusses the subject on one of his recent blog posts .  Looking at the wide range of storage devices, I’ve categorised arrays into the following groups: Monolithic – this architecture is characterised by Hitachi USP, HP XP & EMC DMX and consists of a shared memory architecture and multiple redundant components. Multi-Node – these devices use loosely coupled storage “nodes” with a high-speed interconnect providing scalability by adding extra nodes to the storage “cluster”.  Products in this category include EMC VMAX and 3Par InServ. Closely Coupled Dual Controller – this is the typical “modular” storage architecture characterised by IBM DS8000, EMC CLARiiON, Hitachi AMS and HP EVA

As I have stated in my previous posts on data center transformation , virtualization plays an important role in creating a dynamic pool of server and storage resources and masking the physical infrastructure from the application, so that the provisioning, movement, and refresh of the infrastructure can be done without disruption to the business. While we are masking the physical resources, however, we still need to provide the applications with transparency into the virtual infrastructure to ensure that their service level objectives are being met