Why is EMC focusing so much on software when its roots lie in selling hardware? (Hint: think ‘software defined storage’.)
EMC is the world's largest storage systems company, but despite that the organization is also surprisingly agile. So if you want to understand which way the storage industry is heading, it helps to study EMC.
Now happens to be a particularly good time to put EMC under the microscope. That's because in early May, at its annual EMC World conference in Las Vegas, the company opened the kimono on what products it's been working away on these last few months, and what it's planning for the near future.
The company revealed all kinds of things at its storage fest, but two things in particular stand out and shed the most light on the storage industry's direction. So it's those two that this article will be concentrating on.
The first is the update to EMC's ViPR storage software product that was first released back in September of 2013. The second is what EMC is calling Project Liberty, which provides the features and functionality of EMC's VNX mid-range storage arrays in software – minus the physical storage devices, of course.
So, firstly, ViPR - or ViPR 2.0 as it is in the current iteration. It's not the same as Project Liberty's concept of a software controller split from commodity storage devices. Rather it's software that can be used as a kind of meta-platform on a level above physical arrays, unifying them and integrating them so that each physical array is no longer a separate data silo. EMC calls it a software defined storage platform.
In its first release it was very far from being feature complete - only supporting object data services (with access via Amazon S3 and Hadoop Distributed File System (HDFS) APIs,) having limited support for other vendors' arrays, and only really being suited to a single data center.
With the new release, it adds several features including:
· Additional array support. In addition to commodity hardware (HP SL4540 initially,) ViPR 2.0 now supports Isilon, ScaleIO, VBLOCK, VMAX, and VNX EMC storage, plus third-party storage including Hitachi Data Systems (HDS) and NetApp, and provides access to HP, IBM, and Dell systems via the OpenStack Cinder plug-in.
· Geo-distribution and protection. ViPR 2.0 enables customers to manage a geographically distributed environment of multiple data centers as a single namespace and use metadata-driven policies to distribute and protect content against data center disasters with geo-distribution and replication capabilities.
· Integration with EMC Centera archiving. ViPR 2.0 brings support for the Centera API as ViPR Content-addressable storage (CAS) - full support for its Centera arrays will be introduced in the future.
· Block storage data service provided by ScaleIO, the Israeli startup the company acquired last year.
Those are the facts about ViPR 2.0, but what does this tell us about EMC's vision of storage for the future?
When ViPR was launched in September it was clear that it was significant, and now EMC is fleshing it out further into what is doubtless going to be a key product for the company. But why is it going down this software route with such determination when its roots lie in selling EMC hardware?
The reason, according to Mark Peters, senior analyst at Enterprise Strategy Group, is that the company recognizes that software is changing the storage industry fundamentally, and if you can't beat it then you'd better join it.
"EMC can see the writing on the wall, they see that change is inherent, logical and possible," he says. "So they are getting ready for the change by building up one side of the business while the other side declines – or at least suffers margin attack."
For a company that has specialized in making its own proprietary boxes, one of the more perplexing things is EMC's increasing willingness to support third party hardware – including commodity hardware. It has embraced HDS and NetApp directly, and other vendors' systems indirectly via OpenStack as well.
So to an extent it is supporting competition for its own products by supporting other vendors' systems and commodity disks. What kind of a business model is that?
Peters points out that there have been other platforms that control third party storage systems, from the likes of IBM, Hitachi and even NetApp. "But the truth is that although they offer heterogeneous platforms, they are actually Trojan horses," he says
What's actually going on is that EMC will hope to gain account control. It offers its customers the flexibility to use what arrays they want – at first – while maintaining a presence and income stream from those customers. Effectively it is continuing to "lock in" customers. The difference is that now it is locking them in to its ViPR software, whereas before it was locking them in to its hardware.
The transition from hardware storage to software defined storage is likely to be a long one – in part because many organizations have made significant investments in hardware storage systems. But it's also because while EMC is getting its fingers firmly in the software pie, it doesn't want to destroy its existing and profitable storage business unnecessarily.
For that reason, don't expect too many features to be added to ViPR too quickly going forward. "EMC understands where its money is coming from today, and don't forget that it is in the business of making money," says Peters. "If I sell VMAX, I don't want to put all that functionality in ViPR. So while EMC is adding functionality like block support, it doesn't want to add too much."
So what of Project Liberty, and the idea of offering a software -only VNX? The "project" – and that means that a product may or may not come out of it – is developing virtualized storage software based on the VNX family of arrays. This software will be deployable on a virtual server or perhaps in a hybrid cloud.
EMC suggests that one use case will be to enable IT departments to spin up multiple virtual array instances quickly in test and development environments, and then migrate them to physical VNXs later for superior performance.
The first thing to say about this is that software defined storage uses a software storage controller that is separate from the storage hardware array, whereas a storage system uses storage controller hardware running microcode attached to the storage array.
But what happens when hardware storage controllers are really just made up of controller software running on a commodity PC? "Vendors still call them controllers to keep their margins high," says Peters. In other words, many storage systems are really nothing more than disks and software defined storage controllers. "So Project Liberty is just EMC acknowledging this," Peters adds.
It's not clear why a virtual VNX would be confined to a test and dev arena though, unless it is performance-limited in some way. After all, why would a virtual VNX not be able to provide the performance necessary to support production environments?
EMC, like most of the storage industry, is in a state of flux and is just embarking on the transition from "traditional" integrated storage systems to software-defined storage. But this may prove to be just a waypoint in the journey that storage is taking.
That's because the storage industry's entire raison d'etre was prefaced on the notion that servers themselves didn’t have the raw horsepower to do storage duties as well as computing ones. But now that processing is so cheap and networks are so powerful, it's not even clear that storage systems have a long term future.
"The traditional SAN certainly has a finite future," says Peters. "How long will it be before all storage is stored centrally - perhaps even in the cloud? I think that we will start to see that in under a decade," he concludes.
Photo courtesy of Shutterstock.