Does Data Storage Really Change?

A look back at data storage in 2014 reveals many new developments, but one surprising holdover from earlier years.

 2014's storage highlights: The resurgence of tape, VMware Virtual SAN, marketers making purchasing decisions, 10TB hard drives  and much more: 2014 was a surprisingly exciting year for storage…

In 2014, we saw some well-established themes continuing: Tumbling cloud storage prices, ballooning enterprise storage requirements, talk of array sales peaking. At first glance you could be forgiven for thinking that the storage world in 2014 was much like 2013. And 2012. And 2011.

But there were two key storage themes that defined 2014: flash memory, and responsibility for storage. Both need a bit of explanation.

If you've been following the storage industry for the last twelve months then you can't have failed to detect an uptick in interest for old school tape-based storage. Some people have been saying that "tape is back," although the truth is that it never really went away - it just went out of fashion.

But there's no doubt that the storage medium is being looked at in a new light, and one reason for that is the need for organizations to look more closely at long term, "deep" or "cold" storage - and the success of products like Spectra Logic's Black Pearl Deep Storage Appliance.

The need for cold storage can partly be explained by the rise of Big Data projects and by regulatory requirements forcing enterprises to keep data for long periods, but there's more to it than that, according to Mark Peters, a senior analyst at Enterprise Strategy Group. "The reason that there have been signs of a rise in tape comes down to flash storage and data management capabilities," he says.

This statement certainly requires elucidating. So here goes.

If you put flash storage to one side, the only option for work storage is disk storage, argues Peters. "What the availability of flash has done is make people realize that there are different types of storage and data," he says. "Before, all data was treated as warmish, or lukewarm, or not too cold. But now they realize that flash is for hot data."

Flash storage is more expensive than disk storage, but it's worth using with hot data because of the performance benefits it offers. This hammers home the concept that storage is about more than cost per gigabyte, and more about storing data on the appropriate medium for its temperature.

"So people are beginning to think: 'if we are putting hot data on flash, why are we storing cold storage data on lukewarm disks. Why not try something else?'" says Peters.

That something else is tape. But once again, there is more to it than that. The extra ingredient that's required, and which we have also seen coming to the fore in 2014, is data management software:  software that can move data around (to appropriate storage mediums) automatically.

"So we have the realization that cold data needs to go on tape, and simultaneously we have the availability of sophisticated software to move the data to where we want it," Peters points out. "So tape for cold storage is interesting in 2014, but it's because of flash storage and data management software that it has become interesting."

As well as raising interest in tape, another side effect of flash storage is that it is probably also stimulating interest in high capacity low energy disk drives such as HGST's helium-filled Ultrastar He10 10TB drive, announced in September. "Some people simply aren't prepared to move to tape, so from an economic perspective high capacity, power efficient disks that can be crammed into a small space are very attractive," says Peters.

Responsibility for Storage

So that's the flash memory theme, but what 's the "responsibility for storage" theme about? The answer, essentially, is that throughout 2014, storage responsibility has been drifting away from a dedicated data center storage team on the IT staff roster. Buying storage is increasingly becoming part of the remit of server and virtualization administrators, application owners, and even non-IT staff.  

"What we have been seeing is the central storage team no longer funding, deploying and managing storage in many instances," says Dave Russell, an analyst and vice president at Gartner. Instead server teams and virtualization teams have been making purchasing decisions and  buying integrated systems and server SANS with NAND flash PCIe cards and hard disk drives.

Another group that have been making storage purchasing decisions is application owners (like Oracle database or Hadoop administrators,) he adds. They have been standing up their own storage systems (like a Hadoop cluster) using readily available resources outside the data center. "Often this has involved a return to direct attached storage (DAS), in contrast to the general trend towards shared storage."

In fact other non-IT business units, such as marketing, also carry out IT-related spending – as much as 19% of total IT spend according to a Gartner CIO survey, at least some of which is storage related. And Gartner believes the true figure is 35% or more of total IT spend being carried out by non-IT staff purchasing their own solutions or pursuing SaaS and the public cloud.

"Back in the good old days storage staff made storage decisions, but the genie is well and truly out of the bottle now," Russell says.

A result of this change in responsibility has been a change in the type of storage that was being purchased in 2014. One reason that there has been talk of "peak array" having been passed is that non-storage people are more likely to choose new, alternative storage options.

"Five years ago non-IT staff storage choices may have been similar (to storage staff's choices): SANs, Fibre Channel and so on – whatever was most common and reasonable. But now a virtualization or server team might say "that seems pretty expensive," says Russell.

"They don't have a degree in storage, and new entrants like Tintri – with storage that is optimized for virtualization – or Nutanix – with a virtualization platform  – are tempting because they sound like they are optimized for virtualization," he adds.

It's the threat from this type of storage solution that led to EMC – arguably the company with the most to lose when it comes to traditional array products – announcing initiatives like Project Liberty in mid-2014. This, you may recall, provides the features and functionality of EMC's VNX mid-range storage arrays in software – so a VNX minus the physical storage devices.

It also updated its ViPR software array meta-platform (or perhaps even software defined storage platform) and productized Project Nile into Elastic Cloud Storage.

"EMC wants to look like it is a company that is embracing the future to look progressive," says ESG's Mark Peters. "All this gives it breathing room – it is straddling both sides of the fence, but there is a difference between having products to market and products to sell. EMC still sells more VCE (Vblocks)  and VMAXs  in a day than they sell these new products in a month."

Storage Products

That describes the storage trends of 2014, but what about products? What was the most significant product to appear in the last twelve months? For Gartner's Russell it is (indirectly) another EMC product: VMware's Virtual SAN (VSan). The product made a huge splash when it was finally launched in March after six months' beta testing.

Although only a version 1 release, and functionally incomplete, Russell reasons that it still does what it does well and resonates with mid-market companies and remote offices. It seems that server and virtualization teams are embracing it, and there have already been some deployments at a significant scale, he says.

But he gives two main reasons for choosing it as the most significant product of 2014:

1. As the leading hypervisor vendor turned IT management platform provider, VMware has the ability to provide storage capabilities to a new audience. "They (VMware) can really move the storage industry," he says.

2. VMware has rapidly developed a broad ecosystem of many third-party providers making Ready Nodes for accompanying hardware, and also software add-ons ( from companies including Atlantis Computing and Nexenta) that complement the solution.

And what about ESG's Peters? Does he agree with Russell's choice of VMWare's VSan?  He believes the product is significant – because it represents part of a likely future trend towards buying IT as "blobs that do things"– but Peters' most significant product for 2014 is something more general: flash.

Why flash? "You couldn't do VSan without flash in the server. And tape has a future because of flash. So there's no doubt that flash has caused the biggest storage changes in 2014," Peters concludes.

Photo courtesy of Shutterstock.