Life Cycle of Storage Challenges

Cloud and storage don't go as well together as the vendors would have you believe. Cloud-integrated enterprise storage is an up and coming technology that will likely change this.

In this article, we'll discuss a new class of product that is only just coming center stage but looks poised to seriously alter how storage is done. This type of product is called cloud-integrated enterprise storage, or CES. CES is a new blend of cloud and primary storage that will create radical simplification in the practice of storage and drastically alter the on going costs of storing every piece of data a business ever creates. It almost sounds too good to be true, but after several years of watching this technology evolve and gain real world use, we have little doubt that cloud-integrated enterprise Storage is a force to be reckoned with. It is here, and it is here to stay.

The State of Cloud Storage

Since Amazon first introduced S3 storage buckets, innovators have been looking at how to store more common types of data in the cloud. Storage buckets –- which contain storage objects accessed over the wire via a highly scalable and location agnostic HTTP protocol –- promise unique capabilities in three areas: 1) cost, 2) access and 3) scalability. In pure form, storage bucket technology is far from compatible with business storage. It requires a foreign storage protocol and great tolerance of latency and variable performance, and this doesn't work for mainstream business applications.

Storage buckets have created opportunity among these challenges, and innovation has ensued. The earliest cloudy storage innovators looked to harness these storage buckets and sought obvious IT challenges that were easily matched to the cloud's unique capabilities. On the surface this appears an easy task, but it has required serious innovation around connecting the traditional infrastructure to the cloud and making use of the cloud to enhance data and application access. This has led to a number of first-generation cloud based storage offerings, including special purpose applications, collaboration solutions, backup solutions, data archiving and even cloud-attached storage appliance that use temporary local storage to intermediate the latency and foreign storage protocols of the cloud to enable traditional storage access.

Cloud-Integrated Enterprise Storage

What has so far eluded the focus of most of these innovators is an approach that pairs the cloud with how a business stores the primary data behind the core applications within the data center. Such data poses a challenge, as it is the crown jewel of the business, and is surrounded with even more demanding requirements that are infinitely harder to address with incompatible cloud characteristics.

Cloud-integrated enterprise storage, or CES, is a new category of technology that will tackle the misalignment between cloud access characteristics and the demands of enterprise storage by delivering enterprise-class primary storage first. Then, deeply integrated cloud technology will make this storage uniquely capable, and it will reach far beyond simply adding capacity. Instead, the cloud will fundamentally enhance each and every storage feature, and extend storage capabilities and storage depth so that data can be stored once and for all.

While on the surface such storage may look like a simple attachment to the cloud (see Figure 1), don’t be fooled, it goes much further. CES solutions will go further with innovative technology that optimizes the placement of every stored data byte, and then use the offsite, access-from-anywhere, highly protected and near limitless scalability characteristics of the cloud to change transform the permanence and versatility of primary storage features.

Figure 1

(Click for larger image)

In contrast to the earlier cloud-attached appliances and software, the onsite storage system will be fully authoritative for every stored byte of data, rather than acting purely as a cache or transmission device. That storage system will optimally place every byte of data and continuously optimize placement. This will keep highly active data on high-speed storage inside the enterprise walls (see Figure 2). But this far ranging auto-placement will create far more than just limitless capacity. Why? Because primary storage is the birthplace of nearly ever storage challenge, and well-integrated cloud storage could eradicate many of these challenges.

Figure 2

(Click for larger image)

Continue to Page 2

Primary Storage -- The Origin of a Life Cycle of Storage Challenges

The most complex and costly storage challenges are entirely related to storing and managing data over a life cycle that must be longer than reasonable on rapidly evolving digital systems. CES will have resounding and lasting impact on the practice of storage because that life cycle begins when data is created on primary storage. The integration of cloud and primary storage systems for the first time enables primary storage to deliver features and data management well beyond physically limited devices, systems or data centers. Such storage can eradicate or at least massively reduce the need to move and manipulate stored data.

The lifecycle of data and information management within the business is tremendously long, complex and costly because data must be moved and manipulated. Such a life cycle, which starts with primary storage systems, can involve many different specialized, siloed storage systems and technologies, and create entire stand-alone domains of IT practice within the business (e.g., backup, archive and disaster recovery). Within each of these practices and storage silos, the cloud has so far delivered innovative possibilities for reducing complexity and cost of ownership. But each of these practices, even with new technologies, remains an isolated silo and each carries enormous cost. This makes the life cycle of data management tremendously expensive (see Figure 3).

Figure 3

(Click for larger image)

Pairing the cloud with this primary source of data creation presents the opportunity to touch nearly every piece of business data at the very point in time when it is created, and this can put an end to these separate practices. The consequent cost savings will drive cloud integration to the forestage of the primary storage market. What complex data problems and practices could such a solution take on? A few examples include:

• Retention: Data could be captured for archive and compliance as soon as it is created, and then transferred into the cloud when the time is right, without requiring separate applications, storage appliances, and IT processes. Meanwhile, all data was kept visible and accessible in one consolidated point of access.
• Protection: Copies of data for protection -– snapshots and clones -– could be migrated into the cloud over time, and transform high speed primary protection. By leveraging the ability to automatically place snapshots on-premise, in the cloud or both, a CES array could deliver long-term, low cost, globally dispersed, and on-premise accessible backup that does away with the need for traditional backup and offsite tape. Moreover, based on the solutions available today, such a solution could perform at vastly faster speeds than traditional onsite and offsite backup, irrespective of whether data is recovered from on-premise copies or off-premise copies.
• Disaster: Meanwhile, primary data could still be synchronized with the cloud periodically to create offsite replicas that could, if done right, forever replace complex disaster recovery technologies, dispense with tape and physical transport, and provide a master repository from which recovery data could be accessed, from anywhere. This could unlock highly virtualized, hosted DR on-demand, and do away with wasted standby conventional infrastructure.

These are not small challenges to tackle, but the potential improvements we've highlighted in these examples can create a massive transformation in the practice and costs of data storage and management and reduce a surfeit of technologies to one single data storage and management array.

Core Requirements -- Cloud-Integrated Enterprise Storage

While there is vast room for considerable innovation and differentiation, we can boil the fundamental requirements for delivering CES down to a few capabilities. While each may seem simple or common place on its own, it is the synergistic integration of these capabilities that will create an enormous breakthrough in how we store data and deliver a comprehensive set of features such as those in Figure 4.

Figure 4

(Click for larger image)

Built for Primary Storage.

CES must be engineered for primary storage performance and availability. As usual, primary storage must include the best clustered, active-active controllers designed for non-stop operation, enterprise components, and redundancy throughout. Moreover, today performance is a hotbed of innovation, and customers are steadily coming to expect uniquely integrated and always optimized high IO media (solid state) alongside exceptional approaches to caching that can constantly deliver maximized performance but simultaneously auto-tier data to rotational disk to minimize the cost of capacity.

Built with Primary Storage Features.

Just as importantly, CES requires state of the art primary storage features, that are entirely array-integrated. Such features include internal array virtualization that reduces complexity and harnesses all disks into a single pool of easy to use storage, thin-provisioning, a full set of snapshot and cloning technology, flexible replication, and enterprise caliber management and visibility. There can be absolutely no compromise in the performance and use of these features. First-generation cloud-attached storage devices were too often dependent upon a cloud connection for the use of their features; such systems are not ready to delivery primary storage that must be always-on.

Innovative beyond the Array.

Finally, delivering CES is more than simply turning to the cloud as a parking lot for old data. The cloud requires unique integration, and vendors that fail to innovate may serve up additional capacity, but they will fall far short of enhancing and extending an array to tackle complex lifecycle problems. Innovative integration will reach beyond capacity expansion and provide effortless, always-on backup, seamless offsite data protection, instant DR from anywhere, cloud provider lock-in elimination, and total performance and capacity optimization irrespective of where data is located. The combination of those features will enable a single physical device to reach far beyond physical boundaries and handle all data storage lifecycle needs.

CES will be the most impactful storage technology to emerge from the cloud. It stands to eliminate the most complex storage tasks in the data center, without any more infrastructure alteration than the replacement or addition of a primary storage array. Such potential can yield a massive transformation in the total cost of ownership for storage, and it will foment rapid innovation in this market. But a vendor's attention to integrating both on-premise and off-premise data, and using the combination to address a lifecycle of data management pain points, will differentiate vendors over the long run.

Representative Vendors

Only a few vendors have broken through in delivering what we consider true CES, although there are several who have begun steadfastly moving in the direction of CES. Among major vendors, HP's 3PAR deserves credit for taking first steps here. Some time ago the 3PAR team began crafting innovative cloud partnerships under the banner of their CloudAgile program. Hitachi has more recently moved toward CES by linking their BlueArc (NAS) and Panasas (scale-out file storage) acquisitions to offsite, Hitachi Content Platform storage. But for major vendors, much work and opportunity remains. In each case these traditional legacy vendors have taken the first step by integrating their primary storage with their own "private cloud." We expect the next step will be to integrate with the pubic cloud. But political considerations remain.

Among new innovators, there are many cloud-attached appliances serving up various incarnations of product similar to CES. Vendors include Avere, CTERA, Nasuni, Panzura, Riverbed, TwinStrata, Nasuni, StorSimple, TwinStrata and others. At their forefront today, is clearly StorSimple, and they are the only one yet delivering true CES for core business data. Many vendors still fall short in xyz dimensions (HA, clustering, snapshots that can function locally as well as deliver protection in the cloud, etc. etc.). Many of these vendors are trying to move up the "enterprise primary storage" flagpole, and we expect each to either continue reaching for more primary capabilities as they innovate around cloud attachment, or else rapidly choose a more selective path that targets single storage challenges.

Such solutions will no doubt continue to deliver significant value, but if selecting a vendor on the promise of transforming the entire lifecycle of data storage, pay close attention to their current product and the practicality of their promised roadmap. Cloud-integrated Enterprise Storage stands to be transformative across all data storage practices, and will alter the cost equation of enterprise storage. The question is when, not if, it will someday cross the threshold of your business place.