One to Watch: Kinetic Ethernet-Based Storage Platform

Thursday Sep 24th 2015 by Paul Rubens
Share:

The Kinetic Open Storage Project has gained heavyweight vendor support and promises to disrupt the storage landscape.

Storage controllers, SANs and even file systems are all under threat from Kinetic, an extraordinarily disruptive storage architecture.

The Kinetic storage platform, which is gaining supporters rapidly, uses Ethernet-enabled, key-value store disk drives, providing direct connectivity between applications and object storage. The underlying technology was developed by California-based disk maker Seagate, but the company has now turned its efforts into an open source collaborative project called the Kinetic Open Storage Project (KOSP), run under the auspices of the Linux Foundation.

So far thirteen storage-related companies have joined the project: drive makers Toshiba and Western Digital, as well as storage vendors Scality, NetApp, Red Hat, Cleversafe, Huawei, Dell, Open vStorage, SwiftStack, Digital Sense and Cisco.

Given that Seagate has been developing its Kinetic technology for some time, an obvious question to ask is why it has decided to open source the platform and allow competitors in the hard drive space — like Toshiba and Western Digital — to get involved.

"The reality of our business is that nothing happens with a single company," explains David Burks, Seagate's director of product management for the Kinetic platform. "If Kinetic drives are unique to us and no one could duplicate them then the platform would fail," he says. "Instead of the platform we say that our IP is inside the drive, making it perform well and be reliable."

KOSP will develop an open source standard including the interface and drive command set for accessing and using Kinetic drives. To start the project off Seagate has contributed its existing Kinetic APIs, libraries, tools and utilities such as a simulator.

The idea behind Kinetic is that applications talk to storage drives over Ethernet, and this communication can be routed over switches. That means compute and storage are decoupled. "When you do that, you don't need storage management software and storage servers, or a file system," says Burks. "The benefit is you eliminate the storage servers that break data down into blocks and write it, which means less cost and less power consumption."

There are a couple of challenges to be overcome when implementing a Kinetic object storage platform, though. One is that Kinetic drives — with their Ethernet connectivity — are more expensive than normal drives. "We have had to put additional bridge chips on the drives, which increases the costs by some amount," Burks concedes. "But longer term we anticipate integrating these into the drives' chipset so that they are no more expensive."

Another challenge is that Kinetic object storage still needs to be managed by a storage management application, so existing ones need to be made "Kinetic-aware." That's a job that Burks estimates requires about two man-months of programming effort.

The good news for the project is that some software is already Kinetic-aware: OpenStack's Swift object storage software works with Kinetic today. And SwiftStack's Controller object storage software (which is based on OpenStack Swift) should be ready around October or November of this year, along with Scality's Ring object storage software.

Early next year it should be possible to build a Kinetic system based on OpenvStorage's open-source VM storage router (which is similar in concept to VMware's Virtual SAN product but can use many different types of storage rather than just locally attached disks.) Exablox is also working on its scale-out NAS software to bring support for Kinetic soon, Burks says.

Kinetic will initially be targeted at hyperscale operations like Google, Facebook and Twitter, which use object storage because it is scalable and affordable, Burks says. Later the focus will move to enterprise storage markets too.

"The key-value stores in Kinetic hard drives are a good fit for… distributed file systems (Hadoop Distributed File System, Lustre, GlusterFS) and distributed database (Cassandra) use cases, but will not be a complete replacement for existing enterprise NAS and SAN infrastructures," Henry Baltazar, a senior analyst at Forrester Research, believes.

To build a working Kinetic system you'll also need a chassis which can hold Kinetic drives and provide an Ethernet interface. Today you can buy a chassis from Supermicro or Sanmina to fill with 4TB Seagate Kinetic drives with dual 1Gbps Ethernet interfaces. In the near future Seagate will offer its own 5U, 84 bay chassis to host new 8TB Kinetic drives (for a total of 672TB/chassis). The new drives will have dual 1 or 2.5 Gbps Ethernet connectivity.

Burks expects to see the development of Kinetic solid state drives (SSDs.) Initially these will probably be used to hold object storage metadata to increase performance by making it possible to find object "payloads" faster. Eventually Burks expects a higher-performance tier of object storage to emerge that is fulfilled by a tier of Kinetic SSDs.

Toshiba has already announced Kinetic-compatible drives. These include an 8TB shingled magnetic recording (SMR) Ethernet drive and a hybrid Ethernet drive with spinning and solid state storage. This drive runs its own Linux kernel, which can be used to cache metadata on the solid state storage of the drive. HGST has also announced Ethernet drives.

Will the Kinetic platform take off and become widely deployed? It's impossible to say for sure, but the establishment of a Linux Foundation open source collaborative project certainly makes it more likely, according to Mark Peters, a senior analyst at Enterprise Strategy Group. "The formation of the Kinetic Open Source Project is really quite significant," he says. "Seagate could try and go it alone with Kinetic but then it would be difficult for companies to adopt. They don't want a platform to be too vendor specific: they want standards and not to be beholden to a single vendor."

One of the reasons that the push towards this type of storage architecture is coming from disk vendors rather than more general storage systems vendors is that it provides a way for hard disk drives to become less commoditized items which are supplied at very low cost.

"Everyone wants to grab margin, and these Ethernet drives will cost more," says Peters. "Some of this margin will go to the vendor, so this can be seen as an attempt (by the disk vendors) to increase their margins. Western Digital/HGST, Seagate and Toshiba are all beginning to wrap more functionality into their offerings, so what we are seeing is the disk vendors staking a claim in more of the storage stack, and becoming less of a commodity."

More generally, Peters believes the Kinetic storage architecture may succeed because companies are interested in software-defined storage — however you choose to define it. He says around 90 percent of organizations are either involved in or interested in software-defined storage today.

"Kinetic allows you to avoid the controller layer by moving software somewhere else — which is what people are interested in," he says.

"The most obvious way is to sell software which runs with heterogeneous hardware (like DataCore, Hedvig and IBM.) But Kinetic is a parallel effort. It doesn't provide storage functionality, but it makes software-defined storage easier and cheaper.

"And those," he concludes, "are the key reasons that people are moving to software-defined storage. Ease and cheapness."

Photo courtesy of Shutterstock.

Share:
Home
Mobile Site | Full Site
Copyright 2017 © QuinStreet Inc. All Rights Reserved