Inside Else Inside TEMPlate====>

Cloud Storage Concerns, Considerations and Trends

By Greg Schulz

If you're using or thinking about using cloud storage, these are the things you'll need to think about.

When it comes to cloud computing and, in particular, cloud storage, context matters. Conversations are necessary to address concerns, as well as discuss various considerations, options and alternatives. People frequently ask me questions about the best cloud storage to use, concerns about privacy, security, performance and cost.

Some of the most common cloud conversations topics involve context:

  • Public, private or hybrid cloud; turnkey subscription service or do it yourself (DIY)?
  • Storage, compute server, networking, applications or development tools?
  • Storage application such as file sync and share like Dropbox?
  • Storage resources such as table, queues, objects, file or block?
  • Storage for applications in the cloud, on-site or hybrid?

Do you have cloud storage concerns? If you can identify and list those concerns, you can also prioritize them, then explore options on how to address or work around them.

Some common cloud storage related concerns include the following:

  • Is cloud storage cheaper than traditional storage?
  • How do you access cloud object storage from legacy block and file applications?
  • How do you implement on-site cloud storage?
  • Is enterprise file sync and share (EFSS) safe and secure?
  • Does cloud storage need to be backed up and protected?
  • What geographic location requirements or regulations apply to you?

Cloud Concerns: Storage Options

There is a common industry myth (for some perhaps it is still true) that all cloud storage is object-based. A related myth that is true for some is that accessing local or cloud-based and object storage is difficult, particular for existing applications that are block- or file-based.

While object storage is more commonly discussed, there are also block and file services, as well as tables and message queue based storage services. For example, AWS has file storage (e.g. NAS) through its Elastic File System (EFS) inside the AWS cloud environment for data sharing among EC2 compute instances, containers and other services. Likewise Azure also has file capabilities within their cloud as well as accessible externally from Windows systems using SMB3.

AWS S3 is well known as an object-accessible bulk storage service; however, I find that many are not aware of different S3 tiers (besides Glacier). Within S3 there is standard, as well as Reduced Redundancy (RR) which as its name implies, has a lower level of durability (e.g. number of copies and nines) at a lower cost. There is also Infrequent Access (IA), which is optimized for less frequent access with good durability and lower cost, yet faster access than Glacier. Google, Azure and others have various bulk, object, blob and bucket and container services.

For cold storage, AWS has enhanced Glacier with two new tiers in addition to standard tier. These include Expedited with data access time (time to first byte) of one to five minutes, Standard (three to five hours) and Bulk (five to twelve hours). The cost per retrieval varies by service tier with Expedited being more expensive than Standard and Bulk.

Fees can vary within clouds and across regions. For example, AWS US East (Ohio) region is $0.004 per GB per month plus fees for Glacier. Fees can include API and access, as well as a transfer out of AWS. Besides AWS, Azure and Google, among others, have also enhanced their “cold” cloud storage offerings.

There are numerous gateways. Some are hardware appliances; others are software-defined and virtual appliances. Others are simply plugins for various software, tools, operating systems, and hypervisors and storage systems.

Some of these are browser-based, others map a cloud storage endpoint (with security access credentials) to your system and present as a local, or network-accessed drive, volume, mount point or share. Many cloud storage and, in particular, object- or block-based solutions also support the ability to create what appear to be folders and directory structure, making working with files and objects easy and familiar.

Besides block, file and object access, there are also file sync and share solutions such as Dropbox (among others) that have enterprise options (e.g. EFSS). In addition to browser and mobile app access, some of these offer access via API as well as command line or shell scripts for easier integration with existing environments.

On-premise cloud storage options include OpenStack Swift for object and bulk activity, or if you have an OpenStack compute environment, Cinder and block, as well as Manila folders (e.g. files). Other options for on-premise include Swiftstack, Cloudian, Ceph, IBM (Cleversafe), NetApp, NooBaa, HDS HCP and OpenIO, as well as many others. Speaking of cloud stacks, if you are not aware, Microsoft has an alternative to OpenStack called Azure Stack. As its name implies, it is a derivative of the software used in the Azure service for deployment on premise.

Moving To and Between Cloud Storage Services

Need to move some data, files, buckets, containers, blobs or objects between clouds?

There are also tools for moving objects and blobs among different services. Some tools in my toolbox include AWS CLI (on my EC2 instances), S3 Browser, s3fs (maps S3 buckets to my Linux servers as a mount point), Cloudberry, S3motion (moves buckets, containers, objects, blobs between services or storage systems), curl and REX-Ray, among others. REX-Ray, for those not familiar, enables persistent container storage for Docker among other platforms including accessing EFS (when running in AWS) among other local solutions. Using S3fs, I can do to standard Linux type activities involving files, folders, sub-directors with S3 buckets as though accessing any standard NAS or file server.

The catch is that many of those rely on pulling data out of one cloud and uploading into another. One way to get around being in the middle is run an instance or container at one cloud send and receive at the other.

Sure there is still a process which may have a cost (compute time) in the middle; however, this approach saves on plugging up your network. Also, watch out for network charges when accessing and downloading data.

Another storage migration option is “seeding” by shipping a single HDD, SSD or tape to large scale appliances capable of processing petabytes through services such as AWS Snowball.

Cloud Storage Applications

Besides using cloud storage as a target from on-premise or within clouds, another related topic is storage applications running within cloud services. For example, NetApp ONTAP Cloud software can run on AWS and Azure compute instances. ONTAP is the software that traditionally makes up the FAS storage system appliance. Cloud ONTAP on an AWS (or Azure) instance functions like a traditional storage appliance, except it is running in the cloud using AWS storage such as EBS (block) and S3 (bulk object).

To other NetApp systems, ONTAP Cloud is simply another node accessible from within or outside the cloud service. By being in the cloud using cloud storage while communicating with other nodes, the need for a cloud gateway is also eliminated, not to mention that it provides high availability (HA), business continuity (BC) and disaster recovery (DR) capabilities.

NetApp is not alone with implementing software-defined storage cloud instances of their solutions. Another variation is that VMware has announced support for vSphere and vSAN along with NSX and other tools on AWS. However, VMware tools will not be running on the normal EC2 virtual machines. Instead, they will be on dedicated private servers (DPS). You too can have the equivalent of a DPS with AWS via their Lightsail offerings, and other cloud vendors have similar offerings.

Note that cloud instances such as those at AWS, Azure, Softlayer, Google and many others also support onboard, on-instance direct-attached SSD storage. However, those also tend to be ephemeral, meaning they are not persistent across instance start/stop. Look into your particular preferred service provider as to what they support. Also note that performance will vary between onboard cloud instance SSD storage and general cloud block, file and object resources.

Cloud Storage Options

Cloud storage of some type is in your future, and it might not be just bulk object or file sync and share. There are many different types of clouds and cloud storage. While object and bulk are the most commonly discussed by the industry, there are many other options. The reason for these other options is to simply address the various needs and requirements of those using clouds, as well as what their applications need today.

Greg Schulz is Founder of Server StorageIO, author of several books including Intel Recommended Reading List titles, and a new one available in early 2017 called “Software-Defined Data Infrastructure Essentials.” He is also a Microsoft MVP as well as a VMware vSAN vExpert. Learn more at and Follow on Twitter @StorageIO.

Photo courtesy of Shutterstock.

  This article was originally published on Monday Jan 2nd 2017
Mobile Site | Full Site