Cloud storage offers enterprise organizations the potential to dramatically decrease storage costs. Even so, there are some significant architectural differences between cloud storage and commonly used forms of local storage. These differences can make it challenging to use both local and cloud storage in a seamless manner. Thankfully, the cloud storage market is maturing and there are a number of vendors who offer solutions for bridging the gap between local and cloud storage.
For those who might be new to cloud storage, it's easy to assume that the only real difference between a cloud storage architecture and local storage is the storage location. If this were true, then implementing cloud storage connectivity might be as easy as setting up an iSCSI initiator. In most cases, there are major architectural differences between on-premises and cloud storage. These differences make cloud storage connectivity far more challenging than it would be if cloud storage was based around the same architecture as local storage.
Cloud storage differs from traditional storage in that most -- but not all -- cloud storage offerings are built on an object-based storage platform. The reason for this is that object-based storage platforms can scale to massive levels of capacity and still deliver a very high level of performance.
Cloud storage API protocols
Simple Object Access Protocol: SOAP defines a standard set of rules for Extensible Markup Language (XML)-based message exchange. It uses transport protocols, such as Hypertext Transfer Protocol (HTTP) and Simple Mail Transfer Protocol. Since HTTP requests are usually allowed through firewalls, programs using SOAP can be assured that they can communicate with programs anywhere.
Representational State Transfer: REST is a stateless software architecture that reads webpages containing XML. REST is easier to use than SOAP, which requires writing or using a provided server program (to serve data) and a client program (to request data).
Fast and Secure Protocol: FASP is a patented bulk data moving technology from Aspera that's an alternative to Transmission Control Protocol-based transport technologies. Under this protocol, only those packets that have been confirmed as dropped are re-sent.
For an organization to establish connectivity to object-based cloud storage, there are two main obstacles that must be overcome:
- Local storage tends to be block based. If an organization wants to use local and cloud storage in a seamless way, then they'll need a way to translate between block and object storage.
- Cloud storage is typically exposed as a Web service. As such, standard storage protocols can't be used to communicate with cloud-based storage.
Because cloud storage is based on object storage and exposed as a Web service, access to it must be achieved programmatically via an application programming interface (API). These APIs are usually based around protocols such as the Simple Object Access Protocol.
Cloud storage gateways
In spite of the differences between cloud storage and local storage, there is a way to make cloud storage seamlessly available to the local network. The key is to use a cloud storage gateway, which is either a physical or virtual appliance that translates between commonly used file-based protocols (such as NFS) and cloud storage API protocols.
Although cloud storage gateways are designed primarily as a mechanism for making cloud storage accessible using standard file-based protocols, it's common for vendors to design gateway appliances with features designed to enhance the use of cloud storage.
File-based protocols supported by cloud storage gateways
Common Internet File System: CIFS lets programs make requests for files and services on remote computers on the Internet. It's an open variation of Microsoft's Server Message Block Protocol. CIFS uses the TCP/IP protocol, and is viewed as a complement to protocols such as the File Transfer Protocol and Hypertext Transfer Protocol.
Network File System: NFS is for Unix and Linux operating systems. It's a client/server application that lets a user view, store and update files on a remote computer as though they were on their own computer. Using NFS, a user or a system administrator can mount all or a portion of a file system.
Internet SCSI: ISCSI is an Internet Protocol (IP)-based storage networking standard for linking data storage facilities. By carrying SCSI commands over IP networks, iSCSI facilitates data transfers over intranets and manages storage over long distances.
Some vendors include local storage within cloud gateway appliances. This storage is commonly used for caching purposes. Files that are read the most frequently might be copied to a local read cache so data doesn't have to be remotely retrieved each time it's needed. Read caching provides a better all-around end-user experience.
Similarly, most cloud storage appliances have a write cache. The write cache is important because data can be written to local storage faster than it can be to cloud storage. Having a write cache allows the appliance to quickly store data during write bursts and then copy that data to the cloud storage as available bandwidth allows.
In addition, most cloud storage gateways include a deduplication engine that eliminates redundant data prior to copying the data to the cloud. This helps the data to be transferred more quickly and ensures WAN bandwidth is used efficiently.
Other key features
Accessing cloud storage from your local network is one thing, but using cloud storage in an efficient and meaningful way is quite another. If you plan to use cloud storage for file data, there are two important features that you should look for when selecting a cloud storage provider and a cloud storage gateway.
The first is a unified namespace, sometimes referred to as a global namespace. End users need to access their data, even after it's been moved to the cloud. A unified namespace presents local and cloud storage collectively as a single file system. From a user's standpoint, there's no visual indication that some files reside locally while others do not.
Five capabilities to look for in a cloud storage solution
- Global namespace
- Non-disruptive file migration capabilities
- Automated storage tiering
- Deduplication/WAN optimization
- Internal caching or local storage replication
Another important capability is automatic storage tiering. In a SAN environment, various types of storage are aggregated into storage tiers. For instance, solid-state drive storage can be treated as a high-speed tier, while commodity storage can be treated as a standard tier. Like local storage, cloud-based storage has unique characteristics such as high latency and low cost. From a storage management perspective, that means cloud storage can be treated as a separate tier within your storage architecture.
To derive the greatest possible benefit from storage tiering, an organization must determine what type of data should be stored on each tier. Archive data, by its very nature, tends to be high volume, but accessed infrequently, which makes it an ideal candidate for cloud storage. A good cloud storage gateway should be able to automatically transfer archive data to cloud storage based on rules set forth by the administrator.
Organizations may be able to drive down storage costs by moving aging file data to cloud storage. Before settling on a cloud storage provider however, it's important to investigate the gateway options for connecting to that provider. Cloud storage gateways differ significantly from one another with regard to their feature set.
Do your due diligence when choosing a public cloud provider
From our experts: Essential elements in a cloud storage offering
Using a cloud storage gateway for primary data
Object storage: The building block of a cloud storage infrastructure
Deploying dynamic storage tiering in your environment