Primary data storage options for the cloud
A comprehensive collection of articles, videos and more, hand-picked by our editors
Can you tell me if primary data is a good fit for the cloud or object technologies?
The answer to this question used to be simple: an automatic thumbs-down. Due to latency issues, primary storage just wasn't a good fit for the cloud and object storage. But today that answer, like cloud technology itself, is changing quickly due to powerful improvements in object storage technology.
In the past, transaction-oriented applications such as online transaction processing and online analytical processing didn't find object storage and cloud performance levels acceptable. That's not necessarily the situation today.
Here's the updated answer to the question: It depends on the application, users and your performance expectations. The issue is always latency, and that's because object technologies require more processing and more writing and reading the data -- or objects -- due to the extensive metadata available. The amount of metadata per object is much larger than that in file systems and can be variable. This adds latency. If the object storage utilizes erasure coding, even more processing is required, which adds more latency. To that you can add in the additional latency related to Internet and user connections. More latency equals longer response times or slower performance.
But just as file storage performance has improved significantly over the years, object storage is seeing rapid and, in some cases, radical performance improvement. Several vendors have improved object storage performance through the use of clever algorithms, effective use of caching, parallel processing and/or workload prioritization. The effect has been noticeably reduced latency and better response times that are equivalent to iSCSI storage area network or fast file storage. Indeed, there are scale-out iSCSI or network-attached storage vendors that actually utilize object technologies underneath. Additionally, more and more object storage products have iSCSI, Network File System and/or Common Interface File System interfaces that have the performance normally associated with those systems but at object storage economics and resilience.
When that storage is co-located with the application compute platform -- thereby eliminating the Internet or WAN latency -- it is no longer a given that object storage and the cloud are not good for primary data. A better way to look at object storage and primary application data is whether or not it is good enough for your particular performance requirements.
Related Q&A from Marc Staimer
A hard disk drive failure can put bytes of data at risk. Is multi-copy mirroring or erasure coding the more efficient data protection approach?continue reading
While it has yet to make a large impact in the market, open source software-defined storage is becoming an option for primary data applications.continue reading
Marc Staimer of Dragon Slayer Consulting explains what causes read disturbs and how flash vendors are mitigating the issue.continue reading
Have a question for an expert?
Please add a title for your question
Get answers from a TechTarget expert on whatever's puzzling you.