The University of Southern California (USC) has contracted with Nirvanix Inc. to establish one of the world’s largest private storage clouds, 8.5 petabytes (PBs) of digital archive spread over two sites. Several USC groups will use the digital archive cloud, and it will host the USC Digital Repository, a cloud archiving project that the university will open to external customers looking to digitize physical content -- or move digital content into a secure archive.
The Nirvanix Private Cloud Storage service consists of object-based storage pools maintained in data centers managed by Nirvanix. The service includes around-the-clock data monitoring, security features including Secure Sockets Layer (SSL) transport and AES 256-bit encryption, integration with backup software and policy-based replication. Data can be accessed through CIFS or NFS network-attached storage (NAS) protocols or SOAP/REST interfaces. Nirvanix also handles hardware and software upgrades and migration.
The USC private cloud will hold 107,000 hours of digital video from the USC Shoah Foundation Institute archives, consisting of testimonies of holocaust survivors and witnesses. The Shoah Foundation Institute is one of the partners of the USC Digital Repository, along with the USC Libraries and USC’s Information Technology Services (ITS).
The USC Digital Repository will license digitization, cataloging, content preservation, web access and file-server services to customers who want to create long-term online archives.
“We’re open for business now to researchers bringing us their collections,” said Sam Gustman, chief technology officer at the Shoah Foundation Institute and associate dean at the USC Libraries. “We’re unusual in that we are a repository cloud. Not only do we do storage, not only do we do preservation, but we do all the pieces through digitization, cataloging, preservation and access.”
The USC cloud consists of a 4.25 PB data center on the USC campus and another 4.25 PB data center outside of California. Nirvanix manages both sites and all digital files will have copies at each site.
“Having data out of state is important when you’re in California, in case there’s an earthquake,” Gustman said.
Security and flexibility highlight cloud features
Besides having separate locations, Gustman said the main characteristics he sought in a cloud were security and the flexibility to store content created in many formats without having to worry about those formats becoming obsolete.
“This gives us the ability to have megabytes of data available as you need them,” he said. “You don’t necessarily have to buy once and have the same amount [of storage] for a long period of time. There will be projects where you may need 200 terabytes to 300 terabytes as a workspace, and then you only need 10 terabytes as your final project space. We need to supply that as well.”
Gustman said not having to worry about tech refreshes is a crucial feature of the cloud. “Everything rots,” he said. “Film conservatively lasts 50 years, videotape 20 years, hard drives five years, data tape lasts three years before we see damage and DVD rots in two years sometimes. The newer the technology, the faster we’re seeing rot. As long as we have a system that lets us move files from one piece of media to another, we’re thrilled.”
Most of the Shoah Foundation archive content is on Oracle StorageTek tape libraries purchased three years ago, with a smaller amount on EMC Isilon scale-out NAS systems acquired in the last year. Gustman estimates it will take six months to migrate it all onto the Nirvanix cloud.
Having the data in the cloud will make it easier for the institute to manage it and make sure all the data can be accessed years from now.
Gustman said every piece of content in the Shoah Foundation is mirrored, and he uses digital fingerprints and checksums to check the data’s integrity. “We’re constantly looking at the bits, so if one has a problem, we put in a new tape, copy content over and throw away the old tape that was damaged,” he said. “We don’t trust any digital medium for more than three years, so we migrate off anything older than that.”
He added: “A nice thing about managed cloud storage is that we don’t have to worry about the upgrade path for digital files. The media’s constantly refreshed.”
Drop in prices makes cloud services more affordable
He said he has been searching for years for a better way to archive his data, but hadn’t found anything affordable until this year. He said Nirvanix won the business in a competitive bid. Nirvanix’s service is usage-based; it charges per gigabyte for data uploads, downloads and media services such as transcoding and image resizing. Gustman said he noticed a drastic drop in the price of cloud services this year.
“This is something we wouldn’t have been able to consider a year ago,” Gustman said. “I was pleasantly surprised that the cost of cloud has come down to where this is all possible.”
“Our concern was, what level of ability do we have to make sure the data doesn’t get corrupt?” he said. “USC has strict security protocols about data being encrypted when it’s being transported off campus, as well as being encrypted on any storage system when it’s somewhere else. The Nirvanix cloud supports this. It gives us the control we want of our own infrastructure, but at the cloud level.”