The École Polytechnique Fédérale de Lausanne (EPFL) is using a relatively new technology – object-based storage – for active archiving of more than 5,000 classic jazz concert performances. The archive includes video and audio tapes of greats such as B.B. King, Ella Fitzgerald, Miles Davis and Elvis Costello in performances recorded over the past 45 years at Montreux Jazz Festival.
The festival asked EPFL, a Switzerland-based university, to digitally archive 1 PB of data stored on 24 types of media and formats – mainly audio and video tapes - that had been accumulating since 1967. So, EPFL formed the Montreux Sound Digital Project and migrated the prized data to LTO tapes to ensure it is viable before transferring everything to an Amplidata AmpliStor AS20 object storage system.
Alexandre Delidais, EPFL’s director of operations and development for the project, said he intended to build an online active archive accessible to anyone in the world – while preserving past and future performances from Montreux.
“We want to make the footage of the data available to students and researchers,” Delidais said. “That is the primary goal after saving the archive. We want to build a Montreux Jazz lab inside a building at the school. Then we want to build Montreux Jazz cafes across the world. The plan is to build 20 cafes worldwide.”
EPFL has purchased an AmpliStor AS20 storage system with 128 nodes, with each node containing 8 TB of capacity, said Tom Leyden, Amplidata’s director of business development and marketing. The system has three controllers using Amplidata's BitSpread object storage codec for unstructured data.
BitSpread takes objects from the application, runs them through a controller and splits them into sub-blocks that are encoded. That means they're mathematically transformed into rules, with an algorithm to create redundancy. The encoded sub-blocks are then spread across the storage nodes and disks to create fault tolerance. The system is policy-based, so the customer determines how the sub-blocks are spread out across the nodes or drives.
Delidais said Amplidata’s product satisfied a range of criteria such as low energy consumption, immediate access to data with low latency, strong security, fast and easy replication, as well as open access. EPFL will be able to segment the storage and control access for each segment, which include one for a secondary archive, another for online content in low-resolution Web format and a third for temporary files.
Delidais said the university chose an object-based infrastructure for active archiving because it’s highly redundant and it could handle 1 PB of data that can be accessible via HTTP, REST API, C language APIs and Python CLI. One of the benefits of object storage is that it allows data to be dispersed across wide geographical distances. Delidais said when the project began in mid-2010, he couldn’t find a file-based storage system that met his needs and budget, so he was surprised when he discovered Amplidata and object storage.
“We believe if we have a disk-based architecture, the data migration will be easier,” Delidais said. “We are getting rid of LTO in the long term. We need an active archive. We have about 20 percent of the archive on LTO now. That 20 percent will be moved to the Amplidata product by the end of the year.”