All data deduplication systems aren't created equal, which is something the Anchorage, Alaska school district learned...
the hard way.
The school district turned to data dedupe to help tame its backups last July, installing three Data Domain DD410 data deduplication appliances. Satisfied with the results, Becky Reeves, the district's IT supervisor for systems and operations, began looking for a data deduplication device that could handle larger backups than the DD410.
"One thing about Data Domain is you can't add disk," Reeves said. "What you buy is what you get, it wasn't expandable."
Reeves soon learned that Quantum Corp. had a deduplication system that was expandable and would allow Reeves to add more capacity. But when the school district bought two Quantum DXi5500s with 4 TB of capacity last October, the nightmares began, according to Reeves. "We thought backup was backup, deduplication was deduplication and replication was replication," she said. "Nothing can be further from the truth."
The problems began when Reeves tried to transfer 640 GB of data from the DD410 to the DXi5500. "We transferred what we were backing up to Data Domain and pointed it to the [Quantum appliance]," she said. "We had more storage capacity, and we wanted to pump more to it. The Quantum box just couldn't handle the load we wanted to put on it in the time we have for backup windows. We had more capacity, but we weren't able to use it. We hit bottlenecks in performance."
According to Reeves, the Quantum system broke twice, once causing the district to lose data. Quantum replaced the first system in February, but the replacement unit began showing error messages last month. "We were down for almost a month before they could fix it," she said. "We redirected backups to Data Domain. If we had given up our other equipment, such as tape drives or Data Domain, we would've been up a creek."
In January, the DXi unit that the school district backs up to "died during replication," said Guy Liebeg, systems programmer for the school district. "We lost all the data we were replicating and never got that data back. We missed a whole weekend of full backups. Early [May] we started getting errors during replication, we don't know if our data is getting replicated."
Reeves said Quantum's service representatives told them the system was working correctly despite the error messages. "They tell me it's fine, they're just going through firmware changes," she said. "It doesn't give me a good feeling."
Reeves also doesn't feel good about the way the DXi5500 handles RAID rebuilds after system shutdowns, taking a couple of days while running in degraded mode. "They told us it wouldn't happen on orderly shutdowns, but it does," Reeves said. "That impacts your backup schedule."
Reeves said the district is still trying to decide its next move. One possibility is to buy a bigger Data Domain box, which obviously would be a costly option. But Reeves and Liebeg said they found Data Domain's performance and its user interface superior to Quantum's. Liebeg said it's easier to manage backup and replication through Data Domain's command line interface than through Quantum's GUI.
As for performance, Liebeg said, "We were running about 12 different jobs to one Data Domain. We used the same network connections to the DXi, pointed them to the new [Quantum] storage unit and ran the same jobs to it. It took longer. We throttled jobs back to see how many threads it can run in the same time, and it came to about seven."
Quantum last month began shipping a larger data deduplication unit, the DXi7500, using upgraded software. EMC is licensing the code for Quantum's new dedupe software in three of its new virtual tape libraries (VTL) launched last week. But Quantum still uses the same core software for its DXi5500 and DXi3500 that the Anchorage school district has struggled with.
Quantum executives admit there were problems with its first-generation data dedupe software that began shipping early in 2007. During Quantum's earnings call last month, CEO Rick Belluzzo said the company may have rushed to market with the dedupe technology it acquired when it bought ADIC in 2006. "We chose to move quickly to put a product together, and we made some tradeoffs to get the product to market," Belluzzo said.
When asked specifically about the problems experienced by the Anchorage school board, Quantum DXi product manager Steve Whitner said firmware upgrades have been issued for the DXi5500 to make the system more stable and increase performance. Whitner said the upgrades have improved replication and the way the system handles RAID rebuilds after an "unclean shutdown."
A satisfied customer
Quantum does have happy DXi5500 customers, including one in Anchorage.
The Anchorage Water and Wastewater Utility (AWWU) uses DXi5500s to backup and replicate data between its main office and a remote site. AWWU technology services supervisor Kenneth Rhoades said he purchased Quantum over Data Domain in February on the advice of his VAR and has no regrets and reports none of the problems experienced by the Anchorage school district.
Rhoades said he has reduced backups from 48 hours for full backups to six hours for about 4.5 TB of data. He also finds the DXi5500 "easy to use and administrate. You can do all the administration through the GUI, and you can adjust your replication so it doesn't affect your network. It will run constantly -- as soon as the backup is complete, it is already replicating that data to the far site."