Storage is a 24/7 thing -- it runs around the clock and keeps growing by the minute. Coping with unrestrained capacity growth is no small matter, but protecting all that data effectively is an even bigger challenge. One way to get an upper hand on data backup is to cut the data down to size --literally -- using data deduplication technology.
Six or seven years ago, adding disk to the backup mix was the "big" thing that brought fairly radical changes and benefits to backup operations for most companies. Deduplication technology takes disk-based backup a step or two further by making it a much more practical data protection solution. Take the redundancy out of backups and backup data that used to require hundreds of terabytes of disk or tape capacity can now be wedged onto media a tenth -- or even smaller -- of that size.
All backups have duplicate data, but how much air a dedupe appliance or app can squeeze out of your backups depends on the types of files, how often you back up and a few other variables. There's a wide enough selection of data deduplication products to meet the needs of most organizations, large and small. You can implement dedupe through software (it might even be an option you can turn on in your current backup application), with an appliance or with a virtual tape library. Deduplication can be performed at the data source, as the data travels to its disk target or after the data has landed on the target. These are just some of the basic options available; there are more subtle differences among dedupe products that can help you find the best fit for your backup environment.
For large organizations, perhaps the most compelling development in deduplication technology is the ability for many dedupe to scale to ever-growing and often dispersed backup environments. Global deduplication effectively pools backup data from disparate dedupe installations and further deduplicates across the backup sets.
Still a relatively young technology, data deduplication is already branching out of its backup role and showing up in products designed to reduce redundancies in primary and nearline storage. These primary storage data reduction products cut data down to size where it lives and before it even hits the backup system. For storage managers swimming against the tide of unchecked capacity growth, primary dedupe could be a lifesaver.
If you don't already know that data deduplication is the biggest thing to hit backup in a long, long time, either your inbox's "out of office" sign has been hanging out there for too long or you're so buried by your own backup woes that you've barely had time to survey the scene. Odds are it's the latter case --so we put together this Storage magazine Essential Guide to give you an in-depth dose of all things dedupe. Isn't it time you dug yourself out of that backup hole?
--Rich Castagna, Editorial Director, TechTarget's Storage Media Group
Here's a look at what's inside this deduplication backup essential guide:
Data deduplication explained: It is important to have a firm understanding of its basics if you're considering using it in your storage environment. The purpose of this tutorial is to help you gain that understanding.
Global deduplication and backup: Now that deduplication technology is rapidly becoming a mainstream feature in data backup and recovery, a big question for any data dedupe product is, "Does it do global deduplication?" Learn about the pros and cons of global dedupe.
Source dedupe and remote offices: Source deduplication, also called client-side deduplication, can provide efficiencies in reducing remote-office backup data and bandwidth needs. Learn about who is using it and why in this article.
Primary storage data reduction tips: While not as hot as data deduplication technology for backup, primary storage data reduction, which includes data deduplication and data compression techniques, is getting warmer thanks to a scattering of products that try to shrink the data footprint on tier 1 disk.
FREE DOWNLOAD: Implementing data deduplication technology
This was first published in February 2011