Requires Free Membership to View
Block-level, sometimes called variable block-level deduplication, looks at the data block itself to see if another copy of this block already exists. If so, the second (and subsequent) copies are not stored on the disk/tape, but a link/pointer is created to point to the original copy. For example, John's copy of a file may in fact just be a pointer to Mary's file -- if Mary's file was the first to be archived.
There are pros and cons associated with both file-level and block-level deduplication. For instance, if 1,000 identical attachments are sent out by a benefits coordinator, file-level dedupe will find those 1,000 attachments that are exactly the same, but it won't find the exact duplicate copies you saved (i.e., Benefits_file.Aug, Benefits_file.Sep, Benefits_file.Oct, etc.). Block-level dedupe will find all of the duplicates, even if named differently, and will store the name variations with pointers to the original blocks. Variable block-level will also account for misaligned data sets on disk and would also detect the duplicate files and exact copies with different names.
File-level deduplication will save a relatively small amount of space on your disk/tape archive. Block-level deduplication will save more space on your disk/tape archive, and variable block-level deduplication will save even more space on your disk/tape archive. However, as with any other data storage technology or technique, be advised that your mileage will vary depending on the amount of replicated data you have in your file systems.
For more articles, tips and expert advice on data deduplication, visit the data reduction and deduplication section of our site.
This was first published in January 2010
Join the conversationComment
Share
Comments
Results
Contribute to the conversation