Marks: Native dedupe in Microsoft Hyper-V 3.0

Marks: Native dedupe in Microsoft Hyper-V 3.0

Marks: Native dedupe in Microsoft Hyper-V 3.0

Date: Mar 11, 2013

In this Storage Decisions presentation, Howard Marks, chief scientist at Networks Are Our Lives Inc., outlines how deduplication is performed in Microsoft Hyper-V 3.0.

"So [the] basic function of most data deduplication systems is they divide the data in your system into chunks, use [a] hash to locate other chunks that are similar, [and] do a byte-by-byte compare to see if that data is actually the same. If it turns out those two chunks contain the same data, instead of storing both, they store one, and use a pointer to replace it."

Marks noted in his presentation that Windows Server 8 and Hyper-V 3.0 feature native deduplication of data stored in Microsoft's New Technology File System (NTFS). He said that duplicate chunks of data -- those between 32 KB to 128 KB in size -- are kept in the System Volume Information store to cut down on seeking, which is similar to the technique used for RAM and network I/O.

He said two basic types of data deduplication exist: inline, which determines whether a piece of data is a duplicate in real-time, and post-process, which involves having a review of data at set intervals. He said that Microsoft chose post-process deduplication.

"If you've got 150 Web servers all running Apache -- for the kids who are going to take the Web server development class -- that's gonna deduplicate down into the space used by 1.3 of those servers, and will save you a lot of space…, said Marks."[And] if the data stays deduplicated all the way up the cache in your disk array, it's going to improve your performance as well."

More on Data reduction and deduplication

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: