How to manage unstructured data
How do I manage an explosion of unstructured data, such as emails, software trees, archives of old code, documents, etc.? Many of these are accessed somewhat frequently and are being kept in a NAS
Is there a general vendor-agnostic file compression solution to control this growth?
There is a lot of coverage and focus on data reduction technologies with most discussions centered on data deduplication, single instancing, commonalty factoring, data normalization and differencing among other terms for solutions to address backup, archiving and general data protection. Vendors include Asigra, Atempo, Commvault, Data Domain, Diligent, EMC, Falconstor, Quantum and Sepaton among others.
There is another technology that gets overlooked in coverage and that is traditional compaction and compression at the server, in the network or on a storage device. There are also vendors who offer inline data compaction for NFS and CIFS data such as StoreWiz that can sit in front of an existing NAS farm to reduce your data footprint.
In addition to data reduction technologies, there are also archiving tools for your structured data including databases from vendors including HP and Solix among others.
This was first published in March 2007