Vice President of Editorial
Published: 03 Jun 2011
In our sixth Quality Awards for backup and recovery software, users picked a relative newcomer and a weathered veteran as service and reliability standard-bearers.
It’s been said that familiarity breeds contempt, but in the world of backup and recovery software, it looks like it might be breeding contentment. Backup application users seem more pleased than ever with their programs. The Storage magazine Quality Awards for Backup and Recovery Software surveys have seen overall scores climb steadily for both midrange and enterprise backup apps over the years, and that trend continues in the sixth edition of the award, with the highest winning scores we’ve ever tallied in both categories and the highest average overall scores for all products.
Products in the survey
The following products were included in the Quality Awards for Backup and Recovery Software survey. (The number of responses for finalist products is shown in parentheses.)
Asigra Inc. Cloud Backup*
BakBone Software Inc. NetVault (now part of Quest Software)*
CommVault Systems Inc. Galaxy or Simpana (23)
EMC Corp. NetWorker (42)
Hewlett-Packard (HP) Co. Data Protector (38)
IBM Tivoli Storage Manager (60)
Symantec Corp. NetBackup (76)
Syncsort Inc. BEX (Backup Express)*
Zmanda Inc. Amanda Enterprise*
Acronis Inc. Backup & Recovery (38)
Arkeia Software Network Backup*
Atempo Inc. Time Navigator*
BarracudaWare Yosemite Server Backup*
BridgeHead Software Inc. HT Backup*
CA ARCserve Backup (38)
CMS BounceBack Server*
Copiun Inc. Data Manager*
Druva Software inSync*
EMC Corp. Avamar (22)
FalconStor Software Continuous Data Protector (CDP)*
i365, a Seagate Company EVault Data Backup Software*
Microsoft Corp. Data Protection Manager (DPM) (24)
Paragon Software Group Paragon Drive Backup Server*
PHD Virtual Technologies PHD Virtual Backup (formerly esXpress)*
Quest Software Inc. (formerly Vizioncore) vRanger*
SonicWALL Inc. SonicWALL Continuous Data Protection (CDP)*
StorageCraft Technology Corp. ShadowProtect*
Symantec Corp. Backup Exec (126)
UltraBac Software UltraBac Server or Enterprise Edition*
Veeam Software Veeam Backup & Replication*
Vision Solutions Inc. Double-Take RecoverNow (formerly Double-Take Backup)*
*Did not receive enough responses to be included in the final results
EMC Corp.’s Avamar coasted to victory in the midrange group by sweeping all five rating categories on its way to a very impressive overall 6.69 rating. In the past, Avamar didn’t garner enough survey responses to become a finalist. Among the five enterprise finalist backup apps, Symantec Corp.’s NetBackup prevailed with a strong 6.46 in a tough competition with IBM’s Tivoli Storage Manager (6.33).
To put the high scores in perspective, consider that all five qualifying enterprise products racked up overall scores of 6.00-plus; in the past we’ve never had more than two products break the 6.00 barrier. On the midrange side, only Avamar ended up with a rating higher than 6.00, but the four other midrange finalists all had scores that could have won past competitions.
Thirty-three products were included in the survey (see “Products in the survey” at left); five midrange and five enterprise products emerged as finalists based on 566 product evaluations made by 345 survey respondents.
Able sales teams
Buying backup software is a lot more than just making a simple purchase. It represents a commitment that will likely shape how a company protects its data for years to come. As such, storage managers need the assurance of a capable sales and sales support team before taking the plunge. The Quality Awards sales-force competence rating category endeavors to measure the intangibles that contribute to a confident purchase. In the enterprise product group, Symantec NetBackup and IBM TSM topped the group with average scores of 6.50 and 6.43, respectively, divvying up top honors for the seven statements in the category. IBM came out on top for having a well-prepared sales force, with leading scores for the statements “My sales rep is knowledgeable about my industry,” “My sales rep understands my business” and “My sales rep keeps my interests foremost.” Symantec’s strengths were having a knowledgeable sales support team, being easy to negotiate with, being flexible and having licensing schemes that offered good value. The last is particularly notable as Symantec has taken some heat in the past over its licensing programs.
In the midrange group, EMC Avamar’s category average of 6.71 was the highest sales-force competence rating recorded to date, helping it to easily outdistance the other four products. Avamar’s commanding score was achieved by coming out on top for all seven statements in the category and was highlighted by two 7.00-plus scores: a 7.10 for “The vendor’s sales support team is knowledgeable” and a 7.05 for “The vendor’s licensing formula offers good value.”
With a category score of 5.91, CA ARCserve Backup just barely nosed out Symantec Backup Exec (5.90) to take second place. Microsoft Corp. Data Protection Manager (DPM), a first-time finalist, followed closely with a 5.83.
The worst thing that can happen with software (for both users and vendors) is to have it sit on a shelf or struggle through a long, painful implementation. Judging from the scores in the initial product quality rating category, neither is likely to happen with these vendors and their backup products.
NetBackup cruised to a win among enterprise products by posting the highest ratings for five of the seven statements in the category en route to an average 6.34 rating, which topped EMC NetWorker (6.06) and Hewlett-Packard (HP) Co.’s Data Protector (6.02). NetBackup’s strong scores came for “This product uses tape efficiently” and “This product delivers good value for the money” -- a key statement.
Ironically, IBM’s scores for those two statements were just a bit higher, and it came out on top for the other two statements. However, its overall 5.93 score was pulled down by less impressive ratings for “This product did not require professional services,” as well as configuration and ease-of-use issues.
For midrange apps, Avamar led the category with a 6.32 rating, putting it ahead of Symantec Backup Exec’s 6.13. Avamar had the highest scores for six of the seven statements, highlighted by a 6.86 for “This product delivers good value for the money.” Its margin of victory could have been greater but for a relatively low 5.14 for the professional services statement; Acronis Inc.’s 6.00 was the highest rating for that statement.
Features and functions
A slick sales experience and a painless deployment will fade fast if the product ultimately doesn’t deliver as promised. For a backup application, that means satisfying specific needs and meeting current and future requirements.
IBM’s TSM topped the enterprise app charts with a product features category score of 6.66 that edged out NetBackup (6.59) and CommVault Systems Inc. (6.48); CommVault scored highest for three statements related to management features, file system backup and archiving. But all the enterprise products fared well, with those three leaders closely followed by EMC NetWorker (6.41) and HP Data Protector (6.21) for a remarkably strong showing. It’s clear the spec sheets of enterprise backup applications are very well appointed with needed features.
The midrange products didn’t fare quite as well, with the exception of EMC Avamar, which notched four 7.00-plus statement scores on the way to a sweep and a 6.79 for the category, which handily outpaced Symantec Backup Exec’s 6.01. Avamar’s impressive run through the product features category was highlighted by a staggering 7.37 for a new statement, “This product works well with virtualized servers.”
Day in, day out
Product features are fine . . . as long as the product continues to chug away without stumbling too often. In the product reliability rating category, we capture impressions related to things like platform support, software bugs, patching and upgrades.
The enterprise apps were all closely bunched in this category, with NetBackup’s 6.34 nudging out HP Data Protector (6.27), which ended up in a virtual tie with IBM TSM (6.26). But CommVault, with a 6.13, and EMC NetWorker, at 6.05, also scored very well. Top statement scores were also distributed, with CommVault getting one, IBM two and Symantec the remaining five.
STORAGE/SEARCHSTORAGE.COM BACKUP & RECOVERY SOFTWARE QUALITY AWARDS VI: ENTERPRISE RANKINGS
Enlarge STORAGE/SEARCHSTORAGE.COM BACKUP & RECOVERY SOFTWARE QUALITY AWARDS VI: ENTERPRISE RANKINGS diagram.
It was another story for the midrange group as EMC Avamar scored highest for all statements to win the category by nearly a full point, 6.83 to Symantec Backup Exec’s 5.84. All the scores were solid in the category, but Avamar stood out with a 7.20 for platform support and scores of 6.95 for both “The product meets my service-level requirement” and “Vendor provides comprehensive upgrade guidance.”
Ongoing app support
The difference between a software bug and user error may be tough to define at times, but when help is needed with backup software, it’s usually an emergency situation. All vendors in our survey are doing a good job of providing technical support and advice, as exemplified by the highest-ever scores in both the midrange and enterprise groups.
STORAGE/SEARCHSTORAGE.COM BACKUP & RECOVERY SOFTWARE QUALITY AWARDS VI: MIDRANGE RANKINGS
Enlarge STORAGE/SEARCHSTORAGE.COM BACKUP & RECOVERY SOFTWARE QUALITY AWARDS VI: MIDRANGE RANKINGS diagram.
The enterprise leader for technical support was NetBackup, which led the field for all seven support statements for a very solid category average of 6.52. NetBackup scored high marks for delivering support per contractual agreements and providing good training and support documentation. IBM TSM was a strong second at 6.38, followed by EMC Networker (6.17).
About the Quality Awards
The Storage magazine/SearchStorage.com Quality Awards are designed to identify and recognize products that have proven their quality and reliability in actual use. The results are derived from a survey of qualified readers who assess products in five main categories: sales-force competence, initial product quality, product features, product reliability and technical support. Our methodology incorporates statistically valid polling that eliminates market share as a factor. Indeed, our objective is to identify the most reliable products on the market regardless of vendor name, reputation or size. Products were rated on a scale of 1.00 to 8.00, where 8.00 is the best score. A total of 345 respondents provided 566 backup and recovery software evaluations.
There was no suspense in the midrange group as EMC Avamar scored another impressive category win. Avamar’s 6.78 average was buoyed by a 7.10 for providing support as agreed and a 6.95 for training. Second-place Symantec Backup Exec (5.90) had three 6.00-plus statement scores, led by a 6.17 for providing support as contractually specified.
With 20/20 hindsight
Asked if they would buy the same product again now that they have some experience with it, most respondents showed loyalty to their backup apps. But sometimes we see a disconnect between these “buy again” sentiments and the other product ratings. In the enterprise group, IBM’s buy-again percentage was slightly higher than that of overall winner Symantec NetBackup -- 82% to 80%, respectively, with the other three products in the group ranging from 70% to 79%.
Among midrange products, Avamar netted an unprecedented 100% of users who said they would buy the product again today -- not surprising given the consistently high marks the product received across all categories. Seventy-five percent of Symantec Backup Exec users said they would buy it again; for the remaining products, the range was 61% to 66%.
BIO: Rich Castagna is editorial director of the Storage Media Group.