Managing and protecting all enterprise data


Backup applications get record-breaking Quality Awards results

CommVault returns to the top spot among enterprise backup applications, while Dell AppAssure prevails over seven midrange finalists.

CommVault returns to the top spot among enterprise backup applications in the Storage magazine Quality Awards survey, while Dell AppAssure prevails over seven midrange finalists.

Products in the backup applications Quality Awards survey
About the Quality Awards

In our eighth Quality Awards for backup and recovery software survey, backup application vendors racked up some of the highest scores we've ever seen in this product category. Implementing a backup application is a lot like getting married -- it's a long-term commitment, so you're more than willing to overlook a minor flaw or two in your partner. But judging by our survey results, there seem to be plenty of matches made in heaven in our data centers.

On the enterprise front, CommVault Systems Inc.'s Simpana edged out EMC Corp.'s NetWorker by a very slender margin to take top honors. Symantec Corp.'s NetBackup had finished first in the last two surveys, breaking CommVault's previous string of three straight wins.

Among midrange backup applications, a first-time winner emerged as Dell Inc.'s AppAssure fought off stiff challenges from Microsoft Data Protection Manager (DPM), EMC Avamar and Acronis International's Backup & Recovery.

Overall Rankings

Enterprise. EMC NetWorker garnered the highest scores in three of the five rating categories, but CommVault Simpana emerged on top by a super-slim margin of 0.03 points by running a close second in those same categories and scoring convincing victories in the remaining two. As it has in the past, CommVault showed particular strength in the product features rating category with a score of 7.05 -- the first 7.00-plus category rating we've seen for an enterprise or midrange backup app. CommVault prevailed in the sales-force competence and product features categories, while EMC led the field for initial product quality, product reliability and technical support. IBM Tivoli Storage Manager (TSM) and Hewlett-Packard (HP) Co.'s Data Protector duked it out for third, with IBM achieving a wispy 0.01 advantage over HP (6.27 to 6.26). Rounding out the strong field, Symantec NetBackup's 6.18 would've been good enough to win in five earlier surveys.

Midrange. EMC Avamar's hopes for a three-peat in the midrange group were dashed by Dell AppAssure, which earned a second-highest-ever 6.53 overall ranking for midrange backup apps. It was only Dell AppAssure's second appearance among the finalists; last year, it finished third. This time, the product's 6.53 score gave it a relatively comfortable margin of victory over Microsoft DPM, which earned a personal best of 6.40 for its highest finish. The honors were well distributed, with the Dell product scoring highest in two categories, Acronis on top for two and EMC Avamar taking the fifth. The midrange products broke some records, too, with all-time-high overall averages in four of the five categories.

Sales-Force Competence

Enterprise. CommVault won the sales-force competence category in convincing style by coming out on top for each of the seven statements that comprise the category. Its range of scores was impressively consistent, running from a 6.56 for a licensing formula that offers good value up to a 6.96 for having a knowledgeable sales support team. IBM TSM was a strong second with an average score of 6.52, highlighted by a 6.85 for the knowledgeable sales support team statement and a 6.74 for having reps who are knowledgeable about customers' industries. EMC and HP finished in a dead heat for third (6.36), with both showing strength on the same "knowledge" statements. All the products scored well for sales-force competence, with only one sub-6.00 statement score and a group average that topped all seven previous survey results.

Midrange. EMC Avamar nearly equaled CommVault's category sweep among the midrange products, but had to settle for ranking highest on six of the seven category statements. EMC's 6.68 category score was highlighted by an eye-popping 7.07 for "The vendor's sales support team is knowledgeable." With a 6.81 for knowing about customers' industries, Microsoft DPM's category average of 6.53 put it firmly in second. Acronis nipped Dell App-Assure for third -- 6.24 to 6.23 -- despite Dell's top mark for the licensing statement. Veeam netted all 6.00-plus statement ratings to take the fifth spot with a solid 6.19.

Initial Product Quality

Enterprise. Getting up and running fast and flawlessly is a great way for a backup app to endear itself to a storage manager. EMC NetWorker proved its mettle for initial product quality (6.57) by scoring highest on four of the category's seven rating statements, highlighted by a flashy 7.11 for "This product uses tape efficiently" and a 6.74 for being easy to install. Second-place CommVault (6.36) prevailed on the other two statements -- requiring very little daily intervention (6.37) and delivering good value for the money (6.59) -- and matched EMC on one statement (both earning 6.44 for ease of use). HP Data Protector trailed CommVault by a mere 0.05 points to finish third with a score of 6.31.

Midrange. As a group, the midrange backup apps in our survey had the highest initial product quality score ever, highlighted by Acronis' 6.64, which was nearly matched by Dell's 6.63 in a well-contested category. Acronis scored highest for easy configuration, requiring little daily intervention and for not requiring professional services, and it tied Dell for ease of installation. Dell earned the best statement marks for ease of use and using tape efficiently; third-place Microsoft DPM led the group on the bellwether statement "This product delivers good value for the money." It was the tape efficiency statement that accounted for the 0.01 point difference in the first- and second-place products: Acronis' score was pulled down a bit by a 5.79, while Dell's AppAssure got a boost with a 7.00 (the only 7.00-or-better statement score in the category).

Product Features

Enterprise. Historically, CommVault's strongest Quality Awards showings have been in the product features category. This time around, CommVault outdid its impressive past performances with a category score of 7.05 (an all-time high), earned on the strength of 7.00-plus scores for five category statements. It received an astounding 7.52 for "This product is a complete solution," a 7.38 for scaling well and a 7.22 for file-system backup features. Second-place EMC NetWorker was hardly a slouch, with top scores on three statements: archiving features (6.79), interoperability (6.67) and backup-to-disk functionality (7.21). The remaining products -- IBM (6.33), Symantec (6.30) and HP (6.25) -- also fared very well in the category, adding up to the strongest overall product features average ever.

Midrange. Dell's AppAssure distanced itself from the field in this category with the rare achievement of notching four scores of exactly 7.00: for being a complete solution, and in the categories of file-system backup features, database backup and archiving. Dell also scored best for scalability (6.85), while second-place EMC ranked highest for interoperability and backup-to-disk functionality. Microsoft DPM won the final statement related to management features. Although it didn't score highest on any individual statement, Veeam posted consistently high marks that totaled 6.36, good for a third-place category finish. Category winner Dell's overall score of 6.78 was the second best ever for midrange backup products in this rating category.

Product Reliability

Enterprise. Apparently all five enterprise finalist products have earned their chops as reliable backup applications, as evidenced by the solid scores across the board. With a category score of 6.70, EMC NetWorker led the group, followed by CommVault and IBM. EMC and CommVault split the statements, with each coming out on top for four. Among EMC's winning marks, it earned a 6.89 for requiring few unplanned patches and a 6.78 for providing comprehensive upgrade guidance. EMC also registered a 7.06 for adequate OS/platform support, but was nudged out by CommVault's 7.11 for that statement. CommVault had another above-7.00 grade, a 7.07 for meeting service-level requirements. IBM's best score was a 6.76 for the OS/platform statement.

Midrange. Acronis snared its second category crown by outdoing Dell 6.44 to 6.43. Three different products had top statement scores: Acronis and Dell each led for three statements and EMC prevailed on the final two. EMC and Veeam earned category scores of 6.33 to finish in a tie for third. Acronis' greatest strengths were for requiring few unplanned patches (6.78) and having very few bugs (6.73). Dell had the two highest statement scores in the group, a 6.92 for OS/platform support and a 6.85 for meeting service-level requirements. Error handling was something of an Achilles' heel for the group as all but EMC (6.14) scored below 6.00 for the statement.

Technical support

Enterprise. We've often noted that solid technical support can make up for other product shortcomings, and while these enterprise vendors are obviously delivering solid products, they're not skimping on support either. At 6.42, the group's overall average for the tech support category is the highest ever, led by EMC's 6.78. Once again dueling with CommVault, EMC scored highest on three statements while CommVault came out on top for four. EMC exceeded 7.00 on three statements: having knowledgeable third-party partners (7.07), documentation (7.00) and providing support as contractually specified (7.00). CommVault had its share of 7.00s, highlighted by a 7.19 for the "contractually specified" statement and a 7.00 for having knowledgeable support personnel.

Midrange. Dell's AppAssure netted a 6.92 for having knowledgeable third-party partners among its two leading statement scores on its way to a 6.60 category win. Second-place Microsoft DPM earned top marks for three statements, including a pair of 6.58s for resolving problems in a timely manner and knowledgeable support personnel. Dell and Microsoft tied in one category. The final statement -- "Vendor supplies support as contractually specified" -- was won by third-place finisher EMC with a 7.03, the only over-7.00 mark for the seven finalist products. The overall average for the whole group was 6.07, which was just a hair under the previous best of 6.08.

Would you buy this product again?

Often, when we ask if users, given their experience with a product, would buy that product again, the results don't jibe with other category ratings in the survey. But this time, 93% of CommVault's users said they would repeat their purchases. Overall, 77% of the enterprise backup application rating group said they'd pony up their bucks again to buy the same products.

In the midrange group, fourth-place Acronis had the highest "buy again" rating, with 89% of its users saying they would buy it again today. Veeam's 86% buy again score also demonstrated solid user loyalty, as the mark for the midrange group as a whole averaged 77%. We did see some of those familiar anomalies, however, as CA ARCserve Backup customers showed significant product loyalty despite its category rankings.

About the author: 
Rich Castagna is editorial director of TechTarget's Storage Media Group.

Article 3 of 8

Next Steps

backup software applications are still important, learn why here

Learn more about  enterprise backup software

Dig Deeper on Backup and recovery software

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Storage

Access to all of our back issues View All