Published: 02 Jul 2014
Software just doesn't seem to get the same respect as hardware, at least according to the results of our ninth annual Quality Awards for backup and recovery software survey. Maybe backup users are a tough crowd to please, given how much data they have to protect and how little time they have to do it.
Despite that intimidating environment, two backup applications managed to rack up stellar scores: Asigra Cloud Backup software in the enterprise group, and Veeam Backup & Replication among midrange products. Both marched through their rating categories with barely a stumble.
For the last five years, enterprise backup was represented by the "big five backup apps" of CommVault, EMC, Hewlett-Packard (HP), IBM and Symantec; but Dell NetVault Backup and Asigra Cloud Backup software netted enough responses and joined their ranks this year. When NetVault was still a BakBone product it was a winner in year two of the Quality Awards; this is Asigra's first appearance as a finalist. On the midrange side, we had a record number of finalists, with nine products duking it out for top honors. A total of 455 respondents provided 1,012 product evaluations.
Enterprise. Asigra's out-of-nowhere win was stunning. It rolled up an overall score of 6.38, putting it well ahead of a closely bunched foursome: EMC NetWorker (5.78), CommVault Simpana (5.75), HP Data Protector (5.72) and Symantec NetBackup (5.68). (Asigra provides software for cloud backup services and software for enterprises to install in their data centers; we asked users to comment on the latter product.) Last year we saw some of the highest scores ever for enterprise backup apps, but this year's ratings have come back down to earth. If not for a sub-6.00 score in the initial product quality category, Asigra would have likely ranked among some of the top performers from the past. This doesn't suggest that the rest of the enterprise products fared poorly; in fact, most of the products put up some solid numbers across all rating categories.
Midrange. Among midrange backup and recovery products, Veeam's victory was equally impressive even if its margin of victory was slightly less than Asigra's. Veeam's best finish was second place a couple of years ago, but this time it outran the competition in every rating category. Second-place EMC Avamar (5.77) and BarracudaWare Yosemite Server Backup (5.68) were close to each other, but well off Veeam's pace. A tight cluster followed, with Microsoft, Dell and VMware separated by only 0.05 points. Veeam was impressively consistent, averaging between 6.21 and 6.37 for the five rating categories. The distance between Veeam and second-place EMC was nearly as wide as Asigra's in the enterprise field -- 0.54 points.
Enterprise. Purchasing new backup software is like getting married -- your data center and the new backup app will be hitched, for better or worse, and maybe forever. So a smooth acquisition process is key to getting started on the right foot. Asigra had the highest ratings for each of this category's seven statements, highlighted by a 7.00 for "My sales rep is knowledgeable about my industry" and a 6.97 for a knowledgeable sales support team. CommVault also did well on those statements, notching a pair of 6.28 scores. For the category, CommVault (5.88) and EMC (5.87) finished in a virtual dead heat after Asigra. EMC had ratings of 6.00-plus for reps who understand customers' businesses (6.00) and knowledgeable sales support teams (6.06). HP earned a 6.00 for the latter statement as well, accounting for the group's last 6.00 score.
About the Quality Awards
Find out the methodology behind the Quality Awards, as well as a listing of the 39 products included in the ninth Quality Awards for backup and recovery software survey.
Midrange. Veeam snared all but one statement for sales-force competence on its way to a category win. Veeam's top mark (6.37) was for having a knowledgeable sales support team, but EMC fared slightly better at 6.45. But Veeam showed particular strength for "The vendor's licensing formula offers good value" (6.34), being knowledgeable about users' industries (6.32) and understanding customers' businesses (6.30). EMC and BarracudaWare had the only other 6.00-plus marks in the category: EMC's came for knowing about users' industries (6.21) and for understanding users' businesses (6.0); BarracudaWare bit off a 6.00 for the licensing statement. Third-place VMware had its best score for its industry knowledge (5.60).
Initial Product Quality
Enterprise. In the most closely contested rating category, Asigra scored highest on six of the seven statements to nose out HP Data Protector by a mere 0.04 points and keep its category sweep intact. Users expect software products to install quickly and without a hitch; Asigra's best mark -- 6.65 -- was earned for "This product was easy to install" and its second highest (6.48) was for "This product was easy to configure," which adds up to a very promising initial experience. HP's run through this category was highlighted by a 6.09 for ease of use, a 6.00 for efficient use of tape and another 6.00 for being easy to install. Third-place EMC NetWorker had the highest tape use score at 6.08, and was second to only Asigra (6.23 to 5.92) for the bellwether statement "This product delivers good value for the money."
Midrange. Veeam's (6.37) dominance continued as it scored highest on five of the seven category statements, but it got a run for its money from BarracudaWare (6.10), which netted a leading 6.53 for "The product requires very little daily intervention" and bettered 6.00 on four other statements. Veeam flexed its muscles with a 6.75 for ease of installation and a 6.60 for the good value for the money statement. It also dominated the field for ease of use with a 6.55 rating that easily outdistanced Microsoft's 5.88. Third-place Microsoft tallied three scores of 6.04 or better for easy installation and configuration, and requiring little daily intervention. Symantec Backup Exec's 5.92 for "This product uses tape efficiently" was the highest in the group.
Enterprise. Eventually, the rubber hits the road and a product's features come into focus. For our enterprise backup apps, this was their strongest category, topped again by Asigra, but with four products topping the 6.00 mark. Asigra's 6.82 is the third-highest enterprise features rating we've ever seen, highlighted by a sterling 7.26 for "This product's file-system backup features meet my needs" and a trio of 6.94s for management features, being "a complete solution" and database backup features. Second-place CommVault also racked up a bevy of 6.00s on seven statements. Its best grades were for backup to disk functionality (6.55) and management features (6.37). EMC's third-place finish was highlighted by its high score of 6.28 for file-system backup features. Symantec had five 6.00-plus scores anchored by a 6.39 for being a complete solution.
Midrange. Veeam scored highest for all eight statements in this category en route to a 6.28 rating. But it had some competition from second-place EMC Avamar, which tallied five 6.00-plus scores. Third-place Dell AppAssure was the only other product to secure a statement score over 6.00. Veeam's best rating was for its backup to disk functionality (6.71), but it did nearly as well for file-system backup features (6.54) and management features (6.51). Backup to disk (6.55) and file-system backup (6.37) were also strong points for EMC Avamar. Dell's top grade (6.13) was earned on the same backup to disk statement, which the entire group did well on, yielding its highest group statement average of 5.88.
Enterprise. Backup applications toil day in and day out, so reliability is no small issue. Asigra's sweep through this category suggests a lot of happy users. It posted a 7.10 for meeting users' service-level requirements and 7.00 on the OS/platform support statement; other grades included a 6.71 for providing comprehensive upgrade guidance and a 6.46 for being able to operate "lights-out." Its lowest mark -- 5.81 for error handling -- was still high enough to easily outdistance the next product. EMC NetWorker was second with a category score of 5.66, just ahead of a closely packed group that included HP (5.62), Symantec (5.58) and CommVault (5.57). EMC and HP registered their best marks (6.31 and 6.05, respectively) on the OS/platform support statement, as did Symantec (6.26) and CommVault (6.55).
Midrange. Veeam almost equaled Asigra's romp through the enterprise category, but was eclipsed by EMC Avamar on one statement. Veeam's highest mark came for OS/platform support (6.53), but it also excelled for meeting service-level requirements (6.47) and requiring few unplanned patches/upgrades (6.41). EMC fared well on those statements as well (6.33 and 6.50, respectively), and trumped Asigra with a 6.44 rating for "Vendor provides comprehensive upgrade guidance." BarracudaWare Yosemite Server Backup trailed EMC by a mere 0.05 points, and distinguished itself with solid grades for requiring very few unplanned patches/upgrades (6.14) and platform support (6.07).
Enterprise. When a backup app balks, you need help and some answers pronto. Most of our enterprise backup products are well supported -- particularly Asigra, which just missed another category sweep. Six of the seven products snapped up their highest scores for the statement "Vendor supplies support as contractually specified," led by Asigra's glossy 6.90. The leader also did very well for having knowledgeable support personnel (6.61) and knowledgeable third-party partners (6.46). EMC's best statement showing was a 6.31 for delivering support as promised, followed by a 5.90 for providing adequate documentation and supporting materials. In addition to a 6.26 for the statement on delivering as promised, third-place CommVault posted a 6.18 for having a knowledgeable support staff. Symantec NetBackup's 6.09 was the top score for documentation.
Midrange. Veeam picked up its second category grand slam, leading the field on all seven tech support statements. Its highest marks came for knowledgeable support personnel (6.51) and supplying support as contractually specified (6.50), bolstered by a 6.40 for its documentation and supporting materials. Veeam's lowest rating was 5.97 for "The vendor provides adequate training" -- but it wasn't alone as five of the other eight midrange apps also received their lowest scores for that statement. EMC Avamar finished second for a fourth time, once again barely nudging out BarracudaWare Yosemite by the slimmest of margins -- 5.75 to 5.74. BarracudaWare tallied four scores of 6.00-plus to EMC's one, but EMC's were more consistent.
Would you buy this product again?
After asking our survey respondents to rate their products based on how strongly they agree (or disagree) with a series of statements, we pose a more general question and ask whether they would buy the product again given what they know today. The results have been topsy-turvy over the last nine years, with products that didn't finish near the top in the survey statement categories still earning a high buy-again response. It could be that our respondents are showing some tough love for their backup apps, or maybe having tamed the backup beast it now seems like a friendlier proposition.
Among our enterprise products, Asigra topped off its statement categories sweep with the best buy-again score as 93% of its users are ready for a repeat purchase. The topsy-turvy part figures in with Dell, which struggled through the rating categories but nonetheless earned some serious user loyalty with a 92% buy-again mark.
For the midrange group, Veeam, like Asigra, put an exclamation point on its statement success with 94% of its users saying they would buy the product again. Dell, with its Dell AppAssure Backup and Recovery product, finished second in this group too, despite an otherwise relatively lackluster performance. But with EMC and BarracudaWare in the third and fourth spots, the buy-again results followed a little more closely to form.
About the author:
Rich Castagna is editorial director of TechTarget's Storage Media Group.