Navigating the challenges of OpenStack backup

Brien Posey explains the unique challenges admins face when performing OpenStack backup.

If your organization is considering building a cloud then one very important consideration is that of backup and recovery. There are numerous backup solutions available for clouds that are based around software from companies like Microsoft or VMware, but backing up open source clouds has proven to be somewhat more difficult.

Open source cloud software

Although both Microsoft and VMware offer solid solutions for building private clouds, there is simply no denying that those solutions are expensive. In this age of "do more with less" some IT shops have begun to explore options that are more cost effective such as open source software.

The definitive open source cloud computing software is OpenStack. OpenStack is the result of a joint collaboration between 200 major computer companies. Some of the companies involved in the development of OpenStack include AMD, Dell, Rackspace, Intel, Cisco, and the list goes on.

OpenStack is based around a modular architecture in which a number of different components can be combined together in an effort to offer cloud computing services on standardized hardware. These modules are freely available under the Apache license.

Backing up OpenStack

Backup solutions are typically developed with either operating systems or applications in mind. OpenStack is neither. OpenStack is merely a collection of components that can be combined to provide various types of cloud services. As such, OpenStack administrators must consider what needs to be backed up and how to perform the backup.

OpenStack backups tend to center around backing up configuration files and databases. The configuration files can be backed up at the file level.

The /etc/nova /var/lib/nova folders should be backed up on the cloud controller and the compute nodes. However, you must exclude the /var/lib/nova/instances folder on any compute nodes. This folder contains live KVM instances. Restoring a backup that was made of a live KVM instance will typically result in an unbootable image.

One of the most important folders to include in your backup is /etc/swift/. This folder contains the ring files, ring builder files, and swift configuration files. If the contents of this folder are lost, the cluster data will become inaccessible. As such, it is a good idea to copy the contents of this folder to each storage node so that multiple backups exist within your storage cluster.

Some other folders that contain configuration data and should be included in your backups include:


In addition to the folders listed above, there are also several databases that need to be backed up. Typically the databases will reside on the cloud controller, which doubles as a MySQL Server. This server hosts databases related to the Keystone, Cinder, Nova, and Glance components of OpenStack.

You can back up these databases by using the mysqldump command. You can also use a commercial backup app, so long as it can back up a live database and is designed for use in an open-source environment. The command requires you to specify the names of the databases that you want to back up as well as an output file. For example, if you wanted to back up the keystone database to a file named KeystoneBackup, you could do so with the following command:

# mysqldump –opt keystone > KeystoneBackup.sql

As a shortcut, you can substitute the all-databases parameter in place of the database name. For instance, if you wanted to back up all of the databases to a file named MyCloud, you could use the following command:

# mysqldump –opt –all-databases >MyCloud.sql

What's missing?

Backing up configuration files and databases will allow you to protect your OpenStack configuration, but there are some things that are not protected by this type of backup. This method does not protect individual objects within object storage. Similarly, block storage data is also left unprotected. According to the OpenStack documentation, these types of data are left for users to back up on their own.

You can use any compatible backup application. The OpenStack documentation basically says that it is up to the users to back up data residing on the virtual machines that they create. As such, the backup application would have to be compatible with the virtual machines.

Of course, this raises the question of how you can better protect an OpenStack cloud. One thing to keep in mind is that like any cloud environment, OpenStack makes use of server virtualization. In fact, OpenStack is designed to work with a number of different hypervisors. You can see the full hypervisor support matrix at: https://wiki.openstack.org/wiki/HypervisorSupportMatrix

One way that you can better protect your OpenStack environment is to adopt a backup application that is specifically designed for the hypervisor that you are using. You will still need to protect the OpenStack configuration files and databases, but you can use the backup software to protect the individual virtual machines and their contents

Another thing that you can do is to adopt a backup application that is OpenStack aware. However, this is a tricky proposition. As previously mentioned, OpenStack is a collection of modular components that can be used to construct a private cloud. As such, none of the major backup products come preconfigured to back up OpenStack clouds.

Backup vendor Druva recently made headlines when they announced that their inSync software now supports OpenStack based scale-out storage. The software is designed to access OpenStack storage using the SWIFT OpenStack storage access protocol. It will also have the ability to back up file and object storage, as well as mobile endpoints (laptops, smart phones, etc).

Similarly, Zmanda supports the OpenStack framework with its Amanda enterprise backup software. The software is designed to create backups from the remote server layer.

Both Druva and Zmanda back up specific OpenStack resources, as opposed to the entire OpenStack infrastructure. It should be possible to also use traditional backup apps like NetBackup for Linux to back up the required components. However, NetBackup is not OpenStack aware. It would, therefore, be the backup admin's responsibility to manually configure a backup job that includes all of the required config data and databases.

The key to adequately protecting your OpenStack environment is to determine what it is that needs to be protected and then build a backup solution to meet those needs. While there are commercial products that can back up certain OpenStack resources, those products may not offer the level of protection that you require. You may have to combine commercial backup products with script-based backup techniques.

Dig Deeper on Cloud backup