Simplifying Backups with Zmanda Recovery Manager

Most modern enterprises process immense amounts of data as part of their day-to-day operations. On-demand access to this data is critical and most enterprises invest heavily in relational database technologies to manage their data. Cutting-edge technologies at cost-effective prices have led many enterprises to adopt open source databases such as MySQL. As a result MySQL has become a core component of many data management solutions.

Traditionally, backup of data stored in relational databases like MySQL has been an art form practiced by database administrators who have relied on a mix of command-line wizardry, custom scripts, and sometimes just sheer luck. Successful recovery of data is often seen as black magic, almost. Not surprisingly, most backup and recovery strategies emphasize the backup phase but don’t actually test whether data can be recovered successfully when it’s needed.

Zmanda Inc., a startup based in Sunnyvale, California aims to bring sanity into the backup and recovery process of MySQL databases by incorporating industry best practices in their product — Zmanda Recovery Manager (ZRM) for MySQL. Zmanda’s strong expertise in open source backup and recovery software comes from many years of work by the company’s core engineers on the award-winning Amanda open source backup system, a project which was started at the University of Maryland way back in 1991.

What is Zmanda Recovery Manager (ZRM)

ZRM provides a comprehensive open source solution for backup and recovery of MySQL databases. ZRM eases day-to-day backup and recovery tasks with its Web-based Zmanda Management Console (ZMC). The management console integrates with the Zmanda Network to provide online documentation and other support services for ZRM. Because it is open source and supports common standards as well as open data formats for backups, ZRM frees you from vendor lock-in. ZRM’s command-line interface (CLI) integrates well into existing backup and recovery processes.

Enabling advanced backup and recovery practices, ZRM provides Continuous Data Protection (CDP) for MySQL by combining filesystem level snapshots along with data from MySQL binary logs. Filesystem snapshots help reduce database server downtime, especially when working with large databases. ZRM uses a flexible framework of plugins for snapshotting LVM, NetApp SnapManager, Windows VSS, Veritas VxFS, and Solaris ZFS. Plugins for NetApp SnapManager and Veritas VxFS are sold separately.

Zmanda offers three versions of ZRM for MySQL: the ZRM Enterprise edition, the ZRM Cluster edition and a ZRM Community edition. The Enterprise edition includes a graphical installer, Zmanda Management Console, command-line interface, and plugin framework. Three levels of support (Basic, Standard, Premium) are available for Enterprise edition customers. An annual subscription with basic support starts at $300 (USD). Customers using the NDB storage engine with MySQL Cluster can take advantage of ZRM Cluster edition. The third ZRM version, the Community edition, is a freely downloadable subset of the Enterprise edition licensed under GNU GPLv2 which supports LVM snapshots but does not include the ZMC and the graphical installer.

Review Environment

For this review we evaluated Zmanda Recovery Manager for MySQL Enterprise edition version 2.2. Our test platform (ZRM server) was Red Hat Enterprise Linux 5.1 installed on a PC with an Intel Dual Core 3Ghz processor, 4Gb of system memory, and a 750Gb SATA hard disk. We installed MySQL Community Server version 5.0.51a-0 locally on the ZRM server as well as on another PC with Ubuntu 7.10 server installed. With this configuration we were able to test ZRM’s ability to backup from and restore to local and remote MySQL servers. Note that MySQL versions 4.1.x are also supported by ZRM. Finally, we setup our test databases on a 50Gb LVM (Logical Volume Manager) partition to test ZRM’s hot backup capabilities.

Installing ZRM

Zmanda provides installation packages for ZRM on Red Hat Enterprise Linux, SUSE Linux Enterprise, Fedora, CentOS, Solaris, and OpenSolaris. To install ZRM, we downloaded the ZRM graphical installer binary executable and ZRM license key file from the Zmanda Network site. The license key file has to be placed in ‘/etc/zmanda/zmanda_license‘. A full installation of ZRM Enterprise edition includes command-line tools, a MySQL server instance for use by ZRM, an Apache Web server instance for ZMC, a PHP 5.2.x and Perl 5.8.x environment.

Getting ZRM up and running is easy if you’re familiar with a MySQL environment on Linux. A couple of minor but necessary post-installation tasks included setting up ‘sudo‘ permissions for the ‘mysql‘ system user and creating a MySQL server user account with the right privileges to perform backups and restores.

ZRM’s Core Functionality

Zmanda Management Console (ZMC)

The ZMC is a Web-based interface that acts a control panel for managing the backup and recovery process end-to-end. Each major function (e.g., Backup, Monitor, Report, Admin, Restore) is represented by a tab and corresponding panel in ZMC’s interface. A key concept to understand while using the ZMC is that of a “backup set”. A backup set defines a group of databases or tables in a database on a MySQL server. Once a backup set has been defined, a ZMC user can configure various actions for that backup set. To help you get a feel for ZRM, the following sections highlight ZRM functions accessible through the ZMC.

Backup

ZRM provides several ways to backup MySQL. Important factors to consider while choosing the best way to backup a MySQL database include database availability during backup and the size of your database.

Logical backup should be considered if you’re looking for flexibility. A logical backup captures the output of the mysqldump utility and supports all MySQL storage engines except NDB. You can restore a logical backup to a platform different from the original. For example, a backup of a MySQL database running on an RHEL x86 system can be restored to a MySQL database running on a Solaris Sparc system. However, the flexibility of a logical backup comes at a price. MySQL tables are read-locked during backup. Also, restoring a large database can be slow, since SQL statements are used to recreate the database. Furthermore, the size of the backup can be larger than the actual database or table being backed up.

Raw backups should be considered if you want minimal database downtime and or if your database size is really large. Raw backups can take advantage of filesystem level snapshots to shorten backup times. Also, the backup size of a raw backup is the same as the size of the data being backed up but a raw backup is restrictive because it can only be restored to the exact MySQL server version on the same kind of platform. If you have a large database and or high transaction volume, you should consider using the quick snapshot option to further reduce backup time, since the filesystem snapshot becomes the backup and is not copied to the ZRM server. Note that on Linux, LVM snapshotting is only supported for local MySQL backups.

ZRM makes it easy to run a mix of full or incremental backups. Full backups include all data in a database whereas incremental backups capture the changes since the last successful backup. Incremental backups require binary logging to be enabled on the MySQL server.

If data security is a requirement, ZRM can utilize SSL for remote backups, as well as encrypt backup images using standard encryption tools such as GnuPG.

We tested ZRM’s backup functionality by running variations of full and incremental, logical and raw (with the quick snapshot option) backups of a million record database table. The backups completed without any problems, and we were able to verify this using the summary report page under the Report tab in the ZMC.

Monitoring

ZRM’s monitoring function presents valuable information about the most recently run backup job as well as about currently running jobs. Information includes how much time the backup took, backup size, and any warning or error messages. We found this feature to be very helpful when tracking the progress of long running backup jobs.

Reporting

ZRM provides excellent reports to analyze the details about backup and restore jobs. There are summary reports, custom reports, pre-defined reports, and data integrity reports. ZRM Enterprise edition provides 9 pre-defined reports and allows customized reporting using 30 preset data fields. You can use ZRM’s reporting data within other applications via RSS feeds. You can also configure ZRM to send email alerts about the status of backup jobs.

In our tests we found the Backup Application Performance Report to be quite useful because it provides stats about the amount of time tables were read-locked and total backup time. This information can help a DBA schedule backups to avoid peak periods of database usage.

Administration

ZMC’s administration function provides the ability to define and manage users, manage backup sets, setup ZRM site defaults and customize ZMC preferences. ZMC implements role based access control where a user can be defined as an operator or an administrator. An administrator can manage all backup sets on the ZRM server. However, an operator can only manage the backup sets they own.

Recovery

ZRM simplifies the full or incremental recovery of lost data. Using ZMC, you select a backup set to be restored and specify a MySQL server to restore to. If you want to manually verify your data after recovery, ZMC provides an option to shut down the MySQL server after recovery is complete.

Another option is point-in-time recovery, which gives you the ability to selectively restore data. Using the Visual Log Analyzer in ZMC, you can perform incremental recovery by specifying a given point in time or a binary log position, up to which you want to recover.

To test ZRM’s recovery functionality we successfully restored the backups created from our earlier tests. We also successfully restored the quick snapshot raw backup image of our million record table to our remote MySQL server.

Help

Even though the ZMC interface is relatively easy to understand, you may need more information about a specific ZRM feature. Fortunately, the ZMC provides context sensitive online documentation through the Zmanda Network. Other helpful resources include the Zmanda Wiki, forums, and blogs. Zmanda’s active online community is helpful and responsive.

A Star in the Making

Zmanda Recovery Manager provides a robust, open, and cost-effective backup and recovery solution for organizations relying on MySQL for their critical data. ZRM simplifies an otherwise intricate process of backup and recovery by managing complex open source utilities through its Web based console.

As with any evolving product, there is always room for improvement. Here are some of our suggestions — a smarter graphical installer which automates post-install tasks done manually today; a more sophisticated management console that enhances the user experience; snazzier graphical reports (think Google Analytics); a ZRM virtual appliance (VM image) for easier deployment in virtualized environments.

Try out ZRM’s open source community edition today. You’ll see how effortless MySQL backup and recovery can be!