Zmanda Recovery Manager (ZRM) - FAQ

General questions about backup and recovery of MySQL

What are the top considerations for backup of MySQL?

How fast and how easy do you want the MySQL Recovery process to be?
What will be the impact of MySQL Backup process on your Application?
What will your backup configuration look like? What, Where, When, and How of MySQL Backup
What will your backup configuration look like? What, Where, When, and How of MySQL Backup
How will you manage your backup process and backed up data?
What kind of tracking, reporting and compliance requirements does your business have from your MySQL backup implementation?
In-depth overview of the above considerations

What are the definitions of Cold, Warm and Hot backup?

Cold a.k.a. off-line backup means that during backup the MySQL server is shut down and not available to users.
Warm a.k.a. near-line backup means that MySQL server is running but tables are locked for writes during backup.
Hot a.k.a. on-line or live backup means that MySQL server is fully available to users for both reads and writes.

What is a Logical backup?

A logical backup saves the stream of SQL statements needed to recreate the database as it was in a particular point in time and the output is a text file.
Logical backups are possible regardless of the underlying backup engine (MyISAM, InnoDB, and others).
Logical backups are also portable: they can be restored to a MySQL instance on another platform, or even a different database engine.
Logical backups and especially restores take a very long time to perform because all the MySQL statements must be replayed.
Another disadvantage is the difficulty in predicting the size of logical backups.
Depending on the type of data and your database schema, the size of the logical backup could be larger than the database itself.

What is a Raw backup?

Raw backup provides a consistent copy of the database by storing a binary copy of the data files. There are several advantages of raw backups over logical backups:

Backup and especially recovery are much faster. For example, it not unusual to see that for the very same database the raw backup is 5-10 times faster than the logical backup, and recovery of the raw backup image is 20 times faster than recovery of the logical backup.
Because it is simply a copy of the database files, a raw backup has a predictable size.
It provides better scalability than logical backup, which could be important if your MySQL database is rather large, for example, 10-20 GB or more.

The disadvantage is that raw backups can be recovered only to the same version of MySQL server running on the same operating system as the original data files.

How Snapshots can be used for backup of MySQL?

A snapshot provides a copy of a volume with MySQL data files at a point in time. That copy can be used as a backup source, allowing recovery of the MySQL database to the point in time when snapshot was taken.

For in-depth overview of how snapshots are used for backup of MySQL please review "Zmanda Recovery Manager Guide to MySQL Backup & Recovery" white paper available on Zmanda Network

What are Full and Incremental backups?

A full backup is a complete copy of the specified databases or tables. A full backup can be logical, raw, or snapshot-based.
An incremental backup stores data changed since the last (full or incremental) backup by leveraging MySQL binary logs. When ZRM for MySQL performs an incremental backup, it flushes and saves the MySQL binary logs without a read lock, so it is a "hot backup" with no impact on the MySQL database.

What is Point-in-Time Recovery?

Point-in-Time Recovery means ability to restore your database to any point in time or to any transaction, for example to the point just before a user made an error and dropped a table.

What is Recovery Point Objective?

The Recovery Point Objective (RPO) defines to what point in time you have to recover your database. For example, if a user dropped a table by accident at 12:55 pm, you might want to recover to a point in time just before the delete command.
Please note that in order to obtain your Recovery Point Objective you need Point-in-Time Recovery

What is Recovery Time Objective?

The Recovery Time Objective (RTO) defines how much time it will take to recover the database. You should carefully consider how long your MySQL application can be offline while you do a recovery.

What is Backup Window?

A backup window is the amount of time that the application is unavailable or degraded during backup.

We use Replication as a backup. Why do we need anything else?

MySQL replication is a good mechanism to add robustness to a MySQL environment, but it is not a replacement for a comprehensive backup and archiving strategy: Replication only protects against hardware failure; it can not restore data lost due to user error (such as accidental table deletes) and malicious activity.

ZRM functionality

What is ZRM?

Zmanda Recovery Manager (ZRM) for MySQL is a comprehensive, certified, enterprise backup and recovery solution specifically designed for MySQL databases. With ZRM, MySQL DBAs can manage mission-critical, high-volume transaction processing environments with confidence. ZRM is certified by Sun/MySQL as the preferred backup solution.

What functionality does ZRM provide?

Centrally manage all backup and recovery operations for a single or multiple MySQL servers.
Perform hot backups of online MySQL databases via various snapshot technologies without any application downtime or interruption to online data access.
Schedule full and incremental backups of your MySQL database.
Perform immediate backup or pre-scheduled backups based on thresholds defined by you.
Choose to do more flexible logical or faster raw backups of your database.
Perform backup that is the best match for your storage engine and your MySQL configuration.
Backup your remote MySQL database through a firewall.
Configure compression and/or encryption of your MySQL backups to meet your storage and security needs.
Get e-mail notification about the status of your backups and receive MySQL backup reports via RSS feed.
Monitor and browse your backups.
Define retention policies and delete backups that have expired.
Recover your MySQL database easily to any point-in-time or to any particular transaction, e.g. just before a user made an error.
Visually parse MySQL binary logs for operational and security reasons, and easily identify recovery points.

How does ZRM enable Point-in-Time Recovery?

ZRM provides point-in-time recovery by recovering from the full and incremental backups. The incremental backup stores data changed since the last (full or incremental) backup by leveraging MySQL binary logs. The binary log saves all database events, thus providing a very fine granularity of recovery points. ZRM makes it easy to recover data from incremental backups even when multiple incremental restores are required to achieve the recovery point. Although incremental backups require that MySQL binary logs are enabled, in most cases this results in a performance hit of less than 1%.

How does ZRM take advantage of MySQL Replication?

ZRM takes full advantage of MySQL replication. If you set up MySQL replication, you can use the replication slave as the backup source. Such backups have zero impact on your primary MySQL instance.
ZRM installed on MySQL Slave will ask the MySQL Master to temporarily stop replication, perform raw backup and then continue replication.
Your MySQL Master can be on any operating system, but for ZRM to take advantage of replication, your MySQL Slave has to be on either Linux or Solaris.

How does ZRM backup NDB cluster?

ZRM performs raw hot backup of clusters using MySQL-ndb-tools and is the only product that provides consolidated backup solution for MySQL clusters.
ZRM automatically consolidates backup from each Data Node, which simplifies the recovery process.
ZRM lets you recover data to a cluster with fewer Data Nodes than the original cluster. This important feature lets you recover from a disaster with less hardware than was available in the original cluster configuration.

How is ZRM Enterprise and Cluster Editions are different from Community?

Comparison of ZRM for MySQL Community, Enterprise and Cluster Editions

	Community Edition	Enterprise and Cluster Editions
Which edition should I consider?	A SysAdmin, DBA or MySQL developer who feels comfortable being supported only by Community.	A SysAdmin, DBA or MySQL developer who wants enterprise-strength support and ease of use in production.
Is there a difference in functionality?	Core functionality is the same for all editions. The development effort is focused on providing new functionality as quickly as possible.	Core functionality is the same for all editions. Only Cluster Edition provides support for MySQL Cluster. The focus for Enterprise Edition is on stability, role-based security and ease of use in production environment via the Management Console.
What snapshots are supported for each edition?	Linux LVM and Solaris ZFS snapshots	Linux LVM, Solaris ZFS, Veritas VxFS, Windows VSS, and Network Appliance SnapManager.
What is the Zmanda Recovery Manager (ZRM)?	Currently available only with Enterprise and Cluster Editions.	The Zmanda Management Console (ZMC) is a web based service integrated with Zmanda Network. It provides management and visualization tools for defining all backup activities of a MySQL DBA: - what to backup, e.g. all databases or just the selected tables - where to store the backup images - when to perform backups, e.g. schedule daily, weekly or monthly full and incremental backups The Management Console provides extensive reporting. The unique Database Events Browser allows : - browsing the contents of incremental backups - searching for keywords in the backups, e.g. identifying the exact time of an important event such as dropping of a table - selecting transaction date and time as recovery point
Where do I get this edition?	Community Downloads.	Available through a paid annual subscription to Zmanda Network.
What kind of support is provided?	Support via forums, wiki, and mailing list. Additional resources (e.g. whitepapers, tutorials) are available to registered users of the Zmanda Network.	Three levels of enterprise-strength technical support, knowledgebase, certified software, functionality and security updates as well as optional professional services and training.
How much does it cost?	No cost.	Annual subscription with simple pricing. Support for snapshot requires additional subscription.
What kind of licensing does it have?	GPL and LGPL	Zmanda commercial license.

Download ZRM Enterprise Datasheet (PDF)

What is ZMC?

The Zmanda Management Console (ZMC) is a browser based GUI for configuration, management and monitoring. It provides management and visualization tools for defining all backup and recovery activities of a MySQL DBA:

what to backup, e.g. all databases or just the selected tables
where to store the backup images
when to perform backups, e.g. schedule daily, weekly or monthly full and incremental backups
point-and-click recovery
The Management Console provides extensive monitoring and reporting.

What is Visual Log Analyzer?

Zmanda Recovery Manager for MySQL has a Database Events Viewer aka Visual Log Analyzer built-in. This lets you visually browse the MySQL binary logs used for incremental backup, and select exactly what point in time you wish to restore. Specifically, it allows:

browsing the contents of incremental backups
searching for keywords in the backups, e.g. identifying the exact time of an important event such as dropping of a table
selecting transaction date and time as recovery point

What kind of backup reports are provided?

There are several predefined reports:

Backup report
Backup application performance report
Backup status report
Backup methods report
Backup retention policies report
Backup performance report
Incremental backup report
Backup replication report
Cluster backup report

In addition to the pre-defined reports, you can create and save for future use highly-customized reports on more than 30 different backup and MySQL parameters.

For in-depth overview of ZMC reports please review "Zmanda Recovery Manager Guide to MySQL Backup & Recovery" white paper available on Zmanda Network

What compression and encryption are provided with ZRM?

ZRM supports compression and encryption of the backup images, using any utility supported by the OS. For example you can use gzip for compression (which is the default). Any utility that reads from standard input (stdin) and writes to standard output (stdout) can be used.

Can ZRM use multiple streams in parallel to backup a database?

You can configure ZRM to backup in multiple streams by configuring tables in the database in different backup sets

Can I restore a single database if I configured to backup all databases?

Yes, you can restore a single database, but remember that other databases might be needed for referential integrity.

Can I backup a single table in a database?

Yes, ZRM allows backup and restore of a single table

Does ZRM allow deselecting corrupted records?

ZRM allows selective recovery of consecutive records. You can exclude the corrupted records by recovering consecutive records up to the corrupted one and then running another recovery till the next corrupted record. This approach is acceptable if you have only a few corrupted records.

Can third-party snapshots be integrated with ZRM?

Yes, ZRM has very flexible plugable architecture that allows 3rd party snapshot plugins. See this article for details

What is the performance of ZRM?

Different backup methods have different performance. Raw backups are 6-10 times faster than logical ones.

For in-depth overview of backup performance review “Backup and recovery benchmarks for MySQL with ZRM” white paper available on Zmanda Network.

Can I postpone backup if MySQL is under heavy load?

Yes, you can deploy a special plugin for postponing backups.

What is the dependency of InnoDb during backup process?

There is no dependency on storage engines, but we can take advantage on functionality provided with some engines, e.g. we can use mysqlhotcopy as a data extraction utility for MyISAM. Another example — using snapshots with InnoDB can provide truly hot backup of MySQL.

Having a combination of MyISAM and Innodb, do we need separate jobs for backups?

You don’t. ZRM is designed to handle variety of MySQL configurations and the same backup job can handle different engines.

If my 20 backups are successful and usually they are, I don’t want 20 emails. I only want an email if it “fails”

You may have to tailor the email being sent. This can be done with some scripting. We plan to improve this functionality in the next release of ZRM

When in a replicated environment, can you restore all replicated databases at the same time from one backup?

Yes. You should configure all databases in the same backup set.

Can I resume my backup after a failure instead of starting over?

If there is backup failure, ZRM uses a different backup method. ZRM uses a hierarchy of backup methods depending on configuration and storage engine in use. Currently we don’t have a built-in checkpoint mechanism to resume a particular failed backup run.

When ZRM is remote to MySQL, what gets installed on the DB - an agent? If so, how lightweight is it?

On MySQL server you need to install small footprint software that does data transfer to the ZRM server. This software runs only when needed (during backup run).

From a ZRM standpoint is there any difference backing up a MySQL DB on Linux vs. Windows?

For MySQL on Windows we use either logical or hot VSS snapshot backup. For MySQL on Linux we support logical, raw and variety of snapshots.

Does ZRM allow backup to tape?

ZRM provides backup to disk. It could be any disk that is accessible by ZRM, e.g. local hard disk, network attached storage NAS, Fibre Channel RAID array, etc.
Any enterprise media manager such as Amanda Enterprise, Veritas NetBackup or EMC/Legato Networker can make a tape copy of MySQL backup images.

Can I backup MySQL to Amazon S3?

ZRM provides backup to disk. It could be any disk that is accessible by ZRM, e.g. local hard disk, network attached storage NAS, Fibre Channel RAID array, etc.
Amanda Enterprise can move ZRM backup images to Amazon S3.

We run MySQL 24x7 and we can’t afford a read lock even for a second. What should we do for backup?

We recommend to take advantage of MySQL replication.
Stop replication for duration of backup off the slave
Resume replication.

Storage Engine and OS support

What storage engines are supported?

ZRM supports all storage engines. Note that some engines don’t support all backup methods.

Storage	Logical	Raw	Snapshot
MyISAM	Yes (Warm)	Yes (Warm)	Yes (Hot)
InnoDB;	Yes(Warm)	No	Yes (Hot)
NDB(cluster)	No	Yes (Hot)	No
Archive	Yes (Warm)	Yes (Warm)	Yes (Hot)
Falcon	Yes (Warm)	Yes (Hot with new online backup API)	Yes (Hot)

What Operating Systems are supported?

ZRM itself can be installed on either a Solaris 10 or a Linux-based server, and can back up MySQL databases running on any Windows or UNIX platform.
For ZRM on Linux the following backup methods are supported:

Logical -warm
Raw - warm
LVM snapshot - hot
VxFS snapshot - hot
NetApp snapshot - hot

For ZRM on Solaris the following backup methods are supported:

Logical -warm
Raw - warm
ZFS snapshot - hot
VxFS snapshot - hot
NetApp snapshot - hot

For ZRM on Windows the following backup methods are supported:

Logical -warm
VSS snapshot - hot

For ZRM on OS-X and other UNIX the following backup methods are supported:

Logical -warm

What versions of MySQL are supported?

We test with MySQL 4.1.x and MySQL 5.x, but version 4.0 should work also.

ZRM licensing

How is ZRM licensed?

The Community Edition is covered by GNU Public License (GPL).
ZRM Enterprise Edition and Zmanda Management Console are covered by commercial license.
ZRM enterprise is sold as an annual subscription per MySQL server and you can backup as many databases as you have on this server.
You need to buy additional subscriptions to take advantage of VxFS and NetApp snapshots.
ZRM Cluster edition is sold as an annual subscription per CPU

Competition

How does ZRM compare to InnoDB HotBackup?

InnoDB Hotbackup is a utility that provides hot copy of InnoDB storage engine. It is not a backup management solution. In addition to being limited to hot backup of InnoDB engine only, it is missing some vital functionality such as:

no incremental backups
no point-in-time recovery
no scheduling
no reporting
no monitoring
no e-mail notifications
no GUI
no enterprise technical support (support via e-mail only with no SLAs)

ZRM 2.2 provides support for InnoDB Hotbackup so the existing users of InnoDB Hotbackup can take full advantage of enterprise capabilities provided by ZRM.

How is ZRM different from Bakbone or Arkeia offerings?

BakBone offering for MySQL is just an add-on module for NetVault backup. You have to buy the expensive NetVault software in addition to the module to protect MySQL data.
MySQL data is kept in proprietary BakBone format. You will always need a valid BakBone license to recover your own data.
The module does not take MySQL operational status into consideration. For example, you can not take advantage of near instantaneous backups if snapshots are available.
In summary, the module provides a solution for backup administrator who already purchased BakBone software.
All of the above is applicable to Arkeia as well.

R1Soft claims near continuinous data protection for MySQL. What is that?

Continuous data protection (CDP) means that copy of data is saved every time data changes. The purpose of CDP is the ability to restore data to any point in time. ZRM 2.2 provides CDP functionality for MySQL.
Near CDP means that multiple recovery points are created allowing to restore to a point in time that is close to the Recovery Point Objective.
R1 uses proprietary snapshot technology to make multiple snapshots of Linux volumes.
R1 does not take any advantage of MySQL functionality, and it does not allow recovery to a point-in-time (which is the whole purpose of CDP).
R1 snapshots are done without locking of MySQL. It means that recovered database might not be consistent. It is extremely risky to perform any storage level backup of the MySQL database without taking at least a momentary read lock at the database level.
The price starts at $990 for the first MySQL server + $240 for each additional MySQL server. There are no SLAs for support.
Solution is not open and you can not recover your MySQL data without R1 software.
In summary, R1 solution is available only for MySQL on Linux and only for very specific versions of Linux kernel (because R1 depends on a kernel level driver). Solution does not take any advantage of functionality provided by MySQL, advanced file systems or storage hardware. R1 deploys proprietary copy-on-write of all changed blocks that requires a lot of additional computing resources, e.g. instead of each single write, R1 does two writes and one read.

Acronis claims bare metal recovery for MySQL. What is that?

Bare-metal restore (BMR) means ability to restore a computer from “bare metal” without installing the operating system first. Acronis creates a bootable image of a hard drive that brings the computer to a point in time when image was created (usually once per day).
Instead of using BMR software many system administrators create a bootable “Master copy of OS and applications” to speed up provisioning of new servers.
Acronis claims that it is possible, but it does not guarantee recovery to a hardware that is not exactly the same as the original MySQL server.
BMR solutions are designed to speed up the disaster recovery process for lost server, but they are not optimized to be used as backup for databases such as MySQL. Specifically, Acronis:

does not guarantee that the recovered MySQL database will be in a consistent state, because it does not take a read lock before performing a storage level backup.
does not work with any OS other than Linux and Windows
is not MySQL-aware and does not take any advantage of MySQL functionality such as replication or binary logs. To ensure consistency of MySQL data you have to write your own scripts to suspend and resume MySQL before and after backup image is taken.
does not provide point-in-time recovery for MySQL.

In summary, Acronis solution is not for backup of MySQL database, but rather for restoration of a server that happens to run MySQL without understanding how MySQL-specific backups should be done to ensure recoverability of MySQL data.