Archive for the ‘MySQL Backup and Recovery’ Category

Backup Zen

Monday, April 20th, 2009

ZenOne of the questions that comes up often in the Backup world is “Why can’t I just write a script to do this myself!?”. Well, as a do-it-yourselfer myself, the answer is “Absolutely, a customized backup script can be written, and in fact, the first version of it won’t be that complicated to develop either”. However, a home-grown backup script can quickly become tedious to enhance and maintain.

Lets look at the progression of events following a decision made by a system administrator or a DBA to develop their own backup scripts. Lets take the example of Joe, a MySQL Database Administrator at an e-commerce company SuperWidgets:

1. January: SuperWidgets has been around for one year, and sales of their super widgets have been increasing. The CEO of SuperWidgets, Mary, has faced serious consequences of losing customer data before, and she tells Joe to make sure their MySQL database, which powers their web store, is being backed up.

2. Joe starts looking at ways to backup the web store database (running on Red Hat Linux), and discovers various tools that came with his MySQL installation: mysqldump, mysqlhotcopy and MySQL replication. After spending a couple of days of doing research on various tools, he decides to use the mysqlhotcopy utility to make a quick raw backup of the database.

3. Joe begins digging into the syntax for mysqlhotcopy, and in a day has a script running under cronjob which performs a backup of the web store database at midnight every day.

4. February: SuperWidgets uses Windows as the development platform for their web applications. One morning, Tom the database developer finds that a filesystem level corruption had taken out his database. He used mirroring, but since the corruption was logical rather than physical, both copies are damaged beyond repair. This causes a downtime of two days for the development team. Mary instructs Joe to make sure that all three databases on the development platform are also backed up.

5. Joe discovers that mysqlhotcopy doesn’t work on Windows. So, after doing further research, he writes a custom script to use mysqldump for backing up the development databases nightly via Windows Task Scheduler.

6. Since MySQL database backup has become a hot button for Mary, Joe starts monitoring the status of each of his MySQL backup scripts. He periodically logs onto each of the five systems where MySQL instances are being backed up by his scripts and makes sure that archives were successfully created the previous night.

7. March: SuperWidgets decides to use Alfresco as the Content Management System (CMS) for an internal project, with MySQL as the underlying database. Tom is in charge of the Alfresco implementation. The data stored in this CMS is sensitive and important. Mary gives Tom the responsibility of backing up and, when needed, restoring the CMS data.

8. Joe ports one of his backup scripts to the system running CMS, and trains Tom on nuances of feeding and caring for his script.

9. April: Tom upgrades his MySQL database and discovers that one of the options for mysqldump has changed, causing the backup scripts to fail. He fixes the script to work with the new mysqldump syntax.

9. May: The business of SuperWidgets has gone through the roof. But, one afternoon the webstore is brought to its knees because of an application error causing database to have inconsistent data. Fortunately, Joe’s script worked and he is able to recover the database using an archive from the previous night. Unfortunately, this meant that the transaction data of hundreds of customers since last night is lost. This forced Mary and rest of the management team to take several actions to manage reputation of now well known SuperWidgets.

10. As a result, Joe is instructed to ensure that MySQL can be recovered to any point-in-time, rather than just to the previous night’s status. Also, he has been instructed to send a high-level summary of MySQL backups to the management on a weekly basis. He has also been asked to look at reducing the amount of time the MySQL database is locked up while backups were being done. To add to Joe’s woes, Tom has decided to leave the organization. Joe must now takeover the backup of the CMS. Joe discovers that Tom has modified the original backup scripts for CMS without providing any documentation.

Joe’s situation is not atypical and shows how a Backup solution involves more than just putting a simple script around a utility which makes copy of data. For workloads of even moderate importance, any organization will find the need for cataloging of backup archives, monitoring and reporting to be vital. A common user interface across various backup methods, which is easy for new personnel to learn, has a huge long-term value as well. This is precisely where our backup solutions come in. Specifically for MySQL, our Zmanda Recovery Manager offers a great solution to Joe’s woes, by providing:

– An intelligent MySQL backup solution which figures out the best way to backup a particular MySQL database
– A common user interface across all platforms, whether they are Linux, Solaris or Windows
– A common user interface across all backup methods, whether they are raw backups, logical backups or snapshot based backups
– Integration between backup methods (e.g. snapshots) and MySQL logs to be able to recover MySQL to any point in time
– Role based access control, enabling management and DBAs to have control over who has access to what data
– A centralized backup solution, enabling a quick and automated health check of the entire backup infrastructure
– A customizable Reporting module, enabling automated reporting for desired levels of details

Zen InnovationsOne of Zmanda’s customers, Zen Innovations, initially backed up their data using scripts and manual backup procedures, but soon found that this was not scalable, and opted for Zmanda’s backup solutions. According to Sergio Laberer, Managing Director of Zen Innovations: “Zmanda’s ability to manage multiple platforms over a web based GUI was exactly what we were looking for. Our initial manual processes, scripts and cron jobs quickly started to get complicated as we grew our infrastructure. We needed to do backups regularly and be in a position to recover quickly without too much manual intervention. Our initial approach was neither scalable nor suitable to work efficiently.”

Sun Heats Up Cloud Storage

Wednesday, March 18th, 2009

Today Sun announced the Sun Cloud, including its Sun Cloud Storage Service at CommunityOne East in New York. We announced our partnership with Sun on integrating all our three products: Amanda Enterprise, Zmanda Recovery Manager (ZRM) for MySQL and Zmanda Cloud Backup with Sun Cloud. In near future, our customers will be able to chose between Sun Cloud Storage or Amazon S3 as the destination of their backup archives. Also, users running critical application on the Sun Cloud Compute Service will be able to protect their data using Zmanda’s products.

Sun CloudOf particular interest to us is Sun’s approach to use open and well defined APIs for their cloud services. We got access to Sun Cloud Storage Service (SCSS) few weeks ago. We are able to use either an S3 Compatible API or a WebDAV API for integration. We chose to use S3 APIs for integrating with Amanda, and WebDAV API for integrating with ZRM. In the short-term, these APIs made it easy for us to do the integration. In the long-term, this open architecture will result in rapid innovation. As an industry, open APIs enable us to stand on each other’s shoulders (rather than step on each other’s toes by developing closed solutions from scratch which have overlapping components).

This marks key next step in Zmanda’s cloud backup strategy. We are integrating with storage clouds using an open Cloud API, which is extension of our Device API. We will be the link between on-premises data and any storage cloud of customer’s choice.

Cloud backup space is certainly heating up, and Sun added its warmth to the space today.

Cloud Backup II

Tuesday, February 24th, 2009

In my previous blog on Cloud Backup, I wrote about the solutions we offer to backup to the Storage Cloud (e.g. Amazon S3). In this blog I will talk about backup of cloud, i.e. backup of your applications running on a Compute Cloud (e.g. Amazon EC2).

Let’s say you are migrating some on-premises applications (e.g. a customer facing enterprise app), which are currently being backed up to a tape library, to the cloud (fig 1).
Applications migrating to the cloud

Clouds don’t have a notion of a local tape library. So, your current backup solution will likely not work after this migration.

Backup of Apps on Clouds?So, where do you backup? Note that Compute Cloud vendors do not offer automatic backups. While they may offer storage redundancy features, e.g. replication and snapshots, these are not replacement for a complete backup solution.

You still need backup archives and a backup catalog for those archives to be able to recover from user and application errors. Just like RAID is not a backup solution for on-premises data, storage redundancy features offered by cloud vendors don’t provide a backup solution either. In summary, an automated backup solution is a must-have regardless of where the applications are running: on-premises or in the cloud. Well this is where Zmanda comes into play. We offer three choices as destination for backup of data residing in applications on the cloud(fig 3):

Backup of Apps on Clouds1. Backup to a local Storage Cloud – e.g. Amazon S3 if your applications are running on EC2. This is great option from a cost or performance perspective, but not so great from spreading-your-risk perspective.

2. Backup to a remote cloud. This requires having relationships with two different cloud vendors, but reduces your risks the most.

3. Backup to disks on your premises. This requires local infrastructure, but gives you complete control of your backup archives.

All three of our products: Amanda Enterprise, ZRM for MySQL, and Zmanda Cloud Backup are tested and supported on virtual machines running on the cloud.

So, today you can buy these products and run them on a virtual machine of your choice (as long as it runs an operating system from our compatibility matrix). In near future, we will be shipping virtual appliances which can be run either in your data center or in the cloud. So, lets say you bought and run few VMs in the cloud to run a set of your applications. In order to backup these applications, you will simply buy a Zmanda Backup Appliance (which will be another VM in the same cloud), and quickly configure this appliance to know about your application VMs and the destination for your archives. You don’t have to worry about dependencies, installation issues or optimizing the OS for backup purposes.

Currently we are working actively with Novell and VMware to build our first backup appliance. Novell today announced its partnership with VMware to build SUSE Linux Enterprise Appliances on VMware ESX. We are very excited about this development. SLES is already in Tier 1 of operating systems we test and support. We have hundreds of customers running Amanda Enterprise on SLES (e.g. Zen Innovations). Several of our customers protect guest OS’es running on ESX. We are now polishing up this combination to create an appliance-like experience, presented on a browser via Zmanda Management Console.

Reasons to migrate your applications to cloud are to increase efficiency and to dramatically reduce your IT costs. Our virtual backup appliances will help you with both of these goals.

OpenSolaris is a big step forward if you want to step back in time

Friday, December 12th, 2008

This week Zmanda was invited to participate in the launch of OpenSolaris 2008.11. In his interview Chander Kant, CEO of Zmanda, explained why OpenSolaris is a big step forward from backup and recovery perspective both as a data source for backup and as a repository for backup images.

OpenSolarisThe advanced snapshot capabilities of ZFS enables fast and scalable backups for today’s most demanding workloads for servers and easy to use data protection for workstations. For example, the new Time Slider feature in OpenSolaris provides an automatic way to backup your data on the local disk with intuitive browsing and recovering of files from snapshots using the GNOME file manager. That is great, but what if you loose your local disk? Considering that Time Slider does not support network attached disk yet, you might be in danger of loosing your data. By integrating Amanda Enterprise with Time Slider, we will provide advanced media management for backing up to disk, tape and storage cloud, and centralized administration of snapshots for multiple workstations running Time Slider.

One the server side, OpenSolaris provides an excellent platform for backup to disk. One of the biggest challenges of backup to disk is capacity management. It doesn’t matter how much capacity you have when you start backing up to disk. Sooner or later you will run out of it. Ease of provisioning of ZFS volumes is welcome by backup practitioners because you can add more disk capacity while your backups are running!

The combination of Zmanda backup solutions with Sun’s innovative OpenSolaris operating system, including the advanced ZFS file system, creates one of the most advanced backup-to-disk offerings available on the market today. The best part is you can build such a solution at the price point that will not break your budget. We are also working on getting Amanda and ZRM into OpenSolaris repositories.

ZRM 2.1: Storage snapshots as backups

Tuesday, December 9th, 2008

We released  ZRM for MySQL 2.1 Community Edition today.  ZRM 2.1 includes the ability to treat snapshots as backup images. ZRM has supported backup of MySQL databases using storage snapshots using a plugin interface. LVM (Logical Volume Manager) and Solaris ZFS snapshots are supported plugins in the ZRM Community Edition.
MySQL administrator can perform regular backups (backup images are stored in the ZRM server) or quick backups (snapshots acting as backup images). Quick backups have smaller backup windows and do not consume network bandwidth. When MySQL database is stored on one or more ZFS filesystems, users can do multiple quick backups in a day (even once every hour) and convert one quick backup image to a regular backup image. This schedule provides better recoverability and also reduces the resource consumption.

ZRM for MySQL source packages and binary packages for Linux and Solaris are available here. A whitepaper that describes how to quickly install and configure ZRM 2.1 is available at Zmanda network (free registration).

Get to know Zmanda!

Tuesday, December 9th, 2008

In past few quarters Zmanda has been growing at a break-neck pace. This has transformed us in many different ways. In the past 12 months, we have almost quadrupled our customer base (which is now in 40 countries!), introduced our third product line (Zmanda Internet Backup) and built new relationships with some of the leading IT infrastructure providers in the world.

These changes merit a fresh introduction. Some of you have openly wondered about who we are. Well, here is your chance to hear directly from us. Join us for a webinar on December 17th at 10AM (Pacific Time), titled: Get to Know Zmanda! In this webinar Zmanda CEO Chander Kant will discuss our business model, our philosophy and overview our product lines.

Dmitri Joukovski

Data Protection for Today’s Economy: Amber Road and Zmanda

Monday, November 10th, 2008

Today Sun announced the new 7000 line (aka Amber Road) of open storage appliances. Amber Road runs OpenSolaris and ZFS on industry-standard x86 hardware and includes innovative management software developed by Sun’s FISHworks (Fully Integrated Software and Hardware) group.

Our engineers worked along with Sun’s technologists on Amber Road in Sun’s labs for past few weeks, and today we are announcing support for Amber Road with both of our products Amanda Enterprise and Zmanda Recovery Manager for MySQL.

Amber Road is another example of innovation and value created by combination of open source and open systems. Combine  Amber Road with Zmanda’s open source backup products and you can deploy an extremely scalable and blazing fast backup solution at 1/5th the cost of products from proprietary vendors. We are also offering a free 60 days trial to all Sun Storage Systems customers (please contact your Sun or Zmanda sales rep for this).

New version of ZRM expands platform coverage, adds backup of application files

Tuesday, October 21st, 2008

Today we released version 3.0 of Zmanda Recovery Manger for MySQL. Here are the highlights of the release.

The new version of ZRM supports Zmanda Management Console (ZMC) running on 32- and 64-bit Ubuntu and Debian. We received quite a few requests for providing such support in the past.

In addition to Linux and Solaris, the ZRM server can now run on Windows. We supported logical and VSS-based backup for MySQL on Windows for a long time, but sometimes it was a challenge to introduce ZRM running on Linux/Solaris to all-Windows shops. Based on feedback from customers, prospects and from our partner MySQL, we have developed the version of ZRM that can protect MySQL in all-Windows environments.

We expanded the number of supported snapshot technologies by adding EMC CLARiiON SnapView to the list. This mid-range storage array is a staple for many data centers. Now the users who run MySQL on CLARiiON, will be able to take advantage of very fast and space-efficient snapshot capabilities of SnapView.

Lastly, based on requests of many users, the new version of ZRM provides the capability of doing backup of files at the same time as backup of MySQL database. For example, you can backup the directory and configuration files of application that is powered by MySQL. That simplifies recovery of a database and the application that uses MySQL.
Dmitri Joukovski

Zmanda Recovery Manager for MySQL – Detailed Review by Linux Journal

Friday, September 12th, 2008

Zmanda on Linux Journal Cover

This month’s (September 2008) edition of Linux Journal has an in-depth review of Zmanda Recovery Manager for MySQL (written by Alolita Sharma a veteran of Open Source). Oh, and we made it on the cover as well! If you are browsing Magazines section of your local bookstore, do look us up..

Webinar:Leveraging Replication to Optimize MySQL Backup & Disaster Recovery

Monday, July 21st, 2008

Leveraging Replication to Optimize MySQL Backup & Disaster Recovery 
In a global market, the uptime and performance of your MySQL database is critical. Taking the database offline to perform backups is simply not an option, but using replication is not a backup solution. So, how do you backup your data with minimal impact to your production environment? Using the ZRM for MySQL in conjunction with your replication slave, the ZRM for MySQL can back up your environment with no impact to production.

  1. Introduction to Zmanda.
  2. What is replication?
  3. Replication is not a backup solution!
  4. The ZRM is a backup solution!
  5. Use ZRM to back up a replication server.
  6. Talk about restoring to a master server.
  7. Close.
When: Tuesday, July 22, 2008.
Day Time: 10:00am Pacific / 1:00pm Eastern / 5:00pm GMT
Night Time: 8:00pm Pacific / 11:00pm Eastern / 11:00am Beijing / Noon Tokyo / 8:30am Pune / 1:00pm Sydney