Archive for the ‘LAMP’ Category

Zmanda Recovery Manager for MySQL – What’s New in Version 3.5

Wednesday, March 20th, 2013

As we continue to see MySQL being implemented in bigger and more challenging environments, we are working to ensure Zmanda Recovery Manager for MySQL (ZRM) matches this growth and provides a comprehensive, scalable backup management solution for MySQL that can easily integrate into any network backup infrastructure.

The latest release of ZRM for MySQL is a significant next step, bringing disk space and network usage optimization and enhanced backup reporting, along with simplified management to help configure backups quickly and intelligently.   Additionally, ZRM for MySQL 3.5 now supports backup of MySQL Enterprise Edition, MySQL Community Edition, SkySQL, MariaDB, and MySQL databases running on latest versions of Red Hat Enterprise Linux, CentOS, Debian, Ubuntu and Windows – giving you an open choice for your MySQL infrastructure, now and in future, with confidence that your backup solution will continue to work.

Here is a look at the key updates in ZRM:

Optimization of Disk Space: We’ve implemented streaming for various backup methods so that you don’t need to provide additional disk space on the systems running MySQL servers. This will allow you to do hot backup of your MySQL databases without having to allocate additional space on the system running MySQL. Backup data will get directly stored on the ZRM server.

Optimization of Network Usage: We have implemented client-side compression for various backup methods so you can choose to compress backup data even before it is sent to the ZRM server. Of course, you also have the choice to compress on the backup server; for example, if you don’t want to burden the MySQL server with backup compression operation.

Enhanced Backup Reporting: Backup is often where IT meets compliance. ZRM allows you to generate backup reports for all of the MySQL databases in your environment. With the latest version, now you can generate unified backup reports across backup sets too.

Simplified Management: One of the key features of ZRM is that it hides nuances of particular types of backup method for MySQL behind an easy-to-use GUI, the Zmanda Management Console (ZMC). With the new release, ZMC brings new features for applicable backup methods, such as parallelism, throttling, etc. You will also find several tool tips to help you configure your backups quickly and intelligently, without having to dig through documentation on specific backup methods.

Broad Platform Coverage: MySQL gets implemented in various shapes and forms on various operating systems. We continue to port and test all variants of MySQL on all major operating system platforms. ZRM 3.5 supports backup of MySQL Enterprise Edition, MySQL Community Edition, SkySQL and MariaDB. Backup of MySQL databases running on latest versions of Red Hat Enterprise Linux, CentOS, Debian, Ubuntu and Windows is supported.

Seamless Integration with Backup Infrastructure: ZRM is architected for the MySQL DBAs. In order for DBAs to integrate and comply with the overall backup methodology of their corporate environment, we have made sure that ZRM can integrate well into any of the network backup infrastructures being used. While ZRM is already known to work well with almost all network backup environments, we have completed specific integration and testing of ZRM 3.5 with Amanda Enterprise, Symantec NetBackup, and Tivoli Storage Manager.

If you are putting together a new MySQL based environment, or looking to add a well managed backup solution to your existing MySQL infrastructure, our MySQL backup solutions team is ready to help:

MySQL Backup Updated

Tuesday, April 10th, 2012

As MySQL continues to grow (as a technology and as an ecosystem) the need and importance of creating and deploying robust MySQL backup solutions grows as well. In many circles Zmanda is known as “The MySQL Backup Company”. While we provide backup of a wide variety of environments, we gladly take the label of backing up the most popular open source database in the world, especially as we kick off our presence at the 2012 MySQL Conference.

Here are some of the updates to our MySQL backup technologies that we are announcing at the conference:

Announcing Zmanda Recovery Manager 3.4

We have updated the popular Zmanda Recovery Manager (ZRM) for MySQL product for scalability. Our customers continue to deploy ZRM to backup ever larger MySQL environments. Some of the scalability features include: Better support for hundreds of backup sets within one ZRM installation, support for more aggressive backup schedules, better support for site-wide templates, and deeper integration with NetApp’s snapshot mechanisms. We have also added support for the latest versions of XtraBackup and MySQL Enterprise Backup. We have also added experimental support for backing up Drizzle (via XtraBackup). If you are deploying Drizzle in your environment, we are looking for beta customers.

Many of our customers store their MySQL databases on NetApp storage. ZRM can be used in conjunction with NetApp Snapshot and SnapVault products to create database consistent backups without moving the data out of NetApp storage. ZRM creates snapshots of MySQL database volumes, which it can then move to another Netapp storage using Netapp SnapVault. SnapVault moves the data efficiently between various NetApp filers. This provides customers a way to protect the backups without impacting their corporate LAN. ZRM uses SnapRestore functionality to quickly restore the databases in case of a failure.

Announcing MySQL Backup Agent for Symantec NetBackup

If you have Symantec NetBackup deployed in your environment, and you would like to consolidate your MySQL backups within the umbrella of NetBackup based backup infrastructure, now you have a well integrated solution. We have released MySQL backup Agent, which is deeply integrated with Symantec NetBackup. This agent allows you do perform live backups of your MySQL databases directly from your MySQL servers to your NetBackup server.

NetBackup MySQL Agent

Backup of your MySQL databases to the Cloud

Public or Private Cloud Storage is a great choice for offsite store for backup archives. You can also use compute clouds as inexpensive DR site for your MySQL databases. For MySQL databases running on Windows, our Zmanda Cloud Backup product provides a very easy and inexpensive way to backup to Amazon S3 or Google Cloud Storage.

If you have MySQL databases running on Linux or heterogeneous environments, you have two choices for backing up to the cloud: You can use our Amanda Enterprise product with Amazon S3 or Google Cloud Storage option to move backup images created by ZRM to the cloud. Second option is to use the recently released Amazon Storage Gateway in conjunction with ZRM.

ZRM Backing Up To AWS Gateway Storage

We have published an integration report (available on Zmanda Network under the MySQL Backup section – free registration required) to show how you can deploy AWS Gateway to asynchronously upload backup files created by ZRM to Amazon S3.

As you can see, we have been busy updating our MySQL backup solutions. All of above improvements and feature additions have been done based on feedback provided by MySQL DBAs. If you are visiting the MySQL user conference this week, please do visit us at our booth – we would love to understand and discuss your MySQL backup challenges.

Fast Backups of MySQL Running on Amazon EC2

Saturday, January 23rd, 2010

If you are running your MySQL databases on the Amazon EC2 compute cloud, Zmanda Recovery Manager (ZRM) for MySQL can perform fast full backups of these databases by using Elastic Block Store (EBS) Snapshots. ZRM takes only a momentary read lock on the MySQL database during the creation of the snapshot, in order to ensure consistency of the backed up database archive. MySQL Backups using Amazon EBS snapshots are differential backups, meaning that only the blocks that have changed since your last full backup (via EBS snapshot) will be saved. For example, if you have a database with 100 GBs of data, but only 5 GBs of data has changed since your last snapshot, only the 5 additional GBs of snapshot data will be stored back to Amazon S3 during the current full backup run.

EC2 to S3 mysql backup diagram

ZRM automatically deletes EBS snapshots (containing full backups of MySQL) according to the configured retention policy. Just like other snapshot based full backups, ZRM intelligently correlates EBS Snapshots with incremental backups using MySQL logs, enabling you to recover your MySQL instances running on EC2 to any point in time.

Backups made using EBS snapshots can be recovered on the original EC2 instance or on a new EC2 instance. This also provides a quick and convenient mechanism to instantiate new MySQL database servers based on the database state from a desired point-in-time.

ZRM can run on the same EC2 instance as the MySQL database. On the other hand, if you have multiple EC2 instances with MySQL databases, you can run ZRM on one centralized EC2 instance dedicated for backup purposes. In this case, backup configuration and management for all MySQL databases is performed via Zmanda Management Console from this centralized backup server.

We have created an Amazon Machine Image (AMI) with ZRM pre-configured. This makes implementation of a MySQL backup solution on the cloud even simpler. We have used the “EC2 Small Instance” – which is powerful enough to backup most MySQL workloads in the cloud. This also makes it a very cost-effective option. This AMI is available to all ZRM customers, as part of the ZRM Enterprise subscription. You will need to create your own Amazon EC2 account, and pay standard per hour price to Amazon to run an instance based on this AMI. Note that you can configure your backup server instance to run only during the backup window. So, if you are backing up your databases once a week, and your backups takes less than an hour, then you can have this instance up only during that hour. EC2 pricing is per instance-hour consumed from the time an instance is launched until it is terminated. Each partial instance-hour consumed will be billed as a full hour. In addition to the EC2 compute capacity, you will pay standard storage charges for Amazon S3 (to store EBS Snapshots).

Join us on January 28th for a webinar on MySQL Backups (hosted by Sun/MySQL). Along with an introduction to Zmanda Recovery Manager, we will also discuss backing up MySQL applications on the cloud, and demonstrate the new ZRM AMI.

Zmanda Recovery Manager for MySQL – Detailed Review by Linux Journal

Friday, September 12th, 2008

Zmanda on Linux Journal Cover

This month’s (September 2008) edition of Linux Journal has an in-depth review of Zmanda Recovery Manager for MySQL (written by Alolita Sharma a veteran of Open Source). Oh, and we made it on the cover as well! If you are browsing Magazines section of your local bookstore, do look us up..

Slides from Red Hat Summit “Open Source Backup” Presentation

Monday, June 23rd, 2008

Here are my slides used for presentation at Red Hat Summit in Boston last week:

Zmanda: Open Source Backup (.odp open office format, 1.7MB)Zmanda on Red Hat Exchange

Poll of MySQL Quickpolls

Wednesday, June 11th, 2008

MySQL Quickpolls might be insightful for people who develop products and services for MySQL. Recently I was looking again at “How do you backup your production database” poll. To interpret the results, I wanted to know who are the people answering that and other Quickpolls. Are they the DBAs responsible for running MySQL in production or the developers writing applications that use MySQL? For a backup guy like me knowing that makes a difference.

Every Quickpoll gets a time stamp when opened and tells how many people answered the poll. It occurred to me that the normalized number of people (MySQL polls run for different periods of time) answering each poll could give me some insight. The graph below shows the daily number of people answering each poll in the last 24 months.

MySQL QuickPolls in the last 24 months
Of course, I understand there could be self-selection in answering the polls. For example, the DBAs could be more likely to answer “operational” questions and developers could be more likely to answer the questions relevant to them. However, I still think that the size of the horizontal blue bar is a proxy for relative interest for each poll.

Well, it seems that it is mostly the developers who are answering Quickpolls.The question that generated most interest was the one about primary programming language for MySQL applications. The question about backup was number three by relative interest, but it was only 20% of the most interesting (programming language) question.

Interestingly, the respondents were much more interested in sharing what type of hardware they consider for MySQL server vs. what type of storage they use for MySQL data. I am sure that with MySQL moving up and playing even more significant role for mission critical applications, more people will understand the importance of underlying storage such as Sun Thumper with ZFS, NetApp etc for building scalable and high-performance MySQL implementations.
Dmitri Joukovski

Slides from Zmanda keynote today (Online MySQL Backup)

Thursday, April 17th, 2008

Final slides from keynote delivered this morning at the MySQL user conference. Topic was protecting live MySQL databases.

(Slides render well in both OpenOffice and PowerPoint)

Wow! What a Great First Day at MySQL UC

Tuesday, April 15th, 2008

You have to have been living under a rock, if you did not know that today is the first day of MySQL Users’ Conference and Expo. We at Zmanda are so proud and privileged to have been awarded the “Partner of The Year”. The award is very meaningful to our vision of Simplified, Easy to use, commercial Open Source Backup and Recovery. We appreciate the award and are committed to making the life of the MySQL DBA hassle free. We have had a ton of visitors from all walks of MySQL user community talk to us today. Its fascinating to talk to customer and prospects on how they leverage the power of MySQL. To learn more about how Zmanda provides the Best in Class Backup and Recovery solution for MySQL, you can attend Zmanda’s sessions at the MySQL Conference & Expo include:
What: “Radically Simple Backup & Recovery for Live MySQL”
When: Thursday, April 17, 2008, 10:00 a.m. PT
Where: Santa Clara Convention Center, Ballroom E

In this keynote presentation Zmanda CEO and founder Chander Kant will discuss the most critical task for database administrators – protecting corporate data using online backup and recovery solutions. He will explain how Zmanda enables MySQL DBAs to take advantage of the latest advancements in snapshot technologies and storage engines to take on mission-critical online transaction processing challenges with confidence.

What: “Zmanda Recovery Manager for MySQL”
When: Wednesday, April 16 at 5:15 p.m. PT
Where: Santa Clara Convention Center, Ballroom H

We will present a session on Zmanda Recovery Manager (ZRM) for MySQL and how DBAs can deploy ZRM to protect their MySQL databases. In this session, we will discuss in detail how DBAs can use ZRM to select the right types of backup (e.g. logical, raw, snapshot, full or incremental), optimize backup performance, and activate point-in-time granular recovery right from the MySQL Visual Log Analyzer. we will also outline how ZRM can dramatically simplify and streamline day-to-day backup management tasks via its built-in administration tools, reports, monitoring and alerts.

What: “Top 5 Considerations While Setting up Your MySQL Backup”
When: Wednesday, April 16 at 4:25 p.m. PT
Where: Santa Clara Convention Center, Ballroom G

Protecting CFD (and making more money as a MySQL DBA)

Monday, April 14th, 2008

No, this is not a blog about Computational Fluid Dynamics – my least favorite subject in college. This is about a more exciting (sorry mechanical engineers!) CFD: Customer Facing Data. This is the data that is typically available on the website of an organization that their customers interact with. CFD can range all the way from profiles of users on a social networking site such as Facebook to the customer information database of an e-commerce company such as Travelocity.

CFD represents today’s data protection challenge. Probably the biggest challenge while planning a backup solution for CFD is that it is very hard to figure out what to plan for. You might be starting with a very small database which might grow much more rapidly than what you think. If the data can be segmented based on users or some other characteristic, then you will find that your databases may scale-out instead of scale-up. Also, rate of change can be very rapid, sometimes with lots of small changes (e.g. tags) or sometimes with big data changes (e.g. addition of user generated media content), very similar to the rather unpredictable viral growth pattern of the Internet.

Many organizations are keen to save and analyze behavior of users as they interact with CFD. This metadata can itself pose a data protection challenge of its own, since it may change very rapidly, while the actual data is not changing (e.g. tracking of customer intelligence metadata about which prospects downloaded which whitepapers from your corporate website, so that you can determine the best marketing & sales approach for these prospects).

MySQL is the DBMS of choice for CFD. Some skeptical press and analysts have said a few times that MySQL is not being deployed at the back-end of the enterprise (financials, billing etc.), but only on the web-tier. The implicit (sometimes explicit) implication being that web-tier applications are less important for the organization. I think this analysis fails to realize the actual importance of the data being stored in web-tier applications. Most of this is CFD, loss of which will cause huge pain and costs in the form of lost revenues, customers and reputation.

From its very inception, Zmanda has been focused on technologies powering CFD (MySQL, LAMP stack, and now increasingly Solaris). We have dug deeper than anyone else in understanding the needs of protecting this crucial part of any organization and very rapidly delivered products to address these needs. We are the data protection company for CFD. While we do protect back-end applications and platforms, our technology and business focus remains CFD.

I will be talking about deploying radically simple backup solutions for CFD on Thursday (April 17th, 10AM) at the MySQL user conference. I am in the unenviable position of making a Backup presentation sound interesting after the Facebook keynote (“A Match Made in Heaven? The Social Graph and the Database”). Well, I guess the point that conference organizers are trying to make is that if you have a radically simple MySQL backup and recovery solution, you will have more time to spend on Facebook!

Speaking of analysts, I recently read an analyst report which indicates that, on average, salaries offered to MySQL DBAs can be up to 40% lower than those offered to Oracle or DB2 DBAs. This ironically is considered one of the barriers for entry of MySQL in some environments. Per this report: higher paid database personnel have vested interest to keep MySQL out. Another claim is that somehow the lower salary to MySQL DBAs reflects the relative importance of the MySQL powered application for the business. Well if you are a MySQL DBA, you will do well to make your management realize that you are the keeper of their Customer Facing Data. Any business deploys its best resources for customer facing activities. Hopefully this will start reflecting on your paycheck soon.

If you would like to talk about your data protection challenges, or your salary as a MySQL DBA, stop by our booth #307 at the MySQL user conference.

Highly Scalable MySQL Backups using Snapshots (ZFS or NetApp)

Friday, April 11th, 2008

We have been focusing on providing the best possible backup solution for following scenario: 100 GB+ of data stored in MySQL database, Transaction intensive workload (i.e. rapid rate of change of data), with a business requirement to be able to perform point-in-time restoration of the MySQL database. Oh, the solution also needs to take into account that the database can grow to 1TB or more very quickly.

For such a scenario, we believe that the best possible solution today is a combination of:

  1. Storage level snapshots – a capability built into ZFS (Solaris), NetApp, LVM (Linux), VxFS, and VSS (Windows)
  2. Transaction logs generated by MySQL
  3. Point-and-click restore capability provided by Zmanda Recovery Manager for MySQL

Two reports came out today which go into nitty-gritty of above. First is a joint report written by NetApp and Zmanda engineers, titled “MySQL Backup and Restore Using Zmanda Recovery Manager and NetApp Snapshot Technology“. This report describes how NetApp Snapshot and Zmanda Recovery Manager can be used to back up and restore a MySQL database for NetApp storage systems. Specifically, this report covers the following topics:

  • Infrastructure required to deploy Zmanda Recovery Manager for MySQL with a NetApp storage system
  • Backing up a MySQL database using Zmanda Recovery Manager using NetApp Snapshot plug-in
  • Restoring a MySQL database using Zmanda Recovery Manager

Second one is a how-to blog written by Paddy on O’Reilly Databases: MySQL backups using ZFS snapshot. A key observation is sub-second time spent holding the read lock on the database while the snapshot was being taken.

While performing point-in-time recovery of their MySQL databases, DBAs don’t have to search for specific snapshots and manually combine them with database transaction logs. Zmanda Recovery Manager takes care of that behind the covers. DBAs simply key-in (or point-and-click) the timestamp to which they want to recover to.