Archive for February, 2009

Cloud Backup II

Tuesday, February 24th, 2009

In my previous blog on Cloud Backup, I wrote about the solutions we offer to backup to the Storage Cloud (e.g. Amazon S3). In this blog I will talk about backup of cloud, i.e. backup of your applications running on a Compute Cloud (e.g. Amazon EC2).

Let’s say you are migrating some on-premises applications (e.g. a customer facing enterprise app), which are currently being backed up to a tape library, to the cloud (fig 1).
Applications migrating to the cloud

Clouds don’t have a notion of a local tape library. So, your current backup solution will likely not work after this migration.

Backup of Apps on Clouds?So, where do you backup? Note that Compute Cloud vendors do not offer automatic backups. While they may offer storage redundancy features, e.g. replication and snapshots, these are not replacement for a complete backup solution.

You still need backup archives and a backup catalog for those archives to be able to recover from user and application errors. Just like RAID is not a backup solution for on-premises data, storage redundancy features offered by cloud vendors don’t provide a backup solution either. In summary, an automated backup solution is a must-have regardless of where the applications are running: on-premises or in the cloud. Well this is where Zmanda comes into play. We offer three choices as destination for backup of data residing in applications on the cloud(fig 3):

Backup of Apps on Clouds1. Backup to a local Storage Cloud - e.g. Amazon S3 if your applications are running on EC2. This is great option from a cost or performance perspective, but not so great from spreading-your-risk perspective.

2. Backup to a remote cloud. This requires having relationships with two different cloud vendors, but reduces your risks the most.

3. Backup to disks on your premises. This requires local infrastructure, but gives you complete control of your backup archives.

All three of our products: Amanda Enterprise, ZRM for MySQL, and Zmanda Cloud Backup are tested and supported on virtual machines running on the cloud.

So, today you can buy these products and run them on a virtual machine of your choice (as long as it runs an operating system from our compatibility matrix). In near future, we will be shipping virtual appliances which can be run either in your data center or in the cloud. So, lets say you bought and run few VMs in the cloud to run a set of your applications. In order to backup these applications, you will simply buy a Zmanda Backup Appliance (which will be another VM in the same cloud), and quickly configure this appliance to know about your application VMs and the destination for your archives. You don’t have to worry about dependencies, installation issues or optimizing the OS for backup purposes.

Currently we are working actively with Novell and VMware to build our first backup appliance. Novell today announced its partnership with VMware to build SUSE Linux Enterprise Appliances on VMware ESX. We are very excited about this development. SLES is already in Tier 1 of operating systems we test and support. We have hundreds of customers running Amanda Enterprise on SLES (e.g. Zen Innovations). Several of our customers protect guest OS’es running on ESX. We are now polishing up this combination to create an appliance-like experience, presented on a browser via Zmanda Management Console.

Reasons to migrate your applications to cloud are to increase efficiency and to dramatically reduce your IT costs. Our virtual backup appliances will help you with both of these goals.

Cloud Backup

Tuesday, February 17th, 2009

Today we made two major announcements: Amanda Enterprise 3.0 and GA of Zmanda Cloud Backup. We announced the release of several cloud backup features and optimizations as part of these product offerings.

When we say “Cloud Backup”, we mean two things:

1. Backup to Cloud: Backup of your on-premises data to a Storage Cloud (e.g. Amazon S3)
2. Backup of Cloud: Backup of your applications running on a Compute Cloud (e.g. Amazon EC2)

We offer solutions for both of above. In this blog, I will focus on Backup to Cloud.

Storage Clouds offer an excellent alternative to tapes or optical media for off-site backup storage. Your data is stored off-site and yet is on-line. You can initiate a recovery anytime and from anywhere, as long as you have internet access and your access credentials. You pay only for how much storage and bandwidth you use, and don’t have to invest in local tape infrastructure. Also, even as a small business you get have a profile of a much larger vendor (Tier 1 Storage Cloud vendors spend millions of dollars to ensure robustness of their infrastructure).

We have two different products to securely transfer your backup data to a storage cloud: Integration of Amanda Enterprise with Amazon S3 and Zmanda Cloud Backup.

These two products meet different needs:

Amanda Enterprise network diagram

If you would like to consolidate backup of a heterogeneous set of systems, including those running Solaris, Linux, Windows and OS-X, and then ship this consolidated backup to the storage cloud, our Amanda Enterprise (AE) 3.0 release is optimized for this. We added some key cloud optimizations in AE 3.0:

  • Cloud Vaulting: We allow you to backup to your local media and then vault to the storage cloud on its own independent schedule.
  • Transfer Block Size control: We allow you to control the size of your block transferred to Amazon S3. So far we had it pegged to 10MB. But now depending on your uplink speed and quality, you can pick a transfer block size of anywhere between 1KB and 100MB. So, if you have a fat and reliable pipe and you are not too many hops away from your storage cloud, you can use a much bigger block to get better performance. On the other hand, if you are stuck behind an unreliable pipe you can use a small block size to avoid retries.
  • Location control: We now enable an Amanda administrator to chose between storage cloud in US or Europe as destination for their storage. So, if you are in Europe and would like to backup to a US data center for disaster recovery, you can instruct AE to do so. Or if your country doesn’t allow certain kind of data to leave EU boundaries for compliance purposes, you can restrict the backup location to Europe.
  • Built-in Integrity Check: We do an integrity check on each block transferred to S3. So, any data corruption on transit will be caught right away.
  • Optional Encryption: Until now encryption during data transfer was hard-coded, i.e. you didn’t have a choice but to have your data encrypted during transfer to the storage cloud. We had done so to address the data privacy concern while transferring backup data over the Internet. Well, we got feedback that in many conditions the data being transferred was public data (e.g. postings on a forum). It was important data for business owner for protection, but there was no privacy concern about this data. In this case, the administrator wanted the fastest possible transfer and not incur the overhead of encryption. So, we decided to make encryption while transfer optional. Note that this is independent of encryption of data at rest - which is a configurable parameter on its own.

Zmanda Cloud Backup DiagramZmanda Cloud Backup (ZCB) is our offering designed for small businesses with a handful of Windows servers. It creates a direct link between a single Windows system and Storage Cloud. There is no backup server required. ZCB provides a very simple to use interface and configured automated backup of Windows file systems and applications. It is integrated with Exchange, SQL Server and Oracle using Volume Shadow Copy Service - so, it can backup these applications while they are running. You only pay for actual data transfered and stored (in addition to a small one-time software fees). So, if your data gets compressed 70%, you only pay for 30% of data size being backed up. This is in contrast to other online backup services such as Mozy Pro which charge on the basis of amount of data being backed up.

Also, in contrast to most other backup services, ZCB allows you to set up a retention policy of your choice. So, you can have a retention policy of a month, a year, or seven years (or any other timeframe). You can base this retention policy based on business needs, including any compliance needs.

In addition to file systems and applications, ZCB also backs up and recovers Windows System State (including Active Directory). This enables you to do a bare metal recovery, in case you lose your entire system.

Some leading journalists and bloggers today covered our Cloud Backup news:

BTW, we continue to adhere to our open standards and open formats philosophy. ZCB stores data on the cloud in ZIP64 format. So, any utility which understands this format can extract your files from the archive.

Comprehensive Backup and Recovery for ZFS filesystems

Tuesday, February 3rd, 2009

opensolaris logo

Amanda 2.6.1 release provides Application API and flexibility to configure multiple backup methods for a filesystem or an application. One of the examples of the application API flexibility is backup of ZFS filesystems on OpenSolaris. Amanda performs backups using platform tools to archive a filesystem. In case of ZFS multiple such tools are available to extract data for backup purposes.
Amanda 2.6.1 supports four different methods to backup ZFS filesystem/directories. Each method has pros and cons. They are summarized in the table below:ZFS backup methods
In addition to providing four different backup methods, ZFS snapshot pre-backup plugin is available (regardless of which method the administrator chooses for backup) to backup open files. It provides crash consistent backup of the filesystem/directory. Amanda allows OpenSolaris administrator to configure appropriate backup method for each DLE (disk list entry), which is a unit of Amanda backup configuration.

This work illustrates how availability of open APIs such as Amanda’s Application API fosters development and testing contributions from the Amanda community.