Archive for April, 2007

Amanda at IBM Partnerworld

Friday, April 27th, 2007

If you are visiting IBM Partnerworld in St. Louis next week, be sure to stop by at the “Blades in SMB” ped #309 in the IBM Systems and Technology Group Solutions Village. Amanda backup software will be in action on a IBM BladeCenter server system. From ibm.com:

Combining Linux with System x or BladeCenter servers makes it easier to build a flexible, cost-effective, secure IT environment to help stretch your IT budget and confidently execute your business objectives - even as conditions change.

MySQL Users conference: ZRM for MySQL - Vision and Roadmap

Friday, April 27th, 2007

Thanks to everyone who attended the talk in person.

The slides used in the talk is available at ZRM for MySQL wiki

If you have questions or comments, please send them to me.

Data Protection for the LAMP Economy

Monday, April 23rd, 2007

The value of data stored in LAMP applications is increasing at an exponential pace. Indeed, the LAMP stack fuels an economy of its own - with its own currency, lingo and players. While e-commerce is the clear and present evidence of the LAMP powered economy, the currency for this economy is by no means just monetary. Value is manifested in many factors other than financial gains: personal reputation and legacy, karma points, creativity etc. The LAMP stack fires up innovation by enabling new ideas - you can quickly and cost-effectively prototype a concept which other’s may find bizarre.

User generated content (UGC) is one key currency of the LAMP stack. UGC, even votes (ok, diggs) on other’s UGC store tangible and lasting value. While naming “You”, a proxy for UGC, the Time’s Person of the Year 2006, The Time magazine said: “It’s a story about community and collaboration on a scale never seen before. It’s about the many wresting power from the few and helping one another for nothing and how that will not only change the world, but also change the way the world changes.” With current trends UGC (most of which is stored in LAMP stacks) will continue to pack in increasing value for companies and communities around the world.

Since the cost of deploying applications on the LAMP stack tends to be very low, sometimes their importance to the enterprise may not be perceived accurately. The LAMP stack moves the value of IT infrastructure to business data and applications, which is exactly where it should be (instead of costly underlying technology). The importance of protecting LAMP data can easily be gauged by looking at all LAMP based applications that you rely on. Whether you are using a vBulletin or phpBB based forum for your users, or SugarCRM for your sales force, or Mediawiki for your corporate wiki - loss of data in any of these instances will result in at least lost productivity, if not lost revenue and reputation.

The LAMP economy comes with its own set of challenges and hazards, e.g. crowdhacking, comment storms etc. Dealing with these challenges is especially challenging for IT managers since it is extremely hard to get scheduled downtime on LAMP applications which power a busy website. In addition, such an environment has its own requirements as far as point of time where data should be recovered to (aka Recovery Point Objective). For example, owner of a web based forum may want to recover their data to a point just before a rogue user created a login and started vandalizing the forum.

Several dynamics make data protection for LAMP based applications a more challenging problem than traditional environments. For one, the data stored within LAMP stacks in many cases does not have any physical record. E.g. nobody keeps a printout of all threads in a forum. So, if LAMP data is lost and cannot be recovered, you would simply need to live without that data - there is no way to recreate it from any physical records.

Data in LAMP applications is stored both in MySQL databases and filesystems (typically configuration data). LAMP applications have a tendency to scale out instead of scaling up. One application may be spread across multiple servers (either in form of MySQL Cluster, or simply independent aspects of application distributed on independent LAMP stacks). The application administrator has to take into account multiple servers and locations of their LAMP data while putting together a backup strategy. In such an environment, creating a point-in-time consistent backup is a challenging task.

Frequently, LAMP based applications are hosted at a service provider’s site, instead of a captive data center. This provides additional challenges (and opportunities) for data protection. You will need to carefully plan how the recovery for the whole stack and application will take place at a different location from your hosting provider (e.g. in a hurricane Katrina like situation). In many cases, administrators will need to backup their LAMP data remotely using a secure communication protocol. An interesting alternative here is to use a remote storage grid (e.g. Amazon S3 service) to do the backup of the LAMP applications. Why bother with local tape hardware (and all the idiosyncrasies of tapes), when your data is remote anyway.

It is imperative that today’s IT managers assess the value of data stored in their LAMP stack based applications. They need to architect a backup solution for their LAMP applications based on the impact on application performance, application availability, type of failures to recover from, and the cost of implementing the solution. Administrators need to pay attention to the data in all layers of the LAMP application in order to get a consistent data backup for the whole LAMP application stack.

Zmanda is at an interesting place when it comes to the LAMP stack. We use the LAMP stack in our own products - the new Zmanda Management Console is developed on it, and we are focused on making it simple to protect the value of LAMP application data. Our open source projects extensively use wiki and forums for community collaboration and communication. Our products provide data protection for the entire LAMP stack. Amanda is the leading backup solution for a network of Linux filesystems. Our Zmanda Recover Manager for MySQL product is one of the most popular solutions to backup MySQL databases.

Adspace Mall DisplayOne extensive user of the LAMP stack is Adspace Networks - the largest in-mall digital audio/visual network in the United States. LAMP data is not just displayed on a small browser window - Adspace shows its customer’s advertising artwork on sixty inch plasma displays mounted in 8 foot tall enclosures! Using Adspace’s LAMP based applications, retailers can create new campaigns and upload advertising artwork - which then shows up on the huge screens. LAMP stack enabled Adspace to create and deploy this application in a very aggressive timeframe. Wide availability of LAMP consultants and hosting providers was seen as a big plus while deciding the application framework. Adspace needed a solution which could backup their data more frequently than the traditional nightly backup (due to high value of the customer data). While their applications are deployed at a remote hosting site, they wanted to keep the backup data on their own site. Adspace deployed Zmanda Recovery Manager for MySQL to backup their LAMP data. This solution enabled them to create a consolidated backup solution with point-in-time recovery capability, without having to spend time and expertise in building and architecting a LAMP backup solution.

While Google and Yahoo are the most likely destinations if you want to search for something, if you want to discover stuff one of the coolest destinations is StumbleUpon. StumbleUpon is helping more than 2.1 million users discover and share interesting websites. “Collaborative Opinions” is the currency that StumbleUpon trades in. LAMP stack stores this extremely valuable data of users, their preferences and friends, and all the websites they discover. StumbleUpon stumbled upon Zmanda’s LAMP backup solutions when they were looking to reduce the time and complexity of backing up this data, which is increasing by the minute. The rapid growth of their data mandated incremental backup of their database - full backup every time was just too time-consuming. Today they use Zmanda Recovery Manager for MySQL to backup their database and manage the backed up archives. StumbleUpon implemented Zmanda’s backup solution for its simplicity and effectiveness for their exact needs.

Protecting the LAMP Stack

Zmanda at MySQL Conference and Expo

Monday, April 23rd, 2007

Zmanda is a Diamond sponsor of MySQL users conference. Come Visit us at booth 415.
Various membes of our team are also presenting at the conference. If backup of MySQL databases is
of interest to you, you do not want to miss any of the following presentation:

Zmanda Speakers

Keynote: Data Protection for the LAMP Economy
Presenter: Chander Kant, CEO, Zmanda
Date: Thursday, April 26, 2007
Time: 9:50am-10:20am
Location: Ballrooms E-H

Track: Security & Database Administration
Topic: MySQL Backup: Roadmap & Vision
Presenters: Paddy Sreenivasan, VP Engineering, Zmanda
Lars Thalmann, Replication & Clustering Technology, MySQL

Date: Wednesday, April 25, 2007
Time: 10:45am-11:45am
Location: Ballroom B

Track: Vendor Products & Services
Topic: Zmanda Recovery Manager (ZRM) for MySQL
Presenters: Ram “TK” Krishnamurthy, VP Services & Support, Zmanda
K.K. George, Director Engineering, Zmanda

Date: Wednesday, April 25, 2007
Time: 3:35pm-4:35pm
Location: Ballroom G

Birds Of A Feather

Topic: Backup and Recovery of MySQL
Track: Storage Engine Development and Optimization
Moderators: Paddy Sreenivasan, VP Engineering, Zmanda
Dmitri Joukovski, VP Product Management, Zmanda

Date: Tuesday, April 24
Time: 7:30pm - 9:00pm
Location: Lafayette

Backup vendors continue to ignore Vista and don’t care about Red Hat Enterprise Linux 5 either

Monday, April 16th, 2007

A couple of months ago I wrote about backup vendors not providing support for Microsoft Vista. Today I decided to follow up and check for any news regarding Vista support. Here are the updates:

Three out of six supporting the latest and greatest Windows almost half a year since it was released is actually not that bad. I find it really bizarre though, that BackupExec targeted at smallish Windows workgroups continues to ignore Vista.

While looking for info on Vista, I noticed that NONE of the mentioned vendors support Red Hat Enterprise Linux 5 (RHEL 5) released a month ago.

Interestingly, BakBone who is trying to position itself as Linux-friendly vendor issued a press release claiming full support for RHEL 5. I suspect that PR folks forgot to tell that engineering who usually owns the compatibility guide, because there is nothing about RHEL 5 there http://bakbone.com/support/product_documentation/retrieve_documentation.asp?product=NetVault_Supported_Platforms&page_type=support.

Amanda supports RHEL 5 since the day it was released. That is one of the many reasons why Amanda Enterprise has been selected for the upcoming Red Hat Exchange where customers can purchase everything they need to build and run the LAMP applications.

Dmitri Joukovski

Zmanda Webinar: Backup and Recovery of MySQL Clusters

Tuesday, April 10th, 2007

Event Information

Topic:
Backup and Recovery of MySQL Clusters

Date and Time:
Thursday, April 12th , 2007 10:00 am
Pacific Daylight Time (GMT -07:00, San Francisco)

Panelist(s) Info:
Shailen Patel - Professional Services Engineer,
Paddy Sreenivasan - VP, Engineering,
Ram “TK” Krishnamurthy - VP, Services and Support

Duration: 1 hour

Description:
Zmanda Recovery Manager [ZRM] for MySQL provides a simple and easy way to backup and recover MySQL clusters. Zmanda Engineers will provide an overview of ZRM for MySQL and demonstrate its ability to backup and restore MySQL cluster.

Open standard wins again. This time in tape drive technology for backup and archiving.

Wednesday, April 4th, 2007

In the war of “industry standard” vs. proprietary technology the Digital Equipment Corporation (DEC), or rather its legacy, has lost another battle. This time at stake was a tape format for backup and archiving. The proprietary DLT (Digital Linear Tape) format, originally created by DEC and later purchased by Quantum, has lost to the LTO (Linear Tape Open) technology developed by HP, IBM and Seagate. Interestingly, after Seagate spun off the tape business via Certance, Quantum decided to buy it and became part of LTO consortium itself.

The CEO of Quantum Rick Belluzzo said in an interview with Bryan Betts from The Register that DLT would “slide out of the way” and “there has been massive consolidation onto LTO”. The phase out of DLT technology will occur as quickly as in two years.

I couldn’t find any information about End of Life for DLT at www.quantum.com so I got in touch with Bryan who told me that he interviewed Belluzzo at CeBIT just a couple of weeks ago. Bryan also mentioned that the Wikipedia page on DLT claims that the decision to end development was taken in February.

The end of DLT technology is important news for tens of thousands of backup administrators who write exabytes of data to DLT tape drives. Just recently we conducted survey of Amanda users and almost 30% reported using DLT or SDLT tape drives. For regulatory compliance many organizations are required to keep tapes for 7-10 years or even longer. With Quantum dropping support for DLT, recovering all those DLT and SDLT tapes 10 years from now could be a challenge.

Amanda works well with all tape drive technologies, but often customers ask for our tape recommendations. We always advise to consider the following:

  • Reliability
  • Transfer rate
  • Capacity
  • Cost

From now on, we will add one more factor to consider — probability of tape drive vendor killing its own technology.
——-
Dmitri Joukovski

Open Source “Unconference” in India

Sunday, April 1st, 2007

On Saturday, 24th March ‘07, Open Source “Unconference” was held at Mumbai, India.

It was great learning experience for everyone and a good event to network with MySQL users and developers in India. K. K. George, the project leader for open source Zmanda Recovery Manager for MySQL, gave a talk about MySQL backup and recovery. Zmanda Recovery Manager is Zmanda’s MySQL backup and recovery project that has become quite popular with MySQL administrators. It has scheduling, reporting and monitoring functionalities for MySQL databases. In addition, it can do incremental backups and can also parse the complex binary log files of MySQL dump. Remote MySQL server backup is also supported through SSH and socket options. There was lot of participation and interest from the audience and we got more ideas to enhance the project feature set. India is increasingly becoming a hot market for MySQL and web portals, so this product stands a chance to make it big over here.

Another interesting session was MySQL database tuning. The talk placed emphasis on the importance of using techniques like normalization, datatype selection, query logs to achieve space and performance enhancement.

I was happy to see a community “Open Source Unconference” in India and hope to actively participate in more. Of course, Zmanda is in look out for good engineers to join our MySQL team in Pune and Sunnyvale. If you are interested, please see Zmanda jobs page.