Category Archives: aws

Deleting Old EBS Snapshots

7 Replies

This is a plain and simple way to manage old mysql EBS snapshots. Maintain the latest five recent snapshots of a specific volume. You don’t want to mess with other existing volume snapshots, right? 🙂

1. Install Eric Hammond’s ec2-consistent snapshot

codename=$(lsb_release -cs)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu $codename main" |
sudo tee /etc/apt/sources.list.d/alestic-ppa.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
sudo apt-get update
sudo apt-get install -y ec2-consistent-snapshot
sudo PERL_MM_USE_DEFAULT=1 cpan Net::Amazon::EC2

2. Copy the lines below to a file $HOME/.my.cnf, (make sure to change file permission to read only using chmod 600)

   [client]
   user=root
   password=MYPASSWORD

3. Just copy your AWS access and secret key to $HOME/.awssecret separated by a linefeed.(make sure to change file permission to read only using chmod 600)

  aws_accesskey
  aws_secretkey

4. Download Tim Kay’s AWS tool and make the file executable.

    sudo wget https://github.com/timkay/aws/raw/master/aws -O   /usr/local/sbin/aws
    sudo chmod +x /usr/local/sbin/aws

Updated 12/13-2010: URL for Tim Kay’s aws program.

5. Setup a cronjob. Adjust cronjob to your backup schedules. If you want to maintain 10 snapshots, replace the 5 at sed command. The one-liner code was posted here.

# to snapshot your mysql
0 0 * * * /usr/bin/ec2-consistent-snapshot --mysql --xfs-filesystem /vol vol-XXXXXX >> /mnt/backup.log  2>&1
# Delete the last 5 old snapshots.
0 2 * * * /usr/local/sbin/aws dsnap | grep vol-XXXXXXX | sort -r -k 5  | sed 1,5d | awk '{print "Deleting snapshot: " $2 " Dated:" $8}; system("/usr/local/sbin/aws delsnap " $2 )'  >>  /mnt/backup.log  2>&1

Code Updated fixing redirection.

I’m using Eric Hammond’s Ubuntu Jaunty: ami-ed46a784 for this setup. Note that this ami-ed46a784 is using an older kernel with bug that pose possible access to root account in a multiuser environment.

New Releases of Ubuntu Images for Amazon EC2 (Kernels, Security, Tools, PPA, runurl)
New 2.6.21 kernel+modules: XFS breaks on Ubuntu Hardy, Intrepid

Amazon EC2 AMI Cloner

Leave a reply

Since 2007, I’ve been architecting, implementing, and part of it is bundling EC2 images for my client’s projects. Some clients cannot provide they keys/certs for security reasons and so my usual option is to bundle the AMIs using my account, and once these images are tested or proven to meets my client’s requirements, I’d share those AMIs and let my clients do the bundling.

I was working with Informascale currently doing lots of fun stuff with Amazon AWS, Puppet, Linux, MySQL, Java and Groovy, Apache and Haproxy servers..obviously, most of these are related to scaling virtual servers, on demand applications, and cloud computing technology ideas.

Going back to this EC2 Image Cloner, the good thing about this tool is that you can run its CLI (command line interface) to bundle a shared AMI from AWS accountA to another AWS account(let’s say accountB). Or call it from a PHP application and run the cloning process in the background.

This cloner application needs to save AWS keys and certs of the AccountB to a database, and use this information to launch, bundle and upload the bundle S3.
Currently, it bundles all shared AMI from accountA to accountB. AccountA however needs to set Launch permission to any of its private AMI to be shared to accountB. Before bundling process exits, it tries to make sure to shutdown each of the launch instances being cloned. More features are underway for this tool, one is to be able to have a drag n drop option for setting launch permissions with any AWS account listed on the tool’s db 🙂 Hope there a lot more we can think of. I hope we can get this into a usable tool for everybody.

We had tested to run the cloner inside PHP app and was able to get ubuntu,debian,centos x386 AMIs successfully cloned. I’m about to test Canonical AMI images with this cloner, they have only two AMI’s for now, (checking how it handles x64 machines), so it safe for me to clone them all. 🙂

The cloner is written using Groovy 1.5.2-1, Typica 1.5, and Trilead SSH2 API.

To bundle AMI manually to another account, see these references:

http://entxtech.blogspot.com/2008/03/how-to-move-amazon-ami-ec2-image-from.html
http://austinmills.name/?p=26

Rough screenshot/codes of the EC2 Cloner in action: