Tag Archives: ebs

Deleting Old EBS Snapshots

This is a plain and simple way to manage old mysql EBS snapshots. Maintain the latest five recent snapshots of a specific volume. You don’t want to mess with other existing volume snapshots, right? 🙂

1. Install Eric Hammond’s ec2-consistent snapshot

codename=$(lsb_release -cs)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu $codename main" |
sudo tee /etc/apt/sources.list.d/alestic-ppa.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
sudo apt-get update
sudo apt-get install -y ec2-consistent-snapshot
sudo PERL_MM_USE_DEFAULT=1 cpan Net::Amazon::EC2

2. Copy the lines below to a file $HOME/.my.cnf, (make sure to change file permission to read only using chmod 600)

   [client]
   user=root
   password=MYPASSWORD

3. Just copy your AWS access and secret key to $HOME/.awssecret separated by a linefeed.(make sure to change file permission to read only using chmod 600)

  aws_accesskey
  aws_secretkey

4. Download Tim Kay’s AWS tool and make the file executable.

    sudo wget https://github.com/timkay/aws/raw/master/aws -O   /usr/local/sbin/aws
    sudo chmod +x /usr/local/sbin/aws
 

Updated 12/13-2010: URL for Tim Kay’s aws program.

5. Setup a cronjob. Adjust cronjob to your backup schedules. If you want to maintain 10 snapshots, replace the 5 at sed command. The one-liner code was posted here.

# to snapshot your mysql
0 0 * * * /usr/bin/ec2-consistent-snapshot --mysql --xfs-filesystem /vol vol-XXXXXX >> /mnt/backup.log  2>&1
# Delete the last 5 old snapshots.
0 2 * * * /usr/local/sbin/aws dsnap | grep vol-XXXXXXX | sort -r -k 5  | sed 1,5d | awk '{print "Deleting snapshot: " $2 " Dated:" $8}; system("/usr/local/sbin/aws delsnap " $2 )'  >>  /mnt/backup.log  2>&1

Code Updated fixing redirection.

I’m using Eric Hammond’s Ubuntu Jaunty: ami-ed46a784 for this setup. Note that this ami-ed46a784 is using an older kernel with bug that pose possible access to root account in a multiuser environment.

New Releases of Ubuntu Images for Amazon EC2 (Kernels, Security, Tools, PPA, runurl)
New 2.6.21 kernel+modules: XFS breaks on Ubuntu Hardy, Intrepid

Testing Bootable EBS with different AKI and ARI

The release of AWS share EBS snapshots
How to Boot EC2 Instances from EBS
Cool Things You can do with Shared EBS
Hidden Dangers in Creating Public EBS Snapshots on EC2
Making an AMI use EBS to load the system files

Using the references above I tried to test Bootable EBS volumes, I’ve made some experiments (duh) to test bootable EBS volumes base on Alestic AMIs:
Ubuntu Hardy(ami-e546a78c)
Debian Lenny AMI(ami-ff46a796)

Note that both of these AMI’s are using the same AKI (aki-a71cf9ce) and ARI (ari-a51cf9cc). Therefore the two boot EBS volumes created has this properties. To create the Lenny and Hardy bootable EBS volumes, I simply follow Shlomo’s How to Boot EC2 Instances from EBS tutorial.

The first boot AMI is based on Alestic Ubuntu Hardy(ami-e546a78c) where the device used was set to /dev/sdj instead of /dev/sdp in Shlomo’s tutorial.

It was able to boot and I can successfully connect using SSH to these two instances without problems.

Here are the output of using two different EBS volumes:

Debian Lenny bootable EBS Output using boot AMI based on Alestic Hardy(ami-e546a78c)

EXT3 FS on sda1, internal journal
Making pivot_root device node /new-root-dev
Attempting to mount pivot_root volume on /new-root-dev
1 scsi_mod: no version for "struct_module" found: kernel tainted.
SCSI subsystem initialized
register_blkdev: cannot get major 8 for sd
mount: /new-root-dev is not a valid block device
2 mount: /new-root-dev is not a valid block device
3 mount: /new-root-dev is not a valid block device
4 mount: /new-root-dev is not a valid block device
5 mount: /new-root-dev is not a valid block device
6 mount: /new-root-dev is not a valid block device
7 mount: /new-root-dev is not a valid block device
8 mount: /new-root-dev is not a valid block device
9 mount: /new-root-dev is not a valid block device
10 mount: /new-root-dev is not a valid block device
 sdj: unknown partition table
11 kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Mounted pivot_root volume.
Mount failed for selinuxfs on /selinux:  No such file or directory
INIT: version 2.86 booting
/proc is empty (not mounted ?)
hostname: the specified hostname is invalid
Starting the hotplug events dispatcher: udevd.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...input: PC Speaker as /class/input/input1
done.

Continue reading