Tag Archives: linux

Create EBS Boot AMI

57 Replies

After the release of New Amazon EC2 Feature: Boot from Elastic Block Store , there are folks asking on AWS forum on how to create an EBS Boot AMI and so I’m posting another way to do it. I hope this will be plain and simple to follow. 😉

Note that I’m using Create EBS Boot AMI scripts courtesy of fairchild.

To be able to register EBS Boot AMI, make sure you install the latest EC2 API Tools, you can check that using:

prompt> ec2-version
1.3-45772 2009-10-31

And be sure you set the the EC2 private and cert:

prompt> export EC2_PRIVATE_KEY=/home/tim/ec2-keys/pk.pem
prompt> export EC2_CERT=/home/tim/ec2-keys/cert.pem

I will be using the Canonical Hardy AMI: ami-7e28ca17
The first thing is to get the AKI and ARI of the AMI using ec2-describe-images command or using ElasticFox.

prompt> ec2-describe-images ami-7e28ca17
IMAGE	ami-7e28ca17	ubuntu-images-us/ubuntu-hardy-8.04-i386-server-20091130.manifest.xml	099720109477	available	public		i386	machine	aki-92ba58fb	ari-94ba58fd		instance-store

Here are the AKI: aki-92ba58fb and ARI: ari-94ba58fd information.

Launch the AMI and set the keypair and availability zone:

prompt> ec2-run-instances ami-7e28ca17 -k yourkeypair -z us-east-1a
RESERVATION    r-237fed4a    853279305000    default
INSTANCE  i-d9add0b0  ami-7e28ca17  pending   0  m1.small  2009-11-14T12:38:24+0000  us-east-1a  aki-92ba58fb  ari-94ba58fd  monitoring-disabled

Create the volume and attached it to the instance. Make sure the volume has the same availability zone with the current running instance.

prompt> ec2-create-volume  --size 20 --availability-zone us-east-1a
VOLUME vol-4d826724 20 us-east-1a creating 2009-11-14T12:39:24+0000

prompt> ec2-attach-volume vol-4d826724 -i i-d9add0b0 -d /dev/sdh
ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh attaching 2009-11-14T12:43:24+0000

Make sure the volume has status “attached” before doing the next step.

Now we need to get a copy of instance-to-ebs-ami.sh script and execute it inside the running instance.

prompt> ssh -k yourkeypair ubuntu@public-dns-of-my-instance

prompt> wget https://raw.github.com/gist/1018285/cee984da6acda43c3063cf6d9e8c564d37812428/instance-to-ebs-ami.sh

prompt> chmod +x /tmp/instance-to-ebs-ami.sh
prompt> sudo /tmp/instance-to-ebs-ami.sh

(...output truncated...assuming the script was executed successfuly)
prompt> exit

Now we are back to our machine’s xterm, let’s detach the volume, make snapshot and register as new EBS Boot AMI:

prompt> ec2-detach-volume vol-4d826724
ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh detaching 2009-11-15T00:55:24+0000

prompt> ec2-create-snapshot vol-4d826724 --description "EBS Ubuntu Hardy"
SNAPSHOT  snap-c070c5a9  vol-4d826724  pending 2009-11-15T1:31:29+0000  111111111111  1  EBS Ubuntu Hardy

prompt> ec2-register --snapshot snap-c070c5a9 --kernel aki-92ba58fb \
--ramdisk ari-94ba58fd --description "EBS Ubuntu Hardy" \ 
--name "ebs-ubuntu-hardy-8.04-i386-server-20091130" \
-–block-device-mapping /dev/sda2=ephemeral0 \
--architecture i386 --root-device-name /dev/sda1

Yes of course you can make use the scripts provided on the github link above. But I’m sure now you get the idea. Now start to convert the remaining non-EBS AMI you may have. 😉

By default EBS Boot AMI does not attached the ephemeral storage. See these links for reference:
Attached Ephemeral Storage on EBS Boot AMI
Ephemeral Storage Capacity

There’s also a question of “Can you resize the EBS boot AMI?”. I believe, yes. Will give it a try and post the results here.

Update:
I have posted before how to increase or enlarge the resize EBS volumes.
And about my question of resizing the EBS AMI, I found Eric Hammond’s post which is very helpful:Increasing Root Disk Size of an “EBS Boot” AMI on EC2

Update 01-28-2010:
If your looking for Public EBS Boot AMIs for Ubuntu on Amazon EC2, Eric Hammond have provided them in his site. Thanks Eric.

Update 06-10-2011:
Add label to mkfs.ext3
Add -–block-device-mapping /dev/sda2=ephemeral0 for i386 or –block-device-mapping /dev/sdb=ephemeral0 for x86_64 arch.

References:
http://coderslike.us/2009/12/07/amazon-ec2-boot-from-ebs-and-ami-conversion/
AWS EC2 Thread: Booting from EBS Image Creation

Deleting Old EBS Snapshots

7 Replies

This is a plain and simple way to manage old mysql EBS snapshots. Maintain the latest five recent snapshots of a specific volume. You don’t want to mess with other existing volume snapshots, right? 🙂

1. Install Eric Hammond’s ec2-consistent snapshot

codename=$(lsb_release -cs)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu $codename main" |
sudo tee /etc/apt/sources.list.d/alestic-ppa.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
sudo apt-get update
sudo apt-get install -y ec2-consistent-snapshot
sudo PERL_MM_USE_DEFAULT=1 cpan Net::Amazon::EC2

2. Copy the lines below to a file $HOME/.my.cnf, (make sure to change file permission to read only using chmod 600)

   [client]
   user=root
   password=MYPASSWORD

3. Just copy your AWS access and secret key to $HOME/.awssecret separated by a linefeed.(make sure to change file permission to read only using chmod 600)

  aws_accesskey
  aws_secretkey

4. Download Tim Kay’s AWS tool and make the file executable.

    sudo wget https://github.com/timkay/aws/raw/master/aws -O   /usr/local/sbin/aws
    sudo chmod +x /usr/local/sbin/aws

Updated 12/13-2010: URL for Tim Kay’s aws program.

5. Setup a cronjob. Adjust cronjob to your backup schedules. If you want to maintain 10 snapshots, replace the 5 at sed command. The one-liner code was posted here.

# to snapshot your mysql
0 0 * * * /usr/bin/ec2-consistent-snapshot --mysql --xfs-filesystem /vol vol-XXXXXX >> /mnt/backup.log  2>&1
# Delete the last 5 old snapshots.
0 2 * * * /usr/local/sbin/aws dsnap | grep vol-XXXXXXX | sort -r -k 5  | sed 1,5d | awk '{print "Deleting snapshot: " $2 " Dated:" $8}; system("/usr/local/sbin/aws delsnap " $2 )'  >>  /mnt/backup.log  2>&1

Code Updated fixing redirection.

I’m using Eric Hammond’s Ubuntu Jaunty: ami-ed46a784 for this setup. Note that this ami-ed46a784 is using an older kernel with bug that pose possible access to root account in a multiuser environment.

New Releases of Ubuntu Images for Amazon EC2 (Kernels, Security, Tools, PPA, runurl)
New 2.6.21 kernel+modules: XFS breaks on Ubuntu Hardy, Intrepid