Tag Archives: ebs

MySQL on RAID0 and useful MySQL tools

Maintaining a production DB containing 15-25M rows show best improvement with the use of RAID0 on EC2 platform. Having a MySQL master-slave replication of servers and keeping the documented procedure is very handy for restoring the DB and the RAID volumes.

Documenting the process on downgrading and upgrading RAID0+MySQL from large to xlarge EC2 instance or vice versa helps a lot.

The links below best describes the RAID0 performance for IO operations on EC2 and how to setup the EBS RAID0 and MySQL in your machine.

RAID0 Refrences
IO Performance on EBS
EC2 EBS SINGLE AND-RAID VOLUMES IO BENCHMARK
EC2 EBS RAID
mdadm: A New Tool For Linux Software RAID Management
Running MySQL on Amazon EC2 with Elastic Block Store

MySQL Tools
EC2 Consistent Snapshot
Using Maatkit to restore Slave-Master
Maatkit.Org
MySQL Sandbox
Auto MySQL Backup

Create EBS Boot AMI

After the release of New Amazon EC2 Feature: Boot from Elastic Block Store , there are folks asking on AWS forum on how to create an EBS Boot AMI and so I’m posting another way to do it. I hope this will be plain and simple to follow. 😉

Note that I’m using Create EBS Boot AMI scripts courtesy of fairchild.

To be able to register EBS Boot AMI, make sure you install the latest EC2 API Tools, you can check that using:

prompt> ec2-version
1.3-45772 2009-10-31

And be sure you set the the EC2 private and cert:

prompt> export EC2_PRIVATE_KEY=/home/tim/ec2-keys/pk.pem
prompt> export EC2_CERT=/home/tim/ec2-keys/cert.pem

I will be using the Canonical Hardy AMI: ami-7e28ca17
The first thing is to get the AKI and ARI of the AMI using ec2-describe-images command or using ElasticFox.

prompt> ec2-describe-images ami-7e28ca17
IMAGE	ami-7e28ca17	ubuntu-images-us/ubuntu-hardy-8.04-i386-server-20091130.manifest.xml	099720109477	available	public		i386	machine	aki-92ba58fb	ari-94ba58fd		instance-store

Here are the AKI: aki-92ba58fb and ARI: ari-94ba58fd information.

Launch the AMI and set the keypair and availability zone:

prompt> ec2-run-instances ami-7e28ca17 -k yourkeypair -z us-east-1a
RESERVATION    r-237fed4a    853279305000    default
INSTANCE  i-d9add0b0  ami-7e28ca17  pending   0  m1.small  2009-11-14T12:38:24+0000  us-east-1a  aki-92ba58fb  ari-94ba58fd  monitoring-disabled

Create the volume and attached it to the instance. Make sure the volume has the same availability zone with the current running instance.

prompt> ec2-create-volume  --size 20 --availability-zone us-east-1a
VOLUME vol-4d826724 20 us-east-1a creating 2009-11-14T12:39:24+0000

prompt> ec2-attach-volume vol-4d826724 -i i-d9add0b0 -d /dev/sdh
ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh attaching 2009-11-14T12:43:24+0000

Make sure the volume has status “attached” before doing the next step.

Now we need to get a copy of instance-to-ebs-ami.sh script and execute it inside the running instance.

prompt> ssh -k yourkeypair ubuntu@public-dns-of-my-instance

prompt> wget https://raw.github.com/gist/1018285/cee984da6acda43c3063cf6d9e8c564d37812428/instance-to-ebs-ami.sh

prompt> chmod +x /tmp/instance-to-ebs-ami.sh
prompt> sudo /tmp/instance-to-ebs-ami.sh

(...output truncated...assuming the script was executed successfuly)
prompt> exit

Now we are back to our machine’s xterm, let’s detach the volume, make snapshot and register as new EBS Boot AMI:

prompt> ec2-detach-volume vol-4d826724
ATTACHMENT vol-4d826724 i-6058a509 /dev/sdh detaching 2009-11-15T00:55:24+0000

prompt> ec2-create-snapshot vol-4d826724 --description "EBS Ubuntu Hardy"
SNAPSHOT  snap-c070c5a9  vol-4d826724  pending 2009-11-15T1:31:29+0000  111111111111  1  EBS Ubuntu Hardy

prompt> ec2-register --snapshot snap-c070c5a9 --kernel aki-92ba58fb \
--ramdisk ari-94ba58fd --description "EBS Ubuntu Hardy" \ 
--name "ebs-ubuntu-hardy-8.04-i386-server-20091130" \
-–block-device-mapping /dev/sda2=ephemeral0 \
--architecture i386 --root-device-name /dev/sda1

Yes of course you can make use the scripts provided on the github link above. But I’m sure now you get the idea. Now start to convert the remaining non-EBS AMI you may have. 😉

By default EBS Boot AMI does not attached the ephemeral storage. See these links for reference:
Attached Ephemeral Storage on EBS Boot AMI
Ephemeral Storage Capacity

There’s also a question of “Can you resize the EBS boot AMI?”. I believe, yes. Will give it a try and post the results here.

Update:
I have posted before how to increase or enlarge the resize EBS volumes.
And about my question of resizing the EBS AMI, I found Eric Hammond’s post which is very helpful:Increasing Root Disk Size of an “EBS Boot” AMI on EC2

Update 01-28-2010:
If your looking for Public EBS Boot AMIs for Ubuntu on Amazon EC2, Eric Hammond have provided them in his site. Thanks Eric.

Update 06-10-2011:
Add label to mkfs.ext3
Add -–block-device-mapping /dev/sda2=ephemeral0 for i386 or –block-device-mapping /dev/sdb=ephemeral0 for x86_64 arch.

References:
http://coderslike.us/2009/12/07/amazon-ec2-boot-from-ebs-and-ami-conversion/
AWS EC2 Thread: Booting from EBS Image Creation