Create EBS Boot AMI

Great guide!
This got it working for me.
Made a standard ubuntu intrepid ami based on ebs

Hi, nice update, you might want to update your link to the instance-to-ebs-ami.sh script, http://gist.github.com/249915?file=instance-to-ebs-ami.sh There have been a few updates to the script. The link you have is to a specific, earlier version

Hi Michael,

Thanks for pointing to updated version and of course thank you for your Create EBS Boot AMI scripts.

Youri,

Great that it work for you. As pointed by Michael, I’ve updated the new instance-to-ebs-ami.sh.

Thanks for a clear and concise writeup. My question is – what does one do when there is no identified ARI associated with a particular AMI that we are using?

Rodney, is this correct? It seems you are downloading bundle-to-ebs.sh but saving as /tmp/instance-to-ebs-ami.sh ? Should you instead download instance-to-ebs-ami.sh?
Cheers

Hi Radek,

Thanks for pointing that out. Seems my update changed the link. Fixed.

Rodney

Hey Rodney, thanks a lot! I managed to create the EBS bootable image thanks to your post! I based it on offical canonical ubuntu-hardy-9.10-i386. One thing which is kind of weird is that whenever I terminate the instance the EBS volume attached to that instance is deleted as well… Do you have any idea why this may be happening?

Hi Michael,

You might be using a kernel that does not require ramdisk.
See this post for reference:
Thread: What is the point of the ArI images?
2.6.18 Xen 3.1.0 kernel

Radek,

On EBS AMI, if you terminate the instance it will delete the volume. If you want to make a point in time copy of your EBS AMI, you can stop it using AWS Management console and use Create Image(EBS AMI). Or just create a new snapshot of the volume then you can register it using the command line tools.

Rodney

As posted here by Ton:
http://groups.google.com/group/ec2ubuntu/browse_thread/thread/11b9c0ff3e477a53

Hi Rodney,

After a small change Michael’s scripts worked fine for me in eu-west
zone. The ec2- commands in the script did not have the –region
directive set. In that case they default to ‘us-east-1′, so the script
works out-of-the-box for all people in that region, but in other
regions like ‘eu-west-1′ and ‘us-west-1′ the –region directive must
be explicitly set to that region.

Best regards, Ton

Radek,

Please check the deleteOnTerminate attribute when you run the instance. This will not delete the volumes as describe on the BootFromEBSGuide manual.

Michael,

I’ve built multiple EBS based AMI’s all working great!
But i think I’ve found some minor security issues.
The script doesn’t exclude the authorization_keys found in .ssh which get created on instance initialization.
Also, the file ec2-init-user-data gets chmodded.
But the same file could be named ec2-init like i found on other distributions, so you should check it manually after mounting the volume.

Hope this helps a bit

When I run the following command (filling in my info where appropriate) I just get the help menu for the command. Does anyone see anything wrong? I’m not the most advanced command line person but everything else seemed to go fine.

ec2-register –snapshot snap-c070c5a9 –kernel aki-92ba58fb –ramdisk ari-94ba58fd –description=”EBS Ubuntu Hardy” –name=”–name=”ebs-ubuntu-hardy-8.04-i386-server-20091130″” –architecture i386 –root-device-name /dev/sda1

Ron,

It seems your command contains “-name” twice. Please recheck and let us know how it goes.

Good luck.

Ron – also it could be that you’re running the old version of the ec2 tools without support for boot from EBS.

I’ve tried this over and over again, I’ve got my volume and snapshot dialed, i can’t register the ami from it, whenever I run

ec2-register –snapshot snap-98bb4cf1 –kernel aki-7e0d250a –ramdisk ari-7d0d2509 –description=”my image” –name=”gentoo-stage3″ –architecture i386 –root-device-name /dev/sda1 –region eu-west-1

I get ec2reg/ec2-register usage information, no debugging info, no sign of success or reason of failure… it simply won’t work…

I’ve tried everything, including:

ec2-cmd RegisterImage -n my_image -s snap-98bb4cf1

Thanks for your response. So I took out the double name and ran this. I’ve looked at a lot of examples and tried some of them but…same result. This makes sense anyway. All of the other commands went well so I think my environment is good. Do I need an s3 bucket for this? Do I need to include a manifest? I’ve included the response output. Thanks for your help.

ec2-register –snapshot snap-6db3f004 –kernel aki-5f15f636 –ramdisk ari-0915f660 –description=”ebs based ami” –name=”abc-ami-ebs1″ –architecture i386 –root-device-name /dev/sda1 –region us-east-1d
SYNOPSIS
ec2reg (ec2-register)
ec2reg [GENERAL OPTIONS] MANIFEST
GENERAL NOTES
Any command option/parameter may be passed a value of ‘-’ to indicate
that values for that option should be read from stdin.
DESCRIPTION
Register an AMI manifest for use with EC2.
The MANIFEST parameter is the manifest (in S3) to register.
e.g. mybucketname/image.manifest

GENERAL OPTIONS

-K, –private-key KEY
Specify KEY as the private key to use. Defaults to the value of the
EC2_PRIVATE_KEY environment variable (if set). Overrides the default.

-C, –cert CERT
Specify CERT as the X509 certificate to use. Defaults to the value
of the EC2_CERT environment variable (if set). Overrides the default.

-U, –url URL
Specify URL as the web service URL to use. Defaults to the value of
‘https://ec2.amazonaws.com’ or to that of the EC2_URL environment
variable (if set). Overrides the default.

–region REGION
Specify REGION as the web service region to use.
This option will override the URL specified by the “-U URL” option and EC2_URL environment variable.

-v, –verbose
Verbose output.

-?, –help
Display this help.

-H, –headers
Display column headers.

–debug
Display additional debugging information.

–show-empty-fields
Indicate empty fields.

–connection-timeout TIMEOUT
Specify a connection timeout TIMEOUT (in seconds).

–request-timeout TIMEOUT
Specify a request timeout TIMEOUT (in seconds).

Rares,

Please run the command to check the ec2 tools version you have on your machines. It might be older version without support to EBS boot images as Mark stated above.

You can download the updated API tools here http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

Hey you may be right.
When I run: ec2-version
I get: 1.3-36506 2009-04-04

I downloaded the latest (46266) and put it in /usr/local/ec2-api-tools-1.3-46266. I made that its home.If I run:
$EC2_HOME/bin/ec2-version

I get: 1.3-46266 2009-11-30

As I am just moving to Linux and trying to remember Unix commands from yesteryear I may be doing this wrong. Did I not install the latest ec2 api tools properly? Thanks…

All I needed was a nudge in the right direction. Worked great without the user malfunctions. Thanks….

Great article, helped plenty in moving images to EBS.

Some gotcha’s I faced: firstly the links to the script seemed to miss the instance-to-ebs (only grabbed the bundle-to-ebs). The rsync should also have –exclude=”exclude” with the inverts, otherwise get a nice mess in the target. also exclude “/sys/*”

Before the ec2-register phase, you need to wait for the snapshot to complete. You can hang around and watch, or stick it in:
while ! ec2-describe-snapshots $SNAP | grep -q completed; do
echo “Snapshot is `ec2-describe-snapshots $SNAP | awk ‘{print $6}’` complete”
done

Hi Brett,

Thanks for pointing that out.:)

Thanks for the quick rundown! Worked like a charm. Just wanted to clarify a point made above, which is that there are two typos in this line:

ec2-register –snapshot snap-c070c5a9 –kernel aki-92ba58fb –ramdisk ari-94ba58fd –description=”EBS Ubuntu Hardy” –name=”–name=”ebs-ubuntu-hardy-8.04-i386-server-20091130″” –architecture i386 –root-device-name /dev/sda1

The first is the ‘–name=”–name’ thing, and the second is that for –name and –description, you shouldn’t use equal signs, I don’t think. Anyway, thank you!

Thanks Jonah. Fixed.

[...] to create an EBS image from an existing AMI By cynikh A very good post on how to create an EBS Image from an existing AMI with inputs from developer forums. The rsync method worked for me than the dd method. For the DD [...]

Ah, thanks for catching that jonah

We wrote a tool that does the instance-store/ebs-boot conversion from a web-service: https://cloudyscripts.com/tool/show/2
Might make things easier for you…

[...] Create EBS Boot AMI – 私はこれが好き [...]

well i tried cloudyscripts, plus several other sites then came back to this one. Eventually got my head around linux and managed to get it working. thanks!

here are my notes:

# Setup new repo, update and install latest e2c tools:

sudo apt-get update && sudo apt-get install python-software-properties

sudo add-apt-repository ppa:ubuntu-on-ec2/ec2-tools

sudo apt-get update && sudo apt-get install ec2-api-tools ec2-ami-tools

# Next up, we’ll set some paths in bash profile so the OS knows where the EC2 tools are located.

sudo vim ~/.bash_profile

# Add these lines, replacing “YOURKEYNAME” with the actual file name of your private and public keys, then save.

export EC2_HOME=~/.ec2
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=/home/ubuntu/pk-“YOURKEYNAME”.pem
export EC2_CERT=/home/ubuntu/cert-“YOURKEYNAME”.pem
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/

# To get the changes noticed by the OS immediately, run source.

source ~/.bash_profile

# To test the tools, run the following from a new terminal:

ec2-describe-images -o self -o amazon

# You should receive a list of all the publicly available EC2 images published by Amazon.
# Now to register as new EBS Boot AMI:

ec2-register –snapshot “YOURSNAPSHOT” –description “EBS TEST Ubuntu” –name “ebs-ubuntu-i386-server” –architecture i386 –root-device-name /dev/sda1

# It should return your new AMI number, go into AWS web console to take a look and launch as new instance!

Oh and it took me ages to work out how to get the right certs (download the 2 X.509 certs cert-xxx…pem and pk-xxx…pem) from amazon accounts page and then even longer to figure out how to upload them to the server.

Eventually found some software called winscp which runs over ssh and worked like a treat for uploading the certs to the right place ready for the ec2-register command.

Sure would be a bit easier if the amazon folk integrated register function into the AWS mgmt web interface sigh.

For all the linux old hands out there this stuff must be a piece of cake – for us noobs its like pulling teeth.

Munts,

Thanks for visiting my site.
Glad to hear you got it working.

Nice tutorial. Its strange for me unable to do ssh. The instance created using EBS image is up, but unable to login.

cloudsuser,

Please check this link, this might help:

http://alestic.com/2009/08/ec2-connectivity

hey,
Do you have steps for converting existing instance based AMI to EBS based for Windows platform.

TIA,

Hi,

This was really useful.

I am trying to use a bootable windows ebs volume but I am not able to figure out how to point the boot ami to the windows ebs volume.

Any help is appreciated.

Thanks

FreeMind and narender:

This script dont work with Windows AMI.
You need to use the existing EBS boot Windows AMI and bundle it from the AWS management console or Elasticfox.

Nice one. You should add a note that after cooking up AMI you can remove volume but you need to keep a snapshot you used.

Hai I have created one EBS boot instance from existing instance with reference to your article.But I cannot do ssh to my newly created instance.I don’t no what is the problem.Can you tell me what will be the problem.

Hi Ratheeshekm,

Please follow the steps to check your EC2 connectivity using this link: http://alestic.com/2009/08/ec2-connectivity

one needs to add the option –block-device-mapping /dev/sda2=ephemeral0 to the ec2-register command as of sept 2010 otherwise it hangs on boot and fails to setup the ssh keys

Hi Andy (and all others of course)!

You said: “…otherwise it [...] fails to setup the ssh keys”. What is the exact result of this issue?

My freshly created EBS-backed AMI (Debian 5.0 lenny) starts and keeps running but the SSH server is not started. The system log (accessed via AWS Console) shows the following problem: “Starting OpenBSD Secure Shell server: sshd failed – PRNG is not seeded”.

I know, this also happens when there’s a problem with /dev/urandom or missing host keys but all looks fine. Any thoughts?

(me again)…

I figured out that this issue is associated with the kernel/ramdisk you chose. I could not find a kernel/ramdisk that works with my Debian image but *without* parameters –kernel and –ramdisk in command “ec2-register”, /dev/urandom can be addressed and the SSH server starts.

Downside: the instance uses Amazon’s default kernel version 2.6.16-xenU – pretty old.

I would be more than happy if someone has a suggestion how to address the kernel/ramdisk+SSH issue.

Michael,
Hmm.. why not use, Canonical Ubuntu AMI with newer kernels?

rodney,
Yes, great alternative… but in this case our client had very specific requirements. Their policies dictated to set up an instance from scratch (install a fresh Debian system via bootstrap), not using an existing AMI for security reasons (“don’t trust anybody”) and the instance needed to be Debian lenny

So we bundled our own image, uploaded it to Amazon and run it as an AMI-instance. Afterwards my plan was to convert it to an EBS instance but I struggle at the kernel/ramdisk. Apart of this, the AMI-to-EBS conversation works pretty good (thanks to the infos and links on this page!).

Great tutorial. After going through all the steps, I get an error when I launch a new instance using the ami produced and try to ssh into it:

ssh: connect to host ec2-XXXXXXX.us-west-1.compute.amazonaws.com port 22: Connection refused

All status messages appear fine for the instance, but no-go on shh. Any ideas? Either way, thanks a lot!

Actually, might have a solution: delete the /dev/sdb line from /etc/fstab before creating new ami, as in:
https://forums.aws.amazon.com/thread.jspa?threadID=46483&tstart=0#178690

Bob,

I assume security groups for your instance is setup correctly i.e open port 22.

Try to check for the console output for errors. Have you specified the correct ramdisk/kernel for your new AMI?

What AMI are you trying to bundle? Distro and version.

This link can help you: http://alestic.com/2009/08/ec2-connectivity

Hmm.. A possible for Lucid AMI.

Assuming you already have the rsync’ed EBS volume, just mount the volume to any instance, edit the /etc/fstab and snapshot.

Then register it as a new AMI.

Hey Bob!
As rodney suggested: check security groups and the output of the boot process (system log at AWS console). Maybe you have the same issue I had (unable to start SSH server), see my post above (Michael on 20 September 2010, 9:17 pm).

@Bob:
We’re you able to get your AMI running with SSH?

@Michael:
Was the Debian image working good?

Cheers!

@rodney: sure, the Debian image worked perfectly fine. The issue was (and still is) the kernel, as I wrote in my post on 27 September 2010, 9:32 pm (see above). The solution was to use Amazon’s default kernel version 2.6.16-xenU. With this kernel /dev/urandom works and therefore the SSH server can be started during boot process.

In short: following this guide (for Ubuntu), you can easily set up a Debian lenny image as an EBS Boot AMI, too.

I used your method to create a Debian Squeeze amd64 ami, and I can only launc a micro, small and high-cpu instance. Any idea why that could be happening?

Nvm, I did not specify the architecture, while registering the ami. Also I cant ssh into the instance, I guess the answer lies in one of the comments.
Thanks for the awesome guide Keep up the good work!

Hi Kaustubh,

Glad it work for you.

Hi Rodney,

Thanks for the tutorial. The whole process works for me almost without errors.

However, when I try to connect the instance using the public address, I got timeout error. I also tried to ping it but failed.

Please advise. Thanks.

Hi Leo,

Did you check the security groups?
If you can paste on http://pastie.org/ the console logs, that would be helpful also.