Amazon EC2 and S3 my Invasion

For the past 3 months, I was playing with sample AMI’s for testing purposes. I tried to build an RHEL image from my VMWare and try to bundle it for AWS. So far so good. But after loading and registering it to AWS, and run an instance of it.

My problems encountered:

1. On the first case, I tried to upgrade the ruby version to1.8.5 in my host machine(getting rid of ruby errors)
2. Time not sync with Amazon.. need to run NTP on the host machine before uploading the image.
3. EXT2-fs: unable to read superblock and Kernel Panic
a. Workaround for this one is to make sure your /etc/fstab (ex. /mnt is mounted /dev/sda2)
b. And to copy the 2.6.16-xenU /lib/modules

After this, I started to customize the running AMI at the amazon at and install software as needed. Rebundle and upload it back to Amazon S3.

Rather than having the ready made AMI to work with, I built our own AMI, the learning process is what makes you feel confident, but experience ec2 guys would say “it best to use the existing AMI”. I wont argue, but naturally now,I can build more stuff from using existing AMIs the faster way (confident ec2 user’s way).

Soon, I discoverted the Rightscale and the Firefox addon to manage my instance. But still I can’t do away the command line utilities.

Data Persistent

To make our data persistent is s3sync which is open source. I like this better. I’m currently writing a simple script that will download the data from s3 to ec2 instance upon boot, and upload it back before shutdown. On special cases, if the instance would not crash this is solution will definitely great, but in some cases where my instance crashes, I will be losing some data. The next step, is to use back using s3sync using cron to update the s3 maybe hourly or daily, a script would do this for me.. 🙂

It came to a point where I need a lots of dose of caffiene to get the persistency problem from EC2 to S3. I have tested the ElasticDrive, though there’s a 40Gb limit for free use, thinking in the future if your data gets bigger than that, there extra charges. Reading the forum for ElasticDrive support gets you into S3 as fuse base filesystem. Mounting the S3 buckets as virtual drive. I need to test this further though.

Until next time.

Rodney

Leave a Reply

Your email address will not be published. Required fields are marked *