Tag Archives: ebs

New Amazon EC2 Feature: Boot from Elastic Block Store

Amazon Announced the new Feature: Boot from Elastic Block Store

“You can now launch Amazon EC2 instances from an AMI backed by Amazon EBS (Elastic Block Store). This new functionality enables you to launch an instance with an Amazon EBS volume that serves as the root device.

This new feature brings a number of important performance and operational benefits and also enables some really powerful new features:

* Headroom – The root device (and hence the Amazon Machine Image or AMI) can now occupy up to 1 TB. You can now create more richly featured AMIs, installing additional tools, libraries, applications, and even reference data ahead of time.
* Flexibility – Each AMI can now reference one or more EBS snapshots. An EBS volume will be created from each snapshot and then attached to the new instance before it begins to boot. For instance, you could attach reference data such as a Public Data Set to each new instance.
* Performance – Instances will launch more quickly when they are booted from an EBS snapshot.
* Control – Instances can be stopped and then restarted at a later time. The contents of the volume are, of course, preserved while the instance is stopped, so you get the benefits of a persistent root device without being tied down to a particular piece of compute hardware.”

To test this feature, updated API tools is needed to be installed.

AWS Blog Announcement
Amazon API Tool to Boot from EBS
My previous post: Testing Bootable EBS with different AKI and ARI

Exciting feature to test and make use of it on production systems. 😉

Mount EBS Volume On Startup

“There is more than one way to kill a cat”, though I haven’t killed a cat since birth. And so, there’s another way of mounting EBS volume from boot startup. It’s possible to replace the default volume ID by passing user-data script when launching the instance.

Requirements:
AWS accesskey and secret keys
Tim Kay’s aws tool.

Tested using Ubuntu 9.10 Karmic AMI: ami-1515f67c

How to install the script:

1. Copy the script to at /etc/init.d/mount_ebs_override
2. Replace the default volume vol-xxxxxx in the code.
3. Change the script to have executable permission.
4. Copy access and secret key to /root/.awssecret separated by a newline.
5. Change file mode of /root/.awssecret to 600
6. Run update-rc.d: update-rc.d mount_ebs_override defaults 89
7. Bundle the AMI. 🙂

When the new AMI is launch without value on the user-data, it will mount the default volume ID provided on the script. In order to use a different EBS volume, add the OVERRIDE_EBS_VOLUME=volume-id as value for the user-data and launch the instance.

Note: Just make sure the instance is launch on the same zone of the EBS volume.

#! /bin/bash -ex
#
# description: Assigns an EC2 EBS Volume to a device and mounts the device
# OVERRIDE_EBS_VOLUME variable can be read at user-data to replace the default volume-id.
#
VOL=vol-xxxxxx
DEV=/dev/sdh
MOUNT_POINT=/ebs1

# replaced by Tim Kay's aws tool. must be at /usr/sbin/ and $HOME/.awssecret present.

prog=$(basename $0)
logger="logger -st $prog"
curl="curl --retry 3 --silent --show-error --fail"

MAX_TRIES=120

if [ ! -f /usr/sbin/aws ]; then
   $curl http://timkay.com/aws/aws -o /usr/sbin/aws
   chmod +x /usr/sbin/aws
fi

## REPLACE THIS WITH AWS KEYS.. or copy .awssecret to $HOME
#echo "accesskey" > /root/.awssecret
#echo "secretkey" >> /root/.awssecret
#chmod 600 /root/.awssecret

perl -MIO::Socket::INET -e '
until(new IO::Socket::INET("169.254.169.254:80")){print "Waiting for network...\n" ;sleep 1}
' | $logger

INSTANCE=$($curl http://169.254.169.254/latest/meta-data/instance-id)

CTR=1
TIME_OUT=1
while [ ! -n "$INSTANCE" ]
   do
       if [ $CTR -eq 7 ]; then
           $logger "WARNING: Failed to retrive instance meta-data after `expr $TIME_OUT \* 2` seconds";
           exit 1
      fi
    CTR=`expr $CTR + 1`
    sleep "$TIME_OUT"
    TIME_OUT=`expr $TIME_OUT \* 2`
    INSTANCE=$($curl http://169.254.169.254/latest/meta-data/instance-id)
done

$logger "Got instance id: $INSTANCE"

# start/stop functions for OS

case "$1" in

start)
    ISMOUNTED=$(df | grep $DEV | wc -l)
    if [ $ISMOUNTED -eq 1 ]; then
        /bin/echo"Device already mounted. Exiting.." | $logger
        exit 1
    fi

    CTR=0
    # check if EBS was overriden.
    OVERRIDE_EBS_VOLUME=$(wget -qO- http://169.254.169.254/latest/user-data | awk -F"=" '/^OVERRIDE_EBS_VOLUME=/ {print $2}')

    if [ -n "$OVERRIDE_EBS_VOLUME" ] ; then
        $logger "Using EBS Volume specified in user-data"
        VOL=$(OVERRIDE_EBS_VOLUME)
    fi
    /bin/echo "Mounting Elastic Block Store Volumes. $VOL"  | $logger

    AVAILABLE=`/usr/sbin/aws  dvol $VOL | grep available | wc -l`
    CTR=0
    while [ $AVAILABLE -ne 1 ] 
     do
        sleep 1
        CTR=`expr $CTR + 1`
        if [ $CTR -eq $MAX_TRIES ];  then
           $logger "WARNING: Failed to attach volume $VOL_ID after $MAX_TRIES attempts: volume not available"
           exit 1
        fi
     AVAILABLE=`/usr/sbin/aws  dvol $VOL | grep available | wc -l`
    done

    if [ -e "$DEV" ]; then
         $logger "Opps.. $DEV device already used on this system."
         exit  1
    fi
    /usr/sbin/aws attvol  $VOL -i $INSTANCE -d $DEV 2 >/dev/null | $logger
    while [ ! -e  "$DEV" ]; do
        /bin/sleep 1
        CTR=`expr $CTR + 1`
        if [ $CTR -eq $MAX_TRIES ]; then
            /bin/echo "WARNING: Cannot attach volume $VOL to $DEV -- Giving up after $MAX_TRIES attempts" | $logger
            /bin/echo "Check Zone of EBS and Instance is the same." | $logger
            exit 1
       fi
    done

    if [ ! -d $MOUNT_POINT ]; then
        mkdir $MOUNT_POINT
    fi

    # Updates: assuming the volume is using XFS filesystem.
    echo " $DEV/$MOUNT_POINT xfs noatime 0 0" | sudo tee -a /etc/fstab

    /bin/mount  $DEV $MOUNT_POINT | $logger

    ## add init scripts..launch if exits on EBS.
    if [ -x $MOUNT_POINT/init.files/init.sh ]; then
        $MOUNT_POINT/init.files/init.sh
    fi
   ;;

stop)
    /bin/echo "Unmounting Elastic Block Store Volumes."| $logger

    # only detach if umount is successful.
    /bin/umount $MOUNT_POINT
    /usr/sbin/aws detvol $VOL   2>/dev/null | $logger
   ;;
restart)
   stop
   sleep 5
   start
;;
*)
   echo "Usage: $0 {start|stop|restart}"
   exit 1
;;
esac

exit 0

Update:
Note that sometimes you need to reboot the instance and keep the EBS volume mounted, but this script will unmount and detach the volume. To disable unmounting and detaching of volume, the script can be configured using update.rc.d.

prompt> update-rc.d -f mount_ebs_override remove
prompt> update-rc.d mount_ebs_override start 89 2 3 4 5 .