Category Archives: ec2

Another Fabric Tutorial

Leave a reply

I’ve been playing with  Python here and at the same time Fabric v.0.9 for simple deployment. This is a handy tool to manage several servers for application development,testing and production.

The code below is save to a file named fabfile.py

# Works with fabric v0.9
#
from fabric.api import run, env
from fabric.operations import local,put

def dev_server():
env.user = 'user_name'
env.hosts = ['testserver.domain.com']

def staging_server():
env.user = 'user_name'
env.hosts = ['staging.domain.com']

def production_server():
env.user = 'user_name'
env.hosts = [' prod1.domain.com', 'prod2.domain.com' ]

# define needed functions here.
def host_info():
print 'Checking lsb_release of host: ',  env.host
run('lsb_release -a')

def uptime():
run('uptime')

def simple_deploy():
put ('/tmp/testfile','/tmp')
#
# add deployment codes here

This assumes we can login to any of the server w/o ssh keypair.
See this link for setting up SSH without password.

Tip: Also if you have different keypair for each servers(dev,staging,production), you can add keypair using the command ssh-add. Continue reading →

Mount EBS Volume On Startup

3 Replies

“There is more than one way to kill a cat”, though I haven’t killed a cat since birth. And so, there’s another way of mounting EBS volume from boot startup. It’s possible to replace the default volume ID by passing user-data script when launching the instance.

Requirements:
AWS accesskey and secret keys
Tim Kay’s aws tool.

Tested using Ubuntu 9.10 Karmic AMI: ami-1515f67c

How to install the script:

1. Copy the script to at /etc/init.d/mount_ebs_override
2. Replace the default volume vol-xxxxxx in the code.
3. Change the script to have executable permission.
4. Copy access and secret key to /root/.awssecret separated by a newline.
5. Change file mode of /root/.awssecret to 600
6. Run update-rc.d: update-rc.d mount_ebs_override defaults 89
7. Bundle the AMI. 🙂

When the new AMI is launch without value on the user-data, it will mount the default volume ID provided on the script. In order to use a different EBS volume, add the OVERRIDE_EBS_VOLUME=volume-id as value for the user-data and launch the instance.

Note: Just make sure the instance is launch on the same zone of the EBS volume.

#! /bin/bash -ex
#
# description: Assigns an EC2 EBS Volume to a device and mounts the device
# OVERRIDE_EBS_VOLUME variable can be read at user-data to replace the default volume-id.
#
VOL=vol-xxxxxx
DEV=/dev/sdh
MOUNT_POINT=/ebs1

# replaced by Tim Kay's aws tool. must be at /usr/sbin/ and $HOME/.awssecret present.

prog=$(basename $0)
logger="logger -st $prog"
curl="curl --retry 3 --silent --show-error --fail"

MAX_TRIES=120

if [ ! -f /usr/sbin/aws ]; then
   $curl http://timkay.com/aws/aws -o /usr/sbin/aws
   chmod +x /usr/sbin/aws
fi

## REPLACE THIS WITH AWS KEYS.. or copy .awssecret to $HOME
#echo "accesskey" > /root/.awssecret
#echo "secretkey" >> /root/.awssecret
#chmod 600 /root/.awssecret

perl -MIO::Socket::INET -e '
until(new IO::Socket::INET("169.254.169.254:80")){print "Waiting for network...\n" ;sleep 1}
' | $logger

INSTANCE=$($curl http://169.254.169.254/latest/meta-data/instance-id)

CTR=1
TIME_OUT=1
while [ ! -n "$INSTANCE" ]
   do
       if [ $CTR -eq 7 ]; then
           $logger "WARNING: Failed to retrive instance meta-data after `expr $TIME_OUT \* 2` seconds";
           exit 1
      fi
    CTR=`expr $CTR + 1`
    sleep "$TIME_OUT"
    TIME_OUT=`expr $TIME_OUT \* 2`
    INSTANCE=$($curl http://169.254.169.254/latest/meta-data/instance-id)
done

$logger "Got instance id: $INSTANCE"

# start/stop functions for OS

case "$1" in

start)
    ISMOUNTED=$(df | grep $DEV | wc -l)
    if [ $ISMOUNTED -eq 1 ]; then
        /bin/echo"Device already mounted. Exiting.." | $logger
        exit 1
    fi

    CTR=0
    # check if EBS was overriden.
    OVERRIDE_EBS_VOLUME=$(wget -qO- http://169.254.169.254/latest/user-data | awk -F"=" '/^OVERRIDE_EBS_VOLUME=/ {print $2}')

    if [ -n "$OVERRIDE_EBS_VOLUME" ] ; then
        $logger "Using EBS Volume specified in user-data"
        VOL=$(OVERRIDE_EBS_VOLUME)
    fi
    /bin/echo "Mounting Elastic Block Store Volumes. $VOL"  | $logger

    AVAILABLE=`/usr/sbin/aws  dvol $VOL | grep available | wc -l`
    CTR=0
    while [ $AVAILABLE -ne 1 ] 
     do
        sleep 1
        CTR=`expr $CTR + 1`
        if [ $CTR -eq $MAX_TRIES ];  then
           $logger "WARNING: Failed to attach volume $VOL_ID after $MAX_TRIES attempts: volume not available"
           exit 1
        fi
     AVAILABLE=`/usr/sbin/aws  dvol $VOL | grep available | wc -l`
    done

    if [ -e "$DEV" ]; then
         $logger "Opps.. $DEV device already used on this system."
         exit  1
    fi
    /usr/sbin/aws attvol  $VOL -i $INSTANCE -d $DEV 2 >/dev/null | $logger
    while [ ! -e  "$DEV" ]; do
        /bin/sleep 1
        CTR=`expr $CTR + 1`
        if [ $CTR -eq $MAX_TRIES ]; then
            /bin/echo "WARNING: Cannot attach volume $VOL to $DEV -- Giving up after $MAX_TRIES attempts" | $logger
            /bin/echo "Check Zone of EBS and Instance is the same." | $logger
            exit 1
       fi
    done

    if [ ! -d $MOUNT_POINT ]; then
        mkdir $MOUNT_POINT
    fi

    # Updates: assuming the volume is using XFS filesystem.
    echo " $DEV/$MOUNT_POINT xfs noatime 0 0" | sudo tee -a /etc/fstab

    /bin/mount  $DEV $MOUNT_POINT | $logger

    ## add init scripts..launch if exits on EBS.
    if [ -x $MOUNT_POINT/init.files/init.sh ]; then
        $MOUNT_POINT/init.files/init.sh
    fi
   ;;

stop)
    /bin/echo "Unmounting Elastic Block Store Volumes."| $logger

    # only detach if umount is successful.
    /bin/umount $MOUNT_POINT
    /usr/sbin/aws detvol $VOL   2>/dev/null | $logger
   ;;
restart)
   stop
   sleep 5
   start
;;
*)
   echo "Usage: $0 {start|stop|restart}"
   exit 1
;;
esac

exit 0

Update:
Note that sometimes you need to reboot the instance and keep the EBS volume mounted, but this script will unmount and detach the volume. To disable unmounting and detaching of volume, the script can be configured using update.rc.d.

prompt> update-rc.d -f mount_ebs_override remove
prompt> update-rc.d mount_ebs_override start 89 2 3 4 5 .