I got this idea to test Eric Lubow’s ec2-consistent-snapshot With Mongo.
Using MongoDB 1.8.1 and Ubuntu Lucid AMI from www.alestic.com
On a running Ubuntu Lucid EC2 instance, connect using SSH and do the following:
1. Setup MongoDB. We stop it so we can move it’s files to the RAID volume.
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | \ sudo tee /etc/apt/sources.list.d/10gen.list sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 sudo apt-get update sudo apt-get -y install mongodb-10gen sudo /etc/init.d/mongodb stop
2. Create 4 EBS volumes on the same zone of the instance and attached the volumes using the devices from /dev/sdh to /dev/sdk
3. Setup RAID0 volumes for MongoDb.
sudo apt-get install -y mdadm xfsprogs sudo mdadm --create /dev/md0 --level 0 --chunk=256 \ --metadata=1.1 --raid-devices=2 /dev/sdh /dev/sdi /dev/sdj /dev/sdk echo DEVICE /dev/sdh /dev/sdi /dev/sdj /dev/sdk | sudo tee /etc/mdadm/mdadm.conf sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf sudo mkfs.xfs /dev/md0 echo "/dev/md0 /raiddrive xfs noatime 0 0" | sudo tee -a /etc/fstab sudo mkdir /raiddrive sudo mount /raiddrive sudo mv /var/lib/mongodb /raiddrive/ sudo mv /var/log/mongodb /raiddrive/mongodb-log sudo mkdir /var/lib/mongodb sudo mkdir /var/log/mongodb echo "/raiddrive/mongodb /var/lib/mongodb none bind" | sudo tee -a /etc/fstab echo "/raiddrive/mongodb-log /var/log/mongodb none bind" | \ sudo tee -a /etc/fstab sudo mount /var/lib/mongodb sudo mount /var/lib/mongodb sudo /etc/init.d/mongodb start
4. Install Alestic ec2-consistent-snapshot
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv BE09C571 echo 'deb http://ppa.launchpad.net/alestic/ppa/ubuntu lucid main' | \ sudo tee -a /etc/apt/sources.list.d/alestic-ppa-lucid.list sudo apt-get update sudo apt-get install -y ec2-consistent-snapshot
5. Use ec2-consistent-snapshot with MongoDB support which I modified a bit for Ubuntu. It’s on https://github.com/cocoy/ec2-consistent-snapshot.
sudo apt-get -y install build-essential libio-socket-ssl-perl libdatetime-perl sudo PERL_MM_USE_DEFAULT=1 cpan -fi MongoDB MongoDB::Admin
6. Copy this ec2-consistent-snapshot to /usr/bin/ replacing old file. And make it executable.
sudo wget -q https://github.com/cocoy/ec2-consistent-snapshot/raw/master/ec2-consistent-snapshot \ -O /usr/bin/ec2-consistent-snapshot sudo chmod +x /usr/bin/ec2-consistent-snapshot
7. Run the snapshot.
export AMAZON_ACCESS_KEY_ID=youraccesskeyid export AWS_SECRET_ACCESS_KEY=yoursecretkey sudo ec2-consistent-snapshot --debug \ --mongo \ --xfs-filesystem=/raiddrive/ \ --region us-east-1 \ --description "MongoDB RAID snapshot $(date +'%Y-%m-%d %H:%M:%S')" \ vol-xxxx vol-xxxxx vol-xxxxx vol-xxxxx
It should be resulting to something like:
ec2-consistent-snapshot: Using AWS access key: xxxxxxx ec2-consistent-snapshot: Wed Jun 8 09:11:56 2011: mongo connect on localhost:27017 ec2-consistent-snapshot: Wed Jun 8 09:11:56 2011: locking mongo ec2-consistent-snapshot: Wed Jun 8 09:11:56 2011: mongo locked ec2-consistent-snapshot: Wed Jun 8 09:11:56 2011: sync ec2-consistent-snapshot: Wed Jun 8 09:11:56 2011: xfs_freeze -f /raiddrive/ ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: create EC2 object ec2-consistent-snapshot: Endpoint: https://ec2.us-east-1.amazonaws.com ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: ec2-create-snapshot vol-xxxx snap-xxxxx ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: ec2-create-snapshot vol-xxxx snap-xxxxx ... ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: xfs_freeze -u /raiddrive/ ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: unlocking mongo ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: mongo unlocked ec2-consistent-snapshot: Wed Jun 8 09:11:58 2011: done
References:
ec2-consistent-snapshot With Mongo
EBS RAID volumes
https://launchpad.net/~alestic/+archive/ppa
MongoDB EC2 Backup and Restore