Category Archives: Uncategorized

Moving web application to use Amazon Web Services

Leave a reply

Basic setup technologies and for launching your web application.

1. Servers
These are machines that runs our websites and web applications or services.

Before moving to AWS, always check your application server requirements.
Complex web application need to run programs or web services to work.
For a simple website I think cheaper to go for monthly smaller VPS.

Amazon EC2 offers on demand instance payable per hour.
If you need to scale your site up and down to meet load demands, Amazon EC2 really excels.
Amazon EC2 servers and its pricing

The nice thing about Amazon Web Services is that there are a lots of excellent technologies available and they all play nicely together. And most probably your web application needs one or more of these technologies.

2. SMTP Service
SMTP Mail service handles sending and routing emails.
Depending on the pricing, features and amount of emails to send, a good comparison is here:

SMTP services I have used: Amazon SES,Critsend, AuthSMTP

3. Email Inbox
To open and receive emails we need inbox.
Except for Mailbox service, what’s good with GoogleApp is that it includes most app you need to run your business. Calendar, Notes, spreadsheets that allows sharing with teams.

4. DNS Service
Domain Name Service translates human readable names like www.example.com into the numeric IP addresses like 192.0.1.1 that computers use to connect to each other.

Some of these available services are:
Amazon Route53
Neticca
Namecheap
ClouDNS
ZoneEdit
GeoScaling
XName
DNS Exit
Afraid

5. Monitoring
For basic monitoring Pingdom.com works great for http/https checks.
Having multiple servers with different services running requires more checks.
Opensource are Nagios for alerting, Graphite or Munin for graphing CPU utilization,memory usage, and the like.

In addition, here are some tools that can help too:

6. Code repository
Keeping development and production codes is a good practice.
Contributing back to opensource community also makes development fun.

www.github.com is a nice tool for these kind of stuff.

7. Communication for both developers and teams.

www.HipChat.com offers a cool team messaging system.
Although group messaging in Skype or Yahoo are applicable, but checkout HipChat for more nice features.

8. Easy Deployment/Configuration Management
Deployment and configuration process should be made easy and repeatable.

Ansible is radical push based configuration, orchestration, and deployment tool.

I hope in some ways these list helps you in your current or future projects. 🙂

Resizing kvm disk image using parted tool

Leave a reply

Resizing KVM disk image can be a bit tricky.
One example is a disk image that contains two partition the root and swap which is on the second partition.
Now to be able to extend the root partition, the swap should be moved somewhere at the end of the disk image. After that, the root partition can now be resize to a much bigger size. 

root# qemu-img convert -O raw precise.qcow2 precise2G.raw
root# losetup /dev/loop0 precise2G.raw
root# fdisk -lu /dev/loop0
Disk /dev/loop0: 2684 MB, 2684354560 bytes
4 heads, 32 sectors/track, 40960 cylinders, total 5242880 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d7c37

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1              63     3998046     1998992   83  Linux
/dev/loop0p2         3999744     4997119      498688   82  Linux swap / Solaris

An example of the disk image is something like that above. We convert it to raw format first and mount it
on /dev/loop0. As you can see even if you increase the size using the following:

root# qemu-img convert -O raw precise.qcow2 precise.raw
root# sudo qemu-img convert -O raw precise.qcow2 precise.raw
root# qemu-img create -f raw additional.raw 2G
root# cat additional.raw >> precise.raw

The root partition will have the same size even if we increase the size of the image.
The solution is to move the swap partition to to the end part.
Most of the posts only make use of rescue disk or resizing disk image using graphical method.
Using the tool called parted we can do this without needing to boot CD for GUI. 

root# losetup /dev/loop0 precise.raw
root# sudo parted /dev/loop0
GNU Parted 2.3
Using /dev/loop0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Loopback device (loop)
Disk /dev/loop0: 4832MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system     Flags
 1      32.3kB  2047MB  2047MB  primary  ext4
 2      2048MB  2559MB  511MB   primary  linux-swap(v1)

(parted) move 2 4321 4832
WARNING: you are attempting to use parted to operate on (move) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs.  We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
(parted) print                                                            
Model: Loopback device (loop)
Disk /dev/loop0: 4832MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system     Flags
 1      32.3kB  2047MB  2047MB  primary  ext4
 2      4321MB  4832MB  511MB   primary  linux-swap(v1)

Above we move the swap partion to the end. Subtract 511MB to 4832MB that results to 4321MB as our start.

(parted) unit kB                                                            
(parted) print                                                            
Model: Loopback device (loop)
Disk /dev/loop0: 4831838kB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start      End        Size       Type     File system     Flags
 1      32.3kB     2047000kB  2046968kB  primary  ext4
 2      4321000kB  4831838kB  510838kB   primary  linux-swap(v1)

(parted) rm 1
(parted) mkpart primary 32.3 4321000
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? Ignore                                                     
(parted) print                                                            
Model: Loopback device (loop)
Disk /dev/loop0: 4831838kB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start      End        Size       Type     File system     Flags
 1      32.3kB     4321000kB  4320968kB  primary  ext4
 2      4321000kB  4831838kB  510838kB   primary  linux-swap(v1)

(parted) q                                                                
Information: You may need to update /etc/fstab.

Here we change the start and end of our root partition and finally we need to resize the filesystem to the newly set extended size.

root# fdisk -lu /dev/loop0
Disk /dev/loop0: 4831 MB, 4831838208 bytes
255 heads, 63 sectors/track, 587 cylinders, total 9437184 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d7c37

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1              63     8439452     4219695   83  Linux
/dev/loop0p2         8439453     9437182      498865   82  Linux swap / Solaris

root# resize2fs /dev/loop0p1
resize2fs 1.42 (29-Nov-2011)
Please run 'e2fsck -f /dev/loop0p1' first.

root# e2fsck -f /dev/loop0p1
e2fsck 1.42 (29-Nov-2011)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0p1: 15769/125184 files (0.1% non-contiguous), 130200/499748 blocks

root# resize2fs 1.42 (29-Nov-2011)
Resizing the filesystem on /dev/loop0p1 to 1054923 (4k) blocks.
The filesystem on /dev/loop0p1 is now 1054923 blocks long.

root# fdisk -lu /dev/loop0
Disk /dev/loop0: 4831 MB, 4831838208 bytes
255 heads, 63 sectors/track, 587 cylinders, total 9437184 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d7c37

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1              63     8439452     4219695   83  Linux
/dev/loop0p2         8439453     9437182      498865   82  Linux swap / Solaris

And detach and convert back to qcow format. 

root# losetup -d /dev/loop0
root# qemu-img convert -O qcow2 precise.raw precise4G.qcow2