I’ve found plenty of articles out there explaining how to use KVM with graphical GUI tools. On most of the CentOS servers I administer, however, I use Kickstart to create a customised and minimal GUI-free install to keep things as simple and efficient as possible. Here, therefore, are some guidelines for how to set up a virtualisation environment and virtual machines using KVM on CentOS 6 via the CLI.

I’m using LVM for the disk volumes because that’s supposedly better for IO efficiency than using disk images stored on the host filesystem, so a bit of familiarity with LVM is ideally needed before getting started.

Initial host setup

Firstly it’s necessary to make sure you have all the necessary software installed:

yum -y groupinstall Virtualization "Virtualization Client"
  "Virtualization Platform" "Virtualization Tools" ;
  yum -y install libguestfs-tools

Next check that libvirtd is running:

service libvirtd status

If not, make sure that messagebus and avahi-daemon are running, then start libvirtd:

service messagebus start
service avahi-daemon start
service libvirtd start

Use chkconfig to ensure that all three of these services start automatically on system boot.

Next it’s necessary to set up the network bridge so that the virtual machines can function on the network in the same way as physical servers. To do this, copy /etc/sysconfig/network-scripts/ifcfg-eth0 (or whichever is the file for the active network interface) to /etc/sysconfig/network-scripts/ifcfg-br0.

In /etc/sysconfig/network-scripts/ifcfg-eth0, comment out all the lines for ‘BOOTPROTO’, ‘DNS1′, ‘GATEWAY’, ‘IPADDR’ and ‘NETMASK’, then add this line:

BRIDGE="br0"

Then edit /etc/sysconfig/network-scripts/ifcfg-br0, comment out the ‘HWADDR’ line, change the ‘DEVICE’ to “br0″, and change the ‘TYPE’ to “bridge”.

Then restart the network:

service network restart

The bridge should now be up and running. You can check its status with:

ifconfig
brctl show

Creating the disk volumes for a new virtual machine

We need to create new LVM volumes for the root and swap partitions in the new virtual machine. I’m assuming LVM is already being used, that the volume group is called ‘sysvg’, and that there is sufficient free space available in the sysvg group for the new volumes. If your volume group has a different name then just modify the instructions below accordingly. Change the volume sizes to suit your requirements:

lvcreate -L 20G -n vm-root sysvg
lvcreate -L 4096M -n vm-swap sysvg

Installing the operating system on the new virtual machine

Here I’m installing CentOS 6 on the guest machine using Kickstart, although I will also explain how to perform a normal non-automated installation. You’ll need to modify the instructions accordingly to install different operating systems.

To make CentOS easily available for the installation, firstly make sure you have Apache installed and running. If necessary, install it with:

yum -y install httpd

Then start it with:

service httpd start

Then create the directory /var/www/html/CentOS and copy the contents of the CentOS DVDs into it.

If you’re using Kickstart then you’ll need these lines in your Kickstart config file to make sure that it can find the files from the CentOS DVDs. The IP address of the host in this example is 192.168.1.1, so change that as needed:

install
url --url http://192.168.1.1/CentOS

These lines are also required to make sure that Kickstart can find and use the LVM volumes created earlier:

zerombr
clearpart --all --initlabel
bootloader --location=mbr
part / --fstype ext4 --size 1 --grow --ondrive=vda
part swap --size 1 --grow --ondrive=vdb

Once the Kickstart file is ready, call it ks.cfg and copy it to /var/www/html

This command installs CentOS on the guest using a Kickstart automated installation. The guest is called ‘vm’, it has a dedicated physical CPU core (core number 2) and 1 GB of RAM allocated to it. Again, the IP address of the host is 192.168.1.1, so change that as needed:

virt-install --name=vm --cpuset=2 --ram=1024
  --network bridge=br0 --disk=/dev/mapper/sysvg-vm--root
  --disk=/dev/mapper/sysvg-vm--swap --vnc --vnclisten=0.0.0.0
  --noautoconsole --location /var/www/html/CentOS
  --extra-args "ks=http://192.168.1.1/ks.cfg"

The installation screen can be seen by connecting to the host via VNC. This isn’t necessary for a Kickstart installation (unless something goes wrong). If you want to do a normal install rather than a Kickstart install then you will need to use VNC to get to the installation screen, and in that case you’ll want to use the virt-install command above but just leave off everything from ‘–extra-args’ onwards.

(If virtual servers are already using VNC on the host then you will need to add the appropriate number to the VNC port number to connect to, e.g. the standard VNC port is 5900, and if there are already two virtual servers using VNC on the host then you will need to connect VNC to port 5902 for this install.)

General administration of virtual machines

Once you’ve got your virtual machine installed, you’ll need to know the various commands for everyday administration of KVM virtual machines. In these examples, change the name of the VM from ‘vm’ to whatever yours is called.

To show general info about virtual machines, including names and current state:

virsh list --all

To see a top-style monitor window for all VMs:

virt-top

To show info about a specific virtual machine:

virsh dominfo vm

To start a virtual machine:

virsh start vm

To pause a virtual machine:

virsh suspend vm

To resume a virtual machine:

virsh resume vm

To shut down a virtual machine (the ‘acpid’ service must be running on the guest for this to work):

virsh shutdown vm

To force a hard shutdown of a virtual machine:

virsh destroy vm

To remove a domain (don’t do this unless you’re sure you really don’t want this virtual machine any more):

virsh undefine vm

Cloning virtual machines

To clone a guest VM, firstly it’s necessary to create new disk volumes for the clone, then we use the virt-clone command to clone the existing VM:

lvcreate -L 20G -n newvm-root sysvg
lvcreate -L 4096M -n newvm-swap sysvg
virt-clone -o vm -n newvm -f /dev/mapper/sysvg-newvm--root
  -f /dev/mapper/sysvg-newvm--swap

Then dump the XML for the new VM:

virsh dumpxml newvm > /tmp/newvm.xml

Edit /tmp/newvm.xml. Look for the ‘vcpu’ line and change the ‘cpuset’ number to the CPU core you want to dedicate to this VM. Then make this change effective:

virsh define /tmp/newvm.xml

You’ll also need to grab the MAC address from the XML. Keep this available as we’ll need it in a minute:

grep "mac address" /tmp/newvm.xml | awk -F ' '{print $2}'

Start up the new VM and connect to it via VNC as per the instructions in the Installation section above. Edit /etc/sysconfig/network and change the hostname to whatever you want to use for this new machine. Then edit /etc/sysconfig/network-scripts/ifcfg-eth0 and change the ‘HOSTNAME’ and ‘IPADDR’ to the settings you want for this new machine. Change the ‘HWADDR’ to the MAC address you obtained a moment ago, making sure that the letters are capitalised.

Then reboot and the new VM should be ready.

Backing up and migrating virtual machines

In order to take backups and to be able to move disk volumes from virtual machines to other hosts, we basically need to create disk image files from the LVM volumes. We’ll first snapshot the LVM volume and take the disk image from the snapshot, as this significantly reduces the amount of time that the VM needs to remain paused (i.e. effectively offline) for. We remove the snapshot at the end of the process so that the VM’s IO is not negatively affected.

This disk image, once created, can then be stored in a separate location as a backup, and/or transferred to another host server in order to copy or move the VM there.

So, make sure that the VM is paused or shut down, then create a LVM snapshot, then resume the VM, then create the image from the snapshot, then remove the snapshot:

virsh suspend vm
lvcreate -L 100M -n vm-root-snapshot -s /dev/sysvg/vm-root
virsh resume vm
dd if=/dev/mapper/sysvg-vm--root--snapshot
  of=/tmp/vm-root.img bs=1M
lvremove /dev/mapper/sysvg-vm--root--snapshot

You can then do what you like with /tmp/vm-root.img – store it as a backup, move it to another server, and so forth.

In order to restore from it or create a VM from it on a new server, firstly use ‘lvcreate’ to create the LVM volume for restore if it isn’t already there, then copy the disk image to the LVM volume:

dd if=/tmp/vm-root.img of=/dev/mapper/sysvg-vm--root bs=1M

You may also need to perform this procedure for the swap partition depending on what you are trying to achieve.

You’ll also want to back up the current domain configuration for the virtual machine:

virsh dumpxml vm > /tmp/vm.xml

Then just store the XML file alongside the disk image(s) you’ve taken.

If you’re moving the virtual machine to a new server then once you’ve got the root and swap LVM volumes in place, you’ll need to create the domain for the virtual machine on the new server. Firstly edit the XML file and change the locations of disk volumes to the layout on the new server if it’s different to the old server, then define the new domain:

virsh define /tmp/vm.xml

You should then be able to start up the ‘vm’ virtual machine on the new server.

Resizing partitions on a guest

Let’s say we want to expand the root partition on our VM from 20G to 25G. Firstly make sure the VM is shut down, then use virt-filesystems to get the information we need for the resize procedure:

virsh shutdown vm
virt-filesystems -lh -a /dev/mapper/sysvg-vm--root

This will probably tell you that the available filesystem on that volume is /dev/sda1, which is how these tools see the virtual machine’s /dev/vda1 partition. We’ll proceed on the basis that this is the case, but if the filesystem device name is different then alter the command below accordingly.

Next we create a new volume, then we perform the virt-resize command from the old volume to the new volume, then we set the new volume as the active root partition for our domain:

lvcreate -L 25G -n vm-rootnew sysvg
virt-resize --expand /dev/sda1 /dev/mapper/sysvg-vm--root
  /dev/mapper/sysvg-vm--rootnew
lvrename /dev/sysvg/vm-root /dev/sysvg/vm-rootold
lvrename /dev/sysvg/vm-rootnew /dev/sysvg/vm-root
virsh start vm

Then, when you’re sure the guest is running OK with the new resized partition, remove the old root partition volume:

lvremove /dev/mapper/sysvg-vm--rootold

(This post assumes a PostgreSQL installation located at /var/lib/pgsql on a Red Hat-type Linux system such as Red Hat Enterprise Linux or CentOS. If your system differs from this, you may need to modify some of the paths accordingly.)

In PostgreSQL, sorts larger than a certain size will get performed on disk instead of in memory, and this makes them much slower as a result. Ideally all sorts should be done in memory (except for the ones that are genuinely too big to fit into your available RAM, because swapping to virtual memory should be avoided at all costs).

Seeing which sorts are happening on disk

To see which sorts are being performed on disk, the following parameters need to be enabled in /var/lib/pgsql/data/postgresql.conf:

log_destination = 'syslog'
log_temp_files = 0

PostgreSQL syslog logging goes to the ‘local0′ facility by default, so in /etc/syslog.conf make sure that you have ‘local0.none’ added to the line for /var/log/messages, and add the following line:

local0.* /var/log/pgsql

Then restart syslog and reload PostgreSQL:

service syslog restart
service postgresql reload

You should now start seeing details of your disk-based sorts in /var/log/pgsql. Each one will start with a line containing the phrase ‘LOG: temporary file’ and will show other details below this line including the SQL of the query which contained the sort. After a while you will see patterns emerging.

Making sorts happen in memory instead of on disk

If you have disk-based sorts occurring frequently, you can find the size of the largest of the frequent queries and then set this as the standard threshold for sorts. The parameter for this in /var/lib/pgsql/data/postgresql.conf is ‘work_mem’. For example, if you had frequent disk-spaced sorts using up to 64MB, you would set the following in /var/lib/pgsql/data/postgresql.conf:

work_mem = 64MB

Be very careful, however, because this setting gets used by each query in each thread, so if you set this too high you could rapidly run out of RAM and cause the system to start swapping, which would be disastrous. The best thing to do is increase this bit by bit until as many as possible of your frequent queries are sorting in RAM but without running out of available memory. To see how much RAM is actually available on your system, use the command ‘free’. As Linux tends to use up the majority of available RAM for its disk cache, the number you actually care about is the ‘free’ column in the ‘buffers/cache’ row – this is how much RAM is really available. For example, here we can see that there are just over 11 GB of free RAM available:

# free
total used free shared buffers cached
Mem: 12300988 12013536 287452 0 277788 10801424
-/+ buffers/cache: 934324 11366664
Swap: 4192956 240 4192716

Once you’ve dealt with sorts for frequent queries, you might find there are still the occasional disk-based sorts happening for a small number of particularly intensive queries. For those, you can increase the sort memory threshold on a per-query basis and then put it back to the standard setting once the query is finished. To do this, prepend your query with:

SET work_mem = '500MB';

changing ’500MB’ to whatever size is appropriate for you, then append the query with:

RESET work_mem;

to return it to the standard threshold. Again, be careful that you don’t use up all the system’s available RAM because then the dreaded swapping will occur.

Once you’ve done all this, as many sorts as possible will be happening in memory instead of on disk, and your PostgreSQL installation should be performing considerably better as a result of this tuning.

Getting information from the logfile into a more useful format

When PostgreSQL logs temporary files (disk-based sorts), the log entries can be rather messily distributed in the logfile, especially if you’re logging other things, such as slow queries, at the same time. I knocked up the following Bash script to quickly extract the details of the sorts into a more useful format:

#!/bin/bash

PROG=$(basename $0)
PPROG=$(echo $PROG | awk -F '.' '{print $1}')
TMPFILE1=/tmp/$PPROG.tmp1
TMPFILE2=/tmp/$PPROG.tmp2
LOGFILE=$1

if [ -z "$1" ] ; then
    echo "Usage: $PROG LOGFILE"
    exit
fi

cat /dev/null > $TMPFILE2

grep "temporary file" $LOGFILE > $TMPFILE1

cat $TMPFILE1 | while read LINE ; do
    NO1=$(echo $LINE | awk -F '[][-]' '{print $2}' )
    NO2=$(echo $LINE | awk -F '[][-]' '{print $4}' )
    cat $LOGFILE |
        awk -F '[][-]' "($2 ~ /$NO1/) && ($4 ~ /$NO2/) {print $0}"
        >> $TMPFILE2
    echo "" >> $TMPFILE2
done

cat $TMPFILE2

rm -f $TMPFILE1 $TMPFILE2

For six weeks I am exhibiting my iPhone photography in Cafe Seventy Nine in Primrose Hill, London.

It’s a great space so go along and check it out! There are prints and books of my work for sale there. They also serve great coffee and tea and delicious vegetarian food.

My experience with Lion so far has been, to say the least, mixed. I thought I’d list some of my experiences in the form of a brief review.

Launchpad: Tedious scrolling through multiple screens of apps in order to launch one of them? No thanks. Dreadful.

Mission Control: It seems like a good idea in principle to combine Exposé and Spaces, but in doing so some useful functionality is lost: minimised windows are no longer shown; multiple desktops are now one-dimensional rather than two-dimensional; I no longer have a number in the Menu Bar telling me which desktop I’m currently on; and it’s harder to see the contents of the other desktops. All a bit of a shame really.

Full-Screen Apps: This seems initially useful, but: it doesn’t make use of multiple monitors; the keyboard shortcut only seems to work some of the time; it disables my hot corner to quickly get to the Desktop; switching between windows within a fullscreen app (e.g. Mail) is sometimes impossible; and it adds confusion because all windows already have a Zoom button, so now they all have two maximise buttons instead of one.

Resume/Auto Save/Versions: This seems like a good idea in theory, but I doubt many of the apps I use will implement this for a long time, if ever.

AirDrop: I was really looking forward to this one but didn’t realise that it only works on some recent Macs, i.e. none of mine. So that was very disappointing.

Mail: The redesign is quite nice both aesthetically and ergonomically. Nothing ground-breaking, but nice. I’ve already become very dependent on the conversation view to the point where I now wish they’d hurry up and enable it for my iPhone too.

Address Book/iCal: The horrible graphics in these are ugly and ridiculous, and Address Book has much less space for useful information as a result. I thought Apple had stopped this sort of nonsense years ago and didn’t expect to see it come back now. Also, iCal is having severe problems talking to my work calendar, and the work calendar is iCal Server running on OS X Leopard Server, so it’s particularly annoying that that doesn’t work properly [now fixed by using SSL for CalDAV]. Having said all that, I do like the new ‘planner’ pane on the ‘Day’ view – that’s now replaced ‘Week’ as my standard view as a result.

Front Row: Instead of bringing this up to date, which would have been sensible especially now that Mac minis have HDMI outputs, they’ve just removed it instead. Great. Thanks, Apple. I don’t use it that much in comparison to Plex on my Mac mini-based media centre, but it would have been nice to have functionality added instead of the whole thing being taken away. Very poor show indeed.

Terminal: I love the new ‘Blur’ option for window backgrounds, and I’ve come up with a really nice combination of blur and opacity for my active and inactive windows. But why, when I open new tabs, do they start SSHing into random servers when I don’t want them to? I presume this is something to do with Resume but it’s bloody annoying and I want it to stop! [Turns out it was necessary to change the 'New tabs open with' settings in the Startup tab in Terminal Preferences.]

UI changes: Being able to resize a window from any edge is very welcome – I wish they’d done this years ago. It’s nice to see the back of the oversized Aqua scroll bars at last (but desirable to change the scroll bar setting to show ‘Always’ in the General System Preference because this is a computer, not an iPhone). The overall look and feel is like a more subtle, streamlined version of Snow Leopard, which is quite nice generally. I worried about the icons in sidebars (in Finder and Mail most noticeably) being stupidly oversized, but thankfully I then found the ‘Sidebar icon size’ option in the General System Preference – setting that to ‘Small’ makes everything OK again. And the increased use of animation is quite enjoyable and might occasionally be vaguely useful for new computer users. All the new gesture stuff is irrelevant to me because I don’t have any of the new Macs with big trackpads, and I prefer using mice anyway (though not Apple mice because they’re an ergonomic nightmare). Oh, and suddenly finding scrolling going in the wrong direction wasn’t particularly useful, but thankfully that was easy to turn off in the Mouse System Preference.

System stuff: Make sure you have ‘Show indicator lights for open applications’ ticked in the Dock System Preference, partly because that’s a traditional behaviour which helps to actually make the Dock useful, and partly because if you don’t then Lion will randomly quit open apps without asking. Another bit of weirdness is the ‘Mail, Contacts & Calendars’ System Preference, which seems to serve no useful purpose whatsoever other than to confuse users by duplicating settings which you’ve already got in Mail, iCal, etc.

So there we go. Once you’ve turned all the annoyances off it’s basically just a neater, slicker version of Snow Leopard with a few things missing, so I’m not exactly blown away by this new OS. I think the main problem is that trying to mix Mac OS X and iOS causes a conflicting mess of too many different UI paradigms. Apple should either have stuck with tweaking the Mac OS X foundations or rebuilding the whole thing from the ground up, not ending up with something in between the two which doesn’t quite work.

Edit: It’s quite interesting to compare the rather dark tone of this review to the optimistic positivity of my Snow Leopard and Leopard reviews. It makes Lion feel like even more of a step back than a step forward.

Edit: Reading back on my reviews of previous versions of OS X, I’m reminded of what a mess Apple made of To Do Items in Mail when that was added to Leopard. I see they’ve fixed that by simply removing them again, which I guess is fair enough really.

Photographers are increasingly embracing smartphones, and Apple’s iPhone in particular, as valid alternatives to their SLRs. Adopting the philosophy that ‘the best camera is the one that’s with you’, photographers find with iPhones that they can simply point and shoot in situations where photography would not be an option with a larger camera. For many, this embodies the true essence of photography, in which making a timely visual capture of a particular moment is more important than painstakingly changing lenses and altering aperture settings. More and more photographers are questioning the assumption that bulky, costly photographic equipment is necessary for taking worthwhile photographs. Instead, they are realising that you can never tell when inspiration is going to strike, and being able to take photographs instantly at any time is incredibly advantageous.

As well as making it possible to take photographs at any time, the iPhone’s operating system, iOS, provides the photographer with a portable processing lab in the form of third party ‘apps’ downloaded from the App Store. Other smartphones have photo apps too, but the range available on iOS is far superior to what’s available on Android or other platforms. Quite a few people are now familiar with the popular Hipstamatic app, but this is only one of many. The first iPhone photography apps tended to offer simulations of Lomography cameras and were often quite ‘lo-fi’, which was perhaps natural given the relatively poor quality of the cameras on the first iPhones. However, the camera has been improved on each new iPhone model, and the iPhone 4 has a camera which is on a par with some compact cameras. At the same time, the photography apps available have gone beyond just lo-fi effects and into much more professional territory, offering advanced editing features, layering and masking, a huge range of filters and textures, and other facilities such as HDR. The range of editing possibilities for iPhone photography is now quite incredible.

In addition to the camera itself and the software, there are increasing numbers of hardware accessories for iPhoneography, such as the Owle Bubo mount and stabiliser, and the Rollei telephoto lens for the iPhone 4. The fact that hardware manufacturers such as Rollei are taking the iPhone so seriously is a very good sign for the future of the medium. Another very good sign is that the iPhone 4 has just overtaken the Nikon D90 as the most popular camera in the Flickr community. Many people are now finding that the sheer power of a good camera that’s always available, combined with the advanced editing features offered by the huge range of apps, makes for photographs which are both spontaneous and highly artistic. This has led to a new photographic and artistic subgenre referred to by some as ‘iPhoneography’, which is seen in many ways as highly representative of the current zeitgeist of art meeting technology. iPhone photographs can encompass all photographic genres from street photography to landscapes, but their spontaneity and artistic processing gives them a style of their own. The influence of iPhoneography can be seen increasingly on more traditional photography, as well as on film and TV, where iPhoneography-style processing such as blurring, tilt-shift effects, vignettes and colour filtering is increasingly in vogue.

There is a vibrant community of incredibly talented iPhoneographers on the Internet who are pushing the boundaries of this medium and sharing their results with each other and the rest of the world, and this even includes some professional photographers such as Chase Jarvis who have embraced the iPhone as a valid and exciting part of their toolkit. Many of these iPhoneographers use Flickr to share photos and to give feedback. This makes sense because Flickr has been the most popular photo-sharing site for some time. However, some iPhoneographers use other sites in addition to, or instead of, Flickr. Tumblr has been popular due to its simple microblogging format which is ideal for posting photos and getting feedback on them, although recent technical glitches have made it a less attractive proposition. Some maintain photo blogs using WordPress or Blogger. Twitter is also very popular with iPhoneographers because it enables them to share thoughts and tips, as well as to link to new photos no matter where they are hosted.

In addition to these generic sharing sites, however, there are sites which are specific to mobile photography. Taking the iPhoneography world by storm currently is the mobile photo-sharing phenomenon known as Instagram. Despite its low-resolution square-format restrictions, it enables mobile photographers to share photos and get feedback in a very simple and streamlined way which makes it very addictive for many. Other sites dedicated to mobile photography are EYE’EM and MobiTog. Instagram, EYE’EM and MobiTog cater to all mobile photographers, not just iPhone photographers, so those using Android phones and Blackberrys can also join in the fun.

A rather different approach to photo sharing is taken by P1XELS. This is a curated site which takes the form of a blog, and at P1XELS they are keen to emphasise the importance of iPhoneography as an art form rather than just as a method for taking ‘snaps’. Only photographs taken on iPhones are welcome here, which is probably indicative of how much more advanced the iOS platform is for photography in comparison to other mobile platforms. Anyone remaining cynical about the worth of iPhoneography only has to browse the beautiful images on this site to find out what incredible works of art can be created from photographs taken on the iPhone.

There are various other blogs devoted to showcasing talented iPhoneographers and which also provide news and information about the iPhoneography scene. Glyn Evans’ iPhoneography blog, founded back in 2008 when Glyn coined the term ‘iPhoneography’, is usually the first port of call for information about what’s happening in the iPhoneography world. Other excellent iPhoneography blogs include Marty Yawnick’s Life in LoFi, Edgar Cuevas’ iPhoneogenic, and the bickr blog. As well as these blogs, there is also the very worthwhile iPhoneArt site which has a thriving community and is growing and getting even better all the time, and also the extremely useful iphoneographyCentral site, recently set up by Nicki Fitz-Gerald, which provides iPhoneography tutorials and other handy resources.

iPhoneography exhibitions and events are occurring around the world. Some of these have been arranged by P1XELS, but there have been quite a few others too, including several at various Apple Stores worldwide. More are appearing all the time, such as the HIPSTAMATICS exhibition in London earlier this year, and also the London iPhoneographers: Street Photography event which I’ve helped to organise at the Apple Store on Regent Street in London. As mobile photography becomes ever more prevalent, no doubt we will see even more of these events and exhibitions appearing in the future.

A similar scenario is occurring with books about iPhoneography, which are increasing in number all the time. Current examples of traditionally-published books are The Art of iPhoneography by Stephanie Roberts, Create Great iPhone Photos by Allan Hoffman, iPhone Photography & Video for Dummies by Angelo Micheletti, and iPhone Obsessed: Photo Editing Experiments with Apps by Dan Marcolina. There are also now many self-published books by iPhoneographers on Blurb, including my own iPhoneography.

After reading this article, I hope those skeptical about the future of mobile photography, and iPhoneography in particular, are a bit less cynical. I also hope that those who are already pursuing an interest in this type of photography have found some useful information and have been encouraged to take their interest further. There is much fun to be had, and many great images to be created. My own interest in iPhoneography began in August 2009 (when I wrote a blog post about my new-found enthusiasm for it), and since then, as already mentioned, I’ve helped to organise an iPhoneography event and have published a book of my iPhone photos. I’ve also been showcased and reviewed in various places, been successful in photo contests, been featured on Flickr Explore, and been approached to have one of my photos used for the cover of a novel (though unfortunately this didn’t happen in the end, but it was nice to be considered). Additionally, I run the London iPhoneography Group which I formed in September 2010. You can find me on Flickr as matt_brock, you can check out my Tumblr-based iPhoneography site or look at my work on P1XELS should you wish to do so, and you can find me on Twitter as _mattbrock. My interest in iPhoneography continues to grow, and given the exciting nature of the medium I’m quite sure it will do so for some time to come.

At 7pm on 11th July, the Apple Store on Regent Street will be hosting an iPhoneography event to highlight four talented members of the London iPhoneography Group who will be showing their work and discussing how they use their iPhones for street photography in London: Daniel Holland, Robson Santos, Matthew Burlem and Leyla Bile.

We’re very excited about this because it will be the first iPhoneography event in an Apple Store in England, and also because it will coincide with the first ever London Street Photography Festival which runs from 7th to 17th July. There have been superb iPhoneography events in other Apple Stores around the world, and we’re very pleased to be able to do one of our own in London.

As administrator of the London iPhoneography Group and a keen iPhoneographer and Londoner, I’m providing support for this event (but not showing photos this time).

Useful Links

Daniel Holland (Danny Dutch)
Robson Santos (iPhoneography London)
Matthew Burlem
Leyla Bile

London iPhoneography Group
London Street Photography Festival

Update (18/07/11)

The event was extremely successful and I’ve written a review of it on the iPhoneography blog. Below are some photographs from the event.

Following our successful migration to Amazon’s S3 service for media storage and delivery, we decided to move our entire server infrastructure from its traditional data centre colocation to Amazon’s Elastic Compute Cloud (or ‘EC2′). Using this cloud-based infrastructure instead of data centre colocation provides two main benefits for us.

Firstly, EC2 is much more flexible than traditional colocation. Our infrastructure can now be expanded or altered, quite radically if necessary, in hours or even minutes, just by typing a few commands or clicking a few buttons. Previously, making similar changes would have taken weeks of planning, ordering, delivery, physical installation, software installation, etc.

Secondly, EC2 works out cheaper. EC2 server instances tend to work out cheaper than their physical colocation equivalents, especially when you’re able to plan ahead with pricing by using Amazon’s ‘Reserved’ pricing instead of their standard ‘On-Demand’ pricing. Also, because of EC2′s flexibility, it’s only ever necessary to run precisely the infrastructure required at the present moment, so you’re never paying for more than you need. With a traditional colocation setup, the lack of flexibility means that you will always have more hardware than you need at the present moment in order to cope with possible traffic spikes, hardware failures, etc., and that extra hardware increases the cost even further.

Getting up and running with EC2 is pretty straightforward. You just need to provide payment details, set up security details, and then you’re free to fire up server instances and try it out. There are various instance types to choose from depending on the hardware spec you need. Much of this can now be done via the Amazon Web Services (‘AWS’) Management Console, although you can also install the EC2 command line tools in order to operate EC2 via the CLI. I prefer using the CLI, probably because I’ve been doing it that way for longer, but also I guess there are still some things you can do via the CLI which you can’t do via the AWS Management Console.

Any decent system administrator will want to set up and configure instances according to his own tastes and requirements, and the best way to do this is to start an instance from an image that’s closest to what you want, then modify it accordingly, then create your own image from that running instance. I originally started up an instance from a CentOS image provided by RightScale, then modified it heavily according to my own preferences, then created an image of my own from it. This image is what I called the ‘base’ image.

I then created several different instance types for each of the different types of servers we have in our infrastructure. For each of these I started from the ‘base’ image, installed the applications required for that instance type, then created a new image from it, so that I ended up with several images: one for web server instances, another for mail server instances, and so on. Each instance type has a certain set of events which need to occur when that instance is first run, e.g. working out its own external hostname, setting up certain directories that are needed, etc., and for this I have a startup script in /etc/rc.d/init.d on each instance which works out whether it’s a brand new instance and, if so, performs all these tasks.

Once all the required instance types have been created and made available as images which can be started as required, it’s almost possible to start up a full web application infrastructure with all the components required for it. However, there are a few extra things to think about first.

Firstly, it’s necessary to consider disaster recovery. Conveniently, most of our instances do not contain any unique data which would be missed if the instance died, and if an instance dies then it’s normally just a case of firing up a new one to take its place. Obviously requirements will vary from one company to another, but in our case only the database instance contains unique data, i.e. the live database. I solved the disaster recovery requirement for the database instance by utilising an AWS storage solution called Elastic Block Store (or ‘EBS’). An EBS volume is a storage volume which can be connected to a running instance and which can also be snapshotted. So, all the data for our live database is on an EBS volume connected to the database instance, and this gets backed up and snapshotted every night. I have a Bash script which does this automatically from cron, and which also removes any shapshots older than 28 days.

Therefore, if the database instance dies, I can just fire up a new instance and connect the EBS volume to the new instance. In the unlikely event that the EBS volume dies, I can just create a new one from the latest snapshot and connect it to the database instance. This is therefore a very handy DR solution for the live database, and EBS volumes have the added bonus of being faster than the standard instance-based storage, which is obviously ideal for a database.

All instance images and EBS snapshots are stored on S3. S3 is an exceptionally safe form of storage, so the chances of anything getting lost on it are incredibly low, and this very low level of risk might be reassurance enough for many companies. In our case, however, I’m exceptionally paranoid so I also take backups of everything we have on S3 (including all our media which is also stored there). Every month I use the rsync-style functionality within the very handy s3cmd tool to incrementally backup all our S3 buckets to a disk on a server in our office.

Another consideration is how to load balance the front-end servers. Amazon provide a great solution for this called Elastic Load Balancing. This can be used to quickly and easily create a load balancer which can be configured to distribute whatever traffic you like across whichever instances you need. It even has support for sticky sessions. The only problem I’ve found with ELB is that it requires you to point a CNAME at the load balancer rather than an A record, which means that you can’t use root domains (i.e. apple.com instead of www.apple.com) because the DNS spec does not allow a root domain to be a CNAME. There are various hacks to get around this, but none of them are perfect, and it’s a problem I hope Amazon will solve soon. In the meantime, this thread in the AWS developer forums gives more information.

It’s also necessary to consider the delivery of email. Unfortunately, due to the ease with which server infrastructures can be created on EC2, a lot of spammers use it to send email, which means that EC2 IP addresses are heavily blacklisted. For each EC2 instance you wish to send email from, therefore, you need to follow a process which helps Amazon to ensure that you’re not sending spam. Firstly, you need to assign an Elastic IP to the instance. An Elastic IP is an IP address you can request and then keep for your own use, assigning it to whichever instance you like at any given time instead of using whichever IP address that instance happened to have by default. Once the Elastic IP is assigned to the mail server instance, you just need to tell Amazon to set up reverse DNS for that Elastic IP and remove its email sending limitations. This process works well, and we haven’t had any problems sending mail from our mail server instance since completing it.

As with the general EC2 functions, EBS, ELB and Elastic IPs can be controlled via the AWS Management Console, but the CLI tools tend to be more powerful than the console. CLI commands for EBS and Elastic IPs are included within the EC2 command line tools, but ELB has its own set of CLI tools.

The actual migration of our infrastructure to EC2 was similar to other infrastructure migrations I’ve done in the past, but with the added bonus of EC2′s flexibility. This means that you can set up the infrastructure roughly how you think it will need to be, and then tweak it as you go along by starting and stopping instances, changing instance types, etc., instead of having to predict things as best you can then hope for the best. Since the migration, our entire infrastructure has been successfully running in the cloud for several months, and we continue to be very happy with Amazon Web Services. We closed our account with our colo provider some time ago, then sold all our physical servers.

For those of you who haven’t yet started using EC2, or who haven’t got very far with it yet, lots of useful details on the practicalities of hosting on EC2 can be found in this excellent series of blog posts written by my friend Paul Norman.

There’s a lot more you can do with EC2 which I haven’t covered, but hopefully this post gives an idea of how easy it is to set up a web infrastructure on there, and by reading the documentation and following the updates from Amazon, you can easily work out which additional details you need.

I make extensive use of several Macs and an iPhone, I like using desktop and mobile apps in preference to web apps, and it would drive me mad if I couldn’t keep data constantly and wirelessly in sync between my apps on all my devices. I’m constantly trying to find the best ways of doing this, so I thought I’d share my methods for the benefit of all. This applies to anyone with one or more Macs, iPhones, iPads and iPod Touches.

As I have a MobileMe account, there is a heavy focus on that service, but MobileMe does cost a significant amount of money each year which may not be ideal for everyone, so I’ve given alternatives wherever possible.

Email

Email is the first of several things that I use MobileMe for. On the whole, MobileMe provides a decent, reliable email service with plenty of storage, and it works very smoothly with Mail on iOS and Mac (as you would expect). Also, the MobileMe System Preference enables you to sync all your mail accounts, rules, signatures and smart mailboxes across all your Macs, saving you a lot of configuration hassle. Additionally, MobileMe webmail is very nice for those rare occasions where it’s needed.

A free alternative is Gmail, but this doesn’t work as nicely as MobileMe in Mail on Mac or iOS – in fact it’s pretty messy in Mail on Mac – plus you can’t automatically sync your mail accounts, rules, signatures and smart mailboxes across all your Macs.

Contacts

I’m always amazed when I receive one of those Facebook updates from a friend saying that they’ve lost or changed their phone, and could everyone please contact them to remind them of their number? Now that we all use multiple computers and smartphones, surely nobody should have to manually re-add all their contacts into their new phone.

I use MobileMe to keep contacts synchronised automatically and wirelessly between Address Book on my Macs and Contacts on my iPhone. It works beautifully.

If you’re using Gmail for email then you’ll presumably want to access your Google Contacts. You can do this from iOS with Google Sync, which works OK. It’s harder to get Google Contacts working with Address Book on Mac, though. It generally seems to be pretty glitchy and, at least for certain versions of Address Book, the sync will only occur when you physically connect and sync your iPhone, which is pretty silly. On the whole, this is a somewhat messy and undesirable option from what I’ve seen.

Calendars

Again, I use MobileMe to keep my calendars in sync, and it does a great job of keeping calendar events synchronised across iCal on my Macs and Calendar on iPhone. As well as my regular calendars, I also subscribe to calendars on other sites such as Last.fm, and MobileMe keeps all these calendar subscriptions in sync too, both on my Macs and my iPhone. It even maintains the proper calendar colours across all my devices! Excellent stuff.

The obvious free alternative here is Google Calendar. Using Google Sync this works reasonably well with iPhone, but getting synced with iCal on Mac is fairly complicated and potentially rather unreliable. Also, as far as I am aware, Google Sync doesn’t provide any way of letting you sync calendars you’ve subscribed to on other sites. Again, not a great solution.

To Do Items

MobileMe infuriated me for quite some time with its inability to sync To Do Items from my Macs to my iPhone. However, the new MobileMe calendar enables third-party developers to make apps which ‘plug in’ to the MobileMe calendar to fill in the missing gaps. One such app which has recently been created for iOS is BusyToDo. I’ve just started making use of this app and I can report that it works wonderfully and is fantastic value for money. I can now sync my To Do Items back and forth between my iPhone and iCal on my Macs, and because I make extensive use of ‘to do lists’ I’m extremely pleased about this.

If you’re not a MobileMe subscriber, the best solution I’ve found is to use Google Tasks, which is free. On iPhone, iPad or iPod Touch, go to gmail.com/tasks in Safari, then just log in with your Google account and you’re ready to go. If you click the arrow at the bottom and choose ‘Add to Home Screen’, it will create a nice Google Tasks icon so it can be accessed like a regular app.

Unfortunately, setting up Google Tasks as a Mac app is a little more complicated, but it’s not too complicated and it works well when it’s done. The first thing to do is download and install Fluid, a very handy program which enables you to turn web apps into Mac desktop apps. Once installed, launch Fluid to create a new ‘Site Specific Browser’. The URL should be https://mail.google.com/tasks/ig and the Name should be ‘Google Tasks’, then click Create. You can then go into your Applications folder and run Google Tasks just like a regular app, logging in with your Google account to get to your tasks. To make it look a bit nicer in your Dock, you might want to add a more appropriate icon. I suggest downloading the excellent Fluid Mac Icons from Fairhead Creative Goodies page (and double click to unzip it once it’s downloaded). Then go into Preferences for your Google Tasks app, click the Change button next to ‘Application icon’, and select the ‘Gtasks-Icon.rsrc’ from the ones you just downloaded. You might need to restart the app and possibly re-add it to your Dock for the new icon to take effect.

When the app is running, you should see something which looks like this:

The only problem with this solution is that it doesn’t work when you don’t have any Internet access, but in practice I haven’t found that too much of a problem.

There is also a task list management application called Things, but that’s stupidly overpriced for a glorified task list in my opinion, and it doesn’t even do over the air sync between iOS and multiple Macs.

Notes

I make extensive use of notes for keeping lists of music and films I want to get, making meeting agendas and action lists, writing down ideas for songs, and all sorts of other random things. Notes are, therefore, pretty important for me, and I use MobileMe to keep my notes synced between Mail on Mac and the Notes app on iPhone. In my opinion it’s a bit odd having notes in a mail application, but there we go; it seems to do the job fine, and it helps to keep as many things as possible within MobileMe, which is neater for me.

If you don’t mind in-app adverts and a limited amount of storage for syncing data, a free alternative to consider is Evernote. You can install the Evernote app on multiple Macs and iOS devices and it does a fair job of keeping things in sync wirelessly. I found, however, that the wireless sync on the iPhone was pretty ropey, but they may have improved it since then. One benefit of Evernote is that it allows many types of media as well as just written notes, so you can take a photo with your iPhone and use that as a note, or you can save a web address from Safari and use that as a note, etc. That’s quite cool.

Documents

It’s obviously very useful to have your documents kept in sync between all your devices. I use MobileMe’s iDisk for this. On my Macs I choose the option to sync my iDisk, which keeps a local copy of the iDisk on that Mac to speed things up. On my iPhone I use the iDisk app which enables me to view documents (though, annoyingly, it doesn’t work with OpenOffice documents) and share them. In general, this solution works well for me.

A popular alternative is Dropbox, which has apps for Mac and iOS. When I’ve used Dropbox I’ve found it to be a little clunky, but many people seem very happy with it. Dropbox is free for a limited amount of storage, but prices increase significantly when you add additional storage. Another solution to consider is Google Docs, which allows you to edit documents in the browser on both Macs and iOS devices. As far as I know there are no desktop apps for it, although you could use Fluid to make a desktop app out of it like we did with Google Tasks (above). Google Docs is free, however.

Bookmarks

I depend quite heavily on bookmarks and find it frustrating when I’m using a browser which doesn’t have all my bookmarks in it. By far the simplest way of dealing with this problem is to use Safari on all your Macs and iOS devices and sync bookmarks between all of them using MobileMe. This works well for me because I have a MobileMe account and I consider Safari to be a decent browser.

It’s not easy finding alternatives to this system. Other browsers offer free ways of keeping bookmarks in sync, e.g. Google Chrome features a bookmark sync facility using your Google account, but that only works with Chrome on the desktop and will not sync your bookmarks to Safari on your iPhone/iPad/iPod Touch. If you use MobileMe but you prefer to avoid Safari on the desktop, you can get around this by using Xmarks to sync your bookmarks from your preferred browser to Safari on the desktop, then use MobileMe to sync from desktop Safari to mobile Safari – this is a messy solution, but it does work.

News

RSS is a fantastic way of keeping up to date with your favourite news sites, and it becomes even more useful if you can keep your RSS feeds in sync between all your devices. You can sync RSS feeds over MobileMe but in order to do this you have to use the RSS facilities in Safari which are fiddly and limited, and you can’t sync to iOS, so all in all this is a pretty poor solution.

Google Reader is the best thing to use here, and it’s free. I don’t much like using the Google Reader web app on my Macs, but there are Mac desktop apps which will sync well with Google Reader and provide a good all-round experience. I use NetNewsWire, which has always worked well for me, although I’m not keen on the in-app adverts. There are other options on Mac such as Gruml, but I’ve never felt any huge need to switch.

NetNewsWire is also available on iOS but, like many others, I’ve found it to be increasingly unusable on my iPhone. There are much better alternative (paid) apps which many people swear by such as Reeder and Byline, but at the moment I actually just use Google Reader in Safari because it’s free, it has a decent mobile interface interface, and you can create an app icon for it on your home screen just as we did for Google Tasks (above).

Twitter

Yes, it’s actually possible to keep Twitter in sync across multiple devices. If you’re a fairly heavy Twitter user, you might find it quite annoying that your desktop Twitter app doesn’t know about all the tweets, direct messages, replies, etc. that you dealt with earlier on your iPhone. You can solve this problem by using Echofon on all your Macs and iOS devices. It’s not perfect, but it does a reasonable job of remembering where you were on your iPhone and transferring that across to your Mac, and vice versa. Echofon is free if you don’t mind seeing in-app adverts.

Edit (08/02/11): modified ‘To Do Items’ to reflect the availability of BusyToDo for syncing MobileMe To Do Items to iOS.

2010 was, for me, very much about iPhoneography. During the year, iPhoneography seemed to go from a fun, interesting curiosity to a full-blown photography subgenre or even a ‘movement’, and I’ve become part of a vibrant community of iPhoneographers on the Internet who are pushing the boundaries of this medium and sharing their results with each other and the rest of the world. This feels like being part of something quite special, and I’ve met some cool new people through the community.

Towards the end of the year I created the London iPhoneography Group and I’m hoping to find the time to arrange a meet for the group sometime soon – it will be great to meet some fellow iPhoneographers in real life. I also produced a book of the best iPhone photos I took with my iPhone 3G, and I’ve received lots of nice feedback from people who have bought the book which makes it really worthwhile. During 2011 I hope to produce a book of photos taken with my iPhone 4.

During 2010 I’ve been featured in various places and had some success with contests during the year, details of which are available here (when Tumblr is working). I hope to continue this success in 2011, but the most important thing for me is to keep trying to create interesting iPhone photos and share them with others, and also to keep being inspired by the amazing things other iPhoneographers are doing. Long may it continue!