I have been using KVM virtualization for a while now, and often see folks asking the same questions I've had to discover the answers to, so I thought I'd write up my limited knowledge as a starting place. I do all this with Fedora as the host (fedora 13 currently). I don't use NetworkManager for networking and I disable selinux. This information may or may not be relevent in other environments (in particular selinux is likely to honk at you if you don't get the labeling right on many of the files I edit manually). Networking is the first mystery to solve. I find the default networking provided by libvirt to be worse than useless, interefering with DHCP and iptables by doing mysterious things to them behind my back. The first commands I run after installing fedora get rid of the libvirtd default network: virsh net-destroy default virsh net-undefine default That leaves virtual machines with no network at all, so the next thing to do is setup a bridge. Let me hasten to add, that I have no real idea what a bridge actually is, but the end result allows all my virtual machines to appear to be real machines on the same local network with the host machine. I define the bridge manually (I don't know if there are any tools that can do this) by editing the /etc/sysconfig/network-scripts/ifcfg-eth0 file and constructing a /etc/sysconfig/network-scripts/ifcfg-br0 file. The links here are to the specific examples on my machine, but the basic idea is that all the information defining the IP address and wot-not moves from the eth0 definition to the br0 definition, and the eth0 definition then points to the br0 bridge. Once you reboot after doing all this, you will get a br0 bridge interface defined and eth0 will be a part of it. The libvirt code will recognize that your system has a bridge, and if you use it to define new virtual machines, it will offer bridge networking as the default choice. Important note: If your host machine is running iptables for firewall protection, you will almost certainly need to change any references to eth0 in the firewall rules to br0 instead (or add duplicate rules for br0). If you are running a dhcp server on your network, your virtual machines can now get their network setup over the bridge via dhcp just like the real machines on the same network. I formerly ran dhcpd and named on my fedora host, but have recently stopped that and now run dsnmasq on my dd-wrt enabled netgear router. This is very convenient since all machines on the network can now get their DNS info from the router as well, and the names of virtual machines show up when they are booted. My preferred disk format has become qcow2. I tend to install lots of machines for testing different things in clean environments, and performance is not that important to me. The qcow2 image file format is the most compact, so the image files don't take up too much space, and the performance doesn't really seem to be all that bad. The other gimmick that qcow2 supports is the backing_file option. This makes it simple to do things like deploy several identical machines by installing once, then making new disk images to point to the same backing file. You can also very quickly restore the image to the same base install point by removing and recreating the new image using the same backing file. In the case of my Windows XP virtual machine, which I do use as more of a production system (when only Windows will do), I recently found a way to compact the image after it had grown a bit over the years getting updates and wot-not: Inside the Windows machine, I did all the cleanup operations possible, removing uninstall files for old windows updates, cleaning up temp files, defragmenting several times, etc. I then ran the SDelete utility, using the -c option to fill all the free disk space with zeroes. This takes a while (about 2 hours for me), and blows up the qcow2 image file to max size, but all the extra space is now zero bytes. At this point you can convert that image file from qcow2 format to qcow2 format, and all those zeroes get compacted out of the output qcow2 file. As another way to avoid clutter in my image files, I added a small second virtual disk that just uses the raw image format and changed the system settings so it keeps the swap space (pagefile.sys) on this second disk. I also point various TMP and TEMP environment variables at it to use for temp space. The raw disk image should run faster for swap, and I don't need to backup that disk since there isn't anything in swap (or temp) that is important to backup. I recently (for testing) installed a new XP from scratch and ran into Redhat bug 579348. Following the links in that bugzilla, I found various folks that had discovered ways to fix the broken disk image using hexedit on the image file. Unfortunately, the qcow2 image files aren't that simple to interpret, but I got around that problem by adding the XP image I wanted to fix to an ubuntu virtual machine as a 2nd disk, then ran hexedit in that VM to fix the /dev/vdb disk. No need to understand details of qcow2 internals, just let qemu do it for me :-). The end result was an image file that would boot correctly! Far more important to performance than image file format is the use of virtual I/O drivers. Regardless of the underlying file format, they seem to make a perceptable improvement in performance inside the virtual machine. A recent update in the KVM code apparently broke the old virtio storage drivers I was using, and my production Windows XP machine would no longer boot. I could get it to boot by changing the disk emulation back to IDE, but that drops the performance. After some searching on the net and questions in the fedora mailing lists, I found a pointer to the latest fedora 13 compatible virtio drivers, but installing them was tricky. To get them installed, I finally used this technique: I connected the ISO image with the drivers to the CDROM. I left the boot disk as an IDE disk. I added a dummy 2nd disk and made it be a virtio disk. Then I booted the system and let the new hardware wizard ask me for drivers for the new disk. I pointed it at the CDROM, got the drivers installed, and shutdown the virtual machine. I then removed the 2nd disk from the VM definition and changed the 1st disk back to virtio instead of IDE. At this point I could finally boot again from my main boot disk using the virtio drivers. (Whew!). I have several virtual XP machines defined, all using the same base image, and when I periodically use the cleanup and compact technique described above to generate a fresh base image, I need to recreate all the image files for my copies. I have to fix the names of the machines and like to change the background wallpaper so I can tell at a glance which machine I'm looking at. I previously did this manually, but I recently developed a script using the libguestfs tools and it works great. I called the script update-image, and it recreates the qcow2 C: image file for all my clones of my main Windows XP virtual machine. The two required arguments are the name of the virtual machine and the base name of one of the standard background wallpaper .jpg files available on the windows XP virtual machine, and it creates a new qcow2 image backed by the same base image file that backs my main machine, then modifies the registry entries required to change the machine name (there appear to be a lot of them) and the registry and the cache file to modify the default login user's background image. I've installed the spice drivers in my windows virtual machines, and I'm using a C program qemu-spice-wrapper.c as a wrapper to run instead of qemu-kvm to allow me to use spice and still control the machines under libvirt. It looks for files like winxppro.spice in my images directory to substitute per-machine arguments to pass to qemu-kvm. Fortunately, I have experimented, and determined that all this hackery can go away when I start running Fedora 15. The libvirt in f15 has native spice support, and it was not too difficult to convert the .xml files I was using on f14 to include the native spice support and remove the silly wrapper invocation. The f14-winxppro.xml file is an example of the KVM the way it is defined in Fedora 14 using the wrapper, and f15-winxppro.xml is the converted file that runs a copy of the exact same image with native spice support in fedora 15. The conversion seemed painless. The Windows XP virtual machine apparently didn't notice any change at all, working identically under fedora 15 as it did under 14. |