So, my main virtualisation machine here at home is a Thinkpad R60 running Debian Lenny, with KVM as the virtualisation engine. This has been my main ‘server’ host for over a year and a half now. I have quite a few VMs on it, but most are for testing out stuff, so there is only one or two VMs that are active constantly. One is a Debian Lenny guest running Asterisk for the most part. The other is a Windows 7 VM. I’ve been reasonably happy with this setup. Key positive points to me are;

• I can use the host as a regular linux box with a graphical desktop
• Performance is OK
• The Debian Lenny guest is idle most of the time, and is lucky to consume 2 or 3% of the host cpu.
• It all works with a 32 bit VT capable CPU such as the T2400 core duo in the R60

Negatives of this setup are;

• The T2400 is only 32 bit so no 64 bit guests.
• Physical host CPU use when the Windows 7 guest runs is high; 14% or so. I eventually tracked this down to the USB tablet driver. Once removed, host CPU use for the related kvm process is about 2 to 3% for an idle Windows 7.
• You cannot do easy snapshots and reverts like you can say with Vmware’s products.

So Debian Squeeze is the new stable ‘Debian’, so I have attempted to upgrade my system. I’ve actually done this now several times, first using some of the ‘pre stable’ releases of Squeeze, as well as the current stable Squeeze. Each time I have reverted back to Lenny. Key negatives of Squeeze are;

• Host CPU use seems higher for all guests for me.
• The extra CPU use by the USB tablet driver in a Windows guest still seems to be there. Makes a big difference if you remove it.
• The Windows 7 guest now seems to BSOD (ie. crash) when you shut it down (I’m just using a virtual IDE disk and a virtual RTL3129)
• The Laptop fan seems to stay on more often now.

So this has had me looking at alternatives. I am a big fan of Vmware ESXi, but because of the 32 bit CPU in the R60 I am limited to ESXi 3.5 on it. With that in mind, I did my research and discovered that you can put a T7200 core 2 duo CPU into my R60 which is a 64 bit VT capable CPU. So I ordered one off eaby and have had it running in it for a few days now.

So here’s a quick breakdown of what I’ve tried lately. Some of these I’ve only set up on my desktop core 2 duo system, but I intend to try them all on the R60

• Squeeze 64 bit. Much the same as the problems I’ve had with Squeeze mentioned above.
• ESXi 3.5u5. This works pretty well. On the R60, all the hardware works, except when you do a shutdown on the console, you get a purple screen of death. Reboots are OK though.
• ESXi 4.0 (I will try 4.1 shortly, but I had the 4.0 CD handy). That works pretty good too. The main negatives for ESXi on this box are; 1) no graphical console anymore, 2) one disk has to ‘generally’ be set aside as ESXi vm storage, 3) I have to use Windows to administer it, 4) I can’t read any of the Thinkpad thermal or fan information.
• Scientific Linux 6.0 64 bit. I tried the ‘LiveMiniCD’ which is pretty good. It installs an IceWM desktop. I’ve been running my Asterisk VM and my Windows 7 VM on it under kvm. In this case CPU use for both is low, but the Asterisk VM CPU use is still quite a bit higher than my Lenny setup. Also the Windows 7 VM crashes on shutdown just like under Squeeze. Scientific Linux 6 and Squeeze both use a very similar version of qemu-kvm. I might persists a bit of Scientific Linux. It doesn’t seem too bad (for a Redhat clone). One problem I had with SL 6.0 was a strange flicker on the X desktop every 10 secs. Changing the GRUB kernel args to include ‘nomodeset’ (ie. turn off KMS), cured this.
• Xenserver 5.6 FP1 . I haven’t tried it on the R60, just my desktop core 2 duo box. This I thought worked pretty well … and Citrix have done a lot to make it ‘feel’ just like ESXi for the most part, and the Windows admin tool seems to let you do most stuff that Vmware’s admin client let’s you do. I did have problems installing the guest tools into a Windows 7 guest though. This was a week or too back and I can’t quite remember what it was. But I was impressed by it in general, and I’d like to spend more time looking at it (or XCP below)
• XCP 1.0. This to me looked to be almost exactly the same as Xenserver 5.6 FP1. It had exactly the same problems re the guest tools in Windows 7.

So still undecided at the moment. I need to do a bit more research on the ESXi front. The R60 can only handle 3GB of RAM, and I think the newer ESXi’s have quite a large default memory footprint (but I think there is a way to reduce it a bit). If I could sort out the problem with the Windows 7 BSOD on shutdown then I might consider Scientific Linux. Lot’s of options.

UPDATE (2011.5.2). The Windows 7 BSOD I was getting when performing a shutdown on Scientific Linux or Squeeze was a IRQL_NOT_LESS_OR_EQUAL. I think I’ve found what causes this. All my VMs are created with i686 as the CPU type in the libvirt xml files (since my old T2400 chip was 32 bit only). Of course now I have a 64 bit capable chip (T7200) and the new versions of qemu-kvm don’t seem to like me specifying a i686 for my Windows 7 32 bit instance. If I change the xml file to use x86_64 instead of i686, I can now (finally) gracefully shut the Windows 7 32 bit VM down.;

<type arch=’x86_64′ …

Another useful thing to keep my CPU utilisation low is to disable the memory balloon (which I discovered is enabled by default in more recent libvirt’s). You just need to add this to the <devices> section of your libvirt xml file;

<memballoon model=’none’/>