(Note: This post was initially written when ESXi 4.0 was available. As of late 2010, ESXi 4.1 has been released, and it does actually include a sky2 driver that may or may not work with various Marvell LAN chipsets. The post is still relevant (especially the comments)  if your particular Marvell chipset does not work with the sky2 driver in ESXI 4.1. Also, the post is relevant if you’re interested in porting other network drivers to ESXi)

Well, after somehow getting my Marvell LAN card working with ESXi 3.5u4 (and u3) I thought I’d have a look at  ESXi 4. Again I somehow got it to go. I’m not too sure how good it works, but it works well enough for me at home. If you can’t be bothered reading about me going on and on and on and on about how to compile it, then just scroll to the bottom of the post. The download for the source includes a precompiled module (NB: As per the post about ESXi 3.5, this is all about getting a 88E8053 chipset Marvell LAN working).

ESX/ESXi 4 is quite different from 3.5. The build chain is similar to 64 bit Redhat/Centos 5.2, so I ended up installing a x86_64 Centos 5.3 inside a vmware fusion machine to do my dev work. I just made sure I installed all the dev stuff.  Then I downloaded the VMware-esx-public-source-4.0-162945.tar.gz from VMware’s open source page. It’s a much bigger file (590MB) than the file for 3.5. When you extract the file you end up with a lot of rpm files plus a vmkdrivers-gpl.tgz file. I did the following to extract it all on my test machine;

cd ~
mkdir vmware-oss
cd vmware-oss
tar xvzf ~/VMware-esx-public-source-4.0-162945.tar.gz
mkdir drivers
cd drivers
tar xvzf ../vmkdrivers-gpl.tgz

One of the rpm files included is a kernel source rpm. I’m not exactly sure what it is relevant to ESX, but I installed it anyway for reference. I found I needed the qt-devel and gtk2-devel packages first;

cd ~
cd vmware-oss
yum install qt-devel
yum install gtk2-devel
rpm -iv kernel-sourcecode-400.2.6.18-128.1.1.0.4.159770.x86_64.rpm

I’m pretty sure you don’t need the kernel source to build the drivers, but I kept it handy for reference anyway.

You can try doing a test build of the drivers now. This will build all the drivers built in to ESX/ESXi.

cd ~/vmware-oss/drivers
./build-vmkdrivers.sh

You’ll probably get a few warnings, but it should complete. If you do a find down the ‘bora’ directory you should see a bunch of .o files corresponding to the kernel modules (look under the ‘bora/build/scons/build’ directory).

OK, my approach was to look at the build-vmkdrivers.sh script and basically look at what was done to compile one network driver (I used the forcedeth driver as a reference) and just make a reduced script for my sky2 driver.  As for the source to base the sky2 driver on, instead of using a driver from 2.4.37 like I did with the 3.5 version of the network driver, this time I ended up using the sky2 source from 2.6.26 (or the debian lenny incantation of it). I did originally use the sky2 driver from the kernel-sourcecode…2.6.18…rpm file, but on closer inspection of the tg3 driver that ESX uses, I noticed it actually comes from a 2.6.24.1 kernel (or thereabouts) … so I thought I may as well use a more modern reference source. I had a few minor hiccups trying to get it to compile, but in the end I just had the following shoved into the top of my sky2.c file;

/* Stuff for ESX compile */
#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
#define csum_offset csum
#define bool int
#define PTR_ALIGN(p, a)         ((typeof(p))ALIGN((unsigned long)(p), (a)))

u32 bitreverse(u32 x)
{
        x = (x >> 16) | (x << 16);
        x = (x >> 8 & 0x00ff00ff) | (x << 8 & 0xff00ff00);
        x = (x >> 4 & 0x0f0f0f0f) | (x << 4 & 0xf0f0f0f0);
        x = (x >> 2 & 0x33333333) | (x << 2 & 0xcccccccc);
        x = (x >> 1 & 0x55555555) | (x << 1 & 0xaaaaaaaa);
        return x;
}

The define’s are to remedy compilation errors, and the bitreverse is to satisfy an undefined symbol problem. Note that the undefined symbols errors end up in /var/log/messages on your ESXi 4 box now (unlike 3.5).

So in the end to compile my driver I did a;

./build-sky2.sh

You get a couple of warnings, but if you do a ‘find . -name sky2.o’ you should end up with two sky2.o files. There is a DEBUG and DASHG variable defined at the top of the build script. If you uncomment these it’ll build a lot of debug stuff into the modules.

If you get some errors, its probably because some directories are missing in the build path, so make them first;

mkdir -p bora/build/scons/build/vmkdriver-sky2.o/release/vmkernel64/SUBDIRS/vmkdrivers/src26/drivers/net/sky2
mkdir -p bora/build/scons/build/vmkdriver-sky2.o/release/vmkernel64/SUBDIRS/vmkdrivers/src26/common/

Now, again I used a USB stick with ESXi 4 (build 171294 in my case). To install it, I loopback mounted the VMware ESXi 4 iso file file, extracted the image.tgz file to a temp directory, bunzip2’d the big dd image file, then dd’d it to the whole USB stick (NB: There are some details about how to do this in linux on my other post re ESXi 3.5. See link at the top of the post)

A difference this time is that I didn’t have a simple.map file all pre-prepared to go into the oem.tgz file. I thought the easiest way to get it would be to just boot ESXi and let it fail when it tries to configure a network device, then somehow copy the simple.map file off. So I did this. ESXi 4 merrily boots and eventually you see the dreaded ‘lvmdriver failed’ message. It looks like ESXi is broken at that point, but just type the word ‘unsupported’ and you get a password prompt, and just hit ENTER to get a prompt (You might need to hit alt-f1 first before typing ‘unsupported’)

Because networking is not working, we’ll just copy the simple.map to the Hypervisor1 partition;

cp /etc/vmware/simple.map /vmfs/volumes/Hypervisor1

I just did a ‘sync’ and held in  the power switch ( perhaps type ‘reboot’ if you feel like being more careful). Now get the USB stick to appear as a USB device in your development VM (your centos 5.x environment), and mount partition 5 (or the Hypervisor1 partition) off the USB drive, and you should see an oem.tgz file as well as the simple.map file. You need to make a directory structure up for the new oem.tgz file we’ll be creating;

cd ~
mkdir vmtest
cd vmtest
mkdir -p etc/vmware
mkdir -p usr/lib/vmware/vmkmod

Copy the simple.map off the USB drive into etc/vmware directory in our tree structure. eg.

cp /mnt/simple.map ~/vmtest/etc/vmware

And edit the simple.map so that it includes the PCI ids for your Marvell card. Mine is 11ab:4362 so I added in the bolded line below, but yours could likely be different. If you’re not sure, you could boot ESXi off the USB stick again, do the ‘unsupported’ thing to get a prompt and type lspci -v

1166:0410 0000:0000 storage sata_svw.o
1166:0411 0000:0000 storage sata_svw.o
11ab:4362 0000:0000 network sky2.o
14e4:1600 0000:0000 network tg3.o

Now copy in the sky2.o file that we compiled earlier. The modules are in a different directory compared to 3.5 (NB: the compilation process produces two sky2.o files, so make sure you grab the one shown below)

cd ~/vmware-oss/drivers
cp ./bora/build/scons/build/vmkdriver-sky2.o/release/vmkernel64/sky2.o ~/vmtest/usr/lib/vmware/vmkmod

Now tar it up, and copy it to the USB stick that should be still mounted;

cd ~/vmtest
tar cvzf ~/oem.tgz *
cp ../oem.tgz /mnt

Unmount the USB stick

umount /mnt

Now try booting again. Hopefully you should see a ‘loading sky2’ flash up early in the boot … and it should eventually get to the usual ESX status screen showing the current mgmt IP address. Basically if you don’t see the ‘lvmdriver load failed’ then there’s a good chance it’s working.

And yes here is the sky2-for-esxi4-0.01.tar.gz download. It includes the build script, the modified source, plus directory tree for creating the oem.tgz file including a pre-compiled copy of the module. If you can’t be bothered compiling, you can just extract this file, cd to the vmtest directory and create the oem.tgz file as per the earlier notes.

UPDATE: (2010/02/08) There is also now a driver for the Marvell 88E8001 LAN chipset (see the comments discussion below). This uses the skge driver, not the sky2 driver mentioned above. I don’t own a 88E8001, so thank you to samarium for helping out re the unresolved symbols. Please comment below if you’ve tried the skge driver and it works. I have a new tarball sky2-and-skge-for-esxi4-0.02.tar.gz containing both the sky2 and skge driver

UPDATE: (2010/12/24). xieliwei has a driver that should support the 88E8057 on ESXi 4.1 (as per the latest comments, it seems a bit iffy on 4.0). Anyone else who has this chipset, if you could try the driver and post more information that would be great. The 88E8057 code is here as a local copy ; sky2-1.22-for-esxi4-88e8057-r1-xieliwei.tar.gz or on mediafire; sky2-1.22-for-esxi4-88e8057-r1-xieliwei.tar.gz . Also there’s another version of the driver that will produce copious debug information (only of use if you’re having problems); http://www.mediafire.com/file/m836gce3r3b7ygi/sky2-1.22-for-esxi4-88e8057-r1-oem-debug-xieliwei.tgz