Using a Marvell LAN card with ESXi 4

(Note: This post was initially written when ESXi 4.0 was available. As of late 2010, ESXi 4.1 has been released, and it does actually include a sky2 driver that may or may not work with various Marvell LAN chipsets. The post is still relevant (especially the comments)  if your particular Marvell chipset does not work with the sky2 driver in ESXI 4.1. Also, the post is relevant if you’re interested in porting other network drivers to ESXi)

Well, after somehow getting my Marvell LAN card working with ESXi 3.5u4 (and u3) I thought I’d have a look at  ESXi 4. Again I somehow got it to go. I’m not too sure how good it works, but it works well enough for me at home. If you can’t be bothered reading about me going on and on and on and on about how to compile it, then just scroll to the bottom of the post. The download for the source includes a precompiled module (NB: As per the post about ESXi 3.5, this is all about getting a 88E8053 chipset Marvell LAN working).

ESX/ESXi 4 is quite different from 3.5. The build chain is similar to 64 bit Redhat/Centos 5.2, so I ended up installing a x86_64 Centos 5.3 inside a vmware fusion machine to do my dev work. I just made sure I installed all the dev stuff.  Then I downloaded the VMware-esx-public-source-4.0-162945.tar.gz from VMware’s open source page. It’s a much bigger file (590MB) than the file for 3.5. When you extract the file you end up with a lot of rpm files plus a vmkdrivers-gpl.tgz file. I did the following to extract it all on my test machine;

cd ~
mkdir vmware-oss
cd vmware-oss
tar xvzf ~/VMware-esx-public-source-4.0-162945.tar.gz
mkdir drivers
cd drivers
tar xvzf ../vmkdrivers-gpl.tgz

One of the rpm files included is a kernel source rpm. I’m not exactly sure what it is relevant to ESX, but I installed it anyway for reference. I found I needed the qt-devel and gtk2-devel packages first;

cd ~
cd vmware-oss
yum install qt-devel
yum install gtk2-devel
rpm -iv kernel-sourcecode-400.2.6.18-128.1.1.0.4.159770.x86_64.rpm

I’m pretty sure you don’t need the kernel source to build the drivers, but I kept it handy for reference anyway.

You can try doing a test build of the drivers now. This will build all the drivers built in to ESX/ESXi.

cd ~/vmware-oss/drivers
./build-vmkdrivers.sh

You’ll probably get a few warnings, but it should complete. If you do a find down the ‘bora’ directory you should see a bunch of .o files corresponding to the kernel modules (look under the ‘bora/build/scons/build’ directory).

OK, my approach was to look at the build-vmkdrivers.sh script and basically look at what was done to compile one network driver (I used the forcedeth driver as a reference) and just make a reduced script for my sky2 driver.  As for the source to base the sky2 driver on, instead of using a driver from 2.4.37 like I did with the 3.5 version of the network driver, this time I ended up using the sky2 source from 2.6.26 (or the debian lenny incantation of it). I did originally use the sky2 driver from the kernel-sourcecode…2.6.18…rpm file, but on closer inspection of the tg3 driver that ESX uses, I noticed it actually comes from a 2.6.24.1 kernel (or thereabouts) … so I thought I may as well use a more modern reference source. I had a few minor hiccups trying to get it to compile, but in the end I just had the following shoved into the top of my sky2.c file;

/* Stuff for ESX compile */
#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
#define csum_offset csum
#define bool int
#define PTR_ALIGN(p, a)         ((typeof(p))ALIGN((unsigned long)(p), (a)))

u32 bitreverse(u32 x)
{
        x = (x >> 16) | (x << 16);
        x = (x >> 8 & 0x00ff00ff) | (x << 8 & 0xff00ff00);
        x = (x >> 4 & 0x0f0f0f0f) | (x << 4 & 0xf0f0f0f0);
        x = (x >> 2 & 0x33333333) | (x << 2 & 0xcccccccc);
        x = (x >> 1 & 0x55555555) | (x << 1 & 0xaaaaaaaa);
        return x;
}

The define’s are to remedy compilation errors, and the bitreverse is to satisfy an undefined symbol problem. Note that the undefined symbols errors end up in /var/log/messages on your ESXi 4 box now (unlike 3.5).

So in the end to compile my driver I did a;

./build-sky2.sh

You get a couple of warnings, but if you do a ‘find . -name sky2.o’ you should end up with two sky2.o files. There is a DEBUG and DASHG variable defined at the top of the build script. If you uncomment these it’ll build a lot of debug stuff into the modules.

If you get some errors, its probably because some directories are missing in the build path, so make them first;

mkdir -p bora/build/scons/build/vmkdriver-sky2.o/release/vmkernel64/SUBDIRS/vmkdrivers/src26/drivers/net/sky2
mkdir -p bora/build/scons/build/vmkdriver-sky2.o/release/vmkernel64/SUBDIRS/vmkdrivers/src26/common/

Now, again I used a USB stick with ESXi 4 (build 171294 in my case). To install it, I loopback mounted the VMware ESXi 4 iso file file, extracted the image.tgz file to a temp directory, bunzip2’d the big dd image file, then dd’d it to the whole USB stick (NB: There are some details about how to do this in linux on my other post re ESXi 3.5. See link at the top of the post)

A difference this time is that I didn’t have a simple.map file all pre-prepared to go into the oem.tgz file. I thought the easiest way to get it would be to just boot ESXi and let it fail when it tries to configure a network device, then somehow copy the simple.map file off. So I did this. ESXi 4 merrily boots and eventually you see the dreaded ‘lvmdriver failed’ message. It looks like ESXi is broken at that point, but just type the word ‘unsupported’ and you get a password prompt, and just hit ENTER to get a prompt (You might need to hit alt-f1 first before typing ‘unsupported’)

Because networking is not working, we’ll just copy the simple.map to the Hypervisor1 partition;

cp /etc/vmware/simple.map /vmfs/volumes/Hypervisor1

I just did a ‘sync’ and held in  the power switch ( perhaps type ‘reboot’ if you feel like being more careful). Now get the USB stick to appear as a USB device in your development VM (your centos 5.x environment), and mount partition 5 (or the Hypervisor1 partition) off the USB drive, and you should see an oem.tgz file as well as the simple.map file. You need to make a directory structure up for the new oem.tgz file we’ll be creating;

cd ~
mkdir vmtest
cd vmtest
mkdir -p etc/vmware
mkdir -p usr/lib/vmware/vmkmod

Copy the simple.map off the USB drive into etc/vmware directory in our tree structure. eg.

cp /mnt/simple.map ~/vmtest/etc/vmware

And edit the simple.map so that it includes the PCI ids for your Marvell card. Mine is 11ab:4362 so I added in the bolded line below, but yours could likely be different. If you’re not sure, you could boot ESXi off the USB stick again, do the ‘unsupported’ thing to get a prompt and type lspci -v

1166:0410 0000:0000 storage sata_svw.o
1166:0411 0000:0000 storage sata_svw.o
11ab:4362 0000:0000 network sky2.o
14e4:1600 0000:0000 network tg3.o

Now copy in the sky2.o file that we compiled earlier. The modules are in a different directory compared to 3.5 (NB: the compilation process produces two sky2.o files, so make sure you grab the one shown below)

cd ~/vmware-oss/drivers
cp ./bora/build/scons/build/vmkdriver-sky2.o/release/vmkernel64/sky2.o ~/vmtest/usr/lib/vmware/vmkmod

Now tar it up, and copy it to the USB stick that should be still mounted;

cd ~/vmtest
tar cvzf ~/oem.tgz *
cp ../oem.tgz /mnt

Unmount the USB stick

umount /mnt

Now try booting again. Hopefully you should see a ‘loading sky2’ flash up early in the boot … and it should eventually get to the usual ESX status screen showing the current mgmt IP address. Basically if you don’t see the ‘lvmdriver load failed’ then there’s a good chance it’s working.

And yes here is the sky2-for-esxi4-0.01.tar.gz download. It includes the build script, the modified source, plus directory tree for creating the oem.tgz file including a pre-compiled copy of the module. If you can’t be bothered compiling, you can just extract this file, cd to the vmtest directory and create the oem.tgz file as per the earlier notes.

UPDATE: (2010/02/08) There is also now a driver for the Marvell 88E8001 LAN chipset (see the comments discussion below). This uses the skge driver, not the sky2 driver mentioned above. I don’t own a 88E8001, so thank you to samarium for helping out re the unresolved symbols. Please comment below if you’ve tried the skge driver and it works. I have a new tarball sky2-and-skge-for-esxi4-0.02.tar.gz containing both the sky2 and skge driver

UPDATE: (2010/12/24). xieliwei has a driver that should support the 88E8057 on ESXi 4.1 (as per the latest comments, it seems a bit iffy on 4.0). Anyone else who has this chipset, if you could try the driver and post more information that would be great. The 88E8057 code is here as a local copy ; sky2-1.22-for-esxi4-88e8057-r1-xieliwei.tar.gz or on mediafire; sky2-1.22-for-esxi4-88e8057-r1-xieliwei.tar.gz . Also there’s another version of the driver that will produce copious debug information (only of use if you’re having problems); http://www.mediafire.com/file/m836gce3r3b7ygi/sky2-1.22-for-esxi4-88e8057-r1-oem-debug-xieliwei.tgz

118 Responses to “Using a Marvell LAN card with ESXi 4”

  1. mav1, OK, I had a bit of look into this. I guess the driver you are trying is based on the GPL source that D-link released. I find the built in drivers that come with a linux kernel are a better source when you are trying to adapt a driver for ESXi. Basically, ESXi ‘looks like’ an older 2.6 kernel. So, like a lot of my attempts I used a driver from 2.6.26. From the research I did it looks like the via-rhine driver in the normal linux kernel is meant to work with your DFE 520TX card, but I can’t really confirm that since I don’t have such a card. So I adapted the 2.6.26 via-rhine driver so that it would compile. I had to comment out a heap of mii related stuff in the source to get rid of a lot of unresolved symbols. My thoughts are that the card may still configure itself with these bits missing and might work ‘poorly’ … but again, I’m not too sure until you try it. I also had to comment out some code related to ‘Rhine I’ cards that are based on the VT86C100A chip. I don’t think your DFE card has one of those chips. I’ve uploaded via-rhine-for-esxi4-0.01.tar.gz. That contains a compiled driver, source and an example simple.map. I took a guess and guessed that your PCI IDs (for the simple.map) are 1106:3106. If they’re different, you’ll need to edit the simple.map and change the line with 1106:3106 on it to match your PCI IDs. NB: This is just what I would call a ‘first attempt’. Don’t expect too much.

  2. Hello all. I hope my request that I’m about to make doesn’t annoy or frustrate anyone but this is my dilemma, I’m not Linux savvy at all I’m more of a GUI guy but i know this whole process is all console work. I have a Marvell Yukon 88e8056 with a PID of 11AB:4364. Is it possible for someone to create the files for me and i replace them in the ISO and install? or does it have to be installed and then i replace the files? Any bit of guidance is appreciated. thanks all.

  3. Hello kernel,

    I’m test your driver. It’s works!!! Thank you very much for help.

    With best wishes.

  4. mav1, that’s great news. You should try and copy some large files across the network to get a feel for the performance. Like I said, I commented out a lot of the mii stuff in the driver .. which is often used for setting duplex modes etc and ultimately I wasnt sure just how critical those bits were. If the duplexing is wrong, your network performance will likely suffer.

  5. Jinx, maybe someone else reading this page can you help you. I’m a bit too busy to provide much help at the moment. I guess I always found the USB stick install of ESXi to be pretty easy, and if you google you can probably find some Windows howtos on how to set it up (I’m assuming you’re a Windows user. Apologies if you’re not). The key benefit of installing to USB stick is that you can read and write to it easily. I haven’t tried plugging the USB stick into Windows, but the main partitions on it that you need to copy/edit things on are all FAT partitons, so Windows should be able to mount them. There’s a few comments above by darkdragon that might help point you in the right direction. re your 88E8056 chipset NIC. That’s very similar to the 88E8053 on the motherboard that I have … and I can see that the sky2 driver thing will most likely work … assuming you can create your oem.tgz and get it onto the USB stick. Have a look at my post on setting up the Marvell LAN on ESXi 3.5 (linked to in the main article above). There’s a bunch of stuff in there that is quite relevant … and might help point you in the right direction.

  6. Hi,
    Not working here… Using Marvell Yukon 88E8057 (Galileo Tech Ltd). Using sky driver and module loads but esxcfg-vmknic -l returns nothing. PID is 11ab:4380. Any thoughts would be very appreciated.

    Regards

  7. naik, there’s a PCI ID table near the start of the sky2.c file. It lists all the card IDs. It didn’t include 4380, so I’ve added it in and recompiled. I’ve uploaded sky2-and-skge-for-esxi4-0.03-test1.tar.gz which includes the new sky2.o. Do you want to give that a try and see how you go.

  8. I will and get back to you, many thanks

  9. Hi
    The driver loads but produce the error:
    sky2 0000:03:00.0 unsupported chip type 0xba

    lspci shows the module is loaded for 11ab:4380

    I have a fedora kernel driver source for 2.6.x if that helps:

    http://www.syscore.se/install_v10.85.9.3.tar.bz2

    regards

  10. OK, I’ve added in some more of the chipset detection stuff from the later drivers. Can you try this; sky2-and-skge-for-esxi4-0.03-test2.tar.gz

  11. OK.
    The driver loads. The chip and the two controllers are detected.
    But it seems any protocol access to the host hangs the server.
    I’ve been reading about workarounds regarding the chip and tried the following:

    “ethtool -K vmnic0 sg off” and “ethtool –offload vmnic0 rx off tx
    off sg off tso off” but get the error “function not implemented”

    I can ping the host but thats it. Unfortunately it may be related to the previous post:http://www.mail-archive.com/netdev@vger.kernel.org/msg39532.html

    Any thoughts?
    Regards

  12. Try this one ; sky2-and-skge-for-esxi4-0.03-test3.tar.gz . Only has some minor changes from a later linux driver.

  13. Now I get som more specific error:
    Warning: LinNet: netdev_watchdog vmnic0: transmit timed out
    vmnic0:tx timeout
    transmit ring 477..41 report=41 done=41
    BUG:warning at vmkdrivers/src_v4/vmklinux26/vmware/linux_net.c:3239/netdev_watchdog() (inside vmklinux)
    sky2 vmnic0 disabling interface

    ..tx, rx, sg, autoneg do not work with ethtool: function not implemented

    regards

  14. I’m having the same problem in ESXi 4.1 with Marvell Yukon 88E8056

    “netdev_watchdog vmnic0: transmit timed out”

  15. sorry, it’s 88E8057

  16. Hi again,

    Do you have any more input regarding the Marvell Yukon that could push my case forward?

    I have a question of a more private matter, if you want you can reach me at this temporay email address:

    MgRQcqbcjTy5i3Rj@spam.seydisehirmyo.net

    Regards

  17. Sorry, I guess I’m coming to the conclusion that there is something a bit harder to work out with your card. I’ll try to have another look, but I won’t have much time until the weekend.

  18. I have an Intel Entry Server Board (SE7221BA1-E) with an on-board Marvel 88E8050 card (11AB:4361).

    Currently it is running with VMware ESXi 4.1.0 260247 with your sky2.o driver. All that was required to make it work was add the following line to the simple.map file:

    11ab:4361 0000:0000 network sky2.o

    Thank you very much for your excellent work porting this driver to ESXi.

  19. I have done some more digging around the 88E8057 driver. Earlier the chip was supported under the skge and sk98lin packages. When the code was introduced in sky2 compíled with different kernels it stopped working, for most of the distributions. Here are some sucess stories:
    https://forums.openfiler.com/viewtopic.php?id=4894
    http://linux.derkeiler.com/Newsgroups/comp.os.linux.networking/2009-02/msg00110.html
    http://tima-sls.imag.fr/viewgit/rabbits/?a=viewblob&p=Linux&h=daf961ab68bc27e3fa16ddea89784aa0091b42e2&f=drivers/net/sky2.c

    Unfortunately that’s all I can help you with. I understand your time is limited and are greatful for your support. I’ve also been in contact with Marvel and are awaiting their answer. My earlier email address didn’t work. Here’s a new one: marvellyukon2010@live.se

  20. naik,

    I’ve gone through a much later kernel (2.6.33.7). Try this one ; sky2-and-skge-for-esxi4-0.03-test4.tar.gz .

  21. Hello Kenel, your post about ESXi kernel module literally blowed up whole VMWare Community, THX for that!
    Currently Im trying linux soft raid to get work with ESX, could you please tell your opinion about it, is it possible?

  22. Here is my output when Im trying to load module on ESX:

    [root@ESX ~]# vmkload_mod -v100 vmkdriver-raid1-KLnext/release/vmkernel64/raid1.o
    Verbose:100
    After loop: argc:3 optind:2 optarg:(null)
    modParams is || (0)
    nameIn:vmkdriver-raid1-KLnext/release/vmkernel64/raid1.o
    nameOut:raid1 pathOut:vmkdriver-raid1-KLnext/release/vmkernel64/raid1.o
    fd:4 size:-2346392 st_size:1096059
    VmkModSign_GetSignInfo starting
    vmsignContent does not exist
    name:raid1 addr:0xf7b65000 size:0x10c000 modParams:
    vmkmod: failed at bora/lib/vmkmod/vmkmod.c:VMKMod_LoadModule:240
    vmkmod: failed at bora/lib/vmkmod/vmkmod.c:VMKMod_Load:437
    vmkload_mod: Can not load module vmkdriver-raid1-KLnext/release/vmkernel64/raid1.o: Unresolved symbol

  23. Some time back I did look into porting the md raid driver stuff in linux to ESXi … but quickly concluded it would be a lot of hard work. I haven’t looked at it since.

    Possibly not what you’re after … but there are some interesting sata chipsets made by Jmicron that can seemingly do proper hardware RAID, such as the JMicron 393. They aren’t the cheapest, nor are they easy to find, but they are more likely to come down in price compared to say an Adaptec controller. Here’s a review of two of those Jmicron 393 cards. Admittedly I have not tried out one of these cards.

  24. hi @kernel
    thanks for all your hard work on the 88E8057 card so far.
    i have a Shuttle SX58J3 which has two of them on board.

    using the latest sky2-and-skge-for-esxi4-0.03-test4.tar.gz driver you kindly provided us, the cards are recognized, but do not answer to pings neither they get an IP from DHCP.

    is there something that would help you investigating? some kind of debug output or whatever?
    Do you have such a NIC in an ESXi compatible system? would probably be helpful. seems like a good reason to donate your work! where is the paypal button? 😉

  25. Thanks for the feedback on the sky2-and-skge-for-esxi4-0.03-test4.tar.gz driver. I too am puzzled why this 88E8057 is so hard to make work. From your comment, I am assuming that the module loads OK and esxcfg-nics -l returns what looks like a configured interface.

    I only have an old motherboard with an 88E8053 on it, so any updates to the driver are in many ways ‘compiling blind’ since I don’t have a physical 88E8057 card or motherboard to test with.

    One thing I’m interested in finding out is whether the card actually does work on a recent linux kernel (if you’re using windows, you could just try booting a recent Ubuntu live CD and see whether you have network connectivity … without actually installing linux). The recent linux sky2 driver that I’m using as reference appears to support the 88E8057 … but just because it’s mentioned in the source code doesn’t mean it actually works.

  26. i can confirm now, that it works in a ubuntu 10.4.1 live cd with kernel 2.6.32-24-generic

    i’m trying out no an older version, like 8.04 and also the latest knoppix.
    if there is anything else i can do, just tell me. i’m using windows but am quite familiar with linux, so i could also install a CentOS or whatever if it helps.

  27. forzyte, Thanks for the feedback re the 10.4.1 live CD. Given that I based that test4 code on 2.6.33.7, I’m guessing there is something a bit more fundamental missing, in order for it not to work on ESXi. I’ll have another look at the code. There are always parts of the driver that I end up commenting out (which I gauge as non-critical), because some function calls have changed or don’t exist in the ESXi kernel. Might be the weekend before I get much of a chance to look at it.

  28. hi, i read that some of you guys got marvell yukon 8056 running.
    i got an asus p7f-x. i’ve copied the imagedd of esxi 4.1 to an usb stick. now i can start the server an esxi comes up and tells that it cant find any compatible nics.
    i downloaded sky2 driver and added it to lib path in oem.tgz. of course i added simple.map and the PID.
    now esxi outputs the same message. it cannot find any nics.
    i looked up simple map on real path and for me it seems that the system isn’t copying the new simple.map file.
    the sky2 is copied!

    any idea, someone?

    thanks!

  29. another thing: i did not mount the usb stick to an unix. i worked directly on the esxi console!

  30. irise, I am a bit puzzled how you created the oem.tgz from the esxi console. Did you somehow copy the sky2.o and simple.map from Windows first into one of the FAT partitions on the USB stick, then try and tar them up from on the ESXi console?

  31. i’ve dumped imagedd to usb stick via winimage.

    but now i got it running after editing Hypervisor1 partition and oem.tgz with an ubuntu live cd.

    now i got another problem. esxi won’t get an dhcp lease. and if i configure it manually, i’m neither able to ping it via crossover cable nor via LAN. esxi isn’t able to ping anything, too.

    any idea?

    i used sky2.o and interface state is connected. i also use dmraid for intel ich8r support in oem.tgz.

  32. i’ve found the following lines in message log:

    pci: driver sky2 claimed device …

    vmnic0 not yet opened

    pci: driver sky2 claimed 2 devices

    uplink: 12981: opening device vmnic0

    sky2 vmnic0: enabling the interface
    netport: 982: enabled port 0x2 with mac 00:00:00:00:00:00
    uplink: 131119: vmnic0 is opened

    mod: 4163: initialization of sky2 succeeded with module ID31.
    sky2 loaded successfully.

    ALERT: Elf: 3028: Kernel module sky2 was loaded, but has no signature attached

    same with vmnic1 in between. i’ve only posted messages that seems important for me.

    i donno how to set up the interface with a valid mac address.
    is there something like ifconfig on esxi console?

  33. irise, Have you tried disabling one of the nics in the BIOS? (just to see if it configures properly in that case). Also, there are multiple attempts at updates to the sky2 driver if you look back through the comments above. The latest one (sky2-and-skge-for-esxi4-0.03-test4.tar.gz) is not necessarily the best one for your card. I’d probably try all of them.

    If you look back at the comments above, I can see that samarium had a 88E8056 in his PC, but the discussion at that point was all about getting a 88E8001 running using the skge driver. But there are comments by eo29 who had a P5Q deluxe board (which has a 88E8056 and a 88E8001 I think) … and he confirmed he was able to get both to work.

  34. hi, at this moment i am using sky2-and-skge-for-esxi4-0.03-test4.tar.gz.
    i will try older ones in the next days.
    at vm-help.com forum i’ve read that some guys got 8056 running with sky2.
    i will test and confirm weather that’s right or wrong.
    thanks.

  35. Hi, i got it running with sky2 out of sky2-0.02 package.
    now i want to get raid1 running on p7f-x.
    but dmraid is not working with ich8r.
    think i have to buy a pci raid module.

  36. irise, thanks for the feedback. The sky2-and-skge-for-esxi4-0.03-test1/2/3/… are all various attempts to add in support for the 88E8057 card. There has been a lot of rewriting in each, since in each case so far my attempts did not work, so maybe I’ve introduced a bug into those 0.03 ones that prevent the whole driver from working.

    Anyway, for reference for anyone else reading these comments; the sky2-and-skge-for-esxi4-0.02.tar.gz should support a bunch of chipsets including the 88E8053, 88E8001, 88E8056 (and possibly other Marvell nics). But the sky2-and-skge-for-esxi4-0.03-test1/2/3… downloads are various atempts to get support for the 88E8057. As it stands none of these work correctly for the 88E8057.

  37. I was able to install ESXi 4.1 on a Macmini2,1 with your sky2 driver. Thanks a million!

  38. Ever tried using the marvell provided sk98lin driver? I’m trying to get a 88E8057 working as well.

  39. I have had a brief look at the marvell sk98lin driver. It looked significantly different to the sky2 and skge drivers in the linux kernel, so I didn’t look any further (ie. it looked like a painful effort to port it). I still think there might be some minor error in some of my more later attempts to get the driver working on the 88E8057 (ie. the sky2-and-skge-for-esxi4-0.03-test1/2/3 downloads), as I think these later ones don’t even work with the older Marvell chipsets that do work with the sky2-and-skge-for-esxi4-0.02.tar.gz version). For reference (from memory) the 0.02 release uses a modified driver from a 2.6.26 kernel as this is ‘close’ to the linux-ish kernel in ESXi, and all the 0.03 test versions incorporate code from later linux kernel versions where supposedly the 88E8057 is supported.

  40. Yes, it does look very different with all the compilation units. I was thinking just linking all the resultant objects together would do, but don’t have the skills to do it.

    What is the latest sky.c version (its defined in the file) that you’ve tried? I’m thinking of working my way down from 1.28 (that’s where new DMA defines were introduced, which would require kernel level modification) till I get a working compile.

    Either way I have to get this working, can’t afford a slot for a supported card.

  41. My most recent attempt used the sky2.c out of 2.6.33.7 (which is v1.26). I am pretty sure the PCI defines for the 88E8057 have been around for a while in sky2.c. I looked at several kernels during my various attempts. I have a 2.6.32.24 that has v1.25, and does include the PCI defines for the 88E8057. I also looked at 2.6.31 which includes v1.23 and even that has the defines for the 88E8057. The older sky2.c in the sky2-and-skge-for-esxi4-0.02.tar.gz download I have is from 2.6.26 (but out of a debian lenny source) and it is based on 1.21

    There’s a comment from forzyte higher up indicating that the 88e8057 works on ubuntu with a 2.6.32-24 kernel so that seems to imply that v1.25 should definitely work.

  42. Okay, the highest version I am able to compile is 1.22; before changes in skbuff and netdevice made things too complicated for me. (how did you get past all the changes in struct net_device, pci_dma_mapping_error, vlan_gro_receive, etc?)

    The compiled driver is pretty much useless but actually works for the first minute. I was able to get a DHCP lease and ping from within the console but as time went on, the server starts degenerating. Logs seemed to show that one of the PCPUs was locking up (no heartbeat for 60s) and a NMI was generated. Eventually I got a PSOD about a deadlock.

    I’m looking into it, but would anyone else be interested in my work so far (it is pretty much what kernel has done so far but on the 1.22 driver)? If so I will spend some time cleaning up my current work attach the core dumps and backtraces and package it online.

  43. Oh, the stack trace seems to point to:
    sky2_poll
    napi_poll

    Now I know where to look.

  44. (sorry about the multiple comments in a row)

    I’ve tracked the lockup to the following piece of code (its somewhere in sky2_poll):

    while ((idx = sky2_read16(hw, STAT_PUT_IDX)) != hw->st_idx) {
    work_done += sky2_status_intr(hw, work_limit – work_done, idx);

    if (work_done >= work_limit)
    goto done;
    }

    It appears that eventually the while condition stays true and work_done does not increment anymore.

    I don’t fully understand the driver yet (what that’s about 5000 lines to digest there), but I’m going to try looking into how this condition is reached.

    After that, I’ll just do a simple workaround that breaks out of the loop after X loops and see what happens.

  45. Hey thanks for looking so deeply. Much appreciated

  46. Okay, that was disappointing…

    It appears that the status ring buffer is only 512 (bytes?) long, but the code was set to 2048. (That’s my conjecture at this point, it may turn out to be that some extra initilisation needs to be done, or it is a bug in the code causing this)

    To get it to work, change the define for STATUS_RING_SIZE from 2048 to 512 in sky2.c.

    I’ll leave the server doing a whole lot of pings and repeatedly loading the web client from my computer and pinging the server overnight, just to test the stability. No problems so far for the past 15 minutes and CPU utilisation is low (judged from the CPU temperature).

    If the server isn’t on fire tomorrow when I return, I’ll remove the debug code, package up and leave a copy here. Or kernel, you can actually do it on your side as well, though I’m not sure what changing that will do to other chips (maybe you can test it out?).

    The log seems to be complaining about WOL not available:
    Hostd: Unable to enable WOL Operation not supported: Operation not supported
    WOL capability for nic cannot be turned on.probable bug file against driver: sky2

    and the module being unsigned:
    Kernel module sky2 was loaded, but has no signature attached

    But I can definitely say that I have the 88E8057 running in ESXi. =)

  47. Alright, the driver never missed a tick during the last 8 hours. ping tells me that I have about an average roundtrip time of 0.5ms with a maximum of 4ms over the local network. There were no connection timeouts loading the management page as well.

    I’ve rebooted and enabled both ports and they both work well with failover within 1s if I unplugged the active port.

    There here it is:
    http://www.mediafire.com/file/48dd012ap4xlufy/sky2-1.22-for-esxi4-88e8057-r1-xieliwei.tar.gz

    If kernel won’t mind, would you mirror a copy?

    I’ll be sticking around here a while to see whether anything bad turns up with the driver. I’m kind of satisfied with what I have now (its just for management only) and probably won’t be looking into the 512 limitation unless somebody points out what could be wrong.

    And thanks kernel for most of the work, I wouldn’t have tried if you hadn’t! =)

    (PS, can I place the oem file somewhere such that the ESXi installer can use the sky2 driver? I definitely have to install to harddrive because USB booting fails very regularly with the stick I’m using.)

  48. xieliwei, that is fantastic news for people with 88E8057s. Thanks for the effort. I’ll keep a copy of your driver here as a mirror. And I’ll also put an update in the main post so that it’s easier to find.

    That is odd re the STATUS_RING_SIZE. I’ve noticed looking at later revisions of the (linux kernel) driver, that subtle changes have been made to some of those other #defines in the same section of the code.

    Happy to mirror your oem.tgz if that helps. Thanks again.

  49. While the new driver for the 88e8057 does work in my laptop, it is losing 67% of the packets and has ping times of 1000 ms with an occasional 90 ms. I will try to play with it a little more.

  50. After a restart pinging from the unsupported console to another computer yielded 100% packet loss. Pinging loopback yields good pings but I then tried to ping the external address again and then everything works as expected. I am on esxi 4.0 update 2. i have not tried 4.1 yet.

Leave a Reply

You must be logged in to post a comment.