The Web10G Project: Building a Web10G Kernel

Home 5 Research 5 Networking 5 The Web10G Project 5 Building a Web10G Kernel

The following is generic instruction set for the building and installation of a web10G kernel. These instructions may not be applicable to any one specific distribution. The Web10G team encourages the use of known good methods to patch and build kernels and users should not feel bound to using these instructions. However, the team cannot provide detailed support for every Linux distribution available. When testing or developing for Web10G the team strongly encourages the use of an more established distribution. Instructions for building a custom kernel under Ubuntu can be found at the Ubuntu Wiki

Keep in mind that installation of the kernel will *only* provide the user with a Web10G kernel. The installation of the Web10G Userland (found under the Software tab) will provide the user with the Web10G API and example applications. It is strongly recommended that users install the API to make full use of the Web10G kernel.

The Web10Gig kernel is available as a set of patches from http://www.web10g.org/ under the Software tab.

This document covers general instructions and advice on how to:

  • apply the web10g patches to a linux kernel
  • build the kernel, and
  • install the newly built kernel on the system

IMPORTANT: Currently Web10G patches have only been tested for specific kernels from kernel.org.  The detailed version specific release information appears in README.web10G.

Be sure that you have a good repeatable procedure for building, installing and testing kernels (see below) before applying the web10g patch.  If you are moving to a new base kernel (for example 2.6.32 to 2.6.38), it is especially important that you verify that your new kernel is fully functional before you apply the web10g patch.  As such, we suggest that the user builds a ‘generic’ kernel and validates proper function *before* applying patches. We typically have far less difficulty installing web10g than getting new base kernels properly configured to support all devices and features. While this may add an additional step it makes it much easier to debug any problems with the Web10G kernel.

BEFORE YOU PROCEED

The directions that follow assume that you are enabled as root either via ‘su’ or sudo. However, you may configure and build the kernel as an unprivileged user in your user directory if you prefer. If you use this method you will need to use ‘su’ or ‘sudo’ to install the kernel.

Before you proceed, it may be a good idea to take the following steps
so that you can backtrack if you run into problems:

  • Make a copy of the current kernel
  • Make a copy of the .config file (usually in /usr/src/linux)
  • Tar up the files in the /lib/modules directory corresponding to the current working kernel

GETTING THE KERNEL SOURCE FROM KERNEL.ORG

The other option is to download the appropriate kernel source from kernel.org and patching it manually. Download the appropriate kernel sources for the version of the Web10G patch you are planning to install. For example, if you are using the kernel patch for 2.6.38 you would want the full sources for kernel version 2.6.38. You can obtain the source fia FTP at  ftp.kernel.org or via the web using one of its many mirrors for kernel.org at http://www.kernel.org/mirrors/.

After you have downloaded the kernel sources you’ll need to extract it:
# tar -C /usr/src -xvf linux-2.6.38.tar.gz.

This will create a directory named /usr/src/linux-2.6.38 containing the full kernel sources including all modules. Then make a symbolic link to a working directory:

# rm -rf /usr/src/linux
# ln -s /usr/src/linux-2.6.38 /usr/src/linux

APPLYING WEB10G PATCH

After you have extracted the kernel source you will then apply the appropriate Web10G patch (found on the Software tab of web10g.org)

# cd /usr/src/linux
# patch -p1 < [path-to-web100-dir]/web100-[version-name].patch

NOTE: It is a good idea to save a copy of the patch file you will be
using for web100, so that you can backout of the patch (to get a
clean kernel tree) if necessary.

Unpatching the kernel source is also required before applying the next version of Web10G patch file because the patches are distributed as full patches as opposed to incremental patches. Obviously, this does not apply if the kernel version supported by the patch changes.

To unpatch (when required):

#patch -Rp1 < [path-to-web100-dir]/web100-[version-name].patch

The patch command should complete successfully without any errors. There is, however, one file for which the patch may fail, and that is the patch to the Makefile in the top level.  The top level kernel Makefile defines “EXTRAVERSION” (typically on line 4). EXTRAVERSION determines the suffix of the kernel version name.  It is used for a number of things, including determining the directory name used by loadable kernel modules.

The Makefile patch will fail on all 3rd party kernels, because all 3rd party vendors alter EXTRAVERSION.

If EXTRAVERSION is not updated by Web10G and you ignore this error, your Web10G kernel will attempt to use the same loadable modules as the original kernel.  This will, in many cases, work just fine. However,  sometimes it will lead to problems due to unexpected operational dependences.  For example,  a configuration change to one kernel may make other kernels incompatible with their modules. Additionally, some third party software packages check the kernel version string and abort if they do not recognize it.  People who are installing picky third party software may need to avoid updating the EXTRAVERSION string.

At your discretion we do recommend that you edit the Makefile by hand and append “-web10g” to the EXTRAVERSION string (no spaces between the exisitng string the added string).

Regardless of your decision Web10G will function with or without updated EXTRAVERSION string. However, as previously mentioned, your choice for the EXTRAVERSION string may greatly affect other operational issues, such as module compatibility.

If you are comfortable with it you may want to additionally edit the makefile by hand to reflect your own operational preferences.

CONFIGURING AND BUILDING THE KERNEL

When installing the Web10G kernel we have found it to be safer and easier to force a full rebuild. Use the following commands to remove any previous configuration information and start a new config process.

# cd /usr/src/linux
# cp .config config.save
# make mrproper
# cp config.save .config
# make menuconfig

To enable web100 features select the following options in menuconfig:

Networking support  —>
Networking options  —>
[*]   TCP: Extended TCP statistics (TCP ESTATS) MIB
(32768) TCP: Maximum number of ESTATS connections
(384)     TCP:   ESTATS default file permissions
(0)         TCP:   ESTATS default gid

A note about permisions:
The default gid and file permissions control who has access to connection information.  Under many circumstances, you would want to create a “web10g” group which will have access to all connections, and set default file permissions to 0660. This ensures that only trusted users may access and modify connection information.  Alternatively, you may want to give more users access to the data in which case the permissions would be set to 438 which would give all users access to the ESTATS values (0666). Note that either option may have security implications.  You should carefully consider where and how you’ll be making use of the Web10G kernel.  These values will also be settable at run-time with sysctl variables net.ipv4.estats_fperms and net.ipv4.estats_gid.

Save menuconfig options and exit.

BUILDING AND INSTALLING THE KERNEL

The next steps are platform and distribution dependent, and are for Intel x86-compatible systems only.  For other architechtures, please refer to documentation specific to your system.

Save the old kernel.

If you’re using LILO or GRUB the following section explains the steps:

# cd /usr/src/linux
# make
# make bzImage
# make install
# make modules_install

If you have not modifed the EXTRAVERSION string you may be able to skip building the modules (make modules_install). As this step is generally the longest part of the compilation this can be a significant time saver. However, if you have changed kernel versions you must rebuild the modules. If you have any problems booting from the newly installed kernel it may help to force a module rebuild even if you haven’t changed EXTRAVERSION.

LILO Users

Edit lilo.conf (commonly found at /etc/lilo.conf) and add the following boot configuration lines.
For example, if /dev/hda2 is where /boot resides:

image = /boot/vmlinuz-2.6.38-web10g
label=linux-web10g
root=/dev/hda2

Run # /sbin/lilo -v

Reboot the system. Select “linux-web10g” to boot the web100 kernel. Then go to the “Confirming Web100 Kernel” section.

GRUB Users

Some distributions will have automatically updated the grub configuration system to reflect the newly installed kernel. If it hasn’t you may have to manually build an entry for the new kernel. The following instructions are geared for Fedora. Ubuntu uses a different grub configuration system which is not outlined here at this time.

Edit /boot/grub and add the following lines to ‘menu.lst‘.
For example, if you are if booting from /dev/hda5:

title linux-web100
kernel (hd0, 4)/boot/vmlinuz-2.6.35-web100 root=/dev/hda5

It may also be useful to comment out the ‘hiddenmenu’ option if it exists. Reboot system with new kernel and go to the “Confirming Web100 Kernel” section.

CONFIRMING WEB10G KERNEL

The message “Web10G [VERSION] build [version-name]: Initialization successful” should appear in the /var/log/dmesg file.  The date in [version-name] should match the “date” part from the file name in the patch file.

In addition, check the first line of the file /proc/web100/header.  This line should match the [version-name] in the /var/log/dmesg file.

If system fails to boot, see the “Troubleshooting” section.

TROUBLESHOOTING

If the system doesn’t boot correctly using the Web10G kernel, boot it up with the older kernel that was running on the system and confirm that the kernel was installed correctly. Your distribution may require an initramfs file to exist, the LILO or GRUB options might be incorrect, and so forth. This document cannot explore all the possible ways in which the kernel may have failed to install so it is important to make use of other resources. Both the Fedora and Ubuntu websites have detailed instructions on building custom kernels.  If you have confirmed that the kernel was installed correctly please check the options in the “make menuconfig” step to make sure the kernel options were selected correctly, recompile the kernel and reboot with the Web10G kernel. If you do not explicitly enable Web10G in the makeconfig step the kernel will not provide Web10G functionality.

If you’re unable to get the Web10G kernel up, please post a problem report in the Troubleshooting section of the Web10G Forums on Web10G.org under the Developers tab.

If you use pcmcia drivers or other loadable kernel modules, you may find that you need to rebuild them after applying the Web10G patch.  You should also save copies of your non-Web10G modules in case you need to roll back the installation.

If you have access to a virtual machine via parallels or VMWare it may be a good idea to practice building and installing the kernel in a virtual environment. Set up and tear down is relatively easy and can be a good test bed for people without much experience custom building kernel.

SUBMITTING A PROBLEM REPORT

If you’re unable to get the Web10G kernel installed and booted please post a problem report in the Troubleshooting section of the Web10G Forums on Web10G.org under the Developers tab.  Please try to include as much of the information as possible to assist us in troubleshooting the problems you are experiencing. Depending on the problem you are experiencing, it may not be possible to provide all that is requested below, but please try to provide as much of it as possible.

  • Output of the following commands:
    # uname -a
    # head -1 /proc/web100/header
  • List of third party packages installed
  • The name of the Web10G patch file
  • List of patch failures and the “.rej” file(s)
  • Description of the problem;  Please include specific details of the hardware and software configuration if you suspect that that may be the source of the problem.