Preparing/Debugging linux distro to boot through pxe/tftp

Introduction

The problem

So why another how-to on this subject since a huge number of publication is available on the internet? Because by following step-by-step each of them I always got the same behaviour:

The DHCP/PXE/TFTP servers works fine
Booting either with or without an initrd doesn't work and the constant in it was:
1. Begin: Mounting root file system
  - Begin: Running /script/nfs-top
  - Begin: Retrying nfs mount
  - Begin: Running /script/nfs-premount
  - RPC failed 2
  - No filesystem could mount root, tried …
  - and finally: Kernel Panic
2. mount wrong argument.

So I tried to:

build new kernels with all nfs support →
test both the etch and lenny distro (and also knoppix) →
follow work around on the initrd →

<html> <div style=“width: 75%; border-width: 1px; border: solid; text-align: center; padding: 10px; margin: 10px”> Finally, I found that both the mount and nfsmount present in the initrd doesn't work!<br> So, I <a href=“http://wiki.bolay.net/doku.php?id=operating_systems:linux:debian:nfsboot#with_initrd”>replaced one line in the nfs script present in the initrd and added in /sbin the mount.nfs command</a>. </div> </html>

The answer

In this document I would like to share what I didn't found in once:

Prepare a new distribution which will be the root system for the nfs clients
Building a kernel (basis)
Building an initrd (basis)
Share my configuration
Give the opportunity to boot through PXE on an NFS filesystem.

<html> <div style=“color: red; font-size: large;”> Personally and in accordance with the DEBIAN guidelines, I finally chose to boot with an initrd. </div> </html>

Network structure

server: NFS/TFTP (ETCH - Linux server 2.6.18-6-686)
server2: DHCP/DNS (ETCH - Linux server2 2.6.18-4-686)
plenty of HP desktop machines

Directory structure

This was arbitrarily chosen like this!(Do not ask why!)

Distribution structure

(debootstraped)

The directory structure is (so you have to do a mkdir):

etch: /srv/work/tftp/distro/etch
lenny: /srv/work/tftp/distro/lenny

and for the installation do

root@server:/#aptitude install debootstrap
 
#for etch
root@server:/#debootstrap --include=nfsbooted,dhcp3-client,procps,passwd,vim,less,configure-debian  etch  /srv/work/tftp/distro/etch
 
#for lenny
root@server:/#debootstrap --include=nfsbooted,dhcp3-client,procps,passwd,vim,less,configure-debian  lenny  /srv/work/tftp/distro/lenny

Kernel

(git cloned)

The directory structure is (so you have to do a mkdir):

kernel 2.6.20 (for etch): /srv/work/tftp/distro/linux-kernel-2.6.20
kernel 2.6.26 (for lenny): /srv/work/tftp/distro/linux-kernel-2.6.26

and for the installation do

root@server:/#aptitude install git-core build-essential
 
#for etch
root@server:/#git clone git://git2.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.20.y.git /srv/work/tftp/distro/linux-kernel-2.6.20
root@server:/#cd /srv/work/tftp/distro/linux-kernel-2.6.20
root@server:/srv/work/tftp/distro/linux-kernel-2.6.20#make allmodconfig
 
#for lenny
root@server:/#git clone git://git2.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.20.y.git /srv/work/tftp/distro/linux-kernel-2.6.26
root@server:/#cd /srv/work/tftp/distro/linux-kernel-2.6.26
root@server:/srv/work/tftp/distro/linux-kernel-2.6.26#make allmodconfig

Building a new kernel

Kernel Modules

First you have to determine which module is necessary for your hardware.
Then you can either choose to select the right driver either with =m if you plan to use it with an initrd
or with =y if you do not want to use an initrd.

In my case, I have several HP desktop machines:

Network Card

HP DC7100 SFF: CONFIG_TIGON3
HP DC7700p SFF: CONFIG_E1000

Video Card (FRAMEBUFFER)

(see Document/fb/framebuffer.txt)
http://tldp.org/HOWTO/Framebuffer-HOWTO-5.html
http://ubuntuforums.org/showthread.php?t=165974

HP DC7100 SFF (915G Chipset): FB_INTEL (see in Document/fb/intelfb.txt)
HP DC7700p SFF (965Q Chipset): FB_INTEL (see in Document/fb/intelfb.txt)

Configuring the kernel

If you want to use the configuration menu, you have to install the ncurses libraries.

root@server:/#aptitude install libncurses5-dev

otherwise you can edit by hand the .config file which is in the linux-kernel-2.6.2x directory.

If you choose to use the configuration menu, do:

#for etch
root@server:/srv/work/tftp/distro/linux-kernel-2.6.20#make menuconfig
  
#for lenny
root@server:/srv/work/tftp/distro/linux-kernel-2.6.26#make menuconfig

and then all the procedure of building the kernel, installing the modules in the respective debootstraped directory and finally installing the new kernel also in the respective debootstraped directory. Below are some scripts which does this annoying repetitive sequence alone:

/srv/work/tftp/distro/linux-kernel-2.6.20/.compile.sh

#!/bin/bash
rootPath=/srv/work/tftp/distro/etch
output=$rootPath/kernel
 
make O=$output menuconfig
make 0=$output -j2 bzImage modules
make O=$output INSTALL_MOD_PATH=$rootPath modules_install
make O=$output INSTALL_PATH=$rootPath/boot install
exit

/srv/work/tftp/distro/linux-kernel-2.6.26/.compile.sh

#!/bin/bash
rootPath=/srv/work/tftp/distro/lenny
output=$rootPath/kernel
 
make O=$output menuconfig
make 0=$output -j2 bzImage modules
make O=$output INSTALL_MOD_PATH=$rootPath modules_install
make O=$output INSTALL_PATH=$rootPath/boot install
exit

HINT

I used diff to see the differences between the kernel .20 and .26

root@server:/srv/work/tftp/distro# diff -bwBy --suppress-common-lines linux-kernel-2.6.20/.config linux-kernel-2.6.26/.config > diff.txt
root@server:/srv/work/tftp/distro# vi diff.txt

Building a new initrd

Install necessary tools to compile a new initrd

This is an example with lenny

root@server:/#chroot /srv/work/tft/distro/lenny
root@server:/#apt-get install initramfs-tools

useful info are found in:

man update-initramfs
man initramfs.conf
man initramfs-tools

Create a new initrd

root@server:/#update-initramfs -c -k 2.6.26.3

Update an existing initrd

root@server:/#update-initramfs -u -k 2.6.26.3

View the initrd content

First, create a temporary directory and goes into (here I suppose that you are already chrooted in):

root@server:/#mkdir /tmp/initramfs
root@server:/#cd /tmp/initramfs

Then determine which type of file is your actual initrd (if there is one) and extract its content:

root@server:/tmp/initramfs#file -z /boot/initrd
initrd: ASCII cpio archive (SVR4 with no CRC)
 
root@server:/tmp/initramfs#zcat /boot/initrd | cpio -i
#or (given by man initramfs-tools(8))
root@server:/tmp/initramfs#gunzip -c /boot/initrd | cpio -i -d -H newc --no-absolute-filenames

root@server:/tmp/initramfs#file -z /boot/initrd
initrd: Linux Compressed ROM File System data, little endian size 1372160 version #2
root@server:/tmp/initramfs#gunzip initrd
#or
root@server:/tmp/initramfs#gunzip -c initrd > initrd

PXE/TFTP boot with NFS support

DHCP/TFTP/NFS server installation

First install your DHCP/TFTP/NFS server as widely explained in other how-to.

For information, This is my configuration:

TFTP

served by tftpd-hpa

root@server:/#cat /etc/xinetd.d/tftp

service tftp
{
	socket_type	= dgram
	protocol	= udp
	wait		= yes
	user		= root
	server		= /usr/sbin/in.tftpd
	server_args	= -c -s /srv/work/tftp -vvvvv
	disable		= no 
}

So, my tftp root is in /srv/work/tftp! This will be the root path for the pxe options.

DHCP

served by dhcp3-server

root@server2:#cat /etc/dhcp3/dhcpd.conf

class "pxeclients" {
	match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
	filename "pxelinux.0";
	next-server server; #in fact here I have the IP address of the TFTP server
}

NFS

served by nfs-kernel-server

root@server:#cat /etc/exports

/etch    *(rw,sync,no_root_squash,no_subtree_check)
/lenny    *(rw,sync,no_root_squash,no_subtree_check)

which are in fact links to the debootstraped directories

root@server:/#ls -lh
lrwxrwxrwx    1 root root   27 2008-08-21 13:15 etch -> /srv/work/tftp/distro/etch/
lrwxrwxrwx    1 root root   28 2008-08-21 13:15 lenny -> /srv/work/tftp/distro/lenny/

Configuring/Tunning the debootstraped distribution

I suppose in this section that you are chrooted in your debootstraped distribution. exemple:

root@server:/#chroot /srv/work/tftp/distro/lenny

Package verification/installation

First check that all tools you need are present.

In this example, I show the missing nfs support for the mount command, then the installation of the respective package and finally a successfully test:

root@server:/#mount -t nfs 172.16.30.22:/lenny /tmp
mount: wrong fs type, bad option, bad superblock on 172.16.30.22:/lenny,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
 
root@server:/#aptitude install nfs-common portmap
 
root@server:/#mount -t nfs 172.16.30.22:/lenny /tmp

Configuration

As explained in /usr/share/doc/nfsbooted/README.gz from the nfsbooted package.

/.nfsroot

root@server:/#mkdir /.nfsroot

/etc/fstab

root@server:/etc# cat fstab 
# UNCONFIGURED FSTAB FOR BASE SYSTEM
/               /.nfsroot       none    bind,ro         0 0
proc            /proc           proc    defaults        0 0
 
# copied from /etc/nfsbooted/fstab
/dev/ram        /tmp            ramfs   defaults,rw,auto,dev            0 0
/dev/ram1       /var/run        ramfs   defaults,rw,auto,dev            0 0
/dev/ram2       /var/state      ramfs   defaults,rw,auto,dev            0 0
/dev/ram3       /var/lock       ramfs   defaults,rw,auto,dev            0 0
/dev/ram4       /var/account    ramfs   defaults,rw,auto,dev            0 0
/dev/ram5       /var/log        ramfs   defaults,rw,auto,dev            0 0
/dev/ram6       /var/lib/gdm    ramfs   defaults,rw,auto,dev            0 0
/dev/ram7       /var/tmp        ramfs   defaults,rw,auto,dev            0 0

WITH initrd

(see Documentation/initrd.txt)

http://www.lugatgt.org/articles/booting.inittools/downloads/presentation.pdf
http://www.yolinux.com/TUTORIALS/LinuxTutorialInitProcess.html

First create a new hook script in the right directory and add the execution right on it:

root@server:/#cd /etc/initramfs-tools/hooks
root@server:/etc/initramfs-tools/hooks#touch nfs
root@server:/etc/initramfs-tools/hooks#chmod +x nfs

Then fill the content of this newly created file with:

#!/bin/sh -e
 
PREREQS=""
 
prereqs() { echo "$PREREQS"; }
 
case "$1" in
    prereqs)
    prereqs
    exit 0
    ;;
esac
 
. /usr/share/initramfs-tools/hook-functions
 
# Begin real processing below this line
 
copy_exec /sbin/mount.nfs
copy_exec /sbin/mount.nfs4

Modify the command at line 48 (in the do_nfsmount() function) in the nfs script located in the /usr/share/initramfs-tools/scripts/ directory from

nfsmount -o nolock ${roflag} ${NFSOPTS} ${NFSROOT} ${rootmnt}

/sbin/mount.nfs ${NFSROOT} ${rootmnt} -o nolock ${roflag} ${NFSOPTS}

and finally rebuild your initrd.

WHITOUT initrd

(see Documentation/nfsroot.txt at least in 2.6.20! It has disappeared in 2.6.26!)

/srv/work/tftp/linux/debian/menu/default

label linux-debian-etch-2.6.20
        kernel /distro/etch/boot/vmlinuz
        append rw console=ttyS0,115200n81 console=tty0 root=/dev/nfs ip=::::diskless:eth0:dhcp nfsroot=172.16.30.22:/etch panic=100 vga=791 quiet
 
label linux-debian-lenny-2.6.26
        kernel /distro/lenny/boot/vmlinuz
        append rw console=ttyS0,115200n81 console=tty0 root=/dev/nfs ip=::::diskless:eth0:dhcp nfsroot=172.16.30.22:/lenny panic=100 vga=791 quiet

aptitude install gnome-desktop

if the system hang while starting up the network-manager or the network-manager-dispatcher, edit the respective files in /etc add at the top of file an exit 0 instruction. Then restart your machine and do a

dpkg -a --configure

Netinstall

wget

etch - businesscard: http://cdimage.debian.org/debian-cd/4.0_r4a/i386/iso-cd/debian-40r4a-i386-businesscard.iso
 etch - netinstall: http://cdimage.debian.org/debian-cd/4.0_r4a/i386/iso-cd/debian-40r4a-i386-netinst.iso

root@server:/#cd /srv/work/tftp/linux/debian/etch
root@server:/#mount -o loop -t iso9660 debian-40r4a-i386-businesscard.iso businesscard/

FAQ

Q: mountd[11516]: refused mount request from gateway.iro for /srv/work/tftp/distro/lenny (/srv/work/tftp/distro/lenny): illegal port 12817
 A: add in the /etc/exports the option insecure in the corresponding exporting line

LINKS

This howto was mainly based on jukie.net/~bart work! Many thanks to Bart Trojanowski!
http://www.kernel.org/doc/
http://www.linuxhq.com/kernel/

http://www.tldp.org/HOWTO/BootPrompt-HOWTO.html
http://syslinux.zytor.com/wiki/index.php/PXELINUX
http://www.logilab.org/view?rql=Any%20X%20WHERE%20X%20eid%203281
http://www.onesis.org/NFSroot-HOWTO.php
http://www.howtoforge.com/pxe_booting_debian
http://www.debian-administration.org/articles/478
http://wiki.linuxquestions.org/wiki/Diskless_Workstation
http://linuxfellaz.net/doku.php?id=blog:howtos:etch:disklessboot
http://www.doctort.org/adam/nerd-notes/network-boot-with-debian.html

Table of Contents