Linux booting

inittab

upstart (added 12/Apr/09)

Old-style intro

My Suse 10.2 desktop hardly ever seems to shut down cleanly any more. It typically hangs on 'kernel NFS,' cupsd, or audio. Sooner or later, pulling the plug will corrupt the filesystem, so I need to fix it.

Suspect something has got changed or misconfigured that's causing timeout issues, and so I'll look to add some debugging in the offending scripts, or even remove them, if only temporarily. But having already tried a bit of meddling in /etc/init.d and subdirectories, to suspend shut down vmware machines automatically, it doesn't seem to be doing what I expected.

Update: it was an "unbranded" PCI TV card. If it'd been used, the system hung shutting down.

inittab

I'm going to start with /etc/inittab.

nb - present on Suse 10.2, and Ubuntu 6.06, not present on Ubuntu 8.04. Seems /etc/event.d/ ('upstart') replaced it* in 6.10. (*opens in a new window)

UNIX booting is governed by inittab; its the configuration for init. Unlike AIX, a Solaris or Linux admin who hasn't dug around (or read the manual) might not realise this. AIX starts everything using inittab, with a nod to rc.X directories from version 5.

Other flavours of unix have very few entries in inittab, and the bulk of the work is done by a script which works off the rc.X directories.

kbrequest

Looking in inittab, I found a reference to this. Linking a keypress direct into system initialisation looks useful, so I'm going to return to this.

/etc/init.d/rc

This script is called from inittab to control the run levels.

confirm feature

It has a 'confirm' feature, providing the system is leaving run level 'N', which is turned on via the following test

startpar

There's been quite a lot of debate about starting the system in parallel, using make to manage the dependencies. The following resources seem to play a part in supporting it within the more conventional mechanism.

I'd imagine, broadly, that it would background anything on which nothing else depends, allowing init to get on with starting other stuff instead of waiting. Hmm. Shame it can't do that shutting down my system!

Components referred to in the rc script are as follows; but it looks like there's a little more to it than this.

controlling run levels

When switching levels, it starts by running K* scripts from the __previous__ level's directory. It compares them with the S* scripts for that service at the new level, and executes any which aren't matched. The logic is documented: no point bouncing something that doesn't need it.

To do this, it assumes the 'entry' is linked to a file (usually in /etc/init.d) and uses that to match the start and kill scripts. Following the convention of symlinking would seem prudent, to get predictable results.

The intention is not documented (cheers, guys) but I think K* scripts which aren't symlinked don't get run. Can't be bothered to play with this wonderfully obfuscated code to work out how it works. Its perfectly fine for services to exist at only one run level, so a script which is linked only to one K* or one S* is just fine.

shutting down the system

Stop one would be /etc/init.d/rc0.d; init 0 shuts down the system.

All that's in there is a link to /etc/init.d/halt, which incidentally is linked to /etc/init.d/reboot - the sole file in the rc6.d (reboot) directory.

Watching the system shutdown suggests there are three stages. An initial stage, the section governed by the previous runlevel's kill scripts, and then 'halt'.

Final stage of shutdown: halt

The man page of halt (AKA reboot and poweroff) indicates that at run level 0 or 6 it'll play ball. So, if during shutdown it is reported to the console 'run level 0 reached' then I would imagine halt is about to, or has been called. But that would assume that the 'start' script for halt or reboot ever completes.

By default, halt and its aliases will resort to calling shutdown if the system is at the wrong level. But halt -f and reboot -f will bypass shutdown.

'The system will be halted immediately.' and 'Please stand by while rebooting the system...' are generated by /etc/init.d/halt.

/etc/init.d/boot.d/

It runs any K* scripts in /etc/boot.d. I note with interest that there are S* scripts there as well; for checking the root filesystem, RAID (md) devices, LVM, basic networking and swap. So .. all those initial boot messages after the kernel has initialised come from there.

Hard disk write cache

An interesting bootnote in the halt manpage is that the kernel doesn't flush the write cache on IDE drives, but default behaviour of halt is to put IDE drives into standby mode, flushing their write caches.

However, the only switches that Suse passes to halt is -p (power down - maybe), -d and -f. But not -h. The comments in the script seem to assume that waiting a bit after unmounting filesystems is adequate for the job.

Auto power-off

The system will only power down if it calls halt -p. The default on Intel is plain 'halt' - to attempt a power down, it needs to find /proc/apm, /proc/acpi, or /proc/sys/acpi, or find one of 'apm=smp-power-off' or 'apm=power-off' in the kernel command line, which is stashed in /proc/cmdline

intermediaries like /sbin/shutdown

The man page for shutdown makes this nice and simple: shutdown asks init to do its dirty work; so runlevel 0 is called by shutdown -h, runlevel 6 is called by shutdown -r

It refers the reader to inittab to find out what init does next.

Re-writing the system clock.

This is switched on (default) and off in /etc/sysconfig/clock, on Suse. This, in turn, is sourced by /etc/init.d/halt, which is the only 'start' item that init runs at runlevel 0 (halt) and runlevel 6 (reboot)

Noisy shutdown

On Suse, /etc/sysconfig/shutdown provides a facility to make a noise, through the speaker, when the operating system has shut down, prior to reboot or halt. The file is sourced by /etc/init.d/halt.

The noise is generated using the echo command in /etc/init.d/halt, towards the end.

test "$HALT_SOUND" = "no" || echo -en "\a"

Upstart

Hat tips: (*opens in a new window) ubuntu forums*, launchpad*, linux.com*

Designed to cope with the more dynamic nature of systems, particularly desktops: USB hotplug /events/ and so on. Things get run when something changes. Backwards compatibility with the rc.X folders, and utilities like reboot, shutdown, telinit are provided. Nothing immediately obvious changes: its a different /sbin/init.

Upstart emulates run levels, but Ubuntu systems don't actually have runlevels. Apparently.

Interaction is via initctl.

Events can be triggered using initctl emit.

initctl list suggests that, at least on 8.04, very little of the system has actually moved from /etc/init.d (contrasted, say, with Solaris services) and I suspect this will remain the case: Canonical wants to streamline handling 'events' to improve the Linux experience, but it can't do anything about all the init.d scripts all the Linux applications and components ship with.

ben@akira:~$ sudo initctl list
[sudo] password for ben: 
control-alt-delete (stop) waiting
logd (stop) waiting
rc-default (stop) waiting
rc0 (stop) waiting
rc1 (stop) waiting
rc2 (stop) waiting
rc3 (stop) waiting
rc4 (stop) waiting
rc5 (stop) waiting
rc6 (stop) waiting
rcS (stop) waiting
rcS-sulogin (stop) waiting
sulogin (stop) waiting
tty1 (start) running, process 7961
tty2 (start) running, process 6622
tty3 (start) running, process 6624
tty4 (start) running, process 6619
tty5 (start) running, process 6620
tty6 (start) running, process 6626

A question on the launchpad upstart page, March 2009, asks about using upstart for USB events. udev is recommended. It also doesn't log output from things it runs (unlike, say, Solaris svcs) WTF? I struggle to understand why Canonical bothered ..

Adding a shutdown script

There seems to be a specific process to adding shutdown scripts to Ubuntu, pointed to on Launchpad (opens in a new window), but needing a trip to 'man' to work out the arguments.

root@akira:/etc/init.d# update-rc.d "vmware.suspendall" stop 05 0 6
update-rc.d: error: expected runlevel [0-9S] (did you forget "." ?)
root@akira:/etc/init.d# update-rc.d "vmware.suspendall" stop 05 0 6 .
 Adding system startup for /etc/init.d/vmware.suspendall ...
   /etc/rc0.d/K05vmware.suspendall -> ../init.d/vmware.suspendall
   /etc/rc6.d/K05vmware.suspendall -> ../init.d/vmware.suspendall

add -n as the first argument to get the command to install the script .. not.