Debian Powered Hosts Yet Simple, Version 2 (dphys2)

Simple Automated Installer (SAI) for Debian 3.0 (woody)

Project Basics

The aim is to be able to simply insert the rescue and an modified root disk, answer a minimal amount of questions and then walk away, while the machine does its install by itsself.

For this most of the information asked for by the official installer should be pre-entered while generating an modified root disk.

Second target is automated maintenance of the machines, automatically installing/upgrading and removing packages, controlled by an single file, containing an package status list.

This program requires an network to run, as it will fetch all floppies (appart from the first 2) and all packages from the network. It can use either static IP configuration, or fetch config from DHCP. The floppy images can also be fetched from the net via PXE, avoiding any removable media.

Intended parts in this project are:

• an replacement for /usr/bin/dbootstrap (Debian interactive installer) which already knows what to do (apart from the hostname/IP which vary from host to host, and therefore are faster to just type in at begin of install) and which therefore does not ask any questions
• as part of above /usr/bin/dbootstrap replacement, also routines that pre-modify the installed system, so that the second boot stage questions (by base-config) don't get asked
• an script to generate an modified root disk image using just an HTTP connection to any random Debian mirror and a few files provided in this projects package
• an subset of drivers.tar.gz for the few ethernet cards we use (made also by above script) which gets included on the modified root disk, to reduce disk jockeying
• an script to simply making an rescue disk automatically
• an script to simply auto compile an custom kernel and generate an rescue disk and drivers.tzg file from it
• have all of above configured by an single config file, instead of interactive questioning by dialogs
• an facultative script to extend this with functionality with user specific wishes, such as modifying base-config script set to automagically load (on machine by machine base) all packages we want, including setting their configuratn, and install init.d/cron jobs
• tool to use swap files instead of partitions. Now moved to its own project dphys-swapfile
• tool to build packages containing software, configs as files or postinst actions, dependencies, etc. Now moved to an separate project dphys-pkgtools
• script to automatically build our local packages site. Now also moved to dphys-pkgtools
• script and package that controls initial install and config of our basic package collection onto the system, updating/removing many packages, in controlled sequence, run by cron or init. Now moved to an separate project dphys-admin

For more info on this project read the README file.
For some questions people have asked or may ask read the FAQ list.

Project Status

Done are:

• Investigation of what our present system (dphys) does and what the new system (dphys2) should do
• Investigation of what the official Debian 3.0 installer does, in particular the actions of the dbootstrap program, see the Logfile entry from 2002.07.12 Fri on
• Made an /usr/bin/dbootstrap replacement, as an script for easy adaption to our and others needs
• Made an makeroot script to generate the replacement root disk, containing also the reduced drivers.tgz
• Investigation of how debconf works, and where it gets its settings from, and how we can pre-set them
• Made an debconf-preload perl script called by dbootstrap replacement to do the intended pre-modification of the installed systems debconf database, and pre-generate some of the files that base-config makes

• Made an (now deprecated, use makesourcepackage> makesourcepackage) makepackage script to generate dphys-host-`hostname` and dphys-site packages, that are intended to load all site specific packages via Depends: and install/edit the desired config files
• /usr/bin/dbootstrap added run of an user script at the end, which sets up autoinstall of the hosts dphys-host-`hostname` package and all other packages and configs that this brings with it
• endfirstrun example script to be run by dbootstrap at the end, configures debconf for minimal interaction and loads the dphys-host-`hostname` package and all that it depends on
• Makefile.debian-local to automatically build our debian-local packages servers control files, so that apt-get can find stuff

• endfirstrun extended to be actually usefull, now loads host package and from that other stuff
• Built multiple packages used here on site. Some are available from our local packages server. Note: that you can not access "non-free", as that contains restricted distribution packages
• Extended /usr/bin/dbootstrap to now also allow fetching IP config from DHCP
• Made an (now deprecated, use dphys-admin script and package) dphys-setup.pl script to automatically install/update/remove packages, run by cron or init
• Support for alternatively using the bf2.4 kernel and modules

• Extended /usr/bin/dbootstrap to now also allow no swap partition, for no swap or later swap file
• Made an dphys-swapfile (now on separate project page) package to automatically generate/resize/activate an swap file, default 2*RAM size, and also deactivate/delete it at the approriate times
• Made an makesourcepackage script (now on separate project page) to generate dphys-* and other source based packages, that are intended to load all site specific software the desired config files
• Made an makerescue script to compile an custom kernel and generate and rescue floppy and drivers.tgz archive from ist

• Added dphys2cd (originally makecd) script for generating bootable CDs, with 2.88M rescue floppy image on them, with root file system image inside that
• Added dphys2pxe (originally makepxe) script for extracting PXE/TFTP boot server files, separate kernel and root system image for copying to server
• Changed makesourcepackage to generate Debhelper compatible packages
• All dphys2rescue and dphys2root disk image uploads and dphys2pxe file uploads and and makesourcepackage package uploads to server(s) by scp, not NFS, as that is more often available
• Made an dphys-admin script and package (now on separate project page) to automatically install new packages, update existing ones, or remove obsolete ones
• Moved dphys-admin and dphys-swapfile and dphys-pkgtools to separate projects, as all of these are usable on non-dphys2 systems
• renamed make* to dphys2* for using direct from shell, without calling via Makefile, which may not be available
• Added man pages for all dphys2* and for dphys2(7)
• Converted to run with all files scattered over file system, not with all in an src directory, debianised it

Todo are:

Code Download

• Installation Guide
• entire project: dphys2.tar.gz
• single files:
  • dphys2.7, man page overview of entire dphys2
  • dphys2, default config file, containing only the default settings as comments, to adapt to your site
  • Makefile, control process
  • dphys2rescue, generate custom rescue floppy image and drivers archive
  • dphys2rescue.1, man page for dphys2rescue
  • dphys2root, generate custom root floppy image, possibly using above generated stuff
  • dphys2root.1, man page for dphys2root
  • dphys2cd, generate bootable CD
  • dphys2cd.1, man page for dphys2cd
  • dphys2pxe, extract files for PXE/TFTP bootserver
  • dphys2pxe.1, man page for dphys2pxe
  • dbootstrap, /usr/bin/dbootstrap replacement
  • sfdisk, used by dbootstrap replacement
  • lilo.conf, used by dbootstrap replacement
  • debconf-preload, used by dbootstrap to fudge up 2nd stage process so that there are no dialogs
  • endfirstrun.simple, example endfirstrun user script that at end of forst stage installs an second stage script, that sets up an simple environment for my test installation machine
  • endfirstrun.dphys-admin, example endfirstrun user script that at end of forst stage installs an second stage script, that sets up and runs the dphys-admin package
• Debian packaged:
  • .dsc description file
  • debianised .tar.gz source archive
  • .deb package

• Installation Guide
• entire project: dphys2-20041015.tar.gz
• single files:
  • dphys2.7, man page overview of entire dphys2
  • dphys2, default config file, containing only the default settings as comments, to adapt to your site
  • Makefile, control process
  • dphys2rescue, generate custom rescue floppy image and drivers archive
  • dphys2rescue.1, man page for dphys2rescue
  • dphys2root, generate custom root floppy image, possibly using above generated stuff
  • dphys2root.1, man page for dphys2root
  • dphys2cd, generate bootable CD
  • dphys2cd.1, man page for dphys2cd
  • dphys2pxe, extract files for PXE/TFTP bootserver
  • dphys2pxe.1, man page for dphys2pxe
  • dbootstrap, /usr/bin/dbootstrap replacement
  • sfdisk, used by dbootstrap replacement
  • lilo.conf, used by dbootstrap replacement
  • debconf-preload, used by dbootstrap to fudge up 2nd stage process so that there are no dialogs
  • endfirstrun.simple, example endfirstrun user script that at end of forst stage installs an second stage script, that sets up an simple environment for my test installation machine
  • endfirstrun.dphys-admin, example endfirstrun user script that at end of forst stage installs an second stage script, that sets up and runs the dphys-admin package
  • makesourcepackage (now on separate project page), make dphys-* and other source based packages, that are intended to load all site specific software the desired config files
  • Makefile.debian-local, build an debian-local package Website
  • dphys-swapfile (now with on separate page), size/setup and mount/unmount an swap file
  • dphys-setup.pl (now with on separate page), automatically install/update/remove packages, run by cron or init. This is deprecated, use the new dphys-admin
  • dphys-admin (now with own project page), automatically install new packages, update existing ones, or remove obsolete ones

• Installation Guide
• entire project: dphys2-20040219.tar.gz
• single files:
  • setup-dphys2, adapt to your site
  • Makefile, control process
  • makerescue, script run be Makefile
  • makeroot, script run be Makefile
  • dbootstrap, /usr/bin/dbootstrap replacement
  • sfdisk, used by dbootstrap replacement
  • /etc/lilo.conf, used by dbootstrap replacement
  • debconf-preload, used by dbootstrap to fudge up 2nd stage process so that there are no dialogs
  • endfirstrun, user script that gets run at the end of the first install stage, modifies second stage, so that debconf is quiet and host package is apt-get-ed
  • makepackage, make an package containing Depends: and config files/scripts, for automatic host/site dependant installation/configuration. This is deprecated, use the new makesourcepackage
  • makesourcepackage, make an source based package containing Depends: and config files/scripts, for automatic host/site dependant installation/configuration
  • Makefile.debian-local, build an debian-local package Website
  • dphys-setup.pl, automatically install/update/remove packages, run by cron or init
  • dphys2-swapfile package, generate/activate/update swap file with its dsc file and source archive

• Installation Guide
• entire project: dphys2-20030804.tar.gz
• single files:
  • setup-dphys2, adapt to your site
  • Makefile, control process
  • makeroot, script run be Makefile
  • makerescue, script run be Makefile
  • dbootstrap, /usr/bin/dbootstrap replacement
  • sfdisk, used by dbootstrap replacement
  • /etc/lilo.conf, used by dbootstrap replacement
  • debconf-preload, used by dbootstrap to fudge up 2nd stage process so that there are no dialogs
  • endfirstrun, user script that gets run at the end of the first install stage, modifies second stage, so that debconf is quiet and host package is apt-get-ed
  • makepackage, make an package containing Depends: and config files/scripts, for automatic host/site dependant installation/configuration
  • Makefile.debian-local, build an debian-local package Website
  • dphys-setup.pl, automatically install/update/remove packages, run by cron or init

• Installation Guide
• entire project: dphys2-20030417.tar.gz
  Known bug #3: makepackage second time -p deletes changelog file [fixed in Current since 2003.04.25]
• single files:
  • setup-dphys2, adapt to your site
  • Makefile, control process
  • makeroot, script run be Makefile
  • makerescue, script run be Makefile
  • dbootstrap, /usr/bin/dbootstrap replacement
  • sfdisk, used by dbootstrap replacement
  • /etc/lilo.conf, used by dbootstrap replacement
  • debconf-preload, used by dbootstrap to fudge up 2nd stage process so that there are no dialogs
  • endfirstrun, user script that gets run at the end of the first install stage, modifies second stage, so that debconf is quiet and host package is apt-get-ed
  • makepackage, make an package containing Depends: and config files/scripts, for automatic host/site dependant installation/configuration

• Installation Guide
• entire project: dphys2-20021218.tar.gz
  Known bug #1: dpkg packages list not undated (apt packages list ist) [fixed in Current since 2003.02.14]
  Known bug #2: apt-get upgrade while install is extremely slow (after apt-get is normal) [fixed in Current since 2003.03.13]
• single files:
  • setup-dphys2, adapt to your site
  • Makefile, control process
  • makeroot, script run be Makefile
  • makerescue, script run be Makefile
  • dbootstrap, /usr/bin/dbootstrap replacement
  • sfdisk, used by dbootstrap replacement
  • /etc/lilo.conf, used by dbootstrap replacement
  • debconf-preload, used by dbootstrap to fudge up 2nd stage process so that there are no dialogs

Auxillary documents

Text documents from planning:

• setup of todays Potato 2.2 system
• design for new woody 3.0 system (outdated document)

• saved 20020718 test install of woody, 1st stage, just before reboot (.tar.gz 57M)
• saved 20020718 test install of woody, 2nd stage, just after first login (.tar.gz 34M)
• saved 20030123 test inst of dphys2, 2nd stage, just after first login (.tar.gz 32M)

• saved 20020717 debconf database of woody, 1st stage, just before reboot
• saved 20020718 debconf database of woody, 2nd stage, just after first login
• saved 20030123 debconf database of dphys2, 2nd stage, just after first login
• saved 20030123 debconf database of dphys2, 2nd stage, just after install ssh
• saved 20030123 debconf database of dphys2, 2nd stage, just after install ssh and less

External Stuff (Links)

• Debian main site
• Our Debian mirror
• Rescue disk (i386), used to boot system (from our Debian mirror)
• original Root disk (i386), with install system (from our Debian mirror)
• original drivers file (i386), for modules (from our Debian mirror)
• boot-floppies_3.0.22 source (from our Debian mirror, and local copy)
• Our local packages
• Debian Binary Package Building HOWTO (we use a variant of this method to make our host and site packages in makepackage (deprecated))
• Debian New Maintainers' Guide (we use a variant of this method to make our host and site source based packages in makesourcepackage

This page is by Neil Franklin, last modification 2004.11.25