http://www.phys.ethz.ch/~franklin/Projects/dphys3/Logfile - done and to do author Neil Franklin, last modification see last entry near bottom copyright ETH Zuerich Physics Departement, use under either BSD or GPL license background is the dphys2 project (for Debian 3.0/woody), read its Logfile at: http://www.phys.ethz.ch/~franklin/Projects/dphys2/Logfile this is the follow up dphys3 project, an autoinstaller for Debian 3.1/sarge 2004.10.15 Fri dphys2 release 5th milestone, copy to ../dphys2-20041015, make tar file of it modified index.html and FAQ and INSTALL to show the milestone release deleted old make*, setup*, endfirstrun from current redo ../dphys2.tar.gz tar archive of current state packaged version 20041015.104611-1, changelog: "taken over dphys2 milestone 5 as dphys3 milestone 0 (nothing done, base)" 2004.12.10 Fri started planning for sarge installer, read Debian install docs the new Debian install now has an automatic install mode only needs an config file on install media and setting an kernel parameter so it looks like we do not need an own dbootstrap or debconf-preload any more so totally different smaller project, so name dphys3, not dphys2.1 discussion with heeb, we will use this variant, just make/pack/trigger config we don't want 3rd floppy for net drivers, so also include those 2004.12.16 Thu started project directory, copied everything from dphys2 hagrid test installation, installer is quite intelligent, finds hostname/domain end of 1st stage and after install made .tar.gz of entire filesystem copied to ~/public_html/Projects and unpacked in /scratch/franklin base install installs quite a bit of stuff, less, ssh, gcc, python even if tasksel nothing chosen, just minimal wanted, result 330M, woody 110 ports at long last inetd crappola is closed, but now identd is open 2004.12.17 Fri adapted docs for dphys3, FAQ INSTALL Makefile index.html.en while INSTALL noticed bug, still points to old dphys2/dpyhs2.tar.gz place corrected for dphys3, added symlink for dphys2 (no changes since milestone) also bug .deb section, says dphys-pkgtools, corrected dphys3, dphys2 leave no released milestones yet, so date set to -------- same no Debian packages so date.time set to --------.------ got rid of potato_today and woody_new files, as not relevant any more got rid of dbootstrap and debconf-preload, as not needed any more same for lilo.conf and sfdisk, as most likely also not needed any more rename all dphys2* files to dphys3* also updated names/paths/date in all headers, man page references leave all paths and names in scripts, as these need extensive rework removed dbootstrap and stuff triggered by it reference in headers index.html eliminated all milestone versions, only current reworked intro text to describe the new project, what it will do changed all references in debian/*, and filenames made first .tar.gz archive of this project 2004.12.21 Tue adapted docs for dphys3, INSTALL remark needing large rewrite index.html.en status section index.html.en status section "Milestone" 0: list of stuff taken over from dphys2, Current: what to do planned basic implementation steps looked for where rest of drivers are, as from woody 7 to sarge only 2 floppys tried booting hagrid for looking into stuff, HD image is non bootable was still root floppy image "inserted", tried to boot that seems to be no package drivers* or linux-kernel-drivers* looked into root floppy, contains disk.lbl info file and initrd.gz initrd.gz 1.35->5.2M (smaller than woody) filesystem, few drivers in /lib/mod net includes pcnet32, 3c59x, 8139too, e100+mii.o, e1000, tulip and unix.o and af-packet.o are still modules, totally senseless so we can put our driver collection direct into root disk, no packing tars /dev only console and null, than runs MAKEDEV, so no 1000s of inodes no need for special options when making file system for /dev fitting net-drivers and cd-drivers images are filesystems with .udeb files but are facultative anyway, so we can ignore them, don't need them on target then /boot/* kernel for K7, and initrd K7, and also /lib/modules K7 for own boot floppy (without *-drivers) need different kernel than for sys boot floppy VFAT with linux, initrd.gz(!) and syslinux, with 6k free initrd.gz 0.65->1.88M, run /sbin/init script(!), to fetch 2nd/root floppy but also loads further drivers, usb input block ide scsi on installed system also an initrd, with /sbin/init script, bin and bin2 installation process on root floppy, /sbin/init script, calls busybox init /etc/inittab runs /sbin/debian-installer-startup, then debian-installer run /lib/debian-installer-startup.d/* and respawn /lib/debian-installer.d/* /lib/debian-installer.d/S70menu-linux runs /lib/debian-installer/menu that then runs debconf -o d-i $MENU from /usr/bin/debconf 2005.01.06 Thu read through LinuxPlanet FAI article, needs NFS, and own monitoring daemon decided to stay with our own system, develop new edition for sarge 2005.02.11 Fri vmware new 2 emulated systems, old woody/dphys and new sarge/Debian 2005.02.16 Wed completed index.html.en, added to projects index want new disk container file for sarge vmware, so new install, record questions floppy inserted boot.img Ctrl-Alt-Del boots from floppy Press F1 for help, or Enter to boot: floppy inserted root.img Press Enter when ready, or type a command: Choose a language: (preselected english) Chose country or region other (not preselected United States) Chose country or region Switzerland Load drivers from a floppy no (not preselected yes) Select a keyboard layout Swiss German (not presel American English) Configure the network hagrid (not preselected hagrid1) Configure the network (preselected ethz.ch) Chose a mirror of the Debian archive enter information manually (not preselected Switzerland) Chose a mirror of the Debian archive debian.ethz.ch (no preselect) Chose a mirror of the Debian archive /mirror/debian/ (not presel /debian/) Chose a mirror of the Debian archive (preselected no proxy) Partition disks (preselected erase entire disk) Partition disks (preselected All files in one partition) Partition disks (preselected Finish partitioning and write changes) Partition disks yes (not preselected no) Install the GRUB boot loader on a hard disk (preselected yes) floppy removed Finish the installation (preselected continue) reboots from hard disk GRUB bootscreen, auto continues after a while Debian base system configuration (preselected yes) Time zone configuration yes (not preselected no GMT) Time zone configuration (preselected yes for Europe/Zurich) Configuring passwd secret Configuring passwd secret Configuring passwd franklin Configuring passwd (preselected franklin) Configuring passwd secret Configuring passwd secret Apt configuration http (not preselected cdrom) Apt configuration enter information manually (not preselected Switzerland) Apt configuration debian.ethz.ch (no preselect) Apt configuration /mirror/debian (not presel /debian) Apt configuration (preselected no proxy) Debian software selection (preselected no categories) Configuring Exim v4 (preselected local delivery only) Configuring Exim v4 (preselected franklin for root and postmaster) Debian base system configuration (preselected Ok) login appears, finished looked at system, /root/dbootstrap_settings via /root/install-report.template found /var/log/debian-installer in there saved messages, syslog, partman, status of the install system looked at boot floppies, boot.img syslinux + 761k linux + 658k initrd.gz root.img second 1358k initrd.gz, unpacked both .img and both initrd boot initrd /sbin/init script, uses devfs(!), target later does not, good 2005.02.24 Thu go through Debian installer docs, find out where and how to do preseed stuff manual 5.2 boot prompt preseed/url vs preseed/file, debconf/priotity=critical http://d-i.alioth.debian.org/manual/en.i386/ch05s02.html manual 4.7 automatic install needs critical, DHCPs, then URL fetch preseed can all be set as kernel command line, in syslinux.conf before loading http://d-i.alioth.debian.org/manual/en.i386/ch04s07.html get current data with debconf-get-selections, but better use example file C.1 preconfig file example, lang/country/key must be in syslinux.conf http://d-i.alioth.debian.org/manual/en.i386/apcs01.html there exists an custom-kernel.txt in installer/doc directory (of d-i?) http://d-i.alioth.debian.org/svn/debian-installer/installer/doc/custom-kernel.txt preseed already being used for autoinstall http://interthingy.com/digby/ 2005.02.25 Fri preseed can not set first few questions, can only be read after them, why? boot init -> root init -> /etc/inittab -> /sbin/debian-installer -> /lib/debian-installer.d/S[0-9][0-9]* -> S70menu-linux -> /lib/debian-installer/menu -> exec debconf -o d-i /usr/bin/main-menu main-menu is then a binary, no man page on installed system, dead end build an preseed file, copied from Debian manual appendix C.1 settings from last test install from 2005.02.16 Wed for base-config/late_command extract script from dphys2 simple.endfirstrun no generating script from script, load it with wget directly, then run testrun with long command line, still asks language and all other stuff puts parameters direct into floppy syslinux.cfg, runs until VFS mount there crashes with "Kernel panic: VFS: Unable to mount root fs on 00:00" remove devfs=mount,dall and it runs until trying to read root floppy remove rw (reduce parameter count) has no effect, still 00:00, ram is 01:00 with both but some of the Debian opts gone, it now fails on 09:00(!) 2005.03.02 Wed further preseed experiments, only preseed/url=, no crash but ignored so no vga= and no ramdisk_size= is OK, dito adding own parameters add debconf/priority=, runs now with near no questions but hangs when loading installer components: 1% bugreporter-udeb after long timeout error that failled to get from server our server works, when wget on 2nd console, so must be missconfig looks like it did not use the server given in preseed file pressed on "change server" and automatically starts downloading but then stops with questions on partitioning, not using the preseed added /preseed to root disks initrd.gz and preseed/file= on boot syslinux.cfg dmesg shows that parameters are here, ls -al shows /preseed, and readable but still server not found, change continues, stops at partition dialog after few keypresses, runs further until reboot dialog but stops then again on root password, and again create user seems all base-config stuff not preseeded, apt also not so preseed/url= is same as preseed/file=, drop own root image, use URL fixed a few settings in preseed file, also add language in syslinux.cfg language works, still asks floppy drivers, still hangs in components direct enter when on retry also works, did not even select change reading root initrd /usr/lib/debian-installer/retriever/net-retriever discovered dists/testing/main/debian-installer/binary-i386/Packages lists all udeb packages that build installer system, anna apt replacement looking at install sys /var/log/syslog, many messages no libc6 looking at /var/lib/dpkg/info/prenaseconfig.d, anna net-retriever, binary retriever/floppy/loadnow in preseed fails, too early, put in syslinux.cfg too many options, removed rw, now again VFS panic 00:00 added rw, remove devfs=, now again endless loop looking for root disk removed setting keyboard, annoying, but boots, still silly dialog but different one, floppy read error 2005.03.03 Thu further preseed experiments with syslinux.cfg retriever/floppy/loadnow=false different dialog, can not read floppy, look at ps aux load-floppy.postinst (overwrites with db_set true) and then all to anna floppy-retriever, then /u/l/d-i/retriever/floppy*, fails root initrd /var/lib/dpkg/info/load-floppy.postinst forces false->true remove db_set line, works, now gets without dialog to component loading component loading, ps aux shows in /v/l/d/i/download-installer.postinst there anna net-retriever, using ftp.us.debian.org, not preseeded mirror debian//pool/main/b/..., after return debian//pool/main/d/... after this then again partitioning, also waiting for input why is mirror/http/* not set, preseed not being used? only later? looked at webserver log, only read once, 02/Mar/2005:10:14:53, my URL test test witch todays current floppys, missing component proper error message also stops at partitioner, no preseed file read from webserver looked at debian-boot mailing list, preseed seems to generally be in use case of not being used turned out to be formatting error 2005.03.10 Thu further preseed, try to identify why not loaded, added debug output still hangs on not finding bugreporter-udeb package, and no preseed looked at /var/cache/cdebconf/, preseed/url is set and "seen", without .cfg kernel parameter without .cfg, too long? rename without further looked at /varlib/dpkg/info/*.postinst, found /lib/preseed/preseed.sh that does logging, so /var/log/syslog, aha, is loading /preseed.cfg replaced root with unchanged original, reinstall, now no bugreporter-udeb still loaded from /preseed.cfg, no mentioning of net attempt further analysing /lib/preseed/preseed.sh, used /var/log/debconf-seed in that only single line, most likely the /preseed.cfg file (from where?) manually run main menu "Download debconf preseed file", works, is now here before "chose mirror" and "download installer components", so only floppy looked at main-menu (and libdebian-installer( source, seems to work on lists finding out where they are from, what in them, will be a lot of work also fetched all [initrd|file|network-]preseed[-common] package source that seems to be entirely sensible, so most likely main-manu problem seems that only initrd originally on system, file and network only add later fetched in "download installer components"? then not run because earlier? asked in #debian-boot, fjp preseed/url= may only work with netboot tried preseed/url=, works in initrd and direct on floppy not set in debconf, but no initrd preseed any more, different behaviour bahaviour includes "missing / for installation" from partitioning 2005.03.11 Fri further preseed, went looking for main-menu in Debian bugtracking system others also have ordering trouble with later loaded packages before loader Bug #288053 network-console inserted before download, so not run fetched varios package sources [file|network]-preseed are in menu 14 and 21 but download-installer fetches only at 23, so main-menu eliminates them so these 2 need to be included on floppy, like preseed-common is already while looking in dists/sarge/main/debian-installer/binary-i386/Packages noticed debian-edu-install-udeb at 10, "force install to chose menu items" seems to be server and workstation configs, both based on lvm also looking at how to file bug, pressed packages version noticed bug #276001, better preseeding, all values, no syslinux.cfg try (miss)using "current" floppys and initrd-preseed from them submitted bug report on package network-preseed version 1.01 to Debian got registered as #299059 try with "current" floppys and initrd-preseed, added syslinux.cfg and floppy and Debians preseed.cfg to my preseed, works, just a few dialogs then fails when hitting lilo install, try grub for now now rid of keyboard type question (AT vs USB) floppy error still here, fix script /var/lib/dpkg/info/load-floppy.postinst and lilo install error stays despite demanding grub, fix preseed.cfg now all works, no dialogs, just unwanted 1+5 partitioning and timezone is now at least in Europe, but Amsterdam (was first in Africa) change this to Europe/Zurich for next run 2005.03.17 Thu we don't want the standard partiton 1+5 root+swap, so look into partitioning define an custom partman-auto/expert_recipe with just one root partition also run base-config/late_command from floppy/initrd, no http accesses needed while at it update/expand my script for making modified floppy images add all chages we need up to now (timeout, preseed, floppy load fix) parted fails, thinks we have no / partition, despite us having only that was missing parameter use_filesystem{ } in the partition description timezone it keeps on wanting Europe/Amsterdam, not Europe/Zurich base-config by hand, added tzconfig/select_zone/Europe to debconf DB add this to preseed file and it works, all now installed correctly 2005.03.18 Fri decided on renaming dphys3rescue + dphys3root to dphys3kernel + dphys3preseed as both of these will need to alter both images (drivers and config on both) modified file headers (rest will follow), man pages (all names), Makefile changed index.html.en to reflect new names and functions cut down preseed.cfg to minimal stuff we need (no #-outed possibilites) 2005.03.24 Thu rc3 of the install floppy is out, try our preseed with that get original floppy image directly from our mirror server, not local copy add our preseed stuff to existing file, instead of simply overwriting it preseeding fails completely, no preseed stuff at all on the floppy image but our /preseed.cfg is there, and debconf-set-selections do settings from second shell, but system ignores them, must be earlier mount initrd, chroot, apt-get (anna) there, anna segfaults before chroot copy .udeb into /mnt, then udpkg there, works, files appear changed floppy image and .udeb package fetching to wget (NFS->HTTP) started with modifying dphys3pressed to use logic from add-preseed test script while at it same modifications, as apply, to all other 3 ex dphys2* scripts eliminate all kernel 2.2 stuff, as sarge is allways 2.4 currently no need for 2.6, as custom 2.4 is prefered to stable sarge 2.6 actually remove all 2.2.20 or 2.4.18 and 2.2.* or 2.4.* references and remove BF24 variable naming for 2.4 as that is now standard while at it change all remaining mentionings of woody to sarge also remove woody proposed upgrade stuff, if needed do in config package remove all remaining dbootstrap, sfdisk, lilo.conf and debconf-preload stuff dbootstrap and debconf-preload not needed with pressed, dito sfdisk lilo is now replaced by grub, and that should also be config by package 2005.03.31 Thu further modifying dphys3pressed, DEBUG_* example 2->3, SYS_* file names also no SYS_MODULESMISC for selective copying, only mod /l/m/2.4../k/drivers/net no SYS_DRIVERSTGZ_ON_FLOPPY as no "floppy" archive, drivers direct in /lib no SYS_DPHYS2CONF_ON_FLOPPY as no dbootstrap which uses it endfirstrun elimiated using merged config, as never used anyway NAME and PNAME new names, eliminated /etc/default configs as no /etc/init.d synchronised dphys3/dphys3 and dephys3/debian/dphys3.defaults, del later Makefile put dphys3/dphys3 into example dir, together with endfirstrun removes -w SYS_WORKDIR setting option, as useless, parser+help+manpage tidy up rest of option parsing and help, no mentioning project name now preseed will have to install both floppy images, kernel none update paths on image server, sarge, images/floppy/* put CONF_DEBSERVER and CONF_OWNSERVER into path computation, not URL constr DEBRESCUE and OWNRESCUE renamed to *BOOT, fitting to boot.img file names dphys3cd BOOT*->CD* so that RESC*->BOOT* is possible both kernel and preseed get both boot and root, preseed both own version change drivers tar archive to drivers floppy image, also preseed own vers added code for changing timeout on boot floppy deleting stuff for space and adding net drivers commented out, may use later added code for install .udebs, preseed file, fiy load-floppy 2005.04.01 Fri further modifying dphys3pressed, end[1|2]stage scripts, renamed all references renamed my preseed.bclc to end2stage.simple, merged in old endfirstrun.simple renamed old endfirstrun.dphys-admin for Makefile, but not changed internals dphys3pressed package floppy, upload, fetch/install (temporary), remove now all of add-preseed stuff is in, but still using given preseed.cfg file test runs, fixed a few wrong variables, typos, wrong tests, works converted old endfirstrun.dphys-admin to new end2stage.dphys-admin format extended dphys3pressed to generate preseed.cfg instead of using file converted file to commands, make just active lines, no editing, no comments moved */late_command preseeding into user scripts section, conditional also changed these to fixed filenames when on image and target system 2005.04.07 Thu dphys3pressed add DEBUG_DUMP_PRESEED to dump generated preseed.cfg file convert all dphys3* to using pkgtools-like command variables for output control and also for quiet/verbose/debug options, added COMMAND_OPTIONS while at it set all wget to -q, so they do not bury our messages in crap CMD_INFO_PRINT for progress messages, get rid of long ---------- lines CMD_DEBUG_SLEEP and CMD_DEBUG_PRINT for less framing code, consistency use all CMD_* for errors, split into ERROR and FATAL dphys3pressed generate variable preseed stuff, keyboard layout drop disk selection (allways first), drop IDE DMA (not implemented) swap make 2 recipies (without or with), drop network conf (allways eth0 DHCP) drop lilo (do this later), timezones taken over, root password taken over drop MD5 and shadow (allways used), drop PCMCIA (not implemented) contrib/non_free taken over, drop proposed (do this later), added security drop non_us and security servers (do this later), silence debconf taken over set mirror host and directory from CONF_DEBSERVER, switch testing->sarge tidy up config file, as no debconf-preload, and no base-config numbering update debconf options list in man page, merge into main list test run, works, no errors, installs ok dphys3root chopped down version of dphys3preseed, just -d/-f merged, -i and -r dphys3boot identical file, just changes some names root->boot make man pages dphys3root and dphys3boot, sync with dphys3cd and dphys3pxe deleted stray -n options, added all -q and -v options, updated "see also" updated dphys3 overview manpage, added dphys3root and dphys3boot dphys3cd and dphys3pxe also remove all originals CONF_* and remarks started updating man page dphys3.7, show what boot/root/cd/pxse do and need 2005.04.08 Fri finished updating man page dphys3.7, overview of command and config options dphys3pressed and dphys3kernel removed floppy handling (-f and -i), updates man changes comments and man pages for consistant "download", no "fetch" dphys3pressed eliminated "remove files" section modules handled by dphys3kernel anyway, and other files changed/irrelevant dphys3pressed # lines blocks, removed file del for space, reduced driver stuff test intsall with dphys3pressed -u and then dphys3[boot|root] -di, real floppy fix up dphys3pressed man page, to boot+root instead of root, no drivers arch while at it checked FAQ, updated INSTALL (incl warn what missing), README update dphys3cd, while at it fixed missing dependency on syslinux renamed all 2.88 stuff to boot288, not cd*, as not CD specific got rid of modifying syslinux.cfg for initrd, as that is now standard test run shows CD boot until needing 2nd floppy, as desired inserting root floppy has then the expected effect copy in contents fo root floppy image (final initrd) test run claims not bootable disk, despite syslinux and kernel being on it was root floppy still visible, removed, vmware tries hardware CD, not .iso was vmware PC BIOS, hat CD disabled, now it boots, kernel panic no init root initrd init for kernel 2.2.* removes /sbin/init runs busybox init else copy everything / onto ramdisk /mnt, cd, pivot, chroot busybox init but before that read error because block no larger than device looks like we will need to merge the 2 initrds, look at boot initrd init zcat 2nd floppy initrd.gz to ramdisk /mnt/tmp-initrd -> /mnt, cd, pivot before this loads usb drivers (not needed by us anyway) after this pivots root, deletes /sbin/init, ln -s to busybox, chroot init annother test, ist actually doint pivot stuff, then failing in ram disk further test show it in not pivit failing, init is not even run fails to run because kernel parameter ramdisk_size=1838, too small get rid of that in dphys3pressed when setting syslinux.cfg timeout, works 2005.04.14 Thu update dphys3pxe, not just extract kernel, also now extract initrd test install on dudley, tftp server set up directory dphys3, works but due to only 32M RAM used low memory setup, failed installing dash look at including drivers from net-drivers.img, contains small set of .udebs each quite large, with many drivers of one class in it, so not simply unpack nic-extra-modules-2.4.27-2-386-di.udeb is only really interesting no interest in ipv6, isa, network-preseed, pcmcia, parport or plip and at 590k too large to simply include entirely, so need to extract stuff only content ist /lib/modules/2.4.27-2-386/kernel/drivers/net/* so easy, just udpkg install and then remove the unwanted stuff (near 1.4M) if CONF_MODULESNET get nic-extra-modules from net-drivers.img and extract test run, included selected forcedeth.o module test runs all 5 scripts with full -dgur or -dir or -dgir, all run all man pages fixed bug, ./dphys was killing entire line, do without ./ dphys3kernel, and its man page and index.html.en added warning that this piece of functionality does not work yet, not implemented 2005.04.15 Fri dphys3cd test real burned CD, burn on aragog, reading on pomfrey fails CD is readable on malfoy, pomfreys CD drive (never used) is a dud, replace test now works, boots and installs off CD-RW burned on aragog modules stuff test install on only-forcedeth box, module there but does not want to run on this K7 hardware, already too new we don't use this feature anyway, so leave it untested, as include works first debian packaging, made number, FAQ/INSTALL/Index.html.en updated test shows remove errors if files not there, rm -r -> -f same debugging ls use 2> /dev/null so there is no error if no file user test, dphys3.7 missing comment that dphys3kernel does not yet work also in example config file same remark, so user sees it there example dphys3 configs belong in examples subdir of package name INSTALL file should be installed, as it is mainly a guide but no compile/install in Debian, split into INSTALL+QUICK_GUIDE but simplify it, reduce series of steps from 20 to 13 order of config options in man pages and example config not allways the same standardise these, and also in code set defaults in same order divider lines in config file more visible, as user did not even notice them QUICK_GUIDE on target missing text for PXE, only mentions floppy and CD generally tidy up all floppy vs CD vs PXE variants QUICK_GUIDE remove -u from kernel|preseed and -d from other 4 commands to also make the examples as simple and non-overloading as possible grub crashes with error 18, aragog still has old BIOS that wants /boot so we still need this, despite already last time regarding it as outdated add 30M /boot (sarge uses 5.7M, woody had 23M and used 2.6M) testing changes noticed that $primary and $bootable getting lost, added \ now complains too many primary partitions (2!), make only /boot prim removed $primary from all non-/boot, works /boot now sda1, / sda5 in sda2 aragog now installs without any problems, with users config released 1st milestone, copy to ../dphys3-20050415, make tar file of it modified index.html and FAQ and INSTALL to show the milestone release redo ../dphys3.tar.gz archive for current state packaged version 20050415.131020-1, changelog: "initial release: no kernel, preseed, boot/root floppy, cd, pxe" 2005.05.13 Fri added CONF_IF option to force using 2nd ethernet, set to auto dphys3preseed set and use, dphys3 commented out default, man page doc analog to dphys-admin, dphys-config and dphys-pkgtools rename dphys3 config example into dphys3.example, Makefile, index.html packaged version 20050513.111920-1, changelog: "added CONF_IF support" 2005.06.09 Thu renamed end2stage.dphys-admin to end2stage.dphys, as now more in it reworked it to first config/install/run dphys-config then install/run dphys-admin (config and package list from dphys-config) man pages config files particular project dphys3 -> ./dphys3, as in pkgtools added remark "Standard sh syntax rules apply.", also as in pkgtools user test install of a machine, with end2stage.dphys fixed need for manual apt sources update, and manual running dphys-admin 2005.09.01 Thu dphys3preseed manpage CONF_CONF_MODULESNET removed second CONF_ 2005.09.08 Thu all scripts added exit 0 at end, following new policy 2005.09.09 Thu planning, wanted to use standard kernel-image, but it has no initrd and devfs the later is not allowed in standard kernels, so need separate one only one, p1-1gb, as it will be replaced, no .deb, just blank kernel find out what kernel specific stuff gets stored where on images present has in kernel, boot+root+netdrivers modules + downloads we will need just kernel and our modules, download our standard ones data from senguen, how he patched his custom kernel disks own kernel, initrd root image replaced /lib/modules with fitting /usr/share/discover extend the PCI ID list for new(er) drivers end1stage wget kernel, PATH, LD_LIBRARY_PATH, and install with dpkg preseed call end1stage (existing script knows how to do this) preseed added setting d-i anna/no_kernel_modules boolean true and made an remark, that preseed file had duplicate content 2005.09.15 Thu started dphys3kernel, merge in dphys-kernel-packages> file determination stuff need own kernel, only install, compile only 1 (p1-1gb without ext modules) will be installed on boot floppy, w modules, no .deb needed, no make-kpkg 2005.09.16 Fri all scripts bugfix, first debug output only after debugging commands defined check/set for missing known needed config options CONFIG_BLK_DEV_FD, CONFIG_BLK_DEV_LOOP CONFIG_BLK_DEV_RAM CONFIG_BLK_DEV_INITRD, CONFIG_DEVFS_FS, CONFIG_FAT_FS, CONFIG_VFAT_FS got rid of old config extraction code, and CONF_KERNEL_CONFIG setting reworked actuall installation onto floppys and initrds, kernel no need for modules on boot floppy, as no USB, IDE+SCSI already in, rest add so we can save work for extracting wanted modules from all to insert here but resulting kernel is 1039485, larger than Debians fully modular 764502 reduce size of initrd.gz 658413->354856 by deleting now unneeded modules 2005.09.22 Thu root floppy initrd copy without modules space used, for unused space with zeros so that when gzipped it actually becomes an smaller image.gz file root floppy replace/install modules 2005.09.23 Fri root floppy /usr/share/discover extend the PCI ID list for new(er) drivers decided to leave this unimplemented, not needed for bugfix/sec-hole updates implement it when we actually need an update for new devices drivers 2005.10.14 Fri /usr/share/discover needs extend, already for 2.4.27 -> 2.4.31 and our hardware compile test, failled to modules_install into initrd file system size because too many modules produced, reduce config 2005.10.20 Thu reduced config further, now fits tar copying to new ramdisk complains of Debian file dates which are 1970/0s no option to disable this questionable feature, so 2> /dev/null fails to produce new root image, 0byte File, broken input file, test with -dg look about discovery pci id database, no automatic update possible so have user give an facultative diff file to be applied for tests from now on use official senguen kernel archive and config and now the kernel is too large, even with reduced initrd, to fit floppy so delete even more drivers+stuff not used while install, such as 100k ACPI 2005.10.21 Fri dphys3kernel end1stage script echo "Debian-exim:x:202:" >> /etc/group (for dpkg, else error) but better run entire thing chrooted, no echo, no PATH, no LD_LIBRARY_PATH no anna preseeding, already real dpkg, even apt-get can be used, if config even so problem with needig to adapt dphys3kernel every time we get new external driver module packages, so better separate end1stage script even that requires new building with dphys3preseed, better fetch from server same also applies for existing end2stage script (dphys-[config|admin]) better extend dphys3preseed w ability to fetch server endstage scripts so dphys3kernel is done, update its man page get rid of unused CONF_KERNEL_OWN from dphys3kernel, and also rest of project mainly only in man pages, incl of parts which it was irrelevant for! move author/lastmod before copyright through entire dphys3 2005.10.27 Thu changes SYS_SOURCEDIR to CONF_SOURCEDIR, add -s option to set it no CONF_TARGETDIR, as no debs made like in dphys-kernel-packages dphys3preseed endstage alternative to included files also wget modularised system for building up both late_command strings decided that end1stage runs definitely in /, not /target document everywhere so (comments, man page) and man page show how to use also updated dphy3.example and dphys3.franklin with comments 2005.10.28 Fri end1stage script for installing custom production kernel, designed functions use full apt-get, instead of wget+dpkg, so temporarily patch sources list added end1stage.dphys to Makefile dphys3preseed file check if it has duplicate content, can not see any reason dphys3preseed no security in apt.list, may delay/question, have own list later already hat CONF_* options, set then default to no instead of yes test run, fails to find 1.02 version of preseed-common and initrd-preseed these were taken from pool, and so from etch, now only 1.10 there (and sarge) get current versions, and from now on use these, no more downloads test run, generates image, when runnung modprobe complains no modules.dep root disk then goes until "no network interfaces detected", lsmod empty install system has still got the old Debian 2.4.27 modules typo, forgot B->R, so new initrd overwrites already packed bootfloppy initrd test run, dphys3preseed claims no udpkg on root floppy initrd debug difficult because part gone stuff, use image handling as dphys3kernel test run, still udpkg missing, noticed gunzip "unexpected end of file" and dphys3kernel "gzip: stdout: No space left on device", so initrd broken test for gzip failing, but also other filesystem full stuff, so try set -e runs until there, would gzip -9 to 1.9M, larger than 1440k floppy put in further deletion of modules, after module install, to reduce size now down to 1540k, and with 190k of sk98lin.o in there 2005.11.03 Thu has multiple large drives which may not be needed, ask senguen when back temporarily kill sk98lin to do further testing, works, pressed also test run in vmware, fails modprobe pcnet32, unresolved symbol, crc32_le needed compile without deleting it, fails because initrd image too small change to copy to main disk, work, copy back to empty initrd image now crc323 is back, kernel/lib was never deleted, was also space problem add tests and error message for too little space when packing test run, now finds pcnet32 and crc32 modules but then "Download installer components" complains of no modules found click on yes and it works, despite claim it will most likely not do so gets to preseed/late_command, wegt fetches script to /e2s, runs script script reruns itsself in /target, no loop as no /target, then stops no actions taken, stops long before read dummy occures turns out did get to dummy, but all commands apt-get `dphys-kernel-cputype` and dphys-kernel-cputype failled because of missing /proc, mount that test run, still no modules found, ps auxw says it is anna netretriever fault most likely the test with hw-detect/missing_modules some compiled in here and e1s seems to not have run at all, no wait, no stop at end rewrote using explicite logging, not just stdout into file test run, works, installs proper kernel, boots directly with it 2005.11.04 Fri senguen sk98lin.o reduce size, or preferably remove other drivers as easier found 6 unused ones: dmfe lance tg3 tlan typoon tulip, remove from config then remove deletion of sk98lin.o, so all used ones are back senguen "Download installer components" complains of no modules found is same error that he handled with d-i anna/no_kernel_modules boolean true he got the info to use this from Joey Hess (d-i author) himself boot disk modprobe errors, add an empty/minimal modules.dep, has no effect empty line instead of 0 length also no effekt try running depmod, but that does not even generate an empty file 2005.11.10 Thu senguen reports standard kernel config now fixed, unneded ethernet out so now re-allow sk98lin.o to be included, now fits on disk boot disk modprobe errors, take existing modules.dep, sed release number now modprobe works, but now insmod complains about modules not found they are of course already compiled into kernel, as far as we want them killing modprobe will later fail us, when root disk is inserted fixing up scripts by dropping modprobe may produce syntax errors better make /bin/true out of it, in /sbin/init and /usr/sbin/usb-discover works now, just "not found" errors for uname and cut both are used by /usr/sbin/usb-discover, but not on the boot floppy! root disk installer anna error dialog about missing modules dphys3preseed added an line to preseed, allways, even if no custom kernel /bin/echo >>${PS} "d-i anna/no_kernel_modules boolean true" test run works without any problems update docs to reflect completed state of dphys3 index.html.en, FAQ, QUICK_GUIDE, README Makefile add *preseed*.udeb files to tar archive, and install in lib dphys3preseed use them now from lib, index.html.en link preseed .udebs fixed wrong build dependency tosyslinux, is runtime dependency released 2nd milestone, copy to ../dphys3-20051110, make tar file of it modified index.html and FAQ and INSTALL to show the milestone release redo ../dphys3.tar.gz archive for current state packaged version 20051110.171107-1, changelog: "added endstage from http://, custom kernels" 2005.11.11 Fri senguen Details for Bugreport for bootfloppies boot initrd missing stuff /usr/sbin/usb-discover needs uname and cut to recognize 2.6 kernel but both of these are not present on the initrd, so not found, 2.6 fails for extracting /usr/sbin/usb-discover ran an kernel compile, hangs pstree shows it to be patch, which recieved no input, because file deleted insufficient test for not patching case, if PATCH not set don't run it also reconstruct deleted file, from backuped .img files, put on server packaged version 20051111.151816-1, changelog: "bugfix hangs when no pci.lst diff file" 2005.11.24 Thu looked into what/where boot disk /export and /scratch problem on reinstall are all astro and the new large particle workstations consistent /export with last /export/data and /scratch on hda5|sda5 set by /dphys-setup/extra-fs and /etc/init.d/dphys-setup-extra-fs better ignore the making, just on reinstall test for existance and avoid mkfs test if system is already sarge (not older) and if so just kill directories investigate what actually happens around partitioning/formatting time is installer, but downloaded code, not stuff from root.img initrd so run an standard Debian install and grab an .tar.gz of the install system stupid install busybox tar dosn't know c option, target tar can not find lib chroot target allows target tar, but no installer system access then cp -a does full system (installer+target copy, incl recursive the copy but tar with LD_LIBRARY_PATH= and -f on /target worked 2005.11.25 Fri investigate what d-i partman does, total chaos, >5 level indirection so tried partial install run, looked before partitioning, no /target after "use old" partitioning /target, mount, /t/etc/fstab, /t/media/floppy0 look for mechanism that called by /u/b/main-menu calls /bin/partman seems to at least involve /var/lib/dpkg/info/partman.postinst as last step in between is only an udpkg --configure partman visible, who calls that? all this stuff that is not on boot/root floppy, is downloaded so would need on floppy modifier that patches this at install time alternative do it all by stuffing data into preseed.cfg but then need to get rid of "unclean target" error from base-installer also problem with needing different options for new install and reinstall best strategy seems to be only call partman if new system, else own mounts 2005.12.01 Thu investigate how /usr/bin/main-menu gets to call udpkg --configure partman all other dialogs before partman are result of udpkg --configure no info for order to execute them, so where ist directory with S??* names strings /usr/bin/main-menu has no dir name, but /var/lib/dpkg/status inside that there are special installer-menu-item: ?? lines so we definitely need to modify the /var/lib/dpkg/info/partman.postinst file or possibly add something to /lib/partman/init.d which mounts+terminates as all this is only downloaded while install, edit by inserted script running script before /usr/bin/main-menu is too early, not downloaded yet adding additional item called by main-menu requires modifying /v/l/d/status so we need to hijack something called by main-menu to run our edit script best something which uses an *.d/S??* collection, such as partman init.d but partman runs init.d/S??* in an for loop (but with if [ -x ]) and then fixed while true with ask_user, so we would need to kill parent and this most likely will piss of main-menu because of non-0 return value so better try an attach something to anna, after download/unpack of .udebs there replace partman.postinst with our logic anna and calling is also code which is already present on the root floppy 23 choose-mirror, 23 download-installer | 35 hw-detect-full, 42 partman download-installer.postinst just runs anna net-retriever anna then uses /usr/lib/debian-installer/retriever/$1 that gets called once for Releases and cat-ing of all Packages download then once for each package to fetch, here overwrite partman.postinst or alternatively run something after anna, from download-installer.postinst simpler to modify, just add stuff on end of script, not even exit in way heeb expects it to be 2 different disk sets, new vs re- install to have user decide, no automatic heuristic test, so permanent change heeb we should document all that we have learned, text on web, for others started collecting details which I will want in this 2005.12.02 Fri dphys3preseed started adding reinstall code, is quite site dependant decided better make an generic end0stage support and then external script wrote code to after 23anna delete partman call, make mountpoint + mount save stuff, delete rest, restore stuff, mount /boot, empty it added code to dphys3preseed to call this, added to config files updated man page with info on END0STAGE usage, and example 2005.12.15 Thu test run dphys3preseed with end0stage, "download installer components" fails add exit 0 to end of extended script, but does not get that far anyway fails removing "42" entry, fixed missing / (typo) test run, removing "42" entry now works /target is made but not mounted, misspelled disc as disk, fixed noticed missing /target in many following paths, added test run, no abbort any more, but still runs the partitioner add logging to end0stage, and wait at end of end0stage test run, 42 line is gone, killed download installer postinst, back to menu run "detect hardware" manualy, that calls partitioner, grr 35 hw-detect-full.postinst only runs /bin/hw-detect script, no call partman so menu must be continuing, after seing exit 0 from /bin/hw-detect kill actual partitioner postinst script, nothing in it test run, no partman, but after reboot grub fails to find kernel look at /target, /target/etc and fstab are missing, so nothing to mount fixed -d -> -f tests for fstab file, so do new install, then test again test run, now boots into second stage, works add test for credibility of upgradability, give error and hang if not the case test fails from installer script, works on installer shell, and local shell 2005.12.16 Fri improved credibility tests and there error messages, separate ones tested again disabling of partitioning, just removing 42 line fails tested disabling of partman postinst, that allone works, so no 42 stuff test run, full install, make /export a /scratch, data in them, reinstall, works but at end error message about /e2s, and it was not executed no /e2s installed in /, but a copy of /e1s is there already no e2s on /, so can not be copied to /target, nor run because trying to wget it at runtime, despite filename and not URL, typo case of an bug that only became visible by test case e1s URL and e2s file while at it simplified handling of end[1|2]stage script command generating now simpler to only wget end2stage script after second stage install while at it made some tests more robust, in case blanks in user input document end0stage stuff, QUICK_GUIDE, index.html.en (dphys3preseed.1 already) improved config file comments on how end?stage are run, as root, and where / all CMD_ stuff before option processing, error+normal+output+rest in options overwrite whatever, then print options at end of options section 2005.12.22 Thu CMD_ stuff final code, test runs, works, improved a bit more, test runs, works started writing text about Debian/sarge "d-i" install process preserve/distribute our gained knowledge from dissecting it 2005.12.23 Fri continued writing text about Debian/sarge "d-i" install process drafted entire first install stage, our modifications to it 2006.01.05 Thu continued writing text about Debian/sarge "d-i" install process corrections to finish first stage and mods, convert to HTML 2006.01.06 Fri continued writing text about Debian/sarge "d-i" install process add links, expanded, all pressed one place, added to Makefile and index.html stated on 2nd stage, all done with exeption of base-config 2006.01.12 Thu continued writing text about Debian/sarge "d-i" install process rest of 2nd stage (base-config), found 110->300M base install bloat reason corrections and extensions, text to senguen to read for typos he also got comments on some technical points from debian devels kbd-choser why twice, installer-Menu-Item behaviour, why run-debootstrap base.config why debconf merging not in an menu-called script packaged version 20060112.175737-1, changelog: "added support for non-formatting install, install process description" 2006.05.18 Thu improve online help and man pages analog to makesourcepackage commands/options section split, also no command gives error message while here fix small stuff that has accumulated in -D option shift 1 after all if [ DEBUG_$1 0 stuff dphys3kernel DEBUG_DUMP_CONFIG only .config, not System.map (not yet exist) add set -e to all scripts, as in dphys-pkgtools and dphys-kernel-packages packaged version 20060518.162647-1, changelog: "improved help and man page, commands and options separate" 2006.08.11 Fri added all "packaged version ????????.??????-?, changelog:" lines 2006.10.26 Thu rework for separate upstream and Debian maintainer, only .tar.gz release reworked and updated documentation, FAQ, INSTALL, README, index.html.en also QUICK_GUIDE, added pointers to dphys-kernel-packages index.html.en added better pointers to QUICK_GUIDE and end0stage move planning docs into subdir planning, makes main dir tidyer 2006.10.27 Fri further rework separate upstream and Debian maintainer, only .tar.gz release remove old debian/ so that senguen can make proper debianisation debian/changelog is all info duplicated in this Logfile, deleted it debian/control is just docu text, add to other docs debian/copyright is same as in README, deleted it debian/dphys-config.docs is nothing, deleted it debian/rules has nothing specific, deleted it set new systematic PATH=, removed all absolute paths in commands in end[012]stage missing set -e, added it, code seems OK to run with it if ${DO_DOWN} test if wget exists before using it ./${PNAME} config file, can not collide with install as ./${NAME} checked for cut -f test that can fail if separator missing while at it in dphys3preseed in mirror setting test for http: or ftp: before first /, unlikely to then be file instead of URL quoting added echo "some test", and ${CMD_..} "some text", also VAR="some text" and [] tests with "${}" instead of x${}, also `` quoting noticed still old [ "`grep ''`" = "" ] style, switch to just grep -q '' "${VAR}" up to dphys3kernel 2006.11.02 Thu further quoting, "${VAR}" from dphys3preseed on 2006.11.03 Fri further quoting, "${VAR}" from dphys3pxe on, "${}/*" -> "${}"/*, ~ in "" check for for .* in .*; with ${VAR} needing "" but not allowed automatic test for left overs, missed conversions, fixed, keine echo -e improved text in sarge-install-process.html, better legibility added more details and remarks about selection of what packages get used in particular how tasksel selects what packages it wants to include after this week having a lot of trouble (autoinstall hanging on HTTP 403) after 3.3r3 -> 3.1r4 sarge change screwed this, changed list, lost inetd update end*stage scripts to currently used ones end0stage.dphys is my old original unchanged, just need replacing when tested added better debug wait at end, with CONF_ option for enabling 2006.11.09 Thu further updating end*stage scripts to currently used ones end1stage.dphys also functions unchanded, added debug wait also changed vmware smp missidentification patch vmnet-* name dependant, not from cpu type, so script more portable end2stage.dphys also functions unchanded, runs at end, so no debug wait test runs, dphys3kernel -g -k works, but with -c error that -k wrong was [ -f ], finds filename (symlink) but no file behind it, changed message also when -s wrong, error message with KERNEL=.tar.gz outdated KERNEL= test, fixed to same test as in dphys-kernel-packages works now, but 2.4.33 kernel will not fit into floppy image any more better gzip usage, compress on HD, then copy, to see size, boot and root went through SYS_FORCE_UNCONFIGS list, updated it while doing so, dumped generated .config, nothing removed suggests for loop problem, was tr filter, which was added for security still fails, was new test with grep, which was missing inversion now can pack boot disks, but fails on root disk, also too large is result of far too many modules, inkl unused filesystems, drivers, etc did massive tidy up from 1.9M down zu 1.4M of lib/modules 2006.11.10 Fri further test runs, dphys3preseed, same better gzip usage, only root needed -d fetches wrong disks, is due to needing an option not used in dphys3kernel better use same system, presence of an kernel archive and/or config file while fixing above noticed that all 3 disks own ones fetched but dphys3kernel only generates boot/root, no drivers, fixed this again problem with space running out, too many drivers, remove a few more also get rid of 100k /unifont.bgf font, only for non-alphabetical menues dphys3boot, dphys3root, dphys3cd, dphys3pxe simply worked OK dphys3.example changed to new option usage, expanded comments updated man pages for new dphys3preseed floppy image select method at same time reworded some parts, general text tidy up, fixed errors also extended dphys3kernel man page list of what config options are killed 2006.11.16 Thu further man page improvements, download driver image only if Debian images renaming various CONF_* parameters for better documentation killed seldom/never user -b option to dphys3[kernel|preseed] then merged "${CONF_USER}@${CONF_INST_SERVER}:${CONF_INST_BASE}" like PXE changed man pages examples to reflect this change test all dphys3*, still work OK, no code errors do test installation from CD, fails, no init found fall back to test installation from floppy, abborts with ram disk full is dphys3preseed bootfloppy syslinux.cfg ramdisk_size=, size changed use an '[0-9]* ' to match numer, not constant 2006.11.17 Fri ram disk full because still initial initrd, because mount of new failled this was because tmpfs not in kernel, boot initrd is ramdisk, but root tmpfs changed dphys3kernel to not delete tmpfs, and ensure its presence now runs until failing dhcp, missing socket filter(!) for dhcp changed dphys3kernel to ensure net, packet filter and socket filter install now works compeltely, right up to offering full ssh login 2006.12.01 Fri after all bugfixes and changes new release updated FAQ, INSTALL, index.html.en copy to ../dphys3-20061201, made tar file of that remade ../dphys3.tar.gz archive for current state ------project status doing: todo: dphys3.1 or dphys4 for Etch, not limited any more by 1.44 floppy size or switch to dphys4 for ubuntu instead of normal Debian possible extensions: done: taken over stuff from dphys2, adopted taken over docs removed now unneeded dbootstrap and debconf-preload scripts analysed preseed process, fixed it for floppys, package preseed stuff made dphys3pressed, dphys3root, dphys3boot, updated dphys3cd, dphys3pxe adapted end2stage.dphys to using dphys-config and dphys-admin made dphys3kernel compile, install into image, expand discovery database extended dphys3pressed with endstage scripts download while install made end1stage script to install production kernel extended dphys3pressed with end0stage script for while generating root floppy made end0stage script to clean target filesystem without re-partition/format