Packagemanager-LFS-PackageUser-System git · main
Package-user based package manager for LFS/BLFS
Shell 63.7% Markdown 35.8%git clone https://git.christianimmanuel.de/linux-from-scratch/Packagemanager-LFS-PackageUser-System.gitwget https://git.christianimmanuel.de/linux-from-scratch/Packagemanager-LFS-PackageUser-System/archive/Packagemanager-LFS-PackageUser-System.tar.gzUpdate stacks And fix bugs
BUILD.md | 18 +- HANDOFF.md | 1262 ++------------------------- Makefile | 18 +- NEXT-TASK.md | 156 +--- README.md | 78 +- blfs | 151 +--- kernel.conf | 32 +- lfs | 9 +- lfs-helper | 413 +-------- lfs-kernel | 957 +-------------------- lfs-phases | 95 +- packagemanager | 2233 ++++-------------------------------------------- stacks/app-users.conf | 22 +- stacks/app-users.sh | 120 +-- stacks/avoid.stack | 13 +- stacks/bootscripts.sh | 39 +- stacks/diag.stack | 14 +- stacks/machine.conf | 22 +- stacks/media.stack | 49 +- stacks/net-wifi.sh | 223 +---- stacks/network.stack | 2 +- stacks/services.stack | 62 +- stacks/setid.conf | 50 +- stacks/sway.stack | 423 +++++++-- stacks/vision.stack | 2 +- test_lfs_crosschain.sh | 816 +----------------- 26 files changed, 784 insertions(+), 6495 deletions(-) diff --git a/BUILD.md b/BUILD.md index c33bde9..18b09b2 100644 --- a/BUILD.md +++ b/BUILD.md @@ -190,11 +190,9 @@ updates an installed package. To move packages to what the book now carries: here than the version its page was written against -- not a refusal, just the numbers. -The desktop is three stacks that build as one: `wayland` (session, mesa, -wayland, GTK4, wlroots), `media` (the codec / gstreamer / WebRTC / webkitgtk -chain) and `sway` (the compositor, its tools, your own programs). sway.stack -starts with `include wayland` and `include media`, so a full desktop build is -still one command -- and when a codec or webkitgtk needs redoing you rebuild just that +`media` is the codec / gstreamer / WebRTC / webkitgtk chain. sway.stack +pulls it in with `include media`, so a full desktop build is still one +command -- and when a codec or webkitgtk needs redoing you rebuild just that part: packagemanager stack media --run @@ -213,13 +211,10 @@ usermod -aG audio,video,input <name> # the main_user from the interview export LFS_DESKTOP_USER=<name> packagemanager stack sway --check-refs # every git ref still exists? -packagemanager stack sway --run # desktop: includes wayland + media, then sway, foot, pipewire -packagemanager stack apps --run # zathura, imv -- programs, not the desktop +packagemanager stack sway --run # desktop: wlroots, sway, foot, webkit, pipewire, mesa packagemanager stack network --check-refs packagemanager stack network --run # wpa_supplicant, iptables-nft, net-tools packagemanager stack services --run # bootscripts, fcron, acpid, mytools, shell-env, app-users -packagemanager stack dev --run # gdb -- nothing needs it, so it goes last -packagemanager stack tools --run # trans, jq -- same: nothing needs them ``` `sway` includes `check kernel-sway`, which reads the running kernel's config @@ -300,8 +295,7 @@ generated, so it can be edited before the first build. | `HEADERS`, `V4L2LOOPBACK`, `BOOTLOGO` | as you like | ```sh -lfs-kernel # the plan; nothing is built -lfs-kernel --run # build, install, record as p_linux +lfs-kernel # build, install, record as p_linux lfs-kernel verify # image, modules, depmod, headers, firmware ``` @@ -403,7 +397,7 @@ packagemanager update <pkg> --run # what the book has newer packagemanager which-package /usr/bin/foo packagemanager reclaim <pkg> --run # give files back another package took packagemanager user create firefox --shared --share-dir --launcher # an app account -lfs-kernel latest --run # try a newer kernel beside this one +lfs-kernel latest # try a newer kernel beside this one ``` A new BLFS release: `blfs fetch`, `packagemanager blfs set-default`, diff --git a/HANDOFF.md b/HANDOFF.md index 2754d00..de7d817 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -7,13 +7,13 @@ account per package, so every file on the system has an owner, and Written for a **SysV** system (no systemd) built from the **systemd** BLFS book, because BLFS stopped publishing a SysV edition after 12.4. - all six tools at 1.14.340 (lfs-backup has its own: 1.3.0) - lfs 2f1eb17 - lfs-helper db72098 - packagemanager 8265dde - blfs 80426a3 - lfs-phases 026ca06 - lfs-kernel fce4ce0 (no version line of its own) + all six tools at 1.14.299 (lfs-backup has its own: 1.3.0) + lfs 8b4b96e + lfs-helper 79163cd + packagemanager 9eb593a + blfs a4b278f + lfs-phases 8077458 + lfs-kernel 219215c (no version line of its own) lfs-backup c73acaa (1.3.0; replaces lfs-config) lfs-sysvbook 62ebe64 (1.0.0; untouched since it was written) @@ -38,69 +38,74 @@ is too long to read. A ThinkPad-class AMD Renoir laptop, SysV init, 93 package accounts, collector prefix `nimgnu`, package prefix `p`. -**Built and verified: 319 packages**, `packagemanager verify` clean — the -whole sway stack (wlroots, sway, swaybg, foot, seatd, elogind, webkitgtk, -pipewire, mesa, gtk3/4, gstreamer), the network stack (wpa_supplicant, libnl, -iptables-nft with libnftnl, net-tools, dhcpcd), services (dbus, alsa, -bluetooth, acpid, fcron, blfs-bootscripts, openssh) and bash-completion. - -**Kernel 7.2.4 built, installed, verified and BOOTED.** `su -` works, the -setuid bits are back, and the desktop runs: sway with foot, and the -application-account launchers in it. - -**The desktop is in daily use.** 14 application accounts (browser, claude, -chatgpt, gemini, nrwgpt, grok, mercury, deepseek, github, codeberg, reddit, -youtube, dokus, mpv) with their launchers, menu entries and icons; `~/Shared` -links for the ten that asked for one; passwordless `su` to each through -/etc/pam.d/su_u_user. - -**Wired networking works and now starts itself.** The interfaces are -`enp2s0f0` and `enp5s0` (not the `eth0` of any example), and `cfg bootscripts` -wrote an `/etc/sysconfig/ifconfig.<iface>` for each: ONBOOT, SERVICE=dhcpcd. -sshd's init script and its rc?.d links are installed. - -**Wireless is the one piece still missing.** The interface is `wlan0` on this -kernel — NOT the `wlp3s0` this file said for months, and not `wifi0`. It needs -two files nothing may write for you, because one of them holds a key: -/etc/sysconfig/ifconfig.wlan0 (SERVICE=wpa, handing off to dhcpcd) and a -per-SSID /etc/sysconfig/wpa_supplicant/wpa_supplicant-wlan0.conf, chmod 600. -`packagemanager stack network --only net-wifi --run` prints both, with the -hand-off variable read off this machine's own `/lib/services/wpa`. - -**Not verified since the tools changed under it:** whether the locale step has -run (see below), whether a .desktop entry launches now that the launcher -allocates a pty, and `packagemanager service`, which has never run on this -machine. +**Built and verified: 319 packages**, `packagemanager verify` clean. That +includes the whole sway stack (wlroots, sway, swaybg, foot, seatd, elogind, +webkitgtk, pipewire, mesa, gtk3/4, gstreamer), the network stack (wpa_supplicant, +libnl, iptables-nft with libnftnl, net-tools), services (dbus, alsa, bluetooth, +acpid, fcron, blfs-bootscripts) and bash-completion. + +**Kernel 7.2.4 built, installed and verified** by `lfs-kernel`: image on the +ESP, modules with a depmod index, headers, v4l2loopback, and 15 embedded +firmware blobs. `lfs-kernel verify` reports 32 checks green. + +**7.2.4 has booted** (`Linux version 7.2.4 ... #6 SMP`, built by `p_linux`). +Whether `kernel.last-good` was updated is unchecked; `lfs-kernel verify` +answers it. + +**Nothing in the desktop has ever run.** The last sway attempt failed in the +chroot for want of `/run/udev`, which is the expected result there. + +**Two things the boot exposed, both now fixed in the tools:** + +- **`su -` is fixed** (`lfs-helper setid --run` did it). +- **No wifi.** The AX200 is detected and has no firmware: + `Direct firmware load for iwlwifi-cc-a0-77.ucode failed`, plus + `failed to load regulatory.db`. `intel` in FIRMWARE_ONLY installs + `intel/`, but the driver asks the loader for a FLAT path, so it never + matched. Fix on the machine: `lfs-kernel firmware --missing --fix`, then + `modprobe -r iwlwifi; modprobe iwlwifi`. The shipped kernel.conf now lists + `iwlwifi-*.ucode regulatory.db*` (their edited copy will arrive as `.new`). + It works in the chroot now: the list is derived from the tree's own + modules and this machine's hardware, not only from dmesg. `regulatory.db` + is genuinely not in linux-firmware -- it is wireless-regdb, and the + command says so. +- **`su -` says `setgid: Operation not permitted`.** shadow installs su with + `install -m 4755`; the install wrapper downgraded it to 755 and, unlike the + chmod wrapper beside it, recorded nothing. Fix on the machine: + `lfs-helper setid --run`. This is very likely the same cause as the open + "root cannot log in" item — check `ls -l /usr/bin/su /usr/bin/passwd` + before assuming anything about /etc/shadow. + --- ## What to do next, in order -Everything here is on the machine, not in the tools. The tools are at -1.14.302 and clean. - -1. **The locale.** Nothing sets `$LANG`: lfs-helper wrote an i18n.sh with the - export commented out (1.14.298) and there is no /etc/locale.conf. Every - program is in the C locale and foot says so at every start. - - lfs config locale de_DE.UTF-8 # if it is not that already - packagemanager stack services --only shell-env --run - # then log in again, or: . /etc/profile.d/i18n.sh - -2. **The launchers written before 1.14.295.** A menu entry does nothing when - the launcher does not allocate a pty for `su`, and a launcher these tools - did not write is left alone. `~/bin/claude` is the last hand-made one: - - rm ~/bin/claude - packagemanager stack services --only app-users --run - - Then click a menu entry. If one still does nothing, run it from a - terminal first -- that path prints everything `su` has to say. - -3. **Wireless.** `packagemanager stack network --only net-wifi --run`, then - write the two files it prints for `wlan0`. - -4. **make-ca's weekly job.** The systemd book enables a timer, which the shim - skips; fcron is installed, so the 12.4 book's cron entry goes in by hand: +1. **Point rEFInd at `/boot/efi/EFI/NimGnu/vmlinuz_new`** as a second entry. + Keep the existing `vmlinuz` entry: it is the way back. +2. **Reboot into 7.2.4.** Then `uname -r`, `lfs-kernel verify` (last-good + should read "it is running now"), and `dmesg | grep -i firmware` — that + last one is the definitive `FIRMWARE_ONLY` list for this hardware. +3. **Start sway** as the desktop user, on a tty: `sway` (an alias for their + own `sway_start`). The seat comes from `/usr/bin/seatd-<user>`, a setuid + wrapper compiled for one username; there is no seatd daemon and no `seat` + group. If it fails, `stacks/sway-session.sh`'s launcher checks the three + things that were each a separate bug: the wrapper, `/run/udev`, and + whether this tty is the active console. +4. **Bring up wifi.** The interface is `wlp3s0` (predictable name, not + `wifi0`), so the file has to be `/etc/sysconfig/ifconfig.wlp3s0`. What + is still missing: + + packagemanager stack network --run # dhcpcd, which was NEVER in it + packagemanager stack network --only net-wifi --run # what else is missing + + then `/etc/sysconfig/ifconfig.wlp3s0` (`ONBOOT=yes`, `IFACE=wlp3s0`, + `SERVICE=wpa`, and the hand-off variable that `/lib/services/wpa` + actually reads — `net-wifi` prints it) and a per-SSID file under + `/etc/sysconfig/wpa_supplicant/`, chmod 600. Their `wifi` script calls + `ifup $IFNAME wifi0` with `IFNAME` unset while `INTERFACE` is set and + unused — one of the two should go, and `wifi0` is the wrong name now. +5. **make-ca's weekly job.** The systemd book enables a timer, which the shim + skips. fcron exists now, so add the 12.4 book's cron entry by hand: cat > /etc/cron.weekly/update-pki.sh << "EOF" #!/bin/bash @@ -108,15 +113,12 @@ Everything here is on the machine, not in the tools. The tools are at EOF chmod 754 /etc/cron.weekly/update-pki.sh -5. **Try the new commands once**, so their first use is not in an emergency: - `packagemanager service` (list, then `service sshd start`), and - `packagemanager stack dev --run` for gdb. - Smaller, none blocking: `dbus` was built before `[dbus] systemd = disabled` went into machine.conf (rebuild or accept); a stray `.service` from -at-spi2-core sits in `/usr/lib/systemd/user`; `/etc/bashrc` on this machine -has a mangled line (`# End /etc/bashrcCK=$(tput setaf 0)`) where an append -ran without a leading newline -- harmless, `BLACK` is simply unset. +at-spi2-core sits in `/usr/lib/systemd/user`; root cannot log in after boot +and the cause is unknown — wants `ls -l /etc/shadow`, `getent shadow root`, +and whether `/bin/bash` is in `/etc/shells`. + --- ## The tools @@ -1014,664 +1016,6 @@ ownership creates work for the next install of the same package.** The setid pass and this take-back are two halves of one cycle, and either alone is a bug. -## 1.14.320 -- the stacks, reorganised: wayland / sway / apps - -Asked to recheck every stack for naming and fit, and to take sway and its -tooling out of the Wayland base. sway.stack was 477 lines answering four -questions at once. Now: - - wayland.stack kernel check, dbus/elogind, the audio floor (alsa-lib, - ffmpeg), llvm/glvnd/mesa, wayland, GTK4 + libsoup3, git, - seatd, libdisplay-info, wlroots. Nothing sway-specific. - media.stack unchanged (codecs, gstreamer, WebRTC, webkitgtk, mpv, - yt-dlp). - sway.stack `include wayland`, `include media`, then the audio - server (after media, so pipewire finds gstreamer), SDL3, - scdoc, sway, swaybg, grim/slurp/swayidle, wl-clipboard, - wf-recorder, light, foot (+tllist, fcft), fonts, and the - user's own programs (minibrowser, swbr, swov, swas) and - sway-session. `stack sway --run` is still the whole - desktop. - apps.stack poppler, girara, xxhash, zathura, zathura-pdf-poppler, - imv -- programs, not the desktop. - tools.stack translate-shell, and jq (moved from sway.stack). - -Every entry was carried over by slicing the file at its comment markers, so -the comments travel with their entries; a script compared the entry lists -before and after (none lost, none doubled). One order change, on purpose: -`book git` now comes before media.stack's git entries (libnice, libsrtp, -...) instead of after them. - -On a built machine nothing is rebuilt: `stack sway`'s progress file is keyed -by the stack you run, so its cached entries stay cached; `stack apps` and -`stack tools` have no progress file yet and re-judge their entries, which -are installed and skipped. - -Tests: every check that looked for an ENTRY in sway.stack now reads all five -desktop files in build order ($_DESKTOP_STACK, built in the preamble); a new -block checks the includes, the placement of key entries, and that no entry -is written in two stacks. The expect= "thin probe" heuristic now accepts -`pkg-config --exists` (it flagged librice and xxhash once it saw them). -Full suite: the same failures as before the split, minus two. - -Reviewed and left alone: network (fits), services (see NEXT-TASK -- its name -undersells it), dev, diag, vision, avoid. - -## 1.14.319 -- tools.stack, with translate-shell - -A new stack for "the small programs you type": nothing boots with them and -nothing depends on them, so none of the other stacks is the right answer. -First entry: translate-shell (`trans`) v0.9.7.1, the newest tag. Built and -installed here with the entry's own two commands (`make` is gawk writing one -script; `make PREFIX=/usr install` copies it and the man page). needs=curl: -it fetches with curl and returns nothing without it. expect= only tests the -file -- `trans -V` went to the network and hung in the sandbox. - -## 1.14.318 -- scdoc - -grim stopped on `Program 'scdoc' not found`. 1.14.316 asked grim, slurp -and swayidle for man pages on the assumption that scdoc was present because -sway's own line asks for them -- it was not on the machine. scdoc 1.11.5 -(SLFS's version, from git.sr.ht) is now a git entry in sway.stack, before -sway, so a replay meets it in order. Not built here: sr.ht is not reachable -from this sandbox and no mirror carries the tag. - -## 1.14.317 -- needs= by the package's name, not only the book's anchor - -grim's `needs=libjpeg-turbo` stopped the stack: the book files -libjpeg-turbo-3.2.0 under the anchor `libjpeg`, so the name found no page -and the stack tried to INSTALL a package that was installed. The stack -now says `libjpeg`, and `_resolve_need` maps a name that is not an anchor -but matches exactly one page's own package name (version string minus the -version) to that page -- and says so. Ambiguous or unknown names are used -as given and fail as before. - -## 1.14.316 -- grim, slurp, swayidle - -Three git entries in sway.stack after swaybg, refs checked: grim v1.5.0 from -gitlab.freedesktop.org (the SLFS page's source; the GitHub mirror stopped at -v1.4.0 in 2022), slurp v1.5.0 and swayidle v1.9.0 from GitHub. Options -asked for rather than autodetected: grim jpeg + man pages, slurp man pages. -swayidle is built -Dlogind=disabled: sway takes its seat from seatd-<user>, -so there is no logind session for that half to talk to; the timeouts work -without it. - -## 1.14.315 -- the shared DATA lives in /home/Shared, and app homes are 0700 - -1.14.312 moved only the LINKS to a hub; the data stayed in -<account home>/Shared. Reaching it went through the account's home, which -therefore had to be group-readable (0750) -- and the main user is in every -account's group, so it could read each account's browser profile and -cookies. Pointed out from the machine, with the fix: put the real -directory in /home/Shared and link from the home. - - * /home/Shared/<Name> is the directory (2770 u_<name>:u_<name>); the - hub is root:<main> 0775. `shared_hub=` still moves it. - * The account's own ~/Shared is a link to it, so `--download-dir - ~/Shared` in app-users.conf means the same thing. The main user's - ~/Shared links to /home/Shared. - * Application homes are 0700 now. - * Data is never moved by the tools: a ~/Shared with files in it is left, - and the one-line mv is printed. The machine was migrated by hand, with - commands given in the conversation, as asked. - * `user delete`: the account's hub directory is removed when empty and - kept (with a note) when not -- it is no longer inside the archived home. - * 1.14.312's argument against /home ("a directory there looks like a - home") was about a hub of links; a hub holding the one shared thing is - what lets every home be private, which outweighs it. - -## 1.14.314 -- `book hatchling` was not "make sure it is there" - -1.14.312 put `book hatchling` in front of yt-dlp, as its build backend. On -the machine hatchling 1.28.0 was already installed, and the entry still -planned a chain from the book -- vcs_versioning first, which stopped on -`packaging>=26.2` (installed: older) -- so the stack ended at "0 of 2 done" -and yt-dlp never started. A dependency that every python build on the -machine already needed is not a stack entry; if it is ever missing the -build says `BackendUnavailable: Cannot import 'hatchling.build'`, which -lfs-helper already maps to its page. Removed. - -Not fixed here, noted: a plan that includes a package whose INSTALLED -dependency is too old (vcs_versioning wants packaging >= 26.2) does not put -that dependency's update in front of it. - -## 1.14.313 -- the machine's own files, adopted - -The machine sent its current app-users.conf, machine.conf, sway-session.sh -and install_mytools. The last two were identical to what ships. The first -two differed only by what the machine had added -- a `git` and a `discord` -account, and `USE_GTK4 = ON` under [webkitgtk] -- so those are now in the -shipped copies, byte for byte. Why it matters: `make install` updates a -stack file only while the installed copy matches the last one it shipped; -an edited file is kept and the new one parked as `.new`. With the two -identical, the next install records them as ours again and later fixes -arrive without a manual merge. - -Also: block 121 had failed since 1.14.307 -- its checks still looked for -the "left alone" wording that release replaced with a real answer (branch -moved? newer tag?). They check the new wording now. - -## 1.14.312 -- the shared hub is a system path; yt-dlp - -**The hub.** The links to every shared directory lived in the main user's -~/Shared. Asked to move them "where a package could reach them too", maybe -/home/Shared. They are now in ONE hub, `/home/shared_user/Shared` by -default -- beside the u_* homes they point into -- and ~/Shared is a link to -it, so nothing typed changes. `packagemanager config --shared-hub DIR` -moves it. Not /home/Shared: /home holds homes, and a directory there that -is not one looks like an account to anything that lists /home. - - * root owns the hub, your login's group may add a link by hand (0775). - * The move takes only links; a file of yours in ~/Shared keeps it a - directory, and says so. Re-running is a no-op. - * The SHARED_* variables come from the hub, so every account sees the - same set. Before, each login enumerated its OWN $HOME/Shared -- and in - an app account that is its data directory, so u_claude got one SHARED_* - variable per FILE it had downloaded. - * PKGUSR_SYS_SHARED names the hub. app-users.sh asks the tool for it - rather than reading that variable, which the running shell inherited - from before the move. - * cfg app-users' expect= now fails while ~/Shared is still a real - directory, so the step re-runs once on this machine and does the move. - -What the hub does NOT change: who may open a directory. Each is still 2770 -u_<name>:u_<name>, and a program runs as whoever STARTS it, never as the -package user that owns its file -- so your own programs already reach -everything your login can. Only a daemon running under its own account -needs a membership, and that is `usermod -aG u_<name> <account>`, or a -collector group (`sysgroup create` / `make-group-dir` / `sysgroup add`) for -a directory several accounts share. - -**yt-dlp** in media.stack, from upstream's release tarball (not a BLFS -page). The release asset, not the git tree: the wheel's shared-data names -the man page, README.txt and the completions, which are generated at release -time, and hatchling stops on the first one missing. Built here with the -book's wheel recipe; `yt-dlp --version` answers 2026.08.19. Pinned, with -the two lines to change when YouTube breaks it. (A `book hatchling` line -before it was removed again in 1.14.314.) - -## 1.14.311 -- --clean deleted current files, and its repair had not run since 1.14.16 - - packagemanager update webkitgtk --run --clean # all green - jsc/jsc.h:29:10: fatal error: jsc/JSCOptions.h: No such file or directory - -`packagemanager script install webkitgtk` fixed it, which was the clue: the -files were missing, not wrong. - - * --clean calls a file an orphan when it is in the old pkg.lst and older - than the install. **"Older" is not "no longer installed":** CMake's - install skips a file whose content has not changed ("-- Up-to-date:"), - so webkitgtk's unchanged jsc headers kept their old date. meson and - `install -p` do the same with timestamps. - * The code knew -- it re-ran the install step afterwards "to restore - current files". That step ran `bash ~/install_<account>`, the copy - 1.14.16 stopped writing, with stdout and stderr sent to /dev/null and - the exit status ignored. It failed on `No such file or directory` for - every --clean since, where nobody could see it. - -Now nothing is deleted on the mtime test's word alone. Candidates are -renamed aside in place (`<file>.pkgusr-clean`, one rename, nothing copied), -the install step runs through the same door as `script install` -(`lfs-helper build --phase install`, logged, exit checked), and then: what -it put back is current and kept; what it did not is removed; **if it fails, -every file is put back and the clean is reported as not done.** A ledger -survives an interruption and is undone at the next clean. - -Tested: an unchanged "header" the reinstall re-creates is kept, a real -orphan goes, no `.pkgusr-clean` litter; a failing reinstall puts both back -untouched and leaves no ledger; and the reapply is checked to log, to return -its status, and to use the versioned script. - -## 1.14.310 -- who INSTALLED it, not who holds it - -Reported from the machine a THIRD time, and this time with the evidence that -finally named the mechanism: - - find /usr/sbin -user p_webkitgtk - /usr/sbin/iptables -> xtables-nft-multi (six links, Sep 12 12:56) - grep iptables /usr/src/pkgusr/p_webkitgtk/pkg.lst - /usr/sbin/iptables - /usr/sbin/iptables-restore - /usr/sbin/iptables-save - -Those links were made by `cfg iptables-nft` on a different day. 1.14.308 -guarded the ownership sweep and the damage came back anyway, because the -sweep was not the one doing it. **The lease was.** - - * A build manifest is built BY TIME WINDOW: everything newer than the - build stamp. A webkitgtk build runs for hours, so whatever root wrote - in those hours -- `cfg shadow-pam` touching /usr/bin/su, `cfg - iptables-nft` making six links -- landed in `webkitgtk.files`. - * Before an install phase, `lease_files` reads that manifest and hands - the package everything in it that someone else owns. Its guard for - root's files had two doors, and **"the file carries a setid bit" was - one of them, with no lock on it**: any package whose manifest named - /usr/bin/su was given it. `chown` then cleared the setuid bit, which - is why the file came back as `p_webkitgtk 0711` every time. The user - was mid-rebuild each time they looked: the lease is held for the whole - phase. - * `lease_settle` decides whether the package keeps a leased file by - `cmp`. **For a symlink that test can never say "unchanged":** `cp -a` - copies a relative link as a relative link, so the copy in the lease - directory dangles, `cmp` cannot open it, and the failed compare reads - as "the phase rewrote it". Every leased symlink was kept. That is the - six iptables links, and pkg.lst then recorded them because pkg.lst is - an owner scan. - * And it was self-sustaining. The manifest fed the lease, the lease made - the package the owner, and the manifest filter -- which asks who owns - the file NOW -- saw the owner's own file and kept it. So the same - damage repeated on every rebuild. - -So both answers to the question asked ("did we do something wrong with the -pkg list, or is there an error in the scripts?") are yes, and they are the -same fault: **nothing anywhere asked who INSTALLS a file, only who holds it, -and that was the thing that had gone wrong.** Two facts that do not depend -on current ownership now decide it: - - * `setid.conf` gained a 5th column, the package that installs the file - (`/usr/bin/su | 4711 | root | - | shadow`). shadow may reclaim its own - su; nobody else may borrow it, and a line with no package is leased to - nobody. The old three-column readers are unaffected -- the group is - still field 4. - * **A symlink belongs to whoever owns its target.** Applied in - `lease_settle` (compared with `readlink`, as `lease_restore` already - did), in the ownership sweep, and in the manifest filter. - -### Repairing a machine that already went through those doors - - packagemanager disown --all # what it would hand back - packagemanager disown --all --run # hand it back - lfs-helper setid --run # put the setuid bits back - -`disown` is `reclaim` in the other direction: `reclaim` pushes ownership -towards pkg.lst, which is right when a package lost files and wrong when the -RECORD is what went bad. It hands back only what the two rules above prove -is not the package's, restores the mode from setid.conf (the chown that took -the file cleared it), and strikes the paths from pkg.lst **and** from the -build manifest -- because leaving them in pkg.lst means `update webkitgtk ---clean` would delete the iptables links, and leaving them in the manifest -re-arms the lease on the next build. - -Tested: the lease fixture gained a setuid file setid.conf gives to another -package (not leased), one with no package column (not leased), a relative -symlink the phase did not touch (given back) and one it repointed (kept); -the manifest filter dropping both kinds; and `disown` end to end on real -files and real accounts -- dry run changes nothing, the setuid file goes -back to root at 4711, the link goes to the owner of its target, the -package's own file is left alone, and both records come out clean. - -## 1.14.309 -- turning the machine off is a membership, and nothing re-checked it - -Asked what a good way is for the desktop user to poweroff and reboot, and at -which step. The answer was already built -- `cfg power` in services.stack -- -and it had the same weakness as everything else that sets a mode: - - * halt, poweroff and reboot become 4550 root:<collector>_power. Setuid root - so they can do the job, executable only by the group so not everyone may: - the privilege is a MEMBERSHIP, which is the same shape as seatd-<user> and - su_u_user. The desktop user is added to that group; a package user owns - files, never identity. - * /sys/power/state and /sys/power/disk are made fresh at every boot, so the - step also writes /etc/rc.d/init.d/power-perms and links it into rcS.d. - udev cannot do this: a sysfs attribute has no uevent, so no rule ever - fires for it. - * **The entry had no `expect=`.** Once cached it was never looked at again - -- and these are exactly the modes that vanish quietly: a sysvinit rebuild - reinstalls halt without the setuid bit, and a chown clears it as a side - effect the kernel never reports (1.14.308, on su). It now re-checks the - group, `4550 root:<group>` on halt, and the boot script, on every run. - * setid.conf says why those three are NOT in it: the group's name is - configured, so a `| root | -` line would keep whatever group the file - happens to carry -- p_sysvinit on a machine where `cfg power` never ran, - which would hand shutdown to whoever builds sysvinit. One place knows the - group, and it is the step. - -Answering the question directly: the step is `cfg power`, it belongs in -services.stack after the desktop user exists and after /etc/rc.d/init.d does, -and it is already there. Running it again is free: - - packagemanager stack services --only power --run - -Tested: the expect= read back out of the stack file and checked as valid shell, -and asserted to test the mode, the owner and the boot script; power.sh still -setting 4550 and still re-applying /sys/power/state at boot. - -## 1.14.308 -- chown destroys a setuid bit, and su was root's - - ls -ald /bin/su - -rwx--x--x 1 p_webkitgtk p_webkitgtk 169952 Sep 15 12:43 /bin/su - -Reported from the machine, for the SECOND time. One command explains both -halves: the kernel clears setuid and setgid on chown, so a single -`chown p_webkitgtk /usr/bin/su` changes the owner AND removes the bit that -makes su work. Nothing warns, and nothing in the file says what it used to -be. - -How the sweep reached su at all: ownership is attributed BY TIME WINDOW -- -every file newer than the build stamp -- and a webkitgtk build runs for hours. -Anything that touched su in that window (`lfs-helper setid`, a shadow -reinstall, `cfg shadow-pam`) made it newer than the stamp. It is root-owned, -and the loop's own comment says "files owned by root are fair game (the -temporary system's)". For /usr/src that is true. For /usr/bin/su it is the -opposite of true. - -Two defects, one symptom: - - * **The root-owned rule was applied in one of the two places that needed - it.** The manifest already drops root-owned paths when the phase ran AS - the package user -- a package user cannot have written a root-owned file, - so whatever is root's now was not its doing. The chown loop walked the - RAW lists again and took exactly the files the manifest had refused. - * **A setuid or setgid file is never the sweep's to take**, whoever ran the - phase and whatever the time window says. Those modes are the system's, - declared in setid.conf, and a chown silently destroys them. The sweep now - skips any file with a mode of the form [1-7][0-7][0-7][0-7] and says so. - -Also, because it turned every update into a keypress: `pm-install` asked -"check that p_imv is the right account" on every run. That is a FIRST-INSTALL -question -- a typo in the account name scatters a package across the wrong -home -- and it is worth nothing on an update, where the account exists and -already owns the package's files. It is now asked only when there is no -pkg.lst and no .pkgusr-installed for that account. A prompt people press -through has stopped protecting anything. - -ON THE MACHINE: `packagemanager stack services --only shadow-pam --run` is the -repair -- its expect= is exactly `test -u /usr/bin/su && stat -c %U = root`, -which is why it can see the damage. By hand it is -`chown root:root /usr/bin/su && chmod 4711 /usr/bin/su`. Worth a look for -others: `find / -xdev -perm /6000 ! -user root -ls`. - -Tested: the mode test against 4711, 2755, 6755 (skipped) and 0711, 0755 -(taken); both guards asserted in the source; setid.conf still declaring su as -4711 root; and the prompt's decision against an installed package (silent), an -account with nothing recorded (asks) and no account at all (asks). - -## 1.14.307 -- "updating from its repo" for a package with nothing to update to - - packagemanager update imv - imv: git package -- updating from its repo. - Dry run -- to proceed, re-run with --run - -Everything there is correct -- `--run` is the missing word, as with every -command here that changes the system -- and it still does not answer the -question the person is asking, which is "will this get me anything?". imv is -pinned to v4.3.0 and v4.3.0 is the newest tag exec64/imv has: the rebuild -produces the commit that is already installed. The command works, does -exactly what it says, and changes nothing. - -`update <git-pkg>` now says which of the three it is, before the dry-run line: - - imv: pinned to tag v4.3.0, which is the newest released -- this rebuilds - the same commit - zz: pinned to tag v4.1.0; v4.3.0 is out - a rebuild stays on v4.1.0. To move it: - packagemanager git-ref zz v4.3.0 --run - sway: master moved 4e2a1b3 -> 9c7f0aa (12 commit(s)) -- this brings it in - -`--no-git-check` skips the remote when there is no network. - -Tested against the real exec64/imv (newest tag -> "rebuilds the same commit") -and a copy pinned to v4.1.0 (-> names v4.3.0 and the command to take it). - -## 1.14.306 -- "phased" means it ends in the runner - - packagemanager update imv - imv: git script isn't phased -- it won't fetch/rebuild cleanly. - Regenerate it with: packagemanager template imv --git <repo-url> - -The script was perfectly current -- format v17, build_pkg, install_pkg, and -`. "${PKGUSR_LIB:-lfs-phases}" && pkgusr_run "$@"` at the end. The test asked -whether it defines `unpack_pkg`, which is the RARE OVERRIDE that replaces the -built-in unpack entirely. A correct script does not define it: the runner's -own unpack is what clones git. So every properly phased git package was told -its script was broken. - -The advice was worse than the warning. `template --git` rewrites the file, -and imv's build_pkg carries `-D c_link_args=-licuuc` -- a fix found once and -not written down anywhere else. Following the suggestion would have deleted -it and the next build would have failed at link time for a reason nobody -would connect to this message. - - * `_is_phased_script()` asks what actually matters: does it END IN THE - RUNNER (`pkgusr_run`, PKGUSR_LIB/lfs-phases, or the `### phases:` header)? - That is the thing that makes fetch, resume and single-phase re-runs work. - * When a script really is unphased, the message now says what a phased one - looks like, and warns that `template` does not keep the build commands -- - write it to a new file and move them across. - -Tested: the machine's own imv script (recognised as phased), an old-style -script that clones inline (not phased), and the whole path end to end against -the REAL repository -- `git-status` on imv reports "tag, nothing newer -released", which is true: v4.3.0 is the newest tag exec64/imv has. - -## 1.14.305 -- a branch is not a point in time - -Asked whether `update` could follow a git package that tracks master, and -show the releases for one pinned to a tag. The first half was not a missing -feature, it was a bug: - - * **`git checkout master` does not move the branch.** The unpack phase - fetches (`git fetch --all --tags --prune`) and then checks the ref out -- - and on an existing clone git answers "Already on 'master'" and stays - exactly where it was. So a package pinned to a branch fetched the new - commits, rebuilt the OLD ones, installed them and reported success, for - ever. It now asks the repository what kind of ref it has: a remote - branch is taken with `checkout -B <ref> origin/<ref>` and says what moved - (`master: b21bd64 -> 6429c4b (3 new commit(s))`); a tag or a commit is - checked out DETACHED, because that is what it is and it must not move; - with no ref at all it follows origin's own default branch. - * **`packagemanager git-status [pkg]`** -- for each git package: the ref, - what kind it is *according to the remote* (guessing from the string is - how a tag called `main` goes wrong), the local and remote commits, how - far behind, and for a tag-pinned package every newer tag. `--local-only` - skips the network. - * **`packagemanager git-ref <pkg> [<ref>]`** -- with no ref it lists the - remote's branches and tags with the current one marked; with one it - rewrites `git_ref=` in the install script (backup kept, dry run unless - `--run`), refuses a ref the remote does not have, and NAMES THE STACK - FILE that also carries it -- editing only the script is undone by the - next stack run. - * `update`'s "N packages follow GIT -- left alone" now answers the question - instead of restating it: which are behind their branch (with the command - to rebuild), which are pinned to a tag that has been superseded (with the - command to move), and `--no-git-check` for no network. - -Tested against two real repositories built in the fixture: one package -following a branch that gained a commit (reported as moved, old -> new sha), -one pinned to 2.0.0 with 2.1.0/2.2.0/2.3.0 released (all three offered, -newest suggested); `git-ref` listing, dry-run rewrite, and a refusal for a -ref the remote does not have (rc 1). The branch bug itself is reproduced in -the suite: plain `checkout` leaves the clone where it was, `checkout -B` -moves it. - -## 1.14.304 -- a library something still links against is not an orphan - -From the machine: `pkg-config --modversion webkitgtk-6.0` said 2.50.5 while -the package said 2.52.5. Not a failed install -- the dates tell the story: - - libwebkitgtk-6.0.so.4.13.8 Sep 13 the GTK4 port (2.50.5) - libwebkit2gtk-4.1.so.0.21.9 Sep 15 the GTK3 port (2.52.5) - -The rebuild switched PORTS. The 6.0 files are old-version files this package -still owns and the new version never rewrote -- which is exactly what -`update --clean` deletes, and anything built against libwebkitgtk-6.0.so.4 -would have broken on the spot, with no warning and no way back but a rebuild. - -A package-user system records who INSTALLED every file and nothing about who -USES one. `ldd` answers it for one file; nothing answered it for a library. - - * **`packagemanager used-by <package|soname|path>`** -- every ELF that - records that soname in DT_NEEDED, grouped by the package user that owns - it. grep first (a soname is a literal string in .dynstr, so grep over - /usr/lib is seconds), then readelf only on the hits, because the same - string also appears in scripts, .la files and other libraries' rpaths. - The SONAME is READ from the library (libwebkitgtk-6.0.so.4), not guessed - from the file name (libwebkitgtk-6.0.so.4.13.8). - * **`--clean` now asks that question before it deletes.** A library still - recorded by a file another package owns is KEPT, the packages are named, - and the rebuild command is printed; the symlinks that point at it are kept - with it, or the soname it is found by disappears anyway. A library - nothing links against is swept as before. Files of the package being - cleaned do not count -- they go with it. - * `_needed_by`'s scan directories were a DEFAULT ARGUMENT, bound at import - and therefore impossible to override -- which is how the first test of it - appeared to find nothing. Resolved at call time now. - -Tested for real, not by grep: gcc builds a library with a soname and a -consumer that records it, the consumer is chowned to another account, and -`_orphan_sweep` is run twice -- with the consumer present (library KEPT, no -files removed) and with it gone (both files swept). That fixture is now a -block in the suite, and skips itself where there is no gcc. - -## 1.14.303 -- the suite could not run, so nothing it guards was guarded - -Asked "anything that needs fixing?", so the suite was run instead of guessed -at. It could not run at all, and had not been able to for some time. Five -things, in the order they were found: - - * **`lfs` would not start on Python 3.11.** One f-string reused its own - quote character inside the braces (`f"...{f(lfs, "progress", ...)}"`), - which is legal only from 3.12. `lfs` is the ONE tool that runs on the - HOST, where Python is whatever the distribution ships -- Debian 12 is - 3.11 -- so it did not misbehave there, it refused to start, with a - SyntaxError pointing at a debug print. It also took the whole suite with - it: the preamble builds a real package through a generated script, so - every block downstream failed. That one line was 12 of the failures in - the first block run. - * **The cleanup trap ate the tally.** `trap cleanup EXIT` is inherited by - every subshell, and the suite runs blocks inside `( )` and `$( )` - constantly; the first one to exit removed $T and the counter directory - out from under the run. Every later `ok()` printed a shell error instead - of counting, and the final line read `PASS: FAIL: SKIP: (final)` -- - a suite that reports nothing, which is worse than one that reports a - wrong number. `$$` is the same in a subshell, `$BASHPID` is not; the - trap now returns unless it is the shell that set it. Counters also moved - to a name nothing globs for and are recreated on write. - * **The launcher test had drifted from the launcher.** It neutralised `su` - with `sed '/^exec su - /'`, which assumed one ending; since 1.14.295 the - `exec su` is indented inside an `if` and there is a second path through - `script`, so the sed matched neither and the REAL su ran ("user u_a does - not exist"). It now stubs `su` first in PATH, which tests whatever shape - the launcher has, and asserts the two contracts that were added without - tests: a pty when there is no terminal, and a `{url}` default page. - * **Every stack file said 1.14.292 while the tools said 1.14.302.** The - suite asserts they match ("they ship as one artifact, so they carry one - number") and had been unable to say so for ten releases. Stamped -- and - `make stamp` now does it from `lfs --version`, because a rule only a human - remembers is a rule that drifts. - * **avoid.stack carried debris from the split out of sway.stack**: a second - `# stack-version:` line under a `# sway.stack -- a working Sway desktop` - header halfway down the file, and two orphaned section markers ("# 9. the - nimbin2 family", "# 10. wire the session together") with nothing after - them. All comments, all inert, all misleading to read. - -After: **751 PASS / 80 FAIL / 1 SKIP**, with a final line that prints. The 80 -are this container, not the tools -- it is not a package-user system, so the -/usr/lib/pkgusr wrappers, the book and the chroot blocks have nothing to test -against. `make check` on the machine is the number that counts. - -STILL BROKEN, not fixed here: `--only` / `--from` generate a subset script -that can cut a here-document in half ("here-document at line 667 delimited by -end-of-file") and lose helpers a previous block defined (`detail: command not -found`). The documented limitation is that a block may need what an earlier -one computed; a truncated here-doc is a bug in the extractor. Until it is -fixed, a selection can report failures a full run does not. - -## 1.14.302 -- dev.stack: gdb - -Asked for plainly: "we need gdb in a stack". None of the six fitted. Every -stack here answers one question -- services = what starts at boot, network = -how the machine gets an address, sway = the desktop, diag = is the GPU doing -what I think, media = what plays, vision = computer vision -- and a debugger -is none of them. Putting it in the nearest one is how a stack stops meaning -anything. - - * **stacks/dev.stack**: `book gdb expect="test -x /usr/bin/gdb"`, and the - question it answers is "the program crashed and I want to know why". - Nothing depends on it, so it is safe to build last and safe not to build - at all -- the other reason it is not in a stack the desktop needs. - * gdb's only "required" dependency in the book is the python module `six`, - and that is for gdb's own scripting at runtime; the rest of the list - (guile, doxygen, valgrind, rustc, SystemTap) is for gdb's test suite. - The build line comes from the book page, so nothing is repeated in the - stack. - * The file says out loud that strace, ltrace and valgrind are the obvious - neighbours and are NOT there: an entry in a stack is a package built on - every replay of it, so they wait to be asked for. - * BUILD.md runs it after services. - -Tested: the file through _parse_stack_file -- `include avoid` pulls in all 78 -exclusions and the one real entry parses as book/gdb with its probe. NOT -tested: the build itself (this box has no BLFS book). - -## 1.14.301 -- `packagemanager service`, and an exit code that was thrown away - -Asked for after `cfg bootscripts` installed sshd: a way to see what services -there are and to start one without enabling it. SysV has always kept those -two apart -- the init script starts it, the rc<N>.d links decide whether boot -does -- and nothing here made that easy to use. - - packagemanager service # all of them, at boot, running - packagemanager service sshd start # now, enabled or not - packagemanager service sshd enable|disable # boot only - packagemanager service start sshd # either order reads naturally - - * list: `at boot` (yes / other runlevel / no), `running` (asked of each - script, LSB exit code), and which runlevels. `functions`, `rc` and - `template` are not services and are not listed -- listing them invites - starting them. `--quick` skips the running column. - * start says "not enabled at boot -- starting it anyway" rather than - refusing. That was the point of the request. - * **disable records the links it removed** in /var/lib/pkgusr/services, so - enable restores exactly those. The priority number (S30, S26) comes from - the blfs-bootscripts Makefile and is the one thing that cannot be worked - out again; with no record, the number comes from a surviving K link and - the runlevels from the script's own `Default-Start:` header. disable - removes only the S links and says it has not stopped anything. - * It also lists /lib/services/* separately -- run by ifup, never on their - own, named by SERVICE= in ifconfig.<iface>. Not saying so is how "dhcpcd - is installed and nothing runs it" happens. - * **main() threw away every command's return value**, so packagemanager - exited 0 whatever happened -- including `service <name> status` on a - stopped service, which is the one command people put in an `if`. A - non-zero int now becomes the exit status; anything else is unchanged. - -Tested against a built fake /etc/rc.d (three scripts with INIT INFO headers, -one running, one not, one unlinked, plus a `functions` file): list; start on a -disabled service; enable from nothing (header runlevels, priority 50, seven -links); disable then enable (record written, K links untouched, links restored -identically); reversed word order; an unknown name; and the exit codes 0 / 3 / -1 / 0 for running, stopped, missing and list. NOT tested: a real boot. - -## 1.14.300 -- the build tree is one level deeper than the home - -`cfg bootscripts` from 1.14.299 said - - !! no unpacked blfs-bootscripts tree under /usr/src - -on a machine where it was right there. lfs-phases builds in BUILD_ROOT, -which is `$HOME/src` whenever the package user has one, so the tree is -/usr/src/p_blfs-bootscripts/**src**/blfs-bootscripts-<date>. The step looked -in the home and in the base, never a level deeper, and then told the reader to -unpack something that already existed -- the worst kind of error message, the -one that sends you to redo work that is done. - - * Four patterns now, newest wins: `<base>/*/src/`, `<base>/*/`, `<base>/` - and the sources directory. Then a `find` bounded at depth 4 for a layout - nobody thought of. - * **The tarball is enough.** A build tree can be cleaned away and these - targets only copy files, so with no tree at all the step unpacks - blfs-bootscripts-*.tar.* from $LFS_SOURCES_DIR (/sources) into a - temporary directory, installs from there, and removes it on exit (trap). - It says that those files land owned by root rather than by the package - user, because that is the one thing this path does differently. - * When it really finds nothing it now prints WHERE IT LOOKED, so the next - failure is diagnosable instead of another guess. - -Tested: the real layout (<home>/src/blfs-bootscripts-*, found and installed -from), and the tarball fallback with the tree deleted (unpacked, installed, -temp directory gone afterwards). - ## 1.14.299 -- a package and the thing that starts it come from different tarballs "dhcpcd does not start at boot; did we forget its init script?" There is no @@ -3777,462 +3121,6 @@ nested-repo-path to flat-request, absent), the ownership failure, and blocks - Not run: the regression suite (sandbox slot). `bash -n` / `py_compile` and `--version` only. -## 1.14.340 -- a man page with colons in its name - -XML::Parser: six repair rounds, all of them granting directories, and the -install still failed. The report said - - /usr/share/man/man3/XML missing; parent /usr/share/man/man3 is drwxrwxr-t - --- a file that has never existed. The real name is -`/usr/share/man/man3/XML::Parser.3`, and both path extractors cut it at the -first colon: their character class was `[A-Za-z0-9._/+-]`. So every Perl -man page has been reported under a truncated name, and the file that -actually blocked the install was never named. A trailing colon (from the -message's own "path: Permission denied") is stripped now. - -- THE STICKY BIT IS THE POINT. /usr/share/man/man3 is drwxrwxr-t, so only - a file's OWNER may replace it -- and ExtUtils::Install replaces by - unlinking first. The file belongs to p_xml-parser, an older account for - the same module (this one is p_perl-xml-parser, a different book anchor). - No directory grant can ever fix that, which is why six rounds did not. -- takeover_files_from_log now reads perl's wording -- `Cannot forceunlink - <path>`, `Can't unlink/remove/rename <path>` -- as a whole token, colons - and all. The file is handed to the package that is installing it. -- When two or more files move from an account whose name is this one's - minus a language prefix (xml-parser <- perl-xml-parser), the box says - they are one upstream module under two anchors and offers - `packagemanager remove xml-parser --purge --run` for afterwards. -- Tested: both wordings extracted with their colons intact. - -## 1.14.339 -- install_last is written BEFORE the build - -The answer to 1.14.338's open question, and the reason the stack needed a -package named by hand. - -- lfs-helper copies the script to `install_last` BEFORE running it ("RECORD - WHAT IS INSTALLED BEFORE OVERWRITING THE RECORD"). So a build that dies - at configure leaves one behind. `User.state`, with no manifest and a page - that declares no programs or libraries to check -- cairomm is exactly that - -- then falls through to `if self.meta.get("name_version"): installed`. -- cairomm-1.0 failed at configure with 0 files tracked, was recorded - installed by that rule, and every later plan skipped it as a dependency - that was already there. pangomm then failed on - Run-time dependency cairomm-1.0 found: NO - and only naming cairomm by hand got the chain moving. -- `_install_finished(u)`: a manifest with real paths, or `.pkgusr-installed` - (the marker the runner writes AFTER a successful install), is evidence. - A script copied in before the build is not. The planner skips a - dependency only with that evidence; without it the package goes into the - plan, and the plan says so: - N package(s) are recorded as installed but have no finished-install - record: cairomm-1.0 -- Tested end to end with three shapes: install_last only (planned), - .pkgusr-installed (skipped), a manifest (skipped). -- NOT changed: `state` itself, which many commands read. The rule here is - the PLANNER's, where a wrong "installed" costs a broken chain and a - needless rebuild costs minutes. - -## 1.14.338 -- "recorded as installed" is not "on disk" - -pangomm could not find cairomm-1.0, and the box said cairomm "IS installed -here ... if the .pc is absent, it installed without it". Two very -different things hide behind that: a package that installed and later lost -a file, and one that is RECORDED as installed and put nothing on disk (a -build that failed after its account and home were made). The manifest -answers it in one read, so the box now reads it: - - * manifest empty -> "nothing of it is on disk. Its last build did - not finish, and whatever marked it installed - was wrong." - * manifest, but no .pc -> "records N file(s), and NONE of them is a .pc" - * manifest with .pc -> each one printed, "(on disk)" or "MISSING from - disk", and both the rebuild and the - pkg-config-path checks - -Tested against all three shapes. - -STILL OPEN on that machine: why cairomm-1.0 was skipped as installed at -all. Its previous build failed in the BUILD phase with 0 files tracked, -which `state` should read as "not installed" (no manifest, no -install_last). Either something wrote install_last for a build that never -installed, or a DIFFERENT cairomm (the 1.16 API is its own package) is -being mistaken for it. The next run's box will say which. - -## 1.14.337 -- the build named the option; now the tool reads it - - Run-time dependency Boost found: NO (tried system) - ERROR: Building the test programs requires Boost Test ... and Fontconfig. - Install them or use -Dbuild-tests=false or -Dbuild-tests=if-dependencies-found - -- cairomm stops at configure over TEST programs, which want Boost -- and - Boost is in avoid.stack on purpose. The dependency is never coming; the - option meson prints is the whole answer, and it was on the screen while - the chain sat stopped. -- `_meson_opt_from_log` takes the FIRST `use -D<opt>=<value>` the build - suggested, adds it to the script's `meson setup` line (old script kept as - .bak-<date>) and the retry loop runs the phase again. Only an option the - build itself named, only one that turns something OFF (=false, =disabled, - =no, =if-dependencies-found), only once. The gate lets such a failure - reach the loop, and the "no repair attempted" line no longer claims the - only repairs are permissions. -- A regenerated script loses the edit, so the note prints the machine.conf - section to make it permanent -- and machine.conf now ships - build-tests = false for cairomm-1.0, pangomm-2.48 and glibmm-2.68, which - are the three in this chain with the same Boost test dependency. -- Tested: the option is added once, the second call is a no-op, and a - suggestion that turns something ON is refused. - -## 1.14.336 -- the test suite was in the BUILD block all along - -XML::Parser died in the BUILD phase: - - make: *** [Makefile:1005: test_dynamic] Error 2 - Failed 60/62 test programs. - -- Every Perl module in BLFS uses "the standard build and installation - instructions", which are ONE block: - perl Makefile.PL && - make && - make test - The test suite is not a block to skip -- it is the last LINE of the block - that builds the package, joined with &&. So the block-level test filter - could never see it, and every perl module on this machine has been - running its test suite inside the build. Class::Inspector's passed, - which is why nobody noticed; XML::Parser's fail (60 of 62 files cannot - load the module) and took the gparted chain with them. -- _strip_test_lines drops such a line, leaves the book's own text in a - comment above it, and removes the now-dangling `&&` from the line before. - A block that is ONLY tests still goes through the block-level guard with - its reason. `--test` (PKGUSR_TESTS=1) still runs the suite -- in the - TEST phase, where a failure is reported and not fatal. -- Checked against seven blocks: the perl one, a plain configure+make, a - bare `make check`, `make && LC_ALL=... make check_local`, `make install`, - LVM2's two-target install, and meson+ninja+`ninja test`. -- Script format v19, so every generated script is rebuilt from the book. - -Worth knowing: XML::Parser's tests failing to LOAD the module may mean its -Expat binding is not linking to the installed expat. It is not in the way -of the build any more; after the install, check it: - perl -MXML::Parser -e 'print "ok\n"' - -## 1.14.335 -- a PDF that was never built is not a failed install - -- parted installed 97 files -- every binary, every library -- and then - failed: - install: cannot stat 'doc/parted.pdf': No such file or directory - The book's last line is - install -v -m644 doc/{FAT,API,parted.{html,txt,pdf,ps,dvi}} \ - /usr/share/doc/parted-3.7 - and parted.pdf/ps/dvi and API exist only when texlive built them. - texlive is in avoid.stack, so they never exist here -- for parted, and - for every other page with a doc block like it. -- The install PHASE now defines an `install` function (and only that phase, - which runs in a child process of its own -- nothing outside ever sees - it). For a destination under share/doc it runs the book's command - unchanged first; only if that fails does it retry without the sources - that are not there, and say which they were. Everything else goes - straight to /usr/bin/install. A doc command that fails for any OTHER - reason is re-run and its real error reported. -- Tested: the book's block with two files missing (installs the two that - exist, rc 0, names the others), all sources missing, `-m 644` as two - words, `install -dm755` on a doc directory, and a non-doc destination. -- kernel.conf ships CONFIG_SET="LEGACY_TIOCSTI=n EXFAT_FS=m" -- the kernel - half of exfatprogs, and what this machine already has. (It was the only - difference left in kernel.conf.new.) - -## 1.14.334 -- `mount -o remount,dev /tmp` in an install phase - -LVM2 stopped the gparted chain with - - mount: /tmp: must be superuser to use mount. - !! phase 'install' FAILED (exit 32) - -and the report said "no line matched the usual error patterns". - -- WHERE IT CAME FROM. BLFS 13.0's LVM2 page has three blocks between the - build and the install: a partial install for the test suite, then - `mount -o remount,dev /tmp`, then `LC_ALL=... make check_local`. The - middle one prepares /tmp so the TESTS may create device nodes. All three - went into the install phase. -- `make check_local` was missed by the test filter for two reasons, both - now fixed: the pattern wanted a word boundary after "check" (and - "check_local" has none), and it did not allow the `LC_ALL=...` in front. -- A new rule, judged on the commands and not on the prose: a block - containing mount, umount, modprobe, insmod, mknod, swapon or swapoff is - never part of installing a package. A package user cannot run those and - no collector group can grant them; they are the system's business. The - script says so where the block would have been. -- lfs-helper recognises "must be superuser" and the like: it names the line - and says the step wanted something root-only, instead of falling through - to "no pattern matched". -- apps.stack: `book gparted ignore=lvm2`. LVM2 is only RECOMMENDED by - parted, and this laptop has no logical volumes. -- Tested: the block classifier against all seven blocks of the LVM2 page - (mount -> root-only, check_local -> test, install/configure -> kept). - -## 1.14.333 -- the wpa service reads one directory; we wrote the other - -The boot log, not dhcpcd: - - Starting dhcpcd on the wlp3s0 interface... FAIL - wpa_supplicant configuration file - /etc/sysconfig/wpa_supplicant-wlan0.conf not present FAIL - Bringing up the eth0 interface... Interface eth0 doesn't exist. FAIL - -- THE PATH. This machine's wpa service reads - /etc/sysconfig/wpa_supplicant-<iface>.conf; net-wifi wrote and looked in - /etc/sysconfig/wpa_supplicant/ -- the subdirectory other bootscript - releases use. So wpa_supplicant never started, nothing associated, and - dhcpcd had no link to ask on. The path is now READ out of $svcdir/wpa - ("the wpa service reads: ...") and used for the check and for COPY_WPA; - a config in either layout is found and copied to the one that is read. -- Per interface, the step now says whether the config the service will look - for exists, and prints the one-line copy when it does not. -- DHCP_START without -b is reported: dhcpcd in the foreground waits for a - lease on a wireless link that is not associated yet, which is the - "Starting dhcpcd ... FAIL" line. Our writer has always used -b. -- A config for an interface the kernel does not have is a FAIL line at - every boot. It is reported with the sed that turns ONBOOT off, and - DISABLE_MISSING=yes in net-wifi.conf does it for all of them (backed up; - nothing is deleted -- the card may be back tomorrow). -- Tested in a sandbox with a fake service script, a fake /sys/class/net and - a temporary /etc/sysconfig: the path is detected, the copy lands where - the service reads, and a stale eth0 file is reported. - -## 1.14.332 -- ExtUtils::Install says it in its own words - -- File::ShareDir stopped the plan with - ERROR: Can't create '/usr/lib/perl5/5.42/site_perl/File' - Do not have write permissions on '/usr/lib/perl5/5.42/site_perl/File' - and the box said "no automatic repair attempted: the failure does not - look like a permission or missing-directory problem". It is exactly a - permission problem; ExtUtils::Install simply never writes the words - "Permission denied" for it. Two greps were blind to it: - * the _perm_ish gate (so zero rounds ran), and - * unwritable_dirs_from_log's tail filter (so even a round would have - had nothing to grant). - Both now know `Can't create '<path>'`, `Do not have write permissions` - and `Cannot mkdir`. Checked against the real log text: the gate fires - and the extractor yields /usr/lib/perl5/5.42/site_perl, the directory - the grant needs. -- Round two of the same lesson as 1.14.331: a Perl module's install writes - into three trees owned by three different packages, and every one of them - reports the refusal differently. - -## 1.14.331 -- perllocal.pod is the info/dir of the perl tree - -- `perl-class-inspector` installed both its modules and both its man pages - and was then reported FAILED, for a line make itself had marked - "(ignored)": - /usr/lib/perl5/5.42/core_perl/perllocal.pod: Permission denied - Every Perl module APPENDS to that file in its doc_install step, so it - belongs to whichever module was installed first (p_xml-parser, 664, group - p_xml-parser) and every module after that cannot write it. Granting the - DIRECTORY to the perl collector group -- which the auto-repair did -- - cannot help: the file's own group is one package's. -- It is in shared_files_list now, with the info/dir family: written by many - packages, installed by none. share_shared_files makes it root:install - 664, which every package user can append to. The version is globbed - (/usr/lib/perl5/*/*/perllocal.pod). -- shared_file_dirs SKIPS the perl directories: the file is shared, the - directory is not -- /usr/lib/perl5/<ver>/core_perl is p_perl's install - directory and is in its manifest. Handing it to root:install would make - `verify` right to complain. -- The "exited 0 but the log says otherwise" gate no longer fails a phase - for a shared registry file it could not update. It says so, runs - share_shared_files, and lets the package's own files stand. A real - `install: cannot create ...: Permission denied` still fails, tested - against both logs. - -## 1.14.330 -- an installed dependency is not a thing to upgrade - -- `install gparted --run --recursive` planned FIFTY-THREE packages: gparted, - its missing dependencies -- and updates of bubblewrap, alsa-lib, gtk3, - gtk4, librsvg, glycin, GnuTLS and thirty more that were installed and - working, purely because the book had moved on. Asking for one program is - not asking to rebuild the desktop, and each rebuild is a chance to break - a machine that was fine. A DEPENDENCY that is installed is now left - alone whatever its version; a package NAMED on the command line is - unchanged; `--update-deps` restores the old behaviour. The plan says how - many it left and how to move them on purpose. - (Unit-tested: gparted with gtk3 installed-but-behind plans 2 packages, - and 3 with --update-deps.) -- exfatprogs is a `tar` entry now, not `book`: the BLFS index carries no - exFAT package at all (checked). 1.4.2 from the project's release - tarball, ./configure with an autogen.sh fallback. The relan/exfat the - book mentions is the FUSE implementation -- a different thing, and only - for a kernel without EXFAT_FS. - -## 1.14.329 -- gparted - -- apps.stack: `book gparted`, last in the file (it is the heaviest entry -- - the planner brings gtkmm3 and parted). Confirmed a BLFS page against the - stable index: Gparted-1.7.0. -- It has no graphical privilege step here: the book's page uses polkit and - polkit is avoided on this machine (it arrives only through colord and - systemd). So: run `gparted` from a ROOT terminal inside the session. - The comment in the stack says so. -- The filesystem tools it drives are their own packages; exfatprogs and - btrfs-progs (services.stack) are what stop entries being greyed out. - -## 1.14.328 -- exfatprogs - -- services.stack: `book exfatprogs` (mkfs.exfat, fsck.exfat, exfatlabel), - beside btrfs-progs. No `when=`: the card that needs it is not in the - machine when the stack runs. The kernel side stays separate -- - `lfs-kernel enable exfat` sets EXFAT_FS=m. -- Not tested against the book here: if the page's anchor is not - `exfatprogs`, the stack says NOT IN BOOK and `blfs search exfat` gives - the real one. - -## 1.14.327 -- net-wifi writes what holds no secret, and prefers an antenna - -- `cfg net-wifi` reported and wrote nothing, so a deleted - /etc/sysconfig/ifconfig.<iface> meant pasting a file out of its output. - It now reads stacks/net-wifi.conf and, with WRITE=yes, writes that file - for every wireless interface that has none -- marked as ours, never over - one of yours. The wpa config is still never written: it holds the key. -- PREFER (patterns, best first) decides the dhcpcd route metric: the first - interface PRESENT gets METRIC_PREFERRED, the rest METRIC_OTHER, so a USB - antenna takes over while it is plugged in and the built-in card carries - on when it is gone. A file the step wrote is re-metered when PREFER - changes (backed up); one of yours is reported, not edited. -- USB_RENAME writes a udev rule naming an adapter by its USB ids - (0cf3:9271 -> wl-ext), because its bus name changes with the socket and - ifconfig.<name> cannot follow. It applies when the device next appears. -- COPY_WPA copies an existing wpa_supplicant-<iface>.conf to an interface - that has none (chmod 600), and says so every time -- a key is duplicated. -- A config naming an interface the kernel does not have is now reported. -- diag.stack: usbutils and pciutils (lsusb, lspci) -- the first commands - for a device that did not appear. -- Tested with a fake /sys/class/net (NET_WIFI_SYSFS) and a temporary - /etc/sysconfig: first write, the metric flip when the preferred interface - appears, the wpa copy, the orphan-config note, the udev rule text. - -## 1.14.326 -- a build needs --run, a module can be loaded, a scan is pruned - -- `lfs-kernel` alone no longer builds. It prints what a build WOULD do -- - version, tree state (fetch / continue / already built), config source and - CONFIG_SET, where the image and modules go, firmware, headers, - v4l2loopback -- and says `--run`. `--run` is taken from anywhere on the - line. An hour of compiling that replaces the booting kernel should not - start from a bare word; every other tool here already waited. -- `lfs-kernel load <module> [--boot]` / `unload`: modprobe, confirm it in - /proc/modules (and say when a symbol is built IN rather than missing), - show the dmesg lines that mention it or firmware, and with --boot add it - to /etc/sysconfig/modules (backed up, no duplicates). `modules` points at - it instead of telling people to type modprobe themselves. -- `reload-pkg-list` scans with its own pruned find instead of the hint's - `list_package`: -xdev, and never /proc, /sys, /dev, /run, /tmp, /var/tmp, - /var/cache, /mnt, /media, the snapshots, /sources, /build or the PACKAGE - SOURCE ROOTS. That last one is where the time went -- ninety unpacked - source trees, the kernel's own among them. Still run as the package user - (so unreadable directories are skipped), with a line saying so and - `--root` for the thorough scan. The seconds are now printed. -- Tested: the plan for an already-built tree, load/unload with a stub - modprobe and the boot file, the pruned find, py_compile / bash -n. - -## 1.14.325 -- where are we, and a firmware report that needs no network - -- `lfs-kernel status`: what is running and what is known to boot, whether - the tree's .config carries every CONFIG_SET entry, whether the build is - FINISHED / UNFINISHED (object files, no current image) / not started, how - many modules are in the tree and in /lib/modules, what the ESP holds, and - whether the installed config differs from the tree's. Reads only; no - network; works with the ESP unmounted. Written because an interrupted - build leaves nothing on the machine that says how far it got. -- `firmware --missing` asked kernel.org which kernel is newest, for a - question /lib/modules and the boot log answer -- and EXITED on a machine - with no network. It uses the running kernel's version now. -- Tested: status against a tree with objects only and with a bzImage, - CONFIG_SET read-back, the unmounted-ESP path. - -## 1.14.324 -- a typo built a kernel, and the command line lost to the file - -Two bugs in lfs-kernel's argument handling, both old, both found by one -paste: `lfs-kernel search AR9271` rebuilt and reinstalled 7.2.6. - -- ANY unrecognised word became VERSION. It is now refused, with the - command list and a did-you-mean; only `latest` or something shaped like - 7.2.6 is a version. `search` and `set` became real top-level commands - (they are what a person types) and go to `config search` / `config set`. -- WORSE: `VERSION="$1"` was set BEFORE `. "$CONF"`, and kernel.conf's own - VERSION= then overwrote it. So `lfs-kernel 6.12.40` built whatever the - file said (latest, here) and `lfs-kernel verify 7.2.4` reported on another - kernel -- both documented, neither working, since the config file grew a - VERSION line. The command line is re-applied after the source now. -- `lfs-kernel prune` is named in two comments and has never existed; the - comments are corrected rather than the command invented. -- Tested: unknown word refused, search alias, an explicit version surviving - the config file. - -## 1.14.323 -- hardware by name, and the modules a kernel really has - -- `lfs-kernel enable <name>`: a table of hardware (ath9k-htc, ath9k, ath10k, - ath11k, alx, atl1c, iwlwifi, rtl8xxxu, bluetooth-usb, btrfs, exfat, ntfs3, - nvme, usb-serial, v4l2-usb) -> the symbols AND the firmware it needs, - written into CONFIG_SET and FIRMWARE_ONLY. A bare symbol works too: the - tree says whether it is tristate (=m) or bool (=y). `--list` shows them. -- `lfs-kernel modules [text]` -- what /lib/modules holds for this kernel, - what is built in, what is loaded, with each module's description. -- `config search` now matches a symbol's NAME or its PROMPT and prints the - prompt. It used to list every symbol of every Kconfig file that merely - mentioned the word: 40 lines of MIPS console options for "atheros". -- `config save [--good]` seeds the archive from the running kernel without a - build -- what CONFIG_FROM=last-good needs before the first build ever runs. - `config from <source>` sets CONFIG_FROM and refuses last-good when there - is none. One writer (`_conf_set_var`) for every kernel.conf edit. -- Tested: enable (name, bare symbol, unknown), search prompts, save/from, - modules against a fake /lib/modules. No kernel built here. - -## 1.14.322 -- btrfs-progs when it applies, and the kernel configs are kept - -- Stack entries take `when="<shell>"`: judged and run only where that - command succeeds. `book btrfs-progs when=...` is in services.stack -- - mounted btrfs, or a btrfs partition on any disk (so it is right in a - chroot too). -- lfs-kernel keeps every config it builds with in /etc/pkgusr/kernel-configs - (root's, outside the tree, so `prune` and `packagemanager remove linux` - cannot take it): `config-<ver>-<date>` (newest CONFIG_KEEP=5), - `last-good.config` and `prev-good.config` -- the last two that BOOTED, - rotated where last-good is written, i.e. only from a kernel that ran. - The archive is written BEFORE the compile, so a build that dies keeps its - answers. -- CONFIG_FROM takes last-good | prev-good | saved, and `running` falls back - to last-good instead of exiting when /proc/config.gz is unreadable. -- `lfs-kernel config` -- show | list | diff | search <text> | set SYM=VAL | - unset SYM. set/unset edit CONFIG_SET in kernel.conf with a backup; - search reads the current config and the tree's Kconfig files and says - that a NET_VENDOR_*/WLAN_VENDOR_* symbol alone builds no driver. -- Tested: the config subcommands against a fake tree and config directory - (show/list/diff/search/set/unset), stack parse of the when= entry, - py_compile / bash -n. No kernel was built here. - -## 1.14.321 -- git reinstall, a stack on the svn book, @setup - -- `install <gitpkg> --reinstall [--clean]` builds from the package's own - script and never asks the book. It used to ask three times (map, - `blfs debug`, `blfs search`), take a minute and say "not in the BLFS - book" for imv. A stack git/tar entry that was never built is found too. - `book_version` remembers a miss, so no name is looked up twice. -- `from-book <selector>` in a stack file: its book entries read that book - (versions, deps, scripts) -- permanently, so `update` and `info` agree. - Deps outside the stack keep their own book; a page only the stack book - has is read from it. `blfs path` resolves a selector to the cached file - once per run; `stack`/`update --stack` refresh it (`blfs fetch`) once. - One version map per book (`book-versions-<md5>.json`). The default map - is the default book, no longer "newest file on disk" -- that would have - become svn the moment it was fetched. media.stack now says `from-book - svn`. NOT tested against a real svn book: check the selector exists - (`blfs books --discover`). -- `stack` prints the avoid COUNT; `stack <name> --avoided` lists them. -- app-users.conf `@setup <account|*> <cmd|URL|script>`: runs as the account, - once per line text (`/var/lib/pkgusr/app-setup/<account>`), - `APP_USERS_RERUN=1` repeats. -- Tested: py_compile, bash -n, stack parse (media entries only get svn - inside sway), `--avoided`, the git path with stubs. Suite not run. - ## Deferred tool work None of this blocks the machine. diff --git a/Makefile b/Makefile index 81ac384..df793ac 100644 --- a/Makefile +++ b/Makefile @@ -39,7 +39,7 @@ INSTALL ?= install # mkdir -p leaves an existing directory exactly as it is. MKDIR ?= mkdir -p -.PHONY: all install uninstall check stamp clean help +.PHONY: all install uninstall check clean help all: help @@ -47,7 +47,6 @@ help: @echo "make install [PREFIX=...] [DESTDIR=...] install the tools" @echo "make uninstall [PREFIX=...] remove them" @echo "make check run the tests" - @echo "make stamp stamp the stack files with the tools' version" @echo "make clean remove scratch files" @echo "" @echo "installs into: $(PREFIX)/bin" @@ -215,21 +214,6 @@ uninstall: check: @bash $(TESTS) ./lfs -# ONE ARTIFACT, ONE NUMBER -- and a rule a human has to remember is a rule -# that drifts. The stack files each carry `# stack-version:`, the suite -# asserts it equals the tools' version, and it was ten releases stale before -# anyone ran the suite. This stamps them from `lfs --version`, so cutting a -# release is `make stamp` rather than seven sed commands. -stamp: - @v=$$(./lfs --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1); \ - [ -n "$$v" ] || { echo "cannot read a version from ./lfs"; exit 1; }; \ - for f in stacks/*.stack; do \ - grep -q '^# stack-version:' "$$f" \ - || { echo "$$f has no stack-version line -- add one"; exit 1; }; \ - sed -i "s/^# stack-version: .*/# stack-version: $$v/" "$$f"; \ - done; \ - echo "stamped $$(ls stacks/*.stack | wc -l) stack file(s) with $$v" - # Nothing here is compiled, so clean removes only what running the tools here # leaves behind: Python bytecode and the test suite's scratch directories. # It never touches an installed tree -- that is what uninstall is for. diff --git a/NEXT-TASK.md b/NEXT-TASK.md index 205bfa3..be0af11 100644 --- a/NEXT-TASK.md +++ b/NEXT-TASK.md @@ -1,142 +1,46 @@ # First task for the next session -The machine boots, `su` works, the desktop runs and the application accounts -are in daily use. Everything below is configuration on the machine, not a -build. The tools are at 1.14.320. +## 1. Get root back, then let the stack repair it (1.14.267) -## 0. Repair what the lease took (do this first) +su is not setuid (the earlier-stages claim stripped it; fixed in 1.14.265). +You need root without su: -`/usr/bin/su` and six `/usr/sbin/iptables*` links are owned by `p_webkitgtk`, -and the links are recorded in its `pkg.lst` — so `update webkitgtk --clean` -would delete them. 1.14.310 shuts the doors; the machine still has to be put -back: + * tty login: Ctrl+Alt+F2, log in as root (login runs as root, no setuid + needed -- works if /etc/pam.d/login exists), or + * from the host: enter the chroot as before. - packagemanager disown --all # what it would hand back - packagemanager disown --all --run # hand it back, and clean the records - lfs-helper setid --run # put the setuid bits back - su - # check +Then, from `/` (not ~/lfs, which holds old stack copies): -Then the same rebuild that broke it is safe to repeat. + make install ; lfs run # chroot only; on the booted + # system: make install + cd / + packagemanager stack services --only shadow-pam --run # cached otherwise + packagemanager stack services --run # the rest, incl. power + ls -l /usr/bin/su /usr/sbin/unix_chkpwd # 4711 root, 4755 root + ls /etc/pam.d/ | wc -l # 22 + grep -n pam_exec /etc/pam.d/su # line 1 -## 0a. The stacks are reorganised (1.14.320) +As your normal user, in another terminal: - make install - ls /etc/pkgusr/stacks/*.stack.new # should print nothing + su - # root's password + su - u_browser-mini -c id # no password -wayland.stack is new (the base sway.stack used to carry), sway.stack -includes it, and zathura/imv moved to apps.stack, jq to tools.stack. -Nothing needs rebuilding; `packagemanager stack sway` still means the whole -desktop. See HANDOFF 1.14.320. +If `ls /etc/pam.d/` is short, shadow's configure has not run under the +lease yet: packagemanager script configure shadow -- then the stack again. -## 0b. The shared hub and yt-dlp +## 2. Renumber the machine's ids -Shared data moved to /home/Shared by hand (commands from the 1.14.315 -session), then: +The one-off fix script from chat (1.14.267): collector accounts removed, +collector groups to 90000+, u_* to 30000+. Dry run first; u_* logged out. +Then `stack services --only power --run`. - packagemanager stack services --only app-users --run # links + 0700 homes - packagemanager stack media --only yt-dlp --run +## 3. Still open -Log in again for the SHARED_* variables to name /home/Shared. + * cfg entries are stack-cached, so a repair script is skipped once it + has run. Decide whether cfg entries should always run. -## 1. The locale (one command, then log in again) - -Nothing sets `$LANG`. lfs-helper wrote `/etc/profile.d/i18n.sh` with the -export commented out, and there is no `/etc/locale.conf` — so every program -runs in the C locale and foot says so at every start. Both are fixed in the -tools (1.14.296–1.14.298); the machine still has to run the step: - - lfs config locale de_DE.UTF-8 # if it is not that already - packagemanager stack services --only shell-env --run - # log in again, or right now: . /etc/profile.d/i18n.sh - -It writes `/etc/locale.conf`, replaces that i18n.sh (backed up), and adds one -line to `/etc/bashrc` so a new terminal picks it up without a login. - -## 2. The launchers written before 1.14.295 - -A `.desktop` entry does nothing if its launcher does not give `su` a pty, and -a launcher these tools did not write is left alone — so the old ones survive. -`~/bin/claude` is the last hand-made one: - - rm ~/bin/claude - packagemanager stack services --only app-users --run - -Then click a menu entry. If one still does nothing, run the same name in a -terminal first: that path prints what `su` says. - -## 3. Wireless — the only piece of networking still missing - -Wired works and starts itself (`enp2s0f0`, `enp5s0`). The wireless interface -is **`wlan0`** on this kernel. - - packagemanager stack network --only net-wifi --run - -It prints both files to write: `/etc/sysconfig/ifconfig.wlan0` and a per-SSID -`/etc/sysconfig/wpa_supplicant/wpa_supplicant-wlan0.conf` (chmod 600). Nothing -writes the second one for you — it holds the key. - -## 4. make-ca's weekly job - -The systemd book enables a timer, which the shim skips. fcron is installed: - - cat > /etc/cron.weekly/update-pki.sh << "EOF" - #!/bin/bash - /usr/sbin/make-ca -g - EOF - chmod 754 /etc/cron.weekly/update-pki.sh - -## 5. Try the new commands once - -So their first use is not in an emergency: - - packagemanager service # list; then: service sshd start - packagemanager stack dev --run # gdb - -## 6. Run the test suite once, on the machine - - make check - -It has not been able to report a verdict for some time (1.14.303): `lfs` would -not load on Python < 3.12, and the cleanup trap fired in every subshell and -removed the counters, so the final line printed blank. Both are fixed. Here it -reads 761 PASS / 84 FAIL / 1 SKIP (1.14.320), and those 84 are all "this is not a -package-user system" -- the wrappers, the book and the chroot blocks have -nothing to test against. On the machine they should mostly turn into passes; -whatever is left is a real list worth reading. - -Known and NOT fixed: `--only N` / `--from N` can cut a here-document in half -and lose helpers an earlier block defined, so a selection can report failures -a full run does not. Trust the full run. - -## Still open, none blocking - - * services.stack is misnamed: it holds bash-completion, shell-env, - linux-pam, shadow-pam, app-users, power and mytools as well as the boot - services. `system.stack` would say what it is. Not renamed yet: the - machine's progress file (services.done), NEXT-TASK, BUILD.md and ~20 - test references use the name. Decide, then do it in one release. - * the scdoc, grim, slurp and swayidle entries have not been built yet -- - first real run is on the machine (sr.ht / freedesktop are unreachable - from the sandbox). - * a plan does not put an update of an installed-but-too-old dependency in - front of the package that needs it (vcs_versioning wanted packaging >= - 26.2; HANDOFF 1.14.314). - - * poppler: re-apply the reverted `cd` rule (the patches only "did not fire" - because of the stale index cache, fixed in 1.14.262); diff every generated - script before and after. Do NOT `--regenerate` poppler first. + * poppler: re-apply the reverted `cd` rule (the patches only "did not + fire" because of the stale index cache, fixed in 1.14.262); diff every + generated script before and after. Do NOT `--regenerate` poppler first. * shadow's login.defs block overwrites `.orig` on a second run. * MIT-Kerberos configure transcript blocks (not in any stack). - * `dbus` was built before `[dbus] systemd = disabled` went into - machine.conf — rebuild or accept. - * A stray `.service` from at-spi2-core in `/usr/lib/systemd/user`. - * `/etc/bashrc` has a mangled line (`# End /etc/bashrcCK=$(tput setaf 0)`) - from an append without a leading newline. Harmless: `BLACK` is unset and - nothing uses it. - -## Settled since this list was last written - - * "cfg entries are stack-cached, so a repair script is skipped once it has - run." Answered by convention rather than by changing the cache: every cfg - entry that must stay true now carries `expect=`, which is re-checked on - every run — shadow-pam, app-users, shell-env, bootscripts. diff --git a/README.md b/README.md index 0cf7cef..b23a159 100644 --- a/README.md +++ b/README.md @@ -71,21 +71,13 @@ lfs run # everything up to the chroot, then enter it lfs-helper build-all # inside: chapters 7-9 packagemanager bootstrap --run # wget, Python modules, BLFS book, certificates packagemanager stack # list the stacks -packagemanager stack sway --run # then network, services; apps and tools when wanted -lfs-kernel --run # the kernel, from /etc/pkgusr/kernel.conf +packagemanager stack sway --run # then network, services +lfs-kernel # the kernel, from /etc/pkgusr/kernel.conf ``` `lfs run` and `build-all` continue from wherever they stopped. Run them again after a reboot, a failure, or a cancel. -An entry can carry `when="<shell>"`: it is built only where that command -succeeds. `btrfs-progs` in services.stack is built on btrfs machines only. - -A stack can read its packages from another book. `from-book svn` in a stack -file puts every book entry below it on the development book. Dependencies -outside the stack stay on the default book. `packagemanager stack <name> ---avoided` lists what the stack never builds; the plan shows only the count. - | Command | Does | |---|---| | `lfs build-system next` | What is done, what comes next | @@ -178,9 +170,8 @@ dependency failed is skipped, not attempted. Run it again after fixing anything; it does only what is left. ```sh -packagemanager install <pkg> --recursive --run # missing deps only; installed ones are left alone +packagemanager install <pkg> --recursive --run packagemanager update <pkg> --run -packagemanager install <gitpkg> --reinstall --clean --run # git packages: no book lookup packagemanager which-package /usr/bin/foo blfs order <pkg> # build order, dependencies first blfs sources <pkg> # download URLs @@ -206,8 +197,7 @@ once for your own account; every shared user then joins whatever is there. Many accounts at once come from `/etc/pkgusr/stacks/app-users.conf`, one line per launcher, applied by `packagemanager stack services --only app-users ---run`. A line `@setup <account|*> <command | URL | script>` runs extra -setup as that account, once; a changed line runs again. Undoing them: +--run`. Undoing them: | Command | Removes | |---|---| @@ -227,44 +217,6 @@ login. The step also sources the fragment from `/etc/bashrc`; any shell can refresh itself with `. /etc/profile.d/shared-dirs.sh`, which also drops the variables whose directory is gone. -### Wireless - -`packagemanager stack network --only net-wifi --run` reports what a wireless -link still needs and, with `WRITE=yes` in -`/etc/pkgusr/stacks/net-wifi.conf`, writes the parts that hold no secret: an -`/etc/sysconfig/ifconfig.<iface>` per wireless interface, and a udev rule -that gives a USB adapter the same name in every port. `PREFER` names the -interfaces best first — the first one present gets the low route metric, so -an external antenna is used while it is plugged in. The wpa_supplicant -config is never written (it holds your passphrase), but an existing one is -copied to a new interface when `COPY_WPA=yes`. - ---- - -## Services - -SysV, so a service is an init script in `/etc/rc.d/init.d` plus S/K symlinks -in `/etc/rc.d/rc<N>.d`. **Starting is not enabling** — `start` runs the script -now, whatever the links say. - -```sh -packagemanager service # what exists, at boot, running -packagemanager service sshd start # now, enabled or not -packagemanager service sshd enable # at boot, without starting it -packagemanager service sshd disable # boot only; it keeps running -``` - -`disable` records the links it removed, so `enable` puts back exactly what was -there — the priority number lives in the blfs-bootscripts Makefile and is the -one thing that cannot be worked out again. With no record, the runlevels come -from the script's own `Default-Start`. `status` exits 0 when running, 3 when -not, so it works in an `if`. - -The scripts themselves come from blfs-bootscripts, one `make install-<target>` -per service. `packagemanager stack services --only bootscripts --run` installs -the ones whose package is here and whose script is not, from the mapping in -`/etc/pkgusr/stacks/bootscripts.conf`. - --- ## When ownership starts @@ -372,30 +324,12 @@ own package (`p_linux-firmware`) and the headers update the package that owns ```sh lfs-kernel list # what kernel.org offers -lfs-kernel # what a build would do (nothing is built) -lfs-kernel --run # build the version in /etc/pkgusr/kernel.conf -lfs-kernel 6.12.40 --run # that version, once -lfs-kernel load ath9k_htc --boot # load a module now and at every boot +lfs-kernel # build the version in /etc/pkgusr/kernel.conf +lfs-kernel 6.12.40 # that version, once lfs-kernel verify # is everything installed? lfs-kernel firmware --missing [--fix] # firmware this machine wants and lacks -lfs-kernel status # how far the last build got, what is installed -lfs-kernel enable --list # hardware this tool knows by name -lfs-kernel enable ath9k-htc # its symbols into CONFIG_SET, its firmware - # into FIRMWARE_ONLY -- kernel.conf, backed up -lfs-kernel modules [text] # what this kernel has, and what is loaded -lfs-kernel config # where the next .config comes from -lfs-kernel config search atheros # symbols by name or prompt -lfs-kernel config save --good # keep the running config as a known-good one -lfs-kernel config from last-good # build from it, not from /proc -lfs-kernel config diff # prev-good -> last-good ``` -Every config the tool builds with is copied to `/etc/pkgusr/kernel-configs`, -outside the source tree, so removing the package cannot take it: the last -`CONFIG_KEEP` builds, plus -`last-good.config` and `prev-good.config` — the two that booted. -`CONFIG_FROM=last-good` starts from one of them. - Everything it does is configured in `/etc/pkgusr/kernel.conf`, which is installed rather than generated so it can be edited first. `ESP` and `KERNEL_NAME` together decide whether a build replaces the working kernel or diff --git a/blfs b/blfs index 592baac..0646127 100755 --- a/blfs +++ b/blfs @@ -17,7 +17,6 @@ Subcommands blfs books list known/cached books and the default blfs books --discover probe the live site for available versions blfs set-default <selector|url> choose the default book - blfs path [selector] the local file a book reads (for scripts) blfs search <name> find packages, including python modules blfs script <name|url> [-o dir] write install_<name-version> (install/update) blfs deps <name> [--tree] [--no-recommended] [--optional] @@ -129,7 +128,7 @@ CACHE_VERSION = 56 # v56: a skipped build variant does not keep its install st # tool + generated-script format versions. Bump SCRIPT_VERSION when the shape # of generated install scripts changes (phases, dispatcher, ...), and # CACHE_VERSION when the parsed/cached data structure changes. -BLFS_VERSION = "1.14.340" +BLFS_VERSION = "1.14.299" def _build_id(): @@ -147,7 +146,7 @@ def _build_id(): except OSError: return "unknown" -SCRIPT_VERSION = 19 # v19: a test line chained into the build block is dropped +SCRIPT_VERSION = 17 # v17: the runner moved out into lfs-phases def _tty_out(): @@ -1490,13 +1489,7 @@ def _emit_command_elements(elements, out, nested): out.append(SKIP_MARK + _g) out.append(_commented(text)) else: - text, _dropped = _strip_test_lines(text) - if _dropped: - out.append(SKIP_MARK + "the test suite, chained into " - "this block -- `--test` runs it") - out.append(_commented("\n".join(_dropped))) - if text.strip(): - out.append(text) + out.append(text) elif name == "div": classes = list(el.get("class", [])) if "admon" in classes: @@ -1562,13 +1555,7 @@ _COND_GUARD_RE = re.compile(r"^(if\b|optionally\b|should you\b)", re.I) _TEST_BLOCK_RE = re.compile( r"^(to test\b|to check\b|the test suite\b|tests? (can|may) be run\b)" r"|\bcan be tested\b|\bto test the (results|installation|build)\b" - r"|\bto run the test" - # LVM2 13.0: "If you wish to run the tests, install a minimal set of - # ... first" -- an INSTALL command that is part of the test suite, and - # the only thing marking it as such is the sentence above it. - r"|\bif you (wish|want) to run the test" - r"|\bbefore running the test" - r"|\bfor the test suite\b", re.I) + r"|\bto run the test", re.I) # THE COMMANDS SAY WHAT THEY ARE, AND PROSE IS A SECOND-HAND ACCOUNT. # @@ -1583,17 +1570,7 @@ _TEST_BLOCK_RE = re.compile( # commands, so a rewording upstream cannot bring the bug back. _TEST_CONTENT_RE = re.compile( r"""(?mx) - ^\s*(\w+=\S*\s+)*(make|ninja)\s+(-\w+\s+)*(check|test)\b - # make check / ninja test - | ^\s*(\w+=\S*\s+)*(make|ninja)\s+(-\w+\s+)*(check|test)[_-]\w+ - # LC_ALL=... make check_local - # (the old pattern needed a - # word boundary after - # "check", which - # "check_local" has not -- - # and no env assignment in - # front, which LVM2's page - # has) + ^\s*(make|ninja)\s+(-\w+\s+)*(check|test)\b # make check / ninja test | ^\s*meson\s+test\b | ^\s*ctest\b | ^\s*py\.test\b | ^\s*pytest\b @@ -1604,84 +1581,6 @@ _TEST_CONTENT_RE = re.compile( """) -# A COMMAND THAT ONLY ROOT CAN RUN IS NOT AN INSTALL STEP. -# -# LVM2's page (13.0) puts three blocks between the build and the install: -# a partial install for the test suite, then -# mount -o remount,dev /tmp -# and then `make check_local`. The middle one is not about installing -# anything -- it prepares /tmp so the tests may create device nodes -- and -# it ran in the install phase, as the package user: -# mount: /tmp: must be superuser to use mount. -# !! phase 'install' FAILED (exit 32) -# A package user cannot mount, load a module or make a device node, and no -# BLFS page needs it to: those commands belong to the system, not to the -# package. (`make check_local` was missed too -- the test pattern required -# a word boundary after "check", and "check_local" has none.) -_ROOT_ONLY_RE = re.compile( - r"""(?mx) - ^\s*mount\b | ^\s*umount\b - | ^\s*modprobe\b | ^\s*insmod\b - | ^\s*mknod\b - | ^\s*swapon\b | ^\s*swapoff\b - """) - - -def _root_only_reason(pre): - """Why this block cannot run as the package user, or None.""" - m = _ROOT_ONLY_RE.search(pre.get_text()) - if not m: - return None - hit = " ".join(m.group(0).split()) - return ("it needs root and the system, not this package (`%s`) -- run it " - "yourself if you want it" % hit) - - -# A TEST COMMAND CHAINED INTO THE BUILD BLOCK. -# -# Every Perl module in BLFS uses "the standard build and installation -# instructions", which are ONE block: -# perl Makefile.PL && -# make && -# make test -# So the test suite is not a block to skip -- it is the last line of the -# block that builds the package, joined with &&. XML::Parser's tests fail -# on this machine (62 files, 60 of them cannot even load the module) and -# took the whole gparted chain down in the BUILD phase; Class::Inspector's -# passed, which is why nobody noticed the tests were running at all. -# -# Tests run here only when they are asked for (PKGUSR_TESTS=1, which is -# `--test`), and then the runner runs them ITSELF, in the test phase. So -# the line comes out of the build, with the book's own text left in a -# comment, and the trailing && of the line before it goes too. -_TEST_LINE_RE = re.compile( - r"""^\s*(\w+=\S*\s+)*(make|ninja|meson|ctest|py\.test|pytest)\b[^&|;]*? - \b(check|test)([_-]\w+)?\b[^&|;]*$""", re.X) - - -def _strip_test_lines(text): - """Drop `make test` / `make check` lines from a block that also builds. - - Returns (text, dropped-lines). A block that is ONLY tests comes back - empty and its caller keeps treating it as a test block.""" - lines = text.splitlines() - keep, dropped = [], [] - for ln in lines: - if _TEST_LINE_RE.match(ln.strip()) and not re.search(r"\binstall\b", ln): - dropped.append(ln.strip()) - continue - keep.append(ln) - if not dropped: - return text, [] - # the line before a dropped last line ends in `&&` (or `;`) that now - # chains to nothing -- and `make &&` on its own is a syntax error - while keep and not keep[-1].strip(): - keep.pop() - if keep: - keep[-1] = re.sub(r"\s*(&&|\|\||;)\s*$", "", keep[-1]) - return "\n".join(keep), dropped - - def _test_content_reason(pre): """Why this block is a test suite, judged by its commands, or None.""" txt = pre.get_text() @@ -1709,7 +1608,7 @@ def _conditional_guard(pre): "optional" wins over both: an optional tool is exactly what we do not install.""" # content first: it does not depend on the book's wording or nesting - _by_content = _root_only_reason(pre) or _test_content_reason(pre) + _by_content = _test_content_reason(pre) if _by_content: return _by_content prev = pre.find_previous(["p", "pre", "h3", "h4"]) @@ -1811,14 +1710,6 @@ def extract_commands_phased(sec): stream.append(SKIP_MARK + guard) stream.append(_commented(text)) continue - text, _dropped = _strip_test_lines(text) - if _dropped: - stream.append(SKIP_MARK + "the test suite, chained into this " - "block with && -- run it with `--test`, which " - "runs it in the test phase") - stream.append(_commented("\n".join(_dropped))) - if not text.strip(): - continue stream.append(text) else: # h3 / h4 title = " ".join(el.get_text().split()) @@ -3659,29 +3550,6 @@ def cmd_fetch(args): print("cached: %s" % path) -def cmd_path(args): - """Print the local book file a selector reads -- for other tools. - - packagemanager asks this once per run for every book it needs (the - default, a stack's `from-book svn`). Resolving `svn` through - resolve_book asks the website every time, and a planner that passes - `--book svn` to each of its hundred blfs calls would ask a hundred - times. A cached book is answered from disk; only a book that was never - fetched goes to the network (or fails, with --offline).""" - sel = args.selector or getattr(args, "book", None) or default_selector() - if os.path.isfile(sel): - print(os.path.abspath(sel)) - return - p = _cached_for_selector(sel) - if not (p and os.path.isfile(p)): - if getattr(args, "offline", False): - sys.stderr.write("no cached book for '%s' (blfs fetch %s)\n" - % (sel, sel)) - sys.exit(1) - p = ensure_book(sel, quiet=True) - print(p) - - def cmd_import(args): """Add a local book file to the cache -- no network involved.""" src = os.path.expanduser(args.file) @@ -4302,13 +4170,6 @@ def main(): p.add_argument("selector", nargs="?", help="e.g. stable-systemd, 12.4, svn") p.set_defaults(func=cmd_fetch) - p = sub.add_parser("path", help="print the local file a book selector " - "reads (cached: no network)") - p.add_argument("selector", nargs="?", help="e.g. svn; default: the default") - p.add_argument("--offline", action="store_true", - help="fail instead of fetching a book that is not cached") - p.set_defaults(func=cmd_path) - p = sub.add_parser("import", help="add a local book file (no network)") p.add_argument("file") p.add_argument("--selector", help="name to file it under") diff --git a/kernel.conf b/kernel.conf index 053db7a..e779b54 100644 --- a/kernel.conf +++ b/kernel.conf @@ -39,25 +39,9 @@ PKG=linux # kernel link is memory-hungry. JOBS=-j14 -# CONFIG_FROM: where the .config starts from. -# running /proc/config.gz -- the kernel carrying the machine now. -# If it cannot be read (no CONFIG_IKCONFIG_PROC), the saved -# last-good config is used instead. -# last-good the config of the last kernel that BOOTED -# prev-good the one before it -- the way back from a bad answer -# saved the newest config this tool built with, booted or not -# file:<path> any file -# keep the tree's own .config, untouched -# Every build's config is copied to /etc/pkgusr/kernel-configs, which is -# outside the source tree and survives `packagemanager remove linux`: -# lfs-kernel config list what is saved -# lfs-kernel config diff prev-good -> last-good +# CONFIG_FROM: running (/proc/config.gz) | file:<path> | keep (the tree's own) CONFIG_FROM=running -# CONFIG_KEEP: how many per-build configs to keep in that directory. -# last-good.config and prev-good.config are NOT counted and never pruned. -CONFIG_KEEP=5 - # CONFIG_REFRESH: a config from one kernel is never complete for another -- # every version adds symbols the old file has no answer for, and something # has to decide them: @@ -79,25 +63,13 @@ CONFIG_REFRESH=olddefconfig # Each is read back from the final .config; one the kernel refuses (a missing # dependency, a symbol this version does not have) is named and stops the build. # -# Edit it here, or let the tool do it -- same file, with a backup: -# lfs-kernel enable --list hardware by name -# lfs-kernel enable ath9k-htc its symbols AND its firmware -# lfs-kernel config search atheros which symbol that is -# lfs-kernel config set NET_VENDOR_ATHEROS=y ALX=m -# lfs-kernel config unset ALX -# A NET_VENDOR_* / WLAN_VENDOR_* symbol only opens a submenu: name the -# driver under it too, or nothing is built. -# # LEGACY_TIOCSTI=n no program may type into a terminal it does not own. # With it allowed, shadow's su (4.20+) detaches the shell # of every `su - <user>` run by root -- "cannot set # terminal process group ... no job control" -- because # that user could otherwise inject commands into root's # terminal. Off, su keeps the terminal and job control. -# EXFAT_FS=m the kernel side of exFAT: mkfs.exfat and fsck.exfat -# (exfatprogs, services.stack) work on a filesystem -# the kernel cannot mount without this. -CONFIG_SET="LEGACY_TIOCSTI=n EXFAT_FS=m" +CONFIG_SET="LEGACY_TIOCSTI=n" # ---- firmware ------------------------------------------------------------ # diff --git a/lfs b/lfs index 9a3a0c9..655993f 100755 --- a/lfs +++ b/lfs @@ -60,7 +60,7 @@ except ImportError: " packagemanager pip install beautifulsoup4\n") sys.exit(2) -LFS_VERSION = "1.14.340" +LFS_VERSION = "1.14.299" def _build_id(): @@ -5724,12 +5724,7 @@ def cmd_bs_gen_chroot_scripts(args): for nm, left in unresolved: sys.stderr.write(" %-18s %s\n" % (nm, ", ".join("<%s>" % x for x in left))) - # SINGLE QUOTES INSIDE THE BRACES. Reusing the f-string's own quote - # character is legal only from Python 3.12, and `lfs` is the one tool that - # runs on the HOST -- where Python is whatever the distribution ships - # (Debian 12: 3.11). There it did not fail at this line, it failed to - # start at all, with a SyntaxError pointing at a debug print. - print(f"# step order -> {pkgusr_state(lfs, 'progress', 'steporder')}") + print(f"# step order -> {pkgusr_state(lfs, "progress", "steporder")}") write_chroot_env(lfs) install_helper(lfs) _install_collector_groups(lfs) diff --git a/lfs-helper b/lfs-helper index 929b8db..98985e0 100755 --- a/lfs-helper +++ b/lfs-helper @@ -49,7 +49,7 @@ set -u # Bump on every change that alters behaviour. `lfs-helper --version` makes it # possible to tell at a glance whether the copy inside the chroot is the one # that was just fixed -- guessing at that has wasted a lot of time. -LFS_HELPER_VERSION="1.14.340" +LFS_HELPER_VERSION="1.14.299" # A fingerprint of this file's own contents. A hand-maintained version number # goes stale the moment someone forgets to bump it -- and the reason for @@ -1157,33 +1157,9 @@ PHASES="$STATE_PROGRESS/phases" # job's message can be thousands of lines above the end, so showing the tail # (what you see on screen) is usually useless. show_build_errors() { - local log="$1" hits="" _rootline="" + local log="$1" hits="" [ -f "$log" ] || return 0 say "" - # A COMMAND ONLY ROOT CAN RUN, IN A BUILD THAT IS NOT ROOT. - # - # mount: /tmp: must be superuser to use mount. - # !! phase 'install' FAILED (exit 32) - # ...and the report said "no line matched the usual error patterns", - # because nothing here looks like a compiler error. It is not a - # permission to GRANT either: no collector group makes mount work. The - # command does not belong in a package's install at all (it was LVM2's - # "remount /tmp so the TESTS can make device nodes"), so say exactly - # that and name the line. - _rootline="$(grep -m1 -E "must be superuser|only root can|Operation not permitted \(are you root|must be run as root|requires root privileges" \ - "$log" 2>/dev/null)" - if [ -n "$_rootline" ]; then - fail "--- this step tried to do something only root may do ---" - say " $_rootline" - say " A package builds and installs as its own user. Mounting a" - say " filesystem, loading a module or making a device node is the" - say " system's business, not a package's -- and no collector group" - say " can grant it. Such a command usually comes from a book block" - say " that belongs to the TEST suite (blfs 1.14.334 drops those)." - say " Regenerate the script, and run the command yourself if you" - say " actually want what it does:" - say " packagemanager update <pkg> --reinstall --regenerate --run" - fi hits="$(grep -nE "(^|[^-])\\berror\\b:|\\*\\*\\* \\[|Error [0-9]+|No such file or directory|command not found|undefined reference" \ "$log" 2>/dev/null | grep -vE "Error 1$|Error 2$" | head -12)" if [ -n "$hits" ]; then @@ -2479,12 +2455,8 @@ unwritable_dirs_from_log() { # of it and then running `su ... test -w` on each is minutes of CPU for an # answer that is always in the last few lines. local _tail - # The same three wordings as the gate above: ExtUtils::Install's - # "Can't create '<dir>'" / "Do not have write permissions on '<dir>'" - # and tar's "Cannot mkdir". Without them this list came back EMPTY for - # every Perl module, and the repair had nothing to grant. _tail="$(tail -c 524288 "$log" 2>/dev/null \ - | grep -aE "Permission denied|Operation not permitted|cannot (create|remove|touch|open)|[Cc]an'?t create|[Dd]o not have write permissions|[Cc]annot mkdir|couldn't copy|No such file or directory" \ + | grep -aE "Permission denied|Operation not permitted|cannot (create|remove|touch|open)|couldn't copy|No such file or directory" \ | grep -avE "^[[:space:]]*(File \"|Traceback|at .* line )" \ | head -n 100)" # A RELATIVE PATH NEEDS THE `cd` THAT PRECEDED IT. @@ -2603,7 +2575,7 @@ unwritable_dirs_from_log() { printf '%s\n' "$_cmakedirs" printf '%s\n' "$_cdpaths" printf '%s\n' "$_tail" | grep -aoE "'[^']+'|\"[^\"]+\"" | tr -d "'\"" - printf '%s\n' "$_tail" | grep -aoE "/[A-Za-z0-9._/+:-]{6,}" | sed 's/:*$//'; } \ + printf '%s\n' "$_tail" | grep -aoE "/[A-Za-z0-9._/+-]{6,}"; } \ | while IFS= read -r p; do case "$p" in /*) ;; *) continue ;; esac if ! under_dir "$p" "$SNAP_ROOT"; then @@ -2622,44 +2594,6 @@ unwritable_dirs_from_log() { done } -# A BUILD OPTION THE BUILD ITSELF NAMES. -# -# cairomm 1.14.6 stops at configure: -# Run-time dependency Boost found: NO (tried system) -# ERROR: Building the test programs requires Boost Test ... and Fontconfig. -# Install them or use -Dbuild-tests=false or -Dbuild-tests=if-dependencies-found -# Boost is in avoid.stack -- deliberately, it is an hour of build and this -# desktop wants none of it -- so the dependency is never coming, and the -# option meson prints is the whole answer. A gparted chain that stops for -# an hour on a TEST program nobody asked for is the failure; the fix was on -# the screen the whole time. -# -# Only an option the tool suggested, only one that turns something OFF, only -# if it is not already there, and the old script is kept beside it. -_meson_opt_from_log() { # <log> <script> <name> - local log="$1" script="$2" pkg="${3:-}" opt - [ -f "$log" ] && [ -f "$script" ] || return 1 - opt="$(grep -ohE "use -D[A-Za-z0-9_.-]+=[A-Za-z0-9_.-]+" "$log" 2>/dev/null \ - | head -n1 | sed 's/^use //')" - [ -n "$opt" ] || return 1 - case "$opt" in - *=false|*=disabled|*=no|*=if-dependencies-found) ;; - *) return 1 ;; # never turn something ON - esac - grep -qF -- "$opt" "$script" && return 1 - grep -q "meson setup" "$script" || return 1 - cp -p "$script" "$script.bak-$(date +%Y%m%d-%H%M%S)" || return 1 - sed -i "s|meson setup|meson setup $opt|" "$script" || return 1 - note "# the build named the option itself -- adding it and trying again:" - note "# $opt" - note "# in $script (the old one is beside it as .bak-<date>)" - note "# A REGENERATED SCRIPT LOSES IT. To keep it, put it in" - note "# /etc/pkgusr/stacks/machine.conf:" - note "# [${pkg:-<package>}]" - note "# ${opt#-D}" | sed 's/=/ = /' - return 0 -} - # Grant <user> access to everything in the log it could not write, and say # which groups that took. Returns 0 if anything was granted. @@ -2782,7 +2716,6 @@ takeover_files_from_log() { user_exists "$owner" || continue if real_chown -h "$user:$user" "$f" 2>/dev/null; then warn "!! $f belonged to $owner -- given to $user, which is installing it" - LAST_TAKEOVER_OWNER="$owner" n=$((n + 1)) fi done <<< "$( @@ -2790,39 +2723,7 @@ takeover_files_from_log() { grep -oE "cannot create regular file '[^']+'" "$log" 2>/dev/null | cut -d"'" -f2 grep -oE "cannot unlink '[^']+'" "$log" 2>/dev/null | cut -d"'" -f2 grep -oE "cannot overwrite '[^']+'" "$log" 2>/dev/null | cut -d"'" -f2 - # PERL SAYS IT WITHOUT QUOTES, AND THE PATH CONTAINS COLONS. - # - # Cannot forceunlink /usr/share/man/man3/XML::Parser.3: Permission denied - # - # ExtUtils::Install replaces a file by unlinking it first, and in a - # STICKY directory (/usr/share/man/man3 is drwxrwxr-t) only the - # file's owner may do that -- so granting the directory, which the - # repair did six times, can never help. The file belongs to an - # older account for the same module (p_xml-parser; this one is - # p_perl-xml-parser), and handing it over is the fix. - # The path is taken as a whole token: XML::Parser.3 has colons in - # it, so splitting on ":" loses it. - grep -oE "[Cc]annot forceunlink [^ ]+" "$log" 2>/dev/null \ - | sed 's/^[Cc]annot forceunlink //; s/:$//' - grep -oE "[Cc]an'?t (unlink|remove|rename) [^ ]+" "$log" 2>/dev/null \ - | sed -E "s/^[Cc]an'?t (unlink|remove|rename) //; s/:\$//" } | sort -u)" - # AN OLDER ACCOUNT FOR THE SAME PACKAGE. When several files move from - # one package user whose name is this one's with a language prefix - # dropped (xml-parser <- perl-xml-parser), the two are the same upstream - # module installed twice under two book anchors. Taking the files is - # right; leaving the old account to claim them in `verify` is not. - if [ "$n" -ge 2 ] && [ -n "${LAST_TAKEOVER_OWNER:-}" ]; then - local _a="${user#p_}" _b="${LAST_TAKEOVER_OWNER#p_}" - case "$_a" in - perl-"$_b"|python-"$_b"|"$_b"-perl|"$_b"|py-"$_b") - warn "!! $LAST_TAKEOVER_OWNER looks like an older account for the same" - warn " module as $user -- two book anchors, one upstream package." - warn " The files are $user's now. The old account still claims" - warn " them; when this build is done, take it out of the way:" - warn " packagemanager remove ${_b} --purge --run" ;; - esac - fi [ "$n" -gt 0 ] } @@ -4592,16 +4493,7 @@ diagnose_paths_from_log() { [ -f "$log" ] || return 0 local paths # tail only, and cap the results: see unwritable_dirs_from_log for why - # A PERL MAN PAGE HAS COLONS IN ITS NAME. - # - # /usr/share/man/man3/XML::Parser.3 was reported as - # /usr/share/man/man3/XML missing; parent ... is drwxrwxr-t - # -- the class below had no ":" in it, so the name was cut at the first - # one, and the report named a file that has never existed while the one - # that could not be replaced went unmentioned. A trailing colon (the - # message's own "path: Permission denied") is stripped afterwards. - paths="$(tail -c 524288 "$log" 2>/dev/null | grep -aoE "/[A-Za-z0-9._/+:-]{6,}" \ - | sed 's/:*$//' \ + paths="$(tail -c 524288 "$log" 2>/dev/null | grep -aoE "/[A-Za-z0-9._/+-]{6,}" \ | grep -vE "^/(proc|sys|dev|tmp)/" | sort -u \ | while IFS= read -r p; do case "$p" in @@ -4878,7 +4770,7 @@ is_never_claimed() { } _drop_other_packages_files() { - local owner="$1" p cur _sp _tgt _to + local owner="$1" p cur while IFS= read -r p; do [ -n "$p" ] || continue # a shared index or a generated cache is nobody's file @@ -4888,32 +4780,6 @@ _drop_other_packages_files() { && [ "$cur" != UNKNOWN ] && user_exists "$cur"; then continue # another package installed this fi - # CURRENT OWNERSHIP IS NOT PROOF OF PROVENANCE (1.14.310). - # - # The test above asks who owns the file NOW, and that is circular: - # once the lease had handed /usr/bin/su and six /usr/sbin/iptables* - # links to p_webkitgtk, they read as "the owner's own" and stayed in - # its manifest -- which is the list the NEXT lease reads. The - # manifest fed the lease and the lease fed the manifest, and the - # damage repeated on every rebuild. Two facts break the loop, and - # neither depends on who holds the file today: - # - # * setid.conf names the package that installs a setid file. - # * a symlink is installed by whoever installed its target. - if _sp="$(_setid_pkg_for "$p" 2>/dev/null)" \ - && [ -n "$_sp" ] && [ "$_sp" != "$(unprefix_pkg_user "$owner")" ]; then - continue - fi - if [ -L "$p" ]; then - _tgt="$(readlink -f "$p" 2>/dev/null)" - if [ -n "$_tgt" ] && [ "$_tgt" != "$p" ]; then - _to="$(stat -c %U "$_tgt" 2>/dev/null)" - if [ -n "$_to" ] && [ "$_to" != "$owner" ] && [ "$_to" != root ] \ - && [ "$_to" != UNKNOWN ] && user_exists "$_to"; then - continue - fi - fi - fi printf '%s\n' "$p" done } @@ -5592,29 +5458,6 @@ _setid_policy_for() { END { exit !found }' "$conf" } -# _setid_pkg_for <path> -> the PACKAGE setid.conf names as the file's -# provider (the optional 5th column), or nothing. -# -# WHY A SETID FILE NEEDS AN OWNER'S NAME. The lease hands a root-owned file -# to a package user for the length of a phase, and one of the doors it may -# come through is "the file carries a setid bit". That door had no lock: -# ANY package whose time-window manifest happened to list /usr/bin/su -- and -# a webkitgtk build runs for hours, so its manifest listed whatever root -# wrote meanwhile -- was handed su, 0711 and no longer setuid, until its -# phase ended. `su` is broken for those hours and, if the build is -# interrupted, for good. -# -# Only shadow installs su. With the provider written down, the lease can -# say so: shadow may reclaim it, nobody else may borrow it. -_setid_pkg_for() { - local conf; conf="$(_setid_conf)" || return 1 - awk -F'|' -v p="$1" ' - /^[[:space:]]*(#|$)/ { next } - { f = $1; gsub(/[[:space:]]/, "", f) - if (f == p) { k = $5; gsub(/[[:space:]]/, "", k); print k; found = 1; exit } } - END { exit !found }' "$conf" -} - _lease_dir() { printf '%s/%s\n' "$LEASES" "$(unprefix_pkg_user "$1")"; } # _lease_candidates <owner> <script> <phase> @@ -5643,7 +5486,7 @@ _lease_candidates() { # lease_files <owner> <script> <phase> (root; before the phase) lease_files() { - local owner="$1" script="$2" phase="$3" dir ledger n=0 p u g m accounts _sp _lt + local owner="$1" script="$2" phase="$3" dir ledger n=0 p u g m accounts [ "$(id -u)" = 0 ] || return 0 case "$phase" in install|configure|all|update) ;; *) return 0 ;; esac user_exists "$owner" || return 0 @@ -5691,28 +5534,8 @@ lease_files() { # package's file is root's for two reasons only: a setid bit # (the policy put it there) or its own configure commands write # it. Anything else root owns is not the package's to borrow. - if [ -u "$p" ] || [ -g "$p" ] || _setid_policy_for "$p" >/dev/null 2>&1; then - # ...AND "IT IS SETID" IS NOT A REASON EITHER, unless the - # package is the one that installs it (1.14.310). - # - # This was the widest door in the lease and it had no lock. - # /usr/bin/su is root 4711; any package whose time-window - # manifest listed it -- webkitgtk's did, because `cfg - # shadow-pam` ran during its build -- came through here, and - # `chown` to the package user clears the setuid bit as a - # side effect the kernel never reports. Three times over - # three months the answer to "why is su owned by - # p_webkitgtk and 0711" was this line. - # - # setid.conf names the provider (5th column). shadow may - # reclaim its own su; for anyone else a setid file is the - # system's, not a package's. - _sp="$(_setid_pkg_for "$p" 2>/dev/null)" - if [ -z "$_sp" ] || [ "$_sp" != "$(unprefix_pkg_user "$owner")" ]; then - warn "# $p is setuid/setgid and ${_sp:-no package} installs it -- not leasing it to $owner" - continue - fi - elif ! printf '%s\n' "$_cfg_targets" | grep -qxF "$p"; then + if ! { [ -u "$p" ] || [ -g "$p" ] || _setid_policy_for "$p" >/dev/null 2>&1 \ + || printf '%s\n' "$_cfg_targets" | grep -qxF "$p"; }; then detail "# $p is root's, not setid and not written by $owner's configure -- left alone" continue fi @@ -5785,20 +5608,7 @@ lease_settle() { continue fi fi - # A SYMLINK IS COMPARED BY WHERE IT POINTS, NOT BY `cmp` (1.14.310). - # - # `cp -a` copies a symlink AS a symlink, so the copy in the lease - # directory carries the same *relative* target -- and relative to - # $LEASES/<pkg>/files, not to /usr/sbin. /usr/sbin/iptables -> - # xtables-nft-multi copied aside is a dangling link, `cmp` cannot - # open it, and a failed compare reads here as "the phase rewrote - # it". So every leased symlink was declared the package's own and - # kept -- which is how six /usr/sbin/iptables* links, made by `cfg - # iptables-nft`, ended up owned by p_webkitgtk and written into its - # pkg.lst. lease_restore already compared links with readlink; this - # is the same test, on the other side of the branch. - if { [ -L "$p" ] && [ "$(readlink "$p")" = "$(readlink "$dir/files/$n")" ]; } \ - || { [ ! -L "$p" ] && cmp -s "$dir/files/$n" "$p" 2>/dev/null; }; then + if cmp -s "$dir/files/$n" "$p" 2>/dev/null; then # the phase did not change it: the lease was not needed chown -h "$u:$g" "$p" 2>/dev/null [ -L "$p" ] || chmod "$m" "$p" 2>/dev/null @@ -6015,34 +5825,8 @@ run_phase_as() { # ninja prints `ninja: build stopped: subcommand failed.` with no # `***`, so it needs its own branch -- the one place this check was # supposed to work, and did not. - # A SHARED REGISTRY IS NOT THE PACKAGE. - # - # perl-class-inspector installed both its modules and both its man - # pages, and was then failed for this line: - # .../core_perl/perllocal.pod: Permission denied - # which make itself had already marked "(ignored)". perllocal.pod - # is a list of what has been installed, written by every module and - # owned by the first one; not updating it loses a paragraph of - # bookkeeping, not a file of this package's. The same is true of - # every entry in shared_files_list. They are excluded from this - # test -- and reported, because the reason they cannot be written - # is worth fixing (share_shared_files does it on the next repair). - local _ign _shared_hit="" - _ign="$(mktemp)" - { shared_files_list; printf '%s\n' perllocal.pod; } > "$_ign" 2>/dev/null - _shared_hit="$(grep -Ff "$_ign" "$log" 2>/dev/null \ - | grep -m1 -E ": Permission denied$|Operation not permitted")" - _swallowed="$(grep -vFf "$_ign" "$log" 2>/dev/null \ - | grep -m1 -E "^(make(\[[0-9]+\])?|ninja): \*\*\* .*(Error [0-9]+|build stopped)|^ninja: build stopped|^install: cannot create|: Permission denied$|^CMake Error|^-- Configuring incomplete|^configure: error:|^meson\.build:[0-9]+:[0-9]+: ERROR:|^ERROR: Unhandled python|^error: could not compile|^fatal: could not read Username|^fatal: repository .* not found|^fatal: Authentication failed|BackendUnavailable: Cannot import|^ModuleNotFoundError: No module named")" - rm -f "$_ign" - if [ -z "$_swallowed" ] && [ -n "$_shared_hit" ]; then - warn "a shared registry file could not be updated:" - warn " $_shared_hit" - note " that file is written by every package of its family and" - note " owned by the first one; this package's OWN files installed." - note " Fixed for the next one: lfs-helper fix-perms $name --run" - share_shared_files - fi + _swallowed="$(grep -m1 -E "^(make(\[[0-9]+\])?|ninja): \*\*\* .*(Error [0-9]+|build stopped)|^ninja: build stopped|^install: cannot create|: Permission denied$|^CMake Error|^-- Configuring incomplete|^configure: error:|^meson\.build:[0-9]+:[0-9]+: ERROR:|^ERROR: Unhandled python|^error: could not compile|^fatal: could not read Username|^fatal: repository .* not found|^fatal: Authentication failed|BackendUnavailable: Cannot import|^ModuleNotFoundError: No module named" \ + "$log" 2>/dev/null)" if [ -n "$_swallowed" ]; then warn "!! this phase exited 0, but its log says otherwise:" warn " $_swallowed" @@ -7009,7 +6793,7 @@ cmd_build() { # Directories that belong to the install group are shared infrastructure # (root:install) and must stay that way -- only take the ones this # package created itself. - local n=0 p cur skipped_foreign=0 _sp _lt _lo + local n=0 p cur skipped_foreign=0 while IFS= read -r p; do [ -n "$p" ] || continue # NEVER take a file that already belongs to another package user. @@ -7028,61 +6812,6 @@ cmd_build() { skipped_foreign=$((skipped_foreign+1)) continue fi - # A PACKAGE USER CANNOT HAVE WRITTEN A ROOT-OWNED FILE -- HERE TOO. - # - # The manifest above already drops root-owned paths when the phase - # ran AS the package user (_root_ok). This loop did not: it walked - # the RAW lists again, so a file the manifest correctly refused was - # chowned anyway. One rule, applied in one of the two places that - # needed it. - if [ "$_root_ok" != 1 ] && [ "$cur" = root ]; then - skipped_foreign=$((skipped_foreign+1)) - continue - fi - # CHOWN DESTROYS A SETUID BIT, AND NOTHING SAYS SO. - # - # Attribution is by TIME WINDOW, and a webkitgtk build runs for - # hours. Anything that touched /usr/bin/su in that window -- - # `lfs-helper setid`, a shadow reinstall, `cfg shadow-pam` -- made - # it "newer than the stamp", so this loop took a root-owned file - # that was fair game by every rule above and chowned it: - # -rwx--x--x 1 p_webkitgtk p_webkitgtk /bin/su - # The kernel clears setuid/setgid on chown, so one command changed - # the owner AND removed the bit that makes su work. A file - # carrying either bit is never this sweep's to take, whoever ran - # the phase: the modes in setid.conf are the system's, not a - # package's. - case "$(stat -c %a "$p" 2>/dev/null)" in - [1-7][0-7][0-7][0-7]) - warn "# not taking $p -- it is setuid/setgid ($(stat -c %a "$p")); chown would clear that" - skipped_foreign=$((skipped_foreign+1)) - continue ;; - esac - # ...AND THE SAME TWO PROVENANCE RULES THE MANIFEST USES - # (1.14.310): a setid file belongs to the package setid.conf - # names, and a symlink belongs to whoever owns its target. The - # mode test above cannot see either -- `stat -c %a` on a symlink - # reports the link's own 777, so no link has ever looked setuid - # to it, and a file whose setuid bit a previous chown already - # cleared does not look setuid either. - if _sp="$(_setid_pkg_for "$p" 2>/dev/null)" \ - && [ -n "$_sp" ] && [ "$_sp" != "$name" ]; then - warn "# not taking $p -- setid.conf says $_sp installs it" - skipped_foreign=$((skipped_foreign+1)) - continue - fi - if [ -L "$p" ]; then - _lt="$(readlink -f "$p" 2>/dev/null)" - if [ -n "$_lt" ] && [ "$_lt" != "$p" ]; then - _lo="$(stat -c %U "$_lt" 2>/dev/null)" - if [ -n "$_lo" ] && [ "$_lo" != "$owner" ] && [ "$_lo" != root ] \ - && [ "$_lo" != UNKNOWN ] && user_exists "$_lo"; then - warn "# not taking $p -- it points at $_lo's $_lt" - skipped_foreign=$((skipped_foreign+1)) - continue - fi - fi - fi real_chown -h "$owner:$owner" "$p" 2>/dev/null && n=$((n+1)) done <<< "$(printf '%s\n%s\n' "$newfiles" "$touched" | grep -v '^$' | sort -u)" while IFS= read -r p; do @@ -7121,16 +6850,7 @@ cmd_build() { # Partition first" -- must never be answered by handing directories away. local _perm_ish=0 # "file INSTALL cannot": CMake wraps "Permission denied" over two lines. - # PERL SAYS IT IN ITS OWN WORDS, AND THEY MATCHED NOTHING. - # - # ERROR: Can't create '/usr/lib/perl5/5.42/site_perl/File' - # Do not have write permissions on '/usr/lib/perl5/5.42/site_perl/File' - # make: *** [Makefile:777: pure_site_install] Error 13 - # - # ExtUtils::Install never writes "Permission denied" for this case, so - # the gate said "the failure does not look like a permission problem", - # ran zero rounds, and stopped a plan whose fix is one group. - grep -qiE "permission denied|operation not permitted|cannot (create|make|remove|change|touch)|can(no|')?t (create|make|mkdir)|do not have write permissions|couldn't copy|read-only file system|file INSTALL cannot" \ + grep -qiE "permission denied|operation not permitted|cannot (create|make|remove|change|touch)|couldn't copy|read-only file system|file INSTALL cannot" \ "$log" 2>/dev/null && _perm_ish=1 # A BOOK COMMAND THAT WORKS ONCE. # @@ -7158,17 +6878,13 @@ cmd_build() { # permission problem and granting will never fix it -- but it IS # automatically fixable, so it must reach the retry loop. [ -n "$(missing_dirs_from_log "$log")" ] && _perm_ish=1 - # ...and neither is "use -Dbuild-tests=false", which is the build telling - # us how to finish without a dependency this machine does not have. - grep -qE "use -D[A-Za-z0-9_.-]+=" "$log" 2>/dev/null && _perm_ish=1 # SAY WHY THERE WAS NO RETRY. cmake failed twice on the same missing # directory with no attempt and no explanation, which left the only # evidence -- "did the loop run?" -- invisible. A decision not to act is # a decision, and it has to be reviewable. if [ "$rc" != 0 ] && [ "$auto_fix" = 1 ] && [ "$_perm_ish" != 1 ]; then - detail "# no automatic repair attempted: the failure is not one this" - detail "# tool knows how to fix (a permission, a missing directory," - detail "# or a build option the build itself names)" + detail "# no automatic repair attempted: the failure does not look like" + detail "# a permission or missing-directory problem" fi # Grants are per-directory now (a collector group is permission to install @@ -7200,7 +6916,6 @@ cmd_build() { && _fixed_something=1 fi create_missing_dirs_from_log "$log" "$owner" && _fixed_something=1 - _meson_opt_from_log "$log" "$s" "$name" && _fixed_something=1 # the package's own leftovers from a previous install of the same # version: removing them IS the fix, and it counts as one, so the # phase is retried @@ -7848,47 +7563,13 @@ cmd_build() { fail "!" if [ -n "$_dep_provider" ]; then fail "! '$_dep' is the pkg-config name of '$_dep_provider'," - fail "! which is RECORDED as installed here." - # ASK THE MANIFEST, DO NOT GUESS. - # - # "If the .pc is absent, it installed without it" left the - # reader to find out which of two very different things had - # happened: a package that installed and then lost a file, - # or one that is recorded as installed and put nothing on - # disk at all (a build that failed after its account and - # home were made). The manifest says which, in one read. - local _php _plst _pn=0 _ppc="" - _php="$(pkgusr_home_for "$_dep_provider" 2>/dev/null)" - _plst="$_php/pkg.lst" - if [ -s "$_plst" ]; then - _pn="$(grep -c . "$_plst" 2>/dev/null || echo 0)" - _ppc="$(grep -E '\.pc$' "$_plst" 2>/dev/null | head -n3)" - fi - if [ "${_pn:-0}" -eq 0 ]; then - fail "! ...but its manifest is EMPTY: nothing of it is on" - fail "! disk. Its last build did not finish, and whatever" - fail "! marked it installed was wrong. Build it again:" - fail "! packagemanager install $_dep_provider --run --reinstall" - elif [ -z "$_ppc" ]; then - fail "! It records $_pn file(s), and NONE of them is a .pc" - fail "! -- so it installed without the pkg-config file this" - fail "! build needs. Rebuild it:" - fail "! packagemanager install $_dep_provider --run --reinstall" - else - fail "! It records $_pn file(s), including:" - printf '%s\n' "$_ppc" | while IFS= read -r _l; do - if [ -e "$_l" ]; then - fail "! $_l (on disk)" - else - fail "! $_l -- MISSING from disk" - fi - done - fail "! If the one this build wants is missing, rebuild it;" - fail "! if it is there, the build did not look where it is:" - fail "! pkg-config --exists --print-errors '$_dep'" - fail "! pkg-config --variable pc_path pkg-config" - fail "! packagemanager install $_dep_provider --run --reinstall" - fi + fail "! which IS installed here. So this is a pkg-config" + fail "! problem, not a missing package:" + fail "! pkg-config --exists --print-errors '$_dep'" + fail "! ls /usr/lib/pkgconfig/$_dep.pc" + fail "! If the .pc is absent, $_dep_provider installed without it" + fail "! -- rebuild it:" + fail "! packagemanager install $_dep_provider --run --reinstall" fi _pcdir="$(timeout 10 find "${SNAP_ROOT%/}/usr/lib" "${SNAP_ROOT%/}/usr/share" \ -maxdepth 3 -name "$_dep.pc" 2>/dev/null | head -1)" @@ -10506,28 +10187,12 @@ cmd_pm_install() { say "Install $user_name" say "-------------------------------" say "install file: $install_script" - # "CHECK THAT THIS IS THE RIGHT ACCOUNT" IS A FIRST-INSTALL QUESTION. - # - # It is worth asking once: a typo in the account name on a FIRST install - # scatters a package across the wrong home and is tedious to undo. It is - # worth nothing on an update -- the account exists, it already owns this - # package's files, and the answer was settled the day it was installed. - # Asking anyway turned every `update <pkg> --run` into a keypress, and a - # prompt people press through is a prompt that has stopped protecting - # anything. - local _known=0 - if user_exists "$user_name"; then - [ -s "$(pkgusr_home_for "$user_name")/pkg.lst" ] && _known=1 - [ -f "$(pkgusr_home_for "$user_name")/.pkgusr-installed" ] && _known=1 - fi - if ! assume_yes && [ "$_known" != 1 ]; then + if ! assume_yes; then warn "First check the install file for: link, name, wrong commands" warn "check that \"$user_name\" is the right account" printf 'Press enter to continue ' read -r -n1 _ say "" - elif [ "$_known" = 1 ]; then - say "# $user_name is an existing package with files recorded -- not asking" fi # Phased scripts carry their metadata (install_groups, nimgnu_groups_dir, @@ -11622,22 +11287,6 @@ EOF # the people who must update it. A shared directory is root:install and # group-writable; a shared file is the same rule, one level down. shared_files_list() { - # perllocal.pod IS THE info/dir OF THE PERL TREE. - # - # Every Perl module's `make install` runs doc_install, which APPENDS a - # paragraph to /usr/lib/perl5/<ver>/core_perl/perllocal.pod. So the - # file belongs to whichever module was installed first -- here - # p_xml-parser, mode 664, group p_xml-parser -- and every module after - # that fails on it: - # /usr/lib/perl5/5.42/core_perl/perllocal.pod: Permission denied - # Granting the DIRECTORY to the perl collector group does not help: the - # file's own group is one package's. It is rewritten by many packages - # and installed by none, which is exactly this list's rule. - # The version is in the path, so it is globbed rather than spelled. - local _p - for _p in "${SNAP_ROOT%/}"/usr/lib/perl5/*/*/perllocal.pod; do - [ -e "$_p" ] && printf '%s\n' "$_p" - done printf '%s\n' \ "${SNAP_ROOT%/}/usr/share/info/dir" \ "${SNAP_ROOT%/}/etc/xml/catalog" \ @@ -11672,19 +11321,9 @@ shared_files_list() { # The directories those files live in. They may NOT be sticky -- see below. shared_file_dirs() { - local f d + local f while IFS= read -r f; do - [ -n "$f" ] || continue - d="$(dirname "$f")" - # THE FILE IS SHARED; THE DIRECTORY IS NOT. - # - # /usr/lib/perl5/<ver>/core_perl holds perl's OWN modules and is - # p_perl's install directory, recorded in its manifest. Only - # perllocal.pod inside it is written by everyone. Handing the whole - # directory to root:install would take a directory away from the - # package that installed it and make `verify` right to complain. - case "$d" in */usr/lib/perl5/*) continue ;; esac - printf '%s\n' "$d" + [ -n "$f" ] && dirname "$f" done < <(shared_files_list) | sort -u } diff --git a/lfs-kernel b/lfs-kernel index 92d97bb..bcd52d9 100755 --- a/lfs-kernel +++ b/lfs-kernel @@ -1,36 +1,13 @@ #!/bin/bash # lfs-kernel -- build and install a kernel. # -# lfs-kernel what a build would do (nothing is built) -# lfs-kernel --run build and install the configured version -# lfs-kernel load <module> [--boot] -# load it now, and at every boot with --boot +# lfs-kernel build and install the configured version # lfs-kernel list what kernel.org offers right now -# lfs-kernel 6.12.40 --run that version, once (config untouched) -# lfs-kernel latest --run the newest stable, once +# lfs-kernel 6.12.40 that version, once (config untouched) +# lfs-kernel latest the newest stable, once # lfs-kernel detect-esp print the EFI directory this would use # lfs-kernel verify [ver] check that everything is installed # lfs-kernel firmware install what FIRMWARE_ONLY names, no kernel build -# lfs-kernel status how far the last build got, what is installed -# lfs-kernel enable --list hardware this tool knows by name -# lfs-kernel enable ath9k-htc -# put what that hardware needs into kernel.conf -# lfs-kernel modules [text] what this kernel has, and what is loaded -# lfs-kernel config what the next build's .config comes from -# lfs-kernel config list the saved configs (the last two that BOOTED) -# lfs-kernel config diff [a] [b] -# what changed between two saved configs -# lfs-kernel config search <text> -# which symbols that text means, and their state -# lfs-kernel config set SYM=y|m|n [...] -# add them to CONFIG_SET in kernel.conf -# lfs-kernel config save [--good] -# keep the running config; --good: it BOOTS -# lfs-kernel config from last-good -# build from that config, not from the running -# kernel -# lfs-kernel config unset SYM [...] -# take them out again # lfs-kernel firmware --missing [--fix] [--all] # firmware this machine asks for and has not got, # from the boot log, this tree's modules and the @@ -69,24 +46,6 @@ _detect_esp() { findmnt -rn -t vfat,msdos -o TARGET 2>/dev/null | head -n1 } -# A BUILD IS NOT SOMETHING YOU START BY TYPING FOUR WORDS. -# -# `lfs-kernel` alone compiled a kernel, installed it over the one the -# machine boots, installed the modules, fetched firmware and copied the -# Documentation -- an hour, from a bare command. Every other tool here -# prints the plan and waits for --run; this one now does too. --run is -# taken from anywhere on the line, so `lfs-kernel 7.2.6 --run` and -# `lfs-kernel --run` both read naturally. -_RUN=0 -_rest=() -for _a in "$@"; do - case "$_a" in - --run) _RUN=1 ;; - *) _rest+=("$_a") ;; - esac -done -set -- ${_rest[@]+"${_rest[@]}"} - case "${1:-}" in list) echo "# kernel.org offers:" @@ -96,24 +55,7 @@ case "${1:-}" in detect-esp) printf '%s\n' "$(_detect_esp)"; exit 0 ;; verify) - shift; _VERIFY=1; _CLI_VERSION="${1:-}" ;; - enable) - # hardware by name: what symbols it needs, written into kernel.conf - shift; _ENABLE=1; _ENARGS=("$@") ;; - modules|mods) - # what this kernel actually has, and what is loaded - shift; _MODCMD=1; _MODARGS=("$@") ;; - load|unload) - # load a module now, and (--boot) at every boot - _LOADCMD="$1"; shift; _LOADARGS=("$@") ;; - status|state) - # how far the last build got, and what is installed -- reads only - shift; _STATUS=1 ;; - config) - # the config, without building anything: what it will be made of, - # what the last two booted kernels used, and the one place this - # machine's own answers are written down (CONFIG_SET) - shift; _CFGCMD=1; _CFGARGS=("$@") ;; + shift; _VERIFY=1; VERSION="${1:-}" ;; firmware) # firmware alone, with no kernel build: the blobs are a separate # package and a missing one is a run-time problem, so fixing it must @@ -134,37 +76,8 @@ case "${1:-}" in # new command silently pushes out of range. sed -n '2,/^# Settings:/p' "$0" | sed '$d' | sed 's/^# \?//'; exit 0 ;; "") ;; - search|find) - # `lfs-kernel search AR9271` is what a person types, and it used to - # mean "build the kernel called search" -- see the version check - # below for what that really did. It is the config search. - shift; _CFGCMD=1; _CFGARGS=("search" "$@") ;; - set) - shift; _CFGCMD=1; _CFGARGS=("set" "$@") ;; -*) echo "!! unknown option: $1 (try: lfs-kernel --help)"; exit 1 ;; - *) - # A WORD THAT IS NOT A VERSION IS NOT A VERSION. - # - # Anything unrecognised became the version to build, and the version - # from kernel.conf then overwrote it (below) -- so a typo, or a - # subcommand this tool does not have, silently started a FULL BUILD - # of the configured kernel. `lfs-kernel search AR9271` rebuilt and - # reinstalled 7.2.6, fetched the firmware and said nothing about the - # word `search` at all. - case "$1" in - latest|[0-9]*.[0-9]*) _CLI_VERSION="$1" ;; - *) echo "!! lfs-kernel: '$1' is neither a command nor a version." - echo " commands: status, list, enable, modules, load," - echo " config, search, firmware, verify, detect-esp" - echo " a version looks like 7.2.6, or the word latest." - case "$1" in - conf*|cfg) echo " did you mean: lfs-kernel config" ;; - mod*|lsmod) echo " did you mean: lfs-kernel modules" ;; - en*|add) echo " did you mean: lfs-kernel enable --list" ;; - fw|firm*) echo " did you mean: lfs-kernel firmware --missing" ;; - esac - exit 1 ;; - esac ;; + *) VERSION="$1" ;; esac # The config is INSTALLED, not written on first run: it is a file to edit @@ -178,13 +91,6 @@ if [ ! -f "$CONF" ]; then fi . "$CONF" -# THE COMMAND LINE WINS. `lfs-kernel 6.12.40` sets VERSION, and then this -# file -- sourced after it -- set VERSION=latest straight over the top: the -# documented "that version, once" built whatever kernel.conf said instead, -# and `verify 7.2.4` reported on another kernel entirely. Re-apply it here, -# where nothing can overwrite it again. -[ -n "${_CLI_VERSION:-}" ] && VERSION="$_CLI_VERSION" - # A CONFIG FROM AN OLDER RELEASE. # # kernel.conf is never overwritten once edited, so an installed copy can @@ -230,22 +136,8 @@ _latest_stable() { _releases | awk '$1=="stable"{print $2; exit}'; } # "latest" would ask kernel.org for a version it is never going to look at, # and fail on a machine with no network while reporting on one that is # perfectly installed. -if [ "${_VERIFY:-0}" = 1 ] || [ "${_CFGCMD:-0}" = 1 ] \ - || [ "${_ENABLE:-0}" = 1 ] || [ "${_MODCMD:-0}" = 1 ] \ - || [ "${_STATUS:-0}" = 1 ] || [ -n "${_LOADCMD:-}" ]; then +if [ "${_VERIFY:-0}" = 1 ]; then : -elif [ "${_FWONLY:-0}" = 1 ] && [ "$VERSION" = latest ]; then - # A FIRMWARE REPORT IS ABOUT THIS MACHINE, NOT ABOUT KERNEL.ORG. - # - # `lfs-kernel firmware --missing` asked the website which kernel is - # newest -- for a question answered entirely by /lib/modules and the - # boot log -- printed "newest stable on kernel.org", and on a machine - # with no network it EXITED instead of reporting. - VERSION="$(uname -r | sed "s/-.*//")" - if [ ! -d "/lib/modules/$VERSION" ] && [ ! -d "/lib/modules/$(uname -r)" ]; then - VERSION="$(_latest_stable)" - [ -n "$VERSION" ] || { echo "!! no /lib/modules for the running kernel, and kernel.org is unreachable"; exit 1; } - fi elif [ "$VERSION" = latest ]; then VERSION="$(_latest_stable)" [ -n "$VERSION" ] || { echo "!! could not reach $MIRROR_JSON -- set VERSION in $CONF"; exit 1; } @@ -260,21 +152,12 @@ fi acct="p_$PKG" # `verify` must not CREATE anything -- a report that makes an account is not # a report. It only reads $acct's home below. -if [ "${_VERIFY:-0}" != 1 ] && [ "${_CFGCMD:-0}" != 1 ] \ - && [ "${_ENABLE:-0}" != 1 ] && [ "${_MODCMD:-0}" != 1 ] \ - && [ "${_STATUS:-0}" != 1 ] && [ -z "${_LOADCMD:-}" ] \ - && ! id "$acct" >/dev/null 2>&1; then +if [ "${_VERIFY:-0}" != 1 ] && ! id "$acct" >/dev/null 2>&1; then packagemanager add-user "$PKG" || { echo "!! could not create $acct"; exit 1; } fi -[ "${_VERIFY:-0}" = 1 ] || [ "${_CFGCMD:-0}" = 1 ] \ - || [ "${_ENABLE:-0}" = 1 ] || [ "${_MODCMD:-0}" = 1 ] \ - || [ "${_STATUS:-0}" = 1 ] || [ -n "${_LOADCMD:-}" ] \ - || lfs-helper fix-home "$acct" --run >/dev/null 2>&1 || true +[ "${_VERIFY:-0}" = 1 ] || lfs-helper fix-home "$acct" --run >/dev/null 2>&1 || true SRC="$(getent passwd "$acct" 2>/dev/null | cut -d: -f6)/src" -[ "${_VERIFY:-0}" = 1 ] || [ "${_CFGCMD:-0}" = 1 ] \ - || [ "${_ENABLE:-0}" = 1 ] || [ "${_MODCMD:-0}" = 1 ] \ - || [ "${_STATUS:-0}" = 1 ] || [ -n "${_LOADCMD:-}" ] \ - || install -d -o "$acct" -g "$acct" -m 755 "$SRC" +[ "${_VERIFY:-0}" = 1 ] || install -d -o "$acct" -g "$acct" -m 755 "$SRC" # Everything that touches the tree runs AS THE PACKAGE USER: a root-owned # file left in its home would break the next build and every rule about who @@ -285,739 +168,6 @@ tree="$SRC/linux-$VERSION" tarball="$SRC/linux-$VERSION.tar.xz" major="${VERSION%%.*}" -# -------------------------------------------------------------------------- -# THE CONFIGS ARE KEPT, OUTSIDE THE TREE. -# -# A .config lives in a source tree, and a source tree is deleted by hand or -# by `packagemanager remove linux`, which takes the account's home, -# and a failed build can leave one unusable. What is lost with it is the -# only record of every answer this machine has ever given -- hours of -# menuconfig, and the difference between a kernel that boots and one that -# does not. -# -# So every config this tool builds with is copied to $KCFG_DIR, which is -# root's, small (a config is ~200 kB) and has nothing to do with the tree: -# -# config-<version>-<date> every build, newest CONFIG_KEEP kept -# last-good.config the config of the kernel that BOOTED -# prev-good.config the one before that -# -# The last two are the ones worth having: they are known to work on this -# hardware. CONFIG_FROM=last-good starts from one, `config diff` says what -# changed between them, and `config set` is how an answer is added without -# hand-editing a file that the next build overwrites. -# -------------------------------------------------------------------------- -KCFG_DIR="${KCFG_DIR:-/etc/pkgusr/kernel-configs}" -: "${CONFIG_KEEP:=5}" - -_kcfg_readable() { # a config file with something in it - [ -s "$1" ] && grep -q '^CONFIG_' "$1" 2>/dev/null -} - -_kcfg_newest() { # the newest archived build config, or empty - ls -1t "$KCFG_DIR"/config-* 2>/dev/null | head -n1 -} - -_kcfg_ver() { # the kernel version a config file was built for - sed -n 's/^# Linux\/[^ ]* \([0-9][^ ]*\) Kernel Configuration.*/\1/p' "$1" \ - 2>/dev/null | head -n1 -} - -_kcfg_archive() { # <config> <version> -- keep a copy of what we built with - local src="$1" ver="$2" dst - _kcfg_readable "$src" || return 0 - install -d -m 755 "$KCFG_DIR" || return 0 - dst="$KCFG_DIR/config-$ver-$(date +%Y%m%d-%H%M%S)" - cp "$src" "$dst" 2>/dev/null || return 0 - chmod 644 "$dst" - echo "# config kept as $dst" - # PRUNE THE BUILD SNAPSHOTS ONLY. last-good/prev-good are not in this - # pattern and are never deleted here. - ls -1t "$KCFG_DIR"/config-* 2>/dev/null | tail -n +"$((CONFIG_KEEP + 1))" \ - | while read -r _old; do rm -f "$_old"; done -} - -_kcfg_mark_good() { # <config> <version> -- this kernel has booted - local src="$1" ver="$2" - _kcfg_readable "$src" || return 0 - install -d -m 755 "$KCFG_DIR" || return 0 - # the same version again is the same kernel: refresh it, do not push the - # OTHER working config out with a copy of this one - if [ -f "$KCFG_DIR/last-good.config" ] \ - && [ "$(_kcfg_ver "$KCFG_DIR/last-good.config")" != "$ver" ]; then - cp "$KCFG_DIR/last-good.config" "$KCFG_DIR/prev-good.config" 2>/dev/null - chmod 644 "$KCFG_DIR/prev-good.config" 2>/dev/null - fi - cp "$src" "$KCFG_DIR/last-good.config" 2>/dev/null || return 0 - chmod 644 "$KCFG_DIR/last-good.config" - echo "# the config of $ver is now $KCFG_DIR/last-good.config" -} - -# THE CONFIG THIS MACHINE IS RUNNING ON, wherever it can be read from. -_kcfg_current() { - local c - for c in "$tree/.config" "$KCFG_DIR/last-good.config" "$(_kcfg_newest)"; do - [ -n "$c" ] && _kcfg_readable "$c" && { printf '%s' "$c"; return 0; } - done - if [ -r /proc/config.gz ]; then - c="$(mktemp)"; gunzip < /proc/config.gz > "$c" 2>/dev/null \ - && { printf '%s' "$c"; return 0; } - rm -f "$c" - fi - return 1 -} - -_kcfg_state() { # <config> <SYMBOL> -- what that symbol is, in words - local f="$1" sym="CONFIG_${2#CONFIG_}" l - l="$(grep -E "^($sym=|# $sym is not set)" "$f" 2>/dev/null | head -n1)" - [ -n "$l" ] || { printf 'absent'; return; } - case "$l" in "# $sym is not set") printf 'not set' ;; *) printf '%s' "${l#*=}" ;; esac -} - -# ONE WRITER FOR kernel.conf. `config set`, `config from` and `enable` all -# change one KEY="value" line in a file the user edits by hand, so they all -# back it up the same way and none of them rewrites anything else. -_conf_set_var() { # <KEY> <value> - local key="$1" val="$2" bak tmp - bak="$CONF.bak-$(date +%Y%m%d-%H%M%S)" - cp -p "$CONF" "$bak" || { echo "!! could not back up $CONF"; return 1; } - if grep -qE "^[[:space:]]*$key=" "$CONF"; then - tmp="$(mktemp)" - awk -v v="$key=\"$val\"" -v k="^[[:space:]]*$key=" ' - $0 ~ k && !done { print v; done = 1; next } - { print }' "$CONF" > "$tmp" && cat "$tmp" > "$CONF" - rm -f "$tmp" - else - printf '\n%s="%s"\n' "$key" "$val" >> "$CONF" - fi - echo "# $CONF (backup: $(basename "$bak"))" - echo "# $key=\"$val\"" -} - -# WHAT A SYMBOL IS, ACCORDING TO THE TREE: bool, tristate, or unknown -- -# which decides whether `enable` may say =m. -_ksym_type() { # <SYMBOL> - local sym="${1#CONFIG_}" - [ -d "$tree" ] || return 0 - grep -rhA3 --include='Kconfig*' -E "^[[:space:]]*(menu)?config[[:space:]]+$sym\$" \ - "$tree" 2>/dev/null \ - | grep -m1 -oE '^[[:space:]]*(bool|tristate|string|int|hex)' \ - | tr -d '[:space:]' -} - -# THE SYMBOLS A KCONFIG FILE DEFINES, with their prompt -- the only way a -# search can say "ATH9K_HTC: Atheros HTC based wireless cards" instead of -# printing every symbol in a file that happens to mention the word. -_ksym_scan() { # <pattern> -- SYMBOL<TAB>type<TAB>prompt, for matches only - local pat="$1" - [ -d "$tree" ] || return 0 - grep -rlis --include='Kconfig*' -- "$pat" "$tree" 2>/dev/null | head -n 40 \ - | xargs -r awk -v pat="$pat" ' - function low(x) { return tolower(x) } - /^[[:space:]]*(menu)?config[[:space:]]+[A-Za-z0-9_]+/ { - sym = $2; next } - sym != "" && /^[[:space:]]*(bool|tristate|string|int|hex)([[:space:]]|$)/ { - type = $1 - p = $0 - # the prompt is the FIRST quoted string on the line, if any: - # `tristate "Atheros HTC based wireless cards support"` - if (match(p, /"[^"]*"/)) - p = substr(p, RSTART + 1, RLENGTH - 2) - else - p = "" - if (index(low(sym), low(pat)) || index(low(p), low(pat))) - print sym "\t" type "\t" p - sym = ""; next } - { if ($0 ~ /^[[:space:]]*$/) sym = "" }' 2>/dev/null \ - | sort -u -} - -# THE MODULES THIS KERNEL HAS: which version's /lib/modules to read. -_kmod_ver() { - local r; r="$(uname -r)" - [ -d "/lib/modules/$r" ] && { printf '%s' "$r"; return 0; } - ls -1 /lib/modules 2>/dev/null | sort -V | tail -n1 -} - -# THE NEWEST SOURCE TREE, for a `config` command that names no version. -_kcfg_tree() { - [ -d "${SRC:-}" ] || return 0 - ls -1d "$SRC"/linux-*/ 2>/dev/null | grep -v '\-headers' | sort -V | tail -n1 \ - | sed 's:/$::' -} - -if [ "${_CFGCMD:-0}" = 1 ]; then - [ -d "$tree" ] || tree="$(_kcfg_tree)" - _sub="${_CFGARGS[0]-show}" - case "$_sub" in - show|"") - echo "# the next build's .config" - echo "# CONFIG_FROM = $CONFIG_FROM" - echo "# CONFIG_REFRESH = ${CONFIG_REFRESH:-olddefconfig}" - echo "# CONFIG_SET = ${CONFIG_SET:-(nothing)}" - echo "# kept in $KCFG_DIR (CONFIG_KEEP=$CONFIG_KEEP builds)" - echo - for _f in last-good prev-good; do - if [ -f "$KCFG_DIR/$_f.config" ]; then - printf '# %-16s %s (%s)\n' "$_f.config" \ - "$(_kcfg_ver "$KCFG_DIR/$_f.config")" \ - "$(date -r "$KCFG_DIR/$_f.config" '+%Y-%m-%d %H:%M')" - else - printf '# %-16s -- none yet\n' "$_f.config" - fi - done - echo - echo "# add an option: lfs-kernel config set NET_VENDOR_ATHEROS=y" - echo "# find its name: lfs-kernel config search atheros" - echo "# then build: lfs-kernel" - exit 0 ;; - list|ls) - if [ ! -d "$KCFG_DIR" ]; then - echo "# no saved configs yet ($KCFG_DIR) -- the next build writes one" - exit 0 - fi - echo "# $KCFG_DIR" - for _f in "$KCFG_DIR"/last-good.config "$KCFG_DIR"/prev-good.config; do - [ -f "$_f" ] && printf ' %-34s %-10s %s\n' "$(basename "$_f")" \ - "$(_kcfg_ver "$_f")" "$(date -r "$_f" '+%Y-%m-%d %H:%M')" - done - ls -1t "$KCFG_DIR"/config-* 2>/dev/null | while read -r _f; do - printf ' %-34s %-10s %s\n' "$(basename "$_f")" \ - "$(_kcfg_ver "$_f")" "$(date -r "$_f" '+%Y-%m-%d %H:%M')" - done - echo "# start from one: CONFIG_FROM=file:<path> in $CONF" - echo "# or CONFIG_FROM=last-good / prev-good / saved" - exit 0 ;; - diff) - _a="${_CFGARGS[1]:-$KCFG_DIR/prev-good.config}" - _b="${_CFGARGS[2]:-$KCFG_DIR/last-good.config}" - [ -f "$_a" ] || _a="$KCFG_DIR/$_a" - [ -f "$_b" ] || _b="$KCFG_DIR/$_b" - for _f in "$_a" "$_b"; do - [ -f "$_f" ] || { echo "!! no such config: $_f"; echo " lfs-kernel config list"; exit 1; } - done - echo "# $_a ($(_kcfg_ver "$_a")) -> $_b ($(_kcfg_ver "$_b"))" - echo "# < only in the first, > only in the second" - diff <(grep -E '^(CONFIG_|# CONFIG_)' "$_a" | sort) \ - <(grep -E '^(CONFIG_|# CONFIG_)' "$_b" | sort) \ - | grep -E '^[<>]' | sort -k2 || echo "# identical" - exit 0 ;; - search|find) - _pat="${_CFGARGS[1]:-}" - [ -n "$_pat" ] || { echo "!! lfs-kernel config search <text>"; exit 1; } - _cfg="$(_kcfg_current || true)" - if [ -n "$_cfg" ]; then - echo "# '$_pat' in $_cfg" - grep -iE "^(CONFIG_[A-Z0-9_]*${_pat}[A-Z0-9_]*=|# CONFIG_[A-Z0-9_]*${_pat}[A-Z0-9_]* is not set)" \ - "$_cfg" | head -n 25 | sed 's/^/ /' - fi - echo - if [ -d "$tree" ]; then - # BY NAME OR BY PROMPT, NOT BY FILE. Listing every symbol of - # every Kconfig that mentions the word gave 40 lines of MIPS - # console options for "atheros". A symbol is a hit when its own - # name or its prompt says so, and the prompt is what tells you - # which one you want. - echo "# symbols in $tree (state, type, what it is)" - _hits="$(_ksym_scan "$_pat")" - if [ -n "$_hits" ]; then - printf '%s\n' "$_hits" | head -n 30 | while IFS="$(printf '\t')" read -r _sym _ty _pr; do - printf ' %-26s %-9s %-9s %s\n' "$_sym" \ - "$( [ -n "$_cfg" ] && _kcfg_state "$_cfg" "$_sym" )" \ - "$_ty" "$_pr" - done - _n="$(printf '%s\n' "$_hits" | wc -l)" - [ "$_n" -gt 30 ] && echo " ... $((_n - 30)) more" - else - echo " (nothing whose name or prompt says '$_pat')" - fi - else - echo "# no kernel tree here to search; unpack one with: lfs-kernel <version>" - fi - echo - echo "# a NET_VENDOR_* / WLAN_VENDOR_* symbol only opens the submenu:" - echo "# the DRIVER under it has to be set too." - echo "# known hardware, by name: lfs-kernel enable --list" - echo "# set a symbol: lfs-kernel config set ATH9K_HTC=m" - [ -d "$tree" ] \ - && echo "# read the help text: cd $tree && make menuconfig (/ searches)" - exit 0 ;; - save) - # SEED THE ARCHIVE WITHOUT A BUILD. Before the first build there is - # nothing saved, so CONFIG_FROM=last-good has nothing to read -- and - # the config worth keeping is already on this machine. - _src="${_CFGARGS[1]:-}" - _good=0 - for _a in ${_CFGARGS[@]+"${_CFGARGS[@]}"}; do - [ "$_a" = "--good" ] && _good=1 - done - [ "$_src" = "--good" ] && _src="" - if [ -z "$_src" ]; then - if [ -r /proc/config.gz ]; then - _src="$(mktemp)"; gunzip < /proc/config.gz > "$_src" - _what="the running kernel" - elif _kcfg_readable "$tree/.config"; then - _src="$tree/.config"; _what="$tree" - else - echo "!! nothing to save: no /proc/config.gz and no .config in a tree" - exit 1 - fi - else - _what="$_src" - fi - _kcfg_readable "$_src" || { echo "!! $_src is not a kernel config"; exit 1; } - _ver="$(_kcfg_ver "$_src")" - [ -n "$_ver" ] || _ver="$(uname -r | sed 's/-.*//')" - echo "# saving $_what (kernel $_ver)" - _kcfg_archive "$_src" "$_ver" - if [ "$_good" = 1 ]; then - _kcfg_mark_good "$_src" "$_ver" - else - echo "# it is a build snapshot, not a known-good config." - echo "# --good marks it as one (use it for a kernel that BOOTS)." - fi - echo "# start from it next time: lfs-kernel config from last-good" - exit 0 ;; - from) - _val="${_CFGARGS[1]:-}" - case "$_val" in - running|last-good|good|prev-good|prev|saved|last|keep|file:*) ;; - "") echo "# CONFIG_FROM = $CONFIG_FROM" - echo "# running | last-good | prev-good | saved | keep | file:<path>" - exit 0 ;; - *) echo "!! CONFIG_FROM=$_val is not one of:" - echo " running last-good prev-good saved keep file:<path>"; exit 1 ;; - esac - case "$_val" in - last-good|good) _kcfg_readable "$KCFG_DIR/last-good.config" \ - || { echo "!! there is no $KCFG_DIR/last-good.config yet." - echo " Make one from the kernel running now:" - echo " lfs-kernel config save --good"; exit 1; } ;; - esac - _conf_set_var CONFIG_FROM "$_val" || exit 1 - exit 0 ;; - set|unset) - _items=("${_CFGARGS[@]:1}") - [ "${#_items[@]}" -gt 0 ] || { - echo "!! lfs-kernel config $_sub SYMBOL[=VALUE] ..."; exit 1; } - # the current list, as words - _new="" - for _w in ${CONFIG_SET:-}; do - _keep=1 - for _i in "${_items[@]}"; do - _isym="CONFIG_${_i%%=*}"; _isym="CONFIG_${_isym#CONFIG_}" - _wsym="CONFIG_${_w%%=*}"; _wsym="CONFIG_${_wsym#CONFIG_}" - [ "$_isym" = "$_wsym" ] && _keep=0 - done - [ "$_keep" = 1 ] && _new="$_new $_w" - done - if [ "$_sub" = set ]; then - for _i in "${_items[@]}"; do - case "$_i" in - *=*) ;; - *) echo "!! '$_i' has no value -- SYMBOL=y|m|n|<number>|\"text\""; exit 1 ;; - esac - _val="${_i#*=}" - case "$_val" in - y|m|n|[0-9]*|\"*) ;; - *) echo "# note: '$_val' is taken as a string value" ;; - esac - _new="$_new ${_i#CONFIG_}" - done - fi - _new="$(printf '%s' "$_new" | sed 's/^ *//')" - _conf_set_var CONFIG_SET "$_new" || exit 1 - echo - echo "# nothing is built yet. Apply it:" - echo "# lfs-kernel # rebuild and install" - echo "# Each entry is read back from the final .config: one the" - echo "# kernel refuses (missing dependency, no such symbol) is" - echo "# named and stops the build." - exit 0 ;; - *) echo "!! lfs-kernel config: show | list | diff | search | set | unset" - echo " | save [--good] | from <source>" - exit 1 ;; - esac -fi - -# -------------------------------------------------------------------------- -# HARDWARE BY NAME. -# -# "I have an AR9271 USB adapter" is the question; CONFIG_ATH9K_HTC plus -# WLAN_VENDOR_ATH plus the htc_9271 firmware is the answer, and nothing on -# the machine connects the two. menuconfig will not either: it wants the -# symbol you do not know yet. -# -# Each line is: <name>|<symbols>|<firmware>|<what it is> -# `enable <name>` puts the symbols into CONFIG_SET and the firmware into -# FIRMWARE_ONLY, both in kernel.conf, both backed up. A symbol this kernel -# refuses is reported by the build, which reads every CONFIG_SET entry back. -_KDEV_TABLE=' -ath9k-htc|WLAN_VENDOR_ATH=y ATH9K_HTC=m|ath9k_htc htc_9271*|Atheros AR9271/AR7010 USB wifi (TP-Link TL-WN722N v1, Alfa AWUS036NHA) -ath9k|WLAN_VENDOR_ATH=y ATH9K=m ATH9K_PCI=y|ath9k_htc|Atheros AR9xxx PCI/PCIe wifi -ath10k|WLAN_VENDOR_ATH=y ATH10K=m ATH10K_PCI=m|ath10k|Qualcomm Atheros QCA988x PCI wifi -ath11k|WLAN_VENDOR_ATH=y ATH11K=m ATH11K_PCI=m|ath11k|Qualcomm QCA6390/WCN685x wifi -alx|NET_VENDOR_ATHEROS=y ALX=m|-|Atheros AR816x/AR817x, Killer E2200 ethernet -atl1c|NET_VENDOR_ATHEROS=y ATL1C=m|-|Atheros AR813x/AR815x ethernet -iwlwifi|WLAN_VENDOR_INTEL=y IWLWIFI=m IWLMVM=m|intel iwlwifi-*.ucode|Intel wifi (AX200 and friends) -rtl8xxxu|WLAN_VENDOR_REALTEK=y RTL8XXXU=m|rtlwifi rtw88|Realtek RTL8188/8192 USB wifi -bluetooth-usb|BT=m BT_HCIBTUSB=m BT_RFCOMM=m BT_BNEP=m|rtl_bt intel|USB Bluetooth (most laptop and dongle radios) -btrfs|BTRFS_FS=m BTRFS_FS_POSIX_ACL=y|-|the btrfs filesystem (stack: btrfs-progs) -exfat|EXFAT_FS=m|-|exFAT, the usual format of big SD cards -ntfs3|NTFS3_FS=m NTFS3_LZX_XPRESS=y|-|NTFS read/write (the in-kernel driver) -nvme|BLK_DEV_NVME=y|-|NVMe SSDs -usb-serial|USB_SERIAL=m USB_SERIAL_CP210X=m USB_SERIAL_FTDI_SIO=m USB_SERIAL_CH341=m|-|USB serial adapters (CP210x, FTDI, CH340) -v4l2-usb|MEDIA_SUPPORT=m MEDIA_USB_SUPPORT=y USB_VIDEO_CLASS=m|-|USB webcams (UVC) -' - -_kdev_line() { # <name> -- the table line, or empty - printf '%s\n' "$_KDEV_TABLE" | awk -F'|' -v n="$1" 'NF && $1 == n {print; exit}' -} - -_kdev_list() { - echo "# hardware this tool knows by name:" - printf '%s\n' "$_KDEV_TABLE" | awk -F'|' 'NF { printf " %-15s %s\n", $1, $4 }' - echo - echo "# enable one: lfs-kernel enable ath9k-htc" - echo "# or a symbol: lfs-kernel enable ATH9K_HTC" - echo "# what is set: lfs-kernel config show modules: lfs-kernel modules" -} - -if [ "${_ENABLE:-0}" = 1 ]; then - [ -d "$tree" ] || tree="$(_kcfg_tree)" - _args=("${_ENARGS[@]}") - if [ "${#_args[@]}" = 0 ] || [ "${_args[0]}" = "--list" ] || [ "${_args[0]}" = "list" ]; then - _kdev_list; exit 0 - fi - _syms=""; _fw=""; _unknown="" - for _a in "${_args[@]}"; do - _row="$(_kdev_line "$_a")" - if [ -n "$_row" ]; then - _syms="$_syms $(printf '%s' "$_row" | cut -d'|' -f2)" - _f="$(printf '%s' "$_row" | cut -d'|' -f3)" - [ "$_f" = "-" ] || _fw="$_fw $_f" - echo "# $_a: $(printf '%s' "$_row" | cut -d'|' -f4)" - continue - fi - # not a name in the table: take it as a symbol, and ask the tree - # whether it can be a module before saying =m - case "$_a" in - *=*) _syms="$_syms ${_a#CONFIG_}"; continue ;; - esac - _sym="$(printf '%s' "${_a#CONFIG_}" | tr '[:lower:]' '[:upper:]')" - case "$(_ksym_type "$_sym")" in - tristate) _syms="$_syms $_sym=m" - echo "# $_sym: a tristate symbol -- set to m (a module)" ;; - bool) _syms="$_syms $_sym=y" - echo "# $_sym: a bool symbol -- set to y (built in)" ;; - "") _unknown="$_unknown $_a" ;; - *) _syms="$_syms $_sym=y" ;; - esac - done - if [ -n "$_unknown" ]; then - echo "!! not a name I know, and not a symbol in this tree:$_unknown" - echo " lfs-kernel enable --list # the names" - echo " lfs-kernel config search <text> # the symbols" - echo " lfs-kernel enable SYM=m # if you are sure" - exit 1 - fi - # merge into CONFIG_SET, last value wins, order kept - _new="" - for _w in ${CONFIG_SET:-}; do - _keep=1 - for _i in $_syms; do - [ "CONFIG_${_i%%=*}" = "CONFIG_${_w%%=*}" ] && _keep=0 - done - [ "$_keep" = 1 ] && _new="$_new $_w" - done - for _i in $_syms; do _new="$_new $_i"; done - _new="$(printf '%s' "$_new" | sed 's/^ *//')" - _conf_set_var CONFIG_SET "$_new" || exit 1 - # the firmware, if this hardware needs some and it is not already named - if [ -n "$_fw" ] && [ "${FIRMWARE_FROM:-none}" != none ]; then - _fwnew="${FIRMWARE_ONLY:-}" - if [ -n "$_fwnew" ]; then # empty means "everything": leave it - for _f in $_fw; do - case " $_fwnew " in *" $_f "*) ;; *) _fwnew="$_fwnew $_f" ;; esac - done - [ "$_fwnew" = "${FIRMWARE_ONLY:-}" ] \ - || _conf_set_var FIRMWARE_ONLY "$_fwnew" - fi - fi - echo - echo "# nothing is built yet. Apply it:" - echo "# lfs-kernel # rebuild, install, and the firmware" - echo "# then, after the reboot:" - echo "# lfs-kernel modules ${_args[0]}" - echo "# lfs-kernel firmware --missing" - exit 0 -fi - -if [ "${_MODCMD:-0}" = 1 ]; then - _mver="$(_kmod_ver)" - [ -n "$_mver" ] || { echo "!! no /lib/modules -- no kernel is installed here"; exit 1; } - _md="/lib/modules/$_mver" - _pat="${_MODARGS[0]-}" - _loaded="$(cut -d' ' -f1 /proc/modules 2>/dev/null | tr '_' '-' | sort -u)" - _all="$(find "$_md" -name '*.ko' -o -name '*.ko.*' 2>/dev/null \ - | sed 's:.*/::; s:\.ko.*::' | tr '_' '-' | sort -u)" - _builtin="$(sed 's:.*/::; s:\.ko.*::' "$_md/modules.builtin" 2>/dev/null \ - | tr '_' '-' | sort -u)" - _nall="$(printf '%s\n' "$_all" | grep -c . || true)" - _nld="$(printf '%s\n' "$_loaded" | grep -c . || true)" - _nbi="$(printf '%s\n' "$_builtin" | grep -c . || true)" - echo "# kernel $_mver: $_nall module(s) built, $_nld loaded, $_nbi built in" - [ "$_mver" = "$(uname -r)" ] \ - || echo "# (the running kernel is $(uname -r) -- this is another one)" - if [ -z "$_pat" ]; then - echo "# loaded now:" - printf '%s\n' "$_loaded" | grep . | tr '\n' ' ' | fold -s -w 72 | sed 's/^/ /' - echo - echo "# one subject: lfs-kernel modules ath" - echo "# not built yet: lfs-kernel config search ath9k" - exit 0 - fi - _hit="$(printf '%s\n' "$_all" "$_builtin" | sort -u \ - | grep -i -- "$(printf '%s' "$_pat" | tr '_' '-')" || true)" - if [ -z "$_hit" ]; then - echo "# no module of this kernel is called anything like '$_pat'." - echo "# is it even configured? lfs-kernel config search $_pat" - echo "# what it would be called: lfs-kernel enable --list" - exit 1 - fi - echo "# modules matching '$_pat'" - printf '%s\n' "$_hit" | while read -r _m; do - _st="built" - printf '%s\n' "$_builtin" | grep -qx -- "$_m" && _st="built-in" - printf '%s\n' "$_loaded" | grep -qx -- "$_m" && _st="LOADED" - _desc="$(modinfo -F description "$(printf '%s' "$_m" | tr '-' '_')" \ - 2>/dev/null | head -n1)" - printf ' %-24s %-9s %s\n' "$_m" "$_st" "${_desc:-}" - done - echo - echo "# load one now: lfs-kernel load <name>" - echo "# and at boot: lfs-kernel load <name> --boot" - echo "# (a USB device loads its driver when you plug it in, if udev runs)" - exit 0 -fi - -# -------------------------------------------------------------------------- -# WHERE ARE WE? -# -# A kernel build is long enough to be interrupted -- Ctrl-C, a reboot, a -# session that timed out -- and afterwards nothing on the machine says how -# far it got. The tree holds a .config that may or may not carry your -# answers, object files that may or may not be a finished image, and -# /lib/modules and the ESP each hold a kernel from some earlier day. -# -# `lfs-kernel status` reads all four and says, in order: what is running, -# what the tree has, what is installed, and what to do next. It builds -# nothing, needs no network, and does not care whether the ESP is mounted. -# -------------------------------------------------------------------------- -if [ "${_STATUS:-0}" = 1 ]; then - [ -d "$tree" ] || tree="$(_kcfg_tree)" - _ver="$(basename "${tree:-}" 2>/dev/null | sed 's/^linux-//')" - _run="$(uname -r)"; _runv="$(echo "$_run" | sed 's/-.*//')" - echo "### lfs-kernel status" - echo - echo "# running now" - echo " kernel $_run" - if [ -s /etc/pkgusr/kernel.last-good ]; then - echo " known to boot $(cat /etc/pkgusr/kernel.last-good)" - else - echo " known to boot -- nothing recorded yet" - fi - if [ -d "/lib/modules/$_run" ]; then - echo " its modules /lib/modules/$_run" - else - echo " its modules MISSING (/lib/modules/$_run is not there)" - fi - echo - echo "# the config" - echo " CONFIG_FROM $CONFIG_FROM" - for _f in last-good prev-good; do - if [ -f "$KCFG_DIR/$_f.config" ]; then - echo " $_f $(_kcfg_ver "$KCFG_DIR/$_f.config") ($(date -r "$KCFG_DIR/$_f.config" '+%Y-%m-%d %H:%M'))" - else - echo " $_f -- none" - fi - done - # DOES THE TREE'S CONFIG CARRY WHAT YOU ASKED FOR? This is the question - # after an interrupted run: the answers are applied BEFORE the compile, - # so they are in the file even when no kernel came out of it. - if [ -n "${CONFIG_SET:-}" ] && [ -f "${tree:-}/.config" ]; then - _miss="" - for _cs in $CONFIG_SET; do - _sym="CONFIG_${_cs%%=*}"; _sym="CONFIG_${_sym#CONFIG_}"; _val="${_cs#*=}" - case "$_val" in - n) grep -qE "^$_sym=(y|m)" "$tree/.config" && _miss="$_miss $_cs" ;; - *) grep -qx "$_sym=$_val" "$tree/.config" || _miss="$_miss $_cs" ;; - esac - done - if [ -z "$_miss" ]; then - echo " CONFIG_SET all $(echo $CONFIG_SET | wc -w) in $tree/.config" - else - echo " CONFIG_SET NOT in the tree's .config:$_miss" - echo " (the next build applies them)" - fi - fi - echo - echo "# the tree" - if [ -z "${tree:-}" ] || [ ! -d "$tree" ]; then - echo " none in $SRC -- nothing has been unpacked here" - else - echo " $tree" - _bz="$tree/arch/x86/boot/bzImage" - if [ ! -f "$tree/.config" ]; then - echo " config none yet" - elif [ -f "$_bz" ] && [ "$_bz" -nt "$tree/.config" ]; then - echo " build FINISHED -- bzImage is newer than .config" - echo " $(date -r "$_bz" '+%Y-%m-%d %H:%M')" - elif [ -f "$tree/vmlinux" ] || find "$tree" -maxdepth 2 -name '*.o' -print -quit 2>/dev/null | grep -q .; then - echo " build UNFINISHED -- object files, no current image" - echo " (an interrupted make; lfs-kernel continues" - echo " from here, it does not start over)" - else - echo " build not started" - fi - _nko="$(find "$tree" -name '*.ko' 2>/dev/null | wc -l)" - echo " modules built $_nko in the tree" - fi - echo - echo "# installed" - if [ -n "$_ver" ] && [ -d "/lib/modules/$_ver" ]; then - echo " /lib/modules/$_ver $(test -s "/lib/modules/$_ver/modules.dep" && echo "(depmod has run)" || echo "-- NO modules.dep")" - echo " $(find "/lib/modules/$_ver" -name '*.ko*' 2>/dev/null | wc -l) module(s)" - elif [ -n "$_ver" ]; then - echo " /lib/modules/$_ver -- not there: this kernel was never installed" - fi - if [ -d "${ESP:-}" ]; then - if [ -f "$ESP/$KERNEL_NAME" ]; then - echo " $ESP/$KERNEL_NAME $(date -r "$ESP/$KERNEL_NAME" '+%Y-%m-%d %H:%M')" - fi - for _k in "$ESP/$KERNEL_NAME"-*; do - [ -f "$_k" ] && echo " $(basename "$_k")" - done - if [ -n "$_ver" ] && [ -f "$ESP/config-$_ver" ] && [ -f "$tree/.config" ]; then - if cmp -s "$ESP/config-$_ver" "$tree/.config"; then - echo " the installed kernel was built from the tree's CURRENT config" - else - echo " the installed config DIFFERS from the tree's .config --" - echo " what is in the tree has not been built and installed yet" - fi - fi - else - echo " $ESP is not mounted -- cannot say what is on the EFI partition" - fi - echo - echo "# next" - if [ -n "$_ver" ] && [ -f "$tree/.config" ] \ - && { [ ! -f "$ESP/config-$_ver" ] || ! cmp -s "$ESP/config-$_ver" "$tree/.config"; }; then - echo " lfs-kernel # build and install what the tree now has" - echo " an interrupted build continues; nothing is thrown away." - else - echo " nothing owed: the tree, /lib/modules and the ESP agree." - fi - echo " lfs-kernel modules ath # is a driver there, is it loaded" - echo " lfs-kernel config show # what the next build would use" - exit 0 -fi - -# -------------------------------------------------------------------------- -# LOADING A MODULE, NOW AND AT EVERY BOOT. -# -# A driver built as a module does nothing until something loads it. For a -# USB device udev does that when you plug it in; for everything else, and -# on a machine where udev is not running, it is modprobe -- and then the -# same name in /etc/sysconfig/modules, which is the file LFS's `modules` -# bootscript reads. Two places, one command. -# -------------------------------------------------------------------------- -if [ -n "${_LOADCMD:-}" ]; then - _MODFILE="${MODULES_FILE:-/etc/sysconfig/modules}" - _boot=0; _names=() - for _a in ${_LOADARGS[@]+"${_LOADARGS[@]}"}; do - case "$_a" in - --boot) _boot=1 ;; - -*) echo "!! lfs-kernel $_LOADCMD: unknown option '$_a'"; exit 1 ;; - *) _names+=("$_a") ;; - esac - done - if [ "${#_names[@]}" = 0 ]; then - echo "!! lfs-kernel $_LOADCMD <module> [...] [--boot]" - echo " which modules there are: lfs-kernel modules <text>" - exit 1 - fi - _rc=0 - for _m in "${_names[@]}"; do - _m="$(printf '%s' "$_m" | tr '-' '_')" - if [ "$_LOADCMD" = unload ]; then - if modprobe -r "$_m" 2>&1; then - echo "# $_m: unloaded" - else - echo "!! $_m: could not be unloaded (in use? lsmod | grep $_m)" - _rc=1 - fi - if [ -f "$_MODFILE" ] && grep -qE "^[[:space:]]*$_m([[:space:]]|$)" "$_MODFILE"; then - cp -p "$_MODFILE" "$_MODFILE.bak-$(date +%Y%m%d-%H%M%S)" - _t="$(mktemp)" - grep -vE "^[[:space:]]*$_m([[:space:]]|$)" "$_MODFILE" > "$_t" \ - && cat "$_t" > "$_MODFILE" - rm -f "$_t" - echo "# and taken out of $_MODFILE" - fi - continue - fi - if grep -qE "^${_m}[[:space:]]" /proc/modules 2>/dev/null; then - echo "# $_m: already loaded" - elif modprobe "$_m" 2>&1; then - if grep -qE "^${_m}[[:space:]]" /proc/modules 2>/dev/null; then - echo "# $_m: loaded" - else - # BUILT IN IS NOT A FAILURE. modprobe succeeds silently for - # a symbol that is =y; the module simply never appears in - # /proc/modules because it is part of the image. - echo "# $_m: modprobe returned success but it is not in" - echo "# /proc/modules -- it is probably built INTO the kernel" - echo "# (lfs-kernel modules $_m says which)" - fi - else - echo "!! $_m: modprobe refused it." - echo " is it built? lfs-kernel modules $_m" - echo " is it set? lfs-kernel config search $_m" - _rc=1 - continue - fi - # WHAT THE KERNEL SAID ABOUT IT, which is where a missing firmware - # blob or an unsupported device id turns up. - if command -v dmesg >/dev/null 2>&1; then - _say="$(dmesg 2>/dev/null | tail -n 40 \ - | grep -iE "${_m}|${_m//_/-}|firmware" | tail -n 5)" - [ -n "$_say" ] && printf '%s\n' "$_say" | sed 's/^/ /' - fi - if [ "$_boot" = 1 ]; then - if [ -f "$_MODFILE" ] && grep -qE "^[[:space:]]*$_m([[:space:]]|$)" "$_MODFILE"; then - echo "# $_MODFILE already lists it" - else - [ -f "$_MODFILE" ] \ - && cp -p "$_MODFILE" "$_MODFILE.bak-$(date +%Y%m%d-%H%M%S)" - if [ ! -f "$_MODFILE" ]; then - install -d "$(dirname "$_MODFILE")" - { - echo "# /etc/sysconfig/modules -- one module per line," - echo "# loaded at boot by the 'modules' bootscript." - echo "# Arguments may follow the name." - } > "$_MODFILE" - fi - printf '%s\n' "$_m" >> "$_MODFILE" - echo "# added to $_MODFILE (loaded at every boot)" - fi - else - echo "# at every boot as well: lfs-kernel load $_m --boot" - fi - done - exit $_rc -fi - # -------------------------------------------------------------------------- # the ESP has to be the EFI partition, not a directory of the same name. # Every copy below would SUCCEED on the root filesystem and the firmware @@ -1945,44 +1095,6 @@ fi echo "# kernel $VERSION -> $ESP (tree $tree, $JOBS)" -# ---- the plan, when --run was not given ---------------------------------- -if [ "${_RUN:-0}" != 1 ]; then - echo - echo "### what a build would do (nothing has been built)" - echo " version $VERSION" - if [ -d "$tree" ]; then - _bz="$tree/arch/x86/boot/bzImage" - if [ -f "$tree/.config" ] && [ -f "$_bz" ] && [ "$_bz" -nt "$tree/.config" ]; then - echo " tree $tree (already built -- make would do little)" - elif [ -f "$tree/.config" ]; then - echo " tree $tree (a build was started; make CONTINUES)" - else - echo " tree $tree (unpacked, not configured)" - fi - else - echo " tree $tree -- would be fetched and unpacked" - fi - echo " config from $CONFIG_FROM, refreshed with ${CONFIG_REFRESH:-olddefconfig}" - [ -n "${CONFIG_SET:-}" ] && echo " then set $CONFIG_SET" - echo " installs $ESP/$KERNEL_NAME (previous kept as ${KERNEL_NAME}_old)" - echo " /lib/modules/$VERSION, and the config beside the image" - case "${FIRMWARE_FROM:-none}" in - none) echo " firmware not touched (FIRMWARE_FROM=none)" ;; - *) echo " firmware ${FIRMWARE_FROM}: ${FIRMWARE_ONLY:-everything}" ;; - esac - [ "${HEADERS:-no}" = yes ] \ - && echo " headers /usr/include, as ${HEADERS_PKG:-linux-headers}" - case "${V4L2LOOPBACK:-no}" in yes|1|true) - echo " also v4l2loopback, built against this tree" ;; - esac - echo - echo " This is an hour of compiling on most machines, and it REPLACES" - echo " what $ESP/$KERNEL_NAME boots." - echo " Go ahead: lfs-kernel${_CLI_VERSION:+ $_CLI_VERSION} --run" - echo " Where we are: lfs-kernel status" - exit 0 -fi - # -------------------------------------------------------------------------- # THE COMMANDS. Everything above chooses the version and checks the paths; # everything below is the build, and is meant to be read. @@ -2010,43 +1122,13 @@ if [ -n "${BOOTLOGO:-}" ]; then fi # .config -# running /proc/config.gz -- the kernel carrying the machine -# last-good the config of the last kernel that BOOTED ($KCFG_DIR) -# prev-good the one before it -- the way back from a bad answer -# saved the newest config this tool built with, booted or not -# file:<path> any file; keep the tree's own .config -_cfg_src="" case "$CONFIG_FROM" in - running) - if [ -r /proc/config.gz ]; then - _as_pkg "gunzip < /proc/config.gz > .config" || exit 1 - elif _kcfg_readable "$KCFG_DIR/last-good.config"; then - # NOT A FAILURE WORTH STOPPING FOR. A kernel built without - # CONFIG_IKCONFIG_PROC has no /proc/config.gz, and the config of - # the last kernel that booted is the same answer from a file. - echo "# /proc/config.gz is not readable (no CONFIG_IKCONFIG_PROC?)" - _cfg_src="$KCFG_DIR/last-good.config" - else - echo "!! /proc/config.gz unreadable, and no saved config either." - echo " Set CONFIG_FROM=file:<path> in $CONF, or see:" - echo " lfs-kernel config list" - exit 1 - fi ;; - last-good|good) _cfg_src="$KCFG_DIR/last-good.config" ;; - prev-good|prev) _cfg_src="$KCFG_DIR/prev-good.config" ;; - saved|last) _cfg_src="$(_kcfg_newest)" ;; + running) _as_pkg "gunzip < /proc/config.gz > .config" || { + echo "!! /proc/config.gz unreadable -- set CONFIG_FROM=file:<path>"; exit 1; } ;; file:*) _as_pkg "cp -v '${CONFIG_FROM#file:}' .config" || exit 1 ;; keep) [ -f "$tree/.config" ] || { echo "!! no .config in $tree"; exit 1; } ;; *) echo "!! CONFIG_FROM=$CONFIG_FROM not understood"; exit 1 ;; esac -if [ -n "$_cfg_src" ]; then - _kcfg_readable "$_cfg_src" || { - echo "!! CONFIG_FROM=$CONFIG_FROM, but $_cfg_src is not a config." - echo " lfs-kernel config list # what there is" - exit 1; } - echo "# starting from $_cfg_src (kernel $(_kcfg_ver "$_cfg_src"))" - _as_pkg "cp '$_cfg_src' .config" || exit 1 -fi # ---- refreshing the config for a NEW kernel ------------------------------ # # A config from one kernel is never complete for another: every version adds @@ -2127,11 +1209,6 @@ if [ -n "${CONFIG_SET:-}" ]; then echo "# all $(echo $CONFIG_SET | wc -w) set, read back from .config" fi -# THE CONFIG IS KEPT BEFORE ANYTHING IS COMPILED. It is finished here -- -# source, refresh and CONFIG_SET all applied -- and a build that dies at -# hour two must not take the answers with it. -_kcfg_archive "$tree/.config" "$VERSION" - # what a Wayland compositor needs; a warning, not a refusal for opt in CONFIG_DRM CONFIG_DRM_FBDEV_EMULATION CONFIG_INPUT_EVDEV CONFIG_SYSVIPC; do grep -qE "^$opt=(y|m)" "$tree/.config" || echo "!! $opt is not set (sway needs it)" @@ -2247,8 +1324,7 @@ depmod -a "$VERSION" || exit 1 # perfectly installed. The kernel is on disk at this point; record it. # (Run again after the headers, which add /usr/include.) _record() { - echo "# recording what $acct owns (a scan AS $acct, skipping /proc, the" - echo "# package source trees and the caches; --root if something is missed)" + echo "# recording what $acct owns (scans the filesystem; this is slow) ..." packagemanager reload-pkg-list "$PKG" >/dev/null 2>&1 || return 0 local _h; _h="$(getent passwd "$acct" | cut -d: -f6)" echo "# recorded $(grep -c . "$_h/pkg.lst" 2>/dev/null) file(s) for $acct" @@ -2358,9 +1434,8 @@ esac # A kernel that boots is the only way back from one that does not, and its # TREE is what you need to rebuild or bisect -- not just the bzImage. The # version that is running right now is recorded as last-good before this one -# is; nothing here ever removes that tree. (An older comment here promised -# a `lfs-kernel prune`; there is none, and before 1.14.324 typing it started -# a full build instead of saying so.) +# is; nothing here ever removes that tree, and `lfs-kernel prune` (below) +# refuses to. # ONLY A KERNEL THAT HAS BOOTED IS KNOWN TO BOOT. # # The old `elif` wrote the version just built into last-good whenever the @@ -2375,12 +1450,9 @@ if [ "$_running" = "$VERSION" ]; then # the kernel just built is the one running: it demonstrably boots printf '%s\n' "$VERSION" > "$_lastgood" echo "# $VERSION is running, so it is now the kernel known to boot" - _kcfg_mark_good "$tree/.config" "$VERSION" elif [ -d "$SRC/linux-$_running" ]; then printf '%s\n' "$_running" > "$_lastgood" echo "# keeping $SRC/linux-$_running as the last kernel known to boot" - # ...and ITS config is the one known to work, not the one just built - _kcfg_mark_good "$SRC/linux-$_running/.config" "$_running" else echo "# the running kernel is $_running, and there is no tree for it here," echo "# so nothing is recorded as known-good. $VERSION will be recorded" @@ -2396,6 +1468,5 @@ echo "# $VERSION installed as $ESP/$KERNEL_NAME" echo "# previous kernel kept as $ESP/${KERNEL_NAME}_old" echo "# its source kept at $SRC/linux-$(cat "$_lastgood" 2>/dev/null)" echo "# owned by $acct (packagemanager verify | grep $PKG)" -echo "# its config kept in $KCFG_DIR (lfs-kernel config list)" echo "# running kernel is $(uname -r) -- reboot to use $VERSION" [ "$_fw_ok" = 1 ] || echo "# NOTE: firmware was NOT installed (see the message above)" diff --git a/lfs-phases b/lfs-phases index f97d577..0d79363 100755 --- a/lfs-phases +++ b/lfs-phases @@ -34,7 +34,7 @@ # scripts are also SOURCED by lfs-helper to read their metadata, and this file # comes along -- it must not shadow anything in the sourcing shell. -PKGUSR_LIB_VERSION="1.14.340" +PKGUSR_LIB_VERSION="1.14.299" _pp_build_id() { local f="${BASH_SOURCE[0]}" @@ -292,41 +292,7 @@ _pp_unpack() { # which reads like a build failure and sends people to the log. The # repository is right here and knows what it does have; show it, and # say where the wrong value came from. - # A BRANCH IS NOT A POINT IN TIME, AND `checkout` DOES NOT MOVE IT. - # - # On an existing clone the fetch above brings the new commits in, and - # `git checkout master` then says "Already on 'master'" and stays - # exactly where it was. So a package pinned to a branch rebuilt the - # SAME commit for ever: `update <pkg>` fetched, compiled and installed - # a version it already had, and said it had updated it. A tag or a - # commit is a point in time and must NOT move -- those are checked out - # detached, which is what they are. - _pp_ref="${git_ref:-}" - if [ -z "$_pp_ref" ]; then - # no ref: whatever the remote calls its default branch - _pp_ref="$(git -C "$pkg_dir" symbolic-ref --short refs/remotes/origin/HEAD 2>/dev/null)" - _pp_ref="${_pp_ref#origin/}" - [ -n "$_pp_ref" ] && _pp_say "# no ref given -- following origin's default branch ($_pp_ref)" - fi - if [ -n "$_pp_ref" ] \ - && git -C "$pkg_dir" show-ref --verify --quiet "refs/remotes/origin/$_pp_ref"; then - _pp_before="$(git -C "$pkg_dir" rev-parse --short HEAD 2>/dev/null)" - if ! git -C "$pkg_dir" checkout -B "$_pp_ref" "origin/$_pp_ref"; then - _pp_fail "could not put $pkg_dir on branch $_pp_ref" - return 1 - fi - _pp_after="$(git -C "$pkg_dir" rev-parse --short HEAD 2>/dev/null)" - if [ -n "$_pp_before" ] && [ "$_pp_before" != "$_pp_after" ]; then - _pp_say "# $_pp_ref: $_pp_before -> $_pp_after ($(git -C "$pkg_dir" rev-list --count "$_pp_before..$_pp_after" 2>/dev/null) new commit(s))" - else - _pp_say "# $_pp_ref is already at $_pp_after" - fi - git -C "$pkg_dir" submodule update --init --recursive - echo "$pkg_dir" > "$_pp_dir_marker" - _pp_say "checked out into $BUILD_ROOT/$pkg_dir" - return 0 - fi - if [ -n "${git_ref:-}" ] && ! git -C "$pkg_dir" checkout --detach "$git_ref"; then + if [ -n "${git_ref:-}" ] && ! git -C "$pkg_dir" checkout "$git_ref"; then _pp_fail "'$git_ref' is not a tag, branch or commit in $link" echo " The stack entry (or this script) asks for a ref the" >&2 echo " repository does not have. Recent tags:" >&2 @@ -570,66 +536,9 @@ _pp_warn_unbuilt() { return 0 } -# DOCUMENTATION THAT WAS NEVER BUILT IS NOT A FAILED INSTALL. -# -# parted's page ends with the book's doc block: -# install -v -m644 doc/{FAT,API,parted.{html,txt,pdf,ps,dvi}} \ -# /usr/share/doc/parted-3.7 -# parted.pdf, parted.ps, parted.dvi and API are built only when texlive is -# installed, and texlive is in avoid.stack. So `install` stopped on -# install: cannot stat 'doc/parted.pdf': No such file or directory -# after the package itself -- 97 files, every binary and library -- was -# already on disk. A missing PDF is not a failed install. -# -# Only for a destination under share/doc, only after the command has been -# tried EXACTLY as the book wrote it, and only for sources that are not -# there: everything else is passed through to /usr/bin/install untouched. -# The function is defined inside this phase, which runs in a child process -# of its own, so nothing outside it ever sees an `install` that is not the -# real one. -_pp_install_docs() { - local last="" a; for a in "$@"; do last="$a"; done - case "$last" in - */share/doc/*|*/share/doc|*/share/gtk-doc/*) ;; - *) command install "$@"; return $? ;; - esac - command install "$@" 2>/dev/null && return 0 - local rc=$? - local keep=() prev="" i=0 n=$# missing="" have=0 - for a in "$@"; do - i=$((i + 1)) - if [ "$i" -eq "$n" ]; then keep+=("$a"); continue; fi - case "$prev" in - -m|-o|-g|-t|--mode|--owner|--group|--target-directory) - keep+=("$a"); prev=""; continue ;; - esac - case "$a" in -*) keep+=("$a"); prev="$a"; continue ;; esac - prev="" - if [ -e "$a" ] || [ -L "$a" ]; then - keep+=("$a"); have=$((have + 1)) - else - missing="$missing $a" - fi - done - # nothing missing -> the failure is a real one, and it is the book's - # command that must be reported, not ours - if [ -z "$missing" ]; then - command install "$@" - return $? - fi - if [ "$have" -gt 0 ]; then - command install "${keep[@]}" || { command install "$@"; return $?; } - fi - _pp_note "not installed, because this build did not produce them:$missing" - _pp_note " they are documentation (the book's doc block builds them" - _pp_note " only with texlive and friends); the package itself is in." - return 0 -} - _pp_phase_install() { _pp_enter_build || return 1 _pp_warn_unbuilt - install() { _pp_install_docs "$@"; } install_pkg } diff --git a/packagemanager b/packagemanager index 8a75981..4984503 100755 --- a/packagemanager +++ b/packagemanager @@ -138,38 +138,6 @@ def appuser_home(): APPUSER_HOME_DEFAULT = "/home/shared_user" -# /home/Shared (1.14.315). 1.14.312 put a hub of LINKS in -# /home/shared_user/Shared and argued against /home: a directory there that -# is not a home looks like one. Once the hub holds the DATA, that argument -# turns round: the shared directories are the one thing in /home/shared_user -# that is not private to an account, and taking them out is what lets every -# account home be 0700. -SHARED_HUB_DEFAULT = "/home/Shared" - - -def shared_hub(): - """Where every shared directory lives (1.14.312; the data itself since - 1.14.315 -- see SHARED_HUB_DEFAULT). - - It was ~/Shared of the main user. Two things were wrong with that: - - * ONE PERSON'S HOME IS NOT A SYSTEM PATH. A program of your own, a - script, a launcher, an app account -- anything that wants "the - shared dirs" had to know whose home to look in. A fixed path is the - same answer for every account. - * The SHARED_* variables were enumerated from $HOME/Shared by EVERY - login -- and in an app account $HOME/Shared is its own data - directory, so u_claude got SHARED_<FILENAME> for each file in it. - - Default /home/Shared: the real directories, one per account - (/home/Shared/Claude, 2770 u_claude:u_claude). Each account's ~/Shared - and the main user's ~/Shared are links into it, so nothing you type - changes. `shared_hub=` in packagemanager.conf moves it. - """ - return (load_config().get("shared_hub") or "").rstrip("/") \ - or SHARED_HUB_DEFAULT - - def _app_home(u): """An application user's home: what passwd says, or where they go. @@ -188,7 +156,7 @@ TMP_DIR = "/tmp/packagemanager" CONFIG_PATH = os.environ.get("PKGUSR_CONFIG", "/etc/pkgusr/packagemanager.conf") # Version stamps. Bump PLAN_VERSION when the cached dry-run plan structure # changes (so stale plans are ignored); TOOL_VERSION is the program version. -TOOL_VERSION = "1.14.340" +TOOL_VERSION = "1.14.299" def _build_id(): @@ -717,99 +685,15 @@ def find_blfs(): # dependencies, the generated script -- asks the one named there. _BOOK_FOR_PKG = {} -# A STACK CAN READ ITS PACKAGES FROM ANOTHER BOOK (1.14.321). -# -# media.stack: from-book svn -# -# ...and every book entry of THAT file (not of files it includes) is read -# from the development book: versions, dependencies, the generated script. -# It is a property of the package, not of one run -- `update`, `info` and -# `install --reinstall` ask the same book the stack built it from, or an -# update-all would "update" mpv back to the stable book's older page. -# -# Dependencies outside the stack stay on their own book: an installed one is -# left alone (the svn page naming a newer glib is not a reason to rebuild -# glib), a missing one is built from the default book, and only a page the -# default book does not have at all is read from the stack's book. -_STACK_BOOKS = None # anchor (lower) -> selector -_PLAN_BOOK = None # the stack book a running plan's target reads -_BOOK_PATHS = {} # selector -> local file ("" = could not resolve) - - -def _stack_book_map(): - """anchor -> selector, from every installed stack's `from-book`.""" - global _STACK_BOOKS - if _STACK_BOOKS is not None: - return _STACK_BOOKS - _STACK_BOOKS = {} - # a stack run hands its own file's books to the installs it starts - # (it may be a checkout, not the installed copy) - for kv in (os.environ.get("PM_STACK_BOOKS") or "").split(","): - if "=" in kv: - a, sel = kv.split("=", 1) - _STACK_BOOKS[a.strip().lower()] = sel.strip() - d = STACK_DIR if "STACK_DIR" in globals() else "/etc/pkgusr/stacks" - for f in sorted(glob.glob(os.path.join(d, "*.stack"))): - try: - for r in _parse_stack_file(f): - sel = r["opts"].get("from-book") - if sel and r["kind"] == "book": - _STACK_BOOKS.setdefault(r["target"].lower(), sel) - except (ValueError, OSError): - continue - return _STACK_BOOKS - - -def _book_path_for(selector, refresh=False): - """The local file for a book selector, asked of `blfs path` ONCE. - - With refresh, `blfs fetch` first -- a stack run on the svn book wants - today's svn book; every other command reads the cached copy and never - touches the network.""" - if not selector: - return None - if os.path.isfile(selector): - return selector - if refresh or selector not in _BOOK_PATHS: - if refresh: - set_status("refreshing the %s book ..." % selector) - try: - r = subprocess.run(find_blfs() + ["fetch", selector], - capture_output=True, text=True) - finally: - clear_status() - if r.returncode != 0: - printWarning(" could not refresh the %s book; using the " - "cached one" % selector) - r = subprocess.run(find_blfs() + ["path", selector], - capture_output=True, text=True) - out = (r.stdout or "").strip().splitlines() - _BOOK_PATHS[selector] = out[-1] if r.returncode == 0 and out \ - and os.path.isfile(out[-1]) else "" - if not _BOOK_PATHS[selector]: - printWarning(" no %s book on disk (blfs fetch %s)" - % (selector, selector)) - return _BOOK_PATHS[selector] or None - def _book_override(anchor): - """The book this package should be read from, or None. - - machine.conf `book =` first, then a stack's `from-book`, then -- inside - a plan for a stack-book package -- that book for a page the default - book does not have at all.""" + """The book this package should be read from, or None.""" if not anchor: return None if anchor not in _BOOK_FOR_PKG: opts = _machine_opts_for(anchor) - sel = opts.get("book") or _stack_book_map().get(anchor.lower()) or "" - _BOOK_FOR_PKG[anchor] = (_book_path_for(sel) or sel) if sel else "" - if _BOOK_FOR_PKG[anchor]: - return _BOOK_FOR_PKG[anchor] - if _PLAN_BOOK and not _map_lookup(_versions_for(""), [anchor]) \ - and _map_lookup(_versions_for(_PLAN_BOOK), [anchor]): - return _PLAN_BOOK - return None + _BOOK_FOR_PKG[anchor] = opts.get("book") or "" + return _BOOK_FOR_PKG[anchor] or None def _blfs_book_flags(anchor=None): @@ -840,14 +724,9 @@ _BLFS_ACTIVITY = { _FORWARDED_NOTES = set() -_SAID_BOOK = set() - - -def run_blfs(args, book=None): +def run_blfs(args): """Run blfs; return (returncode, stdout, stderr). rc 127 => blfs missing. - book= reads that book file for this call, whatever the package. - Each call is a subprocess, and the first one for a given book INDEXES it, which takes many seconds. Announce what is happening: an install that sits silent for half a minute before its first output looks frozen.""" @@ -855,15 +734,9 @@ def run_blfs(args, book=None): # what decides whether an override applies _anchor = next((a for a in args[1:] if not a.startswith("-")), None) \ if len(args) > 1 else None - if book: - cmd = find_blfs() + ["--book-file", book] + list(args) - else: - cmd = find_blfs() + _blfs_book_flags(_anchor) + list(args) - # ONCE per package: a plan asks blfs several things about each one - if not book and _anchor and _book_override(_anchor) \ - and _anchor not in _SAID_BOOK: - _SAID_BOOK.add(_anchor) - printInfo(" %s: read from %s" + cmd = find_blfs() + _blfs_book_flags(_anchor) + list(args) + if _anchor and _book_override(_anchor): + printInfo(" %s: read from %s (machine.conf)" % (_anchor, os.path.basename(_book_override(_anchor)))) verb = args[0] if args else "" if verb in _BLFS_ACTIVITY: @@ -905,73 +778,64 @@ def run_blfs(args, book=None): pass -_VERSION_MAPS = {} # book file ("" = default) -> {anchor: nv} -_BOOK_MISSES = set() # names no book had: ask the slow way once - - -def _versions_for(book): - """The whole version map of one book, loaded once per process.""" - if book not in _VERSION_MAPS: - try: - _VERSION_MAPS[book] = book_versions_all(book or None) or {} - except Exception: - _VERSION_MAPS[book] = {} - return _VERSION_MAPS[book] - - -def _map_lookup(vmap, names): - for a in names: - if a in vmap: - return vmap[a] - for a in names: - low = a.lower() - for k, nv in vmap.items(): - if k.lower() == low: - return nv - return None +_VERSION_MAP = None # anchor -> name_version, the whole book, once def book_version(anchor): - """Current name-version of a package in ITS book, or None. + """Current name-version of a package in the book, or None if unavailable. + + ONE PROCESS FOR THE WHOLE BOOK, not one per question. This ran `blfs + debug <anchor>` every time it was called -- a subprocess that opens and + reads the book cache to answer one lookup. A stack preview asks it once + per entry (56 for sway) and the planner asks it for every package it + considers, so a command whose real work is a dictionary lookup spent + minutes launching processes: - One version map per book, loaded once (see book_versions_all); a name - no map has falls back to `blfs debug` once, and a miss is remembered -- - `install imv` asked three slow questions to learn the same "no" three - times. The package's book is machine.conf's, else its stack's - from-book, else the default. + looking up pipewire in the book <- one process + looking up ... in the book <- another - THE PREFIX IS NOT PART OF THE BOOK'S NAME: `info webkitgtk` asks with - the account name p_webkitgtk, which no anchor matches. + `book_versions_all` was already here, used only by `update`. The map is + loaded on first use and answers everything after that; a name the map does + not have (an alias, a URL, a fragment) still falls back to the old call. """ + global _VERSION_MAP + if _VERSION_MAP is None: + try: + _VERSION_MAP = book_versions_all() or {} + except Exception: + _VERSION_MAP = {} + # THE PREFIX IS NOT PART OF THE BOOK'S NAME. + # + # `info webkitgtk` asks with the ACCOUNT name, `p_webkitgtk`, which no + # book anchor matches -- so every call missed the map and fell through to + # `blfs debug p_webkitgtk`, a subprocess that reads the book cache to + # answer one lookup and then fails. That is the pause after + # "looking up p_webkitgtk in the book", and the reason `info` showed no + # book version to compare against. _tries = [anchor] for _pre in (pkgusr_prefix(), user_prefix(), collector_prefix()): if _pre and anchor.startswith(_pre): _tries.append(anchor[len(_pre):]) - own = _book_override(_tries[-1]) - books = ([own] if own and os.path.isfile(own) else []) + [""] - if _PLAN_BOOK and _PLAN_BOOK not in books: - books.append(_PLAN_BOOK) - for b in books: - hit = _map_lookup(_versions_for(b), _tries) + for _a in _tries: + hit = _VERSION_MAP.get(_a) if hit: return hit - if _tries[-1] in _BOOK_MISSES: - return None + if _VERSION_MAP: + for _a in _tries: + low = _a.lower() + for a, nv in _VERSION_MAP.items(): + if a.lower() == low: + return nv rc, out, _ = run_blfs(["debug", _tries[-1]]) - m = re.search(r"^title\s*:\s*(.+)$", out, re.M) if rc == 0 else None + if rc != 0: + return None + m = re.search(r"^title\s*:\s*(.+)$", out, re.M) nv = m.group(1).strip() if m else None - if nv: - _versions_for(books[0])[anchor] = nv # ask once, even for an alias - else: - _BOOK_MISSES.add(_tries[-1]) + if nv and _VERSION_MAP is not None: + _VERSION_MAP[anchor] = nv # ask once, even for an alias return nv -def default_book_version(anchor): - """The version the DEFAULT book has, whatever book the package reads.""" - return _map_lookup(_versions_for(""), [anchor]) - - def book_rdeps(target, no_recommended=False, optional=False): """anchors of packages that recursively depend on `target`, via blfs rdeps.""" argv = ["rdeps", target, "--anchors"] @@ -990,49 +854,18 @@ def book_rdeps(target, no_recommended=False, optional=False): return result -def _versions_cache_path(book=""): - """Where the anchor -> version map is kept between runs: one per book.""" +def _versions_cache_path(): + """Where the anchor -> version map is kept between runs.""" d = os.path.join("/var", "cache", "pkgusr") try: os.makedirs(d, exist_ok=True) except OSError: d = "/tmp" - tag = hashlib.md5(book.encode()).hexdigest()[:8] if book else "default" - return os.path.join(d, "book-versions-%s.json" % tag) - - -def _default_book_file(): - """The file the default book is: BLFS_BOOK_FILE, BLFS_BOOK, `blfs path`. - - NOT the newest file on disk: once a stack fetched the svn book it was - the newest, and every version question on the machine would have been - answered from svn.""" - _bf = os.environ.get("BLFS_BOOK_FILE") or "" - if _bf and os.path.isfile(_bf): - return _bf - if not hasattr(_default_book_file, "_p"): - _p = "" - try: - r = subprocess.run(find_blfs() + ( - ["--book", os.environ["BLFS_BOOK"]] - if os.environ.get("BLFS_BOOK") else []) + ["path", "--offline"], - capture_output=True, text=True) - out = (r.stdout or "").strip().splitlines() - if r.returncode == 0 and out and os.path.isfile(out[-1]): - _p = out[-1] - except OSError: - pass - if not _p: # an older blfs without `path` - _books = sorted((f for f in glob.glob("/usr/share/blfs/books/*.html") - if "svn" not in os.path.basename(f)), - key=lambda f: os.stat(f).st_mtime, reverse=True) - _p = _books[0] if _books else "" - _default_book_file._p = _p - return _default_book_file._p + return os.path.join(d, "book-versions.json") -def book_versions_all(book=None): - """Map anchor -> name_version for the whole book (default, or `book`). +def book_versions_all(): + """Map anchor -> name_version for the whole book. ONE CALL WAS NOT ENOUGH: IT IS ONE CALL PER PROCESS. `blfs versions` re-reads the whole book to build this map, which is eighteen seconds on @@ -1044,8 +877,15 @@ def book_versions_all(book=None): book file changes the cache is ignored, so a stale map cannot outlive the book it came from. """ - # WHICH book: the one asked for, else the default book's file. - _bf = book if book and os.path.isfile(book) else _default_book_file() + # WHICH book: the explicit one if given, else whichever cached book file + # is newest -- the same file `blfs` would read. + _bf = os.environ.get("BLFS_BOOK_FILE") or "" + if not (_bf and os.path.isfile(_bf)): + _bf = "" + _books = sorted(glob.glob("/usr/share/blfs/books/*.html"), + key=lambda f: os.stat(f).st_mtime, reverse=True) + if _books: + _bf = _books[0] _key = None if _bf: try: @@ -1053,7 +893,7 @@ def book_versions_all(book=None): _key = "%s:%d:%d" % (_bf, _st.st_size, int(_st.st_mtime)) except OSError: _key = None - _cp = _versions_cache_path(_bf) + _cp = _versions_cache_path() if _key: try: with open(_cp, encoding="utf-8") as f: @@ -1062,7 +902,7 @@ def book_versions_all(book=None): return _c["versions"] except (OSError, ValueError): pass - rc, out, _ = run_blfs(["versions"], book=_bf or None) + rc, out, _ = run_blfs(["versions"]) versions = {} if rc == 0: for line in out.splitlines(): @@ -1409,39 +1249,6 @@ class User: return self.meta.get("name_version") -def _resolve_need(name): - """A needs= name -> the book's anchor for it (1.14.317). - - The book files a package under an ANCHOR that is not always its name: - libjpeg-turbo-3.2.0 is `libjpeg`. needs=libjpeg-turbo therefore found - no page, the stack tried to INSTALL it, and stopped on - ERROR: 'libjpeg-turbo' not found - with the package installed all along -- the "did you mean" under it even - named the right anchor. So: a name that is not an anchor, and matches - exactly ONE page's own package name (its version string without the - version), is that page. More than one, or none, and the name is used as - given, which fails the way it always did. - """ - try: - rc, out, _e = run_blfs(["search", name]) - except Exception: - return name - anchors, by_pkg = set(), [] - for ln in (out or "").splitlines(): - m = re.match(r"\s*\[package\]\s+(\S+)\s+(\S+)", ln) - if not m: - continue - anchor, ver = m.group(1), m.group(2) - anchors.add(anchor) - base = re.sub(r"-\d[\w.+~-]*$", "", ver) - if base.lower() == name.lower(): - by_pkg.append(anchor) - if name in anchors or len(set(by_pkg)) != 1: - return name - printInfo(" (%s is the book's `%s`)" % (name, by_pkg[0])) - return by_pkg[0] - - def gather_user(name): """Populate a User from the system user db, the on-disk package dir, and its install_last script. @@ -4619,32 +4426,6 @@ def _explain_not_in_book(target): printInfo(f" Nothing similar found either. Try: blfs search {target}") -def _install_finished(u): - """Positive evidence that an install FINISHED -- not that one started. - - THIS IS WHY THE STACK NEEDED HAND-HOLDING. `install_last` is copied - into the package home BEFORE the build runs (lfs-helper: "RECORD WHAT - IS INSTALLED BEFORE OVERWRITING THE RECORD"), so a build that dies at - configure leaves one behind. For a package whose page declares no - programs or libraries to check -- cairomm is one -- `state` then has - nothing else to go on and answers "installed". - - cairomm-1.0 failed at configure with 0 files tracked, was recorded - installed by that rule, and every later plan SKIPPED it as a dependency - that was already there. pangomm found out the hard way: - Run-time dependency cairomm-1.0 found: NO - and the person had to name it by hand to get the chain moving. - - A manifest with real paths, or the marker the runner writes after a - SUCCESSFUL install, is evidence. A script that was copied in before - the build is not. - """ - if getattr(u, "pkg_list", None): - return True - home = getattr(u, "home", "") or "" - return bool(home) and os.path.isfile(os.path.join(home, ".pkgusr-installed")) - - def build_install_plan(args): """Return (plan, missing, why). plan = [(anchor, name_version, reason)] in install order (deps first); reason in {new, update, reinstall}. why maps @@ -4675,7 +4456,6 @@ def build_install_plan(args): "built). If the plan") printInfo(" then looks short, one of them was a dependency: " "packagemanager deps <pkg>") - printInfo(" the list: packagemanager stack avoid --avoided") plan, seen, missing_all, why = [], set(), [], {} # WHAT IS ALREADY DONE, counted as we go. # @@ -4683,8 +4463,6 @@ def build_install_plan(args): # plan that has shrunk 181 -> 151 -> 135 across three runs reads as three # identical fresh starts. The progress was real and invisible. build_install_plan.already = set() - build_install_plan.behind = [] # installed deps the book has moved past - build_install_plan.unproven = [] # "installed", with nothing to show for it _not_found = [] # Reading the book can take many seconds the first time (it is indexed # once per version). Without a word on screen that looks like a hang, so @@ -4697,40 +4475,11 @@ def build_install_plan(args): if anchor in seen or anchor in ignore: return seen.add(anchor) - # A DEPENDENCY OUTSIDE A STACK BOOK READS ITS OWN BOOK. The order - # came from the svn book, so it carries svn's versions for - # everything; a dependency the stack does not own is judged -- and - # built, if missing -- against the book it always reads. - if _PLAN_BOOK and _book_override(anchor) != _PLAN_BOOK: - nv = default_book_version(anchor) or nv u = gather_user(anchor) - _done = u.state == "installed" and _install_finished(u) - if u.state == "installed" and not _done: - # recorded, but nothing says the install ever finished - build_install_plan.unproven.append(anchor) - up_to_date = (_done and (not u.version or u.version == nv)) + up_to_date = (u.state == "installed" + and (not u.version or u.version == nv)) if up_to_date: build_install_plan.already.add(anchor) - # AN INSTALLED DEPENDENCY IS NOT A THING TO UPGRADE. - # - # `install gparted --recursive` planned 53 packages: gparted, its - # real missing dependencies -- and UPDATES of bubblewrap, alsa-lib, - # gtk3, gtk4, librsvg, glycin and thirty others that are installed - # and working, purely because the book has moved on since they were - # built. Asking for one program is not asking to rebuild the - # desktop, and each of those rebuilds is a chance to break a - # machine that was fine. - # - # So: a DEPENDENCY that is installed is left alone, whatever version - # it is. A package NAMED on the command line still goes through - # (that is an instruction), --update-deps restores the old - # behaviour, and `update --stack <name>` is the deliberate way to - # move a whole chain forward. - elif (_done and anchor not in _named - and not getattr(args, "update_deps", False)): - build_install_plan.already.add(anchor) - build_install_plan.behind.append((anchor, u.version or "?", nv)) - return if args.reinstall: plan.append((anchor, nv, "reinstall" if up_to_date else ("update" if u.state == "installed" else "new"))) @@ -4738,12 +4487,8 @@ def build_install_plan(args): plan.append((anchor, nv, "update" if u.state == "installed" else "new")) - global _PLAN_BOOK for target in args.packages: set_status(f"working out what {target} needs ...") - _PLAN_BOOK = None - _tb = _book_override(target) - _PLAN_BOOK = _tb if _tb and os.path.isfile(_tb) else None if not getattr(args, "recursive", False): # DEFAULT: just this package, no dependency walk nv = book_version(target) @@ -4905,67 +4650,7 @@ def install_local(name, path, run, yes, verb="install", test=False, clean=False) printSuccess(f"Done -- {name} {verb}ed. logs: {_pkg_log_dir(name)}") -def _stack_git_entry(name): - """The git/tar entry for `name` in an installed stack, or None.""" - d = STACK_DIR if "STACK_DIR" in globals() else "/etc/pkgusr/stacks" - for f in sorted(glob.glob(os.path.join(d, "*.stack"))): - try: - for r in _parse_stack_file(f): - if r["kind"] in ("git", "tar") and r["target"] == name: - return r - except (ValueError, OSError): - continue - return None - - -def _install_git_packages(args): - """Build the git packages among args.packages from their own scripts. - - NO BOOK. `install --reinstall imv` asked the book about imv three - times (an index lookup, `blfs debug`, `blfs search`), took a minute to - say "not in the BLFS book", and built nothing -- imv is a git entry of - apps.stack, and its script is in its home. A git package is recognised - from that script (or from a stack's git/tar entry, the first time), - before anything asks the book. --clean and --test go through as they - do for `update`. Returns the names it handled.""" - done = [] - for name in list(args.packages or []): - script = _canonical_script_path(name) - if not script: - p = os.path.join(pkgusr_home(name), "install_%s" % name) - script = p if os.path.isfile(p) else None - is_git = bool(script) and _is_git_package(name) - entry = None if is_git else _stack_git_entry(name) - if not is_git and not entry: - continue - done.append(name) - if entry and not script: - script, _ = _stack_git_script( - entry, os.path.join(TMP_DIR, "stack_files")) - u = gather_user(name) - if u.state == "installed" and not args.reinstall: - printSuccess("%s: installed (a %s package). Rebuild it: " - "--reinstall [--clean]; new commits: update %s" - % (name, entry["kind"] if entry else "git", name)) - continue - printInfo("%s: %s package -- building from %s (no book lookup)" - % (name, entry["kind"] if entry else "git", script)) - if script != (_canonical_script_path(name) or "") and entry: - apply_machine_opts(name, script) - install_local(name, script, args.run, args.yes, "install", - test=getattr(args, "test", False), - clean=getattr(args, "clean", False)) - return done - - def cmd_install(args, _replanned=False): - # ---- git packages: their own script, never the book ---- - if not args.local and args.packages: - _git_done = _install_git_packages(args) - if _git_done: - args.packages = [p for p in args.packages if p not in _git_done] - if not args.packages: - return # ---- local package: install a custom script directly, no BLFS ---- if args.local or (len(args.packages) == 1 and book_version(args.packages[0]) is None @@ -5002,29 +4687,6 @@ def cmd_install(args, _replanned=False): return f"depends on {parent}" return f"{kind} by {parent}" - _unproven = getattr(build_install_plan, "unproven", []) or [] - if _unproven: - printWarning(" %d package(s) are recorded as installed but have no" - " finished-install record:" % len(_unproven)) - printWarning(" " + ", ".join(sorted(set(_unproven))[:8]) - + (" ..." if len(set(_unproven)) > 8 else "")) - printInfo(" Their install_last was written before a build that did" - " not finish, so they") - printInfo(" are in this plan rather than skipped -- which is what" - " lets the stack run") - printInfo(" through without naming them by hand.") - _behind = getattr(build_install_plan, "behind", []) or [] - if _behind: - printInfo(" %d installed dependency(ies) are older than the book and" - " are NOT touched:" % len(_behind)) - printInfo(" " + ", ".join("%s (%s -> %s)" % (a, iv, bv) - for a, iv, bv in _behind[:6]) - + (" ..." if len(_behind) > 6 else "")) - printInfo(" that is deliberate: one program is not a reason to" - " rebuild what works.") - printInfo(" move them forward on purpose: packagemanager update" - " <name> --run") - printInfo(" or build this plan against them: add --update-deps") if getattr(args, "select", False): plan = _select_plan_rows( plan, lambda r: f"{r[1]:<30} {r[2]:<10} {why_str(r[0])}") @@ -5258,7 +4920,7 @@ def _pkglist_paths(user): return {p for p in pkglist_entries(pl) if p.startswith("/")} -def _orphan_sweep(user, old_paths, since, remove=True): +def _orphan_sweep(user, old_paths, since): """Remove files that were in the package's OLD manifest but were NOT rewritten by the new version (mtime older than the install start `since`). Only old- manifest files are considered; config paths + the source tree are kept. Does @@ -5291,665 +4953,36 @@ def _orphan_sweep(user, old_paths, since, remove=True): victims.append(p) except OSError: continue - # A LIBRARY SOMETHING IS STILL LINKED TO IS NOT AN ORPHAN. - # - # webkitgtk rebuilt from the GTK4 port to the GTK3 one leaves - # libwebkitgtk-6.0.so.4 in the old manifest, unrewritten and owned by this - # package -- every test above says "orphan", and deleting it breaks every - # program built against it, with no warning and no way back but a rebuild. - # So each library about to go is asked the only question that matters: - # does anything else still record its soname? Files of this package do - # not count -- they go with it. - kept = [] - if victims: - libs = [p for p in victims if re.search(r"\.so(\.\d+)*$", p) - and os.path.isfile(p) and not os.path.islink(p)] - victim_set = set(victims) - for p in libs: - s = _soname_of(p) - if not s: - continue - users = [(f, o) for f, o in _needed_by(s, skip_uid=own_uid) - if f not in victim_set] - if users: - pkgs = sorted({o for _f, o in users}) - printWarning(" --clean: KEEPING %s -- %d file(s) in %s still " - "link against %s" - % (p, len(users), ", ".join(pkgs[:4]), s)) - kept.append((p, s, pkgs)) - if kept: - keep_paths = {p for p, _s, _pk in kept} - # the symlinks that point at a kept library stay with it, or the - # soname it is found by disappears and nothing loads anyway - for p in list(victims): - if os.path.islink(p): - t = os.path.realpath(p) - if t in keep_paths: - keep_paths.add(p) - victims = [p for p in victims if p not in keep_paths] - printInfo(" rebuild those packages, then clean again:") - for _p, _s, pkgs in kept: - printInfo(" packagemanager update %s --reinstall --run" - % " ".join(p[2:] if p.startswith("p_") else p - for p in pkgs[:4])) - break - printInfo(" or see the whole picture: packagemanager used-by %s" - % kept[0][1]) - if not remove: - # the caller decides (_clean_after_install parks them first) - return victims if victims: printInfo(f" --clean: removing {len(victims)} old-version file(s) the " f"upgrade no longer installs (configs / source kept)") remove_paths_as_user(user, victims) - elif not kept: + else: printInfo(" --clean: no leftover files from the old version.") return len(victims) def _reapply_install(user): - """Re-run ONLY the install phase (no recompile), as the user, to put back - the current-version files the mtime test mistook for old ones. - - Returns True only when the phase ran and exited 0. - - IT HAD NOT RUN SINCE 1.14.16 (fixed 1.14.311). It called - `bash ~/install_<account>` -- the runtime copy 1.14.16 removed everywhere - else -- with stdout and stderr thrown away and the exit status ignored. - So every --clean deleted the files the mtime test misjudged, the "restore" - failed on `No such file or directory` where nobody could see it, and the - run ended green. webkitgtk lost /usr/include/webkitgtk-6.0/jsc/*.h that - way: CMake's install skips a file whose content is unchanged ("-- - Up-to-date:"), so the headers kept their old mtime and read as orphans. - - Now: the same script and the same door as `packagemanager script - install` (lfs-helper build), its output kept in a log, and its result - returned so the caller can undo the clean when it fails. - """ - home = pkgusr_home(user) - script = (_versioned_script_in(home) - or os.path.join(home, f"install_{user}")) - if not os.path.isfile(script): - printError(f" --clean: no install script in {home} -- cannot reapply") - return False - helper = shutil.which("lfs-helper") - if helper and _can_su(user): - cmd = [helper, "build", user, "--script", script, - "--phase", "install", "--force"] - else: - inner = f"cd ~ && bash {shlex.quote(script)} install" - cmd = (["su", "-", user, "-c", inner] if _can_su(user) - else ["sh", "-c", inner]) - log = os.path.join(_plans_dir(), - re.sub(r"[^A-Za-z0-9_.-]+", "_", user) + ".clean-reapply.log") + """Re-run ONLY the install phase (no recompile), as the user, to restore any + current-version files that --clean may have removed (e.g. files installed + with preserved timestamps). Orphans stay removed.""" + inner = f"cd ~ && bash {shlex.quote(f'install_{user}')} install" + cmd = (["su", "-", user, "-c", inner] if _can_su(user) + else ["sh", "-c", inner]) printInfo(" --clean: reapplying the install step to restore current files...") - printInfo(" $ " + " ".join(cmd[1:] if cmd[0] == helper else cmd)) - try: - with open(log, "w") as lf: - rc = subprocess.run(cmd, stdout=lf, stderr=subprocess.STDOUT).returncode - except OSError as e: - printError(f" --clean: could not run the install step ({e})") - return False - if rc != 0: - printError(f" --clean: the install step FAILED (exit {rc}) -- log: {log}") - try: - for line in open(log, errors="replace").read().splitlines()[-12:]: - printInfo(" | " + line) - except OSError: - pass - return False - return True - - -_PARK_SUFFIX = ".pkgusr-clean" - - -def _park_ledger(user): - return os.path.join(_plans_dir(), - re.sub(r"[^A-Za-z0-9_.-]+", "_", user) + ".parked") - - -def _unpark_all(user): - """Put every parked file back, from the ledger. Returns how many.""" - ledger = _park_ledger(user) - n = 0 - try: - lines = open(ledger).read().splitlines() - except OSError: - return 0 - for p in lines: - if not p: - continue - q = p + _PARK_SUFFIX - if not os.path.lexists(q): - continue - if os.path.lexists(p): - # the install put the current version there: that one stays - try: - os.unlink(q) - except OSError: - pass - continue - try: - os.rename(q, p) - n += 1 - except OSError as e: - printError(f" could not put back {p} ({e}) -- it is at {q}") - try: - os.unlink(ledger) - except OSError: - pass - return n - - -# =========================================================================== # -# git packages: what they follow, and whether it moved [IMPLEMENTED] -# =========================================================================== # -# -# A book package has a version, and `update` compares it with the book's. A -# git package has a REF, and a ref answers a different question depending on -# what it is: -# a branch -- a moving target. "Is it behind?" is asked of the remote, and -# the answer changes without anyone doing anything. -# a tag -- a point in time. It is never behind; the question is whether -# a NEWER tag exists, and moving to it is a decision. -# a commit -- the same, without even a name to compare. -# Telling them apart is the whole of this. - -_GIT_ENV = dict(os.environ, GIT_TERMINAL_PROMPT="0", GIT_ASKPASS="/bin/true", - GIT_SSH_COMMAND="ssh -oBatchMode=yes") - - -def _script_var(text, var): - """A shell assignment out of an install script: link="..." / git_ref=...""" - m = re.search(r'^\s*%s\s*=\s*(?:"([^"]*)"|\'([^\']*)\'|([^\s#]*))' - % re.escape(var), text or "", re.M) - if not m: - return "" - return (m.group(1) or m.group(2) or m.group(3) or "").strip() - - -def _git_clone_dir(name, link=""): - """Where this package's checkout is. lfs-phases builds in $HOME/src, so - that is where to look -- and the remote is what confirms it, because a - home can hold more than one tree.""" - home = pkgusr_home(name) - for base in (os.path.join(home, "src"), home): - if not os.path.isdir(base): - continue - try: - entries = sorted(os.listdir(base)) - except OSError: - continue - cands = [os.path.join(base, d) for d in entries - if os.path.isdir(os.path.join(base, d, ".git"))] - if link: - for c in cands: - r = subprocess.run(["git", "-C", c, "remote", "get-url", "origin"], - capture_output=True, text=True, env=_GIT_ENV) - if r.returncode == 0 and r.stdout.strip().rstrip("/") == link.rstrip("/"): - return c - if len(cands) == 1: - return cands[0] - return "" - - -def _git(clone, *args, timeout=60): - try: - r = subprocess.run(["git", "-C", clone] + list(args), capture_output=True, - text=True, env=_GIT_ENV, timeout=timeout) - except (OSError, subprocess.SubprocessError): - return "" - return r.stdout.strip() if r.returncode == 0 else "" - - -def _ls_remote(link, pattern=None, timeout=90): - """[(sha, refname)] from the remote, without cloning anything.""" - cmd = ["git", "ls-remote"] - if pattern == "tags": - cmd += ["--tags", "--refs"] - cmd.append(link) - if pattern and pattern != "tags": - cmd.append(pattern) - try: - r = subprocess.run(cmd, capture_output=True, text=True, env=_GIT_ENV, - timeout=timeout) - except (OSError, subprocess.SubprocessError): - return [] - if r.returncode != 0: - return [] - out = [] - for ln in r.stdout.splitlines(): - parts = ln.split(None, 1) - if len(parts) == 2: - out.append((parts[0].strip(), parts[1].strip())) - return out - - -def _vsort_key(tag): - """Sort 1.9.0 before 1.10.0, which a plain string sort does not.""" - return [int(x) if x.isdigit() else x - for x in re.split(r"(\d+)", tag.lstrip("vV"))] - - -def _git_pkg_state(name, remote=True): - """Everything worth saying about one git package. - - Returns a dict, or None when this is not a git package. `remote=False` - asks the disk only -- no network, for the bulk view.""" - script = (_canonical_script_path(name) - or os.path.join(pkgusr_home(name), "install_%s" % name)) - if not script or not os.path.isfile(script): - return None - try: - text = open(script, encoding="utf-8", errors="replace").read() - except OSError: - return None - if not (re.search(r'is_git="?1"?', text) or re.search(r"\bgit\s+clone\b", text)): - return None - link = _script_var(text, "link") - ref = _script_var(text, "git_ref") - clone = _git_clone_dir(name, link) - st = {"name": name, "script": script, "link": link, "ref": ref, - "clone": clone, "kind": "?", "local": "", "remote": "", - "newer_tags": [], "behind": None, "note": ""} - if clone: - st["local"] = _git(clone, "rev-parse", "HEAD") - if not link: - st["note"] = "the script names no repository (link=)" - return st - if not remote: - return st - # WHAT KIND OF REF IS IT? The remote knows; guessing from the string - # ("master" is a branch, "2.18.0" is a tag) is how a tag named `main` - # or a branch named `1.x` goes wrong. - heads = {r.rsplit("/", 1)[-1]: s for s, r in _ls_remote(link, "refs/heads/*")} - tags = [r.rsplit("/", 1)[-1] for _s, r in _ls_remote(link, "tags")] - st["tags"] = tags - if ref and ref in heads: - st["kind"] = "branch" - st["remote"] = heads[ref] - elif ref and ref in tags: - st["kind"] = "tag" - newer = [t for t in tags if _vsort_key(t) > _vsort_key(ref)] - st["newer_tags"] = sorted(newer, key=_vsort_key) - elif not ref: - st["kind"] = "default branch" - for d in ("main", "master"): - if d in heads: - st["remote"] = heads[d] - st["ref"] = d - break - elif ref: - st["kind"] = "commit or missing ref" - st["note"] = "the remote has no branch or tag called %s" % ref - # how far behind, when the clone already has the objects - if st["kind"].endswith("branch") and st["local"] and st["remote"] \ - and st["local"] != st["remote"] and clone: - n = _git(clone, "rev-list", "--count", "%s..%s" % (st["local"], st["remote"])) - st["behind"] = int(n) if n.isdigit() else None - return st - - -def _git_pkg_line(st): - n = st["name"] - ref = st["ref"] or "(default)" - if st["note"] and not st["remote"] and not st["newer_tags"]: - return " %-20s %-14s %s" % (n, ref, st["note"]) - if st["kind"].endswith("branch"): - if not st["remote"]: - return " %-20s %-14s branch -- the remote could not be reached" % (n, ref) - if st["local"][:12] == st["remote"][:12]: - return " %-20s %-14s branch, up to date (%s)" % (n, ref, st["local"][:8]) - behind = ("%d commit(s) behind" % st["behind"]) if st["behind"] \ - else "moved" - return " %-20s %-14s branch, %s: %s -> %s" % ( - n, ref, behind, (st["local"] or "?")[:8], st["remote"][:8]) - if st["kind"] == "tag": - if not st["newer_tags"]: - return " %-20s %-14s tag, nothing newer released" % (n, ref) - nt = st["newer_tags"] - return " %-20s %-14s tag -- %d newer: %s" % ( - n, ref, len(nt), ", ".join(nt[-4:])) - return " %-20s %-14s %s %s" % (n, ref, st["kind"], st["note"]) - - -def cmd_git_status(args): - """What every git package follows, and whether it has moved.""" - names = args.packages or list_all_users() - states = [] - for n in sorted(set(names)): - st = _git_pkg_state(n, remote=not getattr(args, "local_only", False)) - if st: - states.append(st) - if not states: - printInfo("no git-based packages here" - if not args.packages else - "none of those are git-based packages") - return 0 - printHeader("git packages") - print() - print(" %-20s %-14s %s" % ("package", "ref", "state")) - print(" " + "-" * 62) - moved, taggy = [], [] - for st in states: - print(_git_pkg_line(st)) - if st["kind"].endswith("branch") and st["remote"] \ - and st["local"][:12] != st["remote"][:12]: - moved.append(st["name"]) - if st["newer_tags"]: - taggy.append(st) - print() - if moved: - printInfo(" follow the branch (fetch, rebuild, install):") - printInfo(" packagemanager update %s --run" % " ".join(moved)) - for st in taggy: - printInfo(" %s is pinned to %s; to move it:" % (st["name"], st["ref"])) - printInfo(" packagemanager git-ref %s %s" - % (st["name"], st["newer_tags"][-1])) - if not moved and not taggy: - printInfo(" nothing to do: every git package is where its ref says") - return 0 - - -def cmd_git_ref(args): - """Show what a git package could follow, or move it to another ref.""" - name = args.package - st = _git_pkg_state(name) - if st is None: - printError("%s is not a git-based package" % name) - return 1 - if not st["link"]: - printError("%s: its script names no repository" % name) - return 1 - if not args.ref: - tags = sorted(st.get("tags") or [], key=_vsort_key) - heads = [r.rsplit("/", 1)[-1] for _s, r in _ls_remote(st["link"], "refs/heads/*")] - printHeader("%s -- %s" % (name, st["link"])) - print() - print(" now: %s (%s)" % (st["ref"] or "(default)", st["kind"])) - print() - print(" branches: " + (", ".join(heads[:8]) or "(none)")) - print(" tags, newest last:") - for t in tags[-12:]: - print(" %s%s" % (t, " <- current" if t == st["ref"] else "")) - if not tags: - print(" (none -- this repository is not tagged)") - print() - printInfo(" pick one: packagemanager git-ref %s <ref>" % name) - return 0 - want = args.ref - known = set(st.get("tags") or []) | { - r.rsplit("/", 1)[-1] for _s, r in _ls_remote(st["link"], "refs/heads/*")} - if want not in known and not re.fullmatch(r"[0-9a-f]{7,40}", want): - printError("%s: the remote has no tag or branch called %s" % (name, want)) - printInfo(" what it has: packagemanager git-ref %s" % name) - return 1 - # THE SCRIPT IS ONE OF TWO PLACES THE REF LIVES. A stack entry writes it - # into the generated script, so editing only the script is undone by the - # next stack run -- say so, and name the file. - try: - text = open(st["script"], encoding="utf-8", errors="replace").read() - except OSError as e: - printError("%s: %s" % (st["script"], e)) - return 1 - new, n = re.subn(r'^(\s*git_ref\s*=\s*).*$', r'\1"%s"' % want, text, - count=1, flags=re.M) - if not n: - printError("%s has no git_ref= line to change" % st["script"]) - return 1 - if not getattr(args, "run", False): - printInfo(" would set git_ref=\"%s\" in %s (was \"%s\")" - % (want, st["script"], st["ref"])) - printInfo(" dry run -- add --run") - return 0 - bak = st["script"] + ".bak-" + time.strftime("%Y%m%d-%H%M%S") - try: - shutil.copy2(st["script"], bak) - open(st["script"], "w").write(new) - except OSError as e: - printError("could not write %s: %s" % (st["script"], e)) - return 1 - printSuccess("%s: git_ref is now %s (was %s; backup %s)" - % (name, want, st["ref"] or "(default)", os.path.basename(bak))) - src = _STACK_TARGET_SOURCE.get(name) - if src: - printWarning(" %s also carries this ref: a stack run regenerates the" - " script and puts the old one back." % src) - printInfo(" change the entry there too: ref=%s" % want) - printInfo(" then build it: packagemanager update %s --run" % name) - return 0 - - -# =========================================================================== # -# who still links against this? [IMPLEMENTED] -# =========================================================================== # -# -# A package-user system records who INSTALLED every file. It records nothing -# about who USES one -- and a library is used by things that were built -# against it long ago. webkitgtk rebuilt from the GTK4 port to the GTK3 one -# leaves libwebkitgtk-6.0.so.4 behind with nothing to rebuild it, and -# `--clean` is then one keystroke from deleting a library other programs are -# still linked to. ldd answers it for ONE file; this answers it for a -# library. - -_ELF_SCAN_DIRS = ("/usr/bin", "/usr/sbin", "/usr/lib", "/usr/libexec", - "/bin", "/sbin", "/lib", "/opt") -_ELF_CONFIRM_MAX = 600 # readelf calls; a grep hit is cheap, this is not - - -def _soname_of(path): - """The SONAME an ELF library announces -- libwebkitgtk-6.0.so.4, not the - file name libwebkitgtk-6.0.so.4.13.8. That is the name other binaries - record, so it is the name to search for.""" - if not os.path.isfile(path): - return None - try: - r = subprocess.run(["readelf", "-d", path], capture_output=True, - text=True, timeout=30) - except (OSError, subprocess.SubprocessError): - r = None - if r and r.returncode == 0: - m = re.search(r"\(SONAME\)\s+Library soname: \[([^\]]+)\]", r.stdout) - if m: - return m.group(1) - b = os.path.basename(path) - m = re.match(r"^(lib.+?\.so(?:\.\d+)?)(?:\.|$)", b) - return m.group(1) if m else None - - -def _needed_by(soname, skip_uid=None, dirs=None): - """Every ELF that records this soname in DT_NEEDED: [(path, owner)]. - - grep first -- a soname is a literal string in .dynstr, and grep over - /usr/lib is seconds -- then readelf only on the hits, because a string can - also appear in a script, a .la file or another library's rpath. - """ - # RESOLVED HERE, NOT IN THE SIGNATURE. A default argument is bound at - # import, so `dirs=_ELF_SCAN_DIRS` could never be overridden -- including - # by a test, which is how this function first appeared to find nothing. - live = [d for d in (dirs or _ELF_SCAN_DIRS) if os.path.isdir(d)] - if not live: - return [] - try: - r = subprocess.run(["grep", "-rlZ", "--binary-files=binary", "-F", - "--", soname] + live, - capture_output=True, timeout=600) - except (OSError, subprocess.SubprocessError): - return [] - _all = [c.decode("utf-8", "replace") for c in r.stdout.split(b"\0") if c] - cands = _all[:_ELF_CONFIRM_MAX] - if len(_all) > _ELF_CONFIRM_MAX: - # A TRUNCATED ANSWER MUST NOT READ AS A COMPLETE ONE. - printWarning(" %s: %d candidates, only the first %d were confirmed" - % (soname, len(_all), _ELF_CONFIRM_MAX)) - out = [] - for p in cands: - if os.path.islink(p) or not os.path.isfile(p): - continue - if os.path.basename(p).startswith(soname.split(".so")[0] + ".so"): - continue # the library itself, or its siblings - try: - st = os.lstat(p) - if skip_uid is not None and st.st_uid == skip_uid: - continue # its own package may rebuild freely - d = subprocess.run(["readelf", "-d", p], capture_output=True, - text=True, timeout=30) - except (OSError, subprocess.SubprocessError): - continue - if d.returncode != 0: - continue - if re.search(r"\(NEEDED\).*\[%s\]" % re.escape(soname), d.stdout): - owner = "?" - try: - owner = pwd.getpwuid(st.st_uid).pw_name if pwd else "?" - except KeyError: - owner = str(st.st_uid) - out.append((p, owner)) - return out - - -def cmd_used_by(args): - """What still links against this library, or this package's libraries.""" - target = args.name - sonames = [] - if re.search(r"\.so(\.|$)", target): - sonames = [target if os.sep not in target - else (_soname_of(target) or os.path.basename(target))] - else: - # a package: every shared library in its manifest - libs = [p for p in _pkglist_paths(target) or [] - if re.search(r"/lib[^/]*\.so(\.\d+)*$", p) - and os.path.isfile(p) and not os.path.islink(p)] - for p in sorted(libs): - s = _soname_of(p) - if s and s not in sonames: - sonames.append(s) - if not sonames: - printInfo("%s installs no shared library (or has no manifest)" % target) - return 0 - printHeader("What links against %s" % ", ".join(sonames)) - print() - total = 0 - for s in sonames: - users = _needed_by(s) - by_pkg = {} - for p, owner in users: - by_pkg.setdefault(owner, []).append(p) - if not users: - print(" %-34s nothing links against it" % s) - continue - total += len(users) - print(" %s -- %d file(s), %d package(s):" % (s, len(users), len(by_pkg))) - for owner in sorted(by_pkg): - ex = ", ".join(os.path.basename(x) for x in sorted(by_pkg[owner])[:3]) - more = "" if len(by_pkg[owner]) <= 3 else " ..." - print(" %-22s %s%s" % (owner, ex, more)) - print() - if total: - printInfo(" removing a library these were built against breaks them" - " until they are rebuilt:") - printInfo(" packagemanager update <package> --reinstall --run") - else: - printInfo(" nothing needs these; they are safe to remove") - return 0 + subprocess.run(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) def _clean_after_install(user, old_paths, since): """Full --clean sequence run AFTER a successful install: remove old-version orphans, reapply install (to undo any over-removal), then regenerate the - manifest so it reflects the final state. - - PARK, REINSTALL, THEN JUDGE (1.14.311). "Older than the install" is - not "no longer installed": CMake skips a file whose content has not - changed, meson and `install -p` keep timestamps, so a current file can - look old. This used to delete first and trust a reapply to repair the - damage -- a reapply that had not run since 1.14.16, so the damage stayed - (webkitgtk's jsc/*.h, and every build against webkitgtk-6.0 broke). - - Now nothing is deleted on the mtime test's word alone. The candidates - are renamed aside (same directory, so it is one rename each and nothing - is copied), the install phase runs, and only then: - * a path the install put back IS current -- the parked copy goes; - * a path it did not put back is a real orphan -- that one goes; - * if the install step FAILS, every parked file is put back, and the - clean is reported as not done. Nothing is half-cleaned. - """ - # an earlier clean interrupted between park and judge: finish undoing it - if os.path.exists(_park_ledger(user)): - n = _unpark_all(user) - printWarning(f" --clean: an interrupted clean had parked files -- " - f"{n} put back first") - - victims = _orphan_sweep(user, old_paths, since, remove=False) - if not victims: - printInfo(" --clean: no leftover files from the old version.") - printInfo(" --clean: regenerating the file manifest...") - write_pkg_list(user, background=False) - return 0 - - files = [p for p in victims if os.path.islink(p) or not os.path.isdir(p)] - dirs = [p for p in victims if p not in set(files)] - parked = [] - with open(_park_ledger(user), "w") as led: - for p in files: - try: - os.rename(p, p + _PARK_SUFFIX) - parked.append(p) - led.write(p + "\n") - led.flush() - except OSError as e: - printWarning(f" could not set {p} aside ({e}) -- left in place") - printInfo(f" --clean: {len(parked)} file(s) look older than this install " - f"-- set aside while the install step confirms which are gone") - - if not _reapply_install(user): - n = _unpark_all(user) - printError(f" --clean: NOT DONE -- put {n} file(s) back as they were.") - printInfo(" The upgrade itself is installed; only the clean-up was undone.") - printInfo(" Fix the install step, then: packagemanager update " - f"{user[2:] if user.startswith('p_') else user} --reinstall --clean --run") - write_pkg_list(user, background=False) - return 0 - - still = [p for p in parked if os.path.lexists(p)] - gone = [p for p in parked if not os.path.lexists(p)] - # both kinds of parked copy go: for `still` the install wrote the - # current one, for `gone` nothing replaced it -- that is the orphan - remove_paths_as_user(user, [p + _PARK_SUFFIX for p in parked] + dirs) - try: - os.unlink(_park_ledger(user)) - except OSError: - pass - if still: - printInfo(f" --clean: {len(still)} of them are still installed by this " - f"version (unchanged, so the install left their old date) -- kept") - if gone: - printInfo(f" --clean: removed {len(gone)} file(s) the new version no " - f"longer installs") - for p in gone[:10]: - printInfo(" " + p) - if len(gone) > 10: - printInfo(" ... and %d more" % (len(gone) - 10)) - else: - printInfo(" --clean: nothing was an orphan after all") + manifest so it reflects the final state.""" + removed = _orphan_sweep(user, old_paths, since) + if removed: + _reapply_install(user) printInfo(" --clean: regenerating the file manifest...") write_pkg_list(user, background=False) - return len(gone) - - -def _is_phased_script(text): - """Does this install script hand its phases to the runner? - - The runner is what fetches, unpacks, resumes in the build directory and - re-runs one phase -- so "phased" means "ends in pkgusr_run", not "defines - some particular function". Three spellings, because the header and the - library name are each written by a different generation of the tools. - """ - t = text or "" - return bool(re.search(r"\bpkgusr_run\b", t) - or re.search(r"PKGUSR_LIB|lfs-phases", t) - or re.search(r"^###\s*phases:", t, re.M)) + return removed def _is_git_package(name): @@ -6175,19 +5208,6 @@ def _stack_targets(name): except ValueError as e: printError(str(e)) return None - # a stack on another book: refresh it once, and make sure the entries - # read it even when this file is not the installed copy - _sb = {r["target"]: r["opts"]["from-book"] for r in rows - if r["kind"] == "book" and r["opts"].get("from-book")} - if _sb: - _stack_book_map().update({k.lower(): v for k, v in _sb.items()}) - _BOOK_FOR_PKG.clear() - os.environ["PM_STACK_BOOKS"] = ",".join( - "%s=%s" % kv for kv in sorted(_sb.items())) - for _sel in sorted(set(_sb.values())): - _book_path_for(_sel, refresh=True) - printInfo(" %d package(s) of this stack read the %s book" - % (list(_sb.values()).count(_sel), _sel)) return [r["target"] for r in rows if r["kind"] in ("book", "git", "tar")] @@ -6239,61 +5259,10 @@ def cmd_update(args): printWarning(f"{name}: not a git package and not in the book -- " f"skipping (check the name, or use --local).") continue - # IS IT PHASED? ASK THE RIGHT QUESTION. - # - # This looked for `unpack_pkg`, which is the RARE OVERRIDE a script - # defines to replace the built-in unpack entirely. A correct, - # current script does not define it -- the runner's own unpack is - # what clones git -- so every properly phased package was told its - # script "isn't phased", with advice (`template --git`) that - # overwrites the file and takes any hand-tuned build_pkg with it: - # meson setup build ... -D c_link_args=-licuuc - # A phased script is one that ENDS IN THE RUNNER. That is the - # thing that makes fetch/rebuild work, and it is what to test. - _body = open(script, errors="replace").read() - if not _is_phased_script(_body): - printWarning(f"{name}: this script does not end in the phase " - f"runner, so `update` cannot fetch and rebuild it " - f"cleanly.") - printInfo(" a phased script ends with:") - printInfo(' . "${PKGUSR_LIB:-lfs-phases}" && pkgusr_run "$@"') - printInfo(" and puts its commands in build_pkg / install_pkg.") - printInfo(f" `packagemanager template {name} --git <url>` writes" - f" a fresh one -- to a NEW file first, then move your") - printInfo(" build commands across: it does not keep them.") - # SAY WHETHER THERE IS ANYTHING TO UPDATE TO. - # - # "git package -- updating from its repo" is true of a package - # pinned to the newest tag, where the rebuild produces the same - # commit it already has: the command works, does exactly what it - # says, and changes nothing -- and there is no way to tell that - # from the output. The remote knows; ask it. - _st = None if getattr(args, "no_git_check", False) \ - else _git_pkg_state(name) - if _st: - if _st["kind"].endswith("branch") and _st["remote"]: - if _st["local"][:12] == _st["remote"][:12]: - printInfo(" %s: %s is at %s, the same commit as here " - "-- this rebuilds what is already installed" - % (name, _st["ref"], _st["remote"][:8])) - else: - printInfo(" %s: %s moved %s -> %s%s -- this brings it in" - % (name, _st["ref"], (_st["local"] or "?")[:8], - _st["remote"][:8], - (" (%d commit(s))" % _st["behind"]) - if _st["behind"] else "")) - elif _st["kind"] == "tag": - if _st["newer_tags"]: - printInfo(" %s: pinned to tag %s; %s is out" - % (name, _st["ref"], _st["newer_tags"][-1])) - printInfo(" a rebuild stays on %s. To move it:" - % _st["ref"]) - printInfo(" packagemanager git-ref %s %s --run" - % (name, _st["newer_tags"][-1])) - else: - printInfo(" %s: pinned to tag %s, which is the newest " - "released -- this rebuilds the same commit" - % (name, _st["ref"])) + if "unpack_pkg" not in open(script, errors="replace").read(): + printWarning(f"{name}: git script isn't phased -- it won't fetch/" + f"rebuild cleanly. Regenerate it with:\n" + f" packagemanager template {name} --git <repo-url>") printInfo(f"{name}: git package -- updating from its repo.") install_local(name, script, args.run, args.yes, "update", test=getattr(args, "test", False), @@ -6345,10 +5314,6 @@ def cmd_update(args): vlookup = book_versions_all() names = list_all_users() - # a walk from a stack-book target carries that book's versions for - # every dependency; the ones it does not own read their own book - _stack_walk = getattr(args, "recursive", False) and any( - _book_override(t) for t in (args.packages or [])) _bk = _current_blfs_book() _checked = len(names) printInfo(f"Checking {len(names)} package(s) against the book" @@ -6364,13 +5329,6 @@ def cmd_update(args): u = gather_user(name) anchor = u.meta.get("name") or name bv = vlookup.get(anchor) - # EACH PACKAGE AGAINST ITS OWN BOOK: a stack's from-book (or - # machine.conf's book=) -- the default map would "update" an - # svn-built mpv back to the stable page. - if _book_override(anchor): - bv = book_version(anchor) or bv - elif _stack_walk: - bv = default_book_version(anchor) or bv if u.state != "installed": if name in named or anchor in named: not_installed.append(name) @@ -6438,47 +5396,20 @@ def cmd_update(args): f"found) -- use: packagemanager install {name}") if git_tracked: - # "LEFT ALONE" IS NOT AN ANSWER TO "IS IT OUT OF DATE?" - # - # This said `-git` cannot be behind a book version, which is true - # and beside the point: a package that follows a BRANCH is behind - # whenever the branch moves, and the remote can be asked in a - # second. A package pinned to a TAG is not behind at all -- the - # question there is whether a newer tag exists, and taking it is a - # decision, not an update. Both are answered here rather than - # left to the reader. print() - printInfo(" %d package(s) follow GIT, not the book:" + printInfo(" %d package(s) follow GIT, not the book -- left alone:" % len(git_tracked)) - _moved, _tagged = [], [] for _n, _iv, _bv in git_tracked: - _st = _git_pkg_state(_n) if not getattr(args, "no_git_check", - False) else None - if _st: - print(_git_pkg_line(_st)) - if _st["kind"].endswith("branch") and _st["remote"] \ - and _st["local"][:12] != _st["remote"][:12]: - _moved.append(_n) - if _st["newer_tags"]: - _tagged.append(_st) - else: - printInfo(" %-22s tracked as %-20s (the book has %s)" - % (_n, _iv, _bv or "no page")) - if _moved: - printInfo(" behind their branch -- fetch, rebuild, install:") - printInfo(" packagemanager update %s --run" % " ".join(_moved)) - for _st in _tagged: - printInfo(" %s is pinned to %s, and %s is out:" - % (_st["name"], _st["ref"], _st["newer_tags"][-1])) - printInfo(" packagemanager git-ref %s %s --run" - % (_st["name"], _st["newer_tags"][-1])) - if not _moved and not _tagged: - printInfo(" every one of them is where its ref says it" - " should be") - printInfo(" the whole picture: packagemanager git-status") - printInfo(" (taking the BOOK's page instead would be a" - " different build:") - printInfo(" `packagemanager update <pkg> --reinstall --run`.)") + printInfo(" %-22s tracked as %-20s (the book has %s)" + % (_n, _iv, _bv or "no page")) + printInfo(" `-git` is not a version, so it cannot be behind" + " one. To move such a") + printInfo(" package, rebuild its stack entry: packagemanager" + " stack <name> --only <pkg> --run --force") + printInfo(" (that follows the ref in the stack file. Taking" + " the book's page instead") + printInfo(" would be a different build: " + "`packagemanager update <pkg> --reinstall --run`.)") # named packages that are already current: offer to re-apply for anchor, name, iv, bv in uptodate: @@ -7990,32 +6921,7 @@ def _python_module_here(name): return False -# WHAT A MANIFEST SCAN MUST NOT WALK. -# -# `reload-pkg-list p_linux` took minutes, and the time went nowhere near the -# installed files: the scan walked /usr/src/pkgusr -- every package user's -# unpacked source tree, the kernel's own quarter-million files among them -- -# plus /proc, /sys, the snapshots and the caches. None of it can contain an -# INSTALLED file by definition: a package's home is excluded from its own -# manifest, and no package installs into another's source tree. -_SCAN_SKIP = ["/proc", "/sys", "/dev", "/run", "/tmp", "/var/tmp", - "/var/cache", "/mnt", "/media", "/lost+found", - "/.snapshots", "/snapshots", "/sources", "/build"] - - -def _scan_prune_args(): - """find(1) arguments that skip those directories (and stay on one fs).""" - skips = list(_SCAN_SKIP) + [d.rstrip("/") for d in pkgusr_roots()] - out = ["-xdev", "("] - for i, d in enumerate(skips): - if i: - out.append("-o") - out += ["-path", d] - out += [")", "-prune", "-o"] - return out - - -def write_pkg_list(user, background=True, as_root=False): +def write_pkg_list(user, background=True): """Regenerate /usr/src/<user>/pkg.lst. Runs list_package AS the package user (su -) when possible, so it only scans directories that user owns -- much faster than scanning the whole filesystem as root.""" @@ -8069,29 +6975,16 @@ def write_pkg_list(user, background=True, as_root=False): " /\\/lib\\/modules\\/[^/]+\\/source\\// { next } " " { print }'" % shlex.quote(home.rstrip("/") + "/")) - # THE SCAN ITSELF, PRUNED. `list_package` is the hint's script and walks - # everything; this is the same question asked of the same filesystem - # without the directories that cannot hold an installed file. It is run - # AS the package user, which also skips every directory that account - # cannot read -- and that is the difference between minutes and seconds - # on a machine with 90 source trees. --root scans as root instead: it - # reaches directories the account cannot enter (0700 dirs owned by - # somebody else), and takes the long way round. - _find = " ".join(shlex.quote(x) for x in - ["find", "/"] + _scan_prune_args() - + ["-user", user, "-print"]) - _scan = f"{_find} 2>/dev/null | LC_ALL=C sort -u" - if _can_su(user) and not as_root: + if _can_su(user): # su - starts in the user's home, so pkg.lst.new/pkg.lst are ~/... - inner = (f"{_scan} | {_filter} > pkg.lst.new" + inner = (f"{lp} {quser} 2>/dev/null | {_filter} > pkg.lst.new" f" && mv pkg.lst.new pkg.lst") cmd = ["su", "-", user, "-c", inner] else: tmp = shlex.quote(os.path.join(home, "pkg.lst.new")) dst = shlex.quote(os.path.join(home, "pkg.lst")) - cmd = ["sh", "-c", f"{_scan} | {_filter} > {tmp}" - f" && mv {tmp} {dst}" - f" && chown {quser} {dst} 2>/dev/null; true"] + cmd = ["sh", "-c", f"{lp} {quser} 2>/dev/null | {_filter} > {tmp}" + f" && mv {tmp} {dst}"] if background: # detached / disowned subprocess.Popen(cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, @@ -8116,12 +7009,8 @@ def cmd_reload_pkg_list(args): printInfo(f"{acct}: pkg.lst regeneration started" f" (disowned background task).") continue - _as_root = getattr(args, "root", False) - printInfo(f"{acct}: scanning for files it owns " - + ("as root (every directory) ..." if _as_root - else f"as {acct} ...")) - _t0 = time.time() - if not write_pkg_list(user, background=False, as_root=_as_root): + printInfo(f"{acct}: scanning for files it owns ...") + if not write_pkg_list(user, background=False): printError(f"{acct}: could not regenerate pkg.lst") rc = 1 continue @@ -8132,18 +7021,7 @@ def cmd_reload_pkg_list(args): except OSError: n = 0 if n: - printSuccess(f"{acct}: {n} file(s) recorded in {manifest}" - f" ({time.time() - _t0:.0f}s)") - if not _as_root: - # SAY WHAT THE FAST SCAN CANNOT SEE. A file this package - # owns inside a directory it may not read is not in this - # list, and nothing else would ever say so. - printInfo(f" scanned as {acct}: directories it cannot enter" - f" were skipped, and so were") - printInfo(f" {', '.join(_SCAN_SKIP[:6])} ... and the package" - f" source roots.") - printInfo(f" If something is missing: packagemanager" - f" reload-pkg-list {user} --root") + printSuccess(f"{acct}: {n} file(s) recorded in {manifest}") else: printWarning(f"{acct}: no files found -- it owns nothing on disk.") printInfo(" Either it was never installed, or its files ended up" @@ -9126,13 +8004,6 @@ def _cmd_config_interactive(args): if args.main_user: cfg["main_user"] = args.main_user changed = True - if getattr(args, "shared_hub", None): - _h = os.path.abspath(args.shared_hub).rstrip("/") - if cfg.get("shared_hub") != _h: - cfg["shared_hub"] = _h - changed = True - printInfo(" the links move the next time the accounts are set up:") - printInfo(" packagemanager stack services --only app-users --run") # Nothing given on the command line and something is unset? Ask. Leaving # a setting blank and carrying on means it surfaces much later as a failed # install with no obvious cause. @@ -9166,8 +8037,6 @@ def _cmd_config_interactive(args): print(f" user prefix : {cfg['user_prefix']}_ (shared users)") print(f" main user : {cfg.get('main_user') or '(not set)'} " f"(your human login account)") - print(f" shared hub : {shared_hub()} " - f"(links to every shared directory)") print(f" editor : {editor()}") print(f" difftool : {difftool()}") @@ -9213,8 +8082,8 @@ player cannot read your documents, your keys, or another application's data. Creating the account is the easy part; what makes it usable is the rest: - * your home directory stays yours, and the account's stays its own: the - account's home is 0700 + * your home directory stays yours -- the account's home is 0750, so the + application cannot read it * your login joins the account's group, so a shared directory works * `su - u_<name>` from your account without a password, where the hint's /etc/pam.d/su_u_user is installed (skipped where it is not: su then @@ -9921,7 +8790,8 @@ def _relocate_app_home(u): def _repoint_shared_link(u, old_home, new_home): """~/Shared/<Name> of the main user pointed into the old home.""" - hub = shared_hub() + mh = _home_of(main_user() or "") or "" + hub = os.path.join(mh, "Shared") if mh else "" if not hub or not os.path.isdir(hub): return for n in os.listdir(hub): @@ -10063,7 +8933,7 @@ def _setup_application_user(u, args): Creating the account is the easy part. What makes it useful: * your account may `su - <user>` without a password (via PAM) - * the home is 0700, so no other account can read it + * the home is 0750, so other users cannot read it * your account joins the user's group, for the shared directory * an optional shared directory both accounts can write * an optional launcher in ~/bin so the program runs as that user @@ -10077,16 +8947,10 @@ def _setup_application_user(u, args): printWarning(" packagemanager config --main-user <your-login>") return - # 1. the home is private -- 0700, NOT 0750 (1.14.315) - # - # 0750 let the account's GROUP in, and your login is in that group (for - # the shared directory). So "a browser cannot read your documents" held, - # and the reverse did not: you could read its profile and cookies, and - # so could anything running as you. The shared directory lives in the - # hub now, so nothing needs to pass through the home. + # 1. the home is private if os.path.isdir(home): - _run(["chmod", "0700", home]) - printInfo(f" home {home} is 0700 (private to {u})") + _run(["chmod", "0750", home]) + printInfo(f" home {home} is 0750 (private to {u} and its group)") # 2. your account joins the user's group -- this is what makes a shared # directory work, and what lets PAM identify you as entitled @@ -10105,50 +8969,43 @@ def _setup_application_user(u, args): _link_xdg_profile(main, u) # 5. a shared directory, when asked for - # - # THE DIRECTORY LIVES IN THE HUB, THE LINK LIVES IN THE HOME (1.14.315). - # It was the other way round: the data in <account home>/Shared and a - # link to it in the hub -- so reaching it meant passing through the - # account's home, and the home had to be group-readable (0750) for - # that. Your login is in the account's group, so it could read the - # browser profile, the cookies, everything. Now the data is - # /home/Shared/<Name> (2770 u_<name>:u_<name>), the home is 0700, and - # the account's own ~/Shared is a link into the hub -- which keeps - # `--download-dir ~/Shared` in app-users.conf meaning the same thing. if getattr(args, "share_dir", False): - _hub = _ensure_shared_hub(main) - share = os.path.join(_hub, args.name[:1].upper() + args.name[1:]) - if os.path.islink(share): - printWarning(f" {share} is a link (to {os.readlink(share)}) -- the " - f"layout before 1.14.315; left alone") - printInfo(" move the data in, as the 1.14.315 notes say, then " - "run this again") - else: - _run(["mkdir", "-p", share]) - _run(["chown", f"{u}:{u}", share]) - _run(["chmod", "2770", share]) # setgid: new files keep the group - printInfo(f" shared directory: {share} (2770 {u}:{u})") - # ...and the account's ~/Shared points at it - _in_home = os.path.join(home, "Shared") - if os.path.islink(_in_home): - if os.path.realpath(_in_home) != os.path.realpath(share): - printWarning(f" {_in_home} points at " - f"{os.readlink(_in_home)} -- left alone") - elif os.path.isdir(_in_home): - if not os.listdir(_in_home): - os.rmdir(_in_home) - _run(["ln", "-s", share, _in_home]) - _run(["chown", "-h", f"{u}:{u}", _in_home]) - else: - # DATA IS NEVER MOVED BY THIS. It was asked for as a - # hand step; the command is printed, not run. - printWarning(f" {_in_home} still holds files -- not " - f"moved (that is yours to do)") - printInfo(f" mv {_in_home}/* {_in_home}/.[!.]* {share}/ " - f"&& rmdir {_in_home} && ln -s {share} {_in_home}") - elif not os.path.lexists(_in_home): - _run(["ln", "-s", share, _in_home]) - _run(["chown", "-h", f"{u}:{u}", _in_home]) + share = os.path.join(_app_home(u), "Shared") + _run(["mkdir", "-p", share]) + _run(["chown", f"{u}:{u}", share]) + _run(["chmod", "2770", share]) # setgid: new files keep the group + # ONE DIRECTORY OF LINKS, NAMED AFTER THE ACCOUNT. These used to + # land as ~/<name>-shared, one per account, scattered through the + # home directory. ~/Shared/<Name> keeps them together and gives the + # env file below something to enumerate. + _mh = _home_of(main) or "" + link = "" + if _mh: + _hub = os.path.join(_mh, "Shared") + if not os.path.isdir(_hub): + _run(["mkdir", "-p", _hub]) + _run(["chown", f"{main}:{main}", _hub]) + link = os.path.join(_hub, args.name[:1].upper() + args.name[1:]) + if os.path.islink(link): + if os.path.realpath(link) != os.path.realpath(share): + printWarning(f" {link} already points at " + f"{os.path.realpath(link)} -- left alone") + link = "" + elif os.path.exists(link): + # A REAL DIRECTORY IS NOT OURS TO REPLACE. + printWarning(f" {link} exists and is not a symlink -- " + f"left alone") + printInfo(" move it aside yourself if you want the link") + link = "" + else: + _run(["ln", "-s", share, link]) + printInfo(f" shared directory: {share}") + if link: + printInfo(f" reachable from your account as {link}") + _old = os.path.join(_mh, f"{args.name}-shared") if _mh else "" + if _old and os.path.islink(_old): + printInfo(f" (the older link {_old} is still there; remove it " + f"when you are ready)") # 6. a launcher and a menu entry, when the program is really there app = getattr(args, "app", None) or getattr(args, "name", None) \ @@ -10736,99 +9593,9 @@ def _home_is_untouched(home): return True -def _shared_hub(main_home=""): - """Where the per-account links live -- the system hub, not a home - (1.14.312; the argument is kept for the callers and ignored).""" - return shared_hub() - - -def _ensure_shared_hub(main): - """Create the hub, and make the main user's ~/Shared a link to it. - - MOVING, NOT COPYING, AND ONLY OUR LINKS. A ~/Shared that is already a - directory has the links earlier releases made in it; each one moves to - the hub (same name, same target). Anything else in it -- a file, a - directory, a link of your own that clashes with one in the hub -- is - yours, so the directory stays, and says so. Only an EMPTY ~/Shared is - replaced by the link. - """ - hub = shared_hub() - if not os.path.isdir(hub): - _run(["mkdir", "-p", hub]) - # root owns it, your login's group may add a link by hand (0775): the - # hub is a system path that YOU curate, and nobody else may - grp_name = main if (main and _group_exists(main)) else "root" - _run(["chown", f"root:{grp_name}", hub]) - _run(["chmod", "0775" if grp_name != "root" else "0755", hub]) - mh = _home_of(main) if main else None - if not mh: - return hub - home_link = os.path.join(mh, "Shared") - if os.path.islink(home_link): - if os.path.realpath(home_link) != os.path.realpath(hub): - printWarning(f" {home_link} points at {os.readlink(home_link)}, " - f"not the hub {hub} -- left alone") - return hub - if os.path.isdir(home_link): - moved, kept = 0, [] - for n in sorted(os.listdir(home_link)): - p = os.path.join(home_link, n) - dest = os.path.join(hub, n) - if not os.path.islink(p): - kept.append(n) - continue - t = os.readlink(p) - if os.path.lexists(dest): - if os.path.islink(dest) and os.readlink(dest) == t: - os.unlink(p) # already moved on an earlier run - continue - kept.append(n) - continue - os.symlink(t, dest) - os.lchown(dest, 0, 0) - os.unlink(p) - moved += 1 - if moved: - printInfo(f" moved {moved} link(s) from {home_link} to {hub}") - if kept: - printWarning(f" {home_link} still holds {', '.join(kept[:5])}" - f"{' ...' if len(kept) > 5 else ''} -- yours, so it " - f"stays a directory") - printInfo(f" the shared links are in {hub}; move those by hand, " - f"then run this again") - return hub - try: - os.rmdir(home_link) - except OSError: - return hub - os.symlink(hub, home_link) - try: - _pw = pwd.getpwnam(main) - os.lchown(home_link, _pw.pw_uid, _pw.pw_gid) - except (KeyError, OSError): - pass - printInfo(f" {home_link} -> {hub}") - return hub - - -def _remove_empty_hub(mh): - """The hub goes when nothing is left in it -- and ~/Shared, the link to - it, goes with it, or it dangles.""" - hub = shared_hub() - if not (hub and os.path.isdir(hub) and not os.listdir(hub)): - return - try: - os.rmdir(hub) - printInfo(f" removed {hub} (empty)") - except OSError: - return - hl = os.path.join(mh, "Shared") if mh else "" - if hl and os.path.islink(hl) and not os.path.exists(hl): - try: - os.unlink(hl) - printInfo(f" removed {hl} (it pointed at the hub)") - except OSError: - pass +def _shared_hub(main_home): + """The main user's ~/Shared, where the per-account links live.""" + return os.path.join(main_home, "Shared") if main_home else "" def _stale_shared_links(main_home, home=""): @@ -10980,24 +9747,15 @@ def cmd_user_delete(args): printInfo(f" removed {home}") except OSError as e: printError(f" rmtree {home}: {e}") - # ITS SHARED DIRECTORY IS NOT IN ITS HOME ANY MORE (1.14.315), so the - # archive above does not carry it. Empty: removed. With files: kept, - # and said -- those are files you put there, not the account's. - _bare = u[len(user_prefix()):] - _sd = os.path.join(shared_hub(), _bare[:1].upper() + _bare[1:]) - if os.path.isdir(_sd) and not os.path.islink(_sd): - if not os.listdir(_sd): - try: - os.rmdir(_sd) - printInfo(f" removed {_sd} (empty)") - except OSError: - pass - else: - printWarning(f" kept {_sd} -- it has files in it; look, then:" - f" rm -r {_sd}") - # the hub is ours only while it holds our links: an empty hub is - # left-over furniture, one with something else in it is yours. - _remove_empty_hub(mh) + # the hub is ours only while it holds our links: an empty ~/Shared is + # left-over furniture, a ~/Shared with something else in it is yours. + hub = _shared_hub(mh) + if hub and os.path.isdir(hub) and not os.listdir(hub): + try: + os.rmdir(hub) + printInfo(f" removed {hub} (empty)") + except OSError: + pass printSuccess(f"{u} removed.") _shared_vars_note(links) printInfo(" take its lines out of /etc/pkgusr/stacks/app-users.conf, " @@ -11141,7 +9899,13 @@ def cmd_user_purge(args): printInfo(f" removed {p}") except OSError as e: printWarning(f" {p}: {e}") - _remove_empty_hub(mh) + hub = _shared_hub(mh) + if hub and os.path.isdir(hub) and not os.listdir(hub): + try: + os.rmdir(hub) + printInfo(f" removed {hub} (empty)") + except OSError: + pass idir = os.path.join(mh, ".local", "share", "icons", "lfs-apps") if mh else "" if idir and os.path.isdir(idir) and not os.listdir(idir): try: @@ -11212,318 +9976,6 @@ def cmd_user_purge(args): return 0 -# =========================================================================== # -# command: service -- the SysV services, without remembering the layout -# =========================================================================== # -# -# A SysV service is two things in two places: /etc/rc.d/init.d/<name>, which -# starts it, and the S/K symlinks in /etc/rc.d/rc<N>.d, which decide whether -# boot does. Everything here is `ls`, `ln -s` and `rm` on those -- no daemon, -# no database, nothing this tool has to keep in step with the system. -# -# START IS NOT ENABLE. systemd taught everyone that they are different, and -# SysV has always had them separate; `service sshd start` runs the script now -# whatever the links say, and `enable` only touches the links. - -SVC_STATE_DIR = "/var/lib/pkgusr/services" -SVC_ACTIONS = ("start", "stop", "restart", "reload", "force-reload", "status", - "enable", "disable", "list") - - -def _svc_dirs(): - """(init.d, the directory holding rc<N>.d) -- LFS's layout, then the - plain one, because only one of them exists on a given machine.""" - for initd, rcroot in (("/etc/rc.d/init.d", "/etc/rc.d"), ("/etc/init.d", "/etc")): - if os.path.isdir(initd): - return initd, rcroot - return None, None - - -# NOT EVERY FILE IN init.d IS A SERVICE. `functions` is a library the others -# source, `rc` is what init runs, `template` is a skeleton to copy. Listing -# them invites starting them. -_SVC_NOT_SERVICES = {"functions", "rc", "rcS", "template", "README", "Makefile"} - - -def _svc_all(initd): - out = [] - try: - names = sorted(os.listdir(initd)) - except OSError: - return out - for n in names: - if n in _SVC_NOT_SERVICES or n.endswith((".bak", "~", ".orig", ".new")): - continue - p = os.path.join(initd, n) - if os.path.isfile(p) and os.access(p, os.X_OK): - out.append(n) - return out - - -def _svc_links(rcroot, name): - """Every rc<N>.d link for this service: [(level, 'S'|'K', num, path)].""" - out = [] - try: - entries = sorted(os.listdir(rcroot)) - except OSError: - return out - for d in entries: - m = re.match(r"^rc([0-6S])\.d$", d) - if not m: - continue - full = os.path.join(rcroot, d) - try: - for f in sorted(os.listdir(full)): - lm = re.match(r"^([SK])(\d{2})" + re.escape(name) + r"$", f) - if lm: - out.append((m.group(1), lm.group(1), lm.group(2), - os.path.join(full, f))) - except OSError: - continue - return out - - -def _svc_default_runlevel(): - try: - for line in open("/etc/inittab"): - m = re.match(r"^\s*id:([0-6]):initdefault:", line) - if m: - return m.group(1) - except OSError: - pass - return "3" - - -def _svc_header_levels(initd, name): - """Default-Start / Default-Stop from the script's INIT INFO block. - - The book's scripts carry it, so the runlevels a service belongs in are - written down on the machine -- no need to invent them here.""" - start, stop = "", "" - try: - for line in open(os.path.join(initd, name), errors="replace"): - m = re.match(r"^#\s*Default-Start:\s*(.*)$", line) - if m: - start = "".join(re.findall(r"[0-6S]", m.group(1))) - m = re.match(r"^#\s*Default-Stop:\s*(.*)$", line) - if m: - stop = "".join(re.findall(r"[0-6S]", m.group(1))) - if line.startswith("### END INIT INFO"): - break - except OSError: - pass - return start, stop - - -def _svc_enabled_levels(rcroot, name): - return sorted({lv for lv, kind, _n, _p in _svc_links(rcroot, name) if kind == "S"}) - - -def _svc_running(initd, name, quick=True): - """'yes' / 'no' / '?' -- asked of the script itself. - - The LSB convention is 0 for running and 1-4 for not, which every script - from the book follows through statusproc. A script with no `status` says - so by failing differently, and '?' is an honest answer.""" - p = os.path.join(initd, name) - try: - r = subprocess.run([p, "status"], stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, timeout=10 if quick else 60) - except (OSError, subprocess.SubprocessError): - return "?", "" - out = (r.stdout or b"").decode("utf-8", "replace").strip() - if r.returncode == 0: - return "yes", out - if r.returncode in (1, 2, 3, 4): - return "no", out - return "?", out - - -def _svc_state_file(name): - return os.path.join(SVC_STATE_DIR, name + ".links") - - -def cmd_service(args): - initd, rcroot = _svc_dirs() - if not initd: - printError("no /etc/rc.d/init.d (nor /etc/init.d) -- no SysV services here") - return 1 - # `service sshd start` and `service start sshd` both read naturally; - # whichever word is an action is the action. - name, action = args.name, args.action - if name in SVC_ACTIONS and (action is None or action not in SVC_ACTIONS): - name, action = action, name - if not name and not action: - action = "list" - if name and not action: - action = "status" - if action not in SVC_ACTIONS: - printError("%s: not one of %s" % (action, ", ".join(SVC_ACTIONS))) - return 2 - - if action == "list": - return _svc_cmd_list(initd, rcroot, args) - - if not name: - printError("which service? packagemanager service list") - return 2 - script = os.path.join(initd, name) - if not os.path.isfile(script): - printError("%s: no such service" % name) - near = [s for s in _svc_all(initd) if name in s] - if near: - printInfo(" did you mean: %s" % ", ".join(near)) - printInfo(" what there is: packagemanager service list") - return 1 - - if action in ("start", "stop", "restart", "reload", "force-reload", "status"): - # STARTING IS NOT ENABLING, and it does not ask the links for - # permission: this is the whole point of being able to run sshd for an - # afternoon without it coming back at the next boot. - if action == "start" and not _svc_enabled_levels(rcroot, name): - printInfo(" %s is not enabled at boot -- starting it anyway" % name) - if not os.access(script, os.X_OK): - printWarning(" %s is not executable -- fixing that first" % script) - _run(["chmod", "755", script]) - return subprocess.run([script, action]).returncode - - # ---- enable / disable: the links, and only the links ------------------- - links = _svc_links(rcroot, name) - if action == "disable": - gone = [p for _lv, kind, _n, p in links if kind == "S"] - if not gone: - printInfo("%s is already not started at boot" % name) - return 0 - # REMEMBER EXACTLY WHAT WAS THERE. The priority number is the only - # thing that cannot be worked out again -- it lives in the - # blfs-bootscripts Makefile, which may be long gone by then. - try: - os.makedirs(SVC_STATE_DIR, exist_ok=True) - with open(_svc_state_file(name), "w") as f: - f.write("\n".join(gone) + "\n") - except OSError as e: - printWarning(" could not record the links (%s) -- enable will " - "rebuild them from the script's header" % e) - for p in gone: - try: - os.unlink(p) - printInfo(" removed %s" % p) - except OSError as e: - printWarning(" %s: %s" % (p, e)) - printSuccess("%s: disabled (it is not stopped -- `service %s stop` does that)" - % (name, name)) - return 0 - - # enable - if _svc_enabled_levels(rcroot, name): - printInfo("%s is already started at boot (runlevels %s)" - % (name, ",".join(_svc_enabled_levels(rcroot, name)))) - return 0 - # 1. the exact links we removed, if we removed them - restored = [] - try: - saved = [l.strip() for l in open(_svc_state_file(name)) if l.strip()] - except OSError: - saved = [] - for p in saved: - d = os.path.dirname(p) - if not os.path.isdir(d): - continue - try: - os.symlink("../init.d/" + name, p) - restored.append(p) - except OSError as e: - printWarning(" %s: %s" % (p, e)) - if restored: - for p in restored: - printInfo(" %s -> ../init.d/%s" % (p, name)) - try: - os.unlink(_svc_state_file(name)) - except OSError: - pass - printSuccess("%s: enabled (restored as it was)" % name) - return 0 - # 2. otherwise build them: the number from a link that survived, the - # runlevels from the script's own header. - num = args.priority or next((n for _lv, _k, n, _p in links), None) or "50" - start, stop = _svc_header_levels(initd, name) - if args.levels: - start = "".join(re.findall(r"[0-6S]", args.levels)) - start = start or "2345" - stop = stop or "016" - made = [] - for lv in start: - d = os.path.join(rcroot, "rc%s.d" % lv) - if not os.path.isdir(d): - continue - p = os.path.join(d, "S%s%s" % (num, name)) - if not os.path.lexists(p): - try: - os.symlink("../init.d/" + name, p) - made.append(p) - except OSError as e: - printWarning(" %s: %s" % (p, e)) - for lv in stop: - d = os.path.join(rcroot, "rc%s.d" % lv) - if not os.path.isdir(d): - continue - p = os.path.join(d, "K%s%s" % (num, name)) - if not os.path.lexists(p): - try: - os.symlink("../init.d/" + name, p) - made.append(p) - except OSError as e: - printWarning(" %s: %s" % (p, e)) - if not made: - printWarning("%s: nothing to link (no rc<N>.d directories?)" % name) - return 1 - for p in made: - printInfo(" %s -> ../init.d/%s" % (p, name)) - printSuccess("%s: enabled at runlevels %s (priority %s)" % (name, start, num)) - printInfo(" it is not started yet: packagemanager service %s start" % name) - return 0 - - -def _svc_cmd_list(initd, rcroot, args): - names = _svc_all(initd) - if not names: - printInfo("no services in %s" % initd) - return 0 - default_lv = _svc_default_runlevel() - printHeader("services (init scripts in %s, boot links in %s/rc<N>.d)" - % (initd, rcroot)) - print(" default runlevel: %s\n" % default_lv) - print(" %-22s %-10s %-9s %s" % ("service", "at boot", "running", "runlevels")) - print(" " + "-" * 60) - for n in names: - lv = _svc_enabled_levels(rcroot, n) - boot = "yes" if default_lv in lv else ("other" if lv else "no") - run = "-" - if not args.quick: - run, _out = _svc_running(initd, n) - print(" %-22s %-10s %-9s %s" % (n, boot, run, ",".join(lv) or "-")) - print() - printInfo(" packagemanager service <name> start|stop|restart|status") - printInfo(" packagemanager service <name> enable|disable (boot only)") - if args.quick: - printInfo(" (--quick: 'running' not asked; drop it to ask each script)") - # THE SERVICES THAT ARE NOT INIT SCRIPTS. /lib/services/* are what `ifup` - # hands an interface to; they never appear in rc<N>.d and cannot be - # started, and not saying so is how "dhcpcd is installed but nothing runs - # it" happens. - for sd in ("/lib/services", "/usr/lib/services"): - if os.path.isdir(sd): - have = sorted(os.listdir(sd)) - if have: - print() - printInfo(" network services in %s (run by ifup, never on their " - "own): %s" % (sd, ", ".join(have))) - printInfo(" an interface uses one by naming it: SERVICE=\"<name>\"" - " in /etc/sysconfig/ifconfig.<iface>") - break - return 0 - - # =========================================================================== # # command: template [IMPLEMENTED] # =========================================================================== # @@ -12385,8 +10837,7 @@ def _system_env_values(): home = _home_of(main) if main else None if home: vals["PKGUSR_SYS_HOME"] = home - # the hub, whoever asks -- not a path inside one person's home - vals["PKGUSR_SYS_SHARED"] = shared_hub() + vals["PKGUSR_SYS_SHARED"] = os.path.join(home, "Shared") return vals @@ -12564,10 +11015,6 @@ def cmd_env(args): # # File format, one entry per line: # book <target> [key=value ...] a BLFS book package (deps resolved) -# <any> ... when="<shell>" only when that command succeeds -- -# a package this machine needs only -# sometimes (btrfs-progs on a btrfs -# system). It is asked every run. # git <name> url=<repo> [ref=<ref>] [prog=<installed-program>] # tar <name> url=<tarball-url> pkg=<filename> [prog=|have=] [build=|install=] # check <name> root script <stackdir>/<name>.sh; @@ -12638,12 +11085,7 @@ def _parse_stack_file(path): worse than one refused.""" import shlex entries = [] - kinds = ("book", "git", "tar", "check", "cfg", "avoid", "include", - "from-book") - # `from-book svn`: the book entries of THIS file below it read that book - # (an entry's own from-book= wins; `from-book default` ends it). Not - # inherited by an included file, and not passed up to the includer. - _file_book = None + kinds = ("book", "git", "tar", "check", "cfg", "avoid", "include") # THE "INCLUDE ONCE" SET BELONGS TO THE WHOLE PARSE, not to one include # line. Owning it per line meant sway.stack's `include avoid` finished # and cleared it, so `include media` started fresh and media's own @@ -12703,12 +11145,6 @@ def _parse_stack_file(path): # `include <name>` splices another stack's entries in at this # point, in order. The included file is a stack in its own right, # so `packagemanager stack media --run` builds just that part. - if kind == "from-book": - if len(parts) != 2: - raise ValueError("%s:%d: from-book takes one book selector" - " (svn, 13.0-systemd, a file)" % (path, n)) - _file_book = None if parts[1] in ("default", "-") else parts[1] - continue if kind == "include": if len(parts) != 2: raise ValueError("%s:%d: include takes one stack name" @@ -12770,10 +11206,6 @@ def _parse_stack_file(path): raise ValueError("%s:%d: a git entry needs url=" % (path, n)) if kind == "tar" and ("url" not in opts or "pkg" not in opts): raise ValueError("%s:%d: a tar entry needs url= and pkg=" % (path, n)) - if kind == "book" and _file_book: - opts.setdefault("from-book", _file_book) - if opts.get("from-book") in ("default", "-"): - del opts["from-book"] entries.append({"kind": kind, "target": target, "opts": opts}) if not entries: raise ValueError("%s: the stack file has no entries" % path) @@ -13166,32 +11598,6 @@ def cmd_stack(args): avoid = [e["target"] for e in entries if e["kind"] == "avoid"] entries = [e for e in entries if e["kind"] != "avoid"] - if getattr(args, "avoided", False): - # the list the header only counts - printHeader("never built by stack %s (%d)" % (args.name, len(avoid))) - for a in sorted(set(avoid), key=str.lower): - print(" " + a) - return - # THE BOOKS THIS STACK READS. Its from-book entries become the - # packages' book for this run and for every install it starts; the - # book itself is refreshed once here (svn moves daily), and every - # lookup after that reads the file on disk. - _sb = {e["target"]: e["opts"]["from-book"] for e in entries - if e["kind"] == "book" and e["opts"].get("from-book")} - if _sb: - _stack_book_map().update({k.lower(): v for k, v in _sb.items()}) - _BOOK_FOR_PKG.clear() - os.environ["PM_STACK_BOOKS"] = ",".join( - "%s=%s" % kv for kv in sorted(_sb.items())) - for _sel in sorted(set(_sb.values())): - _bp = _book_path_for(_sel, refresh=not getattr(args, "status", - False)) - if not getattr(args, "status", False): - printInfo(" book %s: %d entr%s (%s)" - % (_sel, sum(1 for v in _sb.values() if v == _sel), - "y" if list(_sb.values()).count(_sel) == 1 - else "ies", - os.path.basename(_bp) if _bp else "NOT ON DISK")) if getattr(args, "no_git", False): # git and tar entries are the extras -- cloned tools, fonts. Skipping # them finishes the book part of the desktop first, and matters while @@ -13226,11 +11632,7 @@ def cmd_stack(args): # stack" and then passed nothing to the installer, so the # packages were built anyway. It is the stack's exclusion list # now, handed to every entry's resolve. - # A COUNT, NOT THE LIST: eighty names above every plan pushed - # the plan itself off the screen. - print(" %d package(s) never built by this stack " - "(list: packagemanager stack %s --avoided)\n" - % (len(set(avoid)), args.name)) + print(" never built by this stack: %s\n" % ", ".join(avoid)) print(" file: %s\n" % path) import shutil as _sh @@ -13304,28 +11706,6 @@ def cmd_stack(args): entries = [e for e in entries if e["target"] in _only] printInfo("running only: %s" % ", ".join(e["target"] for e in entries)) print() - # WHEN=: AN ENTRY THIS MACHINE DOES NOT NEED. - # - # btrfs-progs belongs in the stack of a btrfs machine and nowhere else, - # and a second stack file per filesystem is not an answer. The test is - # a shell command, run here, every time -- so a disk added later is - # picked up without editing anything. - _when_out = [] - _kept = [] - for e in entries: - _w = e["opts"].get("when") - if _w and subprocess.run(["/bin/bash", "-c", _w], - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL).returncode != 0: - _when_out.append((e["target"], _w)) - continue - _kept.append(e) - entries = _kept - if _when_out and not getattr(args, "status", False): - for _t, _w in _when_out: - printInfo(" %s: not for this machine -- when=%s" % (_t, _w)) - print() - rows = [] for e in entries: e["_key"] = "%s %s" % (e["kind"], e["target"]) @@ -13589,7 +11969,7 @@ def cmd_stack(args): # `needs=` closes that: the named packages are installed -- # through the same door as everything else -- before the entry # runs. - for _need in [_resolve_need(x.strip()) for x in + for _need in [x.strip() for x in (e["opts"].get("needs") or "").replace(",", " ").split() if x.strip()]: if gather_user(_need).state == "installed": @@ -13950,198 +12330,6 @@ def cmd_reclaim(args): return rc -def _setid_table(): - """setid.conf as {path: (mode, owner, group, package)}. - - The 5th column (package) is the one this command is here for: it is the - only written-down statement of WHICH package installs a setuid file, and - a setuid file held by any other package is damage, not history. - """ - out = {} - for d in (os.environ.get("PM_STACK_DIR") or "/etc/pkgusr/stacks", - os.path.join(os.path.dirname(os.path.abspath(__file__)), "stacks")): - conf = os.path.join(d, "setid.conf") - if not os.path.isfile(conf): - continue - try: - with open(conf, errors="replace") as f: - for line in f: - line = line.strip() - if not line or line.startswith("#"): - continue - col = [c.strip() for c in line.split("|")] - if len(col) < 2 or not col[0].startswith("/"): - continue - col += [""] * (5 - len(col)) - out[col[0]] = (col[1], col[2] or "root", col[3] or "-", col[4]) - except OSError: - continue - break - return out - - -def cmd_disown(args): - """Take back the files a package cannot have installed. - - `reclaim` is the other direction: it pushes ownership TOWARDS pkg.lst, - which is right when a package lost files and wrong when the record is - what went bad. This one distrusts the record, and it exists because the - record did go bad -- three times, the same way: - - -rwx--x--x 1 p_webkitgtk p_webkitgtk /usr/bin/su - lrwxrwxrwx 1 p_webkitgtk p_webkitgtk /usr/sbin/iptables -> xtables-nft-multi - - Neither is webkitgtk's. They were swept into its manifest because - attribution is by TIME WINDOW and a webkitgtk build runs for hours, and - the lease then acted on that manifest -- clearing su's setuid bit on the - way, because `chown` does that and says nothing. 1.14.310 shuts both - doors; this repairs machines that already went through them. - - Two rules, and nothing is guessed from either: - - * setid.conf names the package that installs each setuid file. Anyone - else holding it gives it back to root, with the mode from the table. - * a symlink is installed by whoever installed its target. A link whose - target belongs to another package goes back to that package. - - Whatever is handed back is struck from the package's pkg.lst as well: - an ownership fix that leaves the record claiming the file is a fix that - undoes itself on the next `--clean`, which would DELETE these paths. - """ - if args.run and os.geteuid() != 0: - printError("disown changes ownership; run it as root.") - return 1 - - setid = _setid_table() - if not setid: - printWarning("no setid.conf found -- only the symlink rule can run") - - if getattr(args, "all", False): - names = sorted(_pkg_dirs()) - elif args.packages: - names = args.packages - else: - printError("name a package, or --all") - return 2 - - accounts = set() - for root in pkgusr_roots(): - try: - accounts.update(os.listdir(root)) - except OSError: - pass - - def owner_of(path, follow=False): - try: - st = os.stat(path) if follow else os.lstat(path) - return pwd.getpwuid(st.st_uid).pw_name - except (KeyError, OSError): - return None - - rc, total = 0, 0 - for name in names: - acct = pkgusr_name(name) - pkg = re.sub(r"^" + re.escape(pkgusr_prefix()), "", acct) - lst = os.path.join(pkgusr_home(acct), "pkg.lst") - try: - paths = [l.strip() for l in open(lst, errors="replace") if l.strip()] - except OSError as e: - if getattr(args, "all", False): - continue - printError("%s: cannot read %s (%s)" % (acct, lst, e)) - rc = 1 - continue - - wrong = [] # (path, reason, give-to, mode-or-None) - for path in paths: - if owner_of(path) != acct: - continue - pol = setid.get(path) - if pol and pol[3] != pkg: - who = pol[3] or "no package" - wrong.append((path, "setid.conf says %s installs it" % who, - "root", pol[0])) - continue - if os.path.islink(path): - tgt = os.path.realpath(path) - to = owner_of(tgt, follow=True) - if to and to != acct and to != "root" and to in accounts: - wrong.append((path, "points at %s's %s" % (to, tgt), to, None)) - - if not wrong: - if not getattr(args, "all", False): - printSuccess("%s: nothing it could not have installed (%d checked)" - % (acct, len(paths))) - continue - - total += len(wrong) - printWarning("%s: %d recorded path(s) belong somewhere else" % (acct, len(wrong))) - for path, why, to, _m in wrong: - printInfo(" %-46s -> %-12s %s" % (path, to, why)) - if not args.run: - printInfo(" Dry run -- re-run with --run to hand them back.") - continue - - done = [] - for path, _why, to, mode in wrong: - try: - if to == "root": - os.lchown(path, 0, 0) - # THE MODE COMES BACK TOO. The chown that took the file - # cleared its setuid bit; handing it to root without the - # bit leaves `su` exactly as broken as it was. - if mode and not os.path.islink(path): - os.chmod(path, int(mode, 8)) - else: - os.lchown(path, pwd.getpwnam(to).pw_uid, grp.getgrnam(to).gr_gid) - done.append(path) - except (OSError, KeyError) as e: - printError(" could not hand back %s (%s)" % (path, e)) - rc = 1 - - # ...AND OUT OF THE RECORD, or `update <pkg> --clean` deletes them. - if done: - keep = [p for p in paths if p not in set(done)] - try: - with open(lst + ".new", "w") as f: - f.write("\n".join(keep) + ("\n" if keep else "")) - os.replace(lst + ".new", lst) - except OSError as e: - printError(" %s: could not rewrite pkg.lst (%s)" % (acct, e)) - rc = 1 - # lfs-helper's build manifests: $LFS_PKGUSR_DIR/manifests/<pkg>.files, - # which defaults to /usr/src/lfs-pkgusr. It is the list the LEASE - # reads, so leaving the path in it re-arms the whole thing. - _state = (os.environ.get("LFS_PKGUSR_DIR") - or os.path.join(os.path.dirname(BASE_DIR.rstrip("/")) or "/usr/src", - "lfs-pkgusr")) - for man in dict.fromkeys( - [os.path.join(_state, "manifests", pkg + ".files"), - os.path.join("/usr/src/lfs-pkgusr/manifests", pkg + ".files")]): - if not os.path.isfile(man): - continue - try: - lines = [l.rstrip("\n") for l in open(man, errors="replace")] - left = [l for l in lines if l.strip() not in set(done)] - if len(left) != len(lines): - with open(man + ".new", "w") as f: - f.write("\n".join(left) + ("\n" if left else "")) - os.replace(man + ".new", man) - printInfo(" %s: %d line(s) removed" % (man, len(lines) - len(left))) - except OSError as e: - printError(" %s (%s)" % (man, e)) - rc = 1 - printSuccess("%s: %d path(s) handed back" % (acct, len(done))) - - if total == 0: - printSuccess("nothing to hand back") - elif not args.run: - printInfo("") - printInfo(" After --run, put the setuid bits back and check them:") - printInfo(" lfs-helper setid --run") - return rc - - def cmd_which_package(args): """Which package installed this path. @@ -14487,9 +12675,6 @@ commands, by what you are doing: plan what a run would do, before doing it reload-pkg-list refresh the cached package list reclaim <pkg> | --all give a package back files another took from it - disown <pkg> | --all the other direction: take back what a package - cannot have installed (setuid files, other - packages' symlinks) and clean its pkg.lst which-package <path> which package installed it blfs ... pass a command straight to blfs @@ -14835,9 +13020,6 @@ def main(): p.add_argument("--recursive", action="store_true", help="also resolve and build the package's dependencies " "(default: just the named package(s))") - p.add_argument("--update-deps", action="store_true", - help="also rebuild dependencies that are installed but " - "older than the book (default: leave them alone)") p.add_argument("-f", "--force", action="store_true", help="(deprecated: non-recursive is now the default; kept for " "compatibility)") @@ -14856,9 +13038,6 @@ def main(): p.add_argument("--clean", action="store_true", help="after install, remove stale files from a previous " "version (configs and the source tree kept)") - p.add_argument("--no-git-check", action="store_true", - help="do not ask the git remotes whether a branch moved " - "(no network)") p.add_argument("--regenerate", action="store_true", help="rebuild scripts from the book, ignoring any edited " "script in the package user's home") @@ -14956,9 +13135,6 @@ def main(): p.set_defaults(func=cmd_remove) p = sub.add_parser("reload-pkg-list", help="regenerate /usr/src/<name>/pkg.lst") - p.add_argument("--root", action="store_true", - help="scan as root: slower, but reaches directories the " - "package user cannot enter") p.add_argument("packages", nargs="+") p.add_argument("--background", action="store_true", help="run detached and return at once (the old default)") @@ -15049,8 +13225,6 @@ def main(): p.add_argument("--check-refs", action="store_true", help="ask each git entry's remote whether its ref= still " "exists, and stop -- builds nothing") - p.add_argument("--avoided", action="store_true", - help="list the packages this stack never builds, and stop") p.set_defaults(func=cmd_stack) p = sub.add_parser("reclaim", @@ -15066,17 +13240,6 @@ def main(): help="actually change ownership (default: dry run)") p.set_defaults(func=cmd_reclaim) - p = sub.add_parser("disown", - help="hand back the files a package cannot have " - "installed (setuid files and other packages' " - "symlinks), and strike them from its pkg.lst") - p.add_argument("packages", nargs="*") - p.add_argument("--all", action="store_true", - help="every installed package, not a named list") - p.add_argument("--run", action="store_true", - help="actually change ownership (default: dry run)") - p.set_defaults(func=cmd_disown) - p = sub.add_parser("which-package", aliases=["which-pkg"], help="which package installed a path (reads pkg.lst)") p.add_argument("paths", nargs="+", metavar="PATH") @@ -15183,10 +13346,6 @@ def main(): p.add_argument("--collector-prefix", help="prefix for shared install-dir groups") p.add_argument("--user-prefix", help="prefix for shared users") p.add_argument("--main-user", help="your human login account (e.g. n76310)") - p.add_argument("--shared-hub", metavar="DIR", - help="where the links to the shared directories live " - "(default /home/Shared); your ~/Shared and each " - "account's ~/Shared link into it") p.add_argument("--editor", help="editor for 'verify --set-validate-command' " "(default: $EDITOR or vim)") p.add_argument("--difftool", help="diff tool for 'migrate-scripts --vimdiff' " @@ -15306,47 +13465,6 @@ def main(): q.add_argument("groups", nargs="+") q.set_defaults(func=cmd_user_disallow) - p = sub.add_parser("git-status", aliases=["git"], - help="what each git package follows (branch or tag) and " - "whether the remote has moved past it") - p.add_argument("packages", nargs="*", help="default: every git package here") - p.add_argument("--local-only", action="store_true", - help="do not ask the remotes (no network)") - p.set_defaults(func=cmd_git_status) - - p = sub.add_parser("git-ref", - help="show the branches and tags a git package could " - "follow, or move it to one") - p.add_argument("package") - p.add_argument("ref", nargs="?", help="omit to list what the remote has") - p.add_argument("--run", action="store_true", - help="actually change git_ref= (default dry-run)") - p.set_defaults(func=cmd_git_ref) - - p = sub.add_parser("used-by", aliases=["needed-by"], - help="what still links against a library, or against " - "this package's libraries (DT_NEEDED, not the " - "book's dependency list)") - p.add_argument("name", help="a package name, a soname (libfoo.so.4) or a " - "path to a library") - p.set_defaults(func=cmd_used_by) - - p = sub.add_parser("service", help="the SysV services: list them, start/stop " - "one, enable/disable it at boot (start does not need " - "enable)") - p.add_argument("name", nargs="?", help="the service, as in /etc/rc.d/init.d/<name>") - p.add_argument("action", nargs="?", - help="list | status | start | stop | restart | reload | " - "enable | disable (either order: `service sshd start` " - "or `service start sshd`; no action means status)") - p.add_argument("--levels", help="enable: the runlevels to start in " - "(default: the script's own Default-Start, else 2345)") - p.add_argument("--priority", help="enable: the two-digit order number " - "(default: the one its other links already use)") - p.add_argument("--quick", action="store_true", - help="list: do not ask each script whether it is running") - p.set_defaults(func=cmd_service) - p = sub.add_parser("init", help="set up the install group + make install dirs " "(dry-run unless --run; shows/confirms each command)") p.add_argument("dirs", nargs="*", help="directories to turn into install dirs") @@ -15503,14 +13621,7 @@ def main(): if (not config_exists() and args.command != "config" and sys.stdin.isatty() and sys.stdout.isatty()): first_run_setup() - # THE EXIT CODE IS AN ANSWER. Every command's return value was thrown - # away here, so `packagemanager` exited 0 whatever happened -- including - # `service <name> status` on a service that is not running, which is the - # one command people put in an `if`. Only a non-zero INT becomes the exit - # status; a command that returns something else (or nothing) is unchanged. - rc = args.func(args) - if isinstance(rc, int) and not isinstance(rc, bool) and rc != 0: - sys.exit(rc) + args.func(args) if __name__ == "__main__": diff --git a/stacks/app-users.conf b/stacks/app-users.conf index 9d7d3dc..2b9333a 100644 --- a/stacks/app-users.conf +++ b/stacks/app-users.conf @@ -15,7 +15,7 @@ # account | name | app | share | dir | desktop | launcher | groups | icon # # account the account, without the prefix: browser -> u_browser. Also -# the shared directory's name, capitalised: /home/Shared/Browser +# the shared directory's name, capitalised: ~/Shared/Browser # name the launcher: ~/bin/<name> and <name>.desktop. Empty: the # account's name # app the command line. Its first word must be on PATH. @@ -37,11 +37,8 @@ # full the desktop plus audio and video -- anything that plays # no no session access # An account takes the strongest share of its lines. -# dir yes /home/Shared/<Account> (2770, the account's group -- -# which your login is in), and $SHARED_<ACCOUNT> in your -# next login shell. The account's ~/Shared and yours -# are links into /home/Shared, so its home can stay -# 0700 (`packagemanager config --shared-hub DIR` moves it) +# dir yes <account home>/Shared, linked as ~/Shared/<Account>, +# and $SHARED_<ACCOUNT> in your next login shell # no nothing shared. What the account downloads stays in the # account -- which is the point of the account # desktop yes a menu entry @@ -71,22 +68,13 @@ # share (see below) # # THE SHELL VARIABLES. /etc/profile.d/shared-dirs.sh exports one per -# link in the hub, e.g. SHARED_BROWSER=/home/Shared/Browser, and drops +# ~/Shared link, e.g. SHARED_BROWSER=/home/<you>/Shared/Browser, and drops # the ones whose directory is gone. /etc/profile.d is read by LOGIN shells: # a terminal opened inside the session is not one, so it shows what the # session inherited when you logged in. The step also sources the fragment # from /etc/bashrc, which every interactive shell reads -- and any shell can # refresh itself with: . /etc/profile.d/shared-dirs.sh # -# EXTRA SETUP PER ACCOUNT. A line starting with @setup runs a command, a -# local script or a downloaded bash script AS the account (its own HOME), -# once. `*` means every account. A changed line runs again; -# APP_USERS_RERUN=1 runs all of them again. `|` is allowed here. -# -# @setup mpv mkdir -p ~/.config/mpv && echo "volume=60" > ~/.config/mpv/mpv.conf -# @setup claude https://example.org/claude-setup.sh -# @setup * /etc/pkgusr/stacks/app-common.sh -# # su WITHOUT A PASSWORD: /etc/pam.d/su_u_user lists the accounts your login # may become; this step MERGES it (backup su_u_user.bak-<date> first). @@ -103,10 +91,8 @@ grok | | browser-mini --app-id grok https://grok.com/ --downloa mercury | | browser-mini --app-id mercury https://chat.inceptionlabs.ai/ --download-dir ~/Shared | slim | yes | yes | yes | - | - deepseek | | browser-mini --app-id deepseek https://chat.deepseek.com/ --download-dir ~/Shared | slim | yes | yes | yes | - | github | | browser-mini --app-id github https://github.com/search?q={https://github.com/} --download-dir ~/Shared | slim | yes | yes | yes | - | -git | | browser-mini --app-id git https://git.christianimmanuel.de --download-dir ~/Shared | slim | yes | yes | yes | - | codeberg | | browser-mini --app-id codeberg https://codeberg.org/explore/repos?q={https://codeberg.org/} --download-dir ~/Shared | slim | yes | yes | yes | - | reddit | | browser-mini --app-id reddit https://www.reddit.com/search/?q={https://www.reddit.com/} | full | no | yes | yes | - | -discord | | browser-big --app-id discord https://discord.com/channels/@me | full | no | yes | yes | - | youtube | | browser-mini --app-id youtube https://www.youtube.com/results?search_query={https://www.youtube.com/} | full | no | yes | yes | - | dokus | arte | browser-mini --app-id arte https://www.arte.tv/de/videos/dokumentationen-und-reportagen/ | full | no | yes | yes | - | https://upload.wikimedia.org/wikipedia/commons/3/3b/Arte-Logo.svg dokus | ard | browser-mini --app-id ard https://www.ardmediathek.de/dokus | full | no | yes | yes | - | https://upload.wikimedia.org/wikipedia/commons/7/7c/ARD_2019_logo.svg diff --git a/stacks/app-users.sh b/stacks/app-users.sh index c314f35..f23c2c8 100644 --- a/stacks/app-users.sh +++ b/stacks/app-users.sh @@ -47,14 +47,6 @@ $PM env --system --run >/dev/null 2>&1 \ && note "refreshed /etc/profile.d/pkgusr-system.sh (PKGUSR_SYS_*)" \ || true -# THE HUB: where the links to every shared directory live (1.14.312). Asked -# of the tool, NOT read from $PKGUSR_SYS_SHARED -- this shell inherited that -# at login, and before 1.14.312 it named ~/Shared. /home/Shared -# unless `packagemanager config --shared-hub DIR` says otherwise. -HUB="$($PM env --system 2>/dev/null | sed -n 's/^export PKGUSR_SYS_SHARED="\(.*\)"$/\1/p' | head -n1)" -[ -n "$HUB" ] || HUB="/home/Shared" -note "shared hub: $HUB" - # CONFIRM THE COMMAND EXISTS BEFORE THE LOOP. Every line would otherwise # fail the same way, three times, with the parser's error rather than ours. if ! $PM user setup --help >/dev/null 2>&1; then @@ -79,25 +71,8 @@ declare -A A_SHARE A_DIR A_GROUPS accounts="" L_ACCT=(); L_NAME=(); L_APP=(); L_DESK=(); L_LAUNCH=(); L_ICON=() format=new -S_ACCT=(); S_CMD=() while IFS= read -r raw || [ -n "$raw" ]; do case "$(trim "$raw")" in ""|\#*) continue ;; esac - # @setup <account|*> <command | URL | /path/script> -- run AS the - # account once (see step 1a). Not split on `|`: a command may pipe. - case "$(trim "$raw")" in - @setup[[:space:]]*) - rest="$(trim "${raw#*@setup}")" - sacct="${rest%%[[:space:]]*}" - scmd="$(trim "${rest#"$sacct"}")" - case "$sacct" in - \*|[a-z0-9_-]*) ;; - *) warn "line '$raw': '$sacct' is not an account -- skipped"; continue ;; - esac - [ -n "$scmd" ] || { warn "line '$raw': @setup without a command -- skipped"; continue; } - S_ACCT+=("$sacct"); S_CMD+=("$scmd") - continue ;; - @*) warn "line '$raw': unknown directive -- skipped"; continue ;; - esac IFS='|' read -r f1 f2 f3 f4 f5 f6 f7 f8 f9 <<< "$raw" f1="$(trim "$f1")" # the header tells the format @@ -173,58 +148,6 @@ done echo [ -n "$names" ] || { warn "no usable lines in $CONF"; exit 1; } - -# ---- 1a. @setup: the account's own extra steps ------------------------------ -# A command, a local script or a URL to a bash script, run AS the account -# (login shell, its own HOME) -- never as root. ONCE: a step that ran is -# recorded by its text in $SETUP_DB/<account>, so a later run skips it and -# a CHANGED line runs again. APP_USERS_RERUN=1 runs them all again. -SETUP_DB="${APP_USERS_SETUP_DB:-/var/lib/pkgusr/app-setup}" -fetch_to() { # url file - if command -v curl >/dev/null 2>&1; then curl -fsSL "$1" -o "$2" - elif command -v wget >/dev/null 2>&1; then wget -q -O "$2" "$1" - else echo "no curl or wget" >&2; return 1; fi -} -if [ "${#S_CMD[@]}" -gt 0 ]; then - mkdir -p "$SETUP_DB" && chmod 700 "$SETUP_DB" - for i in "${!S_CMD[@]}"; do - cmd="${S_CMD[$i]}" - if [ "${S_ACCT[$i]}" = "*" ]; then targets="$names"; else targets="${S_ACCT[$i]}"; fi - key="$(printf '%s' "$cmd" | md5sum | cut -c1-12)" - for acct in $targets; do - case " $names " in *" $acct "*) ;; *) warn "@setup $acct: no such account in $CONF -- skipped"; continue ;; esac - db="$SETUP_DB/$acct" - if [ "${APP_USERS_RERUN:-0}" != 1 ] && grep -qx "$key" "$db" 2>/dev/null; then - note "$acct: setup already done -- $cmd" - continue - fi - note "$acct: setup -- $cmd" - case "$cmd" in - http://*|https://*) - tmp="$(mktemp /tmp/app-setup.XXXXXX)" - if ! fetch_to "$cmd" "$tmp"; then - rm -f "$tmp"; warn "$acct: could not download $cmd"; continue - fi - chmod 644 "$tmp" - su - "$PREFIX$acct" -s /bin/bash -c "bash '$tmp'"; rc=$? - rm -f "$tmp" ;; - /*) - if [ -f "$cmd" ]; then - su - "$PREFIX$acct" -s /bin/bash -c "bash '$cmd'"; rc=$? - else - su - "$PREFIX$acct" -s /bin/bash -c "$cmd"; rc=$? - fi ;; - *) su - "$PREFIX$acct" -s /bin/bash -c "$cmd"; rc=$? ;; - esac - if [ "$rc" = 0 ]; then - grep -qx "$key" "$db" 2>/dev/null || echo "$key" >> "$db" - else - warn "$acct: setup failed (exit $rc) -- runs again next time: $cmd" - fi - done - done - echo -fi # TRIM ONCE, HERE. $names is built by appending " $n", so it starts with a # space and ${names%% *} is empty -- which printed `su - u_` twice before, # in two different blocks. One trim, at the top, and every later use is @@ -233,7 +156,7 @@ names="$(printf '%s' "$names" | sed 's/^ *//')" # ---- 1b. a line that STOPPED asking for a shared directory ----------------- # `user setup` makes what the line asks for; it never takes away, so an -# account switched from dir=yes to dir=no keeps its /home/Shared/<Name> -- +# account switched from dir=yes to dir=no keeps its ~/Shared/<Name> link -- # and /etc/profile.d/shared-dirs.sh keeps exporting $SHARED_<NAME> for it. # An EMPTY leftover is removed here (nothing can be lost). One with files in # it is yours: this says where it is and what would remove it. @@ -241,27 +164,15 @@ MAIN_HOME="$(getent passwd "$MAIN" | cut -d: -f6)" for name in $accounts; do [ "${A_DIR[$name]:-}" = yes ] && continue cap="$(printf '%s' "$name" | cut -c1 | tr '[:lower:]' '[:upper:]')$(printf '%s' "$name" | cut -c2-)" - d="$HUB/$cap" - # since 1.14.315 the hub holds the DIRECTORY and the account's ~/Shared - # is the link; before that, the hub held a link. Either is handled. - ah="$(getent passwd "$PREFIX$name" | cut -d: -f6)" - if [ -L "$d" ]; then - tgt="$(readlink -f "$d" 2>/dev/null || true)" - elif [ -d "$d" ]; then - tgt="$d" - else - continue - fi + link="$MAIN_HOME/Shared/$cap" + [ -L "$link" ] || continue + tgt="$(readlink -f "$link" 2>/dev/null || true)" if [ ! -d "$tgt" ] || [ -z "$(ls -A "$tgt" 2>/dev/null)" ]; then - [ -L "$d" ] && rm -f "$d" - [ -d "$tgt" ] && rmdir "$tgt" 2>/dev/null - note "$name: dir=no -- removed the empty $d" - if [ -n "$ah" ] && [ -L "$ah/Shared" ] && [ ! -e "$ah/Shared" ]; then - rm -f "$ah/Shared" && note " and the link $ah/Shared" - fi + rm -f "$link" && note "$name: dir=no -- removed the empty $link" + [ -d "$tgt" ] && rmdir "$tgt" 2>/dev/null && note " and $tgt" else - note "$name: dir=no, but $d still has files in it -- left alone" - note " look, then remove it yourself" + note "$name: dir=no, but $link still has files in it -- left alone" + note " look, then: rm $link (the files stay in $tgt)" fi done echo @@ -417,21 +328,18 @@ fi echo # ---- 3. the shell variables ---------------------------------------------- -# ENUMERATE, DO NOT LIST. The fragment reads the hub at login, so an +# ENUMERATE, DO NOT LIST. The fragment reads ~/Shared at login, so an # account added to the conf file shows up in the next shell without this # file being rewritten -- and a link you make by hand appears too. frag="$PROFILE_D/shared-dirs.sh" if [ -d "$PROFILE_D" ]; then if [ -e "$frag" ] && ! grep -qF "$MARKER" "$frag"; then warn "$frag exists and is not ours -- left alone" - note " it would have exported SHARED_<NAME> for each $HUB/<Name>" + note " it would have exported SHARED_<NAME> for each ~/Shared/<Name>" else cat > "$frag" <<PROF $MARKER -# One variable per shared directory: $HUB/Claude -> \$SHARED_CLAUDE. -# The SAME set in every account: the hub is a system path, not \$HOME/Shared -# (which in an app account is its own data directory -- enumerating that -# exported one SHARED_* per FILE in it, before 1.14.312). +# One variable per shared directory: ~/Shared/Claude -> \$SHARED_CLAUDE. # Enumerated, not listed, so an account added or removed needs no change # here -- and a link you make by hand appears too. # @@ -443,8 +351,8 @@ for _v in \$(env | sed -n 's/^\(SHARED_[A-Z0-9_]*\)=.*/\1/p'); do eval "_p=\\\${\$_v}" [ -d "\$_p" ] || unset "\$_v" done -if [ -d "$HUB" ]; then - for _sd in "$HUB"/*; do +if [ -d "\$HOME/Shared" ]; then + for _sd in "\$HOME"/Shared/*; do [ -e "\$_sd" ] || continue _n="\$(basename "\$_sd" | tr '[:lower:]-' '[:upper:]_')" export "SHARED_\$_n=\$_sd" @@ -503,7 +411,7 @@ PROF else warn "no $PROFILE_D -- cannot install the SHARED_* variables" note " source this from your own shell instead, and it stays yours:" - note " for d in $HUB/*; do export \"SHARED_\$(basename \$d | tr a-z- A-Z_)=\$d\"; done" + note " for d in ~/Shared/*; do export \"SHARED_\$(basename \$d | tr a-z- A-Z_)=\$d\"; done" fi # ---- 3b. the session group, applied by the session ------------------------- diff --git a/stacks/avoid.stack b/stacks/avoid.stack index 24ef08f..805a721 100644 --- a/stacks/avoid.stack +++ b/stacks/avoid.stack @@ -1,5 +1,5 @@ # avoid.stack -- what no stack on this machine builds, and why. -# stack-version: 1.14.320 +# stack-version: 1.14.292 # # An `avoid` entry does not build anything: it tells a dependency plan that # this machine will not have that package, so the plan stops walking into it @@ -25,6 +25,13 @@ # "never built by this stack" line. Harmless, and it made the composed plan # impossible to check against, so now one entry per package. +# sway.stack -- a working Sway desktop with a minimal WebKit browser. +# stack-version: 1.14.292 +# Replay with: packagemanager stack sway --run +# Entries run top to bottom; book deps are resolved from the BLFS book, +# installed entries are skipped, the first failure stops the run. +# git refs are pinned to releases -- true them up if a clone fails. + # Never built by this stack, however they are reached. harfbuzz RECOMMENDS # texlive and LibreOffice; a sway desktop with a webkit browser wants # neither, and texlive alone is hours of build time and a /opt tree. @@ -156,6 +163,10 @@ avoid freeglut # mupdf (avoided) -- no live parent avoid polkit # colord + systemd (both avoided) -- no live parent avoid libgusb # colord +# 9. the nimbin2 family: make all && make PREFIX=/usr install + +# 10. wire the session together (root step) + # TeX is NOT built by this stack. # diff --git a/stacks/bootscripts.sh b/stacks/bootscripts.sh index 82d17e8..c3aa488 100644 --- a/stacks/bootscripts.sh +++ b/stacks/bootscripts.sh @@ -43,49 +43,20 @@ have_any() { [ -r "$CONF" ] || { warn "no $CONF to read"; exit 1; } # ---- 1. the unpacked blfs-bootscripts tree -------------------------------- -# WHERE THE RUNNER PUTS IT, not where it seems it should be. lfs-phases -# builds in BUILD_ROOT, which is `$HOME/src` when the package user has one -- -# /usr/src/p_blfs-bootscripts/SRC/blfs-bootscripts-<date>, one level deeper -# than the home. Looking only in the home found nothing and told the reader -# to unpack a tree that was already there. -SOURCES="${LFS_SOURCES_DIR:-/sources}" -TMPTREE="" +# It is a `tar` entry in services.stack, so its source sits in its package +# user's home like any other. Newest wins: an older copy must not. if [ -z "$SRC" ]; then - for d in "$BASE"/*/src/blfs-bootscripts-* "$BASE"/*/blfs-bootscripts-* \ - "$BASE"/blfs-bootscripts-* "$SOURCES"/blfs-bootscripts-*; do + for d in "$BASE"/*/blfs-bootscripts-* "$BASE"/blfs-bootscripts-*; do [ -f "$d/Makefile" ] || continue if [ -z "$SRC" ] || [ "$d" -nt "$SRC" ]; then SRC="$d"; fi done fi -if [ -z "$SRC" ]; then - # anything those four patterns did not think of, bounded so this can - # never turn into a walk of the whole package tree - d="$(find "$BASE" -maxdepth 4 -type d -name 'blfs-bootscripts-*' 2>/dev/null \ - | sort | tail -n1)" - [ -n "$d" ] && [ -f "$d/Makefile" ] && SRC="$d" -fi -if [ -z "$SRC" ]; then - # THE TARBALL IS ENOUGH. A build tree can be cleaned away; the sources - # directory keeps the tarball, and these targets only copy files. - tb="$(ls -1t "$SOURCES"/blfs-bootscripts-*.tar.* 2>/dev/null | head -n1)" - if [ -n "$tb" ]; then - TMPTREE="$(mktemp -d)" - if tar -xf "$tb" -C "$TMPTREE" 2>/dev/null; then - SRC="$(ls -d "$TMPTREE"/blfs-bootscripts-* 2>/dev/null | head -n1)" - note "no build tree left -- unpacked $tb" - note " (installing from a temporary copy, so these files land" - note " owned by root rather than by the bootscripts package user)" - fi - fi -fi if [ -z "$SRC" ] || [ ! -f "$SRC/Makefile" ]; then - warn "no blfs-bootscripts tree and no tarball -- nothing to install from" - note " looked in: $BASE/*/src, $BASE/*, $BASE, $SOURCES" - note " fetch and unpack it again:" + warn "no unpacked blfs-bootscripts tree under $BASE -- nothing to install from" + note " unpack it again:" note " packagemanager stack services --only blfs-bootscripts --run --force" exit 1 fi -trap '[ -n "$TMPTREE" ] && rm -rf "$TMPTREE"' EXIT note "bootscripts: $SRC" # ---- 2. what is missing --------------------------------------------------- diff --git a/stacks/diag.stack b/stacks/diag.stack index 1dbae23..9b9e7c1 100644 --- a/stacks/diag.stack +++ b/stacks/diag.stack @@ -1,19 +1,9 @@ -# diag.stack -- what the hardware is (lsusb, lspci) and what the GPU does -# (glxinfo, eglinfo, vulkaninfo, vainfo, nvtop). -# stack-version: 1.14.327 +# diag.stack -- GPU diagnostic tools: glxinfo, eglinfo, vulkaninfo, vainfo, nvtop. +# stack-version: 1.14.292 # # WHAT THIS MACHINE NEVER BUILDS is in avoid.stack, included by every stack. include avoid -# WHAT HARDWARE IS THERE AT ALL. `lsusb` is the first command for a USB -# device that did not appear -- it says whether the machine sees the device -# before any question about drivers or firmware makes sense. lspci is its -# counterpart for the cards inside, and both are one small BLFS page each. -book usbutils \ - expect="command -v lsusb >/dev/null" -book pciutils \ - expect="command -v lspci >/dev/null" - # # Replay with: packagemanager stack diag --run # diff --git a/stacks/machine.conf b/stacks/machine.conf index 2317c02..6888b74 100644 --- a/stacks/machine.conf +++ b/stacks/machine.conf @@ -84,7 +84,7 @@ platforms = x11,wayland gallium-drivers = radeonsi,llvmpipe vulkan-drivers = amd,swrast video-codecs = all -# glvnd: the GL dispatcher (libglvnd, wayland.stack, before mesa). Enabled so +# glvnd: the GL dispatcher (libglvnd, sway.stack, before mesa). Enabled so # pkg-config `opengl` exists (libdecor's demo, some games and toolkits ask # for it by that name) and a second GL vendor could be added. Changing it # means rebuilding libglvnd and mesa together -- see the libglvnd entry. @@ -109,23 +109,6 @@ gbm = enabled [iptables] configure_args = --enable-nftables -# cairomm/pangomm/glibmm build TEST PROGRAMS by default, and those need -# Boost Test. Boost is in avoid.stack -- an hour of build for a desktop -# that wants none of it -- so meson stops at configure: -# ERROR: Building the test programs requires Boost Test ... and Fontconfig -# Install them or use -Dbuild-tests=false -# The library itself needs neither. (lfs-helper takes this option from the -# error by itself since 1.14.337; this is where it STAYS, because a -# regenerated script would lose it.) -[cairomm-1.0] -build-tests = false - -[pangomm-2.48] -build-tests = false - -[glibmm-2.68] -build-tests = false - # GST_DEBUG=<level> ONLY WORKS WITH THIS. The book builds the core with # -D gst_debug=false, which compiles the whole debug-log system out; this # replaces that switch (the script's -D gst_debug=false becomes =true). @@ -209,7 +192,7 @@ wrap_mode = nodownload # ENABLE_MEDIA_STREAM the getUserMedia half of the same thing # USE_GSTREAMER_WEBRTC which backend does the WebRTC # -# woff2 must be INSTALLED first (media.stack builds it just before +# woff2 must be INSTALLED first (sway.stack builds it just before # webkitgtk), or the switch turns on support for a library that is not there. # # jobs = 8 rather than [all]'s 13: webkitgtk's link step is the biggest @@ -278,7 +261,6 @@ ENABLE_JOURNALD_LOG = OFF # and a working /proc setup; OFF is what this machine had and what it built # with before. Turn it ON deliberately, not by inheriting a default. ENABLE_BUBBLEWRAP_SANDBOX = OFF -USE_GTK4 = ON USE_LIBRICE = ON USE_WOFF2 = ON ENABLE_WEB_RTC = ON diff --git a/stacks/media.stack b/stacks/media.stack index 1e95d62..9d374e3 100644 --- a/stacks/media.stack +++ b/stacks/media.stack @@ -1,5 +1,5 @@ -# media.stack -- codecs, gstreamer, the WebRTC chain, webkitgtk, mpv, yt-dlp. -# stack-version: 1.14.321 +# media.stack -- codecs, gstreamer, the WebRTC chain, webkitgtk, mpv. +# stack-version: 1.14.292 # # WHAT THIS MACHINE NEVER BUILDS is in avoid.stack, included by every stack # here: the exclusions are a property of the machine, not of one stack. @@ -7,13 +7,6 @@ # cups -- because those exclusions lived in sway.stack alone. include avoid -# THE DEVELOPMENT BOOK. Codecs, gstreamer and webkitgtk move faster than a -# stable book, so the book entries below read svn: versions, dependencies, -# scripts. Dependencies OUTSIDE this file stay on the default book (built if -# missing, never upgraded to svn's page). Per entry: from-book=default. -# `packagemanager stack media` refreshes the svn book once per run. -from-book svn - # # Everything that decides whether a browser can PLAY and PUBLISH: audio and # video codecs, the gstreamer plugin sets, the four WebRTC pieces from git, @@ -28,7 +21,7 @@ from-book svn # an evening at most, and usually minutes. # # ORDER MATTERS, AND THIS IS NOT THE FIRST STACK. It needs, from -# wayland.stack: alsa-lib and ffmpeg (entry 1b -- the audio floor and the +# sway.stack: alsa-lib and ffmpeg (entry 1b -- the audio floor and the # codecs), llvm and mesa, wayland, and gtk4 with pango and libsoup3. # webkitgtk links all of them. sway.stack pulls this file in with # `include media` at exactly the right point, so the full desktop build is @@ -36,7 +29,7 @@ from-book svn # # WHAT IS *NOT* HERE, deliberately: # * ffmpeg, alsa-lib -- the audio and codec floor: too many things below -# them in wayland.stack depend on them being early. +# them in sway.stack depend on them being early. # * x264, x265, dav1d, libaom, svt-av1, fdk-aac, lame -- they arrive as # ffmpeg's own dependencies, so the book's plan builds them. x264enc # (the ELEMENT) comes from gst10-plugins-ugly, which IS here. @@ -222,37 +215,3 @@ book webkitgtk \ expect="ldd /usr/lib/libwebkitgtk-6.0.so.4 2>/dev/null | grep -q woff2" book mpv - -# yt-dlp -- NOT a BLFS page; upstream's release tarball. -# -# The RELEASE ASSET, not the git tree and not the GitHub "source code" -# archive: those lack what the wheel's shared-data section names (the man -# page, README.txt, the shell completions, lazy_extractors.py), all -# generated at release time -- hatchling stops on the first one missing. -# Checked here: this tarball builds with the book's own wheel recipe and -# `yt-dlp --version` answers 2026.08.19. -# -# hatchling is the build backend (>= 1.27) and is NOT an entry here. It is -# already on a machine that has built any python module, and a `book -# hatchling` line does not mean "make sure it is there": it asked the book, -# planned a chain of newer python modules (vcs_versioning, which wants a -# newer `packaging`) and stopped before yt-dlp ever started (1.14.314). If -# it IS missing, the build stops on `BackendUnavailable: Cannot import -# 'hatchling.build'`, which lfs-helper recognises and names the page for. -# ffmpeg is -# not a build dependency but yt-dlp needs it at RUNTIME to merge separate -# video and audio streams -- it comes from wayland.stack. With yt-dlp on PATH, -# mpv plays a URL directly: mpv https://www.youtube.com/watch?v=... -# -# YOUTUBE BREAKS OLD VERSIONS WITHIN WEEKS. A tag is pinned so a replay -# builds what was tested; move it when downloads start failing: -# git ls-remote --tags https://github.com/yt-dlp/yt-dlp | tail -3 -# then edit the two version strings below and -# packagemanager stack media --only yt-dlp --run -tar yt-dlp \ - url=https://github.com/yt-dlp/yt-dlp/releases/download/2026.08.19/yt-dlp.tar.gz \ - pkg=yt-dlp-2026.08.19.tar.gz \ - have=/usr/bin/yt-dlp \ - build="pip3 wheel -w dist --no-build-isolation --no-deps --no-cache-dir ." \ - install="pip3 install --no-index --find-links=dist --no-cache-dir --no-user --no-deps --force-reinstall yt-dlp" \ - expect="yt-dlp --version >/dev/null 2>&1" diff --git a/stacks/net-wifi.sh b/stacks/net-wifi.sh index 0ee69c6..157f59d 100644 --- a/stacks/net-wifi.sh +++ b/stacks/net-wifi.sh @@ -22,23 +22,10 @@ # todo something only you can write -> printed, exit 0 fail=0 warn() { echo "!! $*"; fail=1; } -did() { echo "++ $*"; } todo() { echo "-> $*"; todos=$((todos + 1)); } note() { echo "# $*"; } todos=0 -# ---- 0. the settings ------------------------------------------------------ -CONF="${NET_WIFI_CONF:-/etc/pkgusr/stacks/net-wifi.conf}" -[ -r "$CONF" ] && . "$CONF" -: "${WRITE:=no}"; : "${PREFER:=}"; : "${COPY_WPA:=no}" -: "${METRIC_PREFERRED:=100}"; : "${METRIC_OTHER:=300}" -: "${USB_RENAME:=no}"; : "${USB_NAME:=wl-ext}" -SYSCONF="${NET_WIFI_SYSCONF:-/etc/sysconfig}" -NETSYS="${NET_WIFI_SYSFS:-/sys/class/net}" -UDEVDIR="${NET_WIFI_UDEVDIR:-/etc/udev/rules.d}" -MARKER="# written by stacks/net-wifi.sh -- safe to delete, regenerated" -[ -r "$CONF" ] && note "settings: $CONF" || note "no $CONF -- reporting only" - # ---- 1. the binaries ------------------------------------------------------ for b in /sbin/wpa_supplicant /usr/sbin/wpa_supplicant; do [ -x "$b" ] && wpa="$b" && break @@ -49,43 +36,11 @@ command -v dhcpcd >/dev/null 2>&1 \ command -v ip >/dev/null 2>&1 || command -v ifconfig >/dev/null 2>&1 \ || warn "neither ip nor ifconfig is installed" -# ---- 1b. a stable name for the USB adapter -------------------------------- -# Written before the interfaces are listed, because the name it gives is the -# one the rest of this script should be configuring. -_usb_rule="$UDEVDIR/80-wifi-$USB_NAME.rules" -case "$USB_RENAME" in yes|1|true) - if [ -z "${USB_VENDOR:-}" ] || [ -z "${USB_PRODUCT:-}" ]; then - warn "USB_RENAME=yes but USB_VENDOR/USB_PRODUCT are not set in $CONF" - elif [ -e "$_usb_rule" ] && ! grep -qF "$MARKER" "$_usb_rule"; then - note "$_usb_rule exists and is not ours -- left alone" - else - install -d "$UDEVDIR" - cat > "$_usb_rule" <<RULE -$MARKER -# Name the $USB_VENDOR:$USB_PRODUCT wireless adapter "$USB_NAME" whatever port it -# is in. Without this its name comes from its place on the bus (wlp7s0f4u2), -# which changes with the socket -- and /etc/sysconfig/ifconfig.<name> cannot -# follow that. -SUBSYSTEM=="net", ACTION=="add", ATTRS{idVendor}=="$USB_VENDOR", ATTRS{idProduct}=="$USB_PRODUCT", NAME="$USB_NAME" -RULE - chmod 644 "$_usb_rule" - did "wrote $_usb_rule ($USB_VENDOR:$USB_PRODUCT -> $USB_NAME)" - # IT DOES NOT APPLY TO AN INTERFACE THAT IS ALREADY THERE. Renaming - # happens when the device appears, so a plugged-in adapter keeps the - # name it was given until it is unplugged or the machine reboots. - if [ ! -e "$NETSYS/$USB_NAME" ] \ - && ls -1 "$NETSYS" 2>/dev/null | grep -q '^wl.*u[0-9]'; then - note " unplug the adapter and plug it back in to take the name" - note " (or reboot; udev renames a device when it APPEARS)" - fi - fi ;; -esac - # ---- 2. the interfaces the kernel actually has ---------------------------- # The name matters: ifconfig.<name> must match the interface, and predictable # names (wlp3s0) are not the wlan0/wifi0 that examples use. ifaces="" -for d in "$NETSYS"/*; do +for d in /sys/class/net/*; do [ -e "$d/wireless" ] || [ -e "$d/phy80211" ] || continue ifaces="$ifaces $(basename "$d")" done @@ -115,31 +70,6 @@ if [ -d "$svcdir" ]; then note " from blfs-bootscripts' install-service-dhcpcd target:" note " packagemanager stack services --only blfs-bootscripts --run" ;; esac - # WHERE THE SERVICE LOOKS FOR THE CONFIG -- ASK THE SCRIPT. - # - # This machine's boot said: - # wpa_supplicant configuration file - # /etc/sysconfig/wpa_supplicant-wlan0.conf not present FAIL - # ...while the file sat in /etc/sysconfig/wpa_supplicant/ -- the - # SUBDIRECTORY, which is where other bootscript releases keep it. So - # wpa_supplicant never started, the card never associated, and dhcpcd - # had nothing to ask for an address on. Two directories, one word - # apart, and no output anywhere said which one this machine reads. - # The service script names it; read it rather than remember it. - if [ -r "$svcdir/wpa" ]; then - _wpa_pat="$(grep -ohE '/[A-Za-z0-9_./${}-]*wpa_supplicant[A-Za-z0-9_.${}-]*\.conf' \ - "$svcdir/wpa" 2>/dev/null | head -n1)" - if [ -n "$_wpa_pat" ]; then - WPA_DIR="$(dirname "$_wpa_pat")" - # NET_WIFI_SYSCONF overrides /etc/sysconfig (the tests use it); - # a path read out of the service script has to follow it too. - case "$WPA_DIR" in - /etc/sysconfig*) WPA_DIR="$SYSCONF${WPA_DIR#/etc/sysconfig}" ;; - esac - note "the wpa service reads: $_wpa_pat" - note " -> $WPA_DIR/wpa_supplicant-<iface>.conf" - fi - fi # which variable the wpa service reads to chain onward -- from the script if [ -r "$svcdir/wpa" ]; then chain="$(grep -oE '\$\{?[A-Z_]*SERVICE[A-Z_]*' "$svcdir/wpa" \ @@ -153,79 +83,11 @@ else warn "no $svcdir: blfs-bootscripts is not installed" fi -# A SECOND ADAPTER JOINS THE SAME NETWORKS. -# -# The wpa service reads wpa_supplicant-<iface>.conf, so a new interface has -# nothing to join with even though this machine has known the SSID and the -# key for months. Copying your own file is the whole job; it is said out -# loud every time, because a key is being duplicated. -# The directory the SERVICE reads, when it told us; otherwise the one this -# script has always assumed. -_wpadir="${WPA_DIR:-$SYSCONF/wpa_supplicant}" -case "$COPY_WPA" in yes|1|true) - for i in $ifaces; do - _dst="$_wpadir/wpa_supplicant-$i.conf" - [ -e "$_dst" ] && continue - _srcconf="" - # A CONFIG IN THE OTHER DIRECTORY IS STILL YOUR CONFIG. The two - # layouts -- /etc/sysconfig/wpa_supplicant-<iface>.conf and - # /etc/sysconfig/wpa_supplicant/wpa_supplicant-<iface>.conf -- have - # both been right at some point, and a machine can easily have the - # file in the one its service does not read. - for _c in "$_wpadir"/wpa_supplicant-*.conf \ - "$SYSCONF"/wpa_supplicant-*.conf \ - "$SYSCONF"/wpa_supplicant/wpa_supplicant-*.conf; do - [ -r "$_c" ] || continue - case "$(basename "$_c")" in *__*__*|*example*|*template*|*sample*) continue ;; esac - grep -q 'psk\|key_mgmt=NONE' "$_c" 2>/dev/null || continue - _srcconf="$_c"; break - done - [ -n "$_srcconf" ] || continue - install -d -m 755 "$_wpadir" - cp "$_srcconf" "$_dst" && chmod 600 "$_dst" \ - && did "copied $(basename "$_srcconf") -> $(basename "$_dst") (chmod 600; it holds your key)" - done ;; -esac - # ---- 4. the per-interface configuration ----------------------------------- for i in $ifaces; do - f="$SYSCONF/ifconfig.$i" - # WHICH INTERFACE WINS. The first PREFER pattern that matches something - # this machine has gets the low metric; everything else gets the high - # one, so an external antenna takes over while it is plugged in and the - # card inside carries on when it is not. - _metric="$METRIC_OTHER" - for _p in $PREFER; do - _hit="" - for _c in $ifaces; do - case "$_c" in $_p) _hit="$_c"; break ;; esac - done - [ -n "$_hit" ] || continue - [ "$_hit" = "$i" ] && _metric="$METRIC_PREFERRED" - break - done - if [ ! -r "$f" ] && [ "$WRITE" = yes ]; then - _chain_var="$(printf '%s' "${chain:-WPA_SERVICE}" | awk '{print $1}')" - install -d "$SYSCONF" - cat > "$f" <<IFCFG -$MARKER -# $i -- what ifup reads. No SSID and no key are in here; those live in -# $SYSCONF/wpa_supplicant/wpa_supplicant-$i.conf, which nothing writes for -# you. -m <metric> decides which link the machine prefers when two are up: -# the lower number wins. -ONBOOT="yes" -IFACE="$i" -SERVICE="wpa" -${_chain_var:-WPA_SERVICE}="dhcpcd" -DHCP_START="-b -m $_metric" -DHCP_STOP="-k" -IFCFG - chmod 644 "$f" - did "wrote $f (route metric $_metric$([ "$_metric" = "$METRIC_PREFERRED" ] && echo " -- preferred"))" - fi + f="/etc/sysconfig/ifconfig.$i" if [ ! -r "$f" ]; then todo "write $f -- ifup $i has nothing to read" - note " or let this step do it: WRITE=yes in $CONF" # THIS FILE HOLDS NO SECRET. Only the wpa_supplicant conf does, so # there is no reason to make the reader work out the syntax: print # the file, with the interface that exists and the hand-off variable @@ -260,73 +122,6 @@ IFCFG done grep -qE '^[[:space:]]*ONBOOT=.?yes' "$f" \ || note "$f: ONBOOT is not yes, so boot will not bring it up" - # THE CONFIG THE SERVICE WILL LOOK FOR, FOR THIS INTERFACE. - if grep -qE '^[[:space:]]*SERVICE=.?wpa' "$f"; then - if [ -r "$_wpadir/wpa_supplicant-$i.conf" ]; then - note "$f: wpa config $_wpadir/wpa_supplicant-$i.conf" - else - todo "no $_wpadir/wpa_supplicant-$i.conf -- at boot this reads:" - note " wpa_supplicant configuration file ... not present FAIL" - note " and nothing associates, so dhcpcd has no link to ask on." - _other="$(ls -1 "$SYSCONF"/wpa_supplicant-*.conf \ - "$SYSCONF"/wpa_supplicant/wpa_supplicant-*.conf \ - 2>/dev/null | head -n1)" - [ -n "$_other" ] \ - && note " you have one for another interface: $_other" \ - && note " cp '$_other' '$_wpadir/wpa_supplicant-$i.conf' && chmod 600 \$_" \ - || note " wpa_passphrase '<SSID>' '<key>' > $_wpadir/wpa_supplicant-$i.conf" - fi - fi - # dhcpcd IN THE FOREGROUND MISSES ITS OWN LINK. A wireless interface - # is up before it is associated; a dhcpcd without -b waits for a lease - # that cannot arrive yet and the boot line says FAIL. -b puts it in - # the background, where it keeps trying and succeeds a second later. - if grep -qE '^[[:space:]]*DHCP_START=' "$f" \ - && ! grep -qE '^[[:space:]]*DHCP_START=.*-b' "$f"; then - todo "$f: DHCP_START has no -b, so dhcpcd waits for a lease it cannot" - note " have yet -- that is the 'Starting dhcpcd ... FAIL' at boot." - note " sed -i 's/^DHCP_START=.*/DHCP_START=\"-b\"/' '$f'" - fi - # OURS, AND NOW WRONG. A file this step wrote may carry yesterday's - # metric after PREFER changed; one of yours is never touched. - if grep -qF "$MARKER" "$f" 2>/dev/null \ - && ! grep -qE "^DHCP_START=.*-m $_metric([^0-9]|$)" "$f"; then - if [ "$WRITE" = yes ]; then - cp -p "$f" "$f.bak-$(date +%Y%m%d-%H%M%S)" - sed -i "s/^DHCP_START=.*/DHCP_START=\"-b -m $_metric\"/" "$f" - did "$f: route metric is now $_metric" - else - todo "$f says another metric than PREFER does ($_metric)" - fi - fi -done - -# A CONFIG FOR AN INTERFACE THAT IS NOT HERE. After a rename (or a card -# taken out) its ifconfig file stays, and ifup at boot fails on a name the -# kernel does not have. Named, never deleted: it may be the adapter you -# unplugged this morning. -for f in "$SYSCONF"/ifconfig.*; do - [ -f "$f" ] || continue - case "$f" in *.bak-*) continue ;; esac - _n="$(basename "$f" | sed 's/^ifconfig\.//')" - [ -e "$NETSYS/$_n" ] && continue - grep -qE '^[[:space:]]*ONBOOT=.?yes' "$f" 2>/dev/null || continue - # AT BOOT THIS IS A RED LINE ON THE SCREEN. - # Bringing up the eth0 interface... Interface eth0 doesn't exist. FAIL - # The file is for a card that was taken out, an interface that was - # renamed, or an adapter that is simply not plugged in right now -- so - # it is never deleted here. ONBOOT=no stops the boot from trying. - if [ "${DISABLE_MISSING:-no}" = yes ]; then - cp -p "$f" "$f.bak-$(date +%Y%m%d-%H%M%S)" - sed -i 's/^[[:space:]]*ONBOOT=.*/ONBOOT="no"/' "$f" - did "$f: $_n is not here -- ONBOOT=no (it was failing at every boot)" - note " bring it up by hand when it is: ifup $_n" - else - todo "$f names $_n, which this kernel does not have -- it FAILS at boot" - note " removable, renamed, or gone? Either delete the file, or:" - note " sed -i 's/^ONBOOT=.*/ONBOOT=\"no\"/' '$f'" - note " (DISABLE_MISSING=yes in $CONF does that for every one of them)" - fi done # ---- 4b. the firewall: userspace and kernel must agree -------------------- @@ -368,9 +163,7 @@ echo # ---- 5. the wpa_supplicant configuration ---------------------------------- found_conf="" -for c in "$_wpadir"/wpa_supplicant-*.conf \ - "$SYSCONF"/wpa_supplicant-*.conf \ - "$SYSCONF"/wpa_supplicant/wpa_supplicant-*.conf \ +for c in /etc/sysconfig/wpa_supplicant/wpa_supplicant-*.conf \ /etc/wpa_supplicant/wpa_supplicant*.conf /etc/wpa_supplicant.conf; do [ -r "$c" ] && found_conf="$found_conf $c" done @@ -415,16 +208,6 @@ elif [ "$todos" -gt 0 ]; then else echo "# wireless: the pieces are all in place; bring it up with" echo "# ifup $(set -- $ifaces; echo "$1")" - if [ -n "$PREFER" ]; then - for _p in $PREFER; do - for _c in $ifaces; do - case "$_c" in $_p) - echo "# preferred while it is there: $_c (metric $METRIC_PREFERRED)" - break 2 ;; - esac - done - done - fi fi # -> lines do not fail the step; !! lines do exit $fail diff --git a/stacks/network.stack b/stacks/network.stack index 275f82d..1c327e4 100644 --- a/stacks/network.stack +++ b/stacks/network.stack @@ -1,5 +1,5 @@ # network.stack -- wired and wireless networking, for a SysV system. -# stack-version: 1.14.320 +# stack-version: 1.14.292 # # WHAT THIS MACHINE NEVER BUILDS is in avoid.stack, included by every stack # here: the exclusions are a property of the machine, not of one stack. diff --git a/stacks/services.stack b/stacks/services.stack index 1d28750..e153e92 100644 --- a/stacks/services.stack +++ b/stacks/services.stack @@ -1,5 +1,5 @@ # services.stack -- the boot-time services a SysV desktop needs. -# stack-version: 1.14.330 +# stack-version: 1.14.292 # # WHAT THIS MACHINE NEVER BUILDS is in avoid.stack, included by every stack # here: the exclusions are a property of the machine, not of one stack. @@ -52,39 +52,6 @@ git bash-completion url=https://github.com/scop/bash-completion ref=2.18.0 \ cfg shell-env \ expect='test -f /etc/bashrc && test -f /etc/locale.conf && { test ! -d /etc/profile.d || test -f /etc/profile.d/i18n.sh; }' -# BTRFS, ON A BTRFS MACHINE ONLY. `btrfs check`, `scrub`, `balance`, -# subvolumes and snapshots all come from btrfs-progs, and without it a btrfs -# root can be mounted and nothing else -- no repair, no snapshot, and -# mkinitramfs/fsck have no helper. when= asks the machine: a filesystem -# mounted now, or a partition on any disk (so it is also right in a chroot -# whose target is btrfs and in the installer, before the mount). -book btrfs-progs \ - when="grep -qw btrfs /proc/mounts || lsblk -no FSTYPE 2>/dev/null | grep -qx btrfs || blkid -t TYPE=btrfs >/dev/null 2>&1" \ - expect="command -v btrfs >/dev/null && btrfs --version >/dev/null" - -# EXFAT, WHICH ARRIVES IN A POCKET. mkfs.exfat / fsck.exfat / exfatlabel -# for the cards and sticks that come formatted that way -- no when= test, -# because the medium is not in the machine when the stack runs. -# -# NOT A BLFS PAGE. The book carries no exFAT package at all (checked -# against the stable index, 1.14.330); what it links to in passing is -# relan/exfat, the FUSE implementation -- a different thing. This is the -# userspace for the KERNEL driver, which is the modern path and the one -# this machine uses: -# lfs-kernel enable exfat # EXFAT_FS=m, then lfs-kernel --run -# relan/exfat (mount.exfat-fuse) is only worth having on a kernel without -# that driver. -# -# The release tarball is an autotools dist, so ./configure is in it; the -# build line falls back to autogen.sh if a future tarball is a bare export. -tar exfatprogs \ - url=https://github.com/exfatprogs/exfatprogs/releases/download/1.4.2/exfatprogs-1.4.2.tar.xz \ - pkg=exfatprogs-1.4.2.tar.xz \ - prog=mkfs.exfat \ - build="test -x ./configure || ./autogen.sh; ./configure --prefix=/usr --sbindir=/usr/sbin; make" \ - install="make install" \ - expect="command -v mkfs.exfat >/dev/null && command -v fsck.exfat >/dev/null" - book acpid # power/lid/button events -- what a laptop needs # ONE GROUP CARRIES THE PRIVILEGE. halt/poweroff/reboot must become root, so @@ -139,8 +106,7 @@ cfg shadow-pam \ # this step also does the two things no per-account command can: merge # /etc/pam.d/su_u_user (one file listing every account that may be entered # without a password) and write /etc/profile.d/shared-dirs.sh (one file -# exporting SHARED_<NAME> for each link in the hub, -# /home/Shared -- your ~/Shared links there). +# exporting SHARED_<NAME> for each ~/Shared/<Name>). # # Needs a main user: `packagemanager config --main-user <your-login>`. # Nothing of yours is edited -- no .bashrc, no .bash_profile -- and the PAM @@ -149,29 +115,11 @@ cfg shadow-pam \ # Re-checked on every run (see shadow-pam): the accounts IT MANAGES (the # names in app-users.conf -- not every u_*: a hand-made u_test is not its # business) are not directly in /home, su calls su_u_user, root owns it, and -# it checks the caller, not the target. And (1.14.312) your ~/Shared is -# the link to the hub, not a directory of links: a machine set up before the -# hub existed re-runs this step once and moves them. +# it checks the caller, not the target. cfg app-users \ - expect='test -x /usr/bin/xdg-runtime-share && ! awk -F"|" "\$1 !~ /^[[:space:]]*(#|name[[:space:]]|\$)/ { gsub(/[[:space:]]/, \"\", \$1); print \"u_\" \$1 }" /etc/pkgusr/stacks/app-users.conf | while read -r u; do getent passwd "$u" | cut -d: -f6; done | grep -qE "^/home/[^/]+\$" && { test ! -f /etc/pam.d/su || grep -qE "^[[:space:]]*auth[[:space:]]+sufficient[[:space:]]+pam_exec[.]so.*su_u_user" /etc/pam.d/su; } && { test ! -e /etc/pam.d/su_u_user || { test "$(stat -c %U:%a /etc/pam.d/su_u_user)" = root:755 && ! grep -qF -e "=~ \"\$PAM_USER\"" /etc/pam.d/su_u_user; }; } && { m="$(grep "^main_user=" /etc/pkgusr/packagemanager.conf 2>/dev/null | head -n1 | cut -d= -f2)"; h="$(getent passwd "$m" 2>/dev/null | cut -d: -f6)"; [ -z "$h" ] || [ ! -d "$h/Shared" ] || [ -L "$h/Shared" ]; }' + expect='test -x /usr/bin/xdg-runtime-share && ! awk -F"|" "\$1 !~ /^[[:space:]]*(#|name[[:space:]]|\$)/ { gsub(/[[:space:]]/, \"\", \$1); print \"u_\" \$1 }" /etc/pkgusr/stacks/app-users.conf | while read -r u; do getent passwd "$u" | cut -d: -f6; done | grep -qE "^/home/[^/]+\$" && { test ! -f /etc/pam.d/su || grep -qE "^[[:space:]]*auth[[:space:]]+sufficient[[:space:]]+pam_exec[.]so.*su_u_user" /etc/pam.d/su; } && { test ! -e /etc/pam.d/su_u_user || { test "$(stat -c %U:%a /etc/pam.d/su_u_user)" = root:755 && ! grep -qF -e "=~ \"\$PAM_USER\"" /etc/pam.d/su_u_user; }; }' -# WHO MAY TURN THE MACHINE OFF. A package user owns files, not identity: -# halt, poweroff and reboot have to BECOME root, so they stay root's and a -# group decides who may run them -- 4550 root:<collector>_power, with the -# desktop user in that group. The same step writes the power-perms boot -# script, because /sys/power/state is made fresh at every boot and its group -# has to be re-applied there. -# -# expect= matters more here than almost anywhere else. Without it this entry -# was cached "done" for ever, and the modes it set are exactly the kind that -# get destroyed silently: a sysvinit rebuild reinstalls halt without the -# setuid bit, and a `chown` -- from anything, including this project's own -# ownership sweep before 1.14.308 -- clears setuid as a side effect the -# kernel never reports. Then poweroff stops working for the user and nothing -# says why. Re-checked on every run: the group, the mode and owner of halt, -# and the boot script. -cfg power \ - expect='g="${POWER_GROUP:-}"; if [ -z "$g" ]; then p="$(grep "^collector_prefix=" /etc/pkgusr/packagemanager.conf 2>/dev/null | head -n1 | cut -d= -f2)"; g="${p%_}_power"; fi; h=/usr/bin/halt; [ -f "$h" ] || h=/sbin/halt; getent group "$g" >/dev/null 2>&1 && test -f /etc/rc.d/init.d/power-perms && test "$(stat -Lc "%a %U:%G" "$h" 2>/dev/null)" = "4550 root:$g"' +cfg power book fcron # the cron daemon; make-ca's weekly update needs one # blfs-bootscripts: 20250225 is the FINAL release (BLFS has stopped shipping diff --git a/stacks/setid.conf b/stacks/setid.conf index 56b9a7a..c0a04c0 100644 --- a/stacks/setid.conf +++ b/stacks/setid.conf @@ -11,58 +11,32 @@ # touches these paths) applies them. Nothing here is guessed: each line is # what LFS or BLFS sets on that file. # -# path | mode | owner | group | package +# path | mode | owner | group # # GROUP IS KEPT AS THE PACKAGE USER where that is how the file arrived: # ownership follows the installer in this system, and only the USER needs to # be root for a setuid bit to mean root. `-` keeps whatever is there. -# -# PACKAGE is the 5th column and it is a lock, not documentation (1.14.310). -# The lease hands a root-owned file to a package user for the length of a -# build phase, and "the file is setuid" used to be reason enough. It is not: -# a manifest is built by TIME WINDOW, so a webkitgtk build that happened to -# be running while `cfg shadow-pam` touched /usr/bin/su recorded su as its -# own -- and the next webkitgtk build leased it, chowned it (which clears the -# setuid bit silently) and left `su` as `p_webkitgtk 0711`. Three times. -# Now only the package named here may be handed the file. A line without a -# package is leased to nobody. # Shadow: su must be setuid root or nobody can become another account. # The other four are setuid root on every distribution and in LFS. -/usr/bin/su | 4711 | root | - | shadow -/usr/bin/passwd | 4711 | root | - | shadow -/usr/bin/chage | 4711 | root | - | shadow -/usr/bin/gpasswd | 4711 | root | - | shadow -/usr/bin/newgrp | 4711 | root | - | shadow +/usr/bin/su | 4711 | root | - +/usr/bin/passwd | 4711 | root | - +/usr/bin/chage | 4711 | root | - +/usr/bin/gpasswd | 4711 | root | - +/usr/bin/newgrp | 4711 | root | - # ...AND THE FOUR THE FIRST LIST MISSED. shadow's own install asked for # nine setuid bits and setid.conf declared five of them; these are the rest, # read off that request list rather than guessed. -/usr/bin/chfn | 4711 | root | - | shadow -/usr/bin/chsh | 4711 | root | - | shadow -/usr/bin/newgidmap | 4711 | root | - | shadow -/usr/bin/newuidmap | 4711 | root | - | shadow +/usr/bin/chfn | 4711 | root | - +/usr/bin/chsh | 4711 | root | - +/usr/bin/newgidmap | 4711 | root | - +/usr/bin/newuidmap | 4711 | root | - # Linux-PAM: its own page sets this one, and its install runs as the # package user, so the chmod there is refused the same way. -/usr/sbin/unix_chkpwd | 4755 | root | - | linux-pam +/usr/sbin/unix_chkpwd | 4755 | root | - # D-Bus: BLFS installs the launch helper 4750 root:messagebus. Here the # GROUP is the point -- only messagebus may run it. (This line lived only # inside lfs-helper until 1.14.263; setid.conf is the one table now.) -/usr/libexec/dbus-daemon-launch-helper | 4750 | root | messagebus | dbus - -# NOT HERE, AND ON PURPOSE: halt, poweroff and reboot. -# -# They carry 4550 root:<collector>_power -- setuid root so they can turn the -# machine off, and executable ONLY by that group, which is what makes "the -# desktop user may reboot" a membership rather than a password. The group's -# NAME is configured (collector_prefix, or POWER_GROUP), so it cannot be -# written down here: a line saying `| root | -` would keep whatever group the -# file happens to have, and on a machine where `cfg power` has not run yet -# that is p_sysvinit -- handing shutdown to whoever builds sysvinit. -# -# `cfg power` is the one place that knows the group, and since 1.14.309 it -# carries an expect= that re-checks the mode and owner on every stack run: -# packagemanager stack services --only power --run -# A `chown` clears setuid without a word (see 1.14.308), so this is checked -# rather than assumed. +/usr/libexec/dbus-daemon-launch-helper | 4750 | root | messagebus diff --git a/stacks/sway.stack b/stacks/sway.stack index fdb5d5a..9338cb9 100644 --- a/stacks/sway.stack +++ b/stacks/sway.stack @@ -1,35 +1,184 @@ -# sway.stack -- the Sway desktop: the compositor, its tools, your programs. -# stack-version: 1.14.320 +# sway.stack -- a working Sway desktop with a minimal WebKit browser. +# stack-version: 1.14.292 # # WHAT THIS MACHINE NEVER BUILDS is in avoid.stack, included by every stack # here: the exclusions are a property of the machine, not of one stack. +# Without it, `stack media` planned 274 packages -- systemd, polkit, qt6, +# cups -- because those exclusions lived in sway.stack alone. include avoid # Replay with: packagemanager stack sway --run # Entries run top to bottom; book deps are resolved from the BLFS book, # installed entries are skipped, the first failure stops the run. # git refs are pinned to releases -- true them up if a clone fails. + +# Never built by this stack, however they are reached. harfbuzz RECOMMENDS +# texlive and LibreOffice; a sway desktop with a webkit browser wants +# neither, and texlive alone is hours of build time and a /opt tree. +# NOT BUILT BY THIS STACK. The book's recommendations are followed -- +# a recommendation the build actually needs (curl for cmake, icu for +# libxml2) must not be dropped by a blanket rule. These are the branches +# this desktop does not want, named one at a time, and everything they alone +# pull in goes with them. +# the KDE leaves okular pulls in: named individually, because a package can +# have several parents and pruning by one of them drops shared dependencies +# (graphite2 was lost that way) + +# The desktop-portal tail. The book makes xdg-desktop-portal REQUIRE all +# four backends -- gtk, wlr, gnome AND lxqt -- so one recommendation from +# gtk4 drags in the whole of xfce and lxqt, gnome-desktop and libfm-qt. +# Sway needs none of it to run; screen sharing wants xdg-desktop-portal-wlr, +# which can be built on its own later: +# packagemanager install xdg-desktop-portal-wlr --run + +# Not this desktop either: a systemd package on a SysV system, phone modem +# stacks, a mail/ssh server, spell-checking dictionaries. +# openssh is AVOIDED here and INSTALLED by network.stack, where it belongs: +# it is networking, not desktop. Letting a dependency plan pull it in +# mid-sway would put its system-account question in the middle of a graphics +# build. + +# Tails of things already avoided. Each of these is in the plan ONLY to +# serve a package this stack does not build, and the tools cannot yet work +# that out for themselves (see 1.12.83: pruning by the single parent that +# `blfs order` reports dropped shared dependencies, and broke harfbuzz). +# Checked one by one against the plan's parent column: +# Checked against EVERY parent, not just the one the plan prints: these have +# no parent left in the plan at all. +# NOT avoided, though the plan labels them with an avoided parent: the label +# shows only the FIRST parent. gtk3 is really wanted by adwaita-icon-theme, +# ffmpeg by mpv, polkit by colord... and pulseaudio by pipewire/sdl3/gst. +# (rust and llvm STAY: glycin genuinely requires rust, and llvm is a stack +# entry in its own right for mesa.) + +# 0. verify the kernel (DRM/amdgpu, evdev, snd, seccomp, user ns, render node) +check kernel-sway + +# 1. seat and session (SysV: elogind is the logind; seatd is not in the book) +book dbus +book elogind url=https://github.com/elogind/elogind ref=v255.17 prog=loginctl build="meson setup build --prefix=/usr --buildtype=release -Dcgroup-controller=elogind -Ddbuspolicydir=/etc/dbus-1/system.d; ninja -C build" install="ninja -C build install" + +# 1b. THE AUDIO FLOOR, BEFORE ANYTHING THAT LOOKS FOR IT. # -# THREE FILES, ONE COMMAND (1.14.320). This stack begins with the two it -# stands on, in the order they have to be built: +# alsa-lib used to sit at entry 7, after mpv -- and ffmpeg is built inside +# mesa's plan (entry 2) and SDL3 inside gstreamer's, both of which autodetect +# ALSA. They configured cleanly, installed cleanly, and had no ALSA support, +# because the library did not exist yet. Nothing failed, so nothing said so. # -# wayland.stack session, graphics, wayland, GTK4, seatd, wlroots -# media.stack codecs, gstreamer, WebRTC, webkitgtk, mpv, yt-dlp +# This is the same rule the harfbuzz comment below states for libass: what an +# earlier entry installed is simply there. An optional dependency of +# something built inside a plan can only be placed by entry order. +book alsa-lib + +# ffmpeg and SDL3 are named here, before mesa and gstreamer, so they are +# built in THIS order rather than wherever a dependency plan reaches them -- +# and expect= proves the feature afterwards instead of trusting the order. +book ffmpeg expect="ffmpeg -hide_banner -devices 2>/dev/null | grep -qw alsa" + +# 2. graphics -- llvm feeds mesa; mesa options come from the machine +# (machine.conf wiring pending; book defaults until then) +# cmake (pulled in by llvm and others) has NO required dependencies and four +# recommended ones -- and its own build turns on the system libraries, so a +# missing curl is a hard failure: +# CMAKE_USE_SYSTEM_CURL is ON but a curl is not found! +# This is the trade-off, and the answer is the same one: name +# what the build actually needs. +book nghttp2 +book curl +book libarchive +book libuv +book llvm +# glslang provides glslangValidator, which mesa needs ONLY because +# machine.conf turns on the Vulkan drivers -- the book does not list it as a +# mesa dependency, so no dependency order can place it. A machine option +# that creates a dependency has to name it here. +book glslang +# The font stack goes BEFORE mesa. # -# and then does what only a Sway desktop needs. The applications that used -# to live here (zathura, imv) are apps.stack now, and jq is in tools.stack: -# none of them is part of the desktop, and none needs rebuilding with it. -include wayland +# libass (built inside mesa's plan, via ffmpeg) hard-requires harfbuzz in its +# configure, and the book's libass page does not list it -- so no dependency +# order can place it. What CAN place it is entry order: anything an earlier +# entry installed is simply there. +book harfbuzz # text shaping: freetype recommends it, +book graphite2 # and webkit/pango need both +book fontconfig -# 6b. the media chain, in its own stack so it can be rebuilt on its own. +# LIBGLVND, BEFORE MESA -- because machine.conf builds Mesa with +# glvnd=enabled, and Mesa's configure needs it there. # -# It sits HERE because webkitgtk needs gtk4, pango and libsoup3 from -# wayland.stack, and mesa and ffmpeg from further up. -include media +# glvnd is the GL dispatcher: libGL, libEGL and libGLES are its, and +# Mesa installs only its vendor libraries behind it (libGLX_mesa, +# libEGL_mesa + 50_mesa.json). It is what provides pkg-config `opengl`, +# and what lets another GL driver (NVIDIA's) sit beside Mesa. +# +# NOT a BLFS page -- the 13.1 book only links to GLFS for it. The canonical +# repo is gitlab.freedesktop.org/glvnd/libglvnd; the NVIDIA GitHub copy is +# the one reachable from where this was written. CHECKED there: newest tag +# v1.7.0, and the option names (all left at their defaults). +# +# SWITCHING AN EXISTING MACHINE: libglvnd and mesa must be done in ONE go. +# Between the two, libGL.so.1 is glvnd's with no vendor behind it and every +# GL program fails: +# packagemanager stack sway --only libglvnd,mesa --run +# Programs already built keep working: the sonames (libGL.so.1, libEGL.so.1, +# libGLESv2.so.2) do not change, only who provides them. +git libglvnd \ + url=https://gitlab.freedesktop.org/glvnd/libglvnd.git \ + ref=v1.7.0 \ + needs=xorg7-lib \ + have=/usr/lib/pkgconfig/opengl.pc \ + build="rm -rf build; meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ + install="ninja -C build install" \ + expect="pkg-config --exists opengl && pkg-config --exists glx && test -e /usr/lib/libGLdispatch.so.0" + +# THE SOFTWARE RENDERER IS A SEPARATE DRIVER, and a hardware-only mesa +# fails with "egl: failed to create dri2 screen" and no fallback. The probe +# wants both: radeonsi for the GPU, swrast_dri.so for llvmpipe. (llvmpipe +# installs as swrast_dri.so -- the file is named after the interface, not +# the rasteriser, which is why looking for llvmpipe_dri.so finds nothing.) +book mesa expect="test -e /usr/lib/dri/swrast_dri.so -o -e /usr/lib/dri/kms_swrast_dri.so && test -e /usr/share/glvnd/egl_vendor.d/50_mesa.json && test -e /usr/lib/libGLX_mesa.so.0" +book xwayland +book libinput +book libxkbcommon + +# 3. wayland core, then the compositor from git (not in the book) +book wayland +book wayland-protocols +# 4. a terminal (foot, wayland-native) and its two small deps -# 7. audio: alsa-lib is wayland.stack's entry 1b (it has to be early); -# the server comes after media.stack, so pipewire finds gstreamer +# 5. fonts and 2D +# +# From here on the entries are the RECOMMENDATIONS worth having, named +# instead of inherited: with the book's suggestions no longer +# arrive by themselves, which is the point -- LibreOffice, OpenJDK, CUPS, +# Qt6/okular/VLC, texlive and the xfce/lxqt portals are not part of this +# desktop. Anything missed shows up as a clean "not found" at configure +# time and becomes one more line here. + +# 6. GTK4 + WebKitGTK (6.0 API), gstreamer (bad included -- wanted), mpv +book pango # GTK4 text layout +book gdk-pixbuf # GTK4 image loading +book hicolor-icon-theme +book adwaita-icon-theme # cursors and the default icon set +book gtk4 +book glib-networking # TLS for libsoup, or the browser has no https +book libsoup3 +# The CODECS, before the plugins that look for them. A WebRTC call +# negotiates Opus for audio and VP8 for video; both plugins are autodetected +# (opus in -base, vpx in -good), so a missing library means a call that +# connects and carries nothing. +# 6b. the media chain, in its own stack so it can be rebuilt on its own: +# codecs, gstreamer, the WebRTC pieces, woff2, webkitgtk, mpv. +# +# ONE LIST, TWO ENTRY POINTS. `packagemanager stack media --run` builds +# just that part; this include means the full desktop build is still one +# command and the entries are written down once. It sits HERE because +# webkitgtk needs gtk4, pango and libsoup3 above, and mesa and ffmpeg from +# further up. +include media + +# 7. audio: alsa-lib is entry 1b (it has to be); the tools can come later book alsa-utils book pipewire book wireplumber @@ -43,78 +192,54 @@ book wireplumber # probe fail for the wrong reason book sdl3 expect="strings /usr/lib/libSDL3.so* 2>/dev/null | grep -q libasound" +# 9. the nimbin2 family: make all && make PREFIX=/usr install + +# 10. wire the session together (root step) + + +# TeX is NOT built by this stack. +# +# Putting `book texlive` last only moved the ENTRY: texlive is harfbuzz's +# recommendation, so it was still pulled into an early entry's plan and +# failed there (its configure wants freetype-config, which FreeType removed +# years ago -- an upstream problem, not this desktop's). Nothing here needs +# TeX: the book's `make ... pdf ps` manual builds are commented out (1.12.87) +# and asymptote/biber are avoided. +# +# To build it later, on its own: packagemanager install texlive --run + # --------------------------------------------------------------------- -# THE COMPOSITOR AND ITS TOOLS +# Everything from here needs GIT, or is a plain tarball: kept to the END. +# +# A git entry failed with "git: command not found" -- git is itself a BLFS +# package, so anything cloned has to come after it is built. Grouping them +# here also means the desktop is finished before the extras start, and +# `--no-git` can skip the lot. # --------------------------------------------------------------------- -# scdoc: the man-page generator every sway-family project uses. Not in -# BLFS; SLFS has it (scdoc-1.11.5, from git.sr.ht -- the host seatd already -# comes from). It was assumed present because sway's line below asks for -# -Dman-pages=enabled; it was not, and grim stopped on -# doc/meson.build:1:8: ERROR: Program 'scdoc' not found or not executable -# Before sway, so a replay of this stack meets it in order. No dependencies. -git scdoc url=https://git.sr.ht/~sircmpwn/scdoc ref=1.11.5 prog=scdoc build="make PREFIX=/usr" install="make PREFIX=/usr install" +book git + +# -Dserver=enabled: seatd-user execs /usr/bin/seatd, so the BINARY must +# exist even though no daemon runs at boot. -Dlibseat-seatd=enabled is what +# libseat needs for LIBSEAT_BACKEND=seatd; logind stays enabled as a fallback +# but is unusable here (no pam_elogind). +git seatd url=https://git.sr.ht/~kennylevinsen/seatd ref=0.9.1 prog=seatd build="meson setup build --prefix=/usr --buildtype=release -Dserver=enabled -Dlibseat-seatd=enabled -Dlibseat-logind=elogind; ninja -C build" install="ninja -C build install" +# wlroots' DRM backend needs libdisplay-info; without it meson DISABLES drm +# at configure time and says so only in a summary line, and sway then dies at +# runtime with "Cannot create DRM backend: disabled at compile-time". +book libdisplay-info + +# -Dbackends=drm,libinput,... is explicit on purpose: an auto-detected +# backend list is a silent dependency, and this is the dependency that was +# silently missing. +git wlroots url=https://gitlab.freedesktop.org/wlroots/wlroots.git ref=0.19.2 have=/usr/lib/pkgconfig/wlroots-0.19.pc build="meson setup build --prefix=/usr --buildtype=release -Dbackends=drm,libinput,x11 -Dsession=enabled; ninja -C build" install="ninja -C build install" # needs=: book packages a git entry depends on -- nothing in the book orders # anything before a git build, so its dependencies must be named here. # the author's own options: man pages, swaybar, swaynag, bash completions git sway url=https://github.com/swaywm/sway ref=1.11 prog=sway needs=json-c,cairo,pango build="meson setup build --prefix=/usr --buildtype=release -Dman-pages=enabled -Dswaybar=true -Dswaynag=true -Dbash-completions=true; ninja -C build" install="ninja -C build install" git swaybg url=https://github.com/swaywm/swaybg ref=v1.2.1 prog=swaybg build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" install="ninja -C build install" - -# SCREENSHOTS AND IDLE: grim, slurp, swayidle. None is a BLFS page; grim and -# slurp are SLFS pages (the Sway book), swayidle is swaywm's own. Refs checked: -# grim v1.5.0 -- SLFS 1.5.0 fetches it from gitlab.freedesktop.org. -# NOT github.com/emersion/grim: that mirror stopped at -# v1.4.0 in 2022 when grim moved away from GitHub. -# slurp v1.5.0 -- git ls-remote on github.com/emersion/slurp; SLFS agrees -# swayidle v1.9.0 -- git ls-remote on github.com/swaywm/swayidle -# needs=libjpeg, not libjpeg-turbo: that is the book's ANCHOR for -# libjpeg-turbo-3.2.0 (`packagemanager search jpeg` shows both). -# Their options are asked for, not autodetected: jpeg (grim -t jpeg) and man -# pages (scdoc is here already: sway builds its man pages with it). -# -# grim -g "$(slurp)" ~/shot.png a region, picked with the mouse -# -# swayidle -Dlogind=disabled: its logind half (before-sleep, lock on -# `loginctl lock-session`) needs a logind SESSION, and sway here gets its -# seat from seatd-<user>, not elogind -- so auto-detection would find -# libelogind, build it in, and have nothing to talk to at runtime. Timeouts -# (`swayidle timeout 300 'swaymsg "output * power off"' resume ...`) are the -# part that works without it. -git grim url=https://gitlab.freedesktop.org/emersion/grim.git ref=v1.5.0 prog=grim needs=libpng,pixman,libjpeg build="meson setup build --prefix=/usr --buildtype=release -Djpeg=enabled -Dman-pages=enabled -Dbash-completions=true; ninja -C build" install="ninja -C build install" -git slurp url=https://github.com/emersion/slurp ref=v1.5.0 prog=slurp needs=cairo,libxkbcommon build="meson setup build --prefix=/usr --buildtype=release -Dman-pages=enabled; ninja -C build" install="ninja -C build install" -git swayidle url=https://github.com/swaywm/swayidle ref=v1.9.0 prog=swayidle build="meson setup build --prefix=/usr --buildtype=release -Dlogind=disabled -Dman-pages=enabled -Dbash-completions=true -Dzsh-completions=false -Dfish-completions=false; ninja -C build" install="ninja -C build install" -# wl-clipboard: wl-copy and wl-paste. Not in the book either, and sway is -# unusable without it once anything wants a clipboard from a script. -git wl-clipboard url=https://github.com/bugaevc/wl-clipboard ref=v2.2.1 \ - prog=wl-copy \ - build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ - install="ninja -C build install" \ - expect="wl-paste --version >/dev/null 2>&1" - -# wf-recorder: screen recording for a wlroots compositor. Not in the book. -# -# CHECKED against the repo at v0.6.0 (newest tag; master and two feature -# branches also exist): -# -# * audio is CONDITIONAL. meson_options.txt has `pulse` and `pipewire` -# as auto features and config.h.in has `#mesondefine HAVE_AUDIO`, which -# gates src/audio.cpp -- so with neither library found at configure -# time you get a recorder that cannot record sound. -# * and `--help` lists `-a, --audio` EITHER WAY: that text is not inside -# the #ifdef. So the probe checks what the binary LINKS, which is the -# only honest answer to "was audio built in". -# * `meson build` (as in the by-hand script) still works, but `meson -# setup build` is the spelling meson documents now. -# -# Its codecs come from ffmpeg, which is entry 1b, and the default is -# libx264 -- so gst10-plugins-ugly's x264 is not what it uses; ffmpeg's own -# libx264 support is. -git wf-recorder url=https://github.com/ammen99/wf-recorder.git \ - ref=v0.6.0 prog=wf-recorder \ - build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ - install="ninja -C build install" \ - expect="ldd /usr/bin/wf-recorder 2>/dev/null | grep -qE 'libpulse|libpipewire'" - +git tllist url=https://codeberg.org/dnkl/tllist ref=1.1.0 have=/usr/include/tllist.h build="meson setup build --prefix=/usr" install="ninja -C build install" +git fcft url=https://codeberg.org/dnkl/fcft ref=3.3.2 have=/usr/include/fcft build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" install="ninja -C build install" # light: backlight and keyboard brightness. Not in the book. # # THE UPSTREAM MOVED. haikarainen/light on GitHub is gone -- checked, and @@ -173,9 +298,136 @@ git light url=https://gitlab.com/dpeukert/light.git ref=main prog=light \ install="make install" \ expect="light -V >/dev/null 2>&1" -# foot, the terminal, and its two small libraries -git tllist url=https://codeberg.org/dnkl/tllist ref=1.1.0 have=/usr/include/tllist.h build="meson setup build --prefix=/usr" install="ninja -C build install" -git fcft url=https://codeberg.org/dnkl/fcft ref=3.3.2 have=/usr/include/fcft build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" install="ninja -C build install" +# jq. Not in the book, so this entry is ours -- and it was BUILT to write +# it, not guessed: +# +# - jq-1.8.2 is the newest tag (jq-1.8.2rc1 exists too; the release +# tarball is the non-rc one), md5 e288a84cf2957ea2cfc643aa9e59b473 +# - the release tarball ships `configure`, so no autogen.sh step +# - oniguruma is NOT on the system and does not need to be: configure says +# "Oniguruma was not found. Will use the packaged oniguruma", builds the +# copy in vendor/, and `jq -n '"abc"|test("b")'` answers true. That is +# the probe below -- `jq --version` would pass on a jq with no regex, +# which is the half of jq most scripts use. +tar jq url=https://github.com/jqlang/jq/releases/download/jq-1.8.2/jq-1.8.2.tar.gz \ + pkg=jq-1.8.2.tar.gz prog=jq \ + build="./configure --prefix=/usr; make" \ + install="make install" \ + expect="jq -n '\"abc\" | test(\"b\")' >/dev/null 2>&1" + +# wf-recorder: screen recording for a wlroots compositor. Not in the book. +# +# CHECKED against the repo at v0.6.0 (newest tag; master and two feature +# branches also exist): +# +# * audio is CONDITIONAL. meson_options.txt has `pulse` and `pipewire` +# as auto features and config.h.in has `#mesondefine HAVE_AUDIO`, which +# gates src/audio.cpp -- so with neither library found at configure +# time you get a recorder that cannot record sound. +# * and `--help` lists `-a, --audio` EITHER WAY: that text is not inside +# the #ifdef. So the probe checks what the binary LINKS, which is the +# only honest answer to "was audio built in". +# * `meson build` (as in the by-hand script) still works, but `meson +# setup build` is the spelling meson documents now. +# +# Its codecs come from ffmpeg, which is entry 1b, and the default is +# libx264 -- so gst10-plugins-ugly's x264 is not what it uses; ffmpeg's own +# libx264 support is. +git wf-recorder url=https://github.com/ammen99/wf-recorder.git \ + ref=v0.6.0 prog=wf-recorder \ + build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ + install="ninja -C build install" \ + expect="ldd /usr/bin/wf-recorder 2>/dev/null | grep -qE 'libpulse|libpipewire'" + +# VIEWERS: a document one and an image one. Neither is in the BLFS book. +# +# CHECKED with git ls-remote, all four at once: girara, zathura and +# zathura-pdf-poppler share the release tag **2026.07.18** (they are one +# project, released together -- mixing tags is the usual way to get a +# zathura that cannot open a PDF), and imv's newest is **v4.3.0**. +# imv's repo answers on both github.com/exec64/imv and .../eXeC64/imv; the +# lowercase spelling is used here. +# +# zathura is THREE packages: girara is its widget library, the core opens no +# file formats at all, and the backend is a separate plugin. A zathura with +# no plugin starts, shows a grey window and says nothing -- which is what +# the expect= below is for: it looks for the PLUGIN, not the binary. +# +# poppler (the backend's dependency) is a book page. It was in avoid.stack +# as libcupsfilters' child; it is wanted now. mupdf stays avoided -- it is +# the other backend and one is enough. +book poppler + +# girara installs **girara.pc**, not girara-gtk3.pc: its meson.build says +# `pkg.generate(name: 'girara', ...)`. The old guess made the stack refuse +# to record a girara that had installed perfectly -- "finished, but +# /usr/lib/pkgconfig/girara-gtk3.pc is still missing", which is the have= +# check doing its job on a wrong have=. +git girara url=https://github.com/pwmt/girara.git ref=2026.07.18 \ + have=/usr/lib/pkgconfig/girara.pc \ + build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ + install="ninja -C build install" + +# xxHash, BEFORE zathura. Mentioned eight times in the BLFS book and given +# no page of its own, so this entry is ours. +# +# zathura 2026.07.18 takes it as a hard dependency: +# Run-time dependency libxxhash found: NO (tried pkgconfig and cmake) +# meson.build:57:9: ERROR: Dependency "libxxhash" not found +# +# CHECKED in the checkout: newest tag v0.8.3; it ships libxxhash.pc.in, so +# the pkg-config module really is `libxxhash` -- the name meson asks for -- +# and `make install` runs install_libxxhash.pc among its targets. A plain +# Makefile, no cmake or meson: PREFIX and LIBDIR are what it takes. +git xxhash url=https://github.com/Cyan4973/xxHash.git ref=v0.8.3 \ + have=/usr/lib/pkgconfig/libxxhash.pc \ + build="make PREFIX=/usr LIBDIR=/usr/lib" \ + install="make PREFIX=/usr LIBDIR=/usr/lib install" \ + expect="pkg-config --exists libxxhash" + +# NO -Dsqlite: zathura has no such option. Its meson.build takes sqlite as +# an unconditional dependency -- `dependency('sqlite3', version: '>=3.35.0')` +# -- and its only options are synctex, seccomp, landlock, manpages, +# tests-x11, tests-wayland and convert-icon. `-Dsqlite=enabled` stopped +# meson dead with `Unknown option: "sqlite"`, which is what guessing an +# option name earns. sqlite3 is already installed here (p_sqlite). +git zathura url=https://github.com/pwmt/zathura.git ref=2026.07.18 \ + prog=zathura \ + build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ + install="ninja -C build install" + +git zathura-pdf-poppler url=https://github.com/pwmt/zathura-pdf-poppler.git \ + ref=2026.07.18 \ + have=/usr/lib/zathura/libpdf-poppler.so \ + build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ + install="ninja -C build install" \ + expect="test -e /usr/lib/zathura/libpdf-poppler.so" + +# imv: wayland image viewer. The binary is `imv`, which dispatches to +# imv-wayland or imv-x11 -- so the probe wants the wayland one, or on this +# machine it would pass on an imv that can only open windows under X. +# -licuuc: imv links libicuio and CALLS libicuuc (ubrk_close), and its +# link line carries -Wl,--no-undefined, so the indirect dependency is an +# error rather than a warning: +# undefined reference to symbol 'ubrk_close_78' +# /lib/libicuuc.so.78: error adding symbols: DSO missing from command line +# icu splits its API across libicuuc (common), libicui18n and libicuio; +# imv's meson names only the one it uses directly. Naming the other on the +# link line is the whole fix -- nothing is patched. +git imv url=https://github.com/exec64/imv.git ref=v4.3.0 \ + prog=imv \ + build="meson setup build --prefix=/usr --buildtype=release -D c_link_args=-licuuc; ninja -C build" \ + install="ninja -C build install" \ + expect="command -v imv-wayland >/dev/null 2>&1 || ldd /usr/bin/imv 2>/dev/null | grep -q wayland" + +# wl-clipboard: wl-copy and wl-paste. Not in the book either, and sway is +# unusable without it once anything wants a clipboard from a script. +git wl-clipboard url=https://github.com/bugaevc/wl-clipboard ref=v2.2.1 \ + prog=wl-copy \ + build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" \ + install="ninja -C build install" \ + expect="wl-paste --version >/dev/null 2>&1" + git foot url=https://codeberg.org/dnkl/foot ref=1.28.0 prog=foot build="meson setup build --prefix=/usr --buildtype=release; ninja -C build" install="ninja -C build install" # the rest of the fonts: icons, a patched terminal font, and Noto for # everything DejaVu does not cover. One step, because they are all "unpack @@ -183,11 +435,6 @@ git foot url=https://codeberg.org/dnkl/foot ref=1.28.0 prog=foot build="me cfg fonts tar dejavu-fonts url=https://downloads.sourceforge.net/dejavu/dejavu-fonts-ttf-2.37.tar.bz2 pkg=dejavu-fonts-ttf-2.37.tar.bz2 have=/usr/share/fonts/dejavu build="true" install="install -v -d -m755 /usr/share/fonts/dejavu; install -v -m644 ttf/*.ttf /usr/share/fonts/dejavu; fc-cache /usr/share/fonts/dejavu" - -# --------------------------------------------------------------------- -# YOUR OWN PROGRAMS: the Sw* tools and the browser. SDL3's image and font -# libraries are here because only these use them. -# --------------------------------------------------------------------- git sdl3-image url=https://github.com/libsdl-org/SDL_image ref=release-3.2.4 have=/usr/lib/libSDL3_image.so build="cmake -B build -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release; cmake --build build" install="cmake --install build" git sdl3-ttf url=https://github.com/libsdl-org/SDL_ttf ref=release-3.2.2 have=/usr/lib/libSDL3_ttf.so build="cmake -B build -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release; cmake --build build" install="cmake --install build" # Browser builds two programs, browser-mini and browser-big; the entry keeps diff --git a/stacks/vision.stack b/stacks/vision.stack index 85c2477..e0ba1b5 100644 --- a/stacks/vision.stack +++ b/stacks/vision.stack @@ -1,5 +1,5 @@ # vision.stack -- computer vision, kept out of the desktop stack. -# stack-version: 1.14.320 +# stack-version: 1.14.292 # # WHAT THIS MACHINE NEVER BUILDS is in avoid.stack, included by every stack # here: the exclusions are a property of the machine, not of one stack. diff --git a/test_lfs_crosschain.sh b/test_lfs_crosschain.sh index 668e422..d973042 100644 --- a/test_lfs_crosschain.sh +++ b/test_lfs_crosschain.sh @@ -185,15 +185,6 @@ LFS_BOOK="${2:-$(_find_book)}" PHASES_LIB="$(cd "$(dirname "$LFS_TOOL")" && pwd)/lfs-phases" export PKGUSR_LIB="$PHASES_LIB" T=$(mktemp -d /tmp/lfstest.XXXXXX) -# THE DESKTOP IS FIVE FILES (1.14.320): wayland, media and sway build as one -# (sway.stack includes the other two), apps and tools beside them. A check -# for an ENTRY reads them all, in build order, with the include lines taken -# out so the concatenation parses on its own in $T. -_DESKTOP_STACK="$T/desktop-all.stack" -for _dsf in wayland media sway apps tools; do - [ -f "$(dirname "${LFS_TOOL:-$0}")/stacks/$_dsf.stack" ] \ - && grep -vE '^include ' "$(dirname "${LFS_TOOL:-$0}")/stacks/$_dsf.stack" -done > "$_DESKTOP_STACK" 2>/dev/null PASS=0; FAIL=0 # Count in FILES, not shell variables. # @@ -202,26 +193,19 @@ PASS=0; FAIL=0 # incremented there dies with the subshell, so the suite printed 446 PASS # lines and reported 433. A wrong count is worse than no count: it can hide a # regression behind a number that still looks healthy. -# ...AND IT MUST NOT BE NAMED LIKE THE THING THAT GETS SWEPT UP. This was -# ${TMPDIR:-/tmp}/lfstest-count.$$, beside the /tmp/lfstest.$$ trees the suite -# builds -- and something a block runs sweeps `lfstest*`, so the counters -# vanished mid-run: every later ok() printed a shell error instead of counting, -# and the final line read "PASS: FAIL: SKIP: (final)". A name nothing -# globs for, and a mkdir before each write so a sweep cannot silence it again. -_COUNTDIR="$(mktemp -d "${TMPDIR:-/tmp}/lfscount.XXXXXX")" +_COUNTDIR="${TMPDIR:-/tmp}/lfstest-count.$$" mkdir -p "$_COUNTDIR" : > "$_COUNTDIR/pass" : > "$_COUNTDIR/fail" : > "$_COUNTDIR/skip" -_tick() { [ -d "$_COUNTDIR" ] || mkdir -p "$_COUNTDIR"; echo x >> "$_COUNTDIR/$1"; } -ok() { echo " PASS $1"; _tick pass; } -bad() { echo " FAIL $1"; _tick fail; } +ok() { echo " PASS $1"; echo x >> "$_COUNTDIR/pass"; } +bad() { echo " FAIL $1"; echo x >> "$_COUNTDIR/fail"; } # A check whose PRECONDITION is absent is neither pass nor fail. The rc.site # check needs a SysV book on disk and reported FAIL without one, so a clean # run looked like a broken one and the real failures had to be counted by # hand every time. -skip() { echo " SKIP $1"; _tick skip; } +skip() { echo " SKIP $1"; echo x >> "$_COUNTDIR/skip"; } # Lift ONE function out of a bash source file, to exercise it in isolation. # @@ -270,28 +254,16 @@ _slice_def() { # _slice_def <file> <python def> # the python-heredoc blocks report their own verdict; fold them in the same way # (77 = the block printed SKIP: its precondition is not on this machine) _count_rc() { - if [ "$1" -eq 0 ]; then _tick pass - elif [ "$1" -eq 77 ]; then _tick skip - else _tick fail; fi + if [ "$1" -eq 0 ]; then echo x >> "$_COUNTDIR/pass" + elif [ "$1" -eq 77 ]; then echo x >> "$_COUNTDIR/skip" + else echo x >> "$_COUNTDIR/fail"; fi } -_pass_total() { wc -l < "$_COUNTDIR/pass" 2>/dev/null | tr -d ' ' || echo 0; } -_fail_total() { wc -l < "$_COUNTDIR/fail" 2>/dev/null | tr -d ' ' || echo 0; } -_skip_total() { wc -l < "$_COUNTDIR/skip" 2>/dev/null | tr -d ' ' || echo 0; } +_pass_total() { wc -l < "$_COUNTDIR/pass" | tr -d ' '; } +_fail_total() { wc -l < "$_COUNTDIR/fail" | tr -d ' '; } +_skip_total() { wc -l < "$_COUNTDIR/skip" | tr -d ' '; } -# ONLY THE SHELL THAT SET IT MAY CLEAN UP. -# -# `trap cleanup EXIT` is inherited by every SUBSHELL, and this suite runs -# blocks inside ( ... ) and $( ... ) constantly. Each of those exits, fires -# the trap, and removes $T and the counters out from under the run that is -# still going: from that point every ok() printed a shell error instead of -# counting and the final line read "PASS: FAIL: SKIP: (final)". $$ is -# the same in a subshell; $BASHPID is not. -_MAIN_PID="${BASHPID:-$$}" -cleanup() { - [ "${BASHPID:-$$}" = "$_MAIN_PID" ] || return 0 - rm -rf "$T" "$_COUNTDIR" -} +cleanup() { rm -rf "$T" "$_COUNTDIR"; } trap cleanup EXIT export LFS="$T/lfs" @@ -3755,8 +3727,7 @@ elif [ -f "$_hlL" ]; then _b="$(awk -F: '$3>0 && $3<1000 {print $1}' /etc/passwd | sed -n 2p)" pkg_users_list() { printf '%s\n%s\n' "$_a" "$_b"; } STATE="$R/state"; MANIFESTS="$R/state/manifests"; LEASES="$R/state/leases" - for _f in script_write_targets _setid_conf _setid_policy_for _setid_pkg_for \ - _lease_dir \ + for _f in script_write_targets _setid_conf _setid_policy_for _lease_dir \ _lease_candidates lease_files lease_restore lease_settle \ _setid_state _setid_apply; do eval "$(_slice_fn "$_hlL" "$_f")" @@ -3765,9 +3736,7 @@ elif [ -f "$_hlL" ]; then printf 'other\n' > "$R/etc/shared"; chown "$_b:$_b" "$R/etc/shared"; chmod 644 "$R/etc/shared" printf 'secret\n' > "$R/etc/rootonly"; chmod 600 "$R/etc/rootonly" printf '%s\n' "$R/sbin/tool" > "$MANIFESTS/$_a.files" - # the 5th column is the package that installs it (1.14.310): without - # it the lease refuses, which is the point of the column - printf '%s | 4711 | root | - | %s\n' "$R/sbin/tool" "$_a" > "$R/setid.conf" + printf '%s | 4711 | root | -\n' "$R/sbin/tool" > "$R/setid.conf" export SETID_CONF="$R/setid.conf" printf 'configure_pkg() {\ncat > %s << "X"\nnew\nX\necho x >> %s\n}\n' \ "$R/etc/shared" "$R/etc/rootonly" > "$R/script" @@ -3820,59 +3789,6 @@ elif [ -f "$_hlL" ]; then [ "$(stat -c %U "$R/etc/rootonly")" = root ] \ || out="$out;a root file that is neither setid nor configured was leased" lease_restore "$_a" 2>/dev/null - # 8. A SETUID FILE IS LEASED ONLY TO THE PACKAGE THAT INSTALLS IT - # (1.14.310). /usr/bin/su landed in webkitgtk's time-window - # manifest because `cfg shadow-pam` ran during its build; the - # lease then chowned it, which clears the setuid bit, and `su` - # was p_webkitgtk 0711 until the phase ended. Three times. - printf 'x\n' > "$R/sbin/theirs"; chown root:root "$R/sbin/theirs"; chmod 4711 "$R/sbin/theirs" - printf '%s | 4711 | root | - | %s\n' "$R/sbin/tool" "$_a" > "$R/setid.conf" - printf '%s | 4711 | root | - | someone-else\n' "$R/sbin/theirs" >> "$R/setid.conf" - printf '%s\n' "$R/sbin/theirs" > "$MANIFESTS/$_a.files" - lease_files "$_a" "$R/script" install - [ "$(stat -c '%a %U' "$R/sbin/theirs")" = "4711 root" ] \ - || out="$out;another package's setuid file was leased ($(stat -c '%a %U' "$R/sbin/theirs"))" - lease_restore "$_a" 2>/dev/null - # ...and a setid line with NO package is leased to nobody - printf '%s | 4711 | root | -\n' "$R/sbin/theirs" > "$R/setid.conf" - lease_files "$_a" "$R/script" install - [ "$(stat -c '%a %U' "$R/sbin/theirs")" = "4711 root" ] \ - || out="$out;a setuid file with no package in setid.conf was leased" - lease_restore "$_a" 2>/dev/null - printf '%s | 4711 | root | - | %s\n' "$R/sbin/tool" "$_a" > "$R/setid.conf" - # 9. A LEASED SYMLINK THE PHASE DID NOT TOUCH GOES BACK (1.14.310). - # `cp -a` copies a RELATIVE link as a relative link, so the copy - # in the lease directory dangles, `cmp` cannot open it, and the - # failed compare read as "the phase rewrote it". Six - # /usr/sbin/iptables* links ended up owned by p_webkitgtk and - # written into its pkg.lst that way. - printf 'real\n' > "$R/sbin/xtables-nft-multi"; chown "$_b:$_b" "$R/sbin/xtables-nft-multi" - ln -sf xtables-nft-multi "$R/sbin/iptables"; chown -h "$_b:$_b" "$R/sbin/iptables" - printf '%s\n' "$R/sbin/iptables" > "$MANIFESTS/$_a.files" - lease_files "$_a" "$R/script" install - [ "$(stat -c %U "$R/sbin/iptables")" = "$_a" ] \ - || out="$out;a manifest symlink another package holds was not leased" - lease_settle "$_a" - [ "$(stat -c %U "$R/sbin/iptables")" = "$_b" ] \ - || out="$out;an untouched symlink was kept by the installer ($(stat -c %U "$R/sbin/iptables"))" - # ...and one the phase DID repoint stays with the installer - lease_files "$_a" "$R/script" install - ln -sfn xtables-legacy-multi "$R/sbin/iptables"; chown -h "$_a:$_a" "$R/sbin/iptables" - lease_settle "$_a" - [ "$(stat -c %U "$R/sbin/iptables")" = "$_b" ] \ - && out="$out;a repointed symlink was given back to the old owner" - # 10. the manifest filter drops what provenance says is not ours - eval "$(_slice_fn "$_hlL" _drop_other_packages_files)" - is_never_claimed() { return 1; } - printf '%s | 4711 | root | - | someone-else\n' "$R/sbin/theirs" >> "$R/setid.conf" - ln -sfn xtables-nft-multi "$R/sbin/iptables"; chown -h "$_a:$_a" "$R/sbin/iptables" - _kept="$(printf '%s\n%s\n%s\n' "$R/sbin/theirs" "$R/sbin/iptables" "$R/sbin/plainfile" \ - | _drop_other_packages_files "$_a")" - printf '%s\n' "$_kept" | grep -qxF "$R/sbin/theirs" \ - && out="$out;the manifest kept a setuid file setid.conf gives to another package" - printf '%s\n' "$_kept" | grep -qxF "$R/sbin/iptables" \ - && out="$out;the manifest kept a symlink into another package's file" - chown -h "$_b:$_b" "$R/sbin/iptables" # 6. the earlier-stages claim must not strip a setuid bit (1.14.265) eval "$(_slice_fn "$_hlL" _claim_candidates)" printf 'x\n' > "$R/sbin/suid"; chmod 4711 "$R/sbin/suid" @@ -4498,7 +4414,7 @@ if [ -f "$_au" ] && [ -f "$_ac2" ]; then echo ' "user setup") exit 0;;' echo ' "user create") exit 0;;' echo ' "user --help") exit 0;;' - echo ' "env --system") echo "export PKGUSR_SYS_SHARED=\"/zz/hub\""; exit 0;;' + echo ' "env --system") exit 0;;' echo 'esac' echo 'echo "STUB refused: $*" >&2; exit 2' } > "$_pr/pm"; chmod 755 "$_pr/pm" @@ -4522,14 +4438,8 @@ if [ -f "$_au" ] && [ -f "$_ac2" ]; then ls "$_pr/etc/pam.d/" | grep -q 'bak-' \ || _p="$_p;the PAM file was edited with no backup beside it" # the env fragment enumerates rather than lists - grep -q '"/zz/hub"/\*' "$_pr/etc/profile.d/shared-dirs.sh" 2>/dev/null \ + grep -q 'HOME"/Shared/\*' "$_pr/etc/profile.d/shared-dirs.sh" 2>/dev/null \ || _p="$_p;the SHARED_* variables are a fixed list, not what is there" - # ...and what it enumerates is THE HUB the tool names, not $HOME/Shared: - # in an app account that is its own data directory, and every FILE in it - # became a SHARED_* variable (1.14.312) - grep -v '^[[:space:]]*#' "$_pr/etc/profile.d/shared-dirs.sh" 2>/dev/null \ - | grep -q 'HOME/Shared\|HOME"/Shared' \ - && _p="$_p;the SHARED_* variables are still read from \$HOME/Shared" fi if [ -n "$_p" ]; then printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do @@ -4895,12 +4805,9 @@ PYST || _p="$_p;a git-tracked package is compared against a book version" printf '%s' "$_cu" | grep -q 'git_tracked.append' \ || _p="$_p;a git-tracked package is not reported as left alone" - # (1.14.307 replaced "left alone" with an answer: a branch is asked - # whether it moved, a tag whether a newer one exists. These two checks - # still looked for the old wording until 1.14.313.) - grep -q 'follow GIT, not the book' "$pm_src" \ + grep -q 'follow GIT, not the book -- left alone' "$pm_src" \ || _p="$_p;nothing says why those packages are not in the plan" - grep -q 'behind their branch -- fetch, rebuild, install' "$pm_src" \ + grep -q 'stack <name> --only <pkg> --run --force' "$pm_src" \ || _p="$_p;it does not say how to move a git-tracked package" # ...and that command really does default to everything timeout 60 python3 "$pm_src" update --help 2>&1 \ @@ -5055,7 +4962,7 @@ PYCL # GLVND (1.14.270): machine.conf enables it, so libglvnd must come # before mesa, and --clean must never delete a file that moved to # another package (libGL.so.1 goes from mesa to libglvnd). - _swy="$_DESKTOP_STACK" + _swy="$(dirname "$LFS_TOOL")/stacks/sway.stack" _mcf="$(dirname "$LFS_TOOL")/stacks/machine.conf" _l_g="$(grep -nE '^git +libglvnd' "$_swy" | head -1 | cut -d: -f1)" _l_m="$(grep -nE '^book +mesa' "$_swy" | head -1 | cut -d: -f1)" @@ -9075,54 +8982,15 @@ PY )" _lt="$(printf '%s\n' "$_lt" | tail -1)" if [ -f "$_lt/bin/srch" ]; then - # NEUTRALISE su BY PATH, NOT BY SED. - # - # This rewrote the launcher's last line with `sed '/^exec su - /'`, - # which assumed the launcher ends in exactly one way. Since 1.14.295 - # there are TWO ways out of it -- `exec su` with a terminal, script(1) - # with none -- and the `exec su` is indented inside the if, so the sed - # matched neither: the REAL su ran and the block failed with "user u_a - # does not exist". A stub first in PATH tests whatever shape the - # launcher has, including both of these. - # - # su's own arguments are `- <user> -c <command> --`, five of them, - # and what follows is the argument list the launcher built. - mkdir -p "$_lt/fake" - printf '#!/bin/bash\nshift 5\nprintf "<%%s>" "$@"\n' > "$_lt/fake/su" - chmod 755 "$_lt/fake/su" - # a pty turns \n into \r\n; the stub prints none, but strip it anyway - _runl() { PATH="$_lt/fake:$PATH" bash "$_lt/bin/$1" "${@:2}" 2>/dev/null | tr -d '\r'; } - [ "$(_runl srch a b)" = "<browser-mini><--app-id><srch><https://s.example/r?q=a+b>" ] \ - && [ "$(_runl srch)" = "<browser-mini><--app-id><srch><https://s.example/r>" ] \ - && [ "$(_runl fixed extra)" = "<browser-mini><--app-id><fixed><https://fixed.example/>" ] \ + for _l in srch fixed; do + sed 's/^exec su - [^ ]* -c .exec "\$0" "\$@". -- /printf "<%s>" /' "$_lt/bin/$_l" > "$_lt/$_l.t" + done + [ "$(bash "$_lt/srch.t" a b)" = "<browser-mini><--app-id><srch><https://s.example/r?q=a+b>" ] \ + && [ "$(bash "$_lt/srch.t")" = "<browser-mini><--app-id><srch><https://s.example/r>" ] \ + && [ "$(bash "$_lt/fixed.t" extra)" = "<browser-mini><--app-id><fixed><https://fixed.example/>" ] \ && ok "the launcher passes {} as arguments or a search term, and nothing without {}" \ - || bad "the launcher's argument handling is wrong: $(_runl srch a b) / $(_runl srch) / $(_runl fixed extra)" - # A MENU ENTRY HAS NO TERMINAL (1.14.295). su fails without one and - # its message goes nowhere, so the entry "does nothing". The launcher - # must notice and give it a pty. - if grep -q 'if \[ -t 0 \] && \[ -t 1 \]' "$pmt" && grep -q 'script -qec' "$pmt"; then - ok "with no terminal the launcher gives su a pty" - else - bad "the launcher runs su with no pty: every .desktop entry does nothing" - fi - # ...and a default page for a search launcher that was given nothing - # to search for (1.14.293): {https://site/} rather than the URL cut - # at its ? - _lt2="$( _lh2="$(mktemp -d)"; PKGUSR_CONFIG=/dev/null python3 - "$pmt" "$_lh2" <<'PY' 2>/dev/null -import types, sys -m = types.ModuleType("pm"); exec(compile(open(sys.argv[1]).read(), "pm", "exec"), m.__dict__) -h = sys.argv[2]; m._run = lambda c: True; m._home_of = lambda u: h; m._app_home = lambda u: "/home/x/" + u -m._write_launcher("me", "u_a", "browser-mini https://s.example/r?q={https://s.example/}", "dflt") -print(h) -PY -)" - _lt2="$(printf '%s\n' "$_lt2" | tail -1)" - _rund() { PATH="$_lt/fake:$PATH" bash "$_lt2/bin/dflt" "$@" 2>/dev/null | tr -d '\r'; } - [ "$(_rund)" = "<browser-mini><--app-id><dflt><https://s.example/>" ] \ - && [ "$(_rund x)" = "<browser-mini><--app-id><dflt><https://s.example/r?q=x>" ] \ - && ok "with nothing typed, a {url} default page wins over cutting at the ?" \ - || bad "the {url} default page is wrong: $(_rund) / $(_rund x)" - rm -rf "$_lt" "$_lt2" + || bad "the launcher's argument handling is wrong: $(bash "$_lt/srch.t" a b) / $(bash "$_lt/srch.t") / $(bash "$_lt/fixed.t" extra)" + rm -rf "$_lt" else bad "the launcher could not be generated" fi @@ -14202,7 +14070,7 @@ grep -q '"/etc/pkgusr/stacks"' "$LFS_TOOL" \ || _p="$_p;an installed lfs cannot find the stack files to refresh" grep -q 'no stack files found to refresh' "$LFS_TOOL" \ || _p="$_p;finding no stacks to refresh is silent" -_stk="$_DESKTOP_STACK" +_stk="$(dirname "$LFS_TOOL")/stacks/sway.stack" if [ -f "$_stk" ]; then # The recommendations are FOLLOWED again: a blanket --ignore-recommended # dropped things the builds hard-require (curl for cmake, icu for @@ -18910,7 +18778,7 @@ if command -v git >/dev/null 2>&1; then || _p="$_p;a valid ref no longer checks out" fi # the stack's own pin was the wrong one and is corrected -_ss="$_DESKTOP_STACK" +_ss="$(dirname "$LFS_TOOL")/stacks/sway.stack" [ -f "$_ss" ] && { grep -q 'usrsctp.git ref=1.0.0' "$_ss" \ && _p="$_p;the stack still pins a usrsctp tag that does not exist"; } if [ -n "$_p" ]; then @@ -20243,7 +20111,7 @@ fi # get their dependencies from the book, git entries got nothing. _p="" _pmU="$(dirname "$LFS_TOOL")/packagemanager" -_ssU="$_DESKTOP_STACK" +_ssU="$(dirname "$LFS_TOOL")/stacks/sway.stack" # the stack parser keeps needs=, and the sway entry declares its own if [ -f "$_ssU" ]; then python3 - "$_pmU" "$_ssU" <<'PYNEEDS' || _p="$_p;needs= is not parsed from a stack entry" @@ -20341,7 +20209,7 @@ printf '%s' "$_cr" | grep -q 'y pins" if bad == 1 else "ies pin' \ sed -n '/if getattr(args, "check_refs", False)/,+2p' "$_pmV" | grep -q 'sys.exit' \ || _p="$_p;--check-refs falls through into building the stack" # and the shipped stack no longer pins a ref that never existed -_ssV="$_DESKTOP_STACK" +_ssV="$(dirname "$LFS_TOOL")/stacks/sway.stack" [ -f "$_ssV" ] && { grep -q 'ref=1.24.1' "$_ssV" \ && _p="$_p;the stack still pins foot at a tag that does not exist"; } if [ -n "$_p" ]; then @@ -20373,7 +20241,7 @@ printf '%s' "$_nb" | grep -q 'unreachable += 1' \ printf '%s' "$_nb" | grep -q 'timeout=' \ || _p="$_p;the no-ref probe can hang" # the four local entries point at real repositories, not a placeholder -_ssW="$_DESKTOP_STACK" +_ssW="$(dirname "$LFS_TOOL")/stacks/sway.stack" if [ -f "$_ssW" ]; then grep -qE '^git +(minibrowser|swbr|swov|swas).*github\.com/nimbin2' "$_ssW" \ && _p="$_p;the local sway tools still point at the old placeholder urls" @@ -21820,7 +21688,7 @@ if [ -f "$pm_src" ]; then # the sdl3 probe must not name one library file: SDL3 installs # libSDL3.so.0.<n> and the bare symlink is not always there # $_sw belongs to the next block; name the file, do not borrow a variable - grep -q 'libSDL3.so\*' "$_DESKTOP_STACK" \ + grep -q 'libSDL3.so\*' "$(dirname "$LFS_TOOL")/stacks/sway.stack" \ || _p="$_p;the SDL3 probe names a path that may not exist" # AN UNMATCHED DEPENDENCY IS NOT A CLEAN RESULT. A silent miss would # under-report exactly the thing being looked for. @@ -22091,7 +21959,7 @@ fi # book, so the stack carries them -- and everything about them is ours to # keep true: refs, URLs, and the archive layouts. _p="" -_sw2="$_DESKTOP_STACK" +_sw2="$(dirname "$LFS_TOOL")/stacks/sway.stack" _fs="$(dirname "$LFS_TOOL")/stacks/fonts.sh" if [ -f "$_sw2" ]; then # jq is not in the book either, and its probe has to test the half of jq @@ -22216,10 +22084,8 @@ PYXX [ -n "$_aph" ] || _p="$_p;nothing resolves an application user's real home" printf '%s' "$_aph" | grep -q '_home_of(u) or' \ || _p="$_p;a home moved with usermod is ignored" - # (1.14.315: the directory lives in the hub, not in any home -- the - # account's ~/Shared is a link to it; see the block for 1.14.315) - grep -q 'share = os.path.join(_hub, args.name' "$pm_src" \ - || _p="$_p;the shared directory is not built in the hub" + grep -q 'share = os.path.join(_app_home(u), "Shared")' "$pm_src" \ + || _p="$_p;the shared directory is built from the wrong home" _apo="$(PKGUSR_CONFIG=/dev/null python3 - "$pm_src" 2>&1 <<'PYAH' import sys, types mod = types.ModuleType("pm"); mod.__file__ = sys.argv[1] @@ -22473,7 +22339,10 @@ _p="" # ENTRY has to read both -- eight assertions here failed the moment # the split landed, which is the split being noticed rather than a # fault in it (1.14.223). -_sw="$_DESKTOP_STACK" +_sw="$T/stacks-all.stack" +cat "$(dirname "$LFS_TOOL")/stacks/sway.stack" \ + "$(dirname "$LFS_TOOL")/stacks/media.stack" > "$_sw" 2>/dev/null \ + || _sw="$(dirname "$LFS_TOOL")/stacks/sway.stack" if [ -f "$_sw" ] && [ -f "$pm_src" ]; then # the order, read out of the stack itself _ord="$(python3 - "$pm_src" "$_sw" <<'PYORD' @@ -22513,11 +22382,7 @@ for r in mod._parse_stack_file(sys.argv[2]): # only allowed pipes and grep, so `gst-inspect-1.0 webrtcbin # >/dev/null 2>&1` -- a perfectly good probe -- was called thin.) _e = r["opts"]["expect"] - # (`pkg-config --exists` IS a test -- its exit status is the answer; - # librice and xxhash were called thin once the check read every - # desktop stack, 1.14.320.) - if any(t in _e for t in ("|", "grep", ">/dev/null", "test ", "[ ", - "--exists")): + if any(t in _e for t in ("|", "grep", ">/dev/null", "test ", "[ ")): continue print("THIN %s" % r["target"]) print("COUNT %d" % n) @@ -23651,7 +23516,7 @@ fi # what is on PATH; the ENTRY keeps its name because p_minibrowser is the # account already on the user's disk. _p="" -_ssZ="$_DESKTOP_STACK" +_ssZ="$(dirname "$LFS_TOOL")/stacks/sway.stack" if [ -f "$_ssZ" ]; then grep -qE '^git +wlroots .*have=/usr/lib/pkgconfig/wlroots-0\.19\.pc' "$_ssZ" \ || _p="$_p;wlroots' have= does not match the version it pins" @@ -24547,7 +24412,7 @@ fi # drm when it is absent, mentions it in a summary line, and builds happily. # The cost of that silence is a full desktop that cannot open a screen. _p="" -_ss6="$_DESKTOP_STACK" +_ss6="$(dirname "$LFS_TOOL")/stacks/sway.stack" _pmDR="$(dirname "$LFS_TOOL")/packagemanager" grep -qE '^book +libdisplay-info' "$_ss6" \ || _p="$_p;libdisplay-info is not in the sway stack" @@ -25567,603 +25432,6 @@ else ok "slow steps announce themselves, and headers install as their own package" fi -# ---- a library something still links against is not an orphan (1.14.304) --- # -# `update --clean` removes what the OLD version installed and the new one did -# not rewrite. webkitgtk rebuilt from the GTK4 port to the GTK3 one leaves -# libwebkitgtk-6.0.so.4 exactly there: same package, same owner, old mtime, -# in the old manifest -- an orphan by every test, and deleting it breaks every -# program built against it with no warning and no way back but a rebuild. -# -# Built for real: a library with a soname, a consumer that records it, and the -# consumer owned by ANOTHER account (a package cleans its own files freely). -_p="" -if command -v gcc >/dev/null 2>&1 && command -v readelf >/dev/null 2>&1; then - _g="$T/rdep"; rm -rf "$_g"; mkdir -p "$_g/bin" "$_g/lib" - printf 'int oldfn(void){return 42;}\n' > "$_g/old.c" - printf 'extern int oldfn(void); int main(void){return oldfn();}\n' > "$_g/user.c" - if gcc -shared -fPIC -Wl,-soname,libzzold.so.4 -o "$_g/lib/libzzold.so.4.13.8" \ - "$_g/old.c" 2>/dev/null \ - && ln -sf libzzold.so.4.13.8 "$_g/lib/libzzold.so.4" \ - && gcc -o "$_g/bin/consumer" "$_g/user.c" "$_g/lib/libzzold.so.4.13.8" 2>/dev/null; then - # linked against the FILE, not -lzzold: that needs a libzzold.so dev - # symlink, and DT_NEEDED records the soname either way - # another owner, when this suite runs as root; otherwise the guard's - # "a package cleans its own files" rule hides the case - chown daemon "$_g/bin/consumer" 2>/dev/null || true - _rd="$(python3 - "$pm_src" "$_g" <<'PYRD' -import importlib.machinery as m, os, sys, time -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() -g = sys.argv[2] -lib, link = g + "/lib/libzzold.so.4.13.8", g + "/lib/libzzold.so.4" -pm._ELF_SCAN_DIRS = (g + "/bin", g + "/lib") -gone = [] -pm.remove_paths_as_user = lambda u, paths: gone.extend(paths) -pm.printInfo = pm.printWarning = lambda *a, **k: None -own = os.stat(lib).st_uid -import pwd -user = pwd.getpwuid(own).pw_name -pm._orphan_sweep(user, [lib, link], time.time() + 1000) -print("KEPT" if not gone else "REMOVED") -os.rename(g + "/bin/consumer", g + "/consumer.parked") -gone2 = [] -pm.remove_paths_as_user = lambda u, paths: gone2.extend(paths) -pm._orphan_sweep(user, [lib, link], time.time() + 1000) -print("SWEPT" if len(gone2) == 2 else "STUCK") -PYRD -)" - case "$_rd" in - *KEPT*) ;; - *) _p="$_p;--clean would delete a library another package still links against" ;; - esac - case "$_rd" in - *SWEPT*) ;; - *) _p="$_p;a library nothing links against is no longer cleaned up" ;; - esac - else - _p="$_p;the fixture would not build" - fi - # and the question must be answerable on its own, not only as a side - # effect of --clean - grep -q '^def cmd_used_by' "$pm_src" \ - || _p="$_p;there is no way to ask what links against a library" - grep -q 'SONAME' "$pm_src" \ - || _p="$_p;the soname is guessed from the file name rather than read" - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "--clean keeps a library that is still linked against, and sweeps one that is not" - fi -else - skip "no gcc/readelf here: the library-still-in-use guard cannot be built" -fi - -# ---- turning the machine off is a membership (1.14.309) -------------------- # -# halt/poweroff/reboot are 4550 root:<collector>_power: setuid root so they -# can do it, executable only by the group so not everyone may. That is two -# modes a rebuild or a chown destroys silently, and `cfg power` had no -# expect=, so once it was cached "done" nothing ever looked again. -_p="" -_ss="$(dirname "$LFS_TOOL")/stacks/services.stack" -_pw="$(dirname "$LFS_TOOL")/stacks/power.sh" -if [ -f "$_ss" ] && [ -f "$_pw" ]; then - _pex="$(awk "/^cfg[[:space:]]+power/,/^\\\$/" "$_ss" | grep -c "expect=")" - [ "$_pex" -ge 1 ] || _p="$_p;cfg power has no expect=, so a lost setuid bit is never noticed" - grep -q "4550 root:" "$_ss" \ - || _p="$_p;the expect does not check the mode and owner of halt" - grep -q "power-perms" "$_ss" \ - || _p="$_p;the expect does not check the boot script" - # the step itself must still set both halves and re-apply /sys at boot - grep -q "chmod 4550" "$_pw" || _p="$_p;power.sh no longer sets 4550" - grep -q "sys/power/state" "$_pw" \ - || _p="$_p;nothing re-applies the group to /sys/power/state at boot" - # and the expect must be a runnable shell expression - _pexpr="$(sed -n "/^cfg[[:space:]]\+power/,+2p" "$_ss" | sed -n "s/^ *expect='\(.*\)'$/\1/p")" - if [ -n "$_pexpr" ]; then - bash -n -c "$_pexpr" 2>/dev/null || _p="$_p;the power expect= is not valid shell" - else - _p="$_p;the power expect= could not be read back out of the stack file" - fi - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "poweroff is a group membership, and the stack re-checks the modes that make it one" - fi -else - skip "no services.stack or power.sh here" -fi - -# ---- chown destroys a setuid bit (1.14.308) -------------------------------- # -# Ownership is attributed BY TIME WINDOW: every file newer than the build -# stamp. A webkitgtk build runs for hours, and anything that touched -# /usr/bin/su in that window -- `lfs-helper setid`, a shadow reinstall, `cfg -# shadow-pam` -- made su "newer than the stamp". It is root-owned, which the -# sweep treated as fair game, so it was chowned to p_webkitgtk: -# -rwx--x--x 1 p_webkitgtk p_webkitgtk /bin/su -# The kernel clears setuid/setgid on chown, so ONE command took the file and -# removed the bit that makes su work. Twice, on this machine. -_p="" -_hl="$(dirname "$LFS_TOOL")/lfs-helper" -if [ -f "$_hl" ]; then - grep -q 'it is setuid/setgid' "$_hl" \ - || _p="$_p;a setuid file can still be taken by the ownership sweep" - grep -q '\[ "\$_root_ok" != 1 \] && \[ "\$cur" = root \]' "$_hl" \ - || _p="$_p;the chown pass does not apply the root-owned rule the manifest does" - # the mode test itself, against real files - _sd="$T/setid"; rm -rf "$_sd"; mkdir -p "$_sd" - _bad="" - for _m in 4711 2755 6755 0711 0755; do - : > "$_sd/f$_m"; chmod "$_m" "$_sd/f$_m" - case "$(stat -c %a "$_sd/f$_m" 2>/dev/null)" in - [1-7][0-7][0-7][0-7]) _hit=skip ;; - *) _hit=take ;; - esac - case "$_m:$_hit" in - 4711:skip|2755:skip|6755:skip|0711:take|0755:take) ;; - *) _bad="$_bad $_m($_hit)" ;; - esac - done - [ -z "$_bad" ] || _p="$_p;the setuid test judges these wrongly:$_bad" - # and the mode that matters is declared, so a repair knows what to restore - _si="$(dirname "$LFS_TOOL")/stacks/setid.conf" - if [ -f "$_si" ]; then - grep -qE '^/usr/bin/su[[:space:]]*\|[[:space:]]*4711' "$_si" \ - || _p="$_p;setid.conf does not declare su as 4711 root" - fi - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "the ownership sweep never takes a setuid file, nor a root file a package user could not have written" - fi -else - skip "no lfs-helper here: the ownership sweep cannot be checked" -fi - -# ---- who INSTALLED it, not who holds it (1.14.310) ------------------------- # -# The setuid damage came back a third time, and with it six root-owned -# symlinks: -# lrwxrwxrwx 1 p_webkitgtk p_webkitgtk /usr/sbin/iptables -> xtables-nft-multi -# made by `cfg iptables-nft` on a different day, and recorded in -# /usr/src/pkgusr/p_webkitgtk/pkg.lst. Nothing in the sweep or the lease -# asked WHO INSTALLS a file -- only who owns it now, which is exactly the -# thing that had gone wrong. Two facts that do not depend on current -# ownership: setid.conf names the package behind each setuid file, and a -# symlink belongs to whoever owns its target. -_p="" -_hl3="$(dirname "$LFS_TOOL")/lfs-helper" -_si3="$(dirname "$LFS_TOOL")/stacks/setid.conf" -if [ -f "$_hl3" ] && [ -f "$_si3" ]; then - grep -qE '^/usr/bin/su[[:space:]]*\|[^|]*\|[^|]*\|[^|]*\|[[:space:]]*shadow[[:space:]]*$' "$_si3" \ - || _p="$_p;setid.conf does not name shadow as the package that installs su" - _nocol="$(awk -F'|' '/^[[:space:]]*(#|$)/ { next } - { n = split($0, c, "|"); k = (n >= 5 ? c[5] : "") - gsub(/[[:space:]]/, "", k) - if (k == "") { f = c[1]; gsub(/[[:space:]]/, "", f); print f } }' "$_si3")" - [ -z "$_nocol" ] || _p="$_p;setid.conf lines with no package (leased to nobody): $(echo $_nocol)" - # the reader itself - ( eval "$(_slice_fn "$_hl3" _setid_conf)"; eval "$(_slice_fn "$_hl3" _setid_pkg_for)" - warn() { :; } - SETID_CONF="$_si3" _setid_pkg_for /usr/bin/su | grep -qx shadow ) \ - || _p="$_p;_setid_pkg_for does not read the package column" - # ...and the old three-column reader still sees the group - ( eval "$(_slice_fn "$_hl3" _setid_conf)"; eval "$(_slice_fn "$_hl3" _setid_policy_for)" - SETID_CONF="$_si3" _setid_policy_for /usr/libexec/dbus-daemon-launch-helper \ - | grep -qx '4750|root|messagebus' ) \ - || _p="$_p;the 5th column broke _setid_policy_for's group" - grep -q 'not taking \$p -- setid.conf says' "$_hl3" \ - || _p="$_p;the ownership sweep does not consult setid.conf" - grep -q 'not taking \$p -- it points at' "$_hl3" \ - || _p="$_p;the ownership sweep takes a symlink into another package" - grep -q 'readlink "\$dir/files/\$n"' "$_hl3" \ - || _p="$_p;lease_settle still compares symlinks with cmp" - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "provenance beats current ownership: setid.conf's package column and a symlink's target" - fi -else - skip "no lfs-helper/setid.conf here: provenance cannot be checked" -fi - -# ---- disown: taking back what a package cannot have installed (1.14.310) --- # -# The repair for a machine that already went through those doors. `reclaim` -# pushes ownership towards pkg.lst; this one distrusts pkg.lst, because the -# record is what went wrong. Real files, real accounts, real chown. -_p="" -# self-sufficient: a subset that does not include an earlier block must not -# inherit its variables (the block runs alone under `--only`) -pm_src="$(dirname "$LFS_TOOL")/packagemanager" -if [ "$(id -u)" != 0 ]; then - skip "disown (needs root for chown)" -else - _dw="$(python3 - "$pm_src" <<'PYDIS' -import os, sys, tempfile, pwd, importlib.machinery as m - -R = tempfile.mkdtemp(prefix="lfsdisown.", dir="/var/tmp") -# two real accounts stand in for package users; the prefix is turned off so -# the account name IS the package name -accts = [l.pw_name for l in pwd.getpwall() if 0 < l.pw_uid < 1000][:2] -if len(accts) < 2: - print("SKIP no two system accounts"); raise SystemExit -A, B = accts - -os.makedirs(f"{R}/base/pkgusr/{A}") -os.makedirs(f"{R}/base/pkgusr/{B}") # the package the symlink really is -os.makedirs(f"{R}/stacks"); os.makedirs(f"{R}/sbin") -open(f"{R}/conf", "w").write("pkgusr_prefix=\ncollector_prefix=\n") - -# 1. a setuid file setid.conf gives to someone else, already chowned (so its -# setuid bit is gone -- that is what chown does) -open(f"{R}/sbin/su", "w").write("x") -os.chmod(f"{R}/sbin/su", 0o711) -os.chown(f"{R}/sbin/su", pwd.getpwnam(A).pw_uid, -1) -# 2. a symlink whose target belongs to another package -open(f"{R}/sbin/xtables-nft-multi", "w").write("x") -os.chown(f"{R}/sbin/xtables-nft-multi", pwd.getpwnam(B).pw_uid, -1) -os.symlink("xtables-nft-multi", f"{R}/sbin/iptables") -os.lchown(f"{R}/sbin/iptables", pwd.getpwnam(A).pw_uid, -1) -# 3. a file that really is this package's, which must be left alone -open(f"{R}/sbin/mine", "w").write("x") -os.chown(f"{R}/sbin/mine", pwd.getpwnam(A).pw_uid, -1) - -open(f"{R}/stacks/setid.conf", "w").write( - f"{R}/sbin/su | 4711 | root | - | shadow\n") -open(f"{R}/base/pkgusr/{A}/pkg.lst", "w").write( - f"{R}/sbin/su\n{R}/sbin/iptables\n{R}/sbin/mine\n") -os.makedirs(f"{R}/state/manifests") -open(f"{R}/state/manifests/{A}.files", "w").write( - f"{R}/sbin/su\n{R}/sbin/iptables\n{R}/sbin/mine\n") - -os.environ.update(PKGUSR_BASE=f"{R}/base", PKGUSR_CONFIG=f"{R}/conf", - PM_STACK_DIR=f"{R}/stacks", LFS_PKGUSR_DIR=f"{R}/state") -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() - -class Args: - packages = [A]; all = False; run = False -pm.cmd_disown(Args()) -print("DRYRUN-KEPT", pwd.getpwuid(os.lstat(f"{R}/sbin/su").st_uid).pw_name == A) - -Args.run = True -pm.cmd_disown(Args()) -st = os.lstat(f"{R}/sbin/su") -print("SU-ROOT", st.st_uid == 0) -print("SU-MODE", oct(st.st_mode & 0o7777) == "0o4711") -print("LINK-BACK", - pwd.getpwuid(os.lstat(f"{R}/sbin/iptables").st_uid).pw_name == B) -print("KEPT-MINE", - pwd.getpwuid(os.lstat(f"{R}/sbin/mine").st_uid).pw_name == A) -left = open(f"{R}/base/pkgusr/{A}/pkg.lst").read().split() -print("LST", sorted(left) == [f"{R}/sbin/mine"]) -man = open(f"{R}/state/manifests/{A}.files").read().split() -print("MAN", sorted(man) == [f"{R}/sbin/mine"]) -import shutil; shutil.rmtree(R, ignore_errors=True) -PYDIS -)" - case "$_dw" in - *"SKIP"*) skip "disown (no two system accounts to stand in for packages)" ;; - *) - case "$_dw" in *"DRYRUN-KEPT True"*) ;; *) _p="$_p;a dry run changed ownership" ;; esac - case "$_dw" in *"SU-ROOT True"*) ;; *) _p="$_p;a setuid file setid.conf gives to another package was not returned to root" ;; esac - case "$_dw" in *"SU-MODE True"*) ;; *) _p="$_p;the mode from setid.conf was not put back (su stays broken)" ;; esac - case "$_dw" in *"LINK-BACK True"*) ;; *) _p="$_p;a symlink was not given to the owner of its target" ;; esac - case "$_dw" in *"KEPT-MINE True"*) ;; *) _p="$_p;a file the package really installed was taken away" ;; esac - case "$_dw" in *"LST True"*) ;; *) _p="$_p;pkg.lst still records the paths -- --clean would delete them" ;; esac - case "$_dw" in *"MAN True"*) ;; *) _p="$_p;the build manifest still records them -- the next lease re-arms" ;; esac - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "disown hands back what a package cannot have installed, and clears the record" - fi ;; - esac -fi - -# ---- --clean: park, reinstall, THEN judge (1.14.311) ----------------------- # -# `update webkitgtk --clean --run` ended green and left -# jsc/jsc.h:29: fatal error: jsc/JSCOptions.h: No such file or directory -# CMake skips a file whose content is unchanged, so the jsc headers kept their -# old mtime and --clean called them orphans. The reapply meant to repair that -# ran `bash ~/install_<account>` -- a file 1.14.16 stopped writing -- with its -# output discarded and its exit status ignored. It had not worked for -# ~300 releases. -_p="" -pm_src="$(dirname "$LFS_TOOL")/packagemanager" -_cl="$(python3 - "$pm_src" <<'PYCL' -import importlib.machinery as m, os, sys, tempfile, time -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() -R = tempfile.mkdtemp(prefix="lfsclean.") -pm.TMP_DIR = R + "/pmtmp" -pm.printInfo = pm.printWarning = pm.printError = lambda *a, **k: None -pm.write_pkg_list = lambda *a, **k: None -pm.remove_paths_as_user = lambda u, ps: [os.unlink(p) for p in ps - if os.path.lexists(p) and not os.path.isdir(p)] -user = __import__("pwd").getpwuid(os.getuid()).pw_name -same, orphan = R + "/JSCOptions.h", R + "/old-only.h" -def fixture(): - for p in (same, orphan): - open(p, "w").write("old") - os.utime(p, (1, 1)) -since = time.time() - -# 1. the install step re-creates the unchanged header, not the orphan -fixture() -pm._reapply_install = lambda u: (open(same, "w").write("cur"), True)[1] -pm._clean_after_install(user, [same, orphan], since) -print("KEPT", os.path.exists(same)) -print("ORPHAN-GONE", not os.path.exists(orphan)) -print("NO-LITTER", not any(f.endswith(pm._PARK_SUFFIX) for f in os.listdir(R))) - -# 2. the install step FAILS: everything is put back, nothing deleted -fixture() -pm._reapply_install = lambda u: False -pm._clean_after_install(user, [same, orphan], since) -print("RESTORED", os.path.exists(same) and os.path.exists(orphan) - and open(same).read() == "old") -print("NO-LEDGER", not os.path.exists(pm._park_ledger(user))) -import shutil; shutil.rmtree(R, ignore_errors=True) -PYCL -)" -for _k in KEPT ORPHAN-GONE NO-LITTER RESTORED NO-LEDGER; do - case "$_cl" in *"$_k True"*) ;; *) _p="$_p;--clean: $_k failed" ;; esac -done -_ra="$(python3 - "$pm_src" <<'PYRA' -import sys, re -s = open(sys.argv[1]).read() -f = s[s.index("def _reapply_install("):] -f = f[:f.index("\ndef ", 1)] -print("DEVNULL" if "DEVNULL" in f else "LOGGED") -print("CHECKED" if "return True" in f and "returncode" in f else "UNCHECKED") -print("VERSIONED" if "_versioned_script_in" in f else "STALE-NAME") -PYRA -)" -case "$_ra" in *LOGGED*) ;; *) _p="$_p;the reapply still throws its output away" ;; esac -case "$_ra" in *CHECKED*) ;; *) _p="$_p;the reapply's exit status is still ignored" ;; esac -case "$_ra" in *VERSIONED*) ;; *) _p="$_p;the reapply still runs install_<account>, which no longer exists" ;; esac -if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done -else - ok "--clean deletes only what a reinstall does not put back, and undoes itself when the reinstall fails" -fi - -# ---- the shared hub is a system path, not a home (1.14.312) --------------- # -# The links to every shared directory lived in the main user's ~/Shared, so -# nothing but that one login could find them by a fixed path, and every -# login enumerated ITS OWN $HOME/Shared for the SHARED_* variables -- which -# in an app account is its data directory. Now: one hub, and ~/Shared is a -# link to it. The move keeps anything that is not ours. -_p="" -pm_src="$(dirname "$LFS_TOOL")/packagemanager" -if [ "$(id -u)" != 0 ]; then - skip "the shared hub move (needs root for chown)" -else - _hb="$(python3 - "$pm_src" <<'PYHUB' -import importlib.machinery as m, os, sys, tempfile, pwd, shutil -R = tempfile.mkdtemp(prefix="lfshub.", dir="/var/tmp") -open(R + "/conf", "w").write("shared_hub=%s/hub\nmain_user=root\n" % R) -os.environ["PKGUSR_CONFIG"] = R + "/conf" -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() -pm.printInfo = pm.printWarning = lambda *a, **k: None -home = R + "/home" -pm._home_of = lambda u: home -print("CONFIGURED", pm.shared_hub() == R + "/hub") -print("ENV", pm._system_env_values().get("PKGUSR_SYS_SHARED") == R + "/hub") - -# an old layout: ~/Shared holding one of our links -os.makedirs(home + "/Shared"); os.makedirs(R + "/u_claude/Shared") -os.symlink(R + "/u_claude/Shared", home + "/Shared/Claude") -pm._ensure_shared_hub("root") -print("MOVED", os.path.islink(R + "/hub/Claude") - and os.readlink(R + "/hub/Claude") == R + "/u_claude/Shared") -print("HOMELINK", os.path.islink(home + "/Shared") - and os.path.realpath(home + "/Shared") == os.path.realpath(R + "/hub")) -print("ROOTOWNED", os.stat(R + "/hub").st_uid == 0) -# twice is the same as once -pm._ensure_shared_hub("root") -print("IDEMPOTENT", sorted(os.listdir(R + "/hub")) == ["Claude"]) - -# a ~/Shared with a file of YOURS in it stays a directory -os.unlink(home + "/Shared"); os.makedirs(home + "/Shared") -open(home + "/Shared/notes.txt", "w").write("mine") -os.symlink("/nowhere-x", home + "/Shared/Other") -pm._ensure_shared_hub("root") -print("KEPT-MINE", os.path.isdir(home + "/Shared") and not os.path.islink(home + "/Shared") - and os.path.exists(home + "/Shared/notes.txt")) -print("MOVED-OURS", os.path.islink(R + "/hub/Other") and not os.path.lexists(home + "/Shared/Other")) -shutil.rmtree(R, ignore_errors=True) -PYHUB -)" - for _k in CONFIGURED ENV MOVED HOMELINK ROOTOWNED IDEMPOTENT KEPT-MINE MOVED-OURS; do - case "$_hb" in *"$_k True"*) ;; *) _p="$_p;shared hub: $_k failed" ;; esac - done - _sv="$(dirname "$LFS_TOOL")/stacks/services.stack" - grep -q '\[ -L "\$h/Shared" \]' "$_sv" \ - || _p="$_p;a machine with the old ~/Shared never re-runs cfg app-users" - # 1.14.315: the DATA lives in /home/Shared, the homes are 0700. 0750 - # let the main user -- a member of every account's group, for the - # shared directory -- read the account's profile and cookies. - grep -q '^SHARED_HUB_DEFAULT = "/home/Shared"$' "$pm_src" \ - || _p="$_p;the shared directories do not default to /home/Shared" - _sau="$(_slice_def "$pm_src" _setup_application_user)" - printf '%s' "$_sau" | grep -q '"chmod", "0700", home' \ - || _p="$_p;an application home is still readable by its group" - printf '%s' "$_sau" | grep -q '"chmod", "0750", home' \ - && _p="$_p;an application home is still made 0750 somewhere" - printf '%s' "$_sau" | grep -q 'ln", "-s", share, _in_home' \ - || _p="$_p;the account's ~/Shared is not linked to its hub directory" - printf '%s' "$_sau" | grep -q 'moved (that is yours to do)' \ - || _p="$_p;a ~/Shared with files in it could be replaced" - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "the shared links live in one system hub; ~/Shared is a link to it, and nothing of yours is moved" - fi -fi - -# ---- needs= speaks the package's name; the book files it by anchor (1.14.317) -# `needs=libjpeg-turbo` on grim: the book's anchor is `libjpeg`, so the -# stack tried to INSTALL an installed package and stopped on "not found". -_p="" -pm_src="$(dirname "$LFS_TOOL")/packagemanager" -_rn="$(python3 - "$pm_src" <<'PYRN' -import importlib.machinery as m, sys -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() -pm.printInfo = lambda *a, **k: None -out = (" [package] libjpeg libjpeg-turbo-3.2.0 installed libjpeg-turbo-3.2.0\n" - " [package] openjpeg2 OpenJPEG-2.5.4\n" - " [package] a1 foo-1.0\n [package] a2 foo-2.0\n") -pm.run_blfs = lambda a: (0, out, "") -print("MAP", pm._resolve_need("libjpeg-turbo") == "libjpeg") -print("ANCHOR", pm._resolve_need("libjpeg") == "libjpeg") -print("AMBIG", pm._resolve_need("foo") == "foo") -print("NONE", pm._resolve_need("nothing") == "nothing") -PYRN -)" -for _k in MAP ANCHOR AMBIG NONE; do - case "$_rn" in *"$_k True"*) ;; *) _p="$_p;needs= resolution: $_k failed" ;; esac -done -grep -q '_resolve_need(x.strip())' "$pm_src" \ - || _p="$_p;the stack's needs= loop does not resolve names" -grep -E '^git +grim ' "$(dirname "$LFS_TOOL")/stacks/sway.stack" | grep -q 'libjpeg-turbo' \ - && _p="$_p;sway.stack still names libjpeg-turbo instead of the anchor" -if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done -else - ok "needs= accepts a package's own name when the book files it under another anchor" -fi - -# ---- the desktop is three stacks, each entry written once (1.14.320) ------ # -# sway.stack had grown to 477 lines answering four questions. Split: -# wayland (the base), media (unchanged), sway (compositor + tools + your -# programs), apps (viewers), tools (CLI). `stack sway` must still be the -# whole desktop in one command, and nothing may be written twice. -_p="" -_sd="$(dirname "$LFS_TOOL")/stacks" -grep -nE '^include +(wayland|media)$' "$_sd/sway.stack" | cut -d: -f2 | tr '\n' ' ' \ - | grep -q '^include wayland include media $' \ - || _p="$_p;sway.stack does not include wayland, then media" -for _f in wayland apps tools; do - [ -f "$_sd/$_f.stack" ] || _p="$_p;no $_f.stack" -done -_dup="$(for _f in wayland media sway apps tools vision diag dev network services; do - grep -hoE '^(book|git|tar|cfg|check) +[^ ]+' "$_sd/$_f.stack" 2>/dev/null - done | awk '{print $2}' | sort | uniq -d | tr '\n' ' ')" -[ -z "$_dup" ] || _p="$_p;written in more than one stack: $_dup" -for _e in "wayland mesa" "wayland wlroots" "wayland gtk4" "sway sway" "sway foot" \ - "sway grim" "apps zathura" "apps imv" "tools jq" "tools translate-shell"; do - set -- $_e - grep -qE "^(book|git|tar) +$2( |\$)" "$_sd/$1.stack" \ - || _p="$_p;$2 is not in $1.stack" -done -if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done -else - ok "the desktop is wayland + media + sway in one command, apps and tools beside it, each entry once" -fi - -# ---- "phased" means it ends in the runner (1.14.306) ----------------------- # -# `update <git-pkg>` warned "git script isn't phased" for a perfectly current -# script, because it tested for `unpack_pkg` -- the RARE OVERRIDE that replaces -# the built-in unpack. A correct script does not define it: the runner's own -# unpack is what clones git. The advice that followed (`template --git`) -# rewrites the file and takes any hand-tuned build_pkg with it -- on this -# machine that is imv's `-D c_link_args=-licuuc`. -_p="" -_ok="$(python3 - "$pm_src" <<'PYPH' -import importlib.machinery as m, sys -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() -cur = ('#!/bin/bash\n### phases: all\nname="imv"\nis_git="1"\n' - 'build_pkg() { meson setup build; }\n' - '. "${PKGUSR_LIB:-lfs-phases}" && pkgusr_run "$@"\n') -old = ('#!/bin/bash\nname=imv\ngit clone https://x/y.git\n' - 'cd y && meson setup build && ninja -C build install\n') -print("CUR", pm._is_phased_script(cur)) -print("OLD", pm._is_phased_script(old)) -print("NOUNPACK", "unpack_pkg" not in cur) -PYPH -)" -case "$_ok" in *"CUR True"*) ;; *) _p="$_p;a current phased script is not recognised as phased" ;; esac -case "$_ok" in *"OLD False"*) ;; *) _p="$_p;an unphased script passes as phased" ;; esac -case "$_ok" in *"NOUNPACK True"*) ;; *) _p="$_p;the fixture does not reproduce the case" ;; esac -grep -q 'unpack_pkg" not in open' "$pm_src" \ - && _p="$_p;the phased test still asks for the rare unpack_pkg override" -if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done -else - ok "a script is phased when it ends in the runner, not when it overrides unpack" -fi - -# ---- a branch is not a point in time (1.14.305) ---------------------------- # -# `git checkout master` on an ALREADY CLONED tree says "Already on 'master'" -# and stays exactly where it was -- so a package pinned to a branch fetched -# the new commits and rebuilt the old ones, for ever, while reporting that it -# had updated. A tag must do the opposite and never move. -_p="" -_ph="$(dirname "$LFS_TOOL")/lfs-phases" -if [ -f "$_ph" ] && command -v git >/dev/null 2>&1; then - grep -q 'checkout -B "$_pp_ref" "origin/$_pp_ref"' "$_ph" \ - || _p="$_p;a branch ref is not moved to the remote on a rebuild" - grep -q 'checkout --detach' "$_ph" \ - || _p="$_p;a tag or commit is not checked out detached" - grep -q 'symbolic-ref --short refs/remotes/origin/HEAD' "$_ph" \ - || _p="$_p;with no ref at all nothing follows the default branch" - # and the behaviour itself, against a real repository - _gr="$T/gitref"; rm -rf "$_gr"; mkdir -p "$_gr/up" - ( cd "$_gr/up" && git init -q -b master . \ - && git config user.email t@t && git config user.name t \ - && echo one > V && git add -A && git commit -qm one && git tag v1 ) >/dev/null 2>&1 - git clone -q "$_gr/up" "$_gr/clone" >/dev/null 2>&1 - ( cd "$_gr/up" && echo two > V && git add -A && git commit -qm two ) >/dev/null 2>&1 - ( cd "$_gr/clone" && git fetch -q --all --tags --prune ) >/dev/null 2>&1 - _was="$(git -C "$_gr/clone" rev-parse --short HEAD 2>/dev/null)" - git -C "$_gr/clone" checkout -q master >/dev/null 2>&1 - _plain="$(git -C "$_gr/clone" rev-parse --short HEAD 2>/dev/null)" - git -C "$_gr/clone" checkout -q -B master origin/master >/dev/null 2>&1 - _moved="$(git -C "$_gr/clone" rev-parse --short HEAD 2>/dev/null)" - [ "$_plain" = "$_was" ] \ - || _p="$_p;this git does not reproduce the bug (plain checkout moved)" - [ -n "$_moved" ] && [ "$_moved" != "$_was" ] \ - || _p="$_p;checkout -B did not move the branch to the remote" - # the reporting side: branch vs tag, and what is newer - _gs="$(python3 - "$pm_src" <<'PYGS' -import importlib.machinery as m, sys -pm = m.SourceFileLoader("pm", sys.argv[1]).load_module() -print("VSORT", pm._vsort_key("1.9.0") < pm._vsort_key("1.10.0")) -print("VAR", pm._script_var('link="https://x/y"\ngit_ref="2.0.0"\n', "git_ref")) -print("CMD", hasattr(pm, "cmd_git_status"), hasattr(pm, "cmd_git_ref")) -PYGS -)" - case "$_gs" in *"VSORT True"*) ;; *) _p="$_p;1.10.0 does not sort after 1.9.0" ;; esac - case "$_gs" in *"VAR 2.0.0"*) ;; *) _p="$_p;the ref cannot be read out of an install script" ;; esac - case "$_gs" in *"CMD True True"*) ;; *) _p="$_p;there is no way to see or pick a git ref" ;; esac - if [ -n "$_p" ]; then - printf '%s\n' "${_p#;}" | tr ';' '\n' | while IFS= read -r m; do - [ -n "$m" ] && bad "$m" - done - else - ok "a branch ref follows the remote on rebuild; a tag stays put and newer ones are offered" - fi -else - skip "no lfs-phases or no git here: the branch-ref check cannot run" -fi - #@@ SUMMARY @@ -- everything below runs whichever blocks were selected echo echo " PASS: $(_pass_total) FAIL: $(_fail_total) SKIP: $(_skip_total) (final)"