#+TITLE: Guix North America #+AUTHOR: Collin J. Doering #+begin_abstract This repository contains setup and management instructions for a Guix North American Build Farm. #+end_abstract * Install Guix on debian to be used to bootstrap the Guix os installation Optionally, the below steps can be completed within tmux or screen. Tmux was installed and used in this case using the following. #+begin_src shell sudo apt update sudo apt install tmux tmux #+end_src Following the [[https://guix.gnu.org/manual/en/html_node/Binary-Installation.html][Binary Installation]] section from the Guix manual to install guix. #+begin_src shell sudo apt install -y guix #+end_src This installs the Debian's packaged version of Guix, which likely is older then what's available upstream. As such, update our installation of Guix (following the [[https://guix.gnu.org/manual/en/html_node/Upgrading-Guix.html][Updating Guix]] documentation specific to foreign distros'). #+begin_src shell sudo -i guix pull sudo systemctl restart guix-daemon.service #+end_src * Define Guix operating-system for the machine See: [[file:balg02.scm][balg02.scm]] ** Bootloader configuration For this installation, debian and its bootloader Grub will be left in place. Because we want to retain Guix's interactions with Grub (eg. to allow for restoring from failed upgrades to an earlier generation), we will have debian's Grub chainload Guix's Grub. To do so, we will need to manually adjust Debians' Grub in order to add another menu entry, and set it as the default menu item. Below is a snippet from debian's ~/etc/default/grub~. #+begin_src text GRUB_DEFAULT=0 GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0,115200n8" GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0,115200n8" GRUB_TERMINAL="console serial" GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=1 --word=8 --parity=no --stop=1" #+end_src From this we extract the necessary guix bootloader configuration options (for serial). - serial-unit :: 1 - serial-speed :: 115200 - terminal-inputs :: console serial - terminal-outputs :: console serial *** Manual modifications to Debian's Grub :PROPERTIES: :CUSTOM_ID: manual_modifications_to_debians_grub :END: Modify grub config on debian to add an additional (and default) option to chainload Guix grub. - Add a menuitem for Guix in ~/etc/grub.d/40_custom~, where ~~ is replaced with the efi partition UUID. #+begin_src text menuentry "Gnu Guix" { insmod part_gpt insmod search_fs_uuid insmod chain search --fs-uuid --no-floppy --set=root chainloader ($root)/EFI/Guix/grubx64.efi } #+end_src - Modify ~/etc/default/grub~ setting ~GRUB_DEFAULT="Gnu Guix"~ - Run ~grub-mkconfig -o /boot/grub/grub.cfg~ ** Network configuration Using the a snippet taken from ~/etc/network/interfaces~ on the existing debian installation (below), we can extract the necessary details to configure Guix's static-networking-service. - Interface :: eno8303 - Address :: 216.37.76.55/24 - Gateway :: 216.37.76.1 - DNS Name Servers :: 216.37.64.2 216.37.64.3 - DNS Search :: genenetwork.org #+begin_src text # The primary network interface allow-hotplug eno8303 iface eno8303 inet static address 216.37.76.55/24 gateway 216.37.76.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 216.37.64.2 216.37.64.3 dns-search genenetwork.org #+end_src ** Disk Partitioning :PROPERTIES: :CUSTOM_ID: disk_partitioning :END: For this installation we are using ~/dev/sda~ (a 1.5T ssd which is faster then the alternative 3.6T ssd in the server). First, we require a variety of tools to setup and partition the disk destined for Guix installation. These could be installed on debian, however an alternative approach would be to use Guix from debian as a package manager to temporarily provide the prerequisite tools. This can be done using the shell spawned from the following command. #+begin_src shell guix shell parted btrfs-progs dosfstools #+end_src *** Create disk partition table and layout #+begin_src bash parted /dev/sda mklabel gpt #+end_src *** Create partitions A simple™️ partition layout is used for this installation, consisting of an EFI ESP partition, and the remaining disk partitions for use by btrfs, where btrfs subvolumes and a swapfile will be used. #+begin_src bash parted /dev/sda mkpart primary fat32 0% 512MiB parted /dev/sda mkpart primary 512MiB 100% #+end_src *** Create EFI partition #+begin_src bash parted /dev/sda set 1 esp on mkfs.fat -F32 /dev/sda1 #+end_src *** Create btrfs 'pool' (file-system) and subvolumes **** Create btrfs file-system #+begin_src bash mkfs.btrfs --label root /dev/sda2 #+end_src **** Create btrfs subvolumes First mount the btrfs top-level file-system. #+begin_src bash mount /dev/sda2 /mnt #+end_src Then create the root subvolume, and a subvolume for swapfiles. #+begin_src bash btrfs subvolume create /mnt/@ btrfs subvolume create /mnt/@swap #+end_src Unmount the top-level btrfs file-system. #+begin_src bash umount /mnt #+end_src Mount the root subvolume. #+begin_src bash mount -o subvol=@,compress=zstd /dev/sda2 /mnt #+end_src Create nested subvolumes for ~/gnu/store~ and ~/home~. #+begin_src bash mkdir -p /mnt/gnu btrfs subvolume create /mnt/gnu/store btrfs subvolume create /mnt/home btrfs subvolume create /mnt/var #+end_src *** Create swap #+begin_src bash mkdir /mnt/swap mount -o subvol=@swap /dev/sda2 /mnt/swap chmod 600 /mnt/swap/swapfile touch /mnt/swap/swapfile chattr +C /mnt/swap/swapfile dd if=/dev/zero of=/mnt/swap/swapfile bs=1M count=32768 mkswap /mnt/swap/swapfile #+end_src *** Prepare ~/mnt~ for Guix installation Create ~/boot/efi~ directory for UEFI boot and mount the ESP partition there. #+begin_src bash mkdir -p /mnt/boot/efi mount /dev/sda1 /mnt/boot/efi #+end_src Both root and swap are already mounted and ready due to earlier steps. ** Testing To test the configuration in a vm before deployment, the following can be used. #+begin_src shell $(guix time-machine -C channels.scm -- system vm -e '(@ (guix-na config balg02) %system)') -m 2G -smp 2 -nic user,model=virtio-net-pci #+end_src ** Manual Testing of bootstrapping Guix from a Debian VM To correctly test this deployment, a environment that mimics bal02g should be used. The closest to this is a VM with debian installed, with an additional virtual disk to bootstrap guix onto. This will enable validating bootloader changes required to chainboot Guix's Grub. This testing could be automated, but was done manually as we do not expect to have to bootstrap a system like this often. *** Setup Debian VM 1. Using ~qemu~, ~libvirt~, ~virtualbox~, etc.. create a VM that boots using UEFI firmware. 1. Create an additional virtual disk that will be used to bootstrap Guix onto from Debian. This disk should be ~>20GiB~. 2. Ensure that there is a serial device attached to the VM. 2. Install Debian 12 on the VM created during step 1 (this can be a minimal server installation, no desktop, etc..). 1. It's worth noting that for some reason debian didn't setup a efi boot entry for some reason. Not sure why. To create one I used: #+begin_src shell efibootmgr --create --disk /dev/vda -p 1 -L "Debian" -l "\EFI\debian\grub64.efi" #+end_src After which I would have adjusted the boot order with: #+begin_src shell efibootmgr -o X,Y,... #+end_src However, in my case it was not needed as the boot order had debian first. 3. Reboot VM; further configure Debian. 1. Enable serial for debian grub Modify ~/etc/default/grub~, adjusting ~GRUB_TERMINAL~ and ~GRUB_CMDLINE_LINUX_DEFAULT~ as follows. #+begin_src text GRUB_TERMINAL="console serial" GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0,115200n8" #+end_src 2. Enable getty over serial #+begin_src shell systemctl enable getty@ttyS0.service systemctl start getty@ttyS0.service #+end_src *** Test Bootstrapping Gnu Guix from Debian With the Debian VM setup, we can now apply the documented bootstrapping steps. 1. [[#disk_partitioning][Disk Partitioning]], but with disks adjusted to match the testing VM. 2. [[#bootstrap_guix][Bootstrap Guix]], ensure ~~ matches the VM efi partition used for Guix. 3. [[#manual_modifications_to_debians_grub][Manual modifications to Debian's Grub]], again ensuring ~~ matches the VM efi partition used for Guix. 4. Reboot Following rebooting the VM, its expected that: - Debian Grub boots first, has "Gnu Guix" as its default selected option, which boots Guixs' Grub. - Serial access works for: - Debian and Guix Grub/s - Debian and Guix linux console As this testing is occurring in a VM, its worth noting things that are NOT expected to to be testable. - The network interfaces are not going to match what is on balg02, so its expected that the networking service will not be able to start. * Bootstrap Guix :PROPERTIES: :CUSTOM_ID: bootstrap_guix :END: Using Guix on debian, bootstrap the machine using the configuration in [[*Define Guix operating-system for the machine][Define Guix operating-system for the machine]]. ** Configure Guix Channels First, fetch the most recent channel file from the target machine. #+begin_src shell curl -O https://git.rekahsoft.ca/rekahsoft/guix-north-america/raw/branch/master/channels.scm #+end_src ** Create and Bootstrap System Create a ~bootstrap.scm~ file like below, but where ~~ is replaced with the efi partition UUID. #+begin_src scheme ((@ (guix-na config balg02) balg02) "") #+end_src Use ~guix system init ...~ to instantiate the system, but using guix time-machine to use pinned dependencies. #+begin_src shell guix time-machine -C channels.scm -- system init bootstrap.scm /mnt #+end_src ** Post Boostrapping After guix has been bootstrapped, its useful to do an initial ~guix pull~ using the same channels that were used during bootstrapping. #+begin_src shell guix pull -C /run/current-system/channels.scm #+end_src To ensure your shell refers to the correct guix after its been updated, run ~hash guix~.