352 lines
10 KiB
Org Mode
352 lines
10 KiB
Org Mode
#+TITLE: Guix North America
|
||
#+AUTHOR: Collin J. Doering
|
||
|
||
#+begin_abstract
|
||
This repository contains setup and management instructions for a Guix North American Build
|
||
Farm.
|
||
#+end_abstract
|
||
|
||
* Install Guix on debian to be used to bootstrap the Guix os installation
|
||
|
||
Optionally, the below steps can be completed within tmux or screen. Tmux was installed and
|
||
used in this case using the following.
|
||
|
||
#+begin_src shell
|
||
sudo apt update
|
||
sudo apt install tmux
|
||
tmux
|
||
#+end_src
|
||
|
||
Following the [[https://guix.gnu.org/manual/en/html_node/Binary-Installation.html][Binary Installation]] section from the Guix manual to install guix.
|
||
|
||
#+begin_src shell
|
||
sudo apt install -y guix
|
||
#+end_src
|
||
|
||
This installs the Debian's packaged version of Guix, which likely is older then what's
|
||
available upstream. As such, update our installation of Guix (following the [[https://guix.gnu.org/manual/en/html_node/Upgrading-Guix.html][Updating Guix]]
|
||
documentation specific to foreign distros').
|
||
|
||
#+begin_src shell
|
||
sudo -i guix pull
|
||
sudo systemctl restart guix-daemon.service
|
||
#+end_src
|
||
|
||
* Define Guix operating-system for the machine
|
||
|
||
See: [[file:balg02.scm][balg02.scm]]
|
||
|
||
** Bootloader configuration
|
||
|
||
For this installation, debian and its bootloader Grub will be left in place. Because we want
|
||
to retain Guix's interactions with Grub (eg. to allow for restoring from failed upgrades to
|
||
an earlier generation), we will have debian's Grub chainload Guix's Grub. To do so, we will
|
||
need to manually adjust Debians' Grub in order to add another menu entry, and set it as the
|
||
default menu item.
|
||
|
||
Below is a snippet from debian's ~/etc/default/grub~.
|
||
|
||
#+begin_src text
|
||
GRUB_DEFAULT=0
|
||
GRUB_TIMEOUT=5
|
||
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
|
||
GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0,115200n8"
|
||
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0,115200n8"
|
||
GRUB_TERMINAL="console serial"
|
||
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=1 --word=8 --parity=no --stop=1"
|
||
#+end_src
|
||
|
||
From this we extract the necessary guix bootloader configuration options (for serial).
|
||
|
||
- serial-unit :: 1
|
||
- serial-speed :: 115200
|
||
- terminal-inputs :: console serial
|
||
- terminal-outputs :: console serial
|
||
|
||
*** Manual modifications to Debian's Grub
|
||
:PROPERTIES:
|
||
:CUSTOM_ID: manual_modifications_to_debians_grub
|
||
:END:
|
||
|
||
Modify grub config on debian to add an additional (and default) option to chainload Guix
|
||
grub.
|
||
|
||
- Add a menuitem for Guix in ~/etc/grub.d/40_custom~, where ~<EFI-UUID>~ is replaced with the
|
||
efi partition UUID.
|
||
|
||
#+begin_src text
|
||
menuentry "Gnu Guix" {
|
||
insmod part_gpt
|
||
insmod search_fs_uuid
|
||
insmod chain
|
||
search --fs-uuid --no-floppy --set=root <EFI-UUID>
|
||
chainloader ($root)/EFI/Guix/grubx64.efi
|
||
}
|
||
#+end_src
|
||
|
||
- Modify ~/etc/default/grub~ setting ~GRUB_DEFAULT="Gnu Guix"~
|
||
|
||
- Run ~grub-mkconfig -o /boot/grub/grub.cfg~
|
||
|
||
** Network configuration
|
||
|
||
Using the a snippet taken from ~/etc/network/interfaces~ on the existing debian installation
|
||
(below), we can extract the necessary details to configure Guix's static-networking-service.
|
||
|
||
- Interface :: eno8303
|
||
- Address :: 216.37.76.55/24
|
||
- Gateway :: 216.37.76.1
|
||
- DNS Name Servers :: 216.37.64.2 216.37.64.3
|
||
- DNS Search :: genenetwork.org
|
||
|
||
#+begin_src text
|
||
# The primary network interface
|
||
allow-hotplug eno8303
|
||
iface eno8303 inet static
|
||
address 216.37.76.55/24
|
||
gateway 216.37.76.1
|
||
# dns-* options are implemented by the resolvconf package, if installed
|
||
dns-nameservers 216.37.64.2 216.37.64.3
|
||
dns-search genenetwork.org
|
||
#+end_src
|
||
|
||
** Disk Partitioning
|
||
:PROPERTIES:
|
||
:CUSTOM_ID: disk_partitioning
|
||
:END:
|
||
|
||
For this installation we are using ~/dev/sdb~ (a 1.5T ssd which is faster then the
|
||
alternative 3.6T ssd in the server).
|
||
|
||
First, we require a variety of tools to setup and partition the disk destined for Guix
|
||
installation. These could be installed on debian, however an alternative approach would be to
|
||
use Guix from debian as a package manager to temporarily provide the prerequisite tools. This
|
||
can be done using the shell spawned from the following command.
|
||
|
||
#+begin_src shell
|
||
guix shell parted btrfs-progs dosfstools
|
||
#+end_src
|
||
|
||
*** Create disk partition table and layout
|
||
|
||
#+begin_src bash
|
||
parted /dev/sda mklabel gpt
|
||
#+end_src
|
||
|
||
*** Create partitions
|
||
|
||
A simple™️ partition layout is used for this installation, consisting of an EFI ESP partition,
|
||
and the remaining disk partitions for use by btrfs, where btrfs subvolumes and a swapfile
|
||
will be used.
|
||
|
||
#+begin_src bash
|
||
parted /dev/sda mkpart primary fat32 0% 512MiB
|
||
parted /dev/sda mkpart primary 512MiB 100%
|
||
#+end_src
|
||
|
||
*** Create EFI partition
|
||
|
||
#+begin_src bash
|
||
parted /dev/sda set 1 esp on
|
||
mkfs.fat -F32 /dev/sda1
|
||
#+end_src
|
||
|
||
*** Create btrfs 'pool' (file-system) and subvolumes
|
||
|
||
**** Create btrfs file-system
|
||
|
||
#+begin_src bash
|
||
mkfs.btrfs --label root /dev/sda2
|
||
#+end_src
|
||
|
||
**** Create btrfs subvolumes
|
||
|
||
First mount the btrfs top-level file-system.
|
||
|
||
#+begin_src bash
|
||
mount /dev/sda2 /mnt
|
||
#+end_src
|
||
|
||
Then create the root subvolume, and a subvolume for swapfiles.
|
||
|
||
#+begin_src bash
|
||
btrfs subvolume create /mnt/@
|
||
btrfs subvolume create /mnt/@swap
|
||
#+end_src
|
||
|
||
Unmount the top-level btrfs file-system.
|
||
|
||
#+begin_src bash
|
||
umount /mnt
|
||
#+end_src
|
||
|
||
Mount the root subvolume.
|
||
|
||
#+begin_src bash
|
||
mount -o subvol=@,compress=zstd /dev/sda2 /mnt
|
||
#+end_src
|
||
|
||
Create nested subvolumes for ~/gnu/store~ and ~/home~.
|
||
|
||
#+begin_src bash
|
||
mkdir -p /mnt/gnu
|
||
|
||
btrfs subvolume create /mnt/gnu/store
|
||
btrfs subvolume create /mnt/home
|
||
btrfs subvolume create /mnt/var
|
||
#+end_src
|
||
|
||
*** Create swap
|
||
|
||
#+begin_src bash
|
||
mkdir /mnt/swap
|
||
mount -o subvol=@swap /dev/sda2 /mnt/swap
|
||
chmod 600 /mnt/swap/swapfile
|
||
touch /mnt/swap/swapfile
|
||
chattr +C /mnt/swap/swapfile
|
||
dd if=/dev/zero of=/mnt/swap/swapfile bs=1M count=32768
|
||
|
||
mkswap /mnt/swap/swapfile
|
||
#+end_src
|
||
|
||
*** Prepare ~/mnt~ for Guix installation
|
||
|
||
Create ~/boot/efi~ directory for UEFI boot and mount the ESP partition there.
|
||
|
||
#+begin_src bash
|
||
mkdir -p /mnt/boot/efi
|
||
mount /dev/sda1 /mnt/boot/efi
|
||
#+end_src
|
||
|
||
Both root and swap are already mounted and ready due to earlier steps.
|
||
|
||
** Testing
|
||
|
||
To test the configuration in a vm before deployment, the following can be used.
|
||
|
||
#+begin_src shell
|
||
$(guix time-machine -C channels.scm -- system vm -e '(@ (guix-na config balg02) %system)') -m 2G -smp 2 -nic user,model=virtio-net-pci
|
||
#+end_src
|
||
|
||
** Manual Testing of bootstrapping Guix from a Debian VM
|
||
|
||
To correctly test this deployment, a environment that mimics bal02g should be used. The
|
||
closest to this is a VM with debian installed, with an additional virtual disk to bootstrap
|
||
guix onto. This will enable validating bootloader changes required to chainboot Guix's Grub.
|
||
|
||
This testing could be automated, but was done manually as we do not expect to have to
|
||
bootstrap a system like this often.
|
||
|
||
*** Setup Debian VM
|
||
|
||
1. Using ~qemu~, ~libvirt~, ~virtualbox~, etc.. create a VM that boots using UEFI firmware.
|
||
|
||
1. Create an additional virtual disk that will be used to bootstrap Guix onto from Debian.
|
||
This disk should be ~>20GiB~.
|
||
|
||
2. Ensure that there is a serial device attached to the VM.
|
||
|
||
2. Install Debian 12 on the VM created during step 1 (this can be a minimal server
|
||
installation, no desktop, etc..).
|
||
|
||
1. It's worth noting that for some reason debian didn't setup a efi boot
|
||
entry for some reason. Not sure why. To create one I used:
|
||
|
||
#+begin_src shell
|
||
efibootmgr --create --disk /dev/vda -p 1 -L "Debian" -l "\EFI\debian\grub64.efi"
|
||
#+end_src
|
||
|
||
After which I would have adjusted the boot order with:
|
||
|
||
#+begin_src shell
|
||
efibootmgr -o X,Y,...
|
||
#+end_src
|
||
|
||
However, in my case it was not needed as the boot order had debian first.
|
||
|
||
3. Reboot VM; further configure Debian.
|
||
|
||
1. Enable serial for debian grub
|
||
|
||
Modify ~/etc/default/grub~, adjusting ~GRUB_TERMINAL~ and ~GRUB_CMDLINE_LINUX_DEFAULT~ as
|
||
follows.
|
||
|
||
#+begin_src text
|
||
GRUB_TERMINAL="console serial"
|
||
GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0,115200n8"
|
||
#+end_src
|
||
|
||
2. Enable getty over serial
|
||
|
||
#+begin_src shell
|
||
systemctl enable getty@ttyS0.service
|
||
systemctl start getty@ttyS0.service
|
||
#+end_src
|
||
|
||
*** Test Bootstrapping Gnu Guix from Debian
|
||
|
||
With the Debian VM setup, we can now apply the documented bootstrapping steps.
|
||
|
||
1. [[#disk_partitioning][Disk Partitioning]], but with disks adjusted to match the testing VM.
|
||
2. [[#bootstrap_guix][Bootstrap Guix]], ensure ~<EFI-UUID>~ matches the VM efi partition used for Guix.
|
||
3. [[#manual_modifications_to_debians_grub][Manual modifications to Debian's Grub]], again ensuring ~<EFI-UUID>~ matches the VM efi
|
||
partition used for Guix.
|
||
4. Reboot
|
||
|
||
Following rebooting the VM, its expected that:
|
||
|
||
- Debian Grub boots first, has "Gnu Guix" as its default selected option, which boots Guixs'
|
||
Grub.
|
||
- Serial access works for:
|
||
- Debian and Guix Grub/s
|
||
- Debian and Guix linux console
|
||
|
||
As this testing is occurring in a VM, its worth noting things that are NOT expected to to be
|
||
testable.
|
||
|
||
- The network interfaces are not going to match what is on balg02, so its expected that the
|
||
networking service will not be able to start.
|
||
|
||
* Bootstrap Guix
|
||
:PROPERTIES:
|
||
:CUSTOM_ID: bootstrap_guix
|
||
:END:
|
||
|
||
Using Guix on debian, bootstrap the machine using the configuration in [[*Define Guix operating-system for the machine][Define Guix
|
||
operating-system for the machine]].
|
||
|
||
** Configure Guix Channels
|
||
|
||
First, fetch the most recent channel file from the target machine.
|
||
|
||
#+begin_src shell
|
||
curl -O https://git.rekahsoft.ca/rekahsoft/guix-north-america/raw/branch/master/channels.scm
|
||
#+end_src
|
||
|
||
** Create and Bootstrap System
|
||
|
||
Create a ~bootstrap.scm~ file like below, but where ~<EFI-UUID>~ is replaced with the efi
|
||
partition UUID.
|
||
|
||
#+begin_src scheme
|
||
((@ (guix-na config balg02) balg02) "<EFI-UUID>")
|
||
#+end_src
|
||
|
||
Use ~guix system init ...~ to instantiate the system, but using guix time-machine to use
|
||
pinned dependencies.
|
||
|
||
#+begin_src shell
|
||
guix time-machine -C channels.scm -- system init bootstrap.scm /mnt
|
||
#+end_src
|
||
|
||
** Post Boostrapping
|
||
|
||
After guix has been bootstrapped, its useful to do an initial ~guix pull~ using the same
|
||
channels that were used during bootstrapping.
|
||
|
||
#+begin_src shell
|
||
guix pull -C /run/current-system/channels.scm
|
||
#+end_src
|
||
|
||
To ensure your shell refers to the correct guix after its been updated, run ~hash guix~.
|