guix-machines/README.org

#+TITLE: Guix Machines
#+AUTHOR: Collin J. Doering

#+BEGIN_EXPORT html
<p><a href="https://guix-ci.home.rekahsoft.ca/jobset/guix-machines"><img src="https://guix-ci.home.rekahsoft.ca/jobset/guix-machines/badge.svg?type=0" alt="Cuirass Status"></a></p>
#+END_EXPORT

#+begin_abstract
Guix configurations for all Guix powered servers privately managed by the author. This
includes all virtual machines for my home network and cloud/vps instances. Two mutable
deployment methodologies are supported:

1. A push based model, using ~guix deploy~[fn:1] to remotely deploy changes (useful for example
   from ci/cd).
2. A pull based model, using ~guix~ directly from the target, along with either the entire
   repository, or its channel file.

Immutable deployment is not yet supported, but is certainly possible given Guix's ability to
build an ~operating-system~ configuration into a image.
#+end_abstract

* Repository Structure

- ~channels*.scm~        :: [[*Guix Channel Files][Guix channel files]]
- ~deploy/~              :: Folder containing all ~guix deploy~ configurations
- ~.gitignore~           :: Files ignored by git
- ~.guix/~               :: Guix channel directory
- ~.guix-authorizations~ :: Guix authorizations file[fn:2]
- ~.guix-channel~        :: Guix channel file[fn:3]
- ~.pub-keys/~           :: Folder containing public key files used by Guix configurations
- ~README.org~           :: Org-mode[fn:4] documentation
- ~TODO.org~             :: Org-mode todo's, known issues and future aspirations
- ~unguix/~              :: Docker/docker-compose files used on deployed instances, managed
  outside of guix. Once better support for running docker/docker-compose via shepherd,
  specified declaratively via Guix configuration has been implemented, this directory and all
  files within it should be able to be removed.

** User Supplied Files Required for Push Based Deployment

- ~.deploy-key~ :: Private ssh key used to deploy changes to the target.

** Guix Channel Files

Guix channels[fn:5] allow for Guix to be customized and extended. They are also critical for
replicating a Guix system[fn:6]. As mentioned above, there are two primary classes of
deployments that are managed using this repository, push based and pull based. In both cases,
what specific versions of software that will be installed during deployment depends on the
guix channels in use. To ensure reproducibility, a ~channels.scm~ file is provided in this
repository that is expected to be used during deployment. It pins external guix channels to
specific versions.

If for some reason channels need to be pinned for a specific deployment, a new channel file
named ~channels-<hostname>.scm~ can be created and used in place of normally used channel
file.

*** TODO Updating guix channels used for deployment

**This doesn't work right unless your channels match what is expected by this repository.**

#+begin_src shell
  guix time-machine -- describe -f channels > channels.scm
#+end_src

** ~guix-machines~ the Guix Channel

This repository is itself a Guix channel, which allows operating-system configurations to
come directly from the channel, and the version of this configuration be managed just like
any other guix channel. It also facilitates CI, allowing for changes this channel be
evaluated by Cuirass at [[https://guix-ci.home.rekahsoft.ca]][fn:7]. This channel does not define
any packages, only system configurations and machine specifications for deployment.

At a later date, this also will allow for building of machine images for immutable
deployment, bootstrapping and more.

* TODO Initializing a System

** Using an existing guix installation image

*** TODO Producing an installation image

TODO: It would be convent to be able to produce an image that can be used to install my
normal setup. This would avoid pulling my channels, as well as downloading software (it could
all be pre-packaged in the image).

If you produce and use this installation image, you can skip to [[*Disk Setup and Partitioning][Disk Setup and Partitioning]].

*** TODO Setup Installation

TODO: network needs to be setup; local, keymap, and font too.

*** Configure Guix Channels

First, fetch the most recent channel file from the target machine. For most, this will be
done via the internet using my public mirror.

#+begin_src shell
  curl -O https://git.rekahsoft.ca/rekahsoft/guix-machines/raw/branch/master/channels.scm
#+end_src

When on my network, I pull from my internal git (notice the different URL).

#+begin_src shell
  curl -O https://git.home.rekahsoft.ca/rekahsoft-public/guix-machines/raw/branch/master/channels.scm
#+end_src

Once the channel file is available on the target, refresh the cached command paths with
~hash~ and update guix to use the new channels.

#+begin_src shell
  hash guix
  sudo -i guix pull -C $(realpath channels.scm)
#+end_src

*** TODO Disk Setup and Partitioning

TODO: disks need to be partitioned and appropriately mounted. This varies depending on the setup.

#+begin_src bash
  # Create disk partition table and layout
  parted /dev/nvme0n1 mklabel gpt

  # Create partitions
  parted /dev/nvme0n1 mkpart primary ESP 0% 512MiB
  parted /dev/nvme0n1 mkpart 512MiB 100%

  # Create EFI partition
  parted /dev/nvme0n1p1 set 1 esp on
  mkfs.fat -F32 /dev/nvme0n1p1

  # Create LUKS container on remainder of disk
  cryptsetup luksFormat --label crypt /dev/nvme0n1p2

  # Unlock LUKS container after creation
  cryptsetup luksOpen /dev/nvme0n1p2 crypt

  #
  # Create LVM2 container inside of LUKS container

  # Create Physical Volume (pv)
  pvcreate /dev/mapper/crypt

  # Create Volume Group (vg)
  vgcreate vg0 /dev/mapper/crypt

  # Create Logical Volume/s (vg)
  lvcreate -L <ROOT_VOL_SIZE>G vg0 -n root
  lvcreate -l +100%FREE vg0 -n swap

  #
  # Create btrfs 'pool' (file-system) and subvolumes

  # Create btrfs file-system
  mkfs.btrfs --label root /dev/vg0/root

  # Create btrfs subvolumes
  mount /dev/vg0/root /mnt
  btrfs subvolume create /mnt/@
  umount /mnt

  # Mount the root subvolume.
  mount -o subvol=@,compress=zstd /dev/vg0/root /mnt

  # Create nested subvolumes for /gnu/store, /home, and /var
  mkdir -p /mnt/gnu
  btrfs subvolume create /mnt/gnu/store
  btrfs subvolume create /mnt/home
  btrfs subvolume create /mnt/var

  # Create and activate swap
  mkswap --label swap /dev/vg0/swap
  swapon /dev/vg0/swap

  # Prepare /mnt for Guix installation
  mkdir -p /mnt/boot/efi
  mount /dev/nvme0n1p1 /mnt/boot/efi
#+end_src

*** Bootstrap System

**** Start ~cow-store~ Service

As described in guix documentation, start ~cow-store~ to allow later steps to write store
changes to ~/mnt/gnu/store~.

#+begin_src bash
  herd start cow-store /mnt
#+end_src

**** Optional (but recommended): Use additional or alternative substitute servers

Optionally additional or alternative substitute servers can be setup for guix-daemon. This is
recommended to greatly speed up installation (so that nonguix and rekahsoft-guix (internal
network only) substitutes can be used).

First stop the running ~guix-daemon~.

#+begin_src bash
  herd stop guix-daemon
#+end_src

Then start a new instance of ~guix-daemon~ as it is normally started by its Shepherd service,
but with additional or alternative substitute servers provided.

#+begin_src bash
  guix-daemon \
    --build-users-group guixbuild \
    --max-silent-time 0 \
    --timeout 0 \
    --log-compression gzip \
    --discover=no \
    --substitute-urls https://substitutes.nonguix.org https://guix-ci.home.rekahsoft.ca https://ci.guix.gnu.org https://bordeaux.guix.gnu.org &
#+end_src

**** Create and Bootstrap System

Create a ~bootstrap.scm~ file like this:

#+begin_src scheme
  (@ (rekahsoft guix-config vms <target>) %system)
#+end_src

Use ~guix system init ...~ to instantiate the system.

#+begin_src bash
  guix system init bootstrap.scm /mnt
#+end_src

** TODO Producing an image to be flashed directly

TODO: there are limitations on what images I can produce; namely, lvm and luks cannot be
setup easily (or at all?) it seems?

#+begin_src shell
  guix time-machine -C channels.scm -- system image -e '(@ (rekahsoft guix-config vms <target>) %image)'
#+end_src

* Push Deployment with ~guix deploy~

Push based mutable deployment is the default deployment methodology for the majority of
systems managed by this repository. This is particularity safe because Guix changes are done
as transactions, and thus can easily be rolled back.

To deploy a system use the following (substituting ~<hostname>~ with the appropriate deploy
file).

#+begin_src shell
  guix time-machine -C channels.scm -- deploy deploy/<hostname>.scm
#+end_src

**Note:** Deploy files in [[./deploy]] are named after the hostname that would be used to ssh to
the machine.

* Pull Based Deployment

Pull based mutable deployment is the default deployment methodology for personal computers,
where using a push based method doesn't make sense. It also serves as a secondary deployment
mechanism for systems normally maintained using the push deployment model; for example, this
becomes necessary when facing ~guix deploy~ bugs.

First [[*Configure Guix Channels][Configure Guix Channels]] as described above. Once channels have been updated
successfully, use the following to reconfigure the system.

#+begin_src shell
  sudo -i guix system reconfigure -e '(@ (rekahsoft guix-config vms <target>) %system)'
#+end_src

Alternatively, the same effect can be achieved without first pulling the appropriate channels
by instead using ~guix time-machine~ as follows.

#+begin_src shell
  sudo -i guix time-machine -C $(realpath channels.scm) -- system reconfigure -e '(@ (rekahsoft guix-config vms <target>) %system)'
#+end_src

* Using Local Sources

Regardless of the deployment methodology used, sometimes it is useful to deploy changes that
have not yet been committed. This should be done sparingly, as it can be slightly confusing
if forgotten; that being said, Guix makes this a semi-reasonable thing to do, as how the
system changes is tracked very explicitly by guix generations local to the target.

To manually deploy using local sources, the local sources must exist on the working machine
(of course). The easiest way to do this is via git, from the working machine like so. Most
will pull from my public git mirror.

#+begin_src shell
  git clone https://git.rekahsoft.ca/rekahsoft/guix-machines.git
#+end_src

On my network, internal git is used instead (notice the different URL).

#+begin_src shell
  git clone https://git.home.rekahsoft.ca/rekahsoft-public/guix-machines.git
#+end_src

Once a copy of the sources are available on the working machine, all that remains is
following the normal deployment steps, but with a slight modification; use the
~-L|--load-path~ argument to specify the current working sources, effectively overriding what
is in the ~guix-machines~ channel.

** Building Operating System Configuration

Many times its useful to verify that a change to a system configuration will build correctly,
prior to deploying it. ~guix system build~ is meant for exactly this, and can be run as
follows.

#+begin_src shell
  guix time-machine -C channels.scm -- system build -L .guix -e '(@ (rekahsoft guix-config vms <target>) %system)'
#+end_src

** Running Operating System in VM for testing

#+begin_src shell
  $(guix time-machine -C channels.scm -- system vm -e '(@ (rekahsoft guix-config vms <target>) %system)') -m 2G -smp 2 -nic user,model=virtio-net-pci
#+end_src

** Generating the OS service extension graph

#+begin_src shell
  guix time-machine -C channels.scm -- system extension-graph -e '(@ (rekahsoft guix-config vms <target>) %system)' | guix shell xdot -- xdot -
#+end_src

** Generating the shepherd service graph

#+begin_src shell
  guix time-machine -C channels.scm -- system shepherd-graph -e '(@ (rekahsoft guix-config vms <target>) %system)' | guix shell xdot -- xdot -
#+end_src

** Push Based Deployments

#+begin_src shell
  guix time-machine -C channels.scm -- deploy -L ./.guix deploy/<hostname>.scm
#+end_src

See the [[*Push Deployment with ~guix deploy~][Push Deployment with ~guix deploy~]] section for more details.

** Pull Based Deployments

#+begin_src shell
  sudo -i guix time-machine -C $(realpath channels.scm) -- system reconfigure -L $(realpath ./.guix) -e '(@ (rekahsoft guix-config vms <target>) %system)'
#+end_src

See the [[*Pull Based Deployment][Pull Based Deployment]] section for more details.

* Footnotes

[fn:1] https://guix.gnu.org/manual/en/html_node/Invoking-guix-deploy.html

[fn:2] https://guix.gnu.org/manual/en/html_node/Channel-Authentication.html

[fn:3] https://guix.gnu.org/manual/en/html_node/Package-Modules-in-a-Sub_002ddirectory.html

[fn:4] https://orgmode.org/

[fn:5] https://guix.gnu.org/manual/en/html_node/Channels.html

[fn:6] https://guix.gnu.org/manual/en/html_node/Replicating-Guix.html

[fn:7] Only available in my internal home-network