|
|
|
|
@ -0,0 +1,142 @@
|
|
|
|
|
#+TITLE: Guix Builder
|
|
|
|
|
#+AUTHOR: Collin J. Doering
|
|
|
|
|
|
|
|
|
|
#+begin_abstract
|
|
|
|
|
This repository defines a minimal OCI container containing guix and a few other tools useful
|
|
|
|
|
for CI/CD jobs. The container is expected to be run from a system that already has the
|
|
|
|
|
~guix-daemon~ installed and running, and is most useful in the context of existing CI/CD
|
|
|
|
|
tools that already use an existing OCI container runtime (eg. docker, podman, etc..).
|
|
|
|
|
#+end_abstract
|
|
|
|
|
|
|
|
|
|
* Prerequisites
|
|
|
|
|
|
|
|
|
|
- ~guix-daemon~ installed and running on the host
|
|
|
|
|
- ~dockerd~ to build and run the produced image
|
|
|
|
|
- ~direnv~ (optional)
|
|
|
|
|
|
|
|
|
|
* Repository Structure
|
|
|
|
|
|
|
|
|
|
- ~channels.scm~ :: [[*Guix Channel File][Guix Channel File]].
|
|
|
|
|
- ~Dockerfile~ :: From scratch image that uses guix pack tar.gz output.
|
|
|
|
|
- ~Makefile~ :: Various make targets to build and run the guix-builder OCI image.
|
|
|
|
|
- ~manifest-dev.scm~ :: Packages required for development.
|
|
|
|
|
- ~manifest.scm~ :: Packages to be installed in the OCI image.
|
|
|
|
|
- ~.gitignore~ :: Files ignored by git.
|
|
|
|
|
- ~README.org~ :: Org-mode[fn:1] documentation.
|
|
|
|
|
- ~shell.sh~ :: Launch a containerized development shell.
|
|
|
|
|
|
|
|
|
|
** Guix Channel File
|
|
|
|
|
|
|
|
|
|
Guix channels[fn:2] allow for Guix to be customized and extended. They are also critical for
|
|
|
|
|
replicating a Guix system[fn:3]. To ensure reproducibility, a ~channels.scm~ file is provided
|
|
|
|
|
in this repository that is expected to be used during deployment. It pins external guix
|
|
|
|
|
channels to specific versions.
|
|
|
|
|
|
|
|
|
|
* Setup development environment
|
|
|
|
|
|
|
|
|
|
~direnv~ is the preferred and optimal way to enter a development environment. Use ~direnv allow~.
|
|
|
|
|
|
|
|
|
|
Alternatively, ~./shell.sh~ can be invoked directly
|
|
|
|
|
|
|
|
|
|
* Building the image
|
|
|
|
|
|
|
|
|
|
#+begin_src shell
|
|
|
|
|
make [build]
|
|
|
|
|
#+end_src
|
|
|
|
|
|
|
|
|
|
~build~ is optional, as its the default target.
|
|
|
|
|
|
|
|
|
|
* Using the image
|
|
|
|
|
|
|
|
|
|
Now that the OCI image has been produced, some care must be taken when running it. Namely,
|
|
|
|
|
the following volumes are expected to be mounted.
|
|
|
|
|
|
|
|
|
|
- ~/var/guix/daemon-socket/socket~ :: the ~guix-daemon~ socket
|
|
|
|
|
- ~/gnu/store~ :: The hosts guix store
|
|
|
|
|
- ~/etc/ssl~ :: The hosts ssl certificates (required for ~guix time-machine ...~ commands)
|
|
|
|
|
|
|
|
|
|
Additionally, for ~guix~ to function appropriately, the ~HOME~ environment variable must be
|
|
|
|
|
set. This allows guix to store its cache (mandatory for many operations, eg ~guix pull~).
|
|
|
|
|
Depending on use-case, it is sometimes useful to retain the guix cache between container
|
|
|
|
|
invocations. To do so, mount a volume to ~$HOME/.cache/guix~ (replacing ~$HOME~ with what it
|
|
|
|
|
will be set to within the container).
|
|
|
|
|
|
|
|
|
|
A complete example is given below, where the aforementioned volumes are mounted, ~HOME~ is
|
|
|
|
|
set to ~/tmp~, and a volume for retaining the guix cache between invocations is put in place.
|
|
|
|
|
The working directory within the container is also set with the ~-w~ option, however this is
|
|
|
|
|
not required.
|
|
|
|
|
|
|
|
|
|
#+begin_src shell
|
|
|
|
|
docker run --rm -it \
|
|
|
|
|
-v /var/guix/daemon-socket/socket:/var/guix/daemon-socket/socket \
|
|
|
|
|
-v /gnu/store:/gnu/store:ro \
|
|
|
|
|
-v /etc/ssl:/etc/ssl:ro \
|
|
|
|
|
-v /var/lib/ci/guix-cache:/tmp/.cache/guix \
|
|
|
|
|
-e HOME=/tmp \
|
|
|
|
|
-w /tmp \
|
|
|
|
|
guix-builder:latest
|
|
|
|
|
#+end_src
|
|
|
|
|
|
|
|
|
|
** Using the image with DroneCI / WoodpeckerCI
|
|
|
|
|
|
|
|
|
|
The primary use-case for the image produced by this repository is for ci/cd jobs (mandatorily
|
|
|
|
|
on a host that is running guix-daemon itself). This allows for the hosts guix store to be
|
|
|
|
|
reused, allowing for a node-local caching layer, and enabling optimal ci/cd build/deploy
|
|
|
|
|
times.
|
|
|
|
|
|
|
|
|
|
Any ci/cd tool that can run OCI containers should be compatible with this image, though the
|
|
|
|
|
optimal setup assumes it provides a mechanism to mount volumes and set environment variables
|
|
|
|
|
(by an administrator). Both DroneCI and WoodpeckerCI allow for this, using
|
|
|
|
|
~DRONE_RUNNER_VOLUMES~ and ~WOODPECKER_BACKEND_DOCKER_VOLUMES~ respectively.
|
|
|
|
|
|
|
|
|
|
Note, in WoodpeckerCI, this functionality has not yet made it into a release (see [[https://github.com/woodpecker-ci/woodpecker/pull/1203][PR]]).
|
|
|
|
|
|
|
|
|
|
Below is an example of running ~drone-runner-docker~, setup to spawn ~guix~ images produced
|
|
|
|
|
by this repository:
|
|
|
|
|
|
|
|
|
|
#+begin_src shell
|
|
|
|
|
docker run -d \
|
|
|
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
|
|
|
-e DRONE_RPC_PROTO="https" \
|
|
|
|
|
-e DRONE_RPC_HOST="<REDACTED>" \
|
|
|
|
|
-e DRONE_RPC_SECRET="<REDACTED>" \
|
|
|
|
|
-e DRONE_RUNNER_CAPACITY=4 \
|
|
|
|
|
-e DRONE_RUNNER_NAME="<REDACTED>" \
|
|
|
|
|
-e DRONE_RUNNER_LABELS=guix:on \
|
|
|
|
|
-e DRONE_RUNNER_VOLUMES=/var/guix/daemon-socket/socket:/var/guix/daemon-socket/socket,/gnu/store:/gnu/store,/var/lib/ci/guix-cache:/.cache/guix \
|
|
|
|
|
-p 3001:3000 \
|
|
|
|
|
--restart unless-stopped \
|
|
|
|
|
--name runner-guix \
|
|
|
|
|
drone/drone-runner-docker:1.6.3
|
|
|
|
|
#+end_src
|
|
|
|
|
|
|
|
|
|
* FAQ
|
|
|
|
|
|
|
|
|
|
** Why not produce this image with ~guix pack -f docker ...~ directly?
|
|
|
|
|
|
|
|
|
|
One may ask, why not simply produce a OCI container using ~guix pack~ directly? Eg.
|
|
|
|
|
|
|
|
|
|
#+begin_src shell
|
|
|
|
|
guix pack -f docker -S /bin=bin --entry-point=bin/guix -m manifest.scm
|
|
|
|
|
#+end_src
|
|
|
|
|
|
|
|
|
|
Well, I'm glad you asked! This produces a container that has the guix store in its default
|
|
|
|
|
location ~/gnu/store~. Using the same method as described in [[*Using the image with DroneCI / WoodpeckerCI][Using the image with DroneCI /
|
|
|
|
|
WoodpeckerCI]], this container image works, however it has an implicit constraint: all store
|
|
|
|
|
items used within the pack must also exist in the hosts guix store. This is because the guix
|
|
|
|
|
store is volume mounted into the container at runtime, shadowing the already existing
|
|
|
|
|
~/gnu/store~ directory put in place via ~guix pack~. There are ways one could work around
|
|
|
|
|
this constraint, however the best solution would be to have a guix container that is
|
|
|
|
|
independent of the host (as described in this repository).
|
|
|
|
|
|
|
|
|
|
** What about running a ~guix-daemon~ within a container (so that builds can be completely isolated)?
|
|
|
|
|
|
|
|
|
|
This currently is not possible ... TBD (more detail)
|
|
|
|
|
|
|
|
|
|
* Footnotes
|
|
|
|
|
|
|
|
|
|
[fn:1] https://orgmode.org/
|
|
|
|
|
|
|
|
|
|
[fn:2] https://guix.gnu.org/manual/en/html_node/Channels.html
|
|
|
|
|
|
|
|
|
|
[fn:3] https://guix.gnu.org/manual/en/html_node/Replicating-Guix.html
|
