rekahsoft
/
guix-machines
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
guix-machines/TODO.org

65 KiB
Raw Blame History

guix-machines TODO's

Guix Machines

ARCHIVE   ARCHIVE

DEADLINE: <1990-05-08 Tue>

CANCELED Use 'guix deploy'/lvm based common proxmox-vm configuration for all vms

CLOSED: [2023-03-11 Sat 21:58] DEADLINE: <2023-03-17 Fri>

  • Note taken on [2023-03-12 Sun 13:40]
    Noticed that I actually setup cloud0 for guix deploy-ment. This is likely due to it being used during testing of guix deploy but it was never updated to use my guix channel setup. Doing so is captured as part of Setup cloud0-home-rekahsoft-ca for guix deploy.
  • State "CANCELED" from "DOING" [2023-03-11 Sat 21:58]
    This is superseded by Rotate ssh-host keys
  • Note taken on [2023-03-11 Sat 21:55]
    This was started a long time ago.
  • State "DOING" from "TODO" [2023-03-11 Sat 21:55]

Checked indicates the vm is using a 'guix deploy' style deployment, and is based on the common lvm based proxmox configuration prmox-vm-lvm-minimal.scm.

DONE Rotate ssh-host keys [100%]   home_network

CLOSED: [2023-03-12 Sun 21:15] DEADLINE: <2023-03-11 Sat>

  • State "DONE" from "DOING" [2023-03-12 Sun 21:15]
  • State "DOING" from "TODO" [2022-07-02 Sat 23:14]

I was reminded that I need to rotate host-keys as the guix-template-vm I use used to have it embedded (afair it does not now - [2022-07-01 Fri]).

  ➜ ssh prom0.home.rekahsoft.ca
  The authenticity of host 'prom0.home.rekahsoft.ca (172.16.0.42)' can't be established.
  ED25519 key fingerprint is SHA256:uglOBa3IN3F4wtfjpHSdwM/AdUA1Q8EEGwGjuIMVg0o.
  This host key is known by the following other names/addresses:
      ~/.ssh/known_hosts:22: postgres.home.rekahsoft.ca
      ~/.ssh/known_hosts:29: searx0.home.rekahsoft.ca
      ~/.ssh/known_hosts:36: my0.home.rekahsoft.ca
  Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
  Warning: Permanently added 'prom0.home.rekahsoft.ca' (ED25519) to the list of known hosts.
  Last login: Mon Nov 29 09:59:22 2021 from 172.16.3.12
  • postgres.home.rekahsoft.ca
  • searx0.home.rekahsoft.ca
  • my0.home.rekahsoft.ca
  • dash0.home.rekahsoft.ca
  • prom0.home.rekahsoft.ca
  • ci-runner0.home.rekahsoft.ca
  • ci0.home.rekahsoft.ca
  • git0.home.rekahsoft.ca
  • loki.home.rekahsoft.ca
Notes

As part of doing this, I am going to correctly capture each of the above vms in my guix-machines repository. I will document the process here, however do not expect to have to do it again after completing the procedure for the above vms.

To start, I have to get the most recent configuration defined in the guix-machines repository over to the target machine. I use git archive to make quick work of this.

  git archive --format tar @ | gzip - > "guix-machines-$(git rev-parse --short @).tar.gz"

Then it needs to be copied to the target machine.

  scp guix-machines*.tar.gz <host>:~/

Now that the configuration has been copied to the target machine, it needs to be used to update the system. This will then enable future use of guix deploy (as it adds the necessary bits (eg. the automation user, authorize substitute key)).

On the target machine, extract the tarball to a folder and change directory to it.

  mkdir guix-machines
  tar -xvf guix-machines*.tar.gz -C guix-machines
  cd guix-machines

It seems that for some reason, the guix channel needs to be pulled on its own before pulling this folders channels.scm file (I found a subtle reference to this in old guix irc logs). Until the system is reconfigured with the new configuration, substitute-urls are explicitly set to the upstream mirror (to avoid pulling from my broken nexus guix mirror).

  sudo -i guix pull --substitute-urls="https://ci.guix.gnu.org"

Now that we have successfully pulled the latest guix, update root's guix channels to use the channels.scm file (which will guaranteed be a downgrade).

  sudo -i guix pull -C $(realpath channels.scm) --allow-downgrades --substitute-urls="https://ci.guix.gnu.org"

And then, complete a system update.

  sudo -i guix system reconfigure -e '(@ (rekahsoft guix-config vms <host>-home-rekahsoft-ca) %system)' --substitute-urls="https://ci.guix.gnu.org"

It's possible that this could be done in one step using guix time-machine ... however I have had good success with the above method so far.

Replacing ssh host keys

To replace the ssh host keys (which are 'data' and must be managed manually or by some means outside of guix), we just need to remove them and reboot. I'm not exactly sure what regenerates the host keys upon reboot (its not the ssh-daemon service as I tried that). Either way, this should not be a common occurrence. As root:

  mkdir /etc/ssh/host-keys-backup
  mv /etc/ssh/ssh_host* /etc/ssh/host-keys-backup/
  reboot
Cleanup after successful reboot 
  sudo rm -r /etc/ssh/host-keys-backup

Removed unneeded local configuration.

  sudo rm /etc/config.scm{,~}
Clean up old generations and the gnu store 
  sudo -i guix pull -d 1m
  sudo -i guix gc
Problem while attempting to update searx0

On 'legacy' vms like searx0, my nexus mirror of guix is used. However, I have had issues pulling through my nexus mirror before, and it can be slow. Thus, it may make sense to add --substitute-urls="https://ci.guix.gnu.org" to force using the official mirror.

I ran into issues on old versions of guix, with random failures. I have seen this before, and it can usually be overcome by either running it again, or using guix time-machine.

  sudo -i guix time-machine -- pull $(realpath channels.scm)

It seems I spoke prematurely. The above continues to fail, this time with a different message:

  gunzip -c /var/log/guix/drvs/h7/car69rww65ba18sjh7pabxn8a3xq3h-guix-package-cache.drv.gz
(repl-version 0 1 1)
(exception %exception (non-self-quoting 140737337240624 "#<&message message: \"unsupported manifest format\">"))

Because I never actually checked if the error I was receiving upon pull was recurring or just a random failure, I retried the original command (sans time-machine), and found it failed with the same issue. It said something like this:

  &openpgp-unrecognized-packet-error: #<input: string 7f528ee2d770>

So, what this means is updating to the channels.scm in guix-machines, commit c7f67ad169 resulted in an odd error. Here is current active guix system generation on searx0.

  Generation 9    Oct 17 2020 14:37:32     (current)
    file name: /var/guix/profiles/system-9-link
    canonical file name: /gnu/store/sxz0ay6ryvwvl21v2cq02s4bhd4gxx71-system
    label: GNU with Linux-Libre 5.8.14
    bootloader: grub
    root device: UUID: f354b908-fd6f-46c9-931b-2bebaac61b56
    kernel: /gnu/store/zhbad09wf7waxbjr5vr8ah203svli8mh-linux-libre-5.8.14/bzImage
    channels:
      rekahsoft-guix:
        repository URL: https://git.rekahsoft.ca/rekahsoft/rekahsoft-guix.git
        branch: master
        commit: 7d16b31094446f10d539fcfab8336bdfd38386c8
      guix:
        repository URL: https://git.savannah.gnu.org/git/guix.git
        branch: master
        commit: bf986c3e4325594c1c6a43cf0b67b92262842863
    configuration file: /gnu/store/sqk34984d52zqcp2915rglxkzw59914g-configuration.scm

Because I'm a little at a loss, I decided I will update guix in place, using the existing channels (just default) as well as the configuration in /etc/config.scm. This ended up working, below is the new guix system generation.

  Generation 10   Jul 03 2022 02:33:14    (current)
    file name: /var/guix/profiles/system-10-link
    canonical file name: /gnu/store/7s048bjn23ap0fzc4fb8g15q6q9rhz6l-system
    label: GNU with Linux-Libre 5.18.8
    bootloader: grub
    root device: UUID: f354b908-fd6f-46c9-931b-2bebaac61b56
    kernel: /gnu/store/rs91vnfwf8bjddnjj1q46cw0835agfq2-linux-libre-5.18.8/bzImage
    channels:
      guix:
        repository URL: https://git.savannah.gnu.org/git/guix.git
        branch: master
        commit: cec5a522e190447e64a0a0f193bccc67b7095c31
    configuration file: /gnu/store/sqk34984d52zqcp2915rglxkzw59914g-configuration.scm

Now that I am on a more recent guix, my user's guix is older, which results in errors when calling guix system list-generation. To resolve this, I do a guix pull as my user (though this isn't strictly necessary, it makes sense).

I try pulling channels.scm again, and am met with the same result as earlier. I suspect this could have to do with older profiles (a manifest stored there that is no longer supported?). Following this hunch, I deleted all profiles but the last 2 (generations 9 and 10) and try pulling the channels again. This time, I receive a different error.

  gunzip -c /var/log/guix/drvs/wk/izazsd4va22bwjqjsqddfcy85anrsy-nonguix.drv.gz
(repl-version 0 1 1)
(exception unbound-variable (value #f) (value "Unbound variable: ~S") (value (linux-libre-5.17)) (value #f))

This error is because guix-machines channels.scm (@ c7f67ad1695b13a57629da32e3905e45e665c032) is using version 1a122e06fe046caebf39395edc797515861acd3b of the nonguix channel, prior to the appropriate change being made in nonguix 12da9f0562e4518d297deb821added3f29488565. I update the nonguix channel to its latest commit in channels.scm, which includes the needed fix and retry the pull. And that did the trick! guix pull of the new channels has now succeeded.

One thing that is annoying about this is that the issue was not caught at guix-ci.home.rekahsoft.ca. I am not sure why this is the case and will need to further investigate.

Now that I've been able to pull the channels specified in the guix-machine repository, I can now complete a system update.

  sudo -i guix system reconfigure -e '(@ (rekahsoft guix-config vms searx0-home-rekahsoft-ca) %system)'

This works as expected, and now the system is up-to-date, and uses its original configuration, with the adjustment that its no longer stored on the server at /etc/config.scm, but instead was refactored into a module that lives in the guix-machines repository.

At this point I can do pull based deployment for the server I'm testing with (searx0), however guix deploy from another system (push based deployment) does not yet work, as there are some differences that are not captured in the legacy servers original configuration. I correct this, and re-pull/reconfigure searx0.

At this point, I expected guix deploy to function correctly when given a legacy vm configuration (like searx0), however this is not the case.

  ➜ guix time-machine -C channels.scm -- deploy deploy/searx0-home-rekahsoft-ca.scm
  The following 1 machine will be deployed:
    searx0

  guix deploy: deploying to searx0...
  Backtrace:
            18 (primitive-load "/home/collin/.cache/guix/inferiors/trl…")
  In guix/ui.scm:
     2238:7 17 (run-guix . _)
    2201:10 16 (run-guix-command _ . _)
  In ice-9/boot-9.scm:
    1752:10 15 (with-exception-handler _ _ #:unwind? _ # _)
  In guix/status.scm:
      815:4 14 (call-with-status-report _ _)
  In ice-9/boot-9.scm:
    1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
  In guix/store.scm:
     659:37 12 (thunk)
     1298:8 11 (call-with-build-handler #<procedure 7f788feda360 at g…> …)
  In guix/scripts/deploy.scm:
     274:21 10 (_)
  In guix/store.scm:
    1380:11  9 (map/accumulate-builds #<store-connection 256.99 7f789…> …)
     1298:8  8 (call-with-build-handler #<procedure 7f788feda0c0 at g…> …)
  In ice-9/boot-9.scm:
    1752:10  7 (with-exception-handler _ _ #:unwind? _ # _)
  In guix/scripts/deploy.scm:
     159:29  6 (_)
  In gnu/machine/ssh.scm:
      463:5  5 (deploy-managed-host #<<machine> operating-system: #<<o…>)
  In guix/ssh.scm:
     199:18  4 (remote-authorize-signing-key _ _ _)
  In guix/inferior.scm:
      339:2  3 (port->inferior _ _)
  In ice-9/boot-9.scm:
    1685:16  2 (raise-exception _ #:continuable? _)
    1685:16  1 (raise-exception _ #:continuable? _)
    1685:16  0 (raise-exception _ #:continuable? _)

  ice-9/boot-9.scm:1685:16: In procedure raise-exception:
  ERROR:
    1. &inferior-exception:
        arguments: (%exception #<inferior-object #<&message message: "unsupported manifest format">>)
        inferior: #<inferior pipe (0 1 1) 7f7891abd200>
        stack: ((#f ("ice-9/boot-9.scm" 1779 13)) (raise-exception ("ice-9/boot-9.scm" 1684 16)) (call-with-input-file ("ice-9/ports.scm" 432 17)) (#f ("guix/describe.scm" 109 24)) (package-path-entries ("guix/describe.scm" 122 12)) (#f ("gnu/packages.scm" 149 17)) (primitive-load-path (#f #f #f)) (#f ("ice-9/boot-9.scm" 3563 26)) (save-module-excursion ("ice-9/boot-9.scm" 2835 4)) (#f ("ice-9/boot-9.scm" 3543 20)) (#f ("ice-9/threads.scm" 389 8)) (#f ("ice-9/boot-9.scm" 3252 13)) (#f ("ice-9/threads.scm" 389 8)) (resolve-interface ("ice-9/boot-9.scm" 3326 17)) (map1 ("ice-9/boot-9.scm" 221 17)) (map1 ("ice-9/boot-9.scm" 221 29)) (map1 ("ice-9/boot-9.scm" 221 29)) (map1 ("ice-9/boot-9.scm" 221 29)) (map1 ("ice-9/boot-9.scm" 221 29)) (map1 ("ice-9/boot-9.scm" 221 29)) (map1 ("ice-9/boot-9.scm" 221 29)) (#f ("ice-9/boot-9.scm" 3423 24)) (call-with-deferred-observers ("ice-9/boot-9.scm" 2594 24)) (define-module* ("ice-9/boot-9.scm" 3410 4)) (#f ("ice-9/boot-9.scm" 3922 23)) (primitive-load-path (#f #f #f)) (#f ("ice-9/boot-9.scm" 3563 26)) (save-module-excursion ("ice-9/boot-9.scm" 2835 4)) (#f ("ice-9/boot-9.scm" 3543 20)) (#f ("ice-9/threads.scm" 389 8)) (#f ("ice-9/boot-9.scm" 3252 13)) (#f ("ice-9/threads.scm" 389 8)) (resolve-interface ("ice-9/boot-9.scm" 3326 17)) (#f ("ice-9/boot-9.scm" 3935 31)) (map1 ("ice-9/boot-9.scm" 221 17)) (process-use-modules ("ice-9/boot-9.scm" 3934 20)) (parse ("ice-9/psyntax.scm" 258 10)) (parse ("ice-9/psyntax.scm" 1209 19)) (expand-top-sequence ("ice-9/psyntax.scm" 1217 36)) (primitive-eval ("ice-9/eval.scm" 720 20)) (#f (#f #f #f)) (#f ("guix/repl.scm" 98 21)) (with-exception-handler ("ice-9/boot-9.scm" 1751 10)) (with-exception-handler ("ice-9/boot-9.scm" 1746 15)) (#f ("guix/repl.scm" 125 7)))

I came back to this a few times and attempted to retry deployment of searx0, eventually it succeeded (after a couple retries, and with a different error, but on newer channels). searx0 is now managed as part of guix-machines (rev 8d31ff8) 🎏. This can now be applied across the remaining legacy vms.

Upgrade of ci-runner0 (more involved)

ci-runner0 has some special setup to keep in mind. Namely, it currently can run guix builds within docker, sharing the guix-daemon of the host. The way it does this is as follows.

  1. A docker image is produced.

      guix pack -f docker -S /bin=bin --entry-point=bin/guix guix coreutils bash nss-certs grep gawk sed

  2. The docker image produced in step 1 is pushed to my local docker repository.

      docker load -i <filename-of-pack-tarball>
  docker tag guix-coreutils-bash-nss-certs-grep-gawk:latest docker-local.nexus.home.rekahsoft.ca/guix:latest
  docker push docker-local.nexus.home.rekahsoft.ca/guix:latest

  3. A drone-docker-runner is setup that expects ci users to use the guix image (setup in step 1 & 2) (see setup in unguix/ci-runner0-home-rekahsoft-ca/drone-runner-guix.sh). The most notable bits of this are:

    • Setting of DRONE_RUNNER_LABELS=guix:on

    • Setting of DRONE_RUNNER_VOLUMES

      /var/guix/daemon-socket/socket:/var/guix/daemon-socket/socket
      guix socket
      /gnu/store:/gnu/store
      guix store
      home/collin/guix-ci-cache:.cache/guix
      guix pull cache
Issues
  • Users who use the guix image from their .drone.yaml file are currently references the :latest image (and will pull if-not-exists)
  • The packages that are used within the built container, need to be present on the host system (otherwise, the container is unusable). For instance, if a guix gc occurs, this can delete the packages that are needed by the container.
  • The process is currently manual, but instead should be automated via droneci itself
Solution (shorter term: keep the status quo) [100%]
  • Update the system
  • Get docker containers runner and runner-guix back running

  • Rebuild a container using the same channels that are used by ci-runner0 and push it to my local docker repository

      docker load -i $(guix time-machine -C channels.scm -- pack -f docker -S /bin=bin --entry-point=bin/guix guix coreutils bash nss-certs grep gawk sed)
  docker tag guix-coreutils-bash-nss-certs-grep-gawk:latest docker-local.nexus.home.rekahsoft.ca/guix:latest
  docker push docker-local.nexus.home.rekahsoft.ca/guix:latest

  • Run the same pack on ci-runner0 to ensure the guix store has the same items as will be used in the guix-builder docker container

      docker pull docker.nexus.home.rekahsoft.ca/guix:latest
  • Pull the new docker image on ci-runner0
Solution (longer term)
  • Create a new docker-in-docker droneci runner

  • Create a repository that leverages droneci in order to:

    1. In a step that uses the guix runner, build a docker image tarball (using guix as described in step 1 above)
    2. In a step that uses the dind droneci runner, load the aforementioned tarball and push it to my local container storage (as described in step 2 above).
    3. All existing droneci pipelines that use the guix runner need to either refer to a pinned version, or reference latest (always pulling instead of if-not-exists)

Thinking about this solution, it does not work well in the scenario of multiple droneci worker nodes. This is because the building of the guix-runner image, results in store items that will need to be present on all droneci worker nodes.

The reason this is the case is because /gnu/store is volume mounted over automatically when using the guix-runner. guix pack ... has a -R|--relocatable option that stands out as possibly useful here. Namely, if within the guix-runner container, the /gnu/store could be somewhere else, that would possibly? solve the problem. It seems that -R cannot be used (effectively) when packing docker containers, so this may end up being a multi-step process.

Upgrade of postgres (more involved)

Because postgres.home.rekahsoft.ca is a database server, more care must be taken when upgrading it. It is currently running postgres version 10.13.

  $ guix package -p /run/current-system/profile -I'^postgres'   
  postgresql	10.13	out	/gnu/store/w3rll6ghl5ajls0a1wbyvi13xc7c4gvl-postgresql-10.13

The upgrade I am about to do has the following versions of postgres available.

  $ guix time-machine -C channels.scm -- show postgresql | recfmt '{{name}}@{{version}}
  '               
  postgresql@15.1
  postgresql@14.4
  postgresql@13.9
  postgresql@11.18
  postgresql@10.23

Doing a quick look online for upgrading (what would be 4 major versions from 10 to 15), I came across this article. It suggests the best way to go about my upgrade is to pgdump_all the databases, followed by a pg_restore from the newer version. That is:

  1. Install PostgreSQL 15 binaries (aka complete guix upgrade)
  2. Install PostgreSQL extensions (aka part of step 1)
  3. Initialize PostgreSQL cluster (should be done automatically upon boot)
  4. Perform pg_dumpall/pg_restore
  5. Validation of data/objects (not sure how to do this exactly)

One option I could take is deferring the postgres upgrade, and using the 10.x version available in guix. This seems like the best choice given that nextcloud also needs to be upgraded/replaced (which depends on this postgres instance, but may not be compatible (need to check this)).

Decision and process forward [100%]

I ended up borking my nextcloud upgrade, and am using it as an opportunity to find better alternative solutions. postgres.home.rekahsoft.ca is only currently used for nextcloud, though there is a database that was going to be used from grafana (but its currently only connected to and not actually used). As such, the best way forward has been determined to be:

DONE Convert existing .guix/rekahsoft/guix-config/vms/*.scm files to guile modules

CLOSED: [2022-03-20 Sun 13:00]

DONE Resolve warnings about guix depreciations

CLOSED: [2023-03-03 Fri 21:25]

  • Note taken on [2023-03-03 Fri 21:27]
    Completed in the following commits:

  • State "DONE" from "TODO" [2023-03-03 Fri 21:25]

rekahsoft/guix-config/vms/ci-runner0-home-rekahsoft-ca.scm:25:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/ci0-home-rekahsoft-ca.scm:25:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/dash0-home-rekahsoft-ca.scm:25:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/git0-home-rekahsoft-ca.scm:25:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/loki-home-rekahsoft-ca.scm:26:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/postgres-home-rekahsoft-ca.scm:78:12: warning: 'postgresql-service' is deprecated, use 'postgresql-service-type' instead rekahsoft/guix-config/vms/postgres-home-rekahsoft-ca.scm:32:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/prom0-home-rekahsoft-ca.scm:25:14: warning: the 'target' field is deprecated, please use 'targets' instead rekahsoft/guix-config/vms/searx0-home-rekahsoft-ca.scm:26:14: warning: the 'target' field is deprecated, please use 'targets' instead

DONE Remove user 'collin' configuration (resetting to default) on legacy vms [100%]

CLOSED: [2023-03-12 Sun 20:36]

  • State "DONE" from "DOING" [2023-03-12 Sun 20:36]
  • State "DOING" from "TODO" [2023-03-05 Sun 02:37]

On some legacy vm's, I have user configuration that depends on very old packages, taking up space. Because these vm's are servers, there is no need for such a configuration. As such, remove old guix generations and unnecessary files in my users home.

  • postgres.home.rekahsoft.ca
  • searx0.home.rekahsoft.ca
  • my0.home.rekahsoft.ca
  • dash0.home.rekahsoft.ca
  • prom0.home.rekahsoft.ca
  • ci-runner0.home.rekahsoft.ca
  • ci0.home.rekahsoft.ca
  • git0.home.rekahsoft.ca
  • loki.home.rekahsoft.ca
Process

Remove all packages in my users guix profile.

  guix package -p ~/.guix-profile -I | awk -v OFS=':' '{print $1,$3}' | xargs guix package -r

Delete all older generations.

  guix package -d 1m

Pull the same channels as the current system.

  guix pull -C /run/current-system/channels.scm

Delete old guix pull generations.

  guix pull -d 1m

Cleanup home directory (this is somewhat manual for each machine).

Once the user home is clean, a .zprofile needs to be put in place that sources the system wide profile, as well as a .zshrc file so that the zsh setup function doesn't run.

  touch ~/.zshrc
  echo 'source /etc/profile' > ~/.zprofile

Run a guix garbage collection.

  guix gc

CANCELED Report odd error caused by guix deploy (at 187b9f1)

CLOSED: [2023-03-03 Fri 21:38]

  • State "CANCELED" from "TODO" [2023-03-03 Fri 21:38]
    This is now stale; as such, canceling.

Run a guix deploy:

  guix time-machine -C channels.scm -- deploy deploy/guix-ci0-home-rekahsoft-ca.scm

Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'... Updating channel 'nonguix' from Git repository at 'https://gitlab.com/nonguix/nonguix'... Updating channel 'rekahsoft-guix' from Git repository at 'https://git.rekahsoft.ca/rekahsoft/ Updating channel 'guix-machines' from Git repository at 'https://git.home.rekahsoft.ca/rekahs Authenticating channel 'guix-machines', commits acaa20d to 187b9f1 (2 new commits)… Computing Guix derivation for 'x86_64-linux'… \ substitute: updating substitutes from 'https://substitutes.nonguix.org'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/zwzfvqxz6i53s21f254dx9gpsb0xmcam-profile.drv /gnu/store/1ay2dhg8xfh9gl1k7psmrpb9h4pn3rw4-profile.drv /gnu/store/bqpxvnyqkj16qk24gw6kns4ck40hj1kl-guix-machines.drv /gnu/store/7i7j6jwgx86gjy3zd8i1pqcwb6zmyi1j-inferior-script.scm.drv /gnu/store/i2d5ba5y1n1a53mv88101vqj9x4nfkgv-inferior-script.scm.drv

building /gnu/store/7i7j6jwgx86gjy3zd8i1pqcwb6zmyi1j-inferior-script.scm.drv… building /gnu/store/bqpxvnyqkj16qk24gw6kns4ck40hj1kl-guix-machines.drv… building CA certificate bundle… listing Emacs sub-directories… building fonts directory… building directory of Info manuals… building profile with 4 packages… building /gnu/store/i2d5ba5y1n1a53mv88101vqj9x4nfkgv-inferior-script.scm.drv… building package cache… building profile with 4 packages… The following 1 machine will be deployed: guix-ci0

guix deploy: deploying to guix-ci0… guix deploy: sending 0 store items (0 MiB) to 'guix-ci0.home.rekahsoft.ca'… guix deploy: sending 0 store items (0 MiB) to 'guix-ci0.home.rekahsoft.ca'… substitute: updating substitutes from 'https://substitutes.nonguix.org'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0% 6.1 MB will be downloaded postgresql-10.20 4.8MiB 636KiB/s 00:08 [##################] 100.0% guile-fibers-1.0.0 282KiB 615KiB/s 00:00 [##################] 100.0% cuirass-1.1.0-11.9f08035 836KiB 1.1MiB/s 00:01 [##################] 100.0%substitute: updating substitutes from 'https://substitutes.nonguix.org'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% 88.8 MB will be downloaded linux-libre-5.16.16 84.7MiB 6.3MiB/s 00:13 [##################] 100.0%substitute: updating substitutes from 'https://substitutes.nonguix.org'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/mw226snj90f0ijbigygyvjk57nl3lixp-remote-exp.scm.drv /gnu/store/zf5wsiqx2qz72rkyn3d268fx6nqzx0yk-switch-to-system.scm.drv /gnu/store/nfg7d5h60xirfqa6m6rlhc9nds41im0m-system.drv /gnu/store/cpxnhcz136rgdkpdvxf70bsv170xhy0s-raw-initrd.drv /gnu/store/pimpwfhv4yq9z4bbvfw5qvkyglbf128n-init.drv /gnu/store/h0d9gr6d6iyaln79q572ykx6mqrzsvrl-linux-modules.drv /gnu/store/cs542vsmbfp7aw5lyz2v25h0mdr2i1bw-profile.drv /gnu/store/d7y59lf7gva3j911pl8pkwcld5lz004m-etc.drv /gnu/store/07ri32j59i0rvsc951v6lcghakgfh3g2-dbus-configuration.drv /gnu/store/gz724hw3bia39mm15j4khv0p7f19nd2l-dbus-system-services.drv /gnu/store/6jc565nb5y5i85dkd3rb1d024rklg9id-sudoers.drv /gnu/store/b41c1l6a9a7dz98lpw0l7d7l0rzrw9fb-environment.drv /gnu/store/ks9rjlj9wh6galla3ff0qi76l9vinrqq-rottlog.drv /gnu/store/xahch716lsxpq84gnz847prxcbbfmdkh-rottlog.weekly.drv /gnu/store/lm0m1f99j53j4ad3z3hj3w286lrvmpkz-udev.drv /gnu/store/85wk7pax89pw1dp4yi5zj00hqa68aspm-udev-rules.drv /gnu/store/nbfi4558i9g8jjbhbgiqcijlmbyydvc0-pam.d.drv /gnu/store/nf3s9d33sbxv4wrv19aaw6ih2zfrvfi0-sshd.drv /gnu/store/sy1iwg3pgbanh3s1wkg5i14hxvlwsx67-shells.drv /gnu/store/y1c8120rv3im0zvf5dskqd92wpj5p2an-etc-polkit-1.drv /gnu/store/iis797lbcxf5g2r0z1mwcfqki4xiw81y-provenance.drv /gnu/store/phr0c49z2icjz9nzzpz9zx56wvbd1xsp-profile.drv /gnu/store/vj5f5y76fxwrs5x45i3p136lqrkq2f83-parameters.drv /gnu/store/w4nwh7k1gqnzgvcgrdwnrqrsv9ynphqn-activate.scm.drv /gnu/store/1q7ms0i49i827mf1wykhh9lway1k4wci-activate-service.scm.drv /gnu/store/8znvf1mcyixq1zjqj2rzk42rcfywrrjg-activate-service.scm.drv /gnu/store/fs45w7lrqdxc1ibgazzlfyjgwjx6z93p-openssh-authorized-keys.drv /gnu/store/b47q5z9p9bighs42wxz35y16s6b6l9zc-activate-service.scm.drv /gnu/store/ghcnmarfy9q69rrcskfkqrsc44jqqaii-activate-service.scm.drv /gnu/store/0mr37ba2496synaxvlhl0nbppv1cf7i2-nginx.conf.drv /gnu/store/npryqjbc54mi90vvfjzsgf6pazmycsg2-activate-service.scm.drv /gnu/store/q2q9i8ynhvcrd6wv18qj41zxs7w6x409-acl.drv /gnu/store/v23jm1w2ay2lw85v5cj3r1llcaxfs7fs-activate-service.scm.drv /gnu/store/vka2qaq0vwq24w4j229wjw8l6g7fa7lw-activate-service.scm.drv /gnu/store/wqbk8rkppf9cwh010ndzqgnmdxkn83nq-boot.drv /gnu/store/8vklhk7yb6li7ri921ka6qgwbpkapyiv-shepherd.conf.drv /gnu/store/1b431m7z3xn8aav1sqbimzs805ccfd8z-shepherd-cuirass.go.drv /gnu/store/2ykd5cnwpcfmbl9n9my24b89js292d2j-shepherd-cuirass.scm.drv /gnu/store/1z1qas3pmgkwycn7n4kpk44m93kf5jhf-shepherd-cuirass-web.go.drv /gnu/store/gi5w86wlsvydg48ridr4flww6jq3m1c5-shepherd-cuirass-web.scm.drv /gnu/store/36w0x65d14ig56vk8jg5gbp15ha7y41q-shepherd-nginx.go.drv /gnu/store/35dffx24y1pyhjlj3kvv2iaz0xh6axn1-shepherd-nginx.scm.drv /gnu/store/4x6bpx8lr8hyzsx6mz8xn2w3ggr8319y-shepherd-guix-publish.go.drv /gnu/store/191fbw4zs0f6lzwghv2m3yvz6qdl5vl2-shepherd-guix-publish.scm.drv /gnu/store/5imwai3i8s57fmvk3gn8ycikwiak2jvb-shepherd-guix-daemon.go.drv /gnu/store/qli01q0hb0gdbmv8yr5w6k3fyriaqvpr-shepherd-guix-daemon.scm.drv /gnu/store/79w3gcr50xp6wsn2gqjf666d1rp2q5a7-shepherd-postgres.go.drv /gnu/store/rgvyzg1gfdhy16ykykx43f46ix16nm4p-shepherd-postgres.scm.drv /gnu/store/algyraghsl3ajbywdv84jq3lmxbc6mnk-pg_ctl-wrapper.drv /gnu/store/f9n7xixvwa0n2h3a138zdsyl8hd4gwr8-shepherd-user-homes.go.drv /gnu/store/hrnjmcwr42wz2spcra8bgfcy1845jbcd-shepherd-user-homes.scm.drv /gnu/store/fql8jsv400ga2bgcikx26gw33w9640c5-shepherd-nscd.go.drv /gnu/store/h3s2hhjbhcq96haf34hsk5b415wyggzc-shepherd-ssh-daemon-ssh-sshd.go.drv /gnu/store/9a5irfzc84v5358bzjh6ijzqxhx9nf7p-shepherd-ssh-daemon-ssh-sshd.scm.drv /gnu/store/k34ja2nmnj9ajsf8insj4xz1rxsjdn1y-shepherd-host-name.go.drv /gnu/store/n7pymmbciz7dhhnq2hpy572sy2bbi3bf-shepherd-file-systems.go.drv /gnu/store/psjw14f7r7li2gbhgp1f2ima5pib799b-shepherd-postgres-roles.go.drv /gnu/store/56j2rljjjwf24rg53snnxdx0b7n46amp-shepherd-postgres-roles.scm.drv /gnu/store/wvbzvbba487ldiw1b2nsa6rfdjh253ws-shepherd-prometheus-node-exporter.go.drv /gnu/store/c50c8v3rda7a158m0wnqy878h3ckjpj5-shepherd-prometheus-node-exporter.scm.drv /gnu/store/y29z7fpwfyqbsqvsia2nr9hxmq0v40c6-shepherd-user-file-systems.go.drv

3.8 MB will be downloaded building /gnu/store/b47q5z9p9bighs42wxz35y16s6b6l9zc-activate-service.scm.drv… go-github-com-prometheus-node-exporter-0.18.1 3.6MiB 966KiB/s 00:04 [##################] 100.0% queries 242B 272KiB/s 00:00 [##################] 100.0% sshd_config 335B 218KiB/s 00:00 [##################] 100.0%building /gnu/store/q2q9i8ynhvcrd6wv18qj41zxs7w6x409-acl.drv… building /gnu/store/v23jm1w2ay2lw85v5cj3r1llcaxfs7fs-activate-service.scm.drv… building /gnu/store/npryqjbc54mi90vvfjzsgf6pazmycsg2-activate-service.scm.drv… building /gnu/store/gz724hw3bia39mm15j4khv0p7f19nd2l-dbus-system-services.drv… building /gnu/store/b41c1l6a9a7dz98lpw0l7d7l0rzrw9fb-environment.drv… building /gnu/store/07ri32j59i0rvsc951v6lcghakgfh3g2-dbus-configuration.drv… building /gnu/store/y1c8120rv3im0zvf5dskqd92wpj5p2an-etc-polkit-1.drv… applying 1 graft for guile-squee-0-3.a151fd0 … running profile hook of type 'linux-module-database'… applying 5 grafts for cuirass-1.1.0-11.9f08035 … building /gnu/store/h0d9gr6d6iyaln79q572ykx6mqrzsvrl-linux-modules.drv… applying 1 graft for nginx-1.21.5 … building /gnu/store/pimpwfhv4yq9z4bbvfw5qvkyglbf128n-init.drv… building /gnu/store/0mr37ba2496synaxvlhl0nbppv1cf7i2-nginx.conf.drv… building /gnu/store/fs45w7lrqdxc1ibgazzlfyjgwjx6z93p-openssh-authorized-keys.drv… building /gnu/store/ghcnmarfy9q69rrcskfkqrsc44jqqaii-activate-service.scm.drv… building /gnu/store/8znvf1mcyixq1zjqj2rzk42rcfywrrjg-activate-service.scm.drv… applying 2 grafts for postgresql-10.20 … applying 2 grafts for postgresql-13.4 … building /gnu/store/1q7ms0i49i827mf1wykhh9lway1k4wci-activate-service.scm.drv… building CA certificate bundle… listing Emacs sub-directories… building fonts directory… generating GdkPixbuf loaders cache… generating GLib schema cache… creating GTK+ icon theme cache… building cache files for GTK+ input methods… building directory of Info manuals… building database for manual pages… building /gnu/store/algyraghsl3ajbywdv84jq3lmxbc6mnk-pg_ctl-wrapper.drv… building profile with 1 package… building /gnu/store/iis797lbcxf5g2r0z1mwcfqki4xiw81y-provenance.drv… building /gnu/store/cpxnhcz136rgdkpdvxf70bsv170xhy0s-raw-initrd.drv… building /gnu/store/xahch716lsxpq84gnz847prxcbbfmdkh-rottlog.weekly.drv… building /gnu/store/vj5f5y76fxwrs5x45i3p136lqrkq2f83-parameters.drv… building /gnu/store/ks9rjlj9wh6galla3ff0qi76l9vinrqq-rottlog.drv… building /gnu/store/sy1iwg3pgbanh3s1wkg5i14hxvlwsx67-shells.drv… building /gnu/store/gi5w86wlsvydg48ridr4flww6jq3m1c5-shepherd-cuirass-web.scm.drv… building /gnu/store/2ykd5cnwpcfmbl9n9my24b89js292d2j-shepherd-cuirass.scm.drv… building /gnu/store/1z1qas3pmgkwycn7n4kpk44m93kf5jhf-shepherd-cuirass-web.go.drv… building /gnu/store/1b431m7z3xn8aav1sqbimzs805ccfd8z-shepherd-cuirass.go.drv… building /gnu/store/n7pymmbciz7dhhnq2hpy572sy2bbi3bf-shepherd-file-systems.go.drv… building /gnu/store/qli01q0hb0gdbmv8yr5w6k3fyriaqvpr-shepherd-guix-daemon.scm.drv… building /gnu/store/191fbw4zs0f6lzwghv2m3yvz6qdl5vl2-shepherd-guix-publish.scm.drv… building /gnu/store/5imwai3i8s57fmvk3gn8ycikwiak2jvb-shepherd-guix-daemon.go.drv… building /gnu/store/4x6bpx8lr8hyzsx6mz8xn2w3ggr8319y-shepherd-guix-publish.go.drv… building /gnu/store/k34ja2nmnj9ajsf8insj4xz1rxsjdn1y-shepherd-host-name.go.drv… building /gnu/store/35dffx24y1pyhjlj3kvv2iaz0xh6axn1-shepherd-nginx.scm.drv… building /gnu/store/fql8jsv400ga2bgcikx26gw33w9640c5-shepherd-nscd.go.drv… building /gnu/store/36w0x65d14ig56vk8jg5gbp15ha7y41q-shepherd-nginx.go.drv… building /gnu/store/56j2rljjjwf24rg53snnxdx0b7n46amp-shepherd-postgres-roles.scm.drv… building /gnu/store/rgvyzg1gfdhy16ykykx43f46ix16nm4p-shepherd-postgres.scm.drv… building /gnu/store/psjw14f7r7li2gbhgp1f2ima5pib799b-shepherd-postgres-roles.go.drv… building /gnu/store/79w3gcr50xp6wsn2gqjf666d1rp2q5a7-shepherd-postgres.go.drv… building /gnu/store/c50c8v3rda7a158m0wnqy878h3ckjpj5-shepherd-prometheus-node-exporter.scm.drv… building /gnu/store/9a5irfzc84v5358bzjh6ijzqxhx9nf7p-shepherd-ssh-daemon-ssh-sshd.scm.drv… building /gnu/store/wvbzvbba487ldiw1b2nsa6rfdjh253ws-shepherd-prometheus-node-exporter.go.drv… building /gnu/store/h3s2hhjbhcq96haf34hsk5b415wyggzc-shepherd-ssh-daemon-ssh-sshd.go.drv… building /gnu/store/y29z7fpwfyqbsqvsia2nr9hxmq0v40c6-shepherd-user-file-systems.go.drv… building /gnu/store/hrnjmcwr42wz2spcra8bgfcy1845jbcd-shepherd-user-homes.scm.drv… building /gnu/store/nf3s9d33sbxv4wrv19aaw6ih2zfrvfi0-sshd.drv… building /gnu/store/f9n7xixvwa0n2h3a138zdsyl8hd4gwr8-shepherd-user-homes.go.drv… building /gnu/store/nbfi4558i9g8jjbhbgiqcijlmbyydvc0-pam.d.drv… building /gnu/store/8vklhk7yb6li7ri921ka6qgwbpkapyiv-shepherd.conf.drv… building /gnu/store/6jc565nb5y5i85dkd3rb1d024rklg9id-sudoers.drv… building /gnu/store/85wk7pax89pw1dp4yi5zj00hqa68aspm-udev-rules.drv… building XDG desktop file cache… building /gnu/store/lm0m1f99j53j4ad3z3hj3w286lrvmpkz-udev.drv… building XDG MIME database… building /gnu/store/d7y59lf7gva3j911pl8pkwcld5lz004m-etc.drv… building profile with 57 packages… building /gnu/store/vka2qaq0vwq24w4j229wjw8l6g7fa7lw-activate-service.scm.drv… building /gnu/store/w4nwh7k1gqnzgvcgrdwnrqrsv9ynphqn-activate.scm.drv… building /gnu/store/wqbk8rkppf9cwh010ndzqgnmdxkn83nq-boot.drv… building /gnu/store/nfg7d5h60xirfqa6m6rlhc9nds41im0m-system.drv… building /gnu/store/zf5wsiqx2qz72rkyn3d268fx6nqzx0yk-switch-to-system.scm.drv… building /gnu/store/mw226snj90f0ijbigygyvjk57nl3lixp-remote-exp.scm.drv… guix deploy: sending 258 store items (1,425 MiB) to 'guix-ci0.home.rekahsoft.ca'… guix deploy: sending 0 store items (0 MiB) to 'guix-ci0.home.rekahsoft.ca'… The following derivations will be built: /gnu/store/dhrqvw4y4niq7y2j6i12r36k6ji6rznz-remote-exp.scm.drv /gnu/store/qbn6b0yasn5gdd9d0s3wfcy3mp1z4vif-upgrade-shepherd-services.scm.drv

building /gnu/store/qbn6b0yasn5gdd9d0s3wfcy3mp1z4vif-upgrade-shepherd-services.scm.drv… building /gnu/store/dhrqvw4y4niq7y2j6i12r36k6ji6rznz-remote-exp.scm.drv… guix deploy: sending 2 store items (0 MiB) to 'guix-ci0.home.rekahsoft.ca'… substitute: updating substitutes from 'https://substitutes.nonguix.org'... 100.0% 4.3 MB will be downloaded grub-2.06 4.1MiB 1.5MiB/s 00:03 [##################] 100.0%substitute: updating substitutes from 'https://substitutes.nonguix.org'... 100.0% The following derivations will be built: /gnu/store/k4pz0fa80drp4knpch20yic0q4ljakwx-remote-exp.scm.drv /gnu/store/m96lzbpz36v804drryfsg1glq5nfpxwy-install-bootloader.scm.drv /gnu/store/lnx5p4vp68bjss8ph6n22nvglyz4w915-grub.cfg.drv

0.8 MB will be downloaded grub-locales 753KiB 447KiB/s 00:02 [##################] 100.0%applying 4 grafts for grub-2.06 … building /gnu/store/lnx5p4vp68bjss8ph6n22nvglyz4w915-grub.cfg.drv… building /gnu/store/m96lzbpz36v804drryfsg1glq5nfpxwy-install-bootloader.scm.drv… building /gnu/store/k4pz0fa80drp4knpch20yic0q4ljakwx-remote-exp.scm.drv… ;;; [2022/03/22 22:37:14.445896, 0] [GSSH ERROR] Channel opening failure: channel 66 error (2) open failed: #<input-output: channel (closed) 7fdfaa81dd00> Backtrace: In guix/store.scm: 1410:13 19 (map/accumulate-builds #<store-connection 256.99 7fdfb…> …) 1385:9 18 (map/accumulate-builds #<store-connection 256.99 7fdfb…> …) 1320:8 17 (call-with-build-handler #<procedure 7fdfaa5186c0 at g…> …) In ice-9/boot-9.scm: 1752:10 16 (with-exception-handler _ _ #:unwind? _ # _) In guix/scripts/deploy.scm: 159:6 15 (_) In guix/store.scm: 2129:25 14 (run-with-store #<store-connection 256.99 7fdfb219e1e0> …) In gnu/machine/ssh.scm: 506:32 13 (_ _) In ice-9/boot-9.scm: 1752:10 12 (with-exception-handler _ _ #:unwind? _ # _) In gnu/machine/ssh.scm: 506:32 11 (_) In guix/store.scm: 2129:25 10 (run-with-store #<store-connection 256.99 7fdfab42f370> …) In guix/remote.scm: 138:10 9 (_ _) In guix/store.scm: 2001:38 8 (_ #<store-connection 256.99 7fdfab42f370>) In guix/ssh.scm: 197:18 7 (send-files #<store-connection 256.99 7fdfab42f370> _ # …) 184:15 6 (remote-inferior _ _) In ssh/popen.scm: 64:4 5 (open-remote-pipe* _ "r+" _ . _) In unknown file: 4 (channel-open-session #<input-output: channel (closed) …>) In ice-9/boot-9.scm: 1685:16 3 (raise-exception _ #:continuable? _) 1683:16 2 (raise-exception _ #:continuable? _) 1685:16 1 (raise-exception _ #:continuable? _) 1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception: Throw to key `guile-ssh-error' with args `("channel-open-session" "Channel opening failure: channel 66 error (2) open failed" #<input-output: channel (closed) 7fdfaa81dd00> #f)'.

This appears to create a borked profile.

guix-ci0% guix system list-generations 1d Generation 30 Mar 21 2022 22:08:06 (current) file name: /var/guix/profiles/system-30-link canonical file name: /gnu/store/0nc0iwagsixzzrqsavx6yhf52rynjri3-system label: GNU with Linux-Libre 5.15.16 bootloader: grub root device: label: "root" kernel: /gnu/store/ywy3xy6pkcn3gasm002fz44m5w8ix0j2-linux-libre-5.15.16/bzImage channels: guix: repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: a4eae0c3adce8e4c4ac153a4959d18b9897a67e1 rekahsoft-guix: repository URL: https://git.rekahsoft.ca/rekahsoft/rekahsoft-guix.git branch: master commit: b72c13392a9ffab0dd52dcf9e30fb599e364a78e configuration file: /gnu/store/p150w49v28i8xl0li1sci9l8bpak724b-configuration.scm guix system: warning: unrecognized boot parameters at '/var/guix/profiles/system-32-link/parameters' Backtrace: 10 (primitive-load "home/collin.config/guix/current/bin/…") In guix/ui.scm: 2206:7 9 (run-guix . _) 2169:10 8 (run-guix-command _ . _) In ice-9/boot-9.scm: 1752:10 7 (with-exception-handler _ _ #:unwind? _ # _) In guix/status.scm: 822:3 6 (_) 802:4 5 (call-with-status-report _ _) In ice-9/boot-9.scm: 1747:15 4 (with-exception-handler #<procedure 7f1e13707630 at ic…> …) In srfi/srfi-1.scm: 634:9 3 (for-each #<procedure display-system-generation (numbe…> …) In guix/scripts/system.scm: 492:24 2 (display-system-generation 32 _) In gnu/system.scm: 469:15 1 (read-boot-parameters-file "/var/guix/profiles/system-3…") In ice-9/boot-9.scm: 1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception: In procedure struct-vtable: Wrong type argument in position 1 (expecting struct): #f

DONE Determine how to use guix for ci

CLOSED: [2023-03-06 Mon 20:03]

  • State "DONE" from "DOING" [2023-03-06 Mon 20:03]

  • Note taken on [2023-03-06 Mon 20:00]
    This has been determined (and is in active use). A drone-docker-runner is setup that expects ci users to use the guix image (see setup in unguix/ci-runner0-home-rekahsoft-ca/drone-runner-guix.sh). The most notable bits of this are:

    • Setting of DRONE_RUNNER_LABELS=guix:on

    • Setting of DRONE_RUNNER_VOLUMES

      /var/guix/daemon-socket/socket:/var/guix/daemon-socket/socket
      guix socket
      /gnu/store:/gnu/store
      guix store
      home/collin/guix-ci-cache:.cache/guix
      guix pull cache

    This approach will work for any ci-system (eg. WoodpeckerCI) that allows for the guix-daemon to be shared into a container without the need for a ci-user to be privileged (eg. mount volumes themselves).

  • Note taken on [2023-03-04 Sat 09:53]
    This was actually started (and somewhat completed) a while ago, however there are issues with the setup which I will detail here later.
  • State "DOING" from "TODO" [2023-03-04 Sat 09:53]
See

DONE Setup cloud0-home-rekahsoft-ca for guix deploy

CLOSED: [2023-03-26 Sun 21:44] DEADLINE: <2023-03-18 Sat>

  • State "DONE" from "DOING" [2023-03-26 Sun 21:44]
  • State "DOING" from "NEXT" [2023-03-25 Sat 18:44]
  • State "NEXT" from "TODO" [2023-03-12 Sun 13:50]

This vm was used in my early testing of guix deploy and does not yet use a channel and also has not been updated for some time.

  guix system describe
Generation 26	Jan 08 2022 02:03:33	(current)
  file name: /var/guix/profiles/system-26-link
  canonical file name: /gnu/store/ibhwhg955psms72zihhhsk1li7c26z8g-system
  label: GNU with Linux-Libre 5.15.11
  bootloader: grub
  root device: label: "root"
  kernel: /gnu/store/v5wrrps3xg85knrd03gb1k8q9s5rljic-linux-libre-5.15.11/bzImage
  channels:
    guix:
      repository URL: https://git.savannah.gnu.org/git/guix.git
      branch: master
      commit: 253f02481aad5e16779fbdbf7441fab4eaea75c0
    rekahsoft-guix:
      repository URL: https://git.rekahsoft.ca/rekahsoft/rekahsoft-guix.git
      branch: master
      commit: 4c7f7ff2864152cd90a89a13e56be32ce9ff2c99
  configuration file: /gnu/store/vs7famg181p0hhgy0g9whgqqyyajz0qa-configuration.scm

Recently, an upgrade of nextcloud went sideways, and I decided to move forward without it (finding replacements as part of Investigate replacing nextcloud with another dav + file storage solution). As such, the only thing that matters on this system is ensuring mounting of cephfs continues to work as expected (on guix and otherwise).

Currently on the system, the following version of ceph is in use:

  guix package -p /run/current-system/profile -I'^ceph$'
ceph	16.2.7	out	/gnu/store/b39bpgnp1lz8qb78cynxv0793kd14s1g-ceph-16.2.7

An upgrade will result in the following version of ceph:

  guix time-machine -C channels.scm -- show ceph | recfmt '{{name}}@{{version}}
  '
ceph@17.2.5

This upgrade actually introduces a change to mount.ceph that will impact my existing uses.

ceph.mount documentation:

Specifically, there is a new mount syntax, with support for the 'legacy mount syntax' but with possible minor variations.

Next step is to actually try the upgrade to see what happens (this is of little to no risk as I can simply roll back guix system generations if it fails). However, it will provide valuable insight, as I have a similar issue on my (currently archlinux powered) plex0 vm.

After updating (following the notes in Rotate ssh-host keys (notes)) cloud0 can now be guix deployed to, and unexpectedly, cephfs continues to work (though there is one issue where the shepherd service now tries to mount cephfs before the network is up, so it fails and needs to be mounted (via running the shepherd service, or manually)).

Took the opportunity also refactor mounting of cephfs into a guix service with corresponding configuration type. See guix-machines (9b73115) (moved upstream to rekahsoft-guix channel in guix-machines (78fc788)).

CANCELED Move deploy configs into channel, and reference them from deploy/<hostname>.scm files

CLOSED: [2023-03-28 Tue 14:30]

  • State "CANCELED" from "DOING" [2023-03-28 Tue 14:30]
    Canceling as premature. Currently the guix deploy sub-command does not accept a -e|--install-from=<expression> option, so there is not much point in making this change now.
  • State "DOING" from "TODO" [2022-03-22 Tue 15:55]

This will allow for something like this (once guix deploy supports a -e option similar to other guix commands).

  guix time-machine -C channels.scm -- deploy -e '(@ (rekahsoft guix-config vms <hostname>) machine)'

Note: This also assumes that the guix-machines channel is listed in channels.scm.

Since guix deploy does not (yet) support a -e option, a deploy file can be referenced directly (that itself refers to the appropriate deploy record from this channel), like so:

  guix time-machine -C channels.scm -- deploy deploy/<hostname>.scm

Notice how the -L option and its argument are no longer specified (as it is assumed the appropriate version of guix-machines is set in the channels.scm file).

CANCELED Look into ways to mount nextcloud (cloud0) storage conveniently (nfs, sshfs, etc..)   home_network

CLOSED: [2023-03-28 Tue 14:45] DEADLINE: <2023-06-15 Thu>

  • State "CANCELED" from "TODO" [2023-03-28 Tue 14:45]
    No longer relevant as I'm moving away from nextcloud.

See: https://ownyourbits.com/2017/04/18/different-ways-to-access-your-nextcloud-files/

Backlog   backlog

DEADLINE: <1990-05-08 Tue>

TODO Improve metrics collection [0/4]   metrics

TODO Setup sabnzbd_exporter (Prometheus exporter for sabnzbd)   home_network metrics
TODO Setup transmission-exporter (Prometheus exporter for transmission)   home_network metrics
TODO Setup postgres_exporter (Prometheus postgres exporter)   home_network metrics
TODO Setup prometheus exporter for sonarr/radarr/lidarr   home_network metrics

TODO CI Improvements [0/2]

TODO Look into various droneci enhancements   home_network cicd

See:

TODO Setup drone s3 caching for caching builds of blog-rekahsoft-ca
TODO Investigate woodpecker-ci (foss fork of droneci) as a replacement for droneci   home_network spike

TODO Setup openLDAP for centralized authentication/user-management [0/6]   home_network

TODO Investigate running SSO (eg. keycloak or LDAP)   spike

See:

TODO Hook up my.home.rekahsoft.ca to LDAP
TODO Hook up pve-*.home.rekahsoft.ca to LDAP
TODO Hook up grocy.home.rekahsoft.ca to LDAP
TODO Hook up cloud.home.rekahsoft.ca to LDAP
TODO Hook up center.home.rekahsoft.ca to LDAP

TODO Move all services from Archlinux based plex.home.rekahsoft.ca to GNU Guix [0%]

DEADLINE: <2023-04-30 Sun>

  • Create guix configuration for replacement of plex.home.rekahsoft.ca host

    • This must be based on proxmox-vm-lvm-minimal
    • All of the services also depend on being able to have access to my local CephFS, so it must be automatically mounted on boot

  • Migrate services (all of which are installed and configured as archlinux packages) [0%]

Ideas   spike

DEADLINE: <1990-05-08 Tue>

TODO Setup home-assistant accessible on <SOMETHING>.home.rekahsoft.ca

DEADLINE: <2023-05-01 Mon>

TODO [C] Investigate usefulness of photoprism to manage photos similar to google photos   home_network

DEADLINE: <2023-03-26 Sun>

TODO Investigate and setup Readarr

TODO Setup octoprint accessible on 3d.home.rekahsoft.ca   home_network

TODO Migrate to latest postgres on postgres.home.rekahsoft.ca and cleanup [0%]

This follows maintenance work completed as part of Rotate ssh-host keys.

  • Unpin postgres 10.x and use the latest postgres (multiple major version bump is fine, as we don't care about the existing data)
  • Watch to see how a postgres upgrade on guix works. I expect this will require manual intervention.
  • Remove data (or use pg_upgrade or reset postgres if the data upgrade didn't work out)

TODO Run/configure docker/docker-compose from within guix configurations

Better docker support within guix configuration is required to make this issue go away. Once completed, the unguix directory can be removed.

Checked indicates VMs that use docker/docker-compose (and either depend on files in the unguix directory or docker-compose files on the vm disk.

DOING Add missing vm's to Prometheus configuration [75%]   home_network

DEADLINE: <2023-04-01 Sat>

  • State "DOING" from "TODO" [2022-07-01 Fri 13:09]
  • vault0.home.rekahsoft.ca
  • guix-ci0.home.rekahsoft.ca
  • cloud0.home.rekahsoft.ca
  • grocy0.home.rekahsoft.ca
  • nexus0.home.rekahsoft.ca
  • prom0.home.rekahsoft.ca
  • dash0.home.rekahsoft.ca
  • pbs.home.rekahsoft.ca

DOING [#A] Make prometheus/loki setup production ready and used everywhere   home_network metrics

DEADLINE: <2023-04-01 Sat>

NEXT [#A] Investigate replacing nextcloud with another dav + file storage solution

DEADLINE: <2023-04-01 Sat>

  • State "NEXT" from "TODO" [2023-03-10 Fri 09:44]

I'm currently thinking radicle + SyncThing, but more research is required.

The primary goal of this change would to avoid the maintenance and challenges that come along with running nextcloud. As well as some of the performance challenges I have seen when handling many files (eg. media not loading or crashing the nextcloud android app).

TODO Allow for using guix home for minimal server dotfiles (eg so emacs-tramp works)

DEADLINE: <2023-04-01 Sat>

PAUSE [#A] Update nextcloud   home_network

DEADLINE: <2023-04-02 Sun>

  • State "PAUSE" from "DOING" [2023-03-12 Sun 22:15]
    Pausing this as the update is now borked and I'm using it as an opportunity to replace nextcloud
  • Note taken on [2023-03-10 Fri 09:41]
    I attempted to update right to the latest version out of laziness. I should have done it version by version, because the update paused for some time, and then failed saying multiple major versions upgrades are not supported. Fair, but the update also wrote changes (not sure exactly where, but the /var/www/html volume was adjusted, and perhaps other stuff too). Anyways, the old nextcloud container no longer boots, and the system is officially bored. Though this was unexpected, in a way it could be viewed as a blessing in disguise. Namely, now I can replace nextcloud (which I have had more and more performance issues with as it is more heavily used).
  • State "DOING" from "NEXT" [2023-03-09 Thu 21:16]
  • Note taken on [2023-01-05 Thu 12:36]
    I may just replace nextcloud with radicle + some file storage web/mobile solution (eg. SyncThing)

  • Note taken on [2022-09-01 Thu 16:49]
    Nextcloud reports:

    Nextcloud 22.2.5

    The version you are running is not maintained anymore. Please make sure to update to a supported version as soon as possible.

    Thus, its important to update ASAP

  • State "NEXT" from "TODO" [2022-09-01 Thu 16:49]

TODO Resolve deprecation's in .guix/rekahsoft/guix-config/proxmox-vm-lvm-minimal.scm

DEADLINE: <2023-04-04 Tue>

  .guix/rekahsoft/guix-config/proxmox-vm-lvm-minimal.scm:82:11: warning: 'accountsservice-service' is deprecated, use 'accountsservice-service-type' instead
  .guix/rekahsoft/guix-config/proxmox-vm-lvm-minimal.scm:84:11: warning: 'elogind-service' is deprecated, use 'elogind-service-type' instead
  .guix/rekahsoft/guix-config/proxmox-vm-lvm-minimal.scm:85:11: warning: 'dbus-service' is deprecated, use 'dbus-root-service-type' instead

TODO [A] Setup btrfs backups (using Btrbk + cron) [0%]

DEADLINE: <2023-04-04 Tue>

  • Personal "Work" laptop
  • Personal laptop

DOING Perform manual update of all systems

DEADLINE: <2023-03-31 Fri>