Commit Graph

23 Commits

Author SHA1 Message Date
Ludovic Courtès 76832d3420
Remove most uses of the _IO*F constants.
These constants, for use with 'setvbuf', were deprecated in Guile 2.2
and disappeared in Guile 3.0.  Here we keep these constants in
build-side code where removing them is not feasible.

* guix/build/download-nar.scm (download-nar): Adjust 'setvbuf' calls to
the Guile 2.2+ API.
* guix/build/download.scm (open-socket-for-uri): Likewise.
(open-connection-for-uri, url-fetch): Likewise.
* guix/build/make-bootstrap.scm (make-stripped-libc): Likewise.
* guix/build/union.scm (setvbuf) [guile-2.0]: New conditional wrapper.
(union-build): Adjust to new API.
* guix/ftp-client.scm (ftp-open, ftp-list, ftp-retr): Likewise.
* guix/http-client.scm (http-fetch): Likewise.
* guix/inferior.scm (proxy): Likewise.
* guix/scripts/substitute.scm (fetch, http-multiple-get): Likewise.
* guix/self.scm (compiled-modules): Likewise.
* guix/ssh.scm (remote-daemon-channel, store-import-channel)
(store-export-channel): Likewise.
* guix/ui.scm (initialize-guix): Likewise.
* tests/publish.scm (http-get-port): Likewise.
* guix/store.scm (%newlines): Adjust comment.
2019-01-09 14:47:53 +01:00
Ludovic Courtès ed7b44370f
offload: Use (guix inferior) instead of (ssh dist node).
Using inferiors and thus 'guix repl' simplifies setup on build
machines (no need to worry about GUILE_LOAD_PATH etc.)

Furthermore, the 'guix repl -t machine' protocol running in a remote
pipe addresses several issues with the current implementation of nodes
and RREPLs in Guile-SSH: fewer round trips, doesn't leave a 'guile
--listen' process behind it, stateless (since a new process is started
each time), more efficient (the SSH channel can be reused), more
reliable (no 'pgrep', 'pkill', and shellology; see
<https://github.com/artyom-poptsov/guile-ssh/issues/11> as an example.)

* guix/ssh.scm (inferior-remote-eval): New procedure.
(send-files): Use it instead of 'make-node' and 'node-eval'.
* guix/scripts/offload.scm (node-guile-version): New procedure.
(node-free-disk-space, transfer-and-offload, node-load)
(choose-build-machine, assert-node-has-guix): Use 'remote-inferior'
instead of 'make-node' and 'inferior-eval' instead of 'node-eval'.
(assert-node-can-import, assert-node-can-export): Likewise, and add
'session' parameter.
(check-machine-availability): Likewise, and add calls to
'close-inferior' and 'disconnect!'.
(check-machine-status): Likewise.
* doc/guix.texi (Daemon Offload Setup): Remove bit related to 'guile' in
$PATH and $GUILE_LOAD_PATH; mention 'guix' alone.
2018-12-24 16:06:32 +01:00
Ludovic Courtès af15fe13b6
ssh: Add 'remote-inferior'.
* guix/inferior.scm (<inferior>)[close]: New field.
(port->inferior): New procedure.
(open-inferior): Rewrite in terms of 'port->inferior'.
(close-inferior): Honor INFERIOR's 'close' field.
(inferior-eval-with-store): Add FIXME comment.
* guix/ssh.scm (remote-inferior): New procedure.
2018-12-24 16:06:32 +01:00
Ludovic Courtès 63fd9f084a
ssh: Make 'send-files' more robust.
Possibly fixes <https://bugs.gnu.org/33239>.

* guix/ssh.scm (send-files): Call 'channel-get-exit-status' only when
RESULT is true.
2018-11-24 19:52:16 +01:00
Maxim Cournoyer 2e4d83398b
ssh: Honor the SOCKET-NAME argument of connect-to-remote-daemon.
* guix/ssh.scm (connect-to-remote-daemon): Pass the `socket-name' variable to
the `open-connection' call so that it is honored.
2018-11-05 21:28:59 -05:00
Ludovic Courtès b90d97ec2e
ssh: Display the size of the store items being sent.
* guix/ssh.scm (send-files): Compute the size of MISSING and display
it.
2018-07-29 18:50:28 +02:00
Ludovic Courtès 8446dc5a36
ssh: Work around 'select' bug in Guile.
Fixes <https://bugs.gnu.org/30365>.

* guix/ssh.scm (remote-daemon-channel)[redirect]: Define 'select*' and
use it.
2018-02-09 11:23:27 +01:00
Ludovic Courtès 0dcf675c56
ssh: Switch back to 'get-bytevector-some'.
This mostly reverts 17af5d51de.
Suggested by Andy Wingo <wingo@igalia.com>.

* guix/ssh.scm (remote-daemon-channel)[redirect]: Remove 'read!' FFI
hack.  Use buffered ports.
2018-01-12 23:41:15 +01:00
Ludovic Courtès 4eb0f9ae05
offload: 'test' reports Guile and module errors more nicely.
Fixes <https://bugs.gnu.org/26008>.
Reported by Myles English <mylesenglish@gmail.com>.

* guix/ssh.scm (retrieve-files*): Move error reporting to...
(report-guile-error, report-module-error): ... here.  New procedures.
* guix/scripts/offload.scm (assert-node-repl): Use 'report-guile-error'.
(assert-node-has-guix): Explicitly check for 'use-modules' first.  Use
'report-module-error'.
2018-01-12 23:41:15 +01:00
Ludovic Courtès 0e3c8528af
ssh: 'retrieve-files' now only retrieves what's missing.
* guix/ssh.scm (retrieve-files): Remove the subset of FILES that is
valid in LOCAL.
(store-export-channel): Add comment.
2018-01-12 23:41:15 +01:00
Ludovic Courtès d06d54e338
offload: Fix regression in file retrieval.
This fixes a regression in 'retrieve-files*' introduced in
896fec476f, whereby (guix scripts offload)
would not read the initial sexp now sent by the remote host via
'store-export-channel'.  This would effectively prevent file retrieval
entirely when offloading.

* guix/ssh.scm (retrieve-files*): New procedure, like former
'retrieve-files' but with an extra #:import parameter.
(retrieve-files): Rewrite in terms of 'retrieve-files*'.
(file-retrieval-port): Make private.
* guix/scripts/offload.scm (transfer-and-offload): Pass #:import to
'retrieve-files*'.
(retrieve-files*): Remove.
2018-01-12 23:41:15 +01:00
Ludovic Courtès 55f40fdbcd
ssh: Pass an empty "exceptfds" set to 'select'.
Previously the redirect code could end up exiting prematurely because of
an uninteresting "exceptional condition" on the socket (info "(libc)
Waiting for I/O").

* guix/ssh.scm (remote-daemon-channel): Pass the empty list as the third
argument to 'select'.  It was a mistake to pass a non-empty list there
in the first place.
2018-01-11 00:00:03 +01:00
Ludovic Courtès 17af5d51de
ssh: Work around 'get-bytevector-some' bug.
This works around <https://bugs.gnu.org/30066> and noticeably improves
performance when using GUIX_DAEMON_SOCKET=ssh://HOST (the redirect code
was transferring data to guix-daemon one byte at a time!).

* guix/ssh.scm (remote-daemon-channel)[redirect]: Define 'read!' and use
it instead of 'get-bytevector-some'.
2018-01-11 00:00:02 +01:00
Ludovic Courtès 896fec476f
ssh: Improve error reporting when retrieving files.
'guix copy --from' now reports messages much more useful than "failed to
retrieve files".

* guix/ssh.scm (store-export-channel)[export]: Wrap 'use-modules' in
'catch' and 'with-store' in 'guard'.  Check for invalid items.  Write a
status sexp on stdout.
(raise-error): New macro.
(retrieve-files): Read the initial status sexp and report errors
accordingly.
2018-01-07 23:46:39 +01:00
Ludovic Courtès 52d174d6d1
ssh: Use (guix i18n).
* guix/ssh.scm: Use (guix i18n) instead of (guix ui).
2017-11-19 23:09:01 +01:00
Ludovic Courtès de9d8f0e29
ssh: Improve error reporting when 'send-files' fails.
Fixes <http://bugs.gnu.org/26972>.

* guix/ssh.scm (store-import-channel)[import]: Add 'consume-input'
procedure.  Wrap body in 'catch' and 'guard'.  Use 'open-remote-pipe'
with OPEN_BOTH instead of 'open-remote-output-pipe'.
(send-files): After the 'channel-send-eof' call, do (read port).
Interpret the result sexp and raise an error condition if needed.
2017-06-04 23:00:32 +02:00
Ludovic Courtès 69daee23af
ui: Rename '_' to 'G_'.
This avoids collisions with '_' when the latter is used as a 'match'
pattern for instance.  See
<https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>.

* guix/ui.scm: Rename '_' to 'G_'.
* po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly.
* build-aux/compile-all.scm (warnings): Remove 'format'.
* gnu/packages.scm,
gnu/services.scm,
gnu/services/shepherd.scm,
gnu/system.scm,
gnu/system/shadow.scm,
guix/gnupg.scm,
guix/http-client.scm,
guix/import/cpan.scm,
guix/import/elpa.scm,
guix/import/pypi.scm,
guix/nar.scm,
guix/scripts.scm,
guix/scripts/archive.scm,
guix/scripts/authenticate.scm,
guix/scripts/build.scm,
guix/scripts/challenge.scm,
guix/scripts/container.scm,
guix/scripts/container/exec.scm,
guix/scripts/copy.scm,
guix/scripts/download.scm,
guix/scripts/edit.scm,
guix/scripts/environment.scm,
guix/scripts/gc.scm,
guix/scripts/graph.scm,
guix/scripts/hash.scm,
guix/scripts/import.scm,
guix/scripts/import/cpan.scm,
guix/scripts/import/cran.scm,
guix/scripts/import/crate.scm,
guix/scripts/import/elpa.scm,
guix/scripts/import/gem.scm,
guix/scripts/import/gnu.scm,
guix/scripts/import/hackage.scm,
guix/scripts/import/nix.scm,
guix/scripts/import/pypi.scm,
guix/scripts/import/stackage.scm,
guix/scripts/lint.scm,
guix/scripts/offload.scm,
guix/scripts/pack.scm,
guix/scripts/package.scm,
guix/scripts/perform-download.scm,
guix/scripts/publish.scm,
guix/scripts/pull.scm,
guix/scripts/refresh.scm,
guix/scripts/size.scm,
guix/scripts/substitute.scm,
guix/scripts/system.scm,
guix/ssh.scm,
guix/upstream.scm: Use 'G_' instead of '_'.  Most of this change was
obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
2017-05-03 16:16:17 +02:00
Ludovic Courtès e537833726
ssh: Decompose 'connect-to-remote-daemon'.
* guix/ssh.scm (remote-daemon-channel): New procedure.
(connect-to-remote-daemon): Implement in terms of it.
2017-04-21 19:17:36 +02:00
Ludovic Courtès 615c5298f7
ssh: Move 'open-ssh-session' to (guix ssh).
* guix/scripts/copy.scm (%compression, open-ssh-session): Move to...
* guix/ssh.scm: ... here.  Use '&message' conditions instead of calling
'leave'.
2017-04-21 19:17:34 +02:00
Ludovic Courtès 13164a2102
ssh: 'retrieve-files' detects remote export failures.
* guix/ssh.scm (retrieve-files): Call 'lookahead-u8' and raise a
'&message' condition when it returns EOF.
2016-12-31 18:35:29 +01:00
Ludovic Courtès 23973e4fbf
ssh: 'send-files' returns the list of items sent.
* guix/ssh.scm (send-files): Return MISSING.
2016-12-31 18:35:29 +01:00
Ludovic Courtès e9629e8221
ssh: Allow transfers of complete closures.
* guix/ssh.scm (store-export-channel, send-files)
(file-retrieval-port, retrieve-files): Add #:recursive? parameter and
honor it.
2016-12-31 18:35:29 +01:00
Ludovic Courtès 987a29ba43
Add (guix ssh) module.
* guix/scripts/offload.scm (connect-to-remote-daemon)
(store-import-channel, store-export-channel, send-files)
(retrieve-files): Move to (guix ssh).
(nonce): Add optional 'name' parameter and use it.
(retrieve-files*): New procedure.
(transfer-and-offload): Use it instead of 'retrieve-files', and add
first parameter to 'send-files'.
(assert-node-can-import): Likewise.
(assert-node-can-export): Use 'retrieve-files' instead of
'store-export-channel'.
* guix/ssh.scm: New file.
* configure.ac: Use 'GUIX_CHECK_GUILE_SSH' and define 'HAVE_GUILE_SSH'
Automake conditional.
* Makefile.am (MODULES) [HAVE_GUILE_SSH]: Add guix/ssh.scm.
2016-12-31 01:44:04 +01:00