Commit Graph

56 Commits

Author SHA1 Message Date
Ludovic Courtès bb11825f35
publish: Work around Guile 2.2.5 (web server) bug.
* guix/scripts/publish.scm: Replace (@@ (web http) read-header-line) on
Guile 2.2.5.
2019-06-24 17:50:08 +02:00
Ludovic Courtès b8fa86adfc
publish: '--compression' can be repeated.
This allows 'guix publish' to compress and advertise multiple
compression methods from which users can choose.

* guix/scripts/publish.scm (actual-compression): Rename to...
(actual-compressions): ... this.  Expect REQUESTED to be a list, and
always return a list.
(%default-options): Remove 'compression.
(store-item->recutils): New procedure.
(narinfo-string): Change #:compression to #:compressions (plural).
Adjust accordingly.
(render-narinfo, render-narinfo/cached): Likewise.
(bake-narinfo+nar): Change #:compression to #:compressions.
[compressed-nar-size]: New procedure.
Call 'compress-nar' for each item returned by 'actual-compressions'.
Create a narinfo for each compression.
(effective-compression): New procedure.
(make-request-handler): Change #:compression to #:compressions.
Use 'effective-compression' to determine the applicable compression.
(guix-publish): Adjust handling of '--compression'.
Print a message for each compression that is enabled.
* tests/publish.scm ("/*.narinfo"): Adjust to new narinfo field
ordering.
("/*.narinfo with properly encoded '+' sign"): Likewise.
("/*.narinfo with lzip + gzip"): New test.
("with cache, lzip + gzip"): New test.
* doc/guix.texi (Invoking guix publish): Document it.
2019-06-02 22:01:57 +02:00
Ludovic Courtès 73bddab545
publish: Factorize 'compress-nar'.
* guix/scripts/publish.scm (compress-nar): New procedure.
(bake-narinfo+nar): Use it.
2019-05-29 23:16:49 +02:00
Ludovic Courtès e84e036943
publish: Remove outdated comment.
* guix/scripts/publish.scm (make-request-handler): Remove outdated TODO.
2019-05-29 23:16:49 +02:00
Ludovic Courtès 35d1354fe8
publish: Display the compression method and level in use.
* guix/scripts/publish.scm (guix-publish): Use 'info' instead of
'format' for the initial message.  When COMPRESSION is true, display the
method and level in use.
2019-05-27 22:47:24 +02:00
Ludovic Courtès 66229b04ae
publish: Add support for lzip.
* guix/scripts/publish.scm (show-help, %options): Support '-C METHOD'
and '-C METHOD:LEVEL'.
(default-compression): New procedure.
(bake-narinfo+nar): Add lzip.
(nar-response-port): Likewise.
(string->compression-type): New procedure.
(make-request-handler): Generalize /nar/gzip handler to handle /nar/lzip
as well.
* tests/publish.scm ("/nar/lzip/*"): New test.
("/*.narinfo with lzip compression"): New test.
* doc/guix.texi (Invoking guix publish): Document it.
(Requirements): Mention lzlib.
2019-05-27 22:47:24 +02:00
Ludovic Courtès 493375cdb2
publish: Maintain a hash-part-to-store-item mapping in cache.
Fixes <https://bugs.gnu.org/33897>.

* guix/scripts/publish.scm (hash-part-mapping-cache-file)
(hash-part->path*): New procedures.
* guix/scripts/publish.scm (render-narinfo/cached)[delete-entry]: Delete
the 'hash-part-mapping-cache-file'.
Use 'hash-part->path*' instead of 'hash-part->path'.
* tests/publish.scm ("with cache, vanishing item"): New test.
2019-05-26 01:34:17 +02:00
Ludovic Courtès 9b9de08477
publish: Add a 'Cache-Control' header on /nar responses.
Fixes <https://bugs.gnu.org/33721>.
Reported by Chris Marusich <cmmarusich@gmail.com>.

* guix/scripts/publish.scm (render-nar/cached): Add #:ttl and honor it.
(make-request-handler): Pass #:ttl to 'render-nar/cached'.
* tests/publish.scm ("with cache, uncompressed"): Pass "--ttl=42h" to
'guix publish'.  Check 'Cache-Control' on narinfo response and on nar
response.
2018-12-17 23:33:42 +01:00
Ludovic Courtès ca71942445
Switch to Guile-Gcrypt.
This removes (guix hash) and (guix pk-crypto), which now live as part of
Guile-Gcrypt (version 0.1.0.)

* guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm,
tests/hash.scm, tests/pk-crypto.scm: Remove.
* configure.ac: Test for Guile-Gcrypt.  Remove LIBGCRYPT and
LIBGCRYPT_LIBDIR assignments.
* m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove.
* README: Add Guile-Gcrypt to the dependencies; move libgcrypt as
"required unless --disable-daemon".
* doc/guix.texi (Requirements): Likewise.
* gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm,
guix/git.scm, guix/http-client.scm, guix/import/cpan.scm,
guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm,
guix/import/gnu.scm, guix/import/hackage.scm,
guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm,
guix/pki.scm, guix/scripts/archive.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/scripts/pack.scm,
guix/scripts/publish.scm, guix/scripts/refresh.scm,
guix/scripts/substitute.scm, guix/store.scm,
guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm,
tests/builders.scm, tests/challenge.scm, tests/cpan.scm,
tests/crate.scm, tests/derivations.scm, tests/gem.scm,
tests/nar.scm, tests/opam.scm, tests/pki.scm,
tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm,
tests/store.scm, tests/substitute.scm: Adjust imports.
* gnu/system/vm.scm: Likewise.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this.  Add GUILE-GCRYPT.
(expression->derivation-in-linux-vm)[config]: Remove.
(iso9660-image)[config]: Remove.
(qemu-image)[config]: Remove.
(system-docker-image)[config]: Remove.
* guix/scripts/pack.scm: Adjust imports.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this.  Add GUILE-GCRYPT.
(self-contained-tarball)[build]: Call 'make-config.scm' without
 #:libgcrypt argument.
(squashfs-image)[libgcrypt]: Remove.
[build]: Call 'make-config.scm' without #:libgcrypt.
(docker-image)[config, json]: Remove.
[build]: Add GUILE-GCRYPT to the extensions  Remove (guix config) from
the imported modules.
* guix/self.scm (specification->package): Remove "libgcrypt", add
"guile-gcrypt".
(compiled-guix): Remove #:libgcrypt.
[guile-gcrypt]: New variable.
[dependencies]: Add it.
[*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call.
Add #:extensions.
[*config*]: Remove #:libgcrypt from 'make-config.scm' call.
(%dependency-variables): Remove %libgcrypt.
(make-config.scm): Remove #:libgcrypt.
* build-aux/build-self.scm (guile-gcrypt): New variable.
(make-config.scm): Remove #:libgcrypt.
(build-program)[fake-gcrypt-hash]: New variable.
Add (gcrypt hash) to the imported modules.  Adjust load path
assignments.
* gnu/packages/package-management.scm (guix)[propagated-inputs]: Add
GUILE-GCRYPT.
[arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search
path.
2018-09-04 17:25:11 +02:00
Ludovic Courtès 33988f9b58
publish: Restore gzip compression in cache-less mode.
Fixes <https://bugs.gnu.org/30184>.
Regression introduced in 297e04d660.
Reported by Christopher Baines <mail@cbaines.net>.

* guix/scripts/publish.scm (nar-response-port): Add 'compression'
parameter and honor it.
(http-write): Get 'x-nar-compression' from the initial RESPONSE.
2018-01-22 11:15:50 +01:00
Ludovic Courtès 297e04d660
publish: Remove "Guix-Nar-Compression" header from responses.
This was harmless but non-compliant and unnecessary.

* guix/scripts/publish.scm <top level>: Rename "Guix-Nar-Compression" to
"X-Nar-Compression" as should have always been.
(render-nar, nar-response-port): Adjust accordingly.
(strip-headers): New procedure.
(sans-content-length, with-content-length): Use it.
2018-01-09 23:39:02 +01:00
Ludovic Courtès c04ffadbed
publish: Publish build logs.
* guix/scripts/publish.scm (render-log-file): New procedure.
(make-request-handler): Add "log" case.
* tests/publish.scm ("/log/NAME")
("/log/NAME not found"): New tests.
* doc/guix.texi (Invoking guix publish): Document /log URLs.
2018-01-07 23:51:46 +01:00
Ludovic Courtès 152b7beeac
publish: Use 'x-raw-file' internal response header.
This adjusts the workaround for <http://bugs.gnu.org/21093> so that it's
not limited to a single content-type.

* guix/scripts/publish.scm (render-nar/cached): Add the 'x-raw-file'
header on the response.
(render-content-addressed-file): Likewise.
(with-content-length): Remove the 'x-raw-file' header.
(http-write): Instead of dispatching on 'application/octet-stream',
check whether 'x-raw-file' is set to determine whether to spawn a
thread.
2018-01-07 23:47:22 +01:00
Ludovic Courtès f396611776
publish: Save bandwidth on narinfo 404s.
This saves 18 bytes on each 404 narinfo response.

* guix/scripts/publish.scm (render-narinfo): Pass #:phrase to
'not-found'.
(render-narinfo/cached): Likewise.
2018-01-06 14:43:45 +01:00
Ludovic Courtès e5788ebbe1
publish: Cache uncompressed nars as well.
Fixes <https://bugs.gnu.org/28664>.
Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>.

* guix/scripts/publish.scm (bake-narinfo+nar): When COMPRESSION is
'none, write NAR instead of doing nothing.
(make-request-handler): Use 'render-nar/cached' for /nar URLs with no
compression.
* tests/publish.scm ("with cache", "with cache, uncompressed"): Adjust
expected result accordingly.
2017-10-05 23:05:06 +02:00
Ludovic Courtès c95644f017
publish: Make the cache eviction policy less aggressive.
Suggested by Mark H Weaver <mhw@netris.org>.

* guix/scripts/publish.scm (nar-expiration-time): New procedure.
(render-narinfo/cached): Use it as the #:entry-expiration passed to
'maybe-remove-expired-cache-entries'.
2017-07-21 17:03:25 +02:00
Ludovic Courtès deac674ab4
publish: Avoid 'valid-path?' RPC for non-existent items.
* guix/scripts/publish.scm (render-narinfo/cached): Call 'file-exists?'
before calling 'valid-path?'.  This makes the 404 path slightly faster.
2017-07-21 17:03:25 +02:00
Ludovic Courtès 33463986ba
publish: Remove 'regexp-exec' call from the hot path.
* guix/scripts/publish.scm (extract-narinfo-hash): Rewrite without
resorting to regexps.
2017-07-21 17:03:24 +02:00
Ludovic Courtès 015f17e8b9
derivations: Introduce 'read-derivation-from-file'.
This avoids the open/fstat/close syscalls upon a cache hit that we had
with the previous idiom:

  (call-with-input-file file read-derivation)

where caching happened in 'read-derivation' itself.

* guix/derivations.scm (%read-derivation): Rename to...
(read-derivation): ... this.
(read-derivation-from-file): New procedure.
(derivation-prerequisites, substitution-oracle)
(derivation-prerequisites-to-build):
(derivation-path->output-path, derivation-path->output-paths):
(derivation-path->base16-hash, map-derivation): Use
'read-derivation-from-file' instead of (call-with-input-file …
read-derivation).
* guix/grafts.scm (item->deriver): Likewise.
* guix/scripts/build.scm (log-url, options->things-to-build): Likewise.
* guix/scripts/graph.scm (file->derivation): Remove.
(derivation-dependencies, %derivation-node-type): Use
'read-derivation-from-file' instead.
* guix/scripts/offload.scm (guix-offload): Likewise.
* guix/scripts/perform-download.scm (guix-perform-download): Likewise.
* guix/scripts/publish.scm (load-derivation): Remove.
(narinfo-string): Use 'read-derivation-from-file'.
2017-06-12 17:53:51 +02:00
Ludovic Courtès 8902d0f267
scripts: Set thread names.
This allows 'guix publish' threads as well as 'guix substitute' and
'guix offload' processes to be properly labeled in 'top', 'pstree', etc.

* guix/workers.scm (worker-thunk): Add #:thread-name parameter and honor it.
(make-pool): Likewise.
* guix/scripts/publish.scm (http-write): Add calls to 'set-thread-name'
in bodies of 'call-with-new-thread'.
(guix-publish): Call 'set-thread-name'.   Pass #:thread-name to 'make-pool'.
* guix/scripts/offload.scm (guix-offload): Call 'set-thread-name'.
* guix/scripts/substitute.scm (guix-substitute): Likewise.
2017-05-28 23:13:39 +02:00
Ludovic Courtès 737397355c
publish: Do not bake the same item several times.
Before this patch, there was a possibility that we'd queue several
baking tasks for the same item.  The first one would bake the item but
subsequent tasks wouldn't check whether the item had been baked in the
meantime and would re-bake it.

* guix/scripts/publish.scm (render-narinfo/cached): Don't call
'bake-narinfo+nar' when (file-exists? cached).
2017-05-24 11:17:42 +02:00
Ludovic Courtès ffa5e0a6d2
publish: Fix narinfo rendering for already-compressed items.
Fixes <http://bugs.gnu.org/26975>.
Reported by Mark H Weaver <mhw@netris.org>.

* guix/scripts/publish.scm (bake-narinfo+nar): Pass #f as the 2nd
argument to 'stat' and properly handle #f.
* tests/publish.scm (wait-for-file): New procedure.
("with cache"): Remove 'wait-for-file' procedure.
("with cache, uncompressed"): New test.
2017-05-18 21:35:35 +02:00
Ludovic Courtès 24b21720f7
publish: Advertise a short TTL for "baking" 404s.
* guix/scripts/publish.scm (not-found): Add #:phrase and #:ttl
parameters and honor them.
* tests/publish.scm ("with cache"): Check the 'cache-control' header on
of the 404 response.
2017-05-11 18:05:42 +02:00
Ludovic Courtès 69daee23af
ui: Rename '_' to 'G_'.
This avoids collisions with '_' when the latter is used as a 'match'
pattern for instance.  See
<https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>.

* guix/ui.scm: Rename '_' to 'G_'.
* po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly.
* build-aux/compile-all.scm (warnings): Remove 'format'.
* gnu/packages.scm,
gnu/services.scm,
gnu/services/shepherd.scm,
gnu/system.scm,
gnu/system/shadow.scm,
guix/gnupg.scm,
guix/http-client.scm,
guix/import/cpan.scm,
guix/import/elpa.scm,
guix/import/pypi.scm,
guix/nar.scm,
guix/scripts.scm,
guix/scripts/archive.scm,
guix/scripts/authenticate.scm,
guix/scripts/build.scm,
guix/scripts/challenge.scm,
guix/scripts/container.scm,
guix/scripts/container/exec.scm,
guix/scripts/copy.scm,
guix/scripts/download.scm,
guix/scripts/edit.scm,
guix/scripts/environment.scm,
guix/scripts/gc.scm,
guix/scripts/graph.scm,
guix/scripts/hash.scm,
guix/scripts/import.scm,
guix/scripts/import/cpan.scm,
guix/scripts/import/cran.scm,
guix/scripts/import/crate.scm,
guix/scripts/import/elpa.scm,
guix/scripts/import/gem.scm,
guix/scripts/import/gnu.scm,
guix/scripts/import/hackage.scm,
guix/scripts/import/nix.scm,
guix/scripts/import/pypi.scm,
guix/scripts/import/stackage.scm,
guix/scripts/lint.scm,
guix/scripts/offload.scm,
guix/scripts/pack.scm,
guix/scripts/package.scm,
guix/scripts/perform-download.scm,
guix/scripts/publish.scm,
guix/scripts/pull.scm,
guix/scripts/refresh.scm,
guix/scripts/size.scm,
guix/scripts/substitute.scm,
guix/scripts/system.scm,
guix/ssh.scm,
guix/upstream.scm: Use 'G_' instead of '_'.  Most of this change was
obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
2017-05-03 16:16:17 +02:00
Ludovic Courtès e723178a10
publish: Use a larger zlib buffer for compression with '--cache'.
* guix/scripts/publish.scm (bake-narinfo+nar): Pass #:buffer-size to
'call-with-gzip-output-port'.
2017-05-02 12:32:51 +02:00
Ludovic Courtès dff3189c7d
publish: Produce a "FileSize" narinfo field when possible.
* guix/scripts/publish.scm (narinfo-string): Add #:file-size parameter.
Produce a "FileSize" field when COMPRESSION is eq? to '%no-compression'
or when FILE-SIZE is true.
(bake-narinfo+nar): Pass #:file-size.
* tests/publish.scm ("/*.narinfo")
("/*.narinfo with properly encoded '+' sign")
("with cache"): Check for "FileSize".
2017-05-01 17:28:20 +02:00
Ludovic Courtès e1bbc0e38d
publish: Add a handler for / and /index.html.
Suggested by Quiliro <quiliro@riseup.net>
in <https://bugs.gnu.org/26567>.

* guix/scripts/publish.scm (render-home-page): New procedure.
(make-request-handler): Handle it.
2017-04-19 23:43:51 +02:00
Ludovic Courtès d72b42064b
publish: Remove expired cache entries when '--ttl' is used.
* guix/scripts/publish.scm (narinfo-files): New procedure.
(render-narinfo/cached)[delete-file]: New procedure.  Add call to
'maybe-remove-expired-cache-entries'.
* doc/guix.texi (Invoking guix publish): Document the interation between
--cache and --ttl.
2017-04-18 23:19:30 +02:00
Ludovic Courtès 00753f7038
publish: Add '--cache' and '--workers'.
Fixes <http://bugs.gnu.org/26201>.
Reported by <dian_cecht@zoho.com>.

These options allow nars to be "baked" off-line and cached instead of
being compressed on the fly.  As a side-effect, this allows us to
provide a 'Content-Length' header for nars.

* guix/scripts/publish.scm (show-help, %options): Add '--cache' and
'--workers'.
(%default-options): Add 'workers'.
(nar-cache-file, narinfo-cache-file, run-single-baker): New procedures.
(single-baker): New macro.
(render-narinfo/cached, bake-narinfo+nar)
(render-nar/cached): New procedures.
(make-request-handler): Add #:cache and #:pool parameters and honor
them.
(run-publish-server): Likewise.
(guix-publish): Honor '--cache' and '--workers'.
* tests/publish.scm ("with cache"): New test.
* doc/guix.texi (Invoking guix publish): Document it.
2017-04-18 23:18:41 +02:00
Ludovic Courtès 339a79fd6a
publish: Use 'sendfile' when possible.
* guix/scripts/publish.scm (http-write): In the
'application/octet-stream' case, use 'sendfile' when OUTPUT is a file
port.
2017-04-18 23:17:18 +02:00
Ludovic Courtès 905ae5276d
publish: Introduce 'actual-compression'.
* guix/scripts/publish.scm (actual-compression): New procedure.
(narinfo-string): Use it.
2017-04-18 23:17:18 +02:00
Ludovic Courtès 4bb5e0aeb3
publish: Add '--nar-path'.
* guix/scripts/publish.scm (show-help, %options): Add '--nar-path'.
(%default-options): Add 'nar-path'.
(guix-publish): Honor it.
2017-03-22 14:05:59 +01:00
Ludovic Courtès cdd7a7d210
publish: Make the nar URL prefix a parameter.
* guix/scripts/publish.scm (narinfo-string): Add #:nar-path and honor it.
(render-narinfo): Likewise.
(make-request-handler): Likewise.
(run-publish-server): Likewise.
* tests/publish.scm ("custom nar path"): New test.
2017-03-22 14:05:59 +01:00
Ludovic Courtès 46f58390cb
publish: Add '--public-key' and '--private-key'.
* guix/scripts/publish.scm (show-help, %options): Add --public-key and
--private-key.
* doc/guix.texi (Invoking guix publish): Document it.
2017-03-22 14:05:59 +01:00
Ludovic Courtès ab2a74e4db
publish: The public and private keys are now SRFI-39 parameters.
* guix/scripts/publish.scm (%default-options): Add 'public-key-file' and
'private-key-file'.
(lazy-read-file-sexp): Remove.
(%private-key, %public-key): Turn into SRFI-39 parameters.
(signed-string, render-narinfo): Adjust accordingly.
(guix-publish): Honor 'public-key-file' and 'private-key-file' from
OPTS.  Use 'parameterize'.
* guix/pk-crypto.scm (read-file-sexp): New procedure.
* tests/publish.scm: Initialize '%public-key' and '%private-key'.
2017-03-22 14:05:59 +01:00
Ludovic Courtès 42d07286f4
publish: Factorize 'content-length' addition.
* guix/scripts/publish.scm (with-content-length): New procedure.
(http-write) <application/octet-stream>: Use it.
2016-12-04 22:32:26 +01:00
Ludovic Courtès 2535635f18
Use (ice-9 binary-ports) instead of (rnrs io ports).
This reduces the closure of (guix ui) from 123 to 106 modules.

* guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io
ports).
(map-derivation)[substitute-file]: Use 'read-string' instead of
'get-string-all'.
* guix/ftp-client.scm: Likewise.
* guix/hash.scm: Likewise.
* guix/http-client.scm: Likewise.
* guix/pki.scm (ensure-acl, current-acl): Likewise.
* guix/scripts/archive.scm (authorize-key)[read-key]: Likewise.
* guix/scripts/authenticate.scm (read-canonical-sexp)
(read-hash-data): Likewise.
* guix/scripts/download.scm: Likewise.
* guix/scripts/offload.scm (register-gc-root, remove-gc-roots)
(send-files): Likewise.
* guix/scripts/publish.scm (lazy-read-file-sexp): Likewise.
* guix/scripts/refresh.scm: Likewise.
* guix/scripts/substitute.scm (check-acl-initialized): Likewise.
* guix/serialization.scm (read-maybe-utf8-string): Likewise.
* guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of
'flush-output-port'.
* guix/store.scm (process-stderr): Likewise.
* guix/tests.scm: Likewise.
* guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports)
for 'make-custom-binary-input-port'.
2016-10-19 15:54:10 +02:00
Ludovic Courtès 089b167812
publish: Do not compress already-compressed files.
* guix/scripts/publish.scm (narinfo-string): Force %NO-COMPRESSION when
STORE-PATH matches 'compressed-file?'.
* guix/utils.scm (compressed-file?): New procedure.
* tests/publish.scm ("/*.narinfo for a compressed file"): New test.
2016-08-01 17:58:56 +02:00
Ludovic Courtès ff6638d112
publish: Handle '/file' URLs, for content-addressed files.
* guix/scripts/publish.scm (render-content-addressed-file): New procedure.
(http-write): Add 'application/octet-stream' case.
(make-request-handler): Add /file/NAME/sha256/HASH URLs.
* tests/publish.scm ("/file/NAME/sha256/HASH")
("/file/NAME/sha256/INVALID-NIX-BASE32-STRING")
("/file/NAME/sha256/INVALID-HASH"): New tests.
* doc/guix.texi (Invoking guix publish): Mention the /file URLs.
2016-07-20 16:54:31 +02:00
Ludovic Courtès 053fa76ae7
publish: Keep compression disabled when zlib is missing.
Reported by David Thompson.

* guix/scripts/publish.scm (%options)[--compression]: Warn if LEVEL > 0
and zlib is missing, and return RESULT.
2016-07-19 16:47:19 +02:00
Ludovic Courtès 4a1fc562ae
publish: Add '--compression'.
* guix/scripts/publish.scm (show-help, %options): Add '--compression'.
(<compression>): New record type.
(%no-compression, %default-gzip-compression): New variables.
(%default-options): Add 'compression' key.
(narinfo-string): Add #:compression parameter and honor it.
(render-narinfo): Likewise.
(render-nar): Likewise.
<top level>: Add call to 'declare-header!'.
(swallow-zlib-error): New macro.
(nar-response-port): New procedure.
(http-write): Add call to 'force-output'.  Use 'nar-response-port'
instead of 'response-port'.  Use 'swallow-zlib-error'.
(make-request-handler): Add #:compression parameter and honor it.  Add
"nar/gzip" URL handler.
(run-publish-server): Add #:compression parameter and honor it.
(guix-publish): Honor --compression.
* tests/publish.scm (http-get-port, wait-until-ready): New procedures.
<top level>: Run main server with "-C0".  Call 'wait-until-ready'.
("/nar/gzip/*", "/*.narinfo with compression"): New tests.
* doc/guix.texi (Invoking guix publish): Document it.
2016-07-19 00:07:12 +02:00
Ludovic Courtès e4c7a5f7c8
publish: Add '--ttl'.
* guix/scripts/publish.scm (show-help, %options): Add --ttl.
(render-narinfo): Add #:ttl and honor it.
(make-request-handler): Add #:narinfo-ttl and honor it.
(run-publish-server): Likewise.
(guix-publish): Honor --ttl, pass it to 'run-publish-server'.
2016-06-09 23:34:56 +02:00
Ludovic Courtès 93961f0298
publish: Encore URIs that appear in narinfos.
Fixes <http://bugs.gnu.org/21888>.
Reported by iyzsong@member.fsf.org (宋文武).

* guix/scripts/publish.scm (narinfo-string): Use
'encode-and-join-uri-path' instead of 'string-append' to compute URL.
* tests/publish.scm ("/*.narinfo with properly encoded '+' sign"):
("/nar/ with properly encoded '+' sign"): New tests.
2016-06-08 19:07:58 +02:00
Ludovic Courtès 22572d56cb store: 'path-info-deriver' is #f when there is no deriver.
* guix/store.scm (read-path-info): Use #f when we get the empty string
for DERIVER.
* guix/scripts/publish.scm (narinfo-string): Adjust accordingly.
* tests/store.scm ("path-info-deriver"): New test.
2016-02-26 23:35:29 +01:00
Ludovic Courtès 0043558082 publish: Do not publish nars for invalid store items.
Before that, /nar requests could succeed if the requested store item
exists but is invalid (although such requests were unlikely because the
corresponding narinfo request would have failed.)

* guix/scripts/publish.scm (render-nar): Add 'store' parameter.  Use
'valid-path?' instead of 'file-exists?'.
(make-request-handler): Adjust 'render-nar' call accordingly.
* tests/publish.scm ("/nar/invalid"): New test.
2016-02-25 12:44:41 +01:00
Ludovic Courtès 4d459d8734 publish: Move 'query-path-info' call where it belongs.
* guix/scripts/publish.scm (narinfo-string): Remove 'path-info'
parameter and add 'store' parameter.  Call 'query-path-info'.
(render-narinfo): Adjust accordingly.
2016-02-24 13:11:50 +01:00
Ludovic Courtès cf4e7083ed publish: Fix file descriptor leak.
A client closing the connection while reading from a /nar URL would
leave an open file descriptor in the server.  This patch fixes it.

* guix/scripts/publish.scm (swallow-EPIPE): New macro.
  (http-write): Use it around 'write-file' call.
2015-10-06 23:51:04 +02:00
Alex Kost 88981dd3e2 Add (guix scripts).
* guix/ui.scm: Add missing copyright lines.
  (args-fold*, environment-build-options, %default-argument-handler,
  parse-command-line): Move to ...
* guix/scripts.scm: ...here.  New file.
* guix/scripts/archive.scm: Use it.
* guix/scripts/build.scm: Likewise.
* guix/scripts/download.scm: Likewise.
* guix/scripts/edit.scm: Likewise.
* guix/scripts/environment.scm: Likewise.
* guix/scripts/gc.scm: Likewise.
* guix/scripts/graph.scm: Likewise.
* guix/scripts/hash.scm: Likewise.
* guix/scripts/import/cpan.scm: Likewise.
* guix/scripts/import/cran.scm: Likewise.
* guix/scripts/import/elpa.scm: Likewise.
* guix/scripts/import/gem.scm: Likewise.
* guix/scripts/import/gnu.scm: Likewise.
* guix/scripts/import/hackage.scm: Likewise.
* guix/scripts/import/nix.scm: Likewise.
* guix/scripts/import/pypi.scm: Likewise.
* guix/scripts/lint.scm: Likewise.
* guix/scripts/package.scm: Likewise.
* guix/scripts/publish.scm: Likewise.
* guix/scripts/pull.scm: Likewise.
* guix/scripts/refresh.scm: Likewise.
* guix/scripts/size.scm: Likewise.
* guix/scripts/system.scm: Likewise.
* tests/ui.scm (with-environment-variable, "parse-command-line",
  "parse-command-line and --no options"): Move to ...
* tests/scripts.scm: ...here.  New file.
* Makefile.am (MODULES): Add guix/scripts.scm.
  (SCM_TESTS): Add tests/scripts.scm.
* po/guix/POTFILES.in: Add guix/scripts.scm.
2015-09-18 21:42:06 +03:00
Ludovic Courtès 9d2f48df02 publish: Gracefully handle the lack of a deriver.
* guix/scripts/publish.scm (narinfo-string): Catch 'system-error' around
  'load-derivation' call; return BASE-INFO upon ENOENT.  This allows us
  to return the narinfo even if DERIVER is missing.  Before that, the
  exception would be uncaught, leading to 500 Internal Error on the
  client side.
2015-09-04 00:13:05 +02:00
Ludovic Courtès 69b4ffcfbd Fix typos in translatable strings.
Reported by Anders Jonsson <anders.jonsson@norsjovallen.se>.

* gnu/packages/backup.scm, gnu/packages/databases.scm,
  gnu/packages/linux.scm, gnu/packages/perl.scm,
  gnu/packages/web.scm, guix/scripts/lint.scm,
  guix/scripts/publish.scm: Fix typos in translatable strings.
2015-07-23 09:42:23 +02:00