The v1.7 release brings two new features:
- a
cachix import mycache s3://mybucketcommand to ingest the contents of an existing S3-based binary cache into Cachix. - a standalone daemon mode that opens up a bevy of workflow improvements. Requested many times before.
External cache ingestion
The cachix import command is designed to facilitate the migration of static binary caches to Cachix.
This feature is critical for users seeking to leverage Cachix’s hosted binary cache services while transitioning from their existing setups.
A complete guide can be found in the Cachix Documentation: Importing from S3.
Daemon mode
We’re introducing a new daemon mode to the Cachix CLI. The daemon runs as a standalone process and listens to push requests over a unix socket.
$ cachix daemon run mycache
[2024-01-09 15:43:07][Info] Starting Cachix Daemon
[2024-01-09 15:43:07][Info] Configuration:
PID: 2544950
Socket: /run/user/1000/cachix/cachix-daemon.sock
Workers: 8
Cache name: cachix
Cache URI: https://cachix.cachix.org
Cache public keys: ["cachix.cachix.org-1:eWNHQldwUO7G2VkjpnjDbWwy4KQ/HNxht7H4SSoMckM="]
Cache is public: True
Compression: ZSTD
Clients can asynchronously queue up store paths to be pushed and the daemon will take care of the details.
$ cachix daemon push /nix/store/sgnd3fb22m769gqi577b6cdwwm5s76gk-hello-2.12.1
[2024-01-09 15:43:35][Info] Pushing /nix/store/cv8mfy5wdfwfw4iwhdlkl4ddy8apl667-glibc-2.38-27
[2024-01-09 15:43:35][Info] Pushing /nix/store/by0gd1lfh3vsnk9vfipflcbz2f83bv3r-libunistring-1.1
[2024-01-09 15:43:35][Info] Pushing /nix/store/sgnd3fb22m769gqi577b6cdwwm5s76gk-hello-2.12.1
[2024-01-09 15:43:35][Info] Pushing /nix/store/9cx8xhdxirj0h7yap2vgjxk65360ngg7-libidn2-2.3.4
[2024-01-09 15:43:35][Info] Pushing /nix/store/di87rcanqjsr36jlhfmrh845y4s140iz-xgcc-12.3.0-libgcc
[2024-01-09 15:43:38][Info] Pushed /nix/store/sgnd3fb22m769gqi577b6cdwwm5s76gk-hello-2.12.1
[2024-01-09 15:43:39][Info] Pushed /nix/store/9cx8xhdxirj0h7yap2vgjxk65360ngg7-libidn2-2.3.4
[2024-01-09 15:43:39][Info] Pushed /nix/store/by0gd1lfh3vsnk9vfipflcbz2f83bv3r-libunistring-1.1
[2024-01-09 15:43:39][Info] Pushed /nix/store/di87rcanqjsr36jlhfmrh845y4s140iz-xgcc-12.3.0-libgcc
[2024-01-09 15:43:39][Info] Pushed /nix/store/cv8mfy5wdfwfw4iwhdlkl4ddy8apl667-glibc-2.38-27
So what are the use-cases for this feature?
One approach is to leverage a lesser known Nix feature called the post-build-hook.
This hook lets us register a program to run after every Nix build.
For example, you could wrap the daemon in a systemd service, register a global post-build-hook, and enable automatic caching of every Nix build on your machine.
This general design is quite flexible and enables us to introduce a number of improvements to some common Nix workflows.
A more granular watch-exec
Take, for example, the existing watch-exec command.
It lets you watch the Nix store for new store paths to cache while simulaneously executing some other command.
$ cachix watch-exec cachix -- nix build
It’s convenient to use, but there’s a downside to watching the entire store: it could inadvertently push more store paths than initially intended in a multi-user Nix install.
In v1.7, the new daemonized watch-exec can now leverage the post-build-hook to hook into the Nix build pipeline.
Cachix will push exactly the store paths built. Zero additional configuration required.
Faster CI runs
Our Cachix GitHub action is also getting a daemon-flavoured improvement.
Starting with cachix-action@v14, store paths will be pushed concurrently during CI runs. This drastically reduces the time spent in the post-run step as most paths will have been pushed by the end of the run. We’ve been using this internally for several months now and our post-run times for large rebuilds have gone down from tens of minutes to mere seconds.
We’ve also been running self-hosted GitHub runners for a few customers and this change allows for multi-cache setup.
The trust tradeoff
There is, however, a caveat to the above use-cases.
In multi-user Nix installs, the post-build-hook can only be set by a trusted user. This introduces a tradeoff to be made between convenience and security. A trusted user executing untrusted Nix builds could lead to priveledge escalation.
We plan to look into extending the Nix post-build-hook to execute on the client side, sidestepping the tradeoff.