Blog

Back in 2020, we introduced upstream caches - a feature that avoids pushing store paths already present in cache.nixos.org. At the time, we mentioned:

In the future you are going to define upstream caches within Cachix so that the typical setup of a private and public cache does not duplicate the entries between the two.

That future is now.

What’s New

The simple boolean setting has been replaced with a configurable list of upstream caches. You can now add:

…

At around 2am last night, I got woken up by a PagerDuty call due to our database being unresponsive.

I immediately restarted the database and it was back up in a few minutes.

While waiting for the database to come back up, I noticed that our sign ups were being overwhelmed with spam.

The usual approach to this problem is to add a CAPTCHA. Since we’re already using a honeypot, it felt like a natural next step.

…

The v1.7 release brings two new features:

• a cachix import mycache s3://mybucket command to ingest the contents of an existing S3-based binary cache into Cachix.
• a standalone daemon mode that opens up a bevy of workflow improvements. Requested many times before.

External cache ingestion

The cachix import command is designed to facilitate the migration of static binary caches to Cachix. This feature is critical for users seeking to leverage Cachix’s hosted binary cache services while transitioning from their existing setups.

…

As we deal with networking and databases in our day-to-day work, Haskell exceptions frequently arise in our code.

We’ve been using the Haskell retry package for retrying exceptions so far, but it presents several issues:

• The API is cumbersome to remember, so we’ve always built a thin layer on top.
• It lacks built-in support for retrying HTTP exceptions, which requires careful selection of retryable instances and adherence to the retry-after header.
• To log the exception that occurred, one needs to write a logger as an exception handler.
• It’s not possible to reset the retry state when a long-lived task (such as a WebSocket connection) finally succeeds.

Heavily inspired by Hynek’s Stamina for Python, we’re releasing Stamina for Haskell:

…

We’ve recently reached a couple of milestones:

• It’s been 5 years since we first announced Cachix.
• We sponsored and participated in NixCon 2023.
• We’ve grown to over 6500 users and almost 5000 binary caches

And we’re happy to announce one more milestone: upgraded storage plans for everyone!

Upgraded plans

• Starter: 50 GB
• Standard: 250 GB
• Pro: 1500 GB

Previous plans

• Starter: 20 GB
• Basic: 100 GB
• Pro: 1000 GB

Existing users will be upgraded to the new plans at no extra cost. Enjoy the extra space! Additionally, organizations now enjoy a free plan of 5 GB, just like regular accounts, to deepen our support for Open Source development.

…

Binary caches are an indispensable ally when it comes to avoiding tedious rebuilds by distributing Nix binaries. However, caches can hoard obsolete store paths that aren’t necessary anymore, which occupy precious storage space. As the cache swells beyond its storage capacity, these old store paths eventually have to be evicted. Cachix takes care of this routinely by initiating a process known as garbage collection.

Previously, the decision of what to remove from the cache was not in the hands of the users. Store paths were cleared based on the LRU (Least Recently Used) principle. This could lead to inadvertent removal of significant files that were not accessed regularly.

…

Serving files is the bread and butter of Cachix. Fetching store paths from Cachix is fast — downloads are served directly from fast distributed cloud storage. Uploads, however, are a different story.

Each upload still had to go directly through one of our servers, making our ingest bandwidth a scarce and highly contested resource during peak times.

Many of our users rely on Cachix in their CI pipelines, where slower uploads lead to longer CI runs and a worse developer experience, which is why we’ve been busy reorganizing our internals to remove this bottleneck.

…

We’ve been pioneering and supporting the addition of ZSTD compression to Nix since 2018.

In November 2021, Nix 2.4 was finally released with ZSTD support.

As of today, the Cachix 1.1 client will compress any pushed binaries using zstd by default, which can be reversed back to XZ compression in the binary cache settings.

This will significantly speed up pushing, as it was previously capped at around ~3-5mbit/s due to XZ compression hitting CPU per-core limits.

…

I’m happy to announce the public beta for Cachix Deploy.

Cachix Deploy was built to remove the hassle around managing the life cycle of a Nix profile.

Cachix Deploy provides continuous deployments to a Nix profile (like NixOS, nix-darwin or home-manager) using a binary cache.

It consists of running a simple agent process cachix deploy agent that listens for a new deployment and executes them. There’s no Nix evaluation or building done on the agent. The agent pulls all dependencies from your binary cache and activates the new profile.

…

Last week a dozen or so people gathered on the island of Lanzarote to hack on NixOS.

If it wasn’t for COVID-19, we’d have a week long of sauna and a private lake in Finland last year, but we had to cancel the event before it was even announced.

Wrap up

The weather on Lanzarote is usually sunny all year long.

Last week we were a bit unfortunate to have two days of wind/clouds and we still managed to hack outdoors all days!

…