# nova-cache
[](https://github.com/Novavero-AI/nova-cache/actions/workflows/ci.yml)
[](https://hackage.haskell.org/package/nova-cache)
[](LICENSE)
nova-cache implements the Nix binary cache protocol in Haskell: nix-base32,
NAR serialization (whole-tree and streaming), narinfo parsing, store paths,
Ed25519 signing and upload validation, a WAI application for a cache server,
and an optional server executable.
Bounded xz, zstd and bzip2 codecs are the public `nova-cache:xz`,
`nova-cache:zstandard` and `nova-cache:bzip2` sublibraries, and the zstd codec
also compresses for uploads.
Parsing, formatting, hashing, signing, validation, one-shot xz decompression
and zstd compression are pure. Reading and writing trees, hashing files, the
streaming codecs, zstd and bzip2 decompression, the store and the server run
in `IO`. nova-cache is the protocol layer under
[nova-nix](https://github.com/Novavero-AI/nova-nix).
## Installation
```cabal
build-depends: nova-cache >= 0.12 && < 0.13
```
The codecs are separate sublibraries, named alongside the main library:
```cabal
build-depends: nova-cache:{nova-cache, xz, zstandard} >= 0.12 && < 0.13
```
libbz2 and libzstd are bundled. liblzma comes from the `xz` package, which
prefers a system liblzma found through pkg-config and falls back to its bundled
sources. `constraints: xz -system-xz` in `cabal.project` forces the bundled
copy, as this repository does.
## Usage
```haskell
import NovaCache.Hash (hashBytes, formatNixHash)
import qualified Data.ByteString as BS
-- Hash file contents into sha256:<nix-base32>
hash <- formatNixHash . hashBytes <$> BS.readFile path
```
```haskell
import NovaCache.NarInfo (parseNarInfo)
import NovaCache.Signing (parseSecretKey, sign)
-- Parse a narinfo and sign it
case (parseNarInfo raw, parseSecretKey "mykey:base64...") of
(Right ni, Right sk) -> print (sign sk ni) -- Right "mykey:<base64 sig>"
(Left err, _) -> putStrLn err
(_, Left err) -> putStrLn err
```
```haskell
import NovaCache.Validate (validateFull)
-- Validate an upload: fields, NAR hash, file hash and signatures. Every
-- error is collected rather than stopping at the first.
case validateFull publicKey ni narBytes fileBytes of
Right () -> accept
Left errs -> reject errs
```
```haskell
import qualified Data.ByteString as BS
import NovaCache.NAR (defaultCaseHack, withNarSource)
import qualified NovaCache.Hash as Hash
-- Stream a tree's NAR and hash it in one pass, without holding the archive
-- in memory. NovaCache.NAR.Stream parses NARs incrementally, and the codec
-- sublibraries bound decompression by a narinfo's declared NarSize.
narHash <- withNarSource defaultCaseHack path $ \pull ->
let go ctx = do
chunk <- pull
if BS.null chunk
then pure (Hash.hashFinalize ctx)
else go (Hash.hashUpdate ctx chunk)
in go Hash.hashInit
```
## Server
```bash
CACHE_API_KEY=change-me cabal run --flag server nova-cache-server -- --port 5000 --store ./nix-cache
```
Without `CACHE_API_KEY` the server refuses to start. `--allow-open-writes` runs
it with unauthenticated writes, for local testing only.
The protocol lives in the `NovaCache.Server` library module as a WAI
`Application`, so the cache can be embedded in another server with its own
root page. The bundled executable is one such embedding.
### Configuration
| Variable | Description |
| --- | --- |
| `PORT` | Listen port (default: 5000; also `--port`) |
| `HOST` | Bind host (default: all interfaces; also `--host`) |
| `NIX_CACHE_DIR` | Store directory (default: `./nix-cache`; also `--store`) |
| `CACHE_API_KEY` | Bearer token required for `PUT`. The server refuses to start without it unless `--allow-open-writes` is passed. |
| `SIGNING_KEY_FILE` | Ed25519 secret key file for server-side narinfo signing |
| `LOG_REQUESTS` | Set to `0` to disable request logging |
### Endpoints
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/` | Landing page: live stats and the cache public key |
| `GET` | `/nix-cache-info` | Cache metadata |
| `GET` | `/narinfo-hashes` | All cached narinfo hashes, newline-delimited (authenticated) |
| `GET` | `/<hash>.narinfo` | Fetch a narinfo |
| `GET` | `/nar/<file>` | Fetch a NAR (streamed from disk) |
| `PUT` | `/<hash>.narinfo` | Upload a narinfo (authenticated, validated; refused unless the NAR its `URL` names is stored with its `FileSize` and `FileHash`) |
| `PUT` | `/nar/<file>` | Upload a NAR (authenticated, streamed to disk) |
`HEAD` is answered wherever `GET` is.
### Reclaiming unreferenced NARs
A client uploads each NAR before its narinfo, so a NAR whose narinfo was
refused, or never sent because the push died, stays under `nar/` with nothing
referencing it. No endpoint deletes anything. The `reclaim-nars` subcommand
does, run on the host against the store directory:
```bash
nova-cache-server reclaim-nars --store /var/lib/nix-cache # list
nova-cache-server reclaim-nars --store /var/lib/nix-cache --delete # remove
```
- A NAR is eligible only once it is at least `--min-age-hours` old (default
336, two weeks), measured from its modification time. A NAR is unreferenced
until its push sends the narinfo, and a push can run for hours.
- If any narinfo cannot be read or parsed, or its `URL` is not `nar/<file>`,
the run prints those narinfos, lists and deletes nothing, and exits nonzero:
the NAR such a narinfo names is unknown, so no NAR can be called unreferenced.
- If the store holds NARs and no narinfo at all, the run lists and deletes
nothing and exits nonzero: a store that lost its narinfos looks the same as
one whose every NAR is unreferenced.
- A run is not atomic with the server, so run it with no push in progress. A
narinfo that arrives during a run for a NAR past the minimum age loses that
NAR, which leaves a signed narinfo with nothing behind it.
- A deletion that fails is reported for that file, and the exit status is
nonzero.
- The store root comes from `--store`, then `NIX_CACHE_DIR`, then
`./nix-cache`, as for the server. A missing `narinfo/` or `nar/` directory is
an error, never an empty store.
### Public cache
A public instance runs at `cache.novavero.ai`. It serves the closure of
nova-nix's Windows `hello` recipe, which nova-nix's CI publishes on every push
to its main branch. To use it from Nix, add it to `/etc/nix/nix.conf`. On a
multi-user install, Nix ignores substituters set by users it does not trust.
```
extra-substituters = https://cache.novavero.ai
extra-trusted-public-keys = cache.novavero.ai-1:9gQ7tLWMM+2tdC9H5sKMJltDIPfD7X2GWlZe8Aa8hHQ=
```
## Building from source
Tested with GHC 9.14.1. CI uses the latest cabal-install release.
```bash
cabal update
cabal build all -f server --enable-tests --ghc-options="-Werror"
cabal test -f server --enable-tests --ghc-options="-Werror"
```
## License
Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).