packages feed

nova-cache-0.3.1.0: README.md

<div align="center">
<h1>nova-cache</h1>
<p><strong>Nix Binary Cache Protocol for Haskell</strong></p>
<p>Pure-first implementation of the complete Nix binary cache protocol — base32, NAR, narinfo, Ed25519 signing, store paths — with an optional WAI server.</p>
<p><a href="#quick-start">Quick Start</a> · <a href="#modules">Modules</a> · <a href="#server">Server</a> · <a href="#api-reference">API Reference</a></p>
<p>

[![CI](https://github.com/Novavero-AI/nova-cache/actions/workflows/ci.yml/badge.svg)](https://github.com/Novavero-AI/nova-cache/actions/workflows/ci.yml)
[![Hackage](https://img.shields.io/hackage/v/nova-cache.svg)](https://hackage.haskell.org/package/nova-cache)
![Haskell](https://img.shields.io/badge/haskell-GHC%209.8-purple)
![License](https://img.shields.io/badge/license-BSD--3--Clause-blue)

</p>
</div>

---

## What is nova-cache?

A focused, minimal library implementing the full Nix binary cache protocol:

- **Base32** — Nix-specific encoding with the custom `0123456789abcdfghijklmnpqrsvwxyz` alphabet, O(n) ST-based decode
- **Hash** — SHA-256 hashing with `sha256:<nix-base32>` formatting, composition pipeline over crypton
- **StorePath** — Store path parsing, validation, and hash extraction with enforced invariants via newtypes
- **NAR** — Binary serialization and deserialization of the Nix ARchive format using `Builder` monoid composition
- **NarInfo** — Parse and render the key-value narinfo text format
- **Signing** — Ed25519 fingerprint signing and verification for binary cache trust
- **Compression** — xz compress/decompress for NAR transport (behind `compression` cabal flag, default on)
- **Store** — Filesystem storage backend for narinfo and NAR files
- **Validate** — Pure protocol validation: field semantics, content hashes, Ed25519 signatures — all errors collected, not short-circuited
- **Server** — Optional WAI/Warp HTTP server implementing the cache protocol (behind `server` cabal flag)

Every module is pure by default. IO lives at the boundaries only.

---

## Quick Start

Add to your `.cabal` file:

```cabal
build-depends: nova-cache
```

### Hash a Store Path

```haskell
import NovaCache.Hash (hashBytes, formatNixHash)
import qualified Data.ByteString as BS

main :: IO ()
main = do
  contents <- BS.readFile "/nix/store/abc123-hello/bin/hello"
  let nixHash = hashBytes contents
  putStrLn (show (formatNixHash nixHash))
  -- "sha256:0m6g5r7c..."
```

### Parse a NarInfo

```haskell
import NovaCache.NarInfo (parseNarInfo, niStorePath, niNarHash)

main :: IO ()
main = do
  let raw = "StorePath: /nix/store/abc...-hello\n\
            \URL: nar/abc....nar.xz\n\
            \Compression: xz\n\
            \FileHash: sha256:...\n\
            \FileSize: 1234\n\
            \NarHash: sha256:...\n\
            \NarSize: 5678\n\
            \References: \n"
  case parseNarInfo raw of
    Left err -> putStrLn ("Parse error: " ++ err)
    Right ni -> do
      putStrLn ("Store path: " <> show (niStorePath ni))
      putStrLn ("NAR hash:   " <> show (niNarHash ni))
```

### Sign and Verify

```haskell
import NovaCache.Signing (parseSecretKey, parsePublicKey, sign, verify)

main :: IO ()
main = do
  let Right sk = parseSecretKey "mykey:base64secretkey..."
      Right pk = parsePublicKey "mykey:base64pubkey..."
  case sign sk narinfo of
    Left err  -> putStrLn ("Sign error: " ++ err)
    Right sig -> putStrLn ("Verified: " <> show (verify pk narinfo sig))
```

### Validate an Upload

```haskell
import NovaCache.Validate (validateFull, ValidationError)
import NovaCache.Signing (parsePublicKey)

-- Pure validation — no IO needed
validateUpload :: PublicKey -> NarInfo -> ByteString -> ByteString -> IO ()
validateUpload pk narinfo narBytes fileBytes =
  case validateFull pk narinfo narBytes fileBytes of
    Right ()   -> putStrLn "Upload valid"
    Left errs  -> mapM_ (putStrLn . ("  " ++) . show) errs

-- Individual checks compose too
import NovaCache.Validate (validateNarInfo, validateNarHash)

checkNarInfo :: NarInfo -> Either [ValidationError] NarInfo
checkNarInfo = validateNarInfo  -- collects ALL errors, not just the first
```

---

## Modules

### NovaCache.Base32

Nix uses a non-standard base32 alphabet that omits `e`, `o`, `u`, `t` and encodes in reverse byte order. The encoder extracts 5-bit groups from descending positions; the decoder uses an O(n) ST-based bit scatter with mutable vectors.

```haskell
import NovaCache.Base32 (encode, decode)

encode "\xff"       -- "8z"
decode "0z"         -- Right "\x1e"
decode (encode bs)  -- Right bs  (roundtrip)
```

### NovaCache.Hash

SHA-256 via crypton, formatted as `sha256:<nix-base32>`:

```haskell
import NovaCache.Hash (hashBytes, hashFile, formatNixHash, parseNixHash)

formatNixHash (hashBytes "hello")
-- "sha256:0m6g5r7c..."

nixHash <- hashFile "/nix/store/abc123-hello"
```

### NovaCache.StorePath

Newtypes enforce invariants — store path hashes are always 32 characters, names are validated against the Nix character set:

```haskell
import NovaCache.StorePath

let Right sp = parseStorePath defaultStoreDir "/nix/store/abc123...-hello-1.0"
storePathHashString sp  -- "abc123..."
storePathBaseName sp    -- "abc123...-hello-1.0"
renderStorePath defaultStoreDir sp  -- "/nix/store/abc123...-hello-1.0"
```

### NovaCache.NAR

The NAR tree ADT with `Builder`-based serialization:

```haskell
import NovaCache.NAR

let entry = NarRegular False "file contents"
let bytes = serialise entry
deserialise bytes  -- Right entry (roundtrip)

-- Hash a NAR directly (serialise + SHA-256)
narHash entry  -- NixHash

-- Build from filesystem
tree <- serialiseFromPath "/nix/store/abc123-hello"
```

### NovaCache.Signing

Ed25519 signing and verification of narinfo fingerprints. Keys are `name:base64` pairs:

```haskell
import NovaCache.Signing

-- Fingerprint format: 1;<StorePath>;<NarHash>;<NarSize>;<refs>
fingerprint narinfo
-- "1;/nix/store/abc...;sha256:...;5678;"
```

### NovaCache.Validate

Pure protocol validation — field semantics, content hashes, and signatures. All validators collect every error instead of short-circuiting:

```haskell
import NovaCache.Validate

-- Validate narinfo fields (sizes, store path, hash formats, references)
case validateNarInfo narinfo of
  Right ni   -> processUpload ni
  Left errs  -> rejectWith errs  -- [InvalidStorePath ..., NegativeFileSize ...]

-- Verify content hashes match declared values
validateNarHash narinfo rawNarBytes       -- Either ValidationError ()
validateFileHash narinfo compressedBytes  -- Either ValidationError ()

-- Full pipeline: fields + nar hash + file hash + signatures
validateFull publicKey narinfo narBytes fileBytes
-- Either [ValidationError] ()
```

---

## Server

Enable the WAI server with the `server` cabal flag:

```bash
cabal build --flag server
cabal run nova-cache-server -- --port 5000 --store ./nix-cache
```

Or via environment variables:

```bash
PORT=5000 NIX_CACHE_DIR=./nix-cache nova-cache-server
```

### Environment Variables

| Variable | Description |
|----------|-------------|
| `PORT` | Server listen port (default: 5000) |
| `NIX_CACHE_DIR` | Store directory (default: `./nix-cache`) |
| `CACHE_API_KEY` | Bearer token required for PUT requests. Omit for open writes. |
| `SIGNING_KEY_FILE` | Path to Ed25519 secret key file for server-side narinfo signing. |

### Endpoints

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/nix-cache-info` | Cache metadata (StoreDir, WantMassQuery, Priority) |
| `GET` | `/narinfo-hashes` | All cached narinfo hashes (newline-delimited) |
| `GET` | `/<hash>.narinfo` | Fetch narinfo by store path hash |
| `GET` | `/nar/<file>` | Fetch compressed NAR file |
| `PUT` | `/<hash>.narinfo` | Upload narinfo |
| `PUT` | `/nar/<file>` | Upload NAR file |

### Public Binary Cache

A free, public binary cache is available at `cache.novavero.ai`. Add it to
your Nix configuration:

```nix
# nix.conf or /etc/nix/nix.conf
extra-substituters = https://cache.novavero.ai
extra-trusted-public-keys = cache.novavero.ai-1:2yJK0UZWlDDTpThzEdqfGWaj+j3ljOCGoA50Ims47dM=
```

Or use it directly:

```bash
nix build --substituters "https://cache.nixos.org https://cache.novavero.ai" \
          --trusted-public-keys "cache.nixos.org-1:DLD/YGKmo6OceLp6RsiGCbi5FwMExRzJcoJKanMPe/Q= cache.novavero.ai-1:2yJK0UZWlDDTpThzEdqfGWaj+j3ljOCGoA50Ims47dM="
```

### Self-Hosted Cache

Run your own instance:

```bash
# Push to your cache
nix copy --to http://cache.example.com /nix/store/abc123-hello

# Pull from your cache
nix build --substituters http://cache.example.com --trusted-public-keys "mykey:base64..."
```

---

## Architecture

```
                    Pure Core (no IO)
  ┌──────────────────────────────────────────┐
  │                                          │
  │  Base32 ──→ Hash ──→ StorePath           │
  │                         │                │
  │                      NarInfo ──→ Signing │
  │                         │         │      │
  │                        NAR    Validate   │
  │                                          │
  └──────────────────────────────────────────┘
                       │
              IO Boundary (thin)
  ┌──────────────────────────────────────────────────────┐
  │  Compression (flag)    Store    Server (flag)       │
  └──────────────────────────────────────────────────────┘
```

- **10 modules**, 8 pure + 2 at the IO boundary (Compression optional via flag)
- **74 core tests + 4 compression tests**, hand-rolled harness, no framework dependencies
- **Zero partial functions** — total by construction
- **Strict by default** — bang patterns on all data fields

---

## API Reference

### Base32

```haskell
encode :: ByteString -> Text
decode :: Text -> Either String ByteString
```

### Hash

```haskell
hashBytes     :: ByteString -> NixHash
hashFile      :: FilePath -> IO NixHash
formatNixHash :: NixHash -> Text
parseNixHash  :: Text -> Either String NixHash
```

### StorePath

```haskell
parseStorePath      :: StoreDir -> Text -> Either String StorePath
renderStorePath     :: StoreDir -> StorePath -> Text
storePathHashString :: StorePath -> Text
storePathBaseName   :: StorePath -> Text
```

### NAR

```haskell
serialise         :: NarEntry -> ByteString
deserialise       :: ByteString -> Either String NarEntry
narHash           :: NarEntry -> NixHash
serialiseFromPath :: FilePath -> IO NarEntry
```

### NarInfo

```haskell
parseNarInfo  :: Text -> Either String NarInfo
renderNarInfo :: NarInfo -> Text
```

### Signing

```haskell
parseSecretKey :: Text -> Either String SecretKey
parsePublicKey :: Text -> Either String PublicKey
fingerprint    :: NarInfo -> Text
sign           :: SecretKey -> NarInfo -> Either String Text
verify         :: PublicKey -> NarInfo -> Text -> Bool
```

### Validate

```haskell
validateNarInfo  :: NarInfo -> Either [ValidationError] NarInfo
validateNarHash  :: NarInfo -> ByteString -> Either ValidationError ()
validateFileHash :: NarInfo -> ByteString -> Either ValidationError ()
validateSignature :: PublicKey -> NarInfo -> Either [ValidationError] ()
validateFull     :: PublicKey -> NarInfo -> ByteString -> ByteString -> Either [ValidationError] ()
```

### Compression

```haskell
compressXz   :: ByteString -> ByteString
decompressXz :: ByteString -> IO (Either String ByteString)
```

### Store

```haskell
newFileStore      :: FilePath -> IO FileStore
readNarInfo       :: FileStore -> Text -> IO (Maybe ByteString)
writeNarInfo      :: FileStore -> Text -> ByteString -> IO Bool
readNar           :: FileStore -> Text -> IO (Maybe ByteString)
writeNar          :: FileStore -> Text -> ByteString -> IO Bool
listNarInfoHashes :: FileStore -> IO [Text]
getCacheInfo      :: FileStore -> (Text, Bool, Int)
```

Full Haddock documentation is available on [Hackage](https://hackage.haskell.org/package/nova-cache).

---

## GitHub Action — CI Cache Seeding

A reusable composite action is included for pushing Nix store paths to a nova-cache server from CI:

```yaml
- uses: Novavero-AI/nova-cache/.github/actions/seed@main
  with:
    cache-url: https://cache.example.com
    api-key: ${{ secrets.CACHE_API_KEY }}
```

| Input | Required | Description |
|-------|----------|-------------|
| `cache-url` | yes | Base URL of the nova-cache server |
| `api-key` | yes | Bearer token for authenticating uploads |
| `paths` | no | Explicit store paths (space-separated). Defaults to all paths from `shell.nix` / `default.nix` |
| `parallel` | no | Max concurrent uploads (default: 32) |

The action fetches `GET /narinfo-hashes` to determine which paths are already cached, exports only the missing paths, and uploads narinfo + NAR files in parallel. Falls back to uploading everything if the endpoint is unavailable (servers < 0.3.0.0). Works with any CI that has Nix installed.

---

## Build & Test

```bash
cabal build                              # Build library (compression on by default)
cabal build -f-compression               # Build without lzma C dependency
cabal test                               # Run all tests
cabal build --ghc-options="-Werror"      # Warnings as errors
cabal build --flag server                # Build with WAI server
cabal haddock                            # Generate docs
```

Consumers that only need hashing, NAR, or narinfo can disable the `compression` flag to avoid the system `liblzma` dependency:

```cabal
constraints: nova-cache -compression
```

---

<p align="center">
  <sub>BSD-3-Clause · <a href="https://github.com/Novavero-AI">Novavero AI</a></sub>
</p>