nova-nix 0.4.0.0 → 0.5.0.0

raw patch · 8 files changed

+816/−84 lines, 8 filesdep +ramdep −memorydep ~containersdep ~cryptondep ~http-client-tlsPVP ok

version bump matches the API change (PVP)

Dependencies added: ram

Dependencies removed: memory

Dependency ranges changed: containers, crypton, http-client-tls, nova-cache, time

API changes (from Hackage documentation)

Files

• CHANGELOG.md +12/−0
• README.md +21/−4
• app/Main.hs +164/−15
• nova-nix.cabal +14/−12
• src/Nix/Builder.hs +15/−1
• src/Nix/Push.hs +395/−0
• src/Nix/Store/DB.hs +8/−0
• test/Main.hs +187/−52

@@ -1,5 +1,17 @@ # Changelog +## 0.5.0.0 — 2026-06-11++### Milestone: Push and Pull Against a Binary Cache++A native Windows build can now publish its outputs to a binary cache and substitute them back on another machine. The toolchain seed and the `hello` build push to `cache.novavero.ai` with `nova-nix push`, and a fresh store realizes them by signed download instead of rebuilding.++- **New: `nova-nix push`** — uploads store paths and their full reference closure to a binary cache. Each path is NAR-serialized (via `nova-cache`), its NAR uploaded before its narinfo so a client never sees metadata pointing at absent data, and the server signs each narinfo on receipt. Already-cached paths are skipped via one `GET /narinfo-hashes` diff, the recorded NAR hash is cross-checked against the store database before upload, and a signed round-trip is verified at the end. Published narinfos use the canonical `/nix/store` path form. Usage: `nova-nix push --cache URL --key-file KEY (--all | <paths>)`.+- **New: `build --substituter URL --trusted-key K`** — try a binary cache before building. The trusted key (`name:base64`) is required alongside the substituter, since substituting without signature verification would be unsafe; the build falls back to local compilation for any path the cache lacks.+- **New: `--store DIR`** — run a command against an alternate store directory instead of the platform default, so a build or push can target a scratch store.+- **Reproducible `hello`** — `hello.nix` links with `-Wl,--no-insert-timestamp`, and the builder pins `SOURCE_DATE_EPOCH` (1980-01-01) for every build, so determinism-aware tools write fixed timestamps. Two cold bootstraps produce a byte-identical `hello.exe`.+- **Dependencies modernized** — builds against `nova-cache >= 0.4.2` (Ed25519 signing, `normalizeKeyText`), `crypton >= 1.1` with `ram` replacing the deprecated `memory`, and current upper bounds for `containers`, `http-client-tls`, and `time`. No source change beyond the dependency swap; derivation-hash parity is unaffected (CI-verified).+ ## 0.4.0.0 — 2026-06-10  ### Milestone: The First Native Windows Build

@@ -22,9 +22,9 @@   ...      (17 fixed-output fetches, one per MSYS2 package)   [build]  mingw-w64-seed   [build]  hello-C:\nix\store\4lv3zydln7y7wg0znf56dfmzf5s59h9g-hello+C:\nix\store\zr07i99kqnv48q29n706qxar7h1gfins-hello -$ C:\nix\store\4lv3zydln7y7wg0znf56dfmzf5s59h9g-hello\hello.exe+$ C:\nix\store\zr07i99kqnv48q29n706qxar7h1gfins-hello\hello.exe Hello from the first native Windows Nix build. ``` @@ -57,9 +57,26 @@  $ nova-nix eval FILE.nix                       # evaluate a file $ nova-nix build FILE.nix                       # build a derivation+$ nova-nix push --cache URL --key-file KEY --all # publish the store to a cache $ nova-nix --nix-path nixpkgs=/path eval FILE.nix ``` +## Binary cache++Built outputs are content-addressed, so they can be served and substituted like any Nix store path. `nova-nix push` uploads a path and its closure to a cache (NAR before narinfo, skipping what the cache already has, signed server-side); `build --substituter` pulls from one before compiling.++```console+$ nova-nix build pkgs\windows\hello.nix \+    --store C:\scratch\store \+    --substituter https://cache.novavero.ai \+    --trusted-key cache.novavero.ai-1:9gQ7tLWMM+2tdC9H5sKMJltDIPfD7X2GWlZe8Aa8hHQ=+  [subst]  mingw-w64-x86_64-gcc-16.1.0-5-any.pkg.tar.zst+  ...      (signed download instead of a rebuild)+  [subst]  hello+```++A public instance runs at `cache.novavero.ai`, seeded with the MinGW-w64 toolchain and the `hello` build.+ ## How it works  Six layers — Haskell for logic, C99 for data:@@ -85,13 +102,13 @@  ## Roadmap -**Done** — parser, lazy bytecode evaluator, the Nix `builtins` set, the C99 data layer, content-addressed store, derivation builder, binary-cache substituter, `import <nixpkgs> {}` evaluation, derivation-hash parity with upstream Nix (`hello`'s 253-derivation closure byte-matches `nix-instantiate`), and native Windows builds from a store-pinned MinGW-w64 toolchain (stage 0).+**Done** — parser, lazy bytecode evaluator, the Nix `builtins` set, the C99 data layer, content-addressed store, derivation builder, binary-cache substituter and `push`, `import <nixpkgs> {}` evaluation, derivation-hash parity with upstream Nix (`hello`'s 253-derivation closure byte-matches `nix-instantiate`), and native Windows builds from a store-pinned MinGW-w64 toolchain (stage 0), published to and substituted from a binary cache.  **Next**  - Windows stdenv stage 1 — a setup script and compiler wrappers over the seed; rebuild coreutils and bash from source. - Parity across more of nixpkgs — extend the byte-match check beyond `hello`'s closure.-- Substituter — XZ decompression.+- Cache — serve large NARs from object storage, and compress them.  ## Library usage 

@@ -6,6 +6,7 @@ -- nova-nix eval  FILE.nix                  Evaluate a .nix file, print result -- nova-nix eval  --expr 'EXPR'             Evaluate an inline expression -- nova-nix build FILE.nix                  Build a derivation from a .nix file+-- nova-nix push  --cache URL --all         Push store paths to a binary cache -- @ -- -- Flags:@@ -30,8 +31,10 @@ import Nix.Eval.IO (EvalState (..), newEvalState, runEvalIO) import Nix.Eval.Types (clistFromThunks, clistThunks, thunkToCPtr) import Nix.Parser (parseNix, readFileAutoEncoding)-import Nix.Store (Store (..), closeStore, isValid, openStore, registerPaths, registrationFor, setReadOnly, writeDrv, writeDrvAterm)-import Nix.Store.Path (StorePath, defaultStoreDir, parseStorePath, platformStoreDir, storePathToFilePath)+import Nix.Push (PushConfig (..), PushSummary (..), loadApiKeyFile, pushPaths)+import Nix.Store (Store (..), closeStore, isValid, openStore, queryAllValidPaths, registerPaths, registrationFor, setReadOnly, writeDrv, writeDrvAterm)+import Nix.Store.Path (StoreDir (..), StorePath (..), defaultStoreDir, parseStorePath, platformStoreDir, storePathHashLen, storePathToFilePath)+import Nix.Substituter (CacheConfig (..)) import Paths_nova_nix (getDataDir) import System.Directory (canonicalizePath, copyFile, createDirectoryIfMissing, doesDirectoryExist, getCurrentDirectory, getTemporaryDirectory, listDirectory) import System.Environment (getArgs)@@ -49,6 +52,12 @@   { optNixPaths :: ![T.Text],     optStrict :: !Bool,     optAterm :: !Bool,+    -- | Store directory override (default: the platform store).+    optStore :: !(Maybe FilePath),+    -- | Binary cache URL to substitute from before building.+    optSubstituter :: !(Maybe String),+    -- | Trusted public key (@name:base64@) for the substituter.+    optTrustedKey :: !(Maybe String),     optCommand :: !Command   } @@ -56,10 +65,23 @@   = CmdEvalFile !FilePath   | CmdEvalExpr !T.Text   | CmdBuild !FilePath+  | CmdPush !PushArgs   | CmdHelp +-- | Arguments to the push command.+data PushArgs = PushArgs+  { paCacheUrl :: !(Maybe String),+    paKeyFile :: !(Maybe FilePath),+    paAll :: !Bool,+    paPaths :: ![String]+  }++-- | Push arguments before any flag is parsed.+emptyPushArgs :: PushArgs+emptyPushArgs = PushArgs Nothing Nothing False []+ parseArgs :: [String] -> CliOpts-parseArgs = go (CliOpts [] False False CmdHelp)+parseArgs = go (CliOpts [] False False Nothing Nothing Nothing CmdHelp)   where     go opts [] = opts     go opts ("--nix-path" : val : rest) =@@ -68,9 +90,16 @@       go (opts {optStrict = True}) rest     go opts ("--aterm" : rest) =       go (opts {optAterm = True}) rest+    go opts ("--store" : dir : rest) =+      go (opts {optStore = Just dir}) rest+    go opts ("--substituter" : url : rest) =+      go (opts {optSubstituter = Just url}) rest+    go opts ("--trusted-key" : key : rest) =+      go (opts {optTrustedKey = Just key}) rest     go opts ("eval" : rest) = goEval opts rest     go opts ("build" : path : rest) =       go (opts {optCommand = CmdBuild path}) rest+    go opts ("push" : rest) = goPush opts emptyPushArgs rest     go opts _ = opts     -- Sub-parser for eval: handles --strict and --expr interleaved with the file arg.     goEval opts [] = opts@@ -82,6 +111,18 @@       go (opts {optCommand = CmdEvalExpr (T.pack expr)}) rest     goEval opts (path : rest) =       go (opts {optCommand = CmdEvalFile path}) rest+    -- Sub-parser for push: flags and explicit store paths in any order.+    goPush opts pushArgs [] = opts {optCommand = CmdPush pushArgs}+    goPush opts pushArgs ("--store" : dir : rest) =+      goPush (opts {optStore = Just dir}) pushArgs rest+    goPush opts pushArgs ("--cache" : url : rest) =+      goPush opts (pushArgs {paCacheUrl = Just url}) rest+    goPush opts pushArgs ("--key-file" : path : rest) =+      goPush opts (pushArgs {paKeyFile = Just path}) rest+    goPush opts pushArgs ("--all" : rest) =+      goPush opts (pushArgs {paAll = True}) rest+    goPush opts pushArgs (path : rest) =+      goPush opts (pushArgs {paPaths = paPaths pushArgs ++ [path]}) rest  -- | Merge --nix-path entries, bundled data dir, and NIX_PATH search paths. -- The data dir is appended last so user paths take priority.@@ -106,7 +147,8 @@     CmdEvalExpr expr       | optAterm opts -> evalExprAterm (optNixPaths opts) dataDir expr       | otherwise -> evalExpr (optStrict opts) (optNixPaths opts) dataDir expr-    CmdBuild filePath -> buildFile (optNixPaths opts) dataDir filePath+    CmdBuild filePath -> buildFile opts dataDir filePath+    CmdPush pushArgs -> pushCommand opts pushArgs     CmdHelp -> do       hPutStrLn stderr "Usage: nova-nix [--nix-path NAME=PATH] <command>"       hPutStrLn stderr ""@@ -114,11 +156,17 @@       hPutStrLn stderr "  eval FILE.nix          Evaluate a .nix file, print result"       hPutStrLn stderr "  eval --expr 'EXPR'     Evaluate an inline expression"       hPutStrLn stderr "  build FILE.nix         Build a derivation from a .nix file"+      hPutStrLn stderr "  push --cache URL       Push store paths (and their closures) to a binary cache"       hPutStrLn stderr ""       hPutStrLn stderr "Flags:"       hPutStrLn stderr "  --strict               Deep-force all thunks before printing (warning: OOM on large results)"       hPutStrLn stderr "  --aterm                With eval --expr, print the derivation's .drv ATerm"       hPutStrLn stderr "  --nix-path NAME=PATH   Add search path (repeatable, merged with NIX_PATH)"+      hPutStrLn stderr "  --all                  With push: select every valid path in the store"+      hPutStrLn stderr "  --key-file PATH        With push: file holding the cache API key"+      hPutStrLn stderr "  --store DIR            Use DIR as the store (default: the platform store)"+      hPutStrLn stderr "  --substituter URL      Try this binary cache before building"+      hPutStrLn stderr "  --trusted-key K        Public key (name:base64) for the substituter"       exitFailure  -- | Evaluate a .nix file and print the result.@@ -200,8 +248,9 @@  -- | Parse, evaluate, extract derivation, build, and print result. -- The file argument is canonicalized for the same reason as in 'evalFile'.-buildFile :: [T.Text] -> FilePath -> FilePath -> IO ()-buildFile extraPaths dataDir rawFilePath = do+buildFile :: CliOpts -> FilePath -> FilePath -> IO ()+buildFile opts dataDir rawFilePath = do+  caches <- either failWith pure (substituterConfig (optSubstituter opts) (optTrustedKey opts))   filePath <- canonicalizePath rawFilePath   source <- readFileAutoEncoding filePath   case parseNix (T.pack filePath) source of@@ -210,7 +259,7 @@       exitFailure     Right expr -> do       st0 <- newEvalState (takeDirectory filePath)-      let searchPaths = mergeSearchPaths extraPaths dataDir (esSearchPaths st0)+      let searchPaths = mergeSearchPaths (optNixPaths opts) dataDir (esSearchPaths st0)           st = st0 {esSearchPaths = searchPaths}       result <- runEvalIO st $ do         val <- eval (builtinEnv (esTimestamp st) searchPaths) expr@@ -233,17 +282,17 @@           -- during evaluation; written to the store before building.           drvClosure <- readIORef (esDrvClosure st)           sourceCache <- readIORef (esSourcePathCache st)-          store <- openStore platformStoreDir+          store <- openStore (chosenStoreDir opts)           -- Materialize eval-coerced source paths (src = ./file, path           -- interpolation): evaluation computes their store paths as text           -- only — the parity runner's store is not writable — so the build           -- driver performs the copy and registration.           materializeEvalSources store sourceCache-          buildResult <- buildAndRegister store drvClosure drv drvSP+          buildResult <- buildAndRegister store caches drvClosure drv drvSP           closeStore store           case buildResult of             BuildSuccess sp ->-              TIO.putStrLn (T.pack (storePathToFilePath platformStoreDir sp))+              TIO.putStrLn (T.pack (storePathToFilePath (chosenStoreDir opts) sp))             BuildFailure msg code -> do               TIO.hPutStrLn stderr ("build failed (exit " <> T.pack (show code) <> "): " <> msg)               exitFailure@@ -282,11 +331,35 @@   hPutStrLn stderr "error: result is not a derivation"   exitFailure +-- | The store directory selected by @--store@, or the platform default.+chosenStoreDir :: CliOpts -> StoreDir+chosenStoreDir opts = maybe platformStoreDir StoreDir (optStore opts)++-- | Default priority for a CLI-configured substituter (cache.nixos.org is 40).+substituterPriority :: Int+substituterPriority = 50++-- | Build the cache list from @--substituter@\/@--trusted-key@.  Both or+-- neither: a substituter without a trusted key would skip signature+-- verification, and a key without a substituter is a mistake.+substituterConfig :: Maybe String -> Maybe String -> Either T.Text [CacheConfig]+substituterConfig Nothing Nothing = Right []+substituterConfig Nothing (Just _) = Left "--trusted-key requires --substituter"+substituterConfig (Just _) Nothing = Left "--substituter requires --trusted-key (name:base64)"+substituterConfig (Just url) (Just key) =+  Right+    [ CacheConfig+        { ccUrl = T.dropWhileEnd (== '/') (T.pack url),+          ccPublicKey = T.pack key,+          ccPriority = substituterPriority+        }+    ]+ -- | Write the .drv file to the store and build with dependency resolution. -- The drvPath is the store path of the .drv file itself, extracted from -- the evaluation result alongside the Derivation struct.-buildAndRegister :: Store -> Map.Map T.Text T.Text -> Derivation -> StorePath -> IO BuildResult-buildAndRegister store drvClosure drv drvSP = do+buildAndRegister :: Store -> [CacheConfig] -> Map.Map T.Text T.Text -> Derivation -> StorePath -> IO BuildResult+buildAndRegister store caches drvClosure drv drvSP = do   -- Materialize the full input-.drv closure (every transitive dependency's   -- recipe) to the store.  buildWithDeps reads these back to construct the   -- dependency graph; without them it cannot realize any non-leaf derivation.@@ -297,11 +370,87 @@   -- Build with dependency resolution   tmpDir <- getTemporaryDirectory   let config =-        (defaultBuildConfig platformStoreDir)-          { bcTmpDir = tmpDir+        (defaultBuildConfig (stDir store))+          { bcTmpDir = tmpDir,+            bcCaches = caches           }   buildWithDeps config store drv drvSP +-- ---------------------------------------------------------------------------+-- Push command+-- ---------------------------------------------------------------------------++-- | Push the closure of the selected store paths to a binary cache.+pushCommand :: CliOpts -> PushArgs -> IO ()+pushCommand opts pushArgs = do+  cacheUrl <- case paCacheUrl pushArgs of+    Just url -> pure (T.dropWhileEnd (== '/') (T.pack url))+    Nothing -> failWith "push: --cache URL is required"+  case (paAll pushArgs, paPaths pushArgs) of+    (True, _ : _) -> failWith "push: --all and explicit paths are mutually exclusive"+    (False, []) -> failWith "push: name store paths to push, or pass --all"+    _ -> pure ()+  apiKey <- case paKeyFile pushArgs of+    Nothing -> pure Nothing+    Just path -> do+      loaded <- loadApiKeyFile path+      either failWith (pure . Just) loaded+  store <- openStore (chosenStoreDir opts)+  rootsResult <- resolvePushRoots store pushArgs+  case rootsResult of+    Left err -> do+      closeStore store+      failWith err+    Right roots -> do+      result <- pushPaths (PushConfig cacheUrl apiKey) store roots+      closeStore store+      case result of+        Left err -> failWith ("push failed: " <> err)+        Right summary ->+          TIO.putStrLn+            ( "pushed "+                <> T.pack (show (psPushed summary))+                <> " path(s), "+                <> T.pack (show (psSkipped summary))+                <> " already cached"+            )++-- | Resolve push roots: every valid path with @--all@, otherwise each named+-- path.  Named paths may be full store paths in either store-dir form, or a+-- bare @hash-name@ basename.+resolvePushRoots :: Store -> PushArgs -> IO (Either T.Text [StorePath])+resolvePushRoots store pushArgs+  | paAll pushArgs = do+      pathTexts <- queryAllValidPaths (stDB store)+      pure (traverse parseDbPath pathTexts)+  | otherwise = pure (traverse parseArgPath (paPaths pushArgs))+  where+    parseDbPath txt =+      maybe (Left ("unparseable store DB path: " <> txt)) Right (parseStorePath platformStoreDir txt)+    parseArgPath raw =+      let txt = T.pack raw+          attempts =+            [ parseStorePath platformStoreDir txt,+              parseStorePath defaultStoreDir txt,+              parseBareBasename txt+            ]+       in case catMaybes attempts of+            (sp : _) -> Right sp+            [] -> Left ("not a store path: " <> txt)+    parseBareBasename txt+      | T.length txt >= storePathHashLen + 2,+        (hashPart, rest) <- T.splitAt storePathHashLen txt,+        Just ('-', name) <- T.uncons rest,+        not (T.null name) =+          Just (StorePath hashPart name)+      | otherwise = Nothing++-- | Print an error to stderr and exit.+failWith :: T.Text -> IO a+failWith msg = do+  TIO.hPutStrLn stderr msg+  exitFailure+ -- | Copy eval-coerced source paths into the store and register them.  The -- evaluator's source-path cache maps each coerced filesystem path to its -- @source@ fixed-output store path (text only — eval performs no store@@ -320,7 +469,7 @@           if valid             then pure Nothing             else do-              let dest = storePathToFilePath platformStoreDir sp+              let dest = storePathToFilePath (stDir store) sp               copyPathInto (T.unpack rawPath) dest               setReadOnly dest               Just <$> registrationFor store sp Nothing []

@@ -1,6 +1,6 @@ cabal-version:      3.0 name:               nova-nix-version:            0.4.0.0+version:            0.5.0.0 synopsis:           Windows-native Nix implementation in Haskell and C99 description:   A from-scratch implementation of the Nix package manager that runs@@ -9,8 +9,9 @@   data layer that keeps evaluation data off the GHC heap.  It evaluates   real Nix expressions (including nixpkgs), computes derivations and   content-addressed store paths that byte-match upstream Nix, and builds-  packages natively on Windows from a store-pinned MinGW-w64 toolchain,-  with a derivation builder and binary-cache substituter.+  packages natively on Windows from a store-pinned MinGW-w64 toolchain.+  Built outputs are content-addressed, substituted from a binary cache+  before building, and pushed to one with @nova-nix push@.    Built on @nova-cache@ for NAR serialization, narinfo handling, and   Ed25519-signed binary substitution.@@ -76,6 +77,7 @@     Nix.Derivation     Nix.Builder     Nix.Builder.Unpack+    Nix.Push     Nix.Substituter     Nix.Builtins     Nix.Hash@@ -84,22 +86,22 @@       base                >= 4.16 && < 5     , array               >= 0.5 && < 0.6     , bytestring          >= 0.11 && < 0.13-    , containers          >= 0.6 && < 0.8-    , crypton             >= 1.0 && < 1.1+    , containers          >= 0.6 && < 0.9+    , crypton             >= 1.1 && < 2     , directory           >= 1.3 && < 1.4     , filepath            >= 1.4 && < 1.6     , http-client         >= 0.7 && < 0.8-    , http-client-tls     >= 0.3 && < 0.4+    , http-client-tls     >= 0.3 && < 0.5     , http-types          >= 0.12 && < 0.13-    , memory              >= 0.18 && < 1+    , ram                 >= 0.20 && < 1     , mtl                 >= 2.2 && < 2.4-    , nova-cache          >= 0.3.0 && < 0.4+    , nova-cache          >= 0.4.2 && < 0.5     , process             >= 1.6 && < 1.7     , regex-tdfa          >= 1.3 && < 1.4     , sqlite-simple       >= 0.4 && < 0.5     , tar                 >= 0.7.1 && < 0.8     , text                >= 2.0 && < 2.2-    , time                >= 1.9 && < 1.15+    , time                >= 1.9 && < 1.17     , zstd                >= 0.1 && < 0.2    c-sources:@@ -140,7 +142,7 @@    build-depends:       base                >= 4.16 && < 5-    , containers          >= 0.6 && < 0.8+    , containers          >= 0.6 && < 0.9     , directory           >= 1.3 && < 1.4     , filepath            >= 1.4 && < 1.6     , nova-nix@@ -161,10 +163,10 @@   build-depends:       base                >= 4.16 && < 5     , bytestring          >= 0.11 && < 0.13-    , containers          >= 0.6 && < 0.8+    , containers          >= 0.6 && < 0.9     , directory           >= 1.3 && < 1.4     , filepath            >= 1.4 && < 1.6-    , nova-cache          >= 0.3.0 && < 0.4+    , nova-cache          >= 0.4.2 && < 0.5     , nova-nix     , process             >= 1.6 && < 1.7     , tar                 >= 0.7.1 && < 0.8

@@ -109,6 +109,19 @@ httpStatusOk :: Int httpStatusOk = 200 +-- | Environment variable for the reproducible-builds.org build timestamp.+envSourceDateEpoch :: Text+envSourceDateEpoch = "SOURCE_DATE_EPOCH"++-- | The fixed build timestamp handed to every builder: 1980-01-01 UTC.+-- Determinism-aware tools (binutils @ld@ writes it into the PE header+-- instead of the wall clock; gcc uses it for @__DATE__@\/@__TIME__@)+-- produce identical output across builds.  1980 rather than 0 because+-- zip timestamps cannot represent dates before 1980 — the same value+-- nixpkgs' stdenv uses.  A derivation env may override it.+sourceDateEpochValue :: Text+sourceDateEpochValue = "315532800"+ -- --------------------------------------------------------------------------- -- Configuration -- ---------------------------------------------------------------------------@@ -323,7 +336,8 @@             (envTmpDir, T.pack buildDir),             (homeEnvVar, T.pack buildDir),             (envNixStore, T.pack (unStoreDir (bcStoreDir config))),-            (envPath, buildPath builderPath)+            (envPath, buildPath builderPath),+            (envSourceDateEpoch, sourceDateEpochValue)           ]    in -- Priority: output paths > derivation env > standard env       Map.unions [outputEnv, baseEnv, standardEnv]

@@ -0,0 +1,395 @@+{-# LANGUAGE ScopedTypeVariables #-}++-- | Push store paths to a nova-cache binary cache.+--+-- == The push choreography+--+-- 1. Compute the full reference closure of the requested paths from the+--    store database — a cache must never hold a path without its+--    dependencies, or substitution 404s on cold machines.+-- 2. Fetch @\/narinfo-hashes@ from the cache and skip paths already there.+-- 3. Upload every missing NAR, then every narinfo.  NARs go first+--    globally: a narinfo is the public announcement that a path is+--    available, so it must never appear before its bytes do.+-- 4. Verify one round-trip: fetch a narinfo back and check the server+--    signed it.+--+-- == Path forms+--+-- The store database and filesystem use the platform store dir+-- (@C:\\nix\\store@ on Windows).  Published narinfos always use the+-- canonical store dir (@\/nix\/store@) with forward slashes — the form the+-- store-path hashes were computed against and the form cache servers+-- validate.  'StorePath' itself is directory-agnostic, so translation is+-- just a matter of which renderer runs at which boundary.+--+-- == Compression+--+-- Uploads use @Compression: none@: the substituter round-trips it on every+-- platform today (XZ support is gated behind nova-cache's @compression@+-- flag, which is off on Windows), and the dominant seed payloads are+-- already-compressed source tarballs.+module Nix.Push+  ( -- * Configuration+    PushConfig (..),+    PushSummary (..),++    -- * Pushing+    pushPaths,+    computeClosure,+    loadApiKeyFile,++    -- * Pure pieces (exported for tests)+    mkNarInfo,+    planMissing,+    narFileName,+    stripHashPrefix,+    storePathBasename,+  )+where++import Control.Exception (SomeException, try)+import Control.Monad (forM, forM_, unless, when)+import Control.Monad.Except (ExceptT (..), liftEither, runExceptT, throwError)+import Control.Monad.IO.Class (liftIO)+import qualified Data.ByteString as BS+import Data.List (sort)+import Data.Set (Set)+import qualified Data.Set as Set+import Data.Text (Text)+import qualified Data.Text as T+import qualified Data.Text.Encoding as TE+import qualified Data.Text.IO as TIO+import qualified Network.HTTP.Client as HTTP+import qualified Network.HTTP.Client.TLS as HTTPS+import qualified Network.HTTP.Types as HTTP+import Nix.Store (Store (..), queryDeriver, queryPathInfo, queryReferences)+import qualified Nix.Store.DB as DB+import Nix.Store.Path (StorePath (..), defaultStoreDir, parseStorePath, storePathToFilePath, storePathToText)+import qualified NovaCache.Hash as Hash+import qualified NovaCache.NAR as NAR+import NovaCache.NarInfo (NarInfo (..), parseNarInfo, renderNarInfo)+import NovaCache.Signing (normalizeKeyText)+import System.IO (stderr)++-- ---------------------------------------------------------------------------+-- Named constants+-- ---------------------------------------------------------------------------++-- | Cache endpoint listing all stored narinfo hashes, one per line.+narinfoHashesEndpoint :: Text+narinfoHashesEndpoint = "narinfo-hashes"++-- | URL path segment under which NAR files live.+narDirSegment :: Text+narDirSegment = "nar"++-- | File extension for an uncompressed NAR.+narExtension :: Text+narExtension = ".nar"++-- | Extension for narinfo objects.+narInfoExtension :: Text+narInfoExtension = ".narinfo"++-- | The narinfo @Compression@ value used for uploads.+compressionNone :: Text+compressionNone = "none"++-- | Authorization scheme prefix for the API key.+bearerPrefix :: BS.ByteString+bearerPrefix = "Bearer "++-- | HTTP success status code.+httpStatusOk :: Int+httpStatusOk = 200++-- ---------------------------------------------------------------------------+-- Configuration and results+-- ---------------------------------------------------------------------------++-- | Where and how to push.+data PushConfig = PushConfig+  { -- | Cache base URL, no trailing slash (e.g. @https:\/\/cache.example.com@).+    pcCacheUrl :: !Text,+    -- | Bearer token for authenticated writes.  'Nothing' sends no+    -- Authorization header (only useful against an open-writes server).+    pcApiKey :: !(Maybe Text)+  }+  deriving (Eq, Show)++-- | What a push accomplished.+data PushSummary = PushSummary+  { -- | Paths uploaded (NAR + narinfo).+    psPushed :: !Int,+    -- | Closure paths skipped because the cache already had them.+    psSkipped :: !Int+  }+  deriving (Eq, Show)++-- ---------------------------------------------------------------------------+-- Key loading+-- ---------------------------------------------------------------------------++-- | Read an API key from a file, dropping byte-order marks and surrounding+-- whitespace.  Key files written by Windows tooling are a known source of+-- BOM contamination; 'normalizeKeyText' makes the transfer byte-clean.+loadApiKeyFile :: FilePath -> IO (Either Text Text)+loadApiKeyFile path = do+  attempt <- try (TIO.readFile path)+  pure $ case attempt of+    Left (e :: SomeException) -> Left ("cannot read key file: " <> T.pack (show e))+    Right raw ->+      let key = normalizeKeyText raw+       in if T.null key+            then Left ("key file is empty: " <> T.pack path)+            else Right key++-- ---------------------------------------------------------------------------+-- Closure computation+-- ---------------------------------------------------------------------------++-- | Compute the full reference closure of the given paths from the store+-- database (breadth-first over @Refs@).  Fails if a recorded reference does+-- not parse as a store path — that would mean a corrupt database, and+-- pushing a closure with holes would poison the cache.+computeClosure :: Store -> [StorePath] -> IO (Either Text [StorePath])+computeClosure store roots = runExceptT (go Set.empty [] roots)+  where+    go :: Set Text -> [StorePath] -> [StorePath] -> ExceptT Text IO [StorePath]+    go _ acc [] = pure (reverse acc)+    go seen acc (sp : rest)+      | spHash sp `Set.member` seen = go seen acc rest+      | otherwise = do+          refTexts <- liftIO (queryReferences (stDB store) sp)+          refs <- liftEither (traverse parseRef refTexts)+          go (Set.insert (spHash sp) seen) (sp : acc) (refs ++ rest)+    parseRef txt = case parseStorePath (stDir store) txt of+      Just sp -> Right sp+      Nothing -> Left ("unparseable reference in store DB: " <> txt)++-- ---------------------------------------------------------------------------+-- Pure planning and rendering+-- ---------------------------------------------------------------------------++-- | Paths whose narinfo hash the cache does not already have.+planMissing :: Set Text -> [StorePath] -> [StorePath]+planMissing remote = filter (\sp -> not (spHash sp `Set.member` remote))++-- | The basename form used in narinfo @References@ and @Deriver@ fields:+-- @\<hash\>-\<name\>@ with no store dir.+storePathBasename :: StorePath -> Text+storePathBasename sp = spHash sp <> "-" <> spName sp++-- | Drop a @\<algo\>:@ prefix from a formatted hash, leaving the digest.+stripHashPrefix :: Text -> Text+stripHashPrefix h = case T.breakOn ":" h of+  (_, rest) | not (T.null rest) -> T.drop 1 rest+  _ -> h++-- | The NAR object filename for a given (uncompressed) NAR hash.+narFileName :: Text -> Text+narFileName narHash = stripHashPrefix narHash <> narExtension++-- | Construct the narinfo describing a store path.+--+-- With @Compression: none@ the file fields equal the NAR fields.  The+-- @StorePath@ uses the canonical @\/nix\/store@ form; references and the+-- deriver are basenames, matching the wire format.+mkNarInfo :: StorePath -> Text -> Int -> [StorePath] -> Maybe StorePath -> NarInfo+mkNarInfo sp narHash narSize refs deriver =+  NarInfo+    { niStorePath = storePathToText defaultStoreDir sp,+      niUrl = narDirSegment <> "/" <> narFileName narHash,+      niCompression = compressionNone,+      niFileHash = narHash,+      niFileSize = fromIntegral narSize,+      niNarHash = narHash,+      niNarSize = fromIntegral narSize,+      niReferences = sort (map storePathBasename refs),+      niDeriver = storePathBasename <$> deriver,+      niSigs = [],+      niCA = Nothing+    }++-- ---------------------------------------------------------------------------+-- Push orchestration+-- ---------------------------------------------------------------------------++-- | Push the closure of the given paths to the cache.+--+-- Serializes one NAR at a time (bounded memory), uploads all NARs before+-- any narinfo, and finishes with a signed round-trip check.  Any failure+-- aborts with an error; a partial push leaves only orphaned NARs behind,+-- which are invisible to clients.+pushPaths :: PushConfig -> Store -> [StorePath] -> IO (Either Text PushSummary)+pushPaths cfg store roots = do+  attempt <- try (runExceptT run)+  pure $ case attempt of+    Left (e :: SomeException) -> Left ("push failed: " <> T.pack (show e))+    Right r -> r+  where+    run :: ExceptT Text IO PushSummary+    run = do+      manager <- liftIO newPushManager+      closure <- ExceptT (computeClosure store roots)+      remote <- fetchRemoteHashes manager cfg+      let missing = planMissing remote closure+          skipped = length closure - length missing+      when (skipped > 0) $+        logLine ("[skip]  " <> T.pack (show skipped) <> " path(s) already cached")+      -- Phase 1: NARs.  One at a time; bytes are dropped after upload.+      pairs <- forM missing $ \sp -> do+        narInfo <- uploadNar manager cfg store sp+        pure (sp, narInfo)+      -- Phase 2: narinfos, only after every NAR is in place.+      forM_ pairs $ \(sp, narInfo) -> do+        uploadNarInfo manager cfg sp narInfo+        logLine ("[push]  " <> storePathBasename sp)+      -- Round-trip: the first pushed narinfo must come back signed.+      case pairs of+        ((sp, _) : _) -> verifySignedRoundTrip manager cfg sp+        [] -> pure ()+      pure PushSummary {psPushed = length pairs, psSkipped = skipped}++-- | HTTP manager for pushes.  Large NAR uploads over residential uplinks+-- can take many minutes, so the response timeout is disabled rather than+-- guessed at.+newPushManager :: IO HTTP.Manager+newPushManager =+  HTTP.newManager+    HTTPS.tlsManagerSettings {HTTP.managerResponseTimeout = HTTP.responseTimeoutNone}++-- | Fetch the set of narinfo hashes the cache already stores.+fetchRemoteHashes :: HTTP.Manager -> PushConfig -> ExceptT Text IO (Set Text)+fetchRemoteHashes manager cfg = do+  let url = pcCacheUrl cfg <> "/" <> narinfoHashesEndpoint+  response <- httpRequest manager "GET" url [] Nothing+  let code = HTTP.statusCode (HTTP.responseStatus response)+  unless (code == httpStatusOk) $+    throwError ("GET " <> narinfoHashesEndpoint <> " returned HTTP " <> T.pack (show code))+  body <- decodeUtf8Lenient (responseBytes response)+  pure (Set.fromList (filter (not . T.null) (map T.strip (T.lines body))))++-- | Serialize a store path to a NAR, cross-check the database NAR hash,+-- and upload the bytes.  Returns the narinfo to publish in phase 2.+uploadNar :: HTTP.Manager -> PushConfig -> Store -> StorePath -> ExceptT Text IO NarInfo+uploadNar manager cfg store sp = do+  let physicalPath = storePathToFilePath (stDir store) sp+  narEntry <- liftIO (NAR.serialiseFromPath physicalPath)+  let narBytes = NAR.serialise narEntry+      narHash = Hash.formatNixHash (Hash.hashBytes narBytes)+      narSize = BS.length narBytes+  -- The database recorded this path's NAR hash at registration; a mismatch+  -- now means the path changed on disk — refuse to publish corruption.+  recorded <- liftIO (queryPathInfo (stDB store) sp)+  case recorded of+    Just info+      | DB.piNarHash info /= narHash ->+          throwError+            ( "store integrity: "+                <> storePathBasename sp+                <> " hashes to "+                <> narHash+                <> " but the DB recorded "+                <> DB.piNarHash info+            )+    _ -> pure ()+  refTexts <- liftIO (queryReferences (stDB store) sp)+  refs <- liftEither (traverse (parseRefText store) refTexts)+  deriverText <- liftIO (queryDeriver (stDB store) sp)+  let deriver = deriverText >>= parseStorePath (stDir store)+      narInfo = mkNarInfo sp narHash narSize refs deriver+      url = pcCacheUrl cfg <> "/" <> niUrl narInfo+  logLine+    ( "[nar]   "+        <> storePathBasename sp+        <> " ("+        <> T.pack (show narSize)+        <> " bytes)"+    )+  response <- httpRequest manager "PUT" url (authHeaders cfg) (Just narBytes)+  expectOk ("PUT " <> niUrl narInfo) response+  pure narInfo++-- | Upload a rendered narinfo (the server validates and signs it).+uploadNarInfo :: HTTP.Manager -> PushConfig -> StorePath -> NarInfo -> ExceptT Text IO ()+uploadNarInfo manager cfg sp narInfo = do+  let object = spHash sp <> narInfoExtension+      url = pcCacheUrl cfg <> "/" <> object+      body = TE.encodeUtf8 (renderNarInfo narInfo)+  response <- httpRequest manager "PUT" url (authHeaders cfg) (Just body)+  expectOk ("PUT " <> object) response++-- | Fetch a just-pushed narinfo back and require a signature on it.+verifySignedRoundTrip :: HTTP.Manager -> PushConfig -> StorePath -> ExceptT Text IO ()+verifySignedRoundTrip manager cfg sp = do+  let object = spHash sp <> narInfoExtension+      url = pcCacheUrl cfg <> "/" <> object+  response <- httpRequest manager "GET" url [] Nothing+  expectOk ("GET " <> object) response+  body <- decodeUtf8Lenient (responseBytes response)+  narInfo <- liftEither (either (Left . T.pack) Right (parseNarInfo body))+  when (null (niSigs narInfo)) $+    throwError ("round-trip check failed: " <> object <> " came back unsigned")+  logLine "[ok]    round-trip verified, signature present"++-- ---------------------------------------------------------------------------+-- HTTP plumbing+-- ---------------------------------------------------------------------------++-- | Issue an HTTP request with optional body, converting any exception+-- into a push error.+httpRequest ::+  HTTP.Manager ->+  BS.ByteString ->+  Text ->+  [(HTTP.HeaderName, BS.ByteString)] ->+  Maybe BS.ByteString ->+  ExceptT Text IO (HTTP.Response BS.ByteString)+httpRequest manager method url headers body = ExceptT $ do+  attempt <- try $ do+    request0 <- HTTP.parseRequest (T.unpack url)+    let request =+          request0+            { HTTP.method = method,+              HTTP.requestHeaders = headers,+              HTTP.requestBody = maybe (HTTP.requestBody request0) HTTP.RequestBodyBS body+            }+    response <- HTTP.httpLbs request manager+    pure (BS.toStrict <$> response)+  pure $ case attempt of+    Left (e :: SomeException) -> Left ("HTTP error for " <> url <> ": " <> T.pack (show e))+    Right response -> Right response++-- | Authorization headers for authenticated writes.+authHeaders :: PushConfig -> [(HTTP.HeaderName, BS.ByteString)]+authHeaders cfg = case pcApiKey cfg of+  Nothing -> []+  Just key -> [("Authorization", bearerPrefix <> TE.encodeUtf8 key)]++-- | Require a 200 response, with a readable error otherwise.+expectOk :: Text -> HTTP.Response BS.ByteString -> ExceptT Text IO ()+expectOk label response = do+  let code = HTTP.statusCode (HTTP.responseStatus response)+  unless (code == httpStatusOk) $ do+    body <- decodeUtf8Lenient (responseBytes response)+    throwError (label <> " returned HTTP " <> T.pack (show code) <> ": " <> T.take 200 body)++-- | The response body bytes.+responseBytes :: HTTP.Response BS.ByteString -> BS.ByteString+responseBytes = HTTP.responseBody++-- | Decode response bytes leniently (error bodies may not be UTF-8).+decodeUtf8Lenient :: (Monad m) => BS.ByteString -> ExceptT Text m Text+decodeUtf8Lenient = pure . TE.decodeUtf8With (\_ _ -> Just '\xFFFD')++-- | Parse a reference path text from the database.+parseRefText :: Store -> Text -> Either Text StorePath+parseRefText store txt = case parseStorePath (stDir store) txt of+  Just sp -> Right sp+  Nothing -> Left ("unparseable reference in store DB: " <> txt)++-- | Progress line on stderr (stdout stays reserved for results).+logLine :: Text -> ExceptT Text IO ()+logLine = liftIO . TIO.hPutStrLn stderr

@@ -39,6 +39,7 @@     queryReferences,     queryDeriver,     queryPathInfo,+    queryAllValidPaths,      -- * Constants     metaDirName,@@ -234,6 +235,13 @@       \WHERE vp1.path = ?"       (Only pathText) ::       IO [Only Text]+  pure [p | Only p <- rows]++-- | Query every registered valid path, as full path text in registration+-- order of the table (sorted for determinism).+queryAllValidPaths :: StoreDB -> IO [Text]+queryAllValidPaths db = do+  rows <- query (sdbConn db) "SELECT path FROM ValidPaths ORDER BY path" () :: IO [Only Text]   pure [p | Only p <- rows]  -- | Query the deriver of a registered store path.

@@ -38,8 +38,9 @@ import qualified Nix.Hash as Hash import Nix.Parser (parseNix) import Nix.Parser.Lexer (Located (..), Token (..), tokenize)+import Nix.Push (computeClosure, mkNarInfo, narFileName, planMissing, storePathBasename, stripHashPrefix) import Nix.Store (Store (..), addToStore, closeStore, isValid, openStore, pathExists, scanReferences, setReadOnly, writeDrv)-import Nix.Store.DB (PathInfo (..), PathRegistration (..), closeStoreDB, isValidPath, openStoreDB, queryDeriver, queryPathInfo, queryReferences, registerPath)+import Nix.Store.DB (PathInfo (..), PathRegistration (..), closeStoreDB, isValidPath, openStoreDB, queryDeriver, queryPathInfo, queryReferences, registerPath, registerPaths) import Nix.Store.Path (StoreDir (..), StorePath (..), defaultStoreDir, parseStorePath, platformStoreDirText, storePathToFilePath, storePathToText, windowsStoreDir) import qualified Nix.Substituter as Subst import qualified NovaCache.Hash as CHash@@ -129,7 +130,7 @@ -- at known Git for Windows locations first, then PATH.  Checks known -- paths first to avoid picking up the WSL launcher at -- @C:\\Windows\\System32\\bash.exe@ which exits 1 when WSL is not--- configured.  Real Nix builders always use bash from the store —+-- configured.  Real Nix builders always use bash from the store — -- this bridges the gap until nova-nix bootstraps its own bash -- derivation. findTestShell :: IO Text@@ -225,7 +226,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Literals+-- Tests: Eval — Literals -- ---------------------------------------------------------------------------  testEvalLiterals :: IO [Bool]@@ -247,7 +248,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Variables+-- Tests: Eval — Variables -- ---------------------------------------------------------------------------  testEvalVariables :: IO [Bool]@@ -263,7 +264,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Arithmetic+-- Tests: Eval — Arithmetic -- ---------------------------------------------------------------------------  testEvalArithmetic :: IO [Bool]@@ -295,7 +296,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Comparison+-- Tests: Eval — Comparison -- ---------------------------------------------------------------------------  testEvalComparison :: IO [Bool]@@ -315,7 +316,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Logic+-- Tests: Eval — Logic -- ---------------------------------------------------------------------------  testEvalLogic :: IO [Bool]@@ -339,7 +340,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Strings+-- Tests: Eval — Strings -- ---------------------------------------------------------------------------  testEvalStrings :: IO [Bool]@@ -357,7 +358,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — If/Assert+-- Tests: Eval — If/Assert -- ---------------------------------------------------------------------------  testEvalIfAssert :: IO [Bool]@@ -375,7 +376,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Let+-- Tests: Eval — Let -- ---------------------------------------------------------------------------  testEvalLet :: IO [Bool]@@ -391,7 +392,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Attribute sets+-- Tests: Eval — Attribute sets -- ---------------------------------------------------------------------------  testEvalAttrs :: IO [Bool]@@ -413,7 +414,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Recursive attribute sets+-- Tests: Eval — Recursive attribute sets -- ---------------------------------------------------------------------------  testEvalRecAttrs :: IO [Bool]@@ -427,7 +428,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Lists+-- Tests: Eval — Lists -- ---------------------------------------------------------------------------  testEvalLists :: IO [Bool]@@ -443,7 +444,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Lambda+-- Tests: Eval — Lambda -- ---------------------------------------------------------------------------  testEvalLambda :: IO [Bool]@@ -461,7 +462,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — With+-- Tests: Eval — With -- ---------------------------------------------------------------------------  testEvalWith :: IO [Bool]@@ -526,7 +527,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Builtins+-- Tests: Eval — Builtins -- ---------------------------------------------------------------------------  testEvalBuiltins :: IO [Bool]@@ -548,7 +549,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Errors+-- Tests: Eval — Errors -- ---------------------------------------------------------------------------  testEvalErrors :: IO [Bool]@@ -564,7 +565,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Eval — Higher-order builtins+-- Tests: Eval — Higher-order builtins -- ---------------------------------------------------------------------------  testEvalHigherOrder :: IO [Bool]@@ -879,8 +880,8 @@           (EBinary OpUpdate (EVar "a") (EBinary OpUpdate (EVar "b") (EVar "c"))),       -- Lambda       -- After variable resolution, lambda-bound vars become EResolvedVar.-      -- FormalName "x" → slot 0; FormalSet [a,b] → a=0, b=1;-      -- FormalNamedSet "args" [a] → args=0, a=1.+      -- FormalName "x" → slot 0; FormalSet [a,b] → a=0, b=1;+      -- FormalNamedSet "args" [a] → args=0, a=1.       runTest "parse simple lambda" $         assertParse "lambda" "x: x" (ELambda (FormalName "x") (EResolvedVar 0 0) NoCaptureInfo),       runTest "parse set pattern lambda" $@@ -1108,7 +1109,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 1 — Trivial pure builtins + constants+-- Tests: Batch 1 — Trivial pure builtins + constants -- ---------------------------------------------------------------------------  testBatch1 :: IO [Bool]@@ -1188,7 +1189,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 2 — Arithmetic + bitwise builtins+-- Tests: Batch 2 — Arithmetic + bitwise builtins -- ---------------------------------------------------------------------------  testBatch2 :: IO [Bool]@@ -1233,7 +1234,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 3 — Attrset higher-order builtins+-- Tests: Batch 3 — Attrset higher-order builtins -- ---------------------------------------------------------------------------  testBatch3 :: IO [Bool]@@ -1272,7 +1273,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 4 — String operations+-- Tests: Batch 4 — String operations -- ---------------------------------------------------------------------------  testBatch4 :: IO [Bool]@@ -1316,7 +1317,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 5 — Serialization + hashing+-- Tests: Batch 5 — Serialization + hashing -- ---------------------------------------------------------------------------  testBatch5 :: IO [Bool]@@ -1369,7 +1370,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 6 — tryEval + deepSeq+-- Tests: Batch 6 — tryEval + deepSeq -- ---------------------------------------------------------------------------  testBatch6 :: IO [Bool]@@ -1399,7 +1400,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch 7 — genericClosure+-- Tests: Batch 7 — genericClosure -- ---------------------------------------------------------------------------  testBatch7 :: IO [Bool]@@ -1459,7 +1460,7 @@     st <- newEvalState baseDir     runEvalIO st (eval (builtinEnv (esTimestamp st) (esSearchPaths st)) expr) --- | Run a named IO eval test — single label, no double-wrapping.+-- | Run a named IO eval test — single label, no double-wrapping. runTestIO :: Text -> FilePath -> Text -> NixValue -> IO Bool runTestIO label baseDir source expected = do   result <- evalNixIO baseDir source@@ -1546,7 +1547,7 @@       ]  -- ------------------------------------------------------------------------------ Tests: Batch A — getEnv, currentTime, toPath+-- Tests: Batch A — getEnv, currentTime, toPath -- ---------------------------------------------------------------------------  testBatchA :: IO [Bool]@@ -1579,7 +1580,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch A — IO tests (getEnv)+-- Tests: Batch A — IO tests (getEnv) -- ---------------------------------------------------------------------------  testBatchAIO :: IO [Bool]@@ -1611,7 +1612,7 @@       ]  -- ------------------------------------------------------------------------------ Tests: Batch B — placeholder, storePath+-- Tests: Batch B — placeholder, storePath -- ---------------------------------------------------------------------------  testBatchB :: IO [Bool]@@ -1645,7 +1646,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch C — findFile+-- Tests: Batch C — findFile -- ---------------------------------------------------------------------------  testBatchC :: IO [Bool]@@ -1708,7 +1709,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch D — toFile+-- Tests: Batch D — toFile -- ---------------------------------------------------------------------------  testBatchD :: IO [Bool]@@ -1724,7 +1725,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch E — scopedImport+-- Tests: Batch E — scopedImport -- ---------------------------------------------------------------------------  testBatchE :: IO [Bool]@@ -1762,7 +1763,7 @@       ]  -- ------------------------------------------------------------------------------ Tests: Batch F — fetchurl, fetchTarball, fetchGit+-- Tests: Batch F — fetchurl, fetchTarball, fetchGit -- ---------------------------------------------------------------------------  testBatchF :: IO [Bool]@@ -1784,7 +1785,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch G — ATerm serialization+-- Tests: Batch G — ATerm serialization -- ---------------------------------------------------------------------------  testBatchG :: IO [Bool]@@ -1858,7 +1859,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Batch H — derivation+-- Tests: Batch H — derivation -- ---------------------------------------------------------------------------  testBatchH :: IO [Bool]@@ -2343,8 +2344,7 @@           NarInfo.niReferences = [],           NarInfo.niDeriver = Nothing,           NarInfo.niSigs = [],-          NarInfo.niCA = Nothing,-          NarInfo.niSystem = Nothing+          NarInfo.niCA = Nothing         }  -- ---------------------------------------------------------------------------@@ -2626,6 +2626,87 @@           (parseStorePath windowsStoreDir "C:\\nix\\store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-pkg")     ] +-- ---------------------------------------------------------------------------+-- Tests: Push (pure narinfo construction + closure computation)+-- ---------------------------------------------------------------------------++testPushPure :: IO [Bool]+testPushPure = do+  putStrLn "push/narinfo"+  let hashA = T.replicate 32 "a"+      hashB = T.replicate 32 "b"+      spA = StorePath hashA "hello-1.0"+      spB = StorePath hashB "dep-2.0"+      narHash = "sha256:0123abcdef"+      -- References deliberately unsorted; deriver present.+      ni = mkNarInfo spA narHash 1234 [spB, spA] (Just spB)+  sequence+    [ runTest "narinfo StorePath is canonical /nix/store" $+        assertEqual "StorePath" ("/nix/store/" <> hashA <> "-hello-1.0") (NarInfo.niStorePath ni),+      runTest "narinfo URL is nar/<digest>.nar" $+        assertEqual "URL" "nar/0123abcdef.nar" (NarInfo.niUrl ni),+      runTest "compression none mirrors NAR fields into file fields" $+        assertEqual+          "file fields"+          ("none", NarInfo.niNarHash ni, NarInfo.niNarSize ni)+          (NarInfo.niCompression ni, NarInfo.niFileHash ni, NarInfo.niFileSize ni),+      runTest "narinfo sizes carry through" $+        assertEqual "NarSize" 1234 (NarInfo.niNarSize ni),+      runTest "references are sorted basenames" $+        assertEqual+          "References"+          [hashA <> "-hello-1.0", hashB <> "-dep-2.0"]+          (NarInfo.niReferences ni),+      runTest "deriver renders as a basename" $+        assertEqual "Deriver" (Just (hashB <> "-dep-2.0")) (NarInfo.niDeriver ni),+      runTest "no client-side signatures" $+        assertEqual "Sigs" ([] :: [Text]) (NarInfo.niSigs ni),+      runTest "planMissing keeps only uncached paths" $+        assertEqual+          "missing"+          [hashB]+          (map spHash (planMissing (Set.singleton hashA) [spA, spB])),+      runTest "stripHashPrefix drops the algo tag" $+        assertEqual "tagged" "deadbeef" (stripHashPrefix "sha256:deadbeef"),+      runTest "stripHashPrefix tolerates a bare digest" $+        assertEqual "bare" "deadbeef" (stripHashPrefix "deadbeef"),+      runTest "storePathBasename joins hash and name" $+        assertEqual "basename" (hashA <> "-hello-1.0") (storePathBasename spA),+      runTest "narFileName appends .nar" $+        assertEqual "file" "0123abcdef.nar" (narFileName narHash)+    ]++-- | Closure computation against a real (temp) store database.+testPushClosureIO :: IO [Bool]+testPushClosureIO = do+  putStrLn "push/closure"+  withTempStore $ \store -> do+    let spTop = StorePath (T.replicate 32 "1") "top"+        spMid = StorePath (T.replicate 32 "2") "mid"+        spLeaf = StorePath (T.replicate 32 "3") "leaf"+        regFor sp refs =+          PathRegistration+            { prPath = sp,+              prNarHash = "sha256:" <> T.replicate 52 "0",+              prNarSize = 1,+              prDeriver = Nothing,+              prReferences = refs+            }+    registerPaths (stDB store) [regFor spTop [spMid], regFor spMid [spLeaf], regFor spLeaf []]+    fromTop <- computeClosure store [spTop]+    fromBoth <- computeClosure store [spTop, spLeaf]+    sequence+      [ runTest "closure walks transitive references" $+          assertRight "fromTop" fromTop $ \closure ->+            assertEqual+              "hashes"+              (Set.fromList (map spHash [spTop, spMid, spLeaf]))+              (Set.fromList (map spHash closure)),+        runTest "closure deduplicates across roots" $+          assertRight "fromBoth" fromBoth $ \closure ->+            assertEqual "count" (3 :: Int) (length closure)+      ]+ -- | Helper: create a fresh temp store for IO tests. withTempStore :: (Store -> IO [Bool]) -> IO [Bool] withTempStore action = do@@ -2662,7 +2743,7 @@     Dir.setPermissions path (Dir.setOwnerWritable True perms)  -- | Step 1 of the Windows-stdenv ladder: prove the Builder can run a--- trivial derivation end-to-end natively — a recipe that writes to @$out@,+-- trivial derivation end-to-end natively — a recipe that writes to @$out@, -- with the output landing in the store.  No stdenv, no dependencies: just -- the raw build path (process spawn -> output capture -> addToStore), -- exercised on the host platform (@cmd.exe@ on Windows, @\/bin\/sh@ elsewhere).@@ -2729,6 +2810,57 @@               Fail ("build failed (exit " <> T.pack (show code) <> "): " <> msg)           ] +-- | The builder hands every build a fixed SOURCE_DATE_EPOCH (1980-01-01),+-- the reproducible-builds.org convention that determinism-aware tools+-- (ld's PE timestamp, gcc's __DATE__) honor.  Proven behaviorally: a build+-- that echoes the variable into $out must see the pinned value.+testSourceDateEpochIO :: IO [Bool]+testSourceDateEpochIO = do+  putStrLn "builder/source-date-epoch"+  withTempStore $ \store -> do+    let storeDir = stDir store+        outPath = StorePath (T.replicate 31 "0" <> "5") "sde-probe"+        (builder, args)+          | SI.os == "mingw32" = ("cmd.exe", ["/c", "echo %SOURCE_DATE_EPOCH%>%out%"])+          | otherwise = ("/bin/sh", ["-c", "echo $SOURCE_DATE_EPOCH > $out"])+        drv =+          Derivation+            { drvOutputs =+                [ DerivationOutput+                    { doName = "out",+                      doPath = outPath,+                      doHashAlgo = "",+                      doHash = ""+                    }+                ],+              drvInputDrvs = Map.empty,+              drvInputSrcs = [],+              drvPlatform = currentPlatform,+              drvBuilder = builder,+              drvArgs = args,+              drvEnv = Map.empty+            }+    buildTmp <- (</> "nova-nix-test-sde-build-tmp") <$> getTemporaryDirectory+    forceRemoveIfExists buildTmp+    createDirectoryIfMissing True buildTmp+    let config = (defaultBuildConfig storeDir) {bcTmpDir = buildTmp}+    result <- buildDerivation config store drv+    forceRemoveIfExists buildTmp+    case result of+      BuildFailure msg code ->+        sequence+          [ runTest "SOURCE_DATE_EPOCH build succeeds" $+              Fail ("build failed (exit " <> T.pack (show code) <> "): " <> msg)+          ]+      BuildSuccess sp -> do+        content <- TIO.readFile (storePathToFilePath storeDir sp)+        sequence+          [ runTest "builder pins SOURCE_DATE_EPOCH to 1980-01-01" $+              if "315532800" `T.isInfixOf` content+                then Pass+                else Fail ("unexpected content: " <> T.pack (show content))+          ]+ -- | Zstd compression level for test archives (zstd's own default). testZstdCompressionLevel :: Int testZstdCompressionLevel = 3@@ -2904,7 +3036,7 @@ -- | Step 2 of the ladder: a dependency-aware build end-to-end.  A root -- derivation depends on a leaf; both @.drv@ files are pre-written to the store -- (as the build driver's closure-writing does), and 'buildWithDeps' must read--- the closure, topologically order it, build the leaf first, then the root —+-- the closure, topologically order it, build the leaf first, then the root — -- whose 'validateInputs' requires the leaf's realized output to be valid. -- -- This is the regression guard for the input-@.drv@-closure fix: before it, no@@ -2973,7 +3105,7 @@           ]  -- | Regression tests for the 2026-06-09 audit eval-fidelity fixes.  All are--- parity-safe — none affects a derivation or store-path hash.+-- parity-safe — none affects a derivation or store-path hash. testEvalFidelity :: IO [Bool] testEvalFidelity = do   putStrLn "eval/audit-fidelity"@@ -3024,11 +3156,11 @@         assertEval "leading-dot-type" "builtins.typeOf .5" (mkStr "float"),       runTest "leading-dot float value" $         assertEval "leading-dot-val" ".5 == 0.5" (VBool True),-      -- PureEval tryEval must NOT catch abort — it propagates (matches C++ Nix)+      -- PureEval tryEval must NOT catch abort — it propagates (matches C++ Nix)       runTest "tryEval does not catch abort" $         assertEvalFail "tryeval-abort" "(builtins.tryEval (builtins.abort \"x\")).success",       -- every builtin in the registry (the source of builtinNames/builtinArity)-      -- is actually exposed in the builtins set — guards against builtinRegistry+      -- is actually exposed in the builtins set — guards against builtinRegistry       -- drifting from the exposed builtins       runTest "every registered builtin is exposed" $         let missing = [n | n <- builtinNames, evalNix ("builtins ? \"" <> n <> "\"") /= Right (VBool True)]@@ -3098,7 +3230,7 @@           createDirectoryIfMissing True scanDir           let candidate = StorePath "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" "dep1"               prefix = unStoreDir sd-              -- Only 20 chars of hash — should not match 32-char candidate+              -- Only 20 chars of hash — should not match 32-char candidate               partialRef = prefix <> "/aaaaaaaaaaaaaaaaaaaa"           BS.writeFile (scanDir </> "output.txt") (TE.encodeUtf8 (T.pack partialRef))           refs <- scanReferences sd [candidate] scanDir@@ -3312,9 +3444,9 @@                   then Pass                   else Fail ("output names: " <> T.pack (show names))           _ -> Fail ("expected VDerivation, got " <> T.pack (show val)),-      -- builtinDerivation populates drvEnv with the output paths ($out, …)+      -- builtinDerivation populates drvEnv with the output paths ($out, …)       -- and the build attributes.  Note: the .drv env does NOT contain a-      -- "drvPath" key — matching C++ Nix, which never writes one.+      -- "drvPath" key — matching C++ Nix, which never writes one.       runTest "builtinDerivation populates drvEnv"         $ assertRight           "drvEnv"@@ -3337,7 +3469,7 @@  -- | Create a minimal Derivation for builder tests. -- The shell path is discovered once via 'findTestShell' and threaded through.--- All scripts are POSIX shell — bash is used on every platform.+-- All scripts are POSIX shell — bash is used on every platform. mkTestBuildDrv :: Text -> StorePath -> Text -> Derivation mkTestBuildDrv shell outSP script =   Derivation@@ -3690,7 +3822,7 @@     ]  -- ------------------------------------------------------------------------------ Tests: Phase 4 — search paths, dynamic keys, directory import+-- Tests: Phase 4 — search paths, dynamic keys, directory import -- ---------------------------------------------------------------------------  testPhase4 :: IO [Bool]@@ -3829,7 +3961,7 @@ -- ---------------------------------------------------------------------------  -- | Cast a StablePtr to CThunkPtr for CAttrSet tests.--- CAttrSet stores void* — we use StablePtrs as opaque values in tests.+-- CAttrSet stores void* — we use StablePtrs as opaque values in tests. spToCPtr :: StablePtr a -> CThunkPtr spToCPtr = castPtr . castStablePtrToPtr @@ -3988,7 +4120,7 @@ testCThunk = do   putStrLn "cthunk"   -- Arena is already initialized by main's bracket.-  -- Each test uses the shared arena (thunks accumulate — that's fine).+  -- Each test uses the shared arena (thunks accumulate — that's fine).   sequence     [ runTestM "new pending + state" $ do         sp <- newStablePtr ("pending" :: Text)@@ -4282,7 +4414,7 @@         op <- cbcOpcode idx         count <- cbcShortArg idx         dataOff <- cbcArg1 idx-        -- 3 parts × 2 words each = 6 data words+        -- 3 parts × 2 words each = 6 data words         tag0 <- cbcData dataOff         _val0 <- cbcData (dataOff + 1)         tag1 <- cbcData (dataOff + 2)@@ -4629,6 +4761,7 @@           testStorePaths,           testDerivation,           testTrivialBuildIO,+          testSourceDateEpochIO,           testUnpackBuildIO,           testDependentBuildIO,           testEvalFidelity,@@ -4680,6 +4813,8 @@           testDrvContext,           testDepGraph,           testSubstituter,+          testPushPure,+          testPushClosureIO,           testBuildOrchestrator,           testStoreDB,           testParseStorePath,