nova-nix 0.3.0.0 → 0.4.0.0

raw patch · 31 files changed

+2076/−591 lines, 31 filesdep +tardep +zstdPVP ok

version bump matches the API change (PVP)

Dependencies added: tar, zstd

API changes (from Hackage documentation)

Files

• CHANGELOG.md +11/−0
• README.md +22/−6
• app/Main.hs +77/−8
• cbits/nn_arena.c +10/−57
• cbits/nn_attrset.c +4/−0
• cbits/nn_attrset.h +9/−4
• cbits/nn_env.c +12/−0
• cbits/nn_thunk.c +42/−0
• cbits/nn_thunk.h +10/−1
• data/nix/fetchurl.nix +22/−8
• nova-nix.cabal +10/−3
• src/Nix/Builder.hs +205/−36
• src/Nix/Builder/Unpack.hs +369/−0
• src/Nix/Eval.hs +108/−86
• src/Nix/Eval/CAttrSet.hs +0/−17
• src/Nix/Eval/CBytecode.hs +61/−59
• src/Nix/Eval/CEnv.hs +6/−3
• src/Nix/Eval/CThunk.hs +2/−2
• src/Nix/Eval/Compile.hs +56/−48
• src/Nix/Eval/IO.hs +29/−11
• src/Nix/Eval/Operator.hs +35/−11
• src/Nix/Eval/StringInterp.hs +20/−18
• src/Nix/Eval/Types.hs +101/−33
• src/Nix/Hash.hs +32/−26
• src/Nix/Parser.hs +5/−4
• src/Nix/Parser/Expr.hs +5/−1
• src/Nix/Parser/Lexer.hs +79/−36
• src/Nix/Store.hs +81/−17
• src/Nix/Store/DB.hs +38/−17
• src/Nix/Substituter.hs +118/−41
• test/Main.hs +497/−38

@@ -1,5 +1,16 @@ # Changelog +## 0.4.0.0 — 2026-06-10++### Milestone: The First Native Windows Build++`nova-nix build pkgs/windows/hello.nix` compiles and runs a C program through a Nix derivation, natively on Windows: evaluation records the derivation closure, the builder realizes 17 fixed-output fetches and a toolchain unpack in dependency order, and a compiler that is itself a store path produces `C:\nix\store\…-hello\hello.exe`. The first package built natively on Windows by a Nix implementation.++- **New: `builtin:unpack`** — an in-process archive extractor, the counterpart to `builtin:fetchurl`: the unpacker is the thing being bootstrapped, so it cannot be a store tool, and ambient `tar` is unpinned. Extracts `.tar.zst`/`.tar` archives listed in `srcs` into the single `out` output, in order. MSYS2 packages share the `mingw64/` prefix, so a package set merges into one working toolchain root. Symlink and hardlink entries are materialized as copies (store outputs must not require Developer Mode), pacman per-package metadata is skipped, and cross-archive file collisions and path traversal fail the build loudly. MSYS2's zstd frames carry no pledged content size, so decompression is streaming. New dependencies: `tar >= 0.7.1`, `zstd`.+- **New: `pkgs/windows/seed.nix`** — stage 0 of the native Windows stdenv: the 17-package runtime closure of `mingw-w64-x86_64-gcc`, sha256-pinned against `repo.msys2.org`, fetched and merged into a single toolchain root. `pkgs/windows/hello.nix` is the proof build; its recipe also documents the Windows DLL rule (a subprocess resolves DLLs via `PATH`, not RPATH, so dependencies' `bin` dirs go on the build `PATH`).+- **Fix: the build driver materializes eval-coerced source paths** — `src = ./file` produced only the source store path *text* during evaluation (eval performs no store writes — the parity runner's store is not writable), so input validation failed on the missing path. The driver now copies each coerced source into the store, marks it read-only, and registers it with its real NAR hash before building.+- **`data/nix/fetchurl.nix` documentation rewritten as original prose** — the functional content is fixed byte-for-byte by derivation-hash parity and is unchanged (CI-verified).+ ## 0.3.0.0 — 2026-06-06  ### Milestone: Derivation-Hash Parity with Upstream Nix

@@ -14,15 +14,31 @@  ## Status -nova-nix evaluates real nixpkgs. `import <nixpkgs> {}` resolves the top-level package set (~23,000 attributes) to weak head normal form, and a package evaluates through the stdenv bootstrap down to a derivation:+nova-nix builds natively on Windows. The toolchain is itself a store path (fetched, hash-verified, and unpacked through derivations), and the builder runs it directly:  ```console+$ nova-nix build pkgs\windows\hello.nix+  [build]  mingw-w64-x86_64-gcc-16.1.0-5-any.pkg.tar.zst+  ...      (17 fixed-output fetches, one per MSYS2 package)+  [build]  mingw-w64-seed+  [build]  hello+C:\nix\store\4lv3zydln7y7wg0znf56dfmzf5s59h9g-hello++$ C:\nix\store\4lv3zydln7y7wg0znf56dfmzf5s59h9g-hello\hello.exe+Hello from the first native Windows Nix build.+```++The seed (`pkgs/windows/seed.nix`) is stage 0 of a native Windows stdenv: the sha256-pinned runtime closure of MinGW-w64 GCC, merged into one toolchain root by in-process fetch and unpack builtins.++It also evaluates real nixpkgs. `import <nixpkgs> {}` resolves the top-level package set (~23,000 attributes) to weak head normal form, and a package evaluates through the stdenv bootstrap down to a derivation:++```console $ NIX_PATH=nixpkgs=/path/to/nixpkgs \     nova-nix eval --expr '(import <nixpkgs> { system = "x86_64-linux"; }).hello.drvPath' "/nix/store/gciipqhqkdlqqn803zd4a389v86ran45-hello-2.12.1.drv" ``` -That `drvPath`, and the 253-derivation closure behind it, byte-matches upstream `nix-instantiate` — verified in CI on the same nixpkgs tree. It targets the Nix 2.24 language. Evaluation and derivation-hash parity are done; *building* on Windows is the work ahead.+That `drvPath`, and the 253-derivation closure behind it, byte-matches upstream `nix-instantiate` — verified in CI on the same nixpkgs tree. It targets the Nix 2.24 language.  ## Quickstart @@ -49,7 +65,7 @@ Six layers — Haskell for logic, C99 for data:  1. **Parser** (`Nix.Parser`, `Nix.Expr`) — hand-rolled recursive descent; the full Nix grammar to a 19-constructor AST.-2. **Evaluator** (`Nix.Eval`) — the AST compiles to a flat 24-opcode bytecode that a lazy, thunk-memoizing evaluator runs. Recursive `let`/`rec` are knot-tied; reference cycles are caught by blackhole detection. The evaluator is polymorphic over its effect via `MonadEval` — `PureEval` for tests, `EvalIO` for the real thing.+2. **Evaluator** (`Nix.Eval`) — the AST compiles to a flat 24-opcode bytecode that a lazy, thunk-memoizing evaluator runs. Recursive `let`/`rec` are knot-tied; reference cycles are caught by blackhole detection. The evaluator is polymorphic over its effect via `MonadEval`: `PureEval` for tests, `EvalIO` for the real thing. 3. **Data layer** (`cbits/nn_*.c`) — nine arena-allocated C99 modules (interned symbols, sorted attrsets, thunks, environments, lists, context strings, bytecode, lambdas) hold evaluation data off the GHC heap. Haskell calls C to create and query it; C never calls back. 4. **Store** (`Nix.Store`) — content-addressed `/nix/store` (`C:\nix\store` on Windows) with SQLite metadata and reference scanning. 5. **Builder** (`Nix.Builder`) — dependency graph, topological sort, and binary-cache substitution before local builds.@@ -69,13 +85,13 @@  ## Roadmap -**Done** — parser, lazy bytecode evaluator, the Nix `builtins` set, the C99 data layer, content-addressed store, derivation builder, binary-cache substituter, `import <nixpkgs> {}` evaluation, and derivation-hash parity with upstream Nix (`hello`'s 253-derivation closure byte-matches `nix-instantiate`).+**Done** — parser, lazy bytecode evaluator, the Nix `builtins` set, the C99 data layer, content-addressed store, derivation builder, binary-cache substituter, `import <nixpkgs> {}` evaluation, derivation-hash parity with upstream Nix (`hello`'s 253-derivation closure byte-matches `nix-instantiate`), and native Windows builds from a store-pinned MinGW-w64 toolchain (stage 0).  **Next** -- Windows stdenv — MinGW GCC + MSYS2 coreutils bootstrap for native builds.+- Windows stdenv stage 1 — a setup script and compiler wrappers over the seed; rebuild coreutils and bash from source. - Parity across more of nixpkgs — extend the byte-match check beyond `hello`'s closure.-- Substituter — XZ decompression and real NAR hashing (currently uncompressed-only, with a placeholder hash).+- Substituter — XZ decompression.  ## Library usage 

@@ -17,7 +17,9 @@ module Main (main) where  import Control.Monad (void, (>=>))+import Data.IORef (readIORef) import qualified Data.Map.Strict as Map+import Data.Maybe (catMaybes) import qualified Data.Text as T import qualified Data.Text.IO as TIO import Nix.Builder (BuildConfig (..), BuildResult (..), buildWithDeps, defaultBuildConfig)@@ -28,13 +30,14 @@ import Nix.Eval.IO (EvalState (..), newEvalState, runEvalIO) import Nix.Eval.Types (clistFromThunks, clistThunks, thunkToCPtr) import Nix.Parser (parseNix, readFileAutoEncoding)-import Nix.Store (Store, closeStore, openStore, writeDrv)+import Nix.Store (Store (..), closeStore, isValid, openStore, registerPaths, registrationFor, setReadOnly, writeDrv, writeDrvAterm) import Nix.Store.Path (StorePath, defaultStoreDir, parseStorePath, platformStoreDir, storePathToFilePath) import Paths_nova_nix (getDataDir)-import System.Directory (getCurrentDirectory, getTemporaryDirectory)+import System.Directory (canonicalizePath, copyFile, createDirectoryIfMissing, doesDirectoryExist, getCurrentDirectory, getTemporaryDirectory, listDirectory) import System.Environment (getArgs) import System.Exit (exitFailure) import System.FilePath (takeDirectory)+import qualified System.FilePath as FP import System.IO (BufferMode (..), hPutStrLn, hSetBuffering, stderr, stdout)  -- ---------------------------------------------------------------------------@@ -119,8 +122,13 @@       exitFailure  -- | Evaluate a .nix file and print the result.+--+-- The file argument is canonicalized first: relative path literals inside the+-- file resolve against the file's directory ('esBaseDir'), and a relative base+-- dir would be re-prefixed on every resolution (doubling it). evalFile :: Bool -> [T.Text] -> FilePath -> FilePath -> IO ()-evalFile strict extraPaths dataDir filePath = do+evalFile strict extraPaths dataDir rawFilePath = do+  filePath <- canonicalizePath rawFilePath   source <- readFileAutoEncoding filePath   case parseNix (T.pack filePath) source of     Left err -> do@@ -191,8 +199,10 @@           TIO.putStrLn (toATerm drv)  -- | Parse, evaluate, extract derivation, build, and print result.+-- The file argument is canonicalized for the same reason as in 'evalFile'. buildFile :: [T.Text] -> FilePath -> FilePath -> IO ()-buildFile extraPaths dataDir filePath = do+buildFile extraPaths dataDir rawFilePath = do+  filePath <- canonicalizePath rawFilePath   source <- readFileAutoEncoding filePath   case parseNix (T.pack filePath) source of     Left err -> do@@ -219,8 +229,17 @@           exitFailure         Right val -> do           (drv, drvSP) <- extractDerivation val+          -- The full .drv closure (root + every transitive input) recorded+          -- during evaluation; written to the store before building.+          drvClosure <- readIORef (esDrvClosure st)+          sourceCache <- readIORef (esSourcePathCache st)           store <- openStore platformStoreDir-          buildResult <- buildAndRegister store drv drvSP+          -- Materialize eval-coerced source paths (src = ./file, path+          -- interpolation): evaluation computes their store paths as text+          -- only — the parity runner's store is not writable — so the build+          -- driver performs the copy and registration.+          materializeEvalSources store sourceCache+          buildResult <- buildAndRegister store drvClosure drv drvSP           closeStore store           case buildResult of             BuildSuccess sp ->@@ -266,9 +285,14 @@ -- | Write the .drv file to the store and build with dependency resolution. -- The drvPath is the store path of the .drv file itself, extracted from -- the evaluation result alongside the Derivation struct.-buildAndRegister :: Store -> Derivation -> StorePath -> IO BuildResult-buildAndRegister store drv drvSP = do-  -- Write the .drv file to store at its content-addressed path+buildAndRegister :: Store -> Map.Map T.Text T.Text -> Derivation -> StorePath -> IO BuildResult+buildAndRegister store drvClosure drv drvSP = do+  -- Materialize the full input-.drv closure (every transitive dependency's+  -- recipe) to the store.  buildWithDeps reads these back to construct the+  -- dependency graph; without them it cannot realize any non-leaf derivation.+  writeDrvClosure store drvClosure+  -- Write the root .drv too (idempotent — it is also in the closure) so a build+  -- still works if the closure was not captured (e.g. a pre-built store drv).   writeDrv store drv drvSP   -- Build with dependency resolution   tmpDir <- getTemporaryDirectory@@ -277,6 +301,51 @@           { bcTmpDir = tmpDir           }   buildWithDeps config store drv drvSP++-- | Copy eval-coerced source paths into the store and register them.  The+-- evaluator's source-path cache maps each coerced filesystem path to its+-- @source@ fixed-output store path (text only — eval performs no store+-- writes).  Each entry not already valid is copied in, made read-only, and+-- registered with its real NAR hash.  A copied source carries no references.+materializeEvalSources :: Store -> Map.Map T.Text T.Text -> IO ()+materializeEvalSources store sourceCache = do+  regs <- catMaybes <$> mapM adopt (Map.toList sourceCache)+  registerPaths (stDB store) regs+  where+    adopt (rawPath, spText) =+      case parseStorePath defaultStoreDir spText of+        Nothing -> pure Nothing+        Just sp -> do+          valid <- isValid store sp+          if valid+            then pure Nothing+            else do+              let dest = storePathToFilePath platformStoreDir sp+              copyPathInto (T.unpack rawPath) dest+              setReadOnly dest+              Just <$> registrationFor store sp Nothing []++-- | Recursively copy a file or directory tree to a destination path.+copyPathInto :: FilePath -> FilePath -> IO ()+copyPathInto src dest = do+  isDir <- doesDirectoryExist src+  if isDir+    then do+      createDirectoryIfMissing True dest+      names <- listDirectory src+      mapM_ (\name -> copyPathInto (src FP.</> name) (dest FP.</> name)) names+    else copyFile src dest++-- | Write every recorded @.drv@ ATerm (keyed by its store-path text) to the+-- store.  Keys come from evaluation via 'storePathToText' so they always parse;+-- an unparseable key is skipped defensively.+writeDrvClosure :: Store -> Map.Map T.Text T.Text -> IO ()+writeDrvClosure store = mapM_ writeOne . Map.toList+  where+    writeOne (pathText, aterm) =+      case parseStorePath defaultStoreDir pathText of+        Just sp -> writeDrvAterm store sp aterm+        Nothing -> pure ()  -- --------------------------------------------------------------------------- -- Output formatting

@@ -1,76 +1,29 @@ /*  * nn_arena.c — Batch StablePtr collection from the thunk arena.  *- * Iterates all thunks via nn_thunk_count/nn_thunk_get and identifies- * payloads that are Haskell StablePtrs (as opposed to inline scalar- * values or C pointers).- *- * After M6 bytecode integration, PENDING thunks store (bc_idx,- * StablePtr Env) — the bc_idx replaces the Expr, but the Env stays- * as a StablePtr for knot-tying laziness.+ * The Haskell teardown (Nix.Eval.Arena) needs every Haskell StablePtr the C+ * arena holds so it can free them.  These two entry points delegate to the+ * single-pass block-list walk in nn_thunk.c (nn_thunk_{count,collect}_+ * stableptrs), which is O(total) — index-based iteration via nn_thunk_get+ * would be O(total * blocks), since each get re-walks the block list.  *- * StablePtr sources:- *   1. PENDING/BLACKHOLE: payload = StablePtr Env (all pending thunks)+ * StablePtr sources (see nn_thunk.c):+ *   1. PENDING/BLACKHOLE: payload = StablePtr Env (knot-tying laziness)  *   2. COMPUTED with val_tag == NN_VALUE_PTR: StablePtr NixValue- *- * All other COMPUTED payloads (inline scalars, C pointers) are NOT- * StablePtrs and must NOT be freed.+ * All other COMPUTED payloads (inline scalars, C pointers) are NOT freed.  */  #include "nn_arena.h"-#include "nn_attrset.h" #include "nn_thunk.h" -#include <stddef.h>- uint32_t nn_arena_stableptr_count(void) {-    uint32_t total = nn_thunk_count();-    uint32_t count = 0;-    uint32_t i;--    for (i = 0; i < total; i++) {-        nn_thunk_t *t = nn_thunk_get(i);-        if (!t) continue;--        uint8_t state = nn_thunk_state(t);-        if (state == NN_THUNK_PENDING || state == NN_THUNK_BLACKHOLE) {-            void *p = nn_thunk_payload(t);-            if (p) count++;-        } else if (state == NN_THUNK_COMPUTED) {-            uint8_t vtag = nn_thunk_value_tag(t);-            if (vtag == NN_VALUE_PTR) {-                void *p = nn_thunk_payload(t);-                if (p) count++;-            }-        }-    }-    return count;+    return nn_thunk_count_stableptrs(); }  uint32_t nn_arena_collect_stableptrs(void **output, uint32_t max_count) {-    uint32_t total = nn_thunk_count();-    uint32_t written = 0;-    uint32_t i;--    for (i = 0; i < total && written < max_count; i++) {-        nn_thunk_t *t = nn_thunk_get(i);-        if (!t) continue;--        uint8_t state = nn_thunk_state(t);-        if (state == NN_THUNK_PENDING || state == NN_THUNK_BLACKHOLE) {-            void *p = nn_thunk_payload(t);-            if (p) output[written++] = p;-        } else if (state == NN_THUNK_COMPUTED) {-            uint8_t vtag = nn_thunk_value_tag(t);-            if (vtag == NN_VALUE_PTR) {-                void *p = nn_thunk_payload(t);-                if (p) output[written++] = p;-            }-        }-    }-    return written;+    return nn_thunk_collect_stableptrs(output, max_count); }

@@ -307,6 +307,9 @@  nn_attrset_t *nn_attrset_union(const nn_attrset_t *a, const nn_attrset_t *b) {+    /* The merge-join below assumes both inputs are sorted; every set this+     * module produces is frozen (hence sorted), so guard it in debug. */+    NN_ASSERT(a->frozen && b->frozen, "nn_attrset_union: inputs must be frozen");     /* Merge-join: all keys from both, b wins on conflict. */     uint32_t cap = a->count + b->count;     nn_attrset_t *result = nn_attrset_new(cap);@@ -345,6 +348,7 @@     const nn_symbol_t *keys,     uint32_t key_count) {+    NN_ASSERT(set->frozen, "nn_attrset_remove_keys: input must be frozen");     nn_attrset_t *result = nn_attrset_new(set->count);     if (!result) { fprintf(stderr, "nn_attrset_remove_keys: alloc failed\n"); abort(); }     uint32_t i;

@@ -16,7 +16,10 @@  * Values are opaque void* — Haskell passes StablePtr Thunk.  The C  * side never dereferences them.  *- * Lifecycle: caller owns the nn_attrset_t* and must call nn_attrset_free.+ * Lifecycle: arena-managed.  Every set is tracked at creation and bulk-freed+ * by nn_attrset_free_all at evaluation teardown; nn_attrset_free is that+ * teardown's per-set primitive and must not be called on a tracked set (it+ * would then be double-freed by nn_attrset_free_all).  * Values (StablePtrs) are NOT freed by nn_attrset_free — Haskell owns them.  */ @@ -95,11 +98,13 @@  /* Create a new set that is the right-biased union of `a` and `b`  * (keys in `b` override `a`).  Both inputs must be frozen.- * The result is returned frozen.  Caller owns the result. */+ * The result is returned frozen and is arena-tracked (freed by+ * nn_attrset_free_all) — do NOT pass it to nn_attrset_free. */ nn_attrset_t *nn_attrset_union(const nn_attrset_t *a, const nn_attrset_t *b); -/* Create a new set with the given keys removed.- * Input must be frozen.  Result is returned frozen. */+/* Create a new set with the given keys removed.  Input must be frozen.+ * The result is returned frozen and is arena-tracked (freed by+ * nn_attrset_free_all) — do NOT pass it to nn_attrset_free. */ nn_attrset_t *nn_attrset_remove_keys(     const nn_attrset_t *set,     const nn_symbol_t *keys,

@@ -251,12 +251,24 @@ nn_env_lookup_resolved(const nn_env_t *env, int level, int idx) {     NN_ASSERT(env != NULL, "nn_env_lookup_resolved: NULL env");+    if (env == NULL) {+        return NULL;+    }     while (level > 0) {         NN_ASSERT(env->parent != NULL, "nn_env_lookup_resolved: parent chain too short for level");+        if (env->parent == NULL) {+            return NULL;+        }         env = env->parent;         level--;     }+    /* The resolver supplies these indices; guard the slot read in release+       builds too (NDEBUG strips the assert) so a resolver bug surfaces as a+       NULL rather than an out-of-bounds read past the slot array. */     NN_ASSERT(idx >= 0 && (uint32_t)idx < env->slot_count, "nn_env_lookup_resolved: idx out of bounds");+    if (idx < 0 || (uint32_t)idx >= env->slot_count) {+        return NULL;+    }     return env->slots[idx]; } 

@@ -530,3 +530,45 @@     }     return NULL;  /* unreachable if index < total */ }++/* --- Bulk StablePtr collection (single block-list pass, O(total)) --- */++uint32_t+nn_thunk_count_stableptrs(void)+{+    uint32_t count = 0;+    struct nn_thunk_block *block;+    uint32_t i;+    if (!g_arena) return 0;+    for (block = g_arena->head; block; block = block->next) {+        for (i = 0; i < block->count; i++) {+            const nn_thunk_t *t = &block->slots[i];+            if (t->state == NN_THUNK_PENDING || t->state == NN_THUNK_BLACKHOLE) {+                if (t->payload) count++;+            } else if (t->state == NN_THUNK_COMPUTED && t->val_tag == NN_VALUE_PTR) {+                if (t->payload) count++;+            }+        }+    }+    return count;+}++uint32_t+nn_thunk_collect_stableptrs(void **output, uint32_t max_count)+{+    uint32_t written = 0;+    struct nn_thunk_block *block;+    uint32_t i;+    if (!g_arena) return 0;+    for (block = g_arena->head; block && written < max_count; block = block->next) {+        for (i = 0; i < block->count && written < max_count; i++) {+            const nn_thunk_t *t = &block->slots[i];+            if (t->state == NN_THUNK_PENDING || t->state == NN_THUNK_BLACKHOLE) {+                if (t->payload) output[written++] = t->payload;+            } else if (t->state == NN_THUNK_COMPUTED && t->val_tag == NN_VALUE_PTR) {+                if (t->payload) output[written++] = t->payload;+            }+        }+    }+    return written;+}

@@ -68,7 +68,7 @@     uint8_t   state;     uint8_t   val_tag;     uint16_t  _pad;-    uint32_t  bc_idx;    /* PENDING: bytecode index; COMPUTED: 0 */+    uint32_t  bc_idx;    /* PENDING: bytecode index; otherwise stale (never read) */     void     *payload; } nn_thunk_t; @@ -196,5 +196,14 @@  * Used for StablePtr cleanup iteration before nn_thunk_destroy().  * Index must be < nn_thunk_count(). */ nn_thunk_t *nn_thunk_get(uint32_t index);++/* Single-pass (O(total)) collection of Haskell StablePtr payloads across the+ * whole arena.  Walks the block list directly — nn_thunk_get is O(blocks) per+ * call, so index-based iteration would be O(total * blocks).  StablePtr+ * sources: PENDING/BLACKHOLE payload, and COMPUTED + NN_VALUE_PTR payload.+ * nn_thunk_collect_stableptrs writes up to max_count pointers into `output`+ * and returns the count written.  These back the nn_arena_* teardown helpers. */+uint32_t nn_thunk_count_stableptrs(void);+uint32_t nn_thunk_collect_stableptrs(void **output, uint32_t max_count);  #endif /* NN_THUNK_H */

@@ -1,12 +1,19 @@-# Nix built-in fetchurl — creates a fixed-output derivation that downloads a URL.-# This file is normally shipped with the Nix daemon as <nix/fetchurl.nix>.-# nova-nix bundles it so that nixpkgs bootstrap works without a system Nix install.+# nova-nix's implementation of the <nix/fetchurl.nix> interface.+#+# Nix exposes a built-in fetcher expression under the search-path entry+# <nix/fetchurl.nix>; nixpkgs' bootstrap imports it directly, so nova-nix+# must provide one with the exact same interface and derivation output.+# Every attribute below feeds the .drv ATerm, so the produced derivation+# (and therefore its store path) byte-matches upstream Nix — this is+# enforced by the parity CI job.  The argument names, default chains, and+# derivation attributes are all fixed by that contract; only fetch one+# thing at a time and let the fixed-output hash do the verifying. {-  system ? "", # obsolete+  system ? "", # ignored; kept for interface compatibility   url,   hash ? "", # an SRI hash -  # Legacy hash specification+  # Pre-SRI hash arguments, still accepted everywhere in nixpkgs.   md5 ? "",   sha1 ? "",   sha256 ? "",@@ -42,9 +49,13 @@  derivation (   {+    # Handled in-process by the Builder, not spawned as a subprocess+    # (see Nix.Builder.runBuiltinFetchurl).     builder = "builtin:fetchurl"; -    # New-style output content requirements.+    # An unpacked or executable output is a tree, so its fixed-output+    # hash must be taken over the NAR serialisation rather than the+    # flat file bytes.     outputHashMode = if unpack || executable then "recursive" else "flat";      inherit@@ -56,9 +67,11 @@      system = "builtin"; -    # No need to double the amount of network traffic+    # The fetch is cheaper than copying the result between machines.     preferLocalBuild = true; +    # Proxy configuration may legitimately vary between hosts without+    # changing what gets fetched, so it is allowed through the env.     impureEnvVars = [       "http_proxy"       "https_proxy"@@ -67,7 +80,8 @@       "no_proxy"     ]; -    # To make "nix-prefetch-url" work.+    # Tooling that prefetches (nix-prefetch-url and friends) reads the+    # candidate URLs from this attribute.     urls = [ url ];   }   // (if impure then { __impure = true; } else { inherit outputHashAlgo outputHash; })

@@ -1,6 +1,6 @@ cabal-version:      3.0 name:               nova-nix-version:            0.3.0.0+version:            0.4.0.0 synopsis:           Windows-native Nix implementation in Haskell and C99 description:   A from-scratch implementation of the Nix package manager that runs@@ -8,8 +8,9 @@   Haskell evaluator handles parsing and lazy evaluation, backed by a C99   data layer that keeps evaluation data off the GHC heap.  It evaluates   real Nix expressions (including nixpkgs), computes derivations and-  content-addressed store paths that byte-match upstream Nix, and includes-  a derivation builder and binary-cache substituter.+  content-addressed store paths that byte-match upstream Nix, and builds+  packages natively on Windows from a store-pinned MinGW-w64 toolchain,+  with a derivation builder and binary-cache substituter.    Built on @nova-cache@ for NAR serialization, narinfo handling, and   Ed25519-signed binary substitution.@@ -74,6 +75,7 @@     Nix.DependencyGraph     Nix.Derivation     Nix.Builder+    Nix.Builder.Unpack     Nix.Substituter     Nix.Builtins     Nix.Hash@@ -95,8 +97,10 @@     , process             >= 1.6 && < 1.7     , regex-tdfa          >= 1.3 && < 1.4     , sqlite-simple       >= 0.4 && < 0.5+    , tar                 >= 0.7.1 && < 0.8     , text                >= 2.0 && < 2.2     , time                >= 1.9 && < 1.15+    , zstd                >= 0.1 && < 0.2    c-sources:     cbits/nn_symbol.c@@ -160,9 +164,12 @@     , containers          >= 0.6 && < 0.8     , directory           >= 1.3 && < 1.4     , filepath            >= 1.4 && < 1.6+    , nova-cache          >= 0.3.0 && < 0.4     , nova-nix     , process             >= 1.6 && < 1.7+    , tar                 >= 0.7.1 && < 0.8     , text                >= 2.0 && < 2.2+    , zstd                >= 0.1 && < 0.2  source-repository head   type:     git

@@ -44,16 +44,25 @@  import Control.Exception (SomeException, try) import Control.Monad (filterM, when)+import qualified Data.ByteString as BS+import qualified Data.ByteString.Lazy as LBS import Data.Char (toLower)+import Data.Either (fromRight) import Data.Map.Strict (Map) import qualified Data.Map.Strict as Map+import Data.Maybe (catMaybes) import Data.Text (Text) import qualified Data.Text as T import qualified Data.Text.IO as TIO+import qualified Network.HTTP.Client as HTTP+import qualified Network.HTTP.Client.TLS as HTTPS+import qualified Network.HTTP.Types.Status as HTTP+import Nix.Builder.Unpack (builtinUnpackBuilder, runBuiltinUnpack) import Nix.DependencyGraph (DepGraph, TopoResult (..), buildDepGraph, topoSort) import qualified Nix.DependencyGraph import Nix.Derivation (Derivation (..), DerivationOutput (..), fromATerm)-import Nix.Store (Store (..), addToStore, isValid, scanReferences)+import Nix.Hash (bytesToHexText, hexToBytes, rawHashWithAlgo)+import Nix.Store (PathRegistration, Store (..), isValid, placeInStore, registerPaths, registrationFor, scanReferences, scanTempReferences) import Nix.Store.Path (StoreDir (..), StorePath (..), storePathToFilePath) import Nix.Substituter (CacheConfig, SubstResult (..), trySubstitute) import System.Directory (createDirectoryIfMissing, doesDirectoryExist, doesPathExist, removeDirectoryRecursive)@@ -85,6 +94,21 @@ envPath :: Text envPath = "PATH" +-- | The magic builder string for the built-in URL fetcher.  A derivation with+-- this builder is not executed as a process — the Builder downloads its @url@+-- and verifies it against @outputHash@ (see 'runBuiltinFetchurl').+builtinFetchurlBuilder :: Text+builtinFetchurlBuilder = "builtin:fetchurl"++-- | Derivation environment keys read by @builtin:fetchurl@.+envUrl, envOut :: Text+envUrl = "url"+envOut = "out"++-- | HTTP success status code.+httpStatusOk :: Int+httpStatusOk = 200+ -- --------------------------------------------------------------------------- -- Configuration -- ---------------------------------------------------------------------------@@ -152,8 +176,8 @@  buildDerivationInner :: BuildConfig -> Store -> Derivation -> IO BuildResult buildDerivationInner config store drv = do-  -- 1. Validate input sources exist-  inputsOk <- validateInputs store drv+  -- 1. Validate inputs (sources + input-derivation outputs) exist+  inputsOk <- validateInputs config store drv   case inputsOk of     Left errMsg -> pure (BuildFailure errMsg 1)     Right () -> do@@ -171,7 +195,14 @@            -- 5. Run the builder           builderArgs = map T.unpack (drvArgs drv)-      exitResult <- runBuilder builderPath builderArgs environ buildDir+      -- A "builtin:" builder (e.g. builtin:fetchurl) is handled in-process;+      -- everything else is a normal subprocess.  The output validation and+      -- registration below are shared by both paths.+      exitResult <- case drvBuilder drv of+        b+          | b == builtinFetchurlBuilder -> runBuiltinFetchurl drv outputDirs+          | b == builtinUnpackBuilder -> runBuiltinUnpack drv outputDirs+        _ -> runBuilder builderPath builderArgs environ buildDir       case exitResult of         Left (exitCode, stderrText) -> do           -- 6. Failure: clean up@@ -196,23 +227,47 @@ -- Input validation -- --------------------------------------------------------------------------- --- | Check that all input sources and input derivation outputs are valid.-validateInputs :: Store -> Derivation -> IO (Either Text ())-validateInputs store drv = do+-- | Check that all inputs needed to build a derivation are present in the store.+--+-- Input sources must be valid.  For input derivations, the artifacts a build+-- actually reads are their realized OUTPUT paths, not the @.drv@ recipe files,+-- so we resolve each requested output (by reading the input @.drv@ the build+-- driver materialized) and validate those outputs.  Dependencies are realized+-- in topological order, so a dependency's outputs are present by the time a+-- dependent is validated.+validateInputs :: BuildConfig -> Store -> Derivation -> IO (Either Text ())+validateInputs config store drv = do   -- Check input sources   srcResults <- mapM (isValid store) (drvInputSrcs drv)   let missingSrcs = [sp | (sp, valid) <- zip (drvInputSrcs drv) srcResults, not valid]   if not (null missingSrcs)     then pure (Left ("missing input sources: " <> T.intercalate ", " (map formatSP missingSrcs)))-    else do-      -- Check input derivation outputs-      let inputDrvPaths = Map.keys (drvInputDrvs drv)-      drvResults <- mapM (isValid store) inputDrvPaths-      let missingDrvs = [sp | (sp, valid) <- zip inputDrvPaths drvResults, not valid]-      if not (null missingDrvs)-        then pure (Left ("missing input derivations: " <> T.intercalate ", " (map formatSP missingDrvs)))-        else pure (Right ())+    else case resolveInputOutputs config drv of+      Left err -> pure (Left err)+      Right requiredOutputs -> do+        outResults <- mapM (isValid store) requiredOutputs+        let missingOutputs = [sp | (sp, valid) <- zip requiredOutputs outResults, not valid]+        if not (null missingOutputs)+          then pure (Left ("missing input derivation outputs: " <> T.intercalate ", " (map formatSP missingOutputs)))+          else pure (Right ()) +-- | Resolve each input derivation's requested output names to their store+-- paths by reading the input @.drv@ from the store.  @Left@ if an input @.drv@+-- cannot be read or names an output the derivation does not define.+resolveInputOutputs :: BuildConfig -> Derivation -> Either Text [StorePath]+resolveInputOutputs config drv =+  concat <$> traverse resolveOne (Map.toList (drvInputDrvs drv))+  where+    resolveOne (inputDrvPath, wantedOutputs) = do+      inputDrv <- readDrvFromStore config inputDrvPath+      let outputMap = Map.fromList [(doName o, doPath o) | o <- drvOutputs inputDrv]+      traverse (lookupOutput inputDrvPath outputMap) wantedOutputs+    lookupOutput inputDrvPath outputMap name =+      case Map.lookup name outputMap of+        Just sp -> Right sp+        Nothing ->+          Left ("input derivation " <> formatSP inputDrvPath <> " has no output '" <> name <> "'")+ -- | Format a StorePath for error messages. formatSP :: StorePath -> Text formatSP sp = spHash sp <> "-" <> spName sp@@ -374,10 +429,100 @@ isWindows = System.Info.os == "mingw32"  -- ---------------------------------------------------------------------------+-- Built-in fetcher (builtin:fetchurl)+-- ---------------------------------------------------------------------------++-- | Run a @builtin:fetchurl@ derivation: download its @url@ into @$out@ and+-- verify the bytes against the derivation's @outputHash@.  Nix's bootstrap+-- fetcher is baked into the binary because nothing can be fetched before a+-- fetcher exists.  Returns the same @Either (exit, msg) ()@ shape as+-- 'runBuilder', so the shared output-registration path is reused unchanged.+runBuiltinFetchurl :: Derivation -> [(Text, FilePath)] -> IO (Either (Int, Text) ())+runBuiltinFetchurl drv outputDirs =+  case (Map.lookup envUrl (drvEnv drv), lookup envOut outputDirs, fixedOutput) of+    (Nothing, _, _) -> pure (Left (1, "builtin:fetchurl: derivation has no 'url'"))+    (_, Nothing, _) -> pure (Left (1, "builtin:fetchurl: derivation defines no 'out' output"))+    (_, _, Nothing) -> pure (Left (1, "builtin:fetchurl: 'out' output has no fixed-output hash"))+    (Just url, Just outPath, Just out) -> do+      downloaded <- downloadUrl url+      case downloaded of+        Left err -> pure (Left (1, "builtin:fetchurl: " <> err))+        Right body -> do+          BS.writeFile outPath body+          pure (verifyFetchHash url out body)+  where+    -- builtin:fetchurl derivations are always fixed-output; the expected hash+    -- lives in the canonical output spec (doHashAlgo + doHash), not the env.+    fixedOutput = case drvOutputs drv of+      (out : _) | not (T.null (doHashAlgo out)) -> Just out+      _ -> Nothing++-- | Download a URL to a strict 'BS.ByteString' using nova-nix's own linked+-- HTTP client (the same 'Network.HTTP.Client' the substituter uses) — no+-- external @curl@, which is what makes this a genuine builtin.  Any network+-- exception is turned into a 'Left' so it becomes a clean build failure.+downloadUrl :: Text -> IO (Either Text BS.ByteString)+downloadUrl url = do+  attempt <- try fetch+  pure $ case attempt of+    Left (e :: SomeException) -> Left ("download error: " <> T.pack (show e))+    Right result -> result+  where+    fetch :: IO (Either Text BS.ByteString)+    fetch = do+      manager <- HTTP.newManager HTTPS.tlsManagerSettings+      request <- HTTP.parseRequest (T.unpack url)+      response <- HTTP.httpLbs request manager+      pure (bodyOrError response)+    bodyOrError response+      | code == httpStatusOk = Right (LBS.toStrict (HTTP.responseBody response))+      | otherwise = Left ("HTTP " <> T.pack (show code) <> " fetching " <> url)+      where+        code = HTTP.statusCode (HTTP.responseStatus response)++-- | Verify the fetched bytes against the derivation's fixed-output hash, read+-- from the canonical output spec.  @doHashAlgo@ is @sha256@/@sha512@/… (or+-- @r:sha256@ for recursive); @doHash@ is the base-16 digest eval normalized the+-- user's hash into.  Flat mode is fully supported across algorithms; recursive+-- (unpack/executable) fetches are a separate feature — they require unpacking+-- the download — and fail with a clear message rather than a wrong result.+verifyFetchHash :: Text -> DerivationOutput -> BS.ByteString -> Either (Int, Text) ()+verifyFetchHash url out body+  | recursive =+      Left (1, "builtin:fetchurl: recursive outputHashMode (unpack/executable) not yet supported; fetch flat and unpack in a build phase")+  | otherwise = case (hexToBytes (doHash out), rawHashWithAlgo algo body) of+      (Nothing, _) -> Left (1, "builtin:fetchurl: malformed expected hash for " <> url)+      (_, Nothing) -> Left (1, "builtin:fetchurl: unsupported hash algorithm '" <> algo <> "'")+      (Just expected, Just got)+        | expected == got -> Right ()+        | otherwise ->+            Left+              ( 1,+                "builtin:fetchurl: hash mismatch for "+                  <> url+                  <> "\n  expected: "+                  <> algoField+                  <> ":"+                  <> doHash out+                  <> "\n  got:      "+                  <> algoField+                  <> ":"+                  <> bytesToHexText got+              )+  where+    algoField = doHashAlgo out+    (recursive, algo) = case T.stripPrefix "r:" algoField of+      Just rest -> (True, rest)+      Nothing -> (False, algoField)++-- --------------------------------------------------------------------------- -- Output registration -- --------------------------------------------------------------------------- --- | After a successful build, register each output in the store.+-- | After a successful build, register every output in the store.+--+-- Outputs are placed first and registered together (one transaction) so+-- intra-derivation cross-output references survive (see 'registerPaths'). registerOutputs ::   BuildConfig ->   Store ->@@ -386,47 +531,71 @@   [(Text, FilePath)] ->   IO BuildResult registerOutputs config store drv _buildDir outputDirs = do-  let allCandidates = collectAllCandidates drv+  let -- Candidates for reference scanning: input sources, the input+      -- derivations' realized OUTPUT paths, and this derivation's own outputs.+      inputOutputs = fromRight [] (resolveInputOutputs config drv)+      allCandidates = collectAllCandidates drv ++ inputOutputs       -- Deriver path is not available from the Derivation type alone;       -- the caller (buildWithDeps) would need to pass it through.       -- Register with no deriver for now — queryDeriver will return Nothing.       drvPathText = Nothing-  -- Register each output-  results <- mapM (registerSingleOutput config store allCandidates drvPathText) (zip (drvOutputs drv) outputDirs)-  case sequence results of+      -- (temp output dir, final store path) for every output, used to detect+      -- self- and cross-output references that appear as build-temp paths.+      tempPairs = [(outDir, doPath out) | (out, (_, outDir)) <- zip (drvOutputs drv) outputDirs]+  -- Phase 1: scan references and move each output into the store, collecting a+  -- PathRegistration (no DB writes yet).  Already-valid outputs are skipped.+  prepared <- mapM (prepareOutput config store allCandidates tempPairs drvPathText) (zip (drvOutputs drv) outputDirs)+  case sequence prepared of     Left errMsg -> pure (BuildFailure errMsg 1)-    Right _ ->-      -- Return the first output's store path+    Right regs -> do+      -- Phase 2: register ALL outputs in one transaction.+      registerPaths (stDB store) (catMaybes regs)       case drvOutputs drv of         (firstOut : _) -> pure (BuildSuccess (doPath firstOut))         [] -> pure (BuildFailure "no outputs defined" 1) --- | Register a single output: scan references, move to store, register in DB.-registerSingleOutput ::+-- | Prepare a single output for registration: skip if already valid in the DB,+-- otherwise scan references (input refs plus self/cross-output temp refs) and+-- place it in the store, returning its 'PathRegistration'.  Returns @Nothing@+-- for an output already registered as valid.+prepareOutput ::   BuildConfig ->   Store ->   [StorePath] ->+  [(FilePath, StorePath)] ->   Maybe Text ->   (DerivationOutput, (Text, FilePath)) ->-  IO (Either Text ())-registerSingleOutput config store candidates drvPathText (output, (_outName, outDir)) = do+  IO (Either Text (Maybe PathRegistration))+prepareOutput config store candidates tempPairs drvPathText (output, (_outName, outDir)) = do   let targetSP = doPath output       targetPath = storePathToFilePath (bcStoreDir config) targetSP-  -- Check if output exists (file or directory — builder creates it)+  -- Check the build actually produced the output (file or directory).   exists <- doesPathExist outDir   if not exists     then pure (Left ("output missing: " <> T.pack outDir))     else do-      -- Scan for references in the output-      refs <- scanReferences (bcStoreDir config) candidates outDir-      -- Check if already in store (e.g. from a previous build)-      alreadyExists <- doesPathExist targetPath-      if alreadyExists-        then pure (Right ())+      -- Validity is a DB fact, not mere disk presence: an output left on disk by+      -- an interrupted build is NOT valid and must be (re-)registered.+      valid <- isValid store targetSP+      if valid+        then pure (Right Nothing)         else do-          -- Move to store and register-          addToStore store outDir targetSP drvPathText refs-          pure (Right ())+          onDisk <- doesPathExist targetPath+          -- Scan whichever artifact we will register: a leftover store path if+          -- present, otherwise the fresh build output.+          let scanPath = if onDisk then targetPath else outDir+          inputRefs <- scanReferences (bcStoreDir config) candidates scanPath+          selfRefs <- scanTempReferences tempPairs scanPath+          let refs = dedupStorePaths (inputRefs ++ selfRefs)+          reg <-+            if onDisk+              then registrationFor store targetSP drvPathText refs+              else placeInStore store outDir targetSP drvPathText refs+          pure (Right (Just reg))++-- | Deduplicate store paths by their (unique) hash.+dedupStorePaths :: [StorePath] -> [StorePath]+dedupStorePaths = Map.elems . Map.fromList . map (\sp -> (spHash sp, sp))  -- | Collect all candidate store paths from the derivation's inputs. -- Used for reference scanning.

@@ -0,0 +1,369 @@+{-# LANGUAGE ScopedTypeVariables #-}++-- | __builtin:unpack__ — in-process archive extraction for the bootstrap seed.+--+-- == Why this is a builtin+--+-- The Windows stdenv bootstrap fetches pre-built toolchain tarballs (MSYS2+-- @.pkg.tar.zst@ packages) into the store before any toolchain exists there.+-- Nothing in the store can unpack them — the unpacker is the thing being+-- bootstrapped — and ambient @tar.exe@ is unpinned and may lack zstd support.+-- So, like 'Nix.Builder.runBuiltinFetchurl', extraction runs in-process:+-- a derivation whose @builder@ is @builtin:unpack@ is handled by this module+-- instead of being spawned as a subprocess.+--+-- == Derivation interface+--+-- * @srcs@ — whitespace-separated archive store paths, extracted in order+--   into the single @out@ output.  MSYS2 packages share a top-level+--   @mingw64\/@ prefix, so extracting a package set into one output yields a+--   working toolchain root directly (no separate union step).+-- * @out@ — the merged tree.+--+-- == Determinism+--+-- Extraction is a pure function of the archive bytes: entries are written in+-- archive order, a file appearing in two archives is an error (pacman+-- enforces the same no-conflict invariant), and pacman's per-package+-- metadata entries (@.PKGINFO@, @.MTREE@, …) are skipped — they are not part+-- of the installed tree and would otherwise collide across packages.+--+-- Symlink and hardlink entries are materialized by __copying__ their target:+-- symlinks on Windows require elevation or Developer Mode, so a link in the+-- store would make the output machine-dependent.  Copying is deterministic+-- everywhere at a small size cost.+module Nix.Builder.Unpack+  ( -- * Builder name+    builtinUnpackBuilder,++    -- * Derivation environment keys+    envSrcs,++    -- * Running+    runBuiltinUnpack,+  )+where++import qualified Codec.Archive.Tar as Tar+import qualified Codec.Archive.Tar.Entry as TarEntry+import qualified Codec.Compression.Zstd.Lazy as Zstd+import Control.Exception (SomeException, try)+import Control.Monad (when)+import Data.Bits ((.&.))+import qualified Data.ByteString.Lazy as BL+import Data.Char (toLower)+import Data.List (isPrefixOf, isSuffixOf)+import qualified Data.Map.Strict as Map+import Data.Text (Text)+import qualified Data.Text as T+import Nix.Derivation (Derivation (..))+import System.Directory+  ( copyFile,+    createDirectoryIfMissing,+    doesDirectoryExist,+    doesFileExist,+    doesPathExist,+    getPermissions,+    listDirectory,+    setOwnerExecutable,+    setPermissions,+  )+import System.FilePath (isAbsolute, joinPath, splitDirectories, takeDirectory, (</>))+import qualified System.Info++-- ---------------------------------------------------------------------------+-- Named constants+-- ---------------------------------------------------------------------------++-- | The magic builder string for the built-in archive extractor.  A+-- derivation with this builder is not executed as a process — the Builder+-- extracts its @srcs@ archives into @$out@ (see 'runBuiltinUnpack').+builtinUnpackBuilder :: Text+builtinUnpackBuilder = "builtin:unpack"++-- | Derivation environment key holding the whitespace-separated archive+-- store paths to extract.  Store paths never contain whitespace (store-name+-- validation rejects it), so splitting on words is unambiguous.+envSrcs :: Text+envSrcs = "srcs"++-- | Name of the output the merged tree is extracted into.+unpackOutputName :: Text+unpackOutputName = "out"++-- | Owner-execute bit of a tar header's mode field (octal @0o100@).+ownerExecuteMode :: TarEntry.Permissions+ownerExecuteMode = 0o100++-- | Pax per-file extended header ('x') and global header ('g') type codes.+-- 'Tar.decodeLongNames' already applies pax path\/linkpath overrides; any+-- header entries still present carry only metadata we do not record (mtime,+-- xattrs), so they are skipped rather than rejected.+paxPerFileCode, paxGlobalCode :: Char+paxPerFileCode = 'x'+paxGlobalCode = 'g'++-- | True on a Windows host.  NTFS has no executable bit (PATHEXT decides),+-- so tar mode bits are only materialized on Unix.+isWindowsHost :: Bool+isWindowsHost = System.Info.os == "mingw32"++-- ---------------------------------------------------------------------------+-- Entry types after long-name decoding+-- ---------------------------------------------------------------------------++-- | Entries as produced by 'Tar.decodeLongNames': GNU\/pax long names are+-- resolved, so paths and link targets are plain 'FilePath's.+type DecodedEntries =+  Tar.GenEntries BL.ByteString FilePath FilePath DecodeError++-- | A single decoded entry.+type DecodedEntry = Tar.GenEntry BL.ByteString FilePath FilePath++-- | Format errors from 'Tar.read' or long-name decoding.+type DecodeError = Either Tar.FormatError Tar.DecodeLongNamesError++-- ---------------------------------------------------------------------------+-- Entry point+-- ---------------------------------------------------------------------------++-- | Run a @builtin:unpack@ derivation: extract every archive listed in its+-- @srcs@ env into the @out@ output directory.  Returns the same+-- @Either (exit, msg) ()@ shape as the process runner, so the shared+-- output-validation and registration path in "Nix.Builder" is reused+-- unchanged.+runBuiltinUnpack :: Derivation -> [(Text, FilePath)] -> IO (Either (Int, Text) ())+runBuiltinUnpack drv outputDirs =+  case (Map.lookup envSrcs (drvEnv drv), lookup unpackOutputName outputDirs) of+    (Nothing, _) -> failure "derivation has no 'srcs'"+    (Just srcs, _)+      | null (sourcePaths srcs) -> failure "'srcs' is empty"+    (_, Nothing) -> failure "derivation defines no 'out' output"+    (Just srcs, Just outDir) -> do+      createDirectoryIfMissing True outDir+      result <- unpackAll outDir (sourcePaths srcs)+      pure $ case result of+        Left msg -> Left (1, "builtin:unpack: " <> msg)+        Right () -> Right ()+  where+    failure msg = pure (Left (1, "builtin:unpack: " <> msg))+    sourcePaths = map T.unpack . T.words++-- | Extract archives in order, stopping at the first failure.+unpackAll :: FilePath -> [FilePath] -> IO (Either Text ())+unpackAll _ [] = pure (Right ())+unpackAll outDir (archive : rest) = do+  result <- unpackArchive outDir archive+  case result of+    Left err -> pure (Left err)+    Right () -> unpackAll outDir rest++-- | Unpack one archive into @outDir@.  The decompressor is chosen by file+-- extension; the tar stream is decoded (GNU + pax long names) and extracted+-- entry by entry.  Decompression errors surface lazily mid-stream, so the+-- whole extraction is exception-wrapped into a clean failure.+unpackArchive :: FilePath -> FilePath -> IO (Either Text ())+unpackArchive outDir archivePath = do+  attempt <- try run+  pure $ case attempt of+    Left (e :: SomeException) ->+      Left (T.pack archivePath <> ": " <> T.pack (show e))+    Right result -> result+  where+    run = case decoderFor archivePath of+      Nothing -> pure (Left ("unsupported archive format: " <> T.pack archivePath))+      Just decoder -> do+        raw <- BL.readFile archivePath+        extractEntries outDir (Tar.decodeLongNames (Tar.read (decoder raw)))++-- | Choose a decompressor from the archive file name.  @.tar.zst@ (MSYS2+-- packages) and plain @.tar@ are supported.  MSYS2's zstd frames do not+-- pledge a content size, so the lazy (streaming) zstd decoder is required —+-- the strict single-shot API rejects them.+decoderFor :: FilePath -> Maybe (BL.ByteString -> BL.ByteString)+decoderFor path+  | ".tar.zst" `isSuffixOf` lowered = Just Zstd.decompress+  | ".tar" `isSuffixOf` lowered = Just id+  | otherwise = Nothing+  where+    lowered = map toLower path++-- ---------------------------------------------------------------------------+-- Entry extraction+-- ---------------------------------------------------------------------------++-- | Walk the entry stream, extracting each entry under @outDir@.+extractEntries :: FilePath -> DecodedEntries -> IO (Either Text ())+extractEntries outDir = go+  where+    go stream = case stream of+      Tar.Done -> pure (Right ())+      Tar.Fail err -> pure (Left ("malformed archive: " <> T.pack (show err)))+      Tar.Next entry rest -> do+        result <- extractEntry outDir entry+        case result of+          Left err -> pure (Left err)+          Right () -> go rest++-- | Extract a single entry, after validating its path stays inside the+-- archive root and skipping pacman package metadata.+extractEntry :: FilePath -> DecodedEntry -> IO (Either Text ())+extractEntry outDir entry =+  case entryComponents (TarEntry.entryTarPath entry) of+    Left err -> pure (Left err)+    -- The archive root itself (an entry for "." or "./").+    Right [] -> pure (Right ())+    Right comps+      | isPackageMetadata comps -> pure (Right ())+      | otherwise -> extractContent outDir comps entry++-- | Extract a path-validated entry's content.  @comps@ is non-empty (the+-- empty case is consumed by 'extractEntry').+extractContent :: FilePath -> [FilePath] -> DecodedEntry -> IO (Either Text ())+extractContent outDir comps entry =+  case TarEntry.entryContent entry of+    Tar.Directory -> do+      createDirectoryIfMissing True dest+      pure (Right ())+    Tar.NormalFile bytes _size -> do+      fresh <- freshDestination dest+      case fresh of+        Left err -> pure (Left err)+        Right () -> do+          createDirectoryIfMissing True (takeDirectory dest)+          BL.writeFile dest bytes+          when (executableEntry entry && not isWindowsHost) (markExecutable dest)+          pure (Right ())+    -- A symlink target is relative to the link's own directory.+    Tar.SymbolicLink target ->+      copyLinkTarget "symlink" (resolveLinkTarget parentComps target)+    -- A hardlink target is relative to the archive root.+    Tar.HardLink target ->+      copyLinkTarget "hardlink" (entryComponents target)+    Tar.OtherEntryType code _ _+      | code == paxPerFileCode || code == paxGlobalCode -> pure (Right ())+      | otherwise ->+          pure (Left ("unsupported tar entry type '" <> T.singleton code <> "': " <> pathText))+    Tar.CharacterDevice _ _ -> unsupportedSpecial "character device"+    Tar.BlockDevice _ _ -> unsupportedSpecial "block device"+    Tar.NamedPipe -> unsupportedSpecial "named pipe"+  where+    dest = outDir </> joinPath comps+    parentComps = take (length comps - 1) comps+    pathText = T.pack (TarEntry.entryTarPath entry)+    unsupportedSpecial kind =+      pure (Left ("unsupported " <> kind <> " entry: " <> pathText))+    copyLinkTarget kind resolved = case resolved of+      Left err -> pure (Left err)+      Right targetComps -> do+        let targetPath = outDir </> joinPath targetComps+        isFile <- doesFileExist targetPath+        isDir <- doesDirectoryExist targetPath+        copyTarget kind targetPath isFile isDir+    copyTarget kind targetPath isFile isDir+      | isFile = do+          fresh <- freshDestination dest+          case fresh of+            Left err -> pure (Left err)+            Right () -> do+              createDirectoryIfMissing True (takeDirectory dest)+              copyFile targetPath dest+              pure (Right ())+      | isDir = do+          copyTree targetPath dest+          pure (Right ())+      | otherwise =+          pure+            ( Left+                ( kind+                    <> " target not present (links must follow their targets in the archive): "+                    <> pathText+                    <> " -> "+                    <> T.pack targetPath+                )+            )++-- ---------------------------------------------------------------------------+-- Path validation+-- ---------------------------------------------------------------------------++-- | Split a tar entry path into validated components: relative, no @..@, no+-- @:@ (drive letters, NTFS alternate data streams).  Tar paths use @/@;+-- 'splitDirectories' accepts both separators on Windows.  @.@ components are+-- dropped, so @./mingw64@ and @mingw64@ agree.+entryComponents :: FilePath -> Either Text [FilePath]+entryComponents raw+  | isAbsolute raw = Left ("absolute entry path: " <> T.pack raw)+  | ".." `elem` comps = Left ("entry path escapes archive root: " <> T.pack raw)+  | any (elem ':') comps = Left ("entry path contains ':': " <> T.pack raw)+  | otherwise = Right comps+  where+    comps = filter (/= ".") (splitDirectories raw)++-- | Resolve a symlink target (relative to the link's parent directory)+-- against the archive root.  @..@ components are allowed but must not climb+-- above the root.+resolveLinkTarget :: [FilePath] -> FilePath -> Either Text [FilePath]+resolveLinkTarget parentComps target+  | isAbsolute target = Left ("absolute symlink target: " <> T.pack target)+  | otherwise = walk (reverse parentComps) targetComps+  where+    targetComps = filter (/= ".") (splitDirectories target)+    walk stack remaining = case (stack, remaining) of+      (_, []) -> Right (reverse stack)+      ([], ".." : _) ->+        Left ("symlink target escapes archive root: " <> T.pack target)+      (_ : popped, ".." : rest) -> walk popped rest+      (_, comp : rest)+        | ':' `elem` comp -> Left ("symlink target contains ':': " <> T.pack target)+        | otherwise -> walk (comp : stack) rest++-- | pacman package metadata at the archive root (@.PKGINFO@, @.BUILDINFO@,+-- @.MTREE@, @.INSTALL@): describes the package to pacman, is not part of the+-- installed tree, and collides across packages when several archives merge+-- into one output — so root-level dotfile entries are skipped.+isPackageMetadata :: [FilePath] -> Bool+isPackageMetadata comps = case comps of+  [name] -> "." `isPrefixOf` name+  _ -> False++-- ---------------------------------------------------------------------------+-- Filesystem helpers+-- ---------------------------------------------------------------------------++-- | Guard that nothing was already extracted at @dest@.  Two archives in one+-- seed providing the same file is a packaging error worth failing loudly on+-- (pacman enforces the same no-conflict invariant between its packages).+freshDestination :: FilePath -> IO (Either Text ())+freshDestination dest = do+  exists <- doesPathExist dest+  pure $+    if exists+      then Left ("file collision between archives: " <> T.pack dest)+      else Right ()++-- | True when the entry's tar mode has the owner-execute bit.+executableEntry :: DecodedEntry -> Bool+executableEntry entry = TarEntry.entryPermissions entry .&. ownerExecuteMode /= 0++-- | Materialize the executable bit on Unix hosts.+markExecutable :: FilePath -> IO ()+markExecutable path = do+  perms <- getPermissions path+  setPermissions path (setOwnerExecutable True perms)++-- | Recursively copy a directory tree (used to materialize directory+-- symlinks, which cannot be store-portable links on Windows).+copyTree :: FilePath -> FilePath -> IO ()+copyTree src dest = do+  createDirectoryIfMissing True dest+  names <- listDirectory src+  mapM_ copyOne names+  where+    copyOne name = do+      let from = src </> name+          to = dest </> name+      isDir <- doesDirectoryExist from+      if isDir+        then copyTree from to+        else copyFile from to

@@ -1,4 +1,5 @@ {-# LANGUAGE LambdaCase #-}+{-# LANGUAGE PatternSynonyms #-}  -- | Nix expression evaluator. --@@ -42,7 +43,8 @@      -- * Evaluation monad (re-exported from Types)     MonadEval (..),-    PureEval (..),+    PureEval,+    runPureEval,      -- * Evaluation     eval,@@ -87,7 +89,7 @@ import Foreign.Ptr (Ptr, castPtr, nullPtr, ptrToWordPtr, wordPtrToPtr) import Foreign.Storable (peekElemOff, pokeElemOff) import Nix.Derivation (Derivation (..), DerivationOutput (..), textToPlatform, toATerm, toATermForHash)-import Nix.Eval.CBytecode (cbcArg1, cbcArg2, cbcArg3, cbcData, cbcFlags, cbcOpcode, cbcShortArg)+import Nix.Eval.CBytecode (cbcArg1, cbcArg2, cbcArg3, cbcData, cbcFlags, cbcOpcode, cbcShortArg, pattern OpApp, pattern OpAssert, pattern OpAttrs, pattern OpBinary, pattern OpHasAttr, pattern OpIf, pattern OpIndStr, pattern OpLambda, pattern OpLet, pattern OpList, pattern OpLitBool, pattern OpLitFloat, pattern OpLitInt, pattern OpLitNull, pattern OpLitPath, pattern OpLitUri, pattern OpResolvedVar, pattern OpSearchPath, pattern OpSelect, pattern OpStr, pattern OpUnary, pattern OpVar, pattern OpWith, pattern OpWithVar) import Nix.Eval.CEnv (cenvPushWith) import Nix.Eval.CList (CList (..), clistGet) import Nix.Eval.CThunk (CThunkPtr)@@ -105,7 +107,7 @@     EvalFormals (..),     MonadEval (..),     NixValue (..),-    PureEval (..),+    PureEval,     StringContext (..),     StringContextElement (..),     Thunk (..),@@ -144,6 +146,7 @@     newCEnv,     newMinimalEnv,     readThunkValue,+    runPureEval,     thunkToCPtr,     typeName,     withScopesForCapture,@@ -156,7 +159,7 @@     NixAtom (..),     UnaryOp (..),   )-import Nix.Hash (byteToHex, hashPlaceholder, makeFixedOutputPath, makeOutputPath, makeTextPath, sha256Digest, sha256Hex)+import Nix.Hash (bytesToHexText, hashPlaceholder, hexToBytes, makeFixedOutputPath, makeOutputPath, makeTextPath, sha256Digest, sha256Hex) import Nix.Store.Path (StorePath (..), defaultStoreDir, defaultStoreDirText, parseStorePath, storePathToFilePath, storePathToText) import qualified NovaCache.Base32 as Nix32 import qualified NovaCache.Base64 as B64@@ -188,45 +191,45 @@ evalBytecode env bcIdx =   let opcode = unsafePerformIO (cbcOpcode bcIdx)    in case opcode of-        0 {- LIT_INT -} ->+        OpLitInt ->           let lo = unsafePerformIO (cbcArg1 bcIdx)               hi = unsafePerformIO (cbcArg2 bcIdx)            in pure (VInt (reassembleInt64 lo hi))-        1 {- LIT_FLOAT -} ->+        OpLitFloat ->           let lo = unsafePerformIO (cbcArg1 bcIdx)               hi = unsafePerformIO (cbcArg2 bcIdx)            in pure (VFloat (reassembleDouble lo hi))-        2 {- LIT_BOOL -} ->+        OpLitBool ->           let flag = unsafePerformIO (cbcShortArg bcIdx)            in pure (VBool (flag /= 0))-        3 {- LIT_NULL -} -> pure VNull-        4 {- LIT_URI -} ->+        OpLitNull -> pure VNull+        OpLitUri ->           let sym = unsafePerformIO (cbcArg1 bcIdx)            in pure (mkStr (symbolText (Symbol sym)))-        5 {- LIT_PATH -} ->+        OpLitPath ->           let sym = unsafePerformIO (cbcArg1 bcIdx)            in VPath <$> resolvePathLiteral (symbolText (Symbol sym))-        6 {- STR -} -> evalBcStr env bcIdx-        7 {- IND_STR -} -> evalBcIndStr env bcIdx-        8 {- VAR -} ->+        OpStr -> evalBcStr env bcIdx+        OpIndStr -> evalBcIndStr env bcIdx+        OpVar ->           let sym = unsafePerformIO (cbcArg1 bcIdx)            in evalVar env (symbolText (Symbol sym))-        9 {- WITH_VAR -} ->+        OpWithVar ->           let sym = unsafePerformIO (cbcArg1 bcIdx)               name = symbolText (Symbol sym)            in evalWithVar env name-        10 {- RESOLVED_VAR -} ->+        OpResolvedVar ->           let level = fromIntegral (unsafePerformIO (cbcArg1 bcIdx))               idx = fromIntegral (unsafePerformIO (cbcArg2 bcIdx))            in force (envLookupResolved level idx env)-        11 {- ATTRS -} -> evalBcAttrs env bcIdx-        12 {- LIST -} -> evalBcList env bcIdx-        13 {- SELECT -} -> evalBcSelect env bcIdx-        14 {- HAS_ATTR -} -> evalBcHasAttr env bcIdx-        15 {- APP -} -> evalBcApp env bcIdx-        16 {- LAMBDA -} -> evalBcLambda env bcIdx-        17 {- LET -} -> evalBcLet env bcIdx-        18 {- IF -} ->+        OpAttrs -> evalBcAttrs env bcIdx+        OpList -> evalBcList env bcIdx+        OpSelect -> evalBcSelect env bcIdx+        OpHasAttr -> evalBcHasAttr env bcIdx+        OpApp -> evalBcApp env bcIdx+        OpLambda -> evalBcLambda env bcIdx+        OpLet -> evalBcLet env bcIdx+        OpIf ->           let condIdx = unsafePerformIO (cbcArg1 bcIdx)               thenIdx = unsafePerformIO (cbcArg2 bcIdx)               elseIdx = unsafePerformIO (cbcArg3 bcIdx)@@ -236,7 +239,7 @@                   VBool True -> evalBytecode env thenIdx                   VBool False -> evalBytecode env elseIdx                   _ -> throwEvalError ("'if' condition must be a Boolean, got " <> typeName condVal)-        19 {- WITH -} ->+        OpWith ->           let scopeIdx = unsafePerformIO (cbcArg1 bcIdx)               bodyIdx = unsafePerformIO (cbcArg2 bcIdx)               -- Lazy with: defer forcing the scope until a WITH_VAR lookup@@ -245,7 +248,7 @@               -- eagerly forcing the scope would blackhole.               scopeThunk = cheapThunkBc env scopeIdx            in evalBytecode (pushLazyWithScope scopeThunk env) bodyIdx-        20 {- ASSERT -} ->+        OpAssert ->           let condIdx = unsafePerformIO (cbcArg1 bcIdx)               bodyIdx = unsafePerformIO (cbcArg2 bcIdx)            in do@@ -254,14 +257,14 @@                   VBool True -> evalBytecode env bodyIdx                   VBool False -> throwEvalError "assertion failed"                   _ -> throwEvalError ("assertion condition must be a Boolean, got " <> typeName condVal)-        21 {- UNARY -} ->+        OpUnary ->           let flags_ = unsafePerformIO (cbcFlags bcIdx)               operandIdx = unsafePerformIO (cbcArg1 bcIdx)            in do                 val <- evalBytecode env operandIdx                 evalUnary (decodeUnaryOp flags_) val-        22 {- BINARY -} -> evalBcBinary env bcIdx-        23 {- SEARCH_PATH -} ->+        OpBinary -> evalBcBinary env bcIdx+        OpSearchPath ->           let sym = unsafePerformIO (cbcArg1 bcIdx)            in evalSearchPath env (symbolText (Symbol sym))         _ -> throwEvalError "evalBytecode: unknown opcode"@@ -274,6 +277,8 @@ decodeUnaryOp :: Word8 -> UnaryOp decodeUnaryOp 0 = OpNot decodeUnaryOp 1 = OpNegate+-- Unreachable: the tag is written by Nix.Eval.Compile's encodeUnaryOp, which+-- only emits 0 or 1. decodeUnaryOp n = error ("decodeUnaryOp: unknown tag " <> show n)  -- | Decode a BinaryOp from bytecode flags.@@ -293,6 +298,8 @@ decodeBinaryOp 12 = OpGte decodeBinaryOp 13 = OpConcat decodeBinaryOp 14 = OpUpdate+-- Unreachable: the tag is written by Nix.Eval.Compile's encodeBinaryOp, which+-- only emits 0..14. decodeBinaryOp n = error ("decodeBinaryOp: unknown tag " <> show n)  -- | Evaluate a binary operation from bytecode, with short-circuit@@ -331,12 +338,23 @@   -- String-like: direct concat when both are string/path   (VStr {}, VStr {}) -> evalBinary force OpAdd left right   (VStr {}, VPath _) -> evalBinary force OpAdd left right-  -- Otherwise: coerce both to strings via concatStrings (matching C++ Nix)+  -- Otherwise: string concatenation with STRICT coercion (Nix coerceMore=false)+  -- — only strings and sets with __toString/outPath coerce; numbers, bools,+  -- null, lists, and functions are type errors, matching C++ Nix's `+`.   _ -> do-    (leftStr, leftCtx) <- coerceToString force applyValue left-    (rightStr, rightCtx) <- coerceToString force applyValue right+    (leftStr, leftCtx) <- coerceAddOperand left+    (rightStr, rightCtx) <- coerceAddOperand right     pure (VStr (leftStr <> rightStr) (leftCtx <> rightCtx)) +-- | Strict string coercion for the @+@ operator (Nix @coerceMore = false@):+-- strings, and sets with @__toString@/@outPath@, coerce; numbers, booleans,+-- null, lists, and functions are type errors.+coerceAddOperand :: (MonadEval m) => NixValue -> m (Text, StringContext)+coerceAddOperand v@(VStr {}) = coerceToString False force applyValue v+coerceAddOperand v@(VAttrs {}) = coerceToString False force applyValue v+coerceAddOperand v =+  throwEvalError ("cannot coerce " <> typeName v <> " to a string with the + operator")+ -- | Bytecode short-circuit && evalShortCircuitAnd :: (MonadEval m) => Env -> Word32 -> Word32 -> m NixValue evalShortCircuitAnd env leftIdx rightIdx = do@@ -1812,8 +1830,9 @@   where     forceToStrCtx thunk = do       val <- force thunk-      -- Nix coerces list elements to strings (paths, derivations, etc.)-      coerceToString force applyValue val+      -- Nix coerces list elements to strings (paths, derivations, etc.).+      -- concatStringsSep is strict (coerceMore=False): non-string scalars error.+      coerceToString False force applyValue val builtinConcatStringsSep (VStr _ _) other =   throwEvalError ("builtins.concatStringsSep: expected a list, got " <> typeName other) builtinConcatStringsSep other _ =@@ -1840,16 +1859,18 @@   foldlStrict op stepped rest  builtinSubstring :: (MonadEval m) => NixValue -> NixValue -> NixValue -> m NixValue-builtinSubstring (VInt start) (VInt len) (VStr s ctx) =-  let clampedStart = max 0 (fromIntegral start)-      -- Nix clamps len to available length (negative len means rest of string)-      available = T.length s - clampedStart-      clampedLen =-        if len < 0-          then available-          else min (fromIntegral len) available-   in -- Context is preserved through substring (matching real Nix).-      pure (VStr (T.take clampedLen (T.drop clampedStart s)) ctx)+builtinSubstring (VInt start) (VInt len) (VStr s ctx)+  | start < 0 = throwEvalError "builtins.substring: negative start position"+  | otherwise =+      let startPos = fromIntegral start+          -- Nix clamps len to available length (negative len means rest of string)+          available = T.length s - startPos+          clampedLen =+            if len < 0+              then available+              else min (fromIntegral len) available+       in -- Context is preserved through substring (matching real Nix).+          pure (VStr (T.take clampedLen (T.drop startPos s)) ctx) builtinSubstring _ _ (VStr _ _) =   throwEvalError "builtins.substring: start and length must be integers" builtinSubstring _ _ other =@@ -1890,7 +1911,7 @@     coerceThunk thunk = do       val <- force thunk       coerceToStringPermissive val-coerceToStringPermissive other = coerceToString force applyValue other+coerceToStringPermissive other = coerceToString True force applyValue other  -- | Coerce a value to a string for a DERIVATION field (an env value or an -- arg).  Like 'coerceToStringPermissive', but a path literal is copied into@@ -1919,7 +1940,7 @@   case parseStorePath defaultStoreDir spText of     Just sp -> pure (spText, StringContext (Set.singleton (SCPlain sp)))     Nothing -> pure (spText, mempty)-coerceToStringInterp other = coerceToString force applyValue other+coerceToStringInterp other = coerceToString False force applyValue other  -- | The current system platform string. currentSystemStr :: Text@@ -2124,7 +2145,7 @@   throwEvalError ("builtins.concatLists: expected a list, got " <> typeName other)  builtinLessThan :: (MonadEval m) => NixValue -> NixValue -> m NixValue-builtinLessThan a b = VBool <$> nixCompare a b+builtinLessThan a b = VBool <$> nixCompare force a b  -- --------------------------------------------------------------------------- -- Builtin implementations — arithmetic + bitwise@@ -2171,12 +2192,12 @@  builtinMin :: (MonadEval m) => NixValue -> NixValue -> m NixValue builtinMin a b = do-  aIsLess <- nixCompare a b+  aIsLess <- nixCompare force a b   pure (if aIsLess then a else b)  builtinMax :: (MonadEval m) => NixValue -> NixValue -> m NixValue builtinMax a b = do-  aIsLess <- nixCompare a b+  aIsLess <- nixCompare force a b   pure (if aIsLess then b else a)  builtinBitAnd :: (MonadEval m) => NixValue -> NixValue -> m NixValue@@ -2416,7 +2437,10 @@           let groups = Array.elems match               -- Skip index 0 (full match) — return only capture groups.               captureGroups = drop 1 groups-              toThunk (s, _) = evaluated (mkStr (T.pack s))+              -- A non-participating capture group has offset (-1); C++ Nix+              -- yields null for it, not the empty string.+              toThunk (s, (off, _)) =+                if off < 0 then evaluated VNull else evaluated (mkStr (T.pack s))            in pure (VList (clistFromThunks (map (thunkToCPtr . toThunk) captureGroups)))  -- | @builtins.split regex str@: split a string by a POSIX ERE.@@ -2460,7 +2484,9 @@       before = T.pack (take (matchStart - pos) (drop pos remaining))       -- Capture groups (indices 1..)       groups = drop 1 (Array.elems match)-      groupThunks = map (\(s, _) -> evaluated (mkStr (T.pack s))) groups+      -- A non-participating capture group has offset (-1) → null (as 'match').+      groupThunks =+        map (\(s, (off, _)) -> if off < 0 then evaluated VNull else evaluated (mkStr (T.pack s))) groups       -- Continue after this match       afterPos = matchStart + matchLen    in evaluated (mkStr before)@@ -2480,8 +2506,17 @@  compareVersionParts :: [Text] -> [Text] -> Int64 compareVersionParts [] [] = 0-compareVersionParts [] (_ : _) = -1-compareVersionParts (_ : _) [] = 1+-- When one version runs out, Nix pads the shorter side with an empty+-- component and keeps comparing — so "1.0" > "1.0pre" (empty sorts AFTER+-- "pre"), not "<" as a naive length comparison would give.+compareVersionParts [] (b : bs) =+  case compareComponent "" b of+    EQ -> compareVersionParts [] bs+    cmp -> ordToNix cmp+compareVersionParts (a : as) [] =+  case compareComponent a "" of+    EQ -> compareVersionParts as []+    cmp -> ordToNix cmp compareVersionParts (a : as) (b : bs) =   case compareComponent a b of     EQ -> compareVersionParts as bs@@ -2605,20 +2640,19 @@       ctx = mconcat (map snd results)   pure ("[" <> T.intercalate "," jsonVals <> "]", ctx) valueToJSON (VAttrs attrs) =-  -- Derivation-like attrsets (with outPath) coerce to string in toJSON,-  -- matching C++ Nix behavior.-  case attrSetLookup "outPath" attrs of-    Just outThunk -> do-      outVal <- force outThunk-      (s, ctx) <- coerceToString force applyValue outVal-      pure (jsonEscapeString s, ctx)-    Nothing -> do+  -- C++ Nix serializes an attrset via __toString first, then outPath, and only+  -- falls back to an object when neither is present.+  case (attrSetLookup "__toString" attrs, attrSetLookup "outPath" attrs) of+    (Nothing, Nothing) -> do       let m = attrSetToMap attrs           sortedKeys = Map.keys m       results <- mapM (jsonPair m) sortedKeys       let pairs = map fst results           ctx = mconcat (map snd results)       pure ("{" <> T.intercalate "," pairs <> "}", ctx)+    _ -> do+      (s, ctx) <- coerceToString True force applyValue (VAttrs attrs)+      pure (jsonEscapeString s, ctx)   where     jsonPair attrMap key = case Map.lookup key attrMap of       Nothing -> pure ("", emptyContext)@@ -2793,9 +2827,7 @@   throwEvalError ("builtins.hashString: expected a string, got " <> typeName other)  digestToHex :: (BA.ByteArrayAccess a) => a -> Text-digestToHex digest =-  let bytes = BA.unpack digest-   in T.pack (concatMap byteToHex bytes)+digestToHex = bytesToHexText . BA.convert  -- --------------------------------------------------------------------------- -- Builtin implementations — deep evaluation@@ -3152,7 +3184,7 @@     Nothing -> throwEvalError (builtin <> ": missing required attribute '" <> key <> "'")     Just thunk -> do       val <- force thunk-      result <- catchEvalError (coerceToString force applyValue val)+      result <- catchEvalError (coerceToString True force applyValue val)       case result of         Right (s, _ctx) -> pure s         Left _ -> throwEvalError (builtin <> ": '" <> key <> "' must be a string")@@ -3164,7 +3196,7 @@     Nothing -> pure Nothing     Just thunk -> do       val <- force thunk-      result <- catchEvalError (coerceToString force applyValue val)+      result <- catchEvalError (coerceToString True force applyValue val)       case result of         Right (s, _ctx) -> pure (Just s)         Left _ -> pure Nothing@@ -3262,7 +3294,7 @@       let foPath = makeFixedOutputPath drvName foAlgo foMode foDigest           foPathText = storePathToText defaultStoreDir foPath           algoField = (if foMode == "recursive" then "r:" else "") <> foAlgo-          foHashHex = bytesToHex foDigest+          foHashHex = bytesToHexText foDigest           foModulo =             sha256Digest               (TE.encodeUtf8 ("fixed:out:" <> algoField <> ":" <> foHashHex <> ":" <> foPathText))@@ -3272,7 +3304,7 @@               (Map.insert "out" foPathText baseEnv)           drvSp = makeTextPath drvFileName (sha256Digest (TE.encodeUtf8 (toATerm contents))) drvRefs           drvText = storePathToText defaultStoreDir drvSp-      cacheDrvHash drvText (bytesToHex foModulo)+      cacheDrvHash drvText (bytesToHexText foModulo)       pure (drvText, drvSp, [("out", foPathText)], contents)     Nothing -> do       inputSubst <- mapM resolveInputModulo (Map.toList inputDrvs)@@ -3289,9 +3321,15 @@           moduloUnmasked = sha256Digest (TE.encodeUtf8 (toATermForHash False (Just inputSubst) contents))           drvSp = makeTextPath drvFileName (sha256Digest (TE.encodeUtf8 (toATerm contents))) drvRefs           drvText = storePathToText defaultStoreDir drvSp-      cacheDrvHash drvText (bytesToHex moduloUnmasked)+      cacheDrvHash drvText (bytesToHexText moduloUnmasked)       pure (drvText, drvSp, outPathTexts, contents) +  -- Record this derivation's full .drv ATerm (the exact bytes whose hash is its+  -- store path) so the build driver can materialize the entire input-.drv+  -- closure before building.  Bottom-up eval guarantees every transitive input+  -- is recorded by the time a dependent is.+  recordDrvAterm drvPathText (toATerm completeDrv)+   let mainOutPath = case outPaths of         ((_, p) : _) -> p         [] -> ""@@ -3430,7 +3468,7 @@ forceToText :: (MonadEval m) => Thunk -> m Text forceToText thunk = do   val <- force thunk-  (s, _ctx) <- coerceToString force applyValue val+  (s, _ctx) <- coerceToString True force applyValue val   pure s  -- | Collect all string-coercible attributes for the derivation environment,@@ -3504,7 +3542,7 @@   (algo, rawBytes) <- decodeHashInput attrs hashVal   -- Encode to target format   case toFmt of-    "base16" -> pure (mkStr (bytesToHex rawBytes))+    "base16" -> pure (mkStr (bytesToHexText rawBytes))     "nix32" -> pure (mkStr (Nix32.encode rawBytes))     "base32" -> pure (mkStr (Nix32.encode rawBytes)) -- deprecated alias     "base64" -> pure (mkStr (bytesToBase64 rawBytes))@@ -3561,22 +3599,6 @@       VStr s _ -> pure s       _ -> throwEvalError ("builtins." <> ctx <> ": " <> key <> " must be a string")   Nothing -> throwEvalError ("builtins." <> ctx <> ": missing required attribute '" <> key <> "'")---- | Encode bytes to base16 hex.-bytesToHex :: BS.ByteString -> Text-bytesToHex = T.pack . concatMap byteToHex . BS.unpack---- | Decode hex string to bytes.-hexToBytes :: Text -> Maybe BS.ByteString-hexToBytes t-  | T.null t = Just BS.empty-  | odd (T.length t) = Nothing-  | T.all isHexDigit t = Just (BS.pack (go (T.unpack t)))-  | otherwise = Nothing-  where-    go [] = []-    go (hi : lo : rest) = fromIntegral (digitToInt hi * 16 + digitToInt lo) : go rest-    go [_] = [] -- unreachable due to odd check  -- --------------------------------------------------------------------------- -- Base64 encode/decode — delegates to nova-cache (base64-bytestring under the hood)

@@ -17,8 +17,6 @@      -- * Lifecycle     cattrsetNew,-    cattrsetFree,-    withCAttrSet,      -- * Construction     cattrsetInsert,@@ -62,9 +60,6 @@ foreign import ccall unsafe "nn_attrset_new"   c_nn_attrset_new :: Word32 -> IO CAttrSet -foreign import ccall unsafe "nn_attrset_free"-  c_nn_attrset_free :: CAttrSet -> IO ()- foreign import ccall unsafe "nn_attrset_insert"   c_nn_attrset_insert :: CAttrSet -> Word32 -> Ptr () -> IO () @@ -105,18 +100,6 @@ -- | Allocate a new empty attribute set with the given capacity hint. cattrsetNew :: Word32 -> IO CAttrSet cattrsetNew = c_nn_attrset_new---- | Free a C-allocated attribute set.  Does NOT free thunk values.-cattrsetFree :: CAttrSet -> IO ()-cattrsetFree = c_nn_attrset_free---- | Bracket pattern: allocate, use, free.-withCAttrSet :: Word32 -> (CAttrSet -> IO a) -> IO a-withCAttrSet cap action = do-  set <- cattrsetNew cap-  result <- action set-  cattrsetFree set-  pure result  -- --------------------------------------------------------------------------- -- Construction

@@ -1,3 +1,5 @@+{-# LANGUAGE PatternSynonyms #-}+ -- | C-backed bytecode storage via FFI. -- -- Wraps @cbits/nn_bytecode.c@ — two growable arrays storing flat Nix@@ -6,7 +8,7 @@ -- -- @ -- cbcInit 0 0--- idx <- cbcEmit opLitInt 0 0 42 0 0+-- idx <- cbcEmit OpLitInt 0 0 42 0 0 -- cbcOpcode idx   -- NN_OP_LIT_INT (0) -- cbcArg1 idx     -- 42 -- cbcDestroy@@ -36,30 +38,30 @@     cbcDataCount,      -- * Opcodes-    opLitInt,-    opLitFloat,-    opLitBool,-    opLitNull,-    opLitUri,-    opLitPath,-    opStr,-    opIndStr,-    opVar,-    opWithVar,-    opResolvedVar,-    opAttrs,-    opList,-    opSelect,-    opHasAttr,-    opApp,-    opLambda,-    opLet,-    opIf,-    opWith,-    opAssert,-    opUnary,-    opBinary,-    opSearchPath,+    pattern OpLitInt,+    pattern OpLitFloat,+    pattern OpLitBool,+    pattern OpLitNull,+    pattern OpLitUri,+    pattern OpLitPath,+    pattern OpStr,+    pattern OpIndStr,+    pattern OpVar,+    pattern OpWithVar,+    pattern OpResolvedVar,+    pattern OpAttrs,+    pattern OpList,+    pattern OpSelect,+    pattern OpHasAttr,+    pattern OpApp,+    pattern OpLambda,+    pattern OpLet,+    pattern OpIf,+    pattern OpWith,+    pattern OpAssert,+    pattern OpUnary,+    pattern OpBinary,+    pattern OpSearchPath,      -- * UnaryOp flags     unaryNot,@@ -219,49 +221,49 @@ -- Opcode constants -- --------------------------------------------------------------------------- -opLitInt, opLitFloat, opLitBool, opLitNull :: Word8-opLitInt = 0-opLitFloat = 1-opLitBool = 2-opLitNull = 3+pattern OpLitInt, OpLitFloat, OpLitBool, OpLitNull :: Word8+pattern OpLitInt = 0+pattern OpLitFloat = 1+pattern OpLitBool = 2+pattern OpLitNull = 3 -opLitUri, opLitPath :: Word8-opLitUri = 4-opLitPath = 5+pattern OpLitUri, OpLitPath :: Word8+pattern OpLitUri = 4+pattern OpLitPath = 5 -opStr, opIndStr :: Word8-opStr = 6-opIndStr = 7+pattern OpStr, OpIndStr :: Word8+pattern OpStr = 6+pattern OpIndStr = 7 -opVar, opWithVar, opResolvedVar :: Word8-opVar = 8-opWithVar = 9-opResolvedVar = 10+pattern OpVar, OpWithVar, OpResolvedVar :: Word8+pattern OpVar = 8+pattern OpWithVar = 9+pattern OpResolvedVar = 10 -opAttrs, opList :: Word8-opAttrs = 11-opList = 12+pattern OpAttrs, OpList :: Word8+pattern OpAttrs = 11+pattern OpList = 12 -opSelect, opHasAttr, opApp :: Word8-opSelect = 13-opHasAttr = 14-opApp = 15+pattern OpSelect, OpHasAttr, OpApp :: Word8+pattern OpSelect = 13+pattern OpHasAttr = 14+pattern OpApp = 15 -opLambda, opLet :: Word8-opLambda = 16-opLet = 17+pattern OpLambda, OpLet :: Word8+pattern OpLambda = 16+pattern OpLet = 17 -opIf, opWith, opAssert :: Word8-opIf = 18-opWith = 19-opAssert = 20+pattern OpIf, OpWith, OpAssert :: Word8+pattern OpIf = 18+pattern OpWith = 19+pattern OpAssert = 20 -opUnary, opBinary :: Word8-opUnary = 21-opBinary = 22+pattern OpUnary, OpBinary :: Word8+pattern OpUnary = 21+pattern OpBinary = 22 -opSearchPath :: Word8-opSearchPath = 23+pattern OpSearchPath :: Word8+pattern OpSearchPath = 23  -- --------------------------------------------------------------------------- -- Sub-type flags

@@ -4,7 +4,7 @@ -- hold slot arrays, lazy scopes, parent pointers, and with-scopes. -- All memory is freed in bulk via 'cenvDestroy' at evaluation end. ----- This moves Env records off the GHC heap.  Each @nn_env_t@ is 40+-- This moves Env records off the GHC heap.  Each @nn_env_t@ is 48 -- bytes (arena-allocated, zero GC overhead), replacing the former -- Haskell record (~80-100 bytes with GHC info pointers, Maybe -- constructors, and list cons cells per env).@@ -44,6 +44,7 @@ where  import Data.Word (Word16, Word32)+import Foreign.C.Types (CInt (..)) import Foreign.Ptr (Ptr) import Nix.Eval.CThunk (CThunkPtr) @@ -109,7 +110,7 @@   c_nn_env_with_count :: Ptr NnEnv -> IO Word16  foreign import ccall unsafe "nn_env_lookup_resolved"-  c_nn_env_lookup_resolved :: Ptr NnEnv -> Int -> Int -> IO CThunkPtr+  c_nn_env_lookup_resolved :: Ptr NnEnv -> CInt -> CInt -> IO CThunkPtr  foreign import ccall unsafe "nn_env_root_scope"   c_nn_env_root_scope :: Ptr NnEnv -> IO (Ptr ())@@ -205,7 +206,9 @@ -- | Resolved variable lookup: walk @level@ parent hops, then read -- @slots[idx]@.  Single FFI call — O(level) in C. cenvLookupResolved :: Ptr NnEnv -> Int -> Int -> IO CThunkPtr-cenvLookupResolved = c_nn_env_lookup_resolved+cenvLookupResolved env level idx =+  -- C @int@ params: convert at the boundary (CInt), not HsInt (64-bit).+  c_nn_env_lookup_resolved env (fromIntegral level) (fromIntegral idx)  -- | Walk parent chain to root, return root's lazy scope (NULL if none). cenvRootScope :: Ptr NnEnv -> IO (Ptr ())

@@ -81,7 +81,7 @@  import Data.Int (Int64) import Data.Word (Word32, Word8)-import Foreign.C.Types (CDouble (..))+import Foreign.C.Types (CDouble (..), CInt (..)) import Foreign.Ptr (Ptr, nullPtr)  -- | Phantom type for C-side @nn_thunk_t@.@@ -182,7 +182,7 @@   c_nn_thunk_get_lambda :: CThunkPtr -> IO (Ptr ())  foreign import ccall unsafe "nn_thunk_mark_blackhole"-  c_nn_thunk_mark_blackhole :: CThunkPtr -> IO Int+  c_nn_thunk_mark_blackhole :: CThunkPtr -> IO CInt  foreign import ccall unsafe "nn_thunk_set_computed"   c_nn_thunk_set_computed :: CThunkPtr -> Ptr () -> IO (Ptr ())

@@ -1,3 +1,5 @@+{-# LANGUAGE PatternSynonyms #-}+ -- | Compile Nix 'Expr' AST trees to flat C bytecode. -- -- Post-order traversal: children are compiled before parents, so child@@ -54,34 +56,34 @@     formalName,     formalNamedSet,     formalSet,-    opApp,-    opAssert,-    opAttrs,-    opBinary,-    opHasAttr,-    opIf,-    opIndStr,-    opLambda,-    opLet,-    opList,-    opLitBool,-    opLitFloat,-    opLitInt,-    opLitNull,-    opLitPath,-    opLitUri,-    opResolvedVar,-    opSearchPath,-    opSelect,-    opStr,-    opUnary,-    opVar,-    opWith,-    opWithVar,     strpartInterp,     strpartLit,     unaryNegate,     unaryNot,+    pattern OpApp,+    pattern OpAssert,+    pattern OpAttrs,+    pattern OpBinary,+    pattern OpHasAttr,+    pattern OpIf,+    pattern OpIndStr,+    pattern OpLambda,+    pattern OpLet,+    pattern OpList,+    pattern OpLitBool,+    pattern OpLitFloat,+    pattern OpLitInt,+    pattern OpLitNull,+    pattern OpLitPath,+    pattern OpLitUri,+    pattern OpResolvedVar,+    pattern OpSearchPath,+    pattern OpSelect,+    pattern OpStr,+    pattern OpUnary,+    pattern OpVar,+    pattern OpWith,+    pattern OpWithVar,   ) import Nix.Eval.EvalFormals (EvalFormal (..), EvalFormals (..)) import Nix.Eval.Symbol (Symbol (..), symbolIntern, symbolText)@@ -105,12 +107,12 @@   where     go :: Expr -> IO Word32     go (ELit atom) = compileLit atom-    go (EStr parts) = compileStringParts opStr parts-    go (EIndStr parts) = compileStringParts opIndStr parts-    go (EVar name) = compileSymbolOp opVar name-    go (EWithVar name) = compileSymbolOp opWithVar name+    go (EStr parts) = compileStringParts OpStr parts+    go (EIndStr parts) = compileStringParts OpIndStr parts+    go (EVar name) = compileSymbolOp OpVar name+    go (EWithVar name) = compileSymbolOp OpWithVar name     go (EResolvedVar level idx) =-      cbcEmit opResolvedVar 0 0 (fi level) (fi idx) 0+      cbcEmit OpResolvedVar 0 0 (fi level) (fi idx) 0     go (EAttrs isRec bindings captureInfo) =       compileAttrs isRec bindings captureInfo     go (EList exprs) = compileList exprs@@ -120,7 +122,7 @@     go (EApp func arg) = do       funcIdx <- go func       argIdx <- go arg-      cbcEmit opApp 0 0 funcIdx argIdx 0+      cbcEmit OpApp 0 0 funcIdx argIdx 0     go (ELambda formals body captureInfo) =       compileLambda formals body captureInfo     go (ELet bindings body captureInfo) =@@ -129,23 +131,23 @@       condIdx <- go cond       thenIdx <- go thenExpr       elseIdx <- go elseExpr-      cbcEmit opIf 0 0 condIdx thenIdx elseIdx+      cbcEmit OpIf 0 0 condIdx thenIdx elseIdx     go (EWith scope body) = do       scopeIdx <- go scope       bodyIdx <- go body-      cbcEmit opWith 0 0 scopeIdx bodyIdx 0+      cbcEmit OpWith 0 0 scopeIdx bodyIdx 0     go (EAssert cond body) = do       condIdx <- go cond       bodyIdx <- go body-      cbcEmit opAssert 0 0 condIdx bodyIdx 0+      cbcEmit OpAssert 0 0 condIdx bodyIdx 0     go (EUnary op operand) = do       operandIdx <- go operand-      cbcEmit opUnary (encodeUnaryOp op) 0 operandIdx 0 0+      cbcEmit OpUnary (encodeUnaryOp op) 0 operandIdx 0 0     go (EBinary op left right) = do       leftIdx <- go left       rightIdx <- go right-      cbcEmit opBinary (encodeBinaryOp op) 0 leftIdx rightIdx 0-    go (ESearchPath name) = compileSymbolOp opSearchPath name+      cbcEmit OpBinary (encodeBinaryOp op) 0 leftIdx rightIdx 0+    go (ESearchPath name) = compileSymbolOp OpSearchPath name      -- -----------------------------------------------------------------     -- Literals@@ -154,16 +156,16 @@     compileLit :: NixAtom -> IO Word32     compileLit (NixInt n) =       let (lo, hi) = splitInt64 n-       in cbcEmit opLitInt 0 0 lo hi 0+       in cbcEmit OpLitInt 0 0 lo hi 0     compileLit (NixFloat d) =       let (lo, hi) = splitDouble d-       in cbcEmit opLitFloat 0 0 lo hi 0+       in cbcEmit OpLitFloat 0 0 lo hi 0     compileLit (NixBool b) =-      cbcEmit opLitBool 0 (if b then 1 else 0) 0 0 0+      cbcEmit OpLitBool 0 (if b then 1 else 0) 0 0 0     compileLit NixNull =-      cbcEmit opLitNull 0 0 0 0 0-    compileLit (NixUri u) = compileSymbolOp opLitUri u-    compileLit (NixPath p) = compileSymbolOp opLitPath p+      cbcEmit OpLitNull 0 0 0 0 0+    compileLit (NixUri u) = compileSymbolOp OpLitUri u+    compileLit (NixPath p) = compileSymbolOp OpLitPath p      -- \| Emit an instruction whose only operand is an interned symbol.     compileSymbolOp :: Word8 -> Text -> IO Word32@@ -197,7 +199,7 @@     compileAttrs isRec bindings captureInfo = do       dataOff <- compileBindings bindings       capOff <- compileCaptureInfo captureInfo-      cbcEmit opAttrs (if isRec then 1 else 0) (fi (length bindings)) dataOff capOff 0+      cbcEmit OpAttrs (if isRec then 1 else 0) (fi (length bindings)) dataOff capOff 0      -- -----------------------------------------------------------------     -- Lists (EList)@@ -207,7 +209,7 @@     compileList exprs = do       childIndices <- mapM go exprs       dataOff <- emitWordList childIndices-      cbcEmit opList 0 (fi (length exprs)) dataOff 0 0+      cbcEmit OpList 0 (fi (length exprs)) dataOff 0 0      -- -----------------------------------------------------------------     -- Select / HasAttr@@ -221,13 +223,13 @@         Just d -> go d       (pathOff, pathLen) <- compileAttrPath path       let hasDef = case defExpr of Nothing -> 0; Just _ -> 1-      cbcEmit opSelect hasDef pathLen targetIdx pathOff defIdx+      cbcEmit OpSelect hasDef pathLen targetIdx pathOff defIdx      compileHasAttr :: Expr -> [AttrKey] -> IO Word32     compileHasAttr target path = do       targetIdx <- go target       (pathOff, pathLen) <- compileAttrPath path-      cbcEmit opHasAttr 0 pathLen targetIdx pathOff 0+      cbcEmit OpHasAttr 0 pathLen targetIdx pathOff 0      compileAttrPath :: [AttrKey] -> IO (Word32, Word16)     compileAttrPath path = do@@ -252,7 +254,7 @@       bodyIdx <- go body       formalsOff <- compileFormals formals       capOff <- compileCaptureInfo captureInfo-      cbcEmit opLambda (encodeFormalType formals) 0 formalsOff bodyIdx capOff+      cbcEmit OpLambda (encodeFormalType formals) 0 formalsOff bodyIdx capOff      compileFormals :: Formals -> IO Word32     compileFormals (FormalName name) = do@@ -291,7 +293,7 @@       bodyIdx <- go body       dataOff <- compileBindings bindings       capOff <- compileCaptureInfo captureInfo-      cbcEmit opLet 0 (fi (length bindings)) dataOff bodyIdx capOff+      cbcEmit OpLet 0 (fi (length bindings)) dataOff bodyIdx capOff      -- -----------------------------------------------------------------     -- Bindings (shared by EAttrs and ELet)@@ -456,6 +458,8 @@     ellipsis <- cbcData (dataOff + 2)     formals <- decodeFormalEntries (fromIntegral count) (dataOff + 3)     pure (EFNamedSet (symbolText (Symbol nameSym)) formals (ellipsis /= 0))+  -- Unreachable: the flag is written only by this module's formals compiler,+  -- which emits 0, 1, or 2 — all matched above.   _ -> error "decodeBcFormals: invalid formal type flag"  -- | Decode a list of formal entries from the data buffer.@@ -484,6 +488,8 @@       count <- cbcData (dataOff + 1)       pairs <- decodePairs (fromIntegral count) (dataOff + 2)       pure (CapturesWithScopes pairs)+    -- Unreachable: the tag is written only by this module's capture compiler,+    -- which emits 0, 1, or 2 — all matched above.     _ -> error "decodeBcCaptureInfo: invalid capture type tag"  -- | Decode (level, idx) pairs from the data buffer.@@ -546,6 +552,8 @@       if hasFrom /= 0         then pure (BcInheritFrom fromBcIdx syms, nextOff)         else pure (BcInherit syms, nextOff)+    -- Unreachable: the tag is written only by this module's binding compiler,+    -- which emits 0 or 1 — both matched above.     _ -> error "decodeOneBinding: invalid binding type tag"  -- | Decode attr path keys: pairs of (is_expr, key_or_bc_idx).

@@ -1,5 +1,6 @@ {-# LANGUAGE GeneralizedNewtypeDeriving #-} {-# LANGUAGE LambdaCase #-}+{-# LANGUAGE PatternSynonyms #-} {-# LANGUAGE ScopedTypeVariables #-}  -- | IO-based Nix evaluator.@@ -48,7 +49,7 @@ import Nix.Eval.CList (CList (..)) import Nix.Eval.CThunk (CThunkPtr, cthunkGetAttrs, cthunkGetBcIdx, cthunkGetBool, cthunkGetCtxStr, cthunkGetFloat, cthunkGetInt, cthunkGetLambda, cthunkGetList, cthunkGetPath, cthunkGetStr, cthunkMarkBlackhole, cthunkPayload, cthunkSetComputed, cthunkSetComputedAttrs, cthunkSetComputedBool, cthunkSetComputedCtxStr, cthunkSetComputedFloat, cthunkSetComputedInt, cthunkSetComputedLambda, cthunkSetComputedList, cthunkSetComputedNull, cthunkSetComputedPath, cthunkSetComputedStr, cthunkState, cthunkValueTag) import Nix.Eval.Symbol (Symbol (..), symbolIntern, symbolText)-import Nix.Eval.Types (AttrSet (..), Env (..), MonadEval (..), NixValue (..), Thunk (..), attrSetSize, emptyContext, marshalLambda, marshalStringContext, unmarshalLambdaValue, unmarshalStringContext)+import Nix.Eval.Types (AttrSet (..), Env (..), MonadEval (..), NixValue (..), Thunk (..), attrSetSize, emptyContext, marshalLambda, marshalStringContext, unmarshalLambdaValue, unmarshalStringContext, pattern ValueAttrs, pattern ValueBool, pattern ValueCtxStr, pattern ValueFloat, pattern ValueInt, pattern ValueLambda, pattern ValueList, pattern ValueNull, pattern ValuePath, pattern ValueStr) import Nix.Expr.Types (AttrKey (..), Binding (..), Expr (..), Formal (..), Formals (..), NixAtom (..), StringPart (..)) import Nix.Hash (makeFixedOutputPath, sha256Digest, sha256Hex, truncatedBase32) import Nix.Parser (parseNix, readFileAutoEncoding)@@ -98,6 +99,10 @@     -- bottom-up by 'builtinDerivationStrict' so input derivations can be     -- substituted by their content hashes when computing output paths.     esDrvModuloCache :: !(IORef (Map Text Text)),+    -- | Accumulated @.drv@ closure (drv store path text → its ATerm), recorded+    -- bottom-up by 'builtinDerivationStrict'.  The build driver reads this after+    -- evaluation to write every input @.drv@ to the store before building.+    esDrvClosure :: !(IORef (Map Text Text)),     -- | Cache of source path → its store path (recursive NAR hash), so a path     -- literal used across many derivations is hashed only once.     esSourcePathCache :: !(IORef (Map Text Text)),@@ -113,6 +118,7 @@ newEvalState baseDir = do   cache <- newIORef Map.empty   drvCache <- newIORef Map.empty+  drvClosure <- newIORef Map.empty   srcCache <- newIORef Map.empty   now <- floor <$> getPOSIXTime :: IO Int64   nixPathStr <- lookupEnvText "NIX_PATH"@@ -123,6 +129,7 @@     EvalState       { esImportCache = cache,         esDrvModuloCache = drvCache,+        esDrvClosure = drvClosure,         esSourcePathCache = srcCache,         esBaseDir = baseDir,         esStoreDir = T.unpack SP.platformStoreDirText,@@ -219,6 +226,10 @@     ref <- asks esDrvModuloCache     liftIO (modifyIORef' ref (Map.insert key val)) +  recordDrvAterm key aterm = EvalIO $ do+    ref <- asks esDrvClosure+    liftIO (modifyIORef' ref (Map.insert key aterm))+   storeSourcePath rawPath = do     ref <- EvalIO (asks esSourcePathCache)     cached <- EvalIO (liftIO (Map.lookup rawPath <$> readIORef ref))@@ -246,6 +257,11 @@     when (T.any (== '\0') name) $       throwEvalError ("writeToStore: name must not contain null bytes: " <> name)     storeDir <- EvalIO (asks esStoreDir)+    -- The preimage is built inline against the LIVE storeDir (esStoreDir), not+    -- via Nix.Hash.makeStorePath which pins the canonical defaultStoreDir for+    -- cross-platform .drv-hash parity.  toFile outputs are host-local, so they+    -- must use the running store dir — keep this format in sync with+    -- makeStorePath's preimage (type:sha256:hex:storeDir:name).     let contentHash = sha256Hex (encodeUtf8 contents)         inner = "nix-store:sha256:" <> contentHash <> ":" <> T.pack storeDir <> ":" <> name         pathHash = truncatedBase32 (encodeUtf8 inner)@@ -315,6 +331,8 @@     when (T.any (== '\0') name) $       throwEvalError ("copyPathToStore: name must not contain null bytes: " <> name)     storeDir <- EvalIO (asks esStoreDir)+    -- Inline host-local store-path preimage (see writeToStore's note): uses the+    -- live esStoreDir, not makeStorePath's pinned defaultStoreDir.     let contentHash = sha256Hex (encodeUtf8 ("source:" <> srcPath <> ":" <> name))         inner = "nix-store:sha256:" <> contentHash <> ":" <> T.pack storeDir <> ":" <> name         pathHash = truncatedBase32 (encodeUtf8 inner)@@ -407,22 +425,22 @@ readComputed ptr = do   tag <- cthunkValueTag ptr   case tag of-    0 {- INT -} -> VInt <$> cthunkGetInt ptr-    1 {- FLOAT -} -> VFloat <$> cthunkGetFloat ptr-    2 {- BOOL -} -> (\b -> VBool (b /= 0)) <$> cthunkGetBool ptr-    3 {- NULL -} -> pure VNull-    4 {- STR -} -> do+    ValueInt -> VInt <$> cthunkGetInt ptr+    ValueFloat -> VFloat <$> cthunkGetFloat ptr+    ValueBool -> (\b -> VBool (b /= 0)) <$> cthunkGetBool ptr+    ValueNull -> pure VNull+    ValueStr -> do       sym <- cthunkGetStr ptr       pure (VStr (symbolText (Symbol sym)) emptyContext)-    5 {- PATH -} -> VPath . symbolText . Symbol <$> cthunkGetPath ptr-    6 {- LIST -} -> do+    ValuePath -> VPath . symbolText . Symbol <$> cthunkGetPath ptr+    ValueList -> do       listPtr <- cthunkGetList ptr       pure (VList (CList (castPtr listPtr)))-    7 {- ATTRS -} -> VAttrs . AttrSet . castPtr <$> cthunkGetAttrs ptr-    8 {- CTXSTR -} -> do+    ValueAttrs -> VAttrs . AttrSet . castPtr <$> cthunkGetAttrs ptr+    ValueCtxStr -> do       csptr <- cthunkGetCtxStr ptr       uncurry VStr <$> unmarshalStringContext (castPtr csptr)-    9 {- LAMBDA -} -> do+    ValueLambda -> do       lamPtr <- cthunkGetLambda ptr       unmarshalLambdaValue lamPtr     _ {- PTR -} -> do

@@ -45,14 +45,14 @@   OpDiv -> evalDiv left right   OpEq -> VBool <$> nixEqual forceFn left right   OpNeq -> VBool . not <$> nixEqual forceFn left right-  OpLt -> VBool <$> nixCompare left right+  OpLt -> VBool <$> nixCompare forceFn left right   OpLte -> do-    lt <- nixCompare left right+    lt <- nixCompare forceFn left right     eq <- nixEqual forceFn left right     pure (VBool (lt || eq))-  OpGt -> VBool <$> nixCompare right left+  OpGt -> VBool <$> nixCompare forceFn right left   OpGte -> do-    gt <- nixCompare right left+    gt <- nixCompare forceFn right left     eq <- nixEqual forceFn left right     pure (VBool (gt || eq))   OpConcat -> evalConcat left right@@ -140,20 +140,44 @@ -- ---------------------------------------------------------------------------  -- | Ordering comparison for < (reused for >, <=, >= via argument swap).-nixCompare :: (MonadEval m) => NixValue -> NixValue -> m Bool-nixCompare (VInt a) (VInt b) = pure (a < b)-nixCompare (VInt a) (VFloat b) = pure (fromIntegral a < b)-nixCompare (VFloat a) (VInt b) = pure (a < fromIntegral b)-nixCompare (VFloat a) (VFloat b) = pure (a < b)+nixCompare :: (MonadEval m) => Force m -> NixValue -> NixValue -> m Bool+nixCompare _ (VInt a) (VInt b) = pure (a < b)+nixCompare _ (VInt a) (VFloat b) = pure (fromIntegral a < b)+nixCompare _ (VFloat a) (VInt b) = pure (a < fromIntegral b)+nixCompare _ (VFloat a) (VFloat b) = pure (a < b) -- String comparison ignores context (matching real Nix).-nixCompare (VStr a _) (VStr b _) = pure (a < b)-nixCompare left right =+nixCompare _ (VStr a _) (VStr b _) = pure (a < b)+-- Paths compare as their string representation (Nix semantics).+nixCompare _ (VPath a) (VPath b) = pure (a < b)+-- Lists compare lexicographically, element by element (Nix semantics).+nixCompare forceFn (VList clA) (VList clB) =+  listCompare forceFn (map Thunk (clistThunks clA)) (map Thunk (clistThunks clB))+nixCompare _ left right =   throwEvalError     ( "cannot compare "         <> typeName left         <> " and "         <> typeName right     )++-- | Lexicographic comparison of two thunk lists for the @<@ operator: the+-- first differing element decides; a proper prefix is less than the longer+-- list.  Mirrors 'listEqual'.+listCompare :: (MonadEval m) => Force m -> [Thunk] -> [Thunk] -> m Bool+listCompare _ [] [] = pure False+listCompare _ [] (_ : _) = pure True+listCompare _ (_ : _) [] = pure False+listCompare forceFn (a : as) (b : bs)+  | thunkSameRef a b = listCompare forceFn as bs+  | otherwise = do+      va <- forceFn a+      vb <- forceFn b+      ltAB <- nixCompare forceFn va vb+      if ltAB+        then pure True+        else do+          ltBA <- nixCompare forceFn vb va+          if ltBA then pure False else listCompare forceFn as bs  -- | Deep structural equality.  Forces thunks inside lists and -- attribute sets as needed.

@@ -76,34 +76,36 @@       | c == '\n' = ('\n' : acc, True, 0)       | otherwise = (c : acc, False, dropped) --- | Coerce a Nix value to a string for interpolation.+-- | Coerce a Nix value to a string. ----- Strict coercion: strings, ints, floats, paths, null, bools, and--- attribute sets with @__toString@ or @outPath@.--- Lists and functions without coercion metadata are errors.--- Used by string interpolation (@"${...}"@) and @builtins.toString@.-coerceToString :: (MonadEval m) => Force m -> Apply m -> NixValue -> m (Text, StringContext)-coerceToString _ _ (VStr s ctx) = pure (s, ctx)-coerceToString _ _ (VInt n) = pure (T.pack (show n), emptyContext)-coerceToString _ _ (VFloat n) = pure (formatNixFloat n, emptyContext)-coerceToString _ _ (VPath p) = pure (p, emptyContext)-coerceToString _ _ VNull = pure ("", emptyContext)-coerceToString _ _ (VBool True) = pure ("1", emptyContext)-coerceToString _ _ (VBool False) = pure ("", emptyContext)--- Attribute sets: try __toString first, then outPath-coerceToString forceFn applyFn (VAttrs attrs) =+-- The @coerceMore@ flag mirrors C++ Nix's @coerceToString@ argument: when+-- 'True' (e.g. @builtins.toString@, derivation-env values) ints, floats, bools+-- and null coerce permissively; when 'False' (string interpolation,+-- @builtins.concatStringsSep@) those are type errors, matching C++ Nix.+-- Strings, paths, and attribute sets with @__toString@/@outPath@ coerce in+-- both modes; lists and bare functions are always errors.+coerceToString :: (MonadEval m) => Bool -> Force m -> Apply m -> NixValue -> m (Text, StringContext)+coerceToString _ _ _ (VStr s ctx) = pure (s, ctx)+coerceToString _ _ _ (VPath p) = pure (p, emptyContext)+coerceToString True _ _ (VInt n) = pure (T.pack (show n), emptyContext)+coerceToString True _ _ (VFloat n) = pure (formatNixFloat n, emptyContext)+coerceToString True _ _ VNull = pure ("", emptyContext)+coerceToString True _ _ (VBool True) = pure ("1", emptyContext)+coerceToString True _ _ (VBool False) = pure ("", emptyContext)+-- Attribute sets: try __toString first, then outPath (both modes).+coerceToString coerceMore forceFn applyFn (VAttrs attrs) =   case attrSetLookup "__toString" attrs of     Just toStrThunk -> do       toStrFn <- forceFn toStrThunk       result <- applyFn toStrFn (VAttrs attrs)-      coerceToString forceFn applyFn result+      coerceToString coerceMore forceFn applyFn result     Nothing -> case attrSetLookup "outPath" attrs of       Just outPathThunk -> do         outPathVal <- forceFn outPathThunk-        coerceToString forceFn applyFn outPathVal+        coerceToString coerceMore forceFn applyFn outPathVal       Nothing ->         throwEvalError "cannot coerce a set to a string (missing __toString or outPath)"-coerceToString _ _ other =+coerceToString _ _ _ other =   throwEvalError ("cannot coerce " <> typeName other <> " to a string")  -- | Format a float the way C++ Nix does: 6 fixed decimal places

@@ -1,4 +1,5 @@ {-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE PatternSynonyms #-}  -- | Shared types for the Nix evaluator. --@@ -83,9 +84,25 @@     -- * Display     typeName, +    -- * C thunk-state and value tags (mirror cbits/nn_thunk.h)+    pattern ThunkPending,+    pattern ThunkComputed,+    pattern ThunkBlackhole,+    pattern ValueInt,+    pattern ValueFloat,+    pattern ValueBool,+    pattern ValueNull,+    pattern ValueStr,+    pattern ValuePath,+    pattern ValueList,+    pattern ValueAttrs,+    pattern ValueCtxStr,+    pattern ValueLambda,+     -- * Evaluation monad     MonadEval (..),-    PureEval (..),+    PureEval,+    runPureEval,   ) where @@ -181,11 +198,11 @@ instance Show Thunk where   show (Thunk ptr) =     case unsafePerformIO (cthunkState ptr) of-      1 {- COMPUTED -} -> case readThunkValue (Thunk ptr) of+      ThunkComputed -> case readThunkValue (Thunk ptr) of         Just val -> "Thunk (" ++ show val ++ ")"         Nothing -> "Thunk <computed?>"-      0 -> "Thunk <pending>"-      2 -> "Thunk <blackhole>"+      ThunkPending -> "Thunk <pending>"+      ThunkBlackhole -> "Thunk <blackhole>"       _ -> "Thunk <unknown>"  -- | Equality: pointer identity fast path, then value comparison for@@ -197,6 +214,31 @@         (Just v1, Just v2) -> v1 == v2         _ -> False +-- | Named constants for the C thunk-state byte and value-tag byte, mirroring+-- the @NN_THUNK_*@ / @NN_VALUE_*@ macros in @cbits\/nn_thunk.h@.  Bidirectional+-- pattern synonyms over literals, so a @case@ on a tag keeps the jump-table+-- compilation a bare-literal @case@ gets, while the three thunk-dispatch sites+-- ('readThunkValue', @PureEval@'s 'forceThunk', and @Nix.Eval.IO@'s+-- @readComputed@) share one source of truth instead of triplicating literals.+-- These MUST stay in lockstep with @cbits\/nn_thunk.h@.  The remaining value+-- tag (PTR, a StablePtr payload) is the wildcard at each dispatch site.+pattern ThunkPending, ThunkComputed, ThunkBlackhole :: Word8+pattern ThunkPending = 0+pattern ThunkComputed = 1+pattern ThunkBlackhole = 2++pattern ValueInt, ValueFloat, ValueBool, ValueNull, ValueStr, ValuePath, ValueList, ValueAttrs, ValueCtxStr, ValueLambda :: Word8+pattern ValueInt = 0+pattern ValueFloat = 1+pattern ValueBool = 2+pattern ValueNull = 3+pattern ValueStr = 4+pattern ValuePath = 5+pattern ValueList = 6+pattern ValueAttrs = 7+pattern ValueCtxStr = 8+pattern ValueLambda = 9+ -- | Read a COMPUTED thunk's value without forcing. -- Returns 'Nothing' for PENDING or BLACKHOLE thunks. -- Uses 'unsafePerformIO' — safe because C reads are idempotent.@@ -204,25 +246,25 @@ readThunkValue (Thunk ptr) =   unsafePerformIO $ do     state <- cthunkState ptr-    if state /= 1+    if state /= ThunkComputed       then pure Nothing       else do         tag <- cthunkValueTag ptr         fmap Just $ case tag of-          0 {- INT -} -> VInt <$> cthunkGetInt ptr-          1 {- FLOAT -} -> VFloat <$> cthunkGetFloat ptr-          2 {- BOOL -} -> (\b -> VBool (b /= 0)) <$> cthunkGetBool ptr-          3 {- NULL -} -> pure VNull-          4 {- STR -} -> (\sym -> VStr (symbolText (Symbol sym)) emptyContext) <$> cthunkGetStr ptr-          5 {- PATH -} -> VPath . symbolText . Symbol <$> cthunkGetPath ptr-          6 {- LIST -} -> do+          ValueInt -> VInt <$> cthunkGetInt ptr+          ValueFloat -> VFloat <$> cthunkGetFloat ptr+          ValueBool -> (\b -> VBool (b /= 0)) <$> cthunkGetBool ptr+          ValueNull -> pure VNull+          ValueStr -> (\sym -> VStr (symbolText (Symbol sym)) emptyContext) <$> cthunkGetStr ptr+          ValuePath -> VPath . symbolText . Symbol <$> cthunkGetPath ptr+          ValueList -> do             listPtr <- cthunkGetList ptr             pure (VList (CList (castPtr listPtr)))-          7 {- ATTRS -} -> VAttrs . AttrSet . castPtr <$> cthunkGetAttrs ptr-          8 {- CTXSTR -} -> do+          ValueAttrs -> VAttrs . AttrSet . castPtr <$> cthunkGetAttrs ptr+          ValueCtxStr -> do             csptr <- cthunkGetCtxStr ptr             uncurry VStr <$> unmarshalStringContext (castPtr csptr)-          9 {- LAMBDA -} -> do+          ValueLambda -> do             lamRaw <- cthunkPayload ptr             unmarshalLambdaValue (castPtr lamRaw)           _ {- PTR -} -> do@@ -407,7 +449,7 @@  -- | Evaluation environment — C-backed scope chain. ----- Arena-allocated @nn_env_t@ struct (40 bytes, zero GC overhead).+-- Arena-allocated @nn_env_t@ struct (48 bytes, zero GC overhead). -- All env data (slots, lazy scope, parent, with-scopes) lives in C. -- Haskell holds only the pointer.  Variable lookup has two paths: --@@ -1059,22 +1101,47 @@   -- A no-op in pure evaluators (no memoization).   cacheDrvHash :: Text -> Text -> m () +  -- | Record a derivation's serialized @.drv@ ATerm under its @.drv@ store+  -- path, as each derivation is computed (bottom-up).  Unlike 'cacheDrvHash'+  -- (which stores only the modulo hash), this retains the full ATerm so the+  -- build driver can materialize the entire input-@.drv@ closure to the store+  -- before a dependency-aware build.  A no-op in pure evaluators.+  recordDrvAterm :: Text -> Text -> m ()+   -- | Compute the store path a source file/directory gets when copied into   -- the store (recursive NAR sha256 → @source@ fixed-output path), WITHOUT   -- performing the copy.  Used when a path literal is coerced in a derivation   -- argument or environment value.  Unavailable in pure evaluation.   storeSourcePath :: Text -> m Text --- | Pure evaluation monad — wraps 'Either Text'.+-- | Error raised during pure evaluation.  'PAbort' (from @abort@ and infinite+-- recursion) must escape 'tryEval' exactly as in C++ Nix; 'PThrow' is catchable.+data PureError = PThrow !Text | PAbort !Text++-- | Pure evaluation monad — wraps @Either PureError@. -- IO builtins ('readFile', 'import') are unavailable; -- everything else evaluates identically to the IO version.-newtype PureEval a = PureEval {runPureEval :: Either Text a}+newtype PureEval a = PureEval (Either PureError a)   deriving (Functor, Applicative, Monad) +-- | Run a pure evaluation, flattening the internal error into 'Text'.  An abort+-- is prefixed @\"evaluation aborted: \"@, matching 'EvalIO'.+runPureEval :: PureEval a -> Either Text a+runPureEval (PureEval (Right a)) = Right a+runPureEval (PureEval (Left (PThrow t))) = Left t+runPureEval (PureEval (Left (PAbort t))) = Left ("evaluation aborted: " <> t)+ instance MonadEval PureEval where-  throwEvalError msg = PureEval (Left msg)-  abortEvaluation msg = PureEval (Left ("evaluation aborted: " <> msg))-  catchEvalError (PureEval action) = PureEval (Right action)+  throwEvalError msg = PureEval (Left (PThrow msg))+  abortEvaluation msg = PureEval (Left (PAbort msg))++  -- Catch a throw (tryEval sees the error), but let an abort / infinite+  -- recursion propagate past tryEval, matching EvalIO and C++ Nix.+  catchEvalError (PureEval action) =+    PureEval $ case action of+      Left (PThrow t) -> Right (Left t)+      Left (PAbort t) -> Left (PAbort t)+      Right a -> Right (Right a)   readFileText _ = throwEvalError "readFile: not available in pure evaluation"   doesPathExist _ = pure False   listDirectory _ = throwEvalError "builtins.readDir: not available in pure evaluation"@@ -1090,6 +1157,7 @@   traceMessage _ = pure ()   lookupDrvHash _ = pure Nothing   cacheDrvHash _ _ = pure ()+  recordDrvAterm _ _ = pure ()    -- Pure eval cannot read files: a path coerces to itself (no store copy);   -- the real copy-to-store happens only under 'EvalIO'.@@ -1101,30 +1169,30 @@     -- Does NOT mark blackholes — PureEval may re-force the same thunk.     -- Dispatches on val_tag for computed scalars (no StablePtr deref).     case unsafePerformIO (cthunkState ptr) of-      1 {- COMPUTED -} ->+      ThunkComputed ->         let tag = unsafePerformIO (cthunkValueTag ptr)          in case tag of-              0 {- INT -} -> pure (VInt (unsafePerformIO (cthunkGetInt ptr)))-              1 {- FLOAT -} -> pure (VFloat (unsafePerformIO (cthunkGetFloat ptr)))-              2 {- BOOL -} -> pure (VBool (unsafePerformIO (cthunkGetBool ptr) /= 0))-              3 {- NULL -} -> pure VNull-              4 {- STR -} ->+              ValueInt -> pure (VInt (unsafePerformIO (cthunkGetInt ptr)))+              ValueFloat -> pure (VFloat (unsafePerformIO (cthunkGetFloat ptr)))+              ValueBool -> pure (VBool (unsafePerformIO (cthunkGetBool ptr) /= 0))+              ValueNull -> pure VNull+              ValueStr ->                 let sym = unsafePerformIO (cthunkGetStr ptr)                  in pure (VStr (symbolText (Symbol sym)) emptyContext)-              5 {- PATH -} ->+              ValuePath ->                 let sym = unsafePerformIO (cthunkGetPath ptr)                  in pure (VPath (symbolText (Symbol sym)))-              6 {- LIST -} ->+              ValueList ->                 let listPtr = unsafePerformIO (cthunkGetList ptr)                  in pure (VList (CList (castPtr listPtr)))-              7 {- ATTRS -} ->+              ValueAttrs ->                 let p = unsafePerformIO (cthunkGetAttrs ptr)                  in pure (VAttrs (AttrSet (castPtr p)))-              8 {- CTXSTR -} ->+              ValueCtxStr ->                 let csptr = unsafePerformIO (cthunkGetCtxStr ptr)                     (t, ctx) = unsafePerformIO (unmarshalStringContext (castPtr csptr))                  in pure (VStr t ctx)-              9 {- LAMBDA -} ->+              ValueLambda ->                 let lamRaw = unsafePerformIO (cthunkPayload ptr)                     val = unsafePerformIO (unmarshalLambdaValue lamRaw)                  in pure val@@ -1132,7 +1200,7 @@                 let payload = unsafePerformIO (cthunkPayload ptr)                     val = unsafePerformIO (deRefStablePtr (castPtrToStablePtr payload))                  in pure val-      2 {- BLACKHOLE -} ->+      ThunkBlackhole ->         -- Non-catchable: must escape tryEval like in C++ Nix         abortEvaluation "infinite recursion encountered"       _ {- PENDING -} ->

@@ -26,19 +26,16 @@ --   network transfer integrity. -- -- nova-cache already handles output hashes and file hashes.  This module--- adds input hash (derivation hash) computation for the evaluator, plus--- the @makeStorePath@ family that constructs content-addressed store paths--- exactly as C++ Nix does.+-- provides the shared hashing utilities and the @makeStorePath@ family that+-- constructs content-addressed store paths exactly as C++ Nix does. module Nix.Hash-  ( -- * Derivation hashing-    DrvHash (..),-    hashDerivation,--    -- * Shared hashing utilities+  ( -- * Shared hashing utilities     sha256Hex,     truncatedBase32,     hashPlaceholder,     byteToHex,+    hexToBytes,+    rawHashWithAlgo,      -- * Store path construction (Nix @makeStorePath@ family)     makeStorePath,@@ -60,6 +57,7 @@ import Data.Bits (xor) import qualified Data.ByteArray as BA import qualified Data.ByteString as BS+import Data.Char (digitToInt, isHexDigit) import Data.List (foldl') import qualified Data.Set as Set import Data.Text (Text)@@ -70,24 +68,6 @@ import NovaCache.Base32 (encode) import NovaCache.Hash (formatNixHash, hashBytes, parseNixHash) --- | A derivation hash — the input hash that determines the store path.--- This is computed from the derivation's inputs, NOT from the build output.--- Stored as the Nix-formatted hash string (e.g. "sha256:0abc...").-newtype DrvHash = DrvHash {unDrvHash :: Text}-  deriving (Eq, Show)---- | Hash a serialized derivation (.drv file contents) to produce the--- input hash used in store path computation.------ The derivation is first converted to ATerm format, then SHA-256 hashed,--- then truncated to 160 bits (20 bytes) and encoded in Nix base-32.--- This produces the 32-character hash in the store path.-hashDerivation :: Text -> DrvHash-hashDerivation drvText =-  let drvBytes = encodeUtf8 drvText-      nixHash = hashBytes drvBytes-   in DrvHash (formatNixHash nixHash)- -- | SHA-256 hex digest of a ByteString. sha256Hex :: BS.ByteString -> Text sha256Hex bs =@@ -153,6 +133,32 @@ -- | Lowercase base-16 of raw bytes (two hex characters per byte). bytesToHexText :: BS.ByteString -> Text bytesToHexText = T.pack . concatMap byteToHex . BS.unpack++-- | Decode a lowercase/uppercase base-16 string to raw bytes.  Returns+-- 'Nothing' for odd length or non-hex characters.  This is the canonical+-- form a @.drv@ stores a fixed-output hash in, so the builder reads it back+-- with this rather than re-parsing the user's original SRI/base32 string.+hexToBytes :: Text -> Maybe BS.ByteString+hexToBytes t+  | T.null t = Just BS.empty+  | odd (T.length t) = Nothing+  | T.all isHexDigit t = Just (BS.pack (pairs (T.unpack t)))+  | otherwise = Nothing+  where+    pairs [] = []+    pairs (hi : lo : rest) = fromIntegral (digitToInt hi * 16 + digitToInt lo) : pairs rest+    pairs [_] = [] -- unreachable: even length checked above++-- | Raw digest of a 'BS.ByteString' under a named algorithm (@sha256@,+-- @sha512@, @sha1@, @md5@).  Returns 'Nothing' for an unknown algorithm.+-- Used to verify a fixed-output fetch against its declared @outputHashAlgo@.+rawHashWithAlgo :: Text -> BS.ByteString -> Maybe BS.ByteString+rawHashWithAlgo algo bytes = case algo of+  "sha256" -> Just (BA.convert (CH.hash bytes :: CH.Digest CH.SHA256))+  "sha512" -> Just (BA.convert (CH.hash bytes :: CH.Digest CH.SHA512))+  "sha1" -> Just (BA.convert (CH.hash bytes :: CH.Digest CH.SHA1))+  "md5" -> Just (BA.convert (CH.hash bytes :: CH.Digest CH.MD5))+  _ -> Nothing  -- | The core store-path construction primitive.  Given a @type@ string, the -- inner content digest (raw SHA-256 bytes), and a name, produce the store

@@ -54,6 +54,7 @@ import Data.Text (Text) import qualified Data.Text as T import qualified Data.Text.Encoding as TE+import Data.Text.Encoding.Error (lenientDecode) import Nix.Expr.ClosureTrim (trimClosures) import Nix.Expr.Resolve (resolveVars) import Nix.Expr.Types (Expr)@@ -110,15 +111,15 @@   | BS.length bytes >= 2,     BS.index bytes 0 == 0xFF,     BS.index bytes 1 == 0xFE =-      TE.decodeUtf16LE (BS.drop 2 bytes)+      TE.decodeUtf16LEWith lenientDecode (BS.drop 2 bytes)   | BS.length bytes >= 2,     BS.index bytes 0 == 0xFE,     BS.index bytes 1 == 0xFF =-      TE.decodeUtf16BE (BS.drop 2 bytes)+      TE.decodeUtf16BEWith lenientDecode (BS.drop 2 bytes)   | BS.length bytes >= 3,     BS.index bytes 0 == 0xEF,     BS.index bytes 1 == 0xBB,     BS.index bytes 2 == 0xBF =-      TE.decodeUtf8 (BS.drop 3 bytes)+      TE.decodeUtf8With lenientDecode (BS.drop 3 bytes)   | otherwise =-      TE.decodeUtf8 bytes+      TE.decodeUtf8With lenientDecode bytes

@@ -7,7 +7,11 @@ -- ! / + - / * \\/ / ++ / ? / negate / application / selection -- @ ----- Matches the C++ Nix parser (parser.y) precedence exactly.+-- Close to the C++ Nix parser (parser.y), with two inert deviations: here+-- @++@ binds looser than multiplication\/addition (parser.y binds it tighter)+-- and unary negate binds looser than @?@ (parser.y makes negate tightest).+-- Both only affect mixed expressions that are type errors regardless, so no+-- program that type-checks parses differently. -- Entirely pure. No IO, no Megaparsec, no Parsec. module Nix.Parser.Expr   ( -- * Entry point

@@ -175,6 +175,9 @@         Just c2 <- safeHead (T.drop 1 rest),         c2 == '/' ->           lexPath st acc+      | Just d <- safeHead rest,+        isDigit d ->+          lexLeadingDotFloat st acc     '.' -> emit1 st TokDot acc     ',' -> emit1 st TokComma acc     ';' -> emit1 st TokSemicolon acc@@ -224,16 +227,12 @@     '>' -> emit1 st TokGt acc     '/' | Just '/' <- safeHead rest -> emit2 st TokUpdate acc     '/'-      -- After an expression token, / is always division-      | prevCanEndExpr -> emit1 st TokSlash acc-      -- Not after expression: / followed by path char starts an absolute path-      | Just c2 <- safeHead rest, isPathChar c2 -> lexPath st acc-      -- Otherwise, division+      -- A '/' that begins a path segment (slash followed by a path char)+      -- starts a path: /abs/path.  A bare '/' (followed by whitespace) is the+      -- division operator: a / b.  Relative paths like a/b are caught by the+      -- path guard further down, before the identifier/number cases.+      | looksLikePath (lsInput st) -> lexPath st acc       | otherwise -> emit1 st TokSlash acc-      where-        prevCanEndExpr = case acc of-          (Located _ _ tok : _) -> canFollowWithDivision tok-          [] -> False     '$'       | Just '{' <- safeHead rest ->           -- Increment brace depth so the closing } is TokRBrace, not@@ -246,6 +245,9 @@     '~'       | Just '/' <- safeHead rest ->           lexPath st acc+    -- A path-char run that contains a '/' segment is a path, not an+    -- identifier or number: a/b and 6/2 lex as paths, matching Nix.+    _ | looksLikePath (lsInput st) -> lexPath st acc     _ | isDigit c -> lexNumber st acc     _ | isIdentStart c -> lexIdentOrKeyword st acc     _ ->@@ -475,16 +477,28 @@           | Just (d, _) <- T.uncons after2,             isDigit d ->               let (decimals, after3) = T.span isDigit after2-                  fullLen = T.length digits + 1 + T.length decimals-                  val = readDouble digits decimals+                  (expVal, after4, expLen) = lexExponent after3+                  fullLen = T.length digits + 1 + T.length decimals + expLen+                  val = applyExponent (readDouble digits decimals) expVal                   tok = Located (lsLine st) (lsCol st) (TokFloat val)-                  newSt = advanceCol fullLen st {lsInput = after3}+                  newSt = advanceCol fullLen st {lsInput = after4}                in lexNormalMode newSt (tok : acc)-        _ ->-          let val = fromIntegral (readInteger digits) :: Int64-              tok = Located (lsLine st) (lsCol st) (TokInt val)-              newSt = advanceCol len st {lsInput = after}-           in lexNormalMode newSt (tok : acc)+        _+          -- C++ Nix rejects out-of-range integer literals rather than+          -- silently wrapping modulo 2^64.+          | readInteger digits > toInteger (maxBound :: Int64) ->+              Left+                ParseError+                  { peFile = lsFile st,+                    peLine = lsLine st,+                    peCol = lsCol st,+                    peMessage = "integer literal out of range: " <> digits+                  }+          | otherwise ->+              let val = fromIntegral (readInteger digits) :: Int64+                  tok = Located (lsLine st) (lsCol st) (TokInt val)+                  newSt = advanceCol len st {lsInput = after}+               in lexNormalMode newSt (tok : acc)  -- | Read an integer from text without using the partial 'read'. readInteger :: Text -> Integer@@ -499,6 +513,42 @@       frac = fromIntegral (readInteger decPart) / (decimalBase' ^ fracDigits)    in whole + frac +-- | Consume an optional exponent @[eE][+-]?[0-9]+@ after a float's digits.+-- Returns the signed exponent, the remaining input, and the characters+-- consumed.  An @e@ not followed by digits is not an exponent (consumes 0), so+-- e.g. @1.5e@ lexes as the float @1.5@ followed by the identifier @e@.+lexExponent :: Text -> (Integer, Text, Int)+lexExponent input =+  case T.uncons input of+    Just (e, afterE)+      | e == 'e' || e == 'E' ->+          let (sign, afterSign, signLen) = case T.uncons afterE of+                Just ('+', rest) -> (1, rest, 1)+                Just ('-', rest) -> (-1, rest, 1)+                _ -> (1, afterE, 0)+              (expDigits, afterExp) = T.span isDigit afterSign+           in if T.null expDigits+                then (0, input, 0)+                else (sign * readInteger expDigits, afterExp, 1 + signLen + T.length expDigits)+    _ -> (0, input, 0)++-- | Scale a float mantissa by @10 ^^ exp@ (exp may be negative).+applyExponent :: Double -> Integer -> Double+applyExponent mantissa expVal = mantissa * (10 ^^ expVal)++-- | Lex a leading-dot float like @.5@ or @.5e3@ (Nix's @0?\\.[0-9]+@ form).+-- The current input begins with the @.@.+lexLeadingDotFloat :: LexState -> [Located] -> Either ParseError [Located]+lexLeadingDotFloat st acc =+  let afterDot = T.drop 1 (lsInput st)+      (decimals, after3) = T.span isDigit afterDot+      (expVal, after4, expLen) = lexExponent after3+      fullLen = 1 + T.length decimals + expLen+      val = applyExponent (readDouble "" decimals) expVal+      tok = Located (lsLine st) (lsCol st) (TokFloat val)+      newSt = advanceCol fullLen st {lsInput = after4}+   in lexNormalMode newSt (tok : acc)+ -- | Base for decimal digit accumulation. decimalBase :: Integer decimalBase = 10@@ -622,30 +672,23 @@ isPathChar :: Char -> Bool isPathChar c = isAlphaNum c || c `elem` ("/.~_-+" :: [Char]) +-- | Does the maximal path-char run at the start of the input contain a+-- @/@-segment (a slash followed by a non-slash path char)?  If so it lexes+-- as a path rather than an identifier, number, or division operator —+-- matching Nix, where @a/b@ and @/abs/path@ are paths, @a / b@ (slash+-- surrounded by whitespace) is division, and @a // b@ is the update operator.+looksLikePath :: Text -> Bool+looksLikePath input =+  case T.break (== '/') (T.takeWhile isPathChar input) of+    (_, slashAndRest) -> case T.uncons (T.drop 1 slashAndRest) of+      Just (afterSlash, _) -> isPathChar afterSlash && afterSlash /= '/'+      Nothing -> False+ isSearchPathChar :: Char -> Bool isSearchPathChar c = isAlphaNum c || c `elem` ("/.~_-+" :: [Char])  isUriChar :: Char -> Bool isUriChar c = isAlphaNum c || c `elem` ("%/?:@&=+$,#._~!-" :: [Char])---- | Whether a token can be the last token of an expression.--- Used to decide if @/@ is division or the start of an absolute path.--- Real Nix uses the same context-dependent rule in its lexer.-canFollowWithDivision :: Token -> Bool-canFollowWithDivision (TokIdent _) = True-canFollowWithDivision (TokInt _) = True-canFollowWithDivision (TokFloat _) = True-canFollowWithDivision (TokPath _) = True-canFollowWithDivision TokRParen = True-canFollowWithDivision TokRBracket = True-canFollowWithDivision TokRBrace = True-canFollowWithDivision TokInterpClose = True-canFollowWithDivision TokStringClose = True-canFollowWithDivision TokIndStringClose = True-canFollowWithDivision TokTrue = True-canFollowWithDivision TokFalse = True-canFollowWithDivision TokNull = True-canFollowWithDivision _ = False  -- --------------------------------------------------------------------------- -- Emit helpers

@@ -42,9 +42,13 @@      -- * Store operations     addToStore,+    placeInStore,+    registrationFor,     scanReferences,+    scanTempReferences,     setReadOnly,     writeDrv,+    writeDrvAterm,      -- * Re-exports     module Nix.Store.Path,@@ -64,6 +68,8 @@ import Nix.Derivation (Derivation, toATerm) import Nix.Store.DB import Nix.Store.Path+import qualified NovaCache.Hash as Hash+import qualified NovaCache.NAR as NAR import System.Directory   ( copyFile,     createDirectoryIfMissing,@@ -121,18 +127,46 @@   [StorePath] ->   IO () addToStore store srcPath sp deriver refs = do+  reg <- placeInStore store srcPath sp deriver refs+  registerPath (stDB store) reg++-- | Move a build output into the store (read-only) and compute its+-- registration (NAR hash, size, references) WITHOUT writing to the database.+--+-- Splitting placement from registration lets a multi-output build place every+-- output first and then register them together, so intra-derivation+-- cross-output references are preserved (see 'registerPaths').+placeInStore ::+  Store ->+  FilePath ->+  StorePath ->+  Maybe Text ->+  [StorePath] ->+  IO PathRegistration+placeInStore store srcPath sp deriver refs = do   let destPath = storePathToFilePath (stDir store) sp   -- Move (or copy) source to store   moveOutput srcPath destPath   -- Set read-only permissions   setReadOnly destPath-  -- Register in database-  registerPath-    (stDB store)+  registrationFor store sp deriver refs++-- | Compute the registration metadata for a store path already present on+-- disk, without moving anything.  Used to (re-)register an output that an+-- interrupted build left in place but never recorded in the database.+registrationFor :: Store -> StorePath -> Maybe Text -> [StorePath] -> IO PathRegistration+registrationFor store sp deriver refs = do+  let destPath = storePathToFilePath (stDir store) sp+  -- Compute the NAR hash and size of the final store contents.  The NAR+  -- serialization is canonical (entries sorted, 8-byte padding), so this is+  -- exactly the NarHash/NarSize a binary cache reports for the path.+  narEntry <- NAR.serialiseFromPath destPath+  let narBytes = NAR.serialise narEntry+  pure     PathRegistration       { prPath = sp,-        prNarHash = "sha256:0000000000000000000000000000000000000000000000000000", -- NAR hashing deferred until nova-cache integration-        prNarSize = 0, -- Computed size deferred until NAR serialization is wired in+        prNarHash = Hash.formatNixHash (Hash.hashBytes narBytes),+        prNarSize = BS.length narBytes,         prDeriver = deriver,         prReferences = refs       }@@ -176,21 +210,39 @@ scanReferences storeDir candidates dir = do   let candidateSet = Set.fromList [(spHash sp, sp) | sp <- candidates]       storeDirStr = unStoreDir storeDir-      -- Scan for both forward-slash and backslash store prefixes.-      -- On Windows, binaries may contain either separator style.+      -- Scan for both forward-slash and backslash store prefixes.  On Windows,+      -- binaries may contain either separator style.  Both separators are a+      -- single ASCII byte, so the two prefixes share a length.       prefixFwd = TE.encodeUtf8 (T.pack (storeDirStr <> "/"))       prefixBwd = TE.encodeUtf8 (T.pack (storeDirStr <> "\\"))-      prefixBytes = prefixFwd-      prefixLen = BS.length prefixBytes-      hashLen = 32+      prefixLen = BS.length prefixFwd   files <- collectRegularFiles dir   foundHashes <- foldlIO Set.empty files $ \acc filePath -> do     contents <- BS.readFile filePath     -- Scan with forward-slash prefix, then backslash (for Windows)-    let acc1 = scanBytes prefixFwd prefixLen hashLen contents acc-    pure (scanBytes prefixBwd prefixLen hashLen contents acc1)+    let acc1 = scanBytes prefixFwd prefixLen storePathHashLen contents acc+    pure (scanBytes prefixBwd prefixLen storePathHashLen contents acc1)   pure [sp | (h, sp) <- Set.toList candidateSet, Set.member h foundHashes] +-- | Scan an output for references to build-temp output locations.+--+-- The builder runs under a temp directory, so an output that embeds its own or+-- a sibling output's path embeds the TEMP path — which 'scanReferences' (store+-- prefix only) cannot see.  Given @(tempDir, storePath)@ for every output of+-- the derivation, returns the store paths whose temp location is referenced+-- from the scanned output, capturing self- and cross-output references.+--+-- This records the dependency edge; it does not rewrite the embedded bytes+-- (self-reference hash rewriting is a separate, future concern).+scanTempReferences :: [(FilePath, StorePath)] -> FilePath -> IO [StorePath]+scanTempReferences tempPairs dir = do+  let needles = [(TE.encodeUtf8 (T.pack tempDir), sp) | (tempDir, sp) <- tempPairs]+  files <- collectRegularFiles dir+  foundHashes <- foldlIO Set.empty files $ \acc filePath -> do+    contents <- BS.readFile filePath+    pure (Set.union acc (Set.fromList [spHash sp | (needle, sp) <- needles, needle `BS.isInfixOf` contents]))+  pure [sp | (_, sp) <- tempPairs, Set.member (spHash sp) foundHashes]+ -- | Collect all regular files under a path, recursively. -- If the path is itself a regular file, returns it directly. collectRegularFiles :: FilePath -> IO [FilePath]@@ -235,7 +287,13 @@   acc <- f z x   foldlIO acc xs f --- | Recursively set a directory and all its contents to read-only.+-- | Recursively mark a store path and its contents read-only after a build.+--+-- On Windows the directory read-only attribute does not prevent adding or+-- removing entries — only the per-file read-only attribute protects a file.+-- Immutability here is therefore enforced at FILE granularity (every file is+-- made read-only); hardening the directory itself against entry changes would+-- require ACLs and is deferred. setReadOnly :: FilePath -> IO () setReadOnly path = do   isDir <- doesDirectoryExist path@@ -251,10 +309,16 @@         perms <- Dir.getPermissions path         setPermissions path (Dir.setOwnerWritable False perms) --- | Serialize a derivation to ATerm and write it to the store at the given path.-writeDrv :: Store -> Derivation -> StorePath -> IO ()-writeDrv store drv sp = do+-- | Write an already-serialized derivation ATerm to its store path.  Used to+-- materialize the input @.drv@ closure (root plus every transitive input)+-- before a dependency-aware build: evaluation computes these ATerms but does+-- no store IO, so the build driver writes them here.+writeDrvAterm :: Store -> StorePath -> Text -> IO ()+writeDrvAterm store sp aterm = do   let destPath = storePathToFilePath (stDir store) sp-      aterm = toATerm drv   createDirectoryIfMissing True (unStoreDir (stDir store))   TIO.writeFile destPath aterm++-- | Serialize a derivation to ATerm and write it to the store at the given path.+writeDrv :: Store -> Derivation -> StorePath -> IO ()+writeDrv store drv sp = writeDrvAterm store sp (toATerm drv)

@@ -22,7 +22,7 @@ -- @\/nix\/store\/.nova-nix\/db.sqlite@ (or @C:\\nix\\store\\.nova-nix\\db.sqlite@). module Nix.Store.DB   ( -- * Database handle-    StoreDB (..),+    StoreDB,      -- * Types     PathRegistration (..),@@ -34,6 +34,7 @@      -- * Registration     registerPath,+    registerPaths,     isValidPath,     queryReferences,     queryDeriver,@@ -159,26 +160,46 @@ -- Registration -- --------------------------------------------------------------------------- --- | Register a store path as valid with its metadata and references.--- Uses a transaction for atomicity.  Idempotent — re-registering an--- existing path is a no-op (INSERT OR IGNORE).+-- | Register a single store path as valid with its metadata and references.+-- A convenience wrapper over 'registerPaths' for one path. registerPath :: StoreDB -> PathRegistration -> IO ()-registerPath db reg = withTransaction (sdbConn db) $ do-  let conn = sdbConn db-      pathText = T.pack (storePathToFilePath (sdbDir db) (prPath reg))-  -- Insert the path (skip if already present)+registerPath db reg = registerPaths db [reg]++-- | Register several store paths as valid in one transaction.+--+-- ALL path rows are inserted BEFORE any reference edge, so references among the+-- paths in this batch — e.g. intra-derivation cross-output references — are+-- never dropped.  (Registering one path at a time loses an edge whenever a+-- referrer is registered before its referent.)+--+-- Re-registering an existing path refreshes its metadata (NAR hash, size,+-- deriver) via @ON CONFLICT DO UPDATE@, so a path first registered with a+-- placeholder hash is corrected on a later real registration.+registerPaths :: StoreDB -> [PathRegistration] -> IO ()+registerPaths db regs = withTransaction (sdbConn db) $ do+  mapM_ (insertPathRow db) regs+  mapM_ (insertPathRefs db) regs++-- | Insert (or refresh) a single ValidPaths row.+insertPathRow :: StoreDB -> PathRegistration -> IO ()+insertPathRow db reg = do+  let pathText = T.pack (storePathToFilePath (sdbDir db) (prPath reg))   execute-    conn-    "INSERT OR IGNORE INTO ValidPaths (path, hash, registrationTime, deriver, narSize) \-    \VALUES (?, ?, strftime('%s','now'), ?, ?)"+    (sdbConn db)+    "INSERT INTO ValidPaths (path, hash, registrationTime, deriver, narSize) \+    \VALUES (?, ?, strftime('%s','now'), ?, ?) \+    \ON CONFLICT(path) DO UPDATE SET hash = excluded.hash, narSize = excluded.narSize, deriver = excluded.deriver"     (pathText, prNarHash reg, prDeriver reg, prNarSize reg)-  -- Lookup the referrer's id++-- | Insert the reference edges for a path whose row already exists.+insertPathRefs :: StoreDB -> PathRegistration -> IO ()+insertPathRefs db reg = do+  let conn = sdbConn db+      pathText = T.pack (storePathToFilePath (sdbDir db) (prPath reg))   referrerRows <- query conn "SELECT id FROM ValidPaths WHERE path = ?" (Only pathText) :: IO [Only Int]   case referrerRows of-    (Only referrerId : _) -> do-      -- Insert references-      mapM_ (insertRef conn referrerId) (prReferences reg)-    [] -> pure () -- Should not happen after INSERT OR IGNORE+    (Only referrerId : _) -> mapM_ (insertRef conn referrerId) (prReferences reg)+    [] -> pure () -- Should not happen: the row was just inserted above.   where     insertRef conn referrerId refPath = do       let refPathText = T.pack (storePathToFilePath (sdbDir db) refPath)@@ -186,7 +207,7 @@       case refRows of         (Only refId : _) ->           execute conn "INSERT OR IGNORE INTO Refs (referrer, reference) VALUES (?, ?)" (referrerId, refId)-        [] -> pure () -- Reference not yet registered — skip+        [] -> pure () -- Reference not in this batch or the store yet — skip.  -- --------------------------------------------------------------------------- -- Queries

@@ -42,6 +42,8 @@     -- * Pure helpers (exported for testing)     sortCaches,     verifySigs,+    verifyNarHash,+    narInfoMatchesPath,     decompressNar,     unpackNarEntry,     parseReferences,@@ -62,7 +64,8 @@ import qualified Network.HTTP.Types.Status as HTTP import Nix.Store (Store (..), setReadOnly) import Nix.Store.DB (PathRegistration (..), registerPath)-import Nix.Store.Path (StoreDir, StorePath (..), parseStorePath, storePathToFilePath)+import Nix.Store.Path (StoreDir, StorePath (..), parseStorePath, storePathHashLen, storePathToFilePath)+import qualified NovaCache.Hash as Hash import qualified NovaCache.NAR as NAR import qualified NovaCache.NarInfo as NarInfo import qualified NovaCache.Signing as Signing@@ -116,7 +119,9 @@ trySubstitute :: Store -> [CacheConfig] -> StorePath -> IO SubstResult trySubstitute _ [] _ = pure SubstNotFound trySubstitute store caches sp = do-  manager <- HTTPS.newTlsManager+  -- Reuse the process-global TLS manager (connection pooling / keep-alive)+  -- rather than creating a fresh one per call and per output.+  manager <- HTTPS.getGlobalManager   tryCaches manager (sortCaches caches) sp   where     tryCaches _ [] _ = pure SubstNotFound@@ -145,15 +150,28 @@   narInfoResult <- fetchNarInfo mgr cache sp   case narInfoResult of     Left notFoundOrErr -> pure notFoundOrErr-    Right narInfo -> do-      -- 2–4. Verify, download, decompress (pure pipeline after fetch)-      case verifyAndDecompress cache mgr narInfo of-        Left err -> pure (SubstError err)-        Right fetchDecompress -> do-          narBytes <- fetchDecompress-          case narBytes of+    Right narInfo+      -- 1b. The served narinfo must describe the requested path.  A+      -- misconfigured or hostile cache could return a validly-signed narinfo+      -- for a DIFFERENT path under this hash's URL.+      | not (narInfoMatchesPath sp narInfo) ->+          pure+            ( SubstError+                ( "narinfo identity mismatch: requested "+                    <> spHash sp+                    <> ", narinfo names "+                    <> NarInfo.niStorePath narInfo+                )+            )+      | otherwise ->+          -- 2–4. Verify, download, decompress (pure pipeline after fetch)+          case verifyAndDecompress cache mgr narInfo of             Left err -> pure (SubstError err)-            Right rawNar -> unpackAndRegister store sp narInfo rawNar+            Right fetchDecompress -> do+              narBytes <- fetchDecompress+              case narBytes of+                Left err -> pure (SubstError err)+                Right rawNar -> unpackAndRegister store sp narInfo rawNar  -- | Pure pipeline: verify signature, then produce an IO action -- that downloads and decompresses.@@ -169,29 +187,73 @@     downloaded <- downloadNar mgr cache narInfo     pure $ downloaded >>= decompressNar compression --- | Deserialize a NAR, unpack to the store, set permissions, and register.+-- | Verify the NAR hash, deserialize, unpack to the store, set permissions,+-- and register. unpackAndRegister :: Store -> StorePath -> NarInfo.NarInfo -> BS.ByteString -> IO SubstResult unpackAndRegister store sp narInfo rawNar =-  case NAR.deserialise rawNar of-    Left err -> pure (SubstError ("NAR deserialisation failed: " <> T.pack err))-    Right narEntry -> do-      let destPath = storePathToFilePath (stDir store) sp-      unpackResult <- try (unpackNarEntry destPath narEntry)-      case (unpackResult :: Either SomeException ()) of-        Left err -> pure (SubstError ("unpack failed: " <> T.pack (show err)))-        Right () -> do-          setReadOnly destPath-          registerPath-            (stDB store)-            PathRegistration-              { prPath = sp,-                prNarHash = NarInfo.niNarHash narInfo,-                prNarSize = fromInteger (NarInfo.niNarSize narInfo),-                prDeriver = NarInfo.niDeriver narInfo,-                prReferences = parseReferences (stDir store) (NarInfo.niReferences narInfo)-              }-          pure (SubstSuccess sp)+  -- Verify the downloaded NAR's hash matches the (signed) narinfo BEFORE+  -- trusting its bytes.  The NAR hash is the content-addressed integrity+  -- contract; the signature only attests to the narinfo, not the body, so+  -- network corruption or a compromised cache must be caught here.+  case verifyNarHash narInfo rawNar of+    Left err -> pure (SubstError err)+    Right () -> case NAR.deserialise rawNar of+      Left err -> pure (SubstError ("NAR deserialisation failed: " <> T.pack err))+      Right narEntry -> do+        let destPath = storePathToFilePath (stDir store) sp+        unpackResult <- try (unpackNarEntry destPath narEntry)+        case (unpackResult :: Either SomeException (Either Text ())) of+          Left err -> pure (SubstError ("unpack failed: " <> T.pack (show err)))+          Right (Left err) -> pure (SubstError ("unpack failed: " <> err))+          Right (Right ()) -> do+            setReadOnly destPath+            registerPath+              (stDB store)+              PathRegistration+                { prPath = sp,+                  prNarHash = NarInfo.niNarHash narInfo,+                  prNarSize = fromInteger (NarInfo.niNarSize narInfo),+                  prDeriver = NarInfo.niDeriver narInfo,+                  prReferences = parseReferences (stDir store) (NarInfo.niReferences narInfo)+                }+            pure (SubstSuccess sp) +-- | Verify that the downloaded NAR bytes hash to the narinfo's declared+-- NarHash.  Compares decoded hash bytes, so any valid encoding of the digest+-- validates; @prNarHash@ stays sourced from the (now-verified) narinfo.+verifyNarHash :: NarInfo.NarInfo -> BS.ByteString -> Either Text ()+verifyNarHash narInfo rawNar =+  case Hash.parseNixHash (NarInfo.niNarHash narInfo) of+    Left err -> Left ("invalid narinfo NarHash: " <> T.pack err)+    Right declared+      | declared == actual -> Right ()+      | otherwise ->+          Left+            ( "NAR hash mismatch: narinfo declares "+                <> NarInfo.niNarHash narInfo+                <> " but downloaded bytes hash to "+                <> Hash.formatNixHash actual+            )+  where+    actual = Hash.hashBytes rawNar++-- | Whether a served narinfo describes the requested path: the hash component+-- of its declared StorePath must equal the requested path's hash.  Only the+-- hash is compared, since the cache may use a different store directory.+narInfoMatchesPath :: StorePath -> NarInfo.NarInfo -> Bool+narInfoMatchesPath sp narInfo =+  storePathHashOf (NarInfo.niStorePath narInfo) == Just (spHash sp)++-- | Extract the leading hash from a full store path's basename, if well-formed+-- (@\<hash\>-\<name\>@ with a hash of the expected length).+storePathHashOf :: Text -> Maybe Text+storePathHashOf path =+  let base = T.takeWhileEnd (\c -> c /= '/' && c /= '\\') path+      (hashPart, rest) = T.splitAt storePathHashLen base+   in if T.length hashPart == storePathHashLen && T.isPrefixOf "-" rest+        then Just hashPart+        else Nothing+ -- --------------------------------------------------------------------------- -- HTTP fetching -- ---------------------------------------------------------------------------@@ -221,6 +283,10 @@         else pure (Left (SubstError ("narinfo fetch failed: HTTP " <> T.pack (show code))))  -- | Download the NAR file referenced by a narinfo.+--+-- The whole NAR is realized in memory (nova-cache's 'NAR.deserialise' consumes+-- a strict 'BS.ByteString'), so a very large path briefly spikes RSS.  Streaming+-- would need a streaming NAR parser that nova-cache does not yet provide. downloadNar :: HTTP.Manager -> CacheConfig -> NarInfo.NarInfo -> IO (Either Text BS.ByteString) downloadNar mgr cache narInfo = do   let narUrl = T.unpack (ccUrl cache) <> "/" <> T.unpack (NarInfo.niUrl narInfo)@@ -273,8 +339,10 @@ -- NAR unpacking -- --------------------------------------------------------------------------- --- | Unpack a NarEntry tree to a filesystem destination.-unpackNarEntry :: FilePath -> NAR.NarEntry -> IO ()+-- | Unpack a NarEntry tree to a filesystem destination.  Returns @Left@ on an+-- unsafe entry name (path traversal); these come from untrusted cache data, so+-- a typed failure is used instead of a partial 'error'.+unpackNarEntry :: FilePath -> NAR.NarEntry -> IO (Either Text ()) unpackNarEntry path entry = case entry of   NAR.NarRegular isExec contents -> do     createDirectoryIfMissing True (takeDirectory path)@@ -282,24 +350,33 @@     when isExec $ do       perms <- Dir.getPermissions path       setPermissions path (Dir.setOwnerExecutable True perms)+    pure (Right ())   NAR.NarSymlink target -> do     createDirectoryIfMissing True (takeDirectory path)     -- On Windows, symlinks require elevated permissions.     -- Fall back to writing the target as a text file.     result <- try (Dir.createFileLink (T.unpack target) path)     case result of-      Right () -> pure ()-      Left (_ :: SomeException) ->+      Right () -> pure (Right ())+      Left (_ :: SomeException) -> do         writeFile path (T.unpack target)+        pure (Right ())   NAR.NarDirectory entries -> do     createDirectoryIfMissing True path-    mapM_-      ( \(name, child) ->-          if isSafeNarName name-            then unpackNarEntry (path </> T.unpack name) child-            else error ("unpackNarEntry: unsafe directory entry name: " <> T.unpack name)-      )-      entries+    unpackChildren path entries++-- | Unpack directory children, short-circuiting with a typed failure on the+-- first unsafe entry name rather than crashing on untrusted input.+unpackChildren :: FilePath -> [(Text, NAR.NarEntry)] -> IO (Either Text ())+unpackChildren _ [] = pure (Right ())+unpackChildren path ((name, child) : rest)+  | not (isSafeNarName name) =+      pure (Left ("unsafe NAR directory entry name: " <> name))+  | otherwise = do+      result <- unpackNarEntry (path </> T.unpack name) child+      case result of+        Left err -> pure (Left err)+        Right () -> unpackChildren path rest  -- | Validate that a NAR entry name is safe (no path traversal). isSafeNarName :: Text -> Bool

@@ -1,12 +1,17 @@ {-# LANGUAGE LambdaCase #-}+{-# LANGUAGE PatternSynonyms #-}  module Main (main) where +import qualified Codec.Archive.Tar as Tar+import qualified Codec.Archive.Tar.Entry as TarEntry+import qualified Codec.Compression.Zstd.Lazy as ZstdL import Control.Exception (bracket_) import Control.Monad (filterM, void, when) import qualified Data.ByteString as BS+import qualified Data.ByteString.Lazy as BL import qualified Data.Map.Strict as Map-import Data.Maybe (isJust)+import Data.Maybe (fromMaybe, isJust) import qualified Data.Set as Set import Data.Text (Text) import qualified Data.Text as T@@ -15,13 +20,14 @@ import Foreign.Ptr (castPtr) import Foreign.StablePtr (StablePtr, castPtrToStablePtr, castStablePtrToPtr, deRefStablePtr, freeStablePtr, newStablePtr) import Nix.Builder (BuildConfig (..), BuildResult (..), buildDerivation, buildWithDeps, defaultBuildConfig)+import Nix.Builder.Unpack (builtinUnpackBuilder, envSrcs) import Nix.Builtins (builtinEnv, parseNixPath) import qualified Nix.DependencyGraph as DepGraph import Nix.Derivation (Derivation (..), DerivationOutput (..), Platform (..), currentPlatform, fromATerm, platformToText, toATerm)-import Nix.Eval (NixValue (..), StringContext (..), StringContextElement (..), attrSetFromMap, attrSetLookup, attrSetNull, attrSetSize, emptyContext, emptyEnv, eval, force, mkStr, readThunkValue, runPureEval)+import Nix.Eval (NixValue (..), StringContext (..), StringContextElement (..), attrSetFromMap, attrSetLookup, attrSetNull, attrSetSize, builtinNames, emptyContext, emptyEnv, eval, force, mkStr, readThunkValue, runPureEval) import Nix.Eval.Arena (arenaDestroy, arenaInit) import Nix.Eval.CAttrSet (cattrsetFreeze, cattrsetInsert, cattrsetKeys, cattrsetLookup, cattrsetNew, cattrsetSize, cattrsetUnion)-import Nix.Eval.CBytecode (binaryAdd, captureSlots, captureWithScopes, cbcArg1, cbcArg2, cbcArg3, cbcData, cbcFlags, cbcOpCount, cbcOpcode, cbcShortArg, formalName, formalNamedSet, formalSet, opApp, opAssert, opAttrs, opBinary, opHasAttr, opIf, opIndStr, opLambda, opLet, opList, opLitBool, opLitFloat, opLitInt, opLitNull, opLitPath, opLitUri, opResolvedVar, opSelect, opStr, opUnary, opVar, opWith, opWithVar, strpartInterp, strpartLit, unaryNegate)+import Nix.Eval.CBytecode (binaryAdd, captureSlots, captureWithScopes, cbcArg1, cbcArg2, cbcArg3, cbcData, cbcFlags, cbcOpCount, cbcOpcode, cbcShortArg, formalName, formalNamedSet, formalSet, strpartInterp, strpartLit, unaryNegate, pattern OpApp, pattern OpAssert, pattern OpAttrs, pattern OpBinary, pattern OpHasAttr, pattern OpIf, pattern OpIndStr, pattern OpLambda, pattern OpLet, pattern OpList, pattern OpLitBool, pattern OpLitFloat, pattern OpLitInt, pattern OpLitNull, pattern OpLitPath, pattern OpLitUri, pattern OpResolvedVar, pattern OpSelect, pattern OpStr, pattern OpUnary, pattern OpVar, pattern OpWith, pattern OpWithVar) import Nix.Eval.CThunk (CThunkPtr, cthunkCount, cthunkGet, cthunkMarkBlackhole, cthunkNew, cthunkNewComputed, cthunkPayload, cthunkSetComputed, cthunkState) import Nix.Eval.Compile (compileExpr) import qualified Nix.Eval.Context as Context@@ -29,12 +35,15 @@ import Nix.Eval.Symbol (Symbol (..), symbolCount, symbolIntern, symbolLen, symbolText) import Nix.Eval.Types (emptyCList) import Nix.Expr.Types+import qualified Nix.Hash as Hash import Nix.Parser (parseNix) import Nix.Parser.Lexer (Located (..), Token (..), tokenize) import Nix.Store (Store (..), addToStore, closeStore, isValid, openStore, pathExists, scanReferences, setReadOnly, writeDrv) import Nix.Store.DB (PathInfo (..), PathRegistration (..), closeStoreDB, isValidPath, openStoreDB, queryDeriver, queryPathInfo, queryReferences, registerPath) import Nix.Store.Path (StoreDir (..), StorePath (..), defaultStoreDir, parseStorePath, platformStoreDirText, storePathToFilePath, storePathToText, windowsStoreDir) import qualified Nix.Substituter as Subst+import qualified NovaCache.Hash as CHash+import qualified NovaCache.NarInfo as NarInfo import System.Directory (createDirectoryIfMissing, doesDirectoryExist, getPermissions, getTemporaryDirectory, removeDirectoryRecursive, writable) import qualified System.Directory as Dir import System.Exit (ExitCode (..), exitFailure, exitSuccess)@@ -276,7 +285,13 @@       runTest "negate int" $         assertEval "negate" "- 5" (VInt (-5)),       runTest "division by zero" $-        assertEvalFail "div0" "1 / 0"+        assertEvalFail "div0" "1 / 0",+      runTest "absolute path lexes as path, not division" $+        assertEval "path-abs" "builtins.typeOf /abs/path" (mkStr "path"),+      runTest "bareword relative path lexes as path" $+        assertEval "path-rel" "builtins.typeOf foo/bar" (mkStr "path"),+      runTest "function applied to an absolute path argument" $+        assertEval "app-abs-path" "(p: builtins.typeOf p) /abs/path" (mkStr "path")     ]  -- ---------------------------------------------------------------------------@@ -2286,8 +2301,51 @@         assertEqual "default-url" "https://cache.nixos.org" (Subst.ccUrl Subst.defaultCacheConfig),       -- defaultCacheConfig has priority 40       runTest "defaultCacheConfig priority" $-        assertEqual "default-prio" 40 (Subst.ccPriority Subst.defaultCacheConfig)+        assertEqual "default-prio" 40 (Subst.ccPriority Subst.defaultCacheConfig),+      -- verifyNarHash: matching NAR hash accepted (HIGH#2 integrity gate)+      runTest "verifyNarHash accepts matching hash" $+        assertEqual "narhash-match" (Right ()) (Subst.verifyNarHash (sampleNarInfo sampleNarHash) sampleNarBytes),+      -- verifyNarHash: mismatched NAR hash rejected+      runTest "verifyNarHash rejects mismatched hash" $+        case Subst.verifyNarHash (sampleNarInfo wrongNarHash) sampleNarBytes of+          Left _ -> Pass+          Right () -> Fail "expected mismatch rejection",+      -- verifyNarHash: malformed NAR hash rejected+      runTest "verifyNarHash rejects malformed hash" $+        case Subst.verifyNarHash (sampleNarInfo "not-a-hash") sampleNarBytes of+          Left _ -> Pass+          Right () -> Fail "expected malformed-hash rejection",+      -- narInfoMatchesPath: identity match accepted+      runTest "narInfoMatchesPath accepts matching identity" $+        if Subst.narInfoMatchesPath (StorePath sampleHash "hello") (sampleNarInfo sampleNarHash)+          then Pass+          else Fail "expected identity match",+      -- narInfoMatchesPath: identity mismatch rejected+      runTest "narInfoMatchesPath rejects mismatched identity" $+        if Subst.narInfoMatchesPath (StorePath (T.replicate 32 "b") "hello") (sampleNarInfo sampleNarHash)+          then Fail "expected identity mismatch"+          else Pass     ]+  where+    sampleNarBytes = "nova-nix nar sample bytes" :: BS.ByteString+    sampleNarHash = CHash.formatNixHash (CHash.hashBytes sampleNarBytes)+    wrongNarHash = CHash.formatNixHash (CHash.hashBytes ("different bytes" :: BS.ByteString))+    sampleHash = T.replicate 32 "a"+    sampleNarInfo narHash =+      NarInfo.NarInfo+        { NarInfo.niStorePath = "/nix/store/" <> sampleHash <> "-hello",+          NarInfo.niUrl = "nar/sample.nar",+          NarInfo.niCompression = "none",+          NarInfo.niFileHash = sampleNarHash,+          NarInfo.niFileSize = 0,+          NarInfo.niNarHash = narHash,+          NarInfo.niNarSize = fromIntegral (BS.length sampleNarBytes),+          NarInfo.niReferences = [],+          NarInfo.niDeriver = Nothing,+          NarInfo.niSigs = [],+          NarInfo.niCA = Nothing,+          NarInfo.niSystem = Nothing+        }  -- --------------------------------------------------------------------------- -- Tests: Build orchestrator (Phase 3, Batch 7)@@ -2603,6 +2661,402 @@     perms <- getPermissions path     Dir.setPermissions path (Dir.setOwnerWritable True perms) +-- | Step 1 of the Windows-stdenv ladder: prove the Builder can run a+-- trivial derivation end-to-end natively — a recipe that writes to @$out@,+-- with the output landing in the store.  No stdenv, no dependencies: just+-- the raw build path (process spawn -> output capture -> addToStore),+-- exercised on the host platform (@cmd.exe@ on Windows, @\/bin\/sh@ elsewhere).+testTrivialBuildIO :: IO [Bool]+testTrivialBuildIO = do+  putStrLn "builder/trivial-native-build"+  withTempStore $ \store -> do+    let storeDir = stDir store+        outPath = StorePath (T.replicate 31 "0" <> "1") "trivial"+        (builder, args)+          | SI.os == "mingw32" = ("cmd.exe", ["/c", "echo hi>%out%"])+          | otherwise = ("/bin/sh", ["-c", "echo hi > $out"])+        drv =+          Derivation+            { drvOutputs =+                [ DerivationOutput+                    { doName = "out",+                      doPath = outPath,+                      doHashAlgo = "",+                      doHash = ""+                    }+                ],+              drvInputDrvs = Map.empty,+              drvInputSrcs = [],+              drvPlatform = currentPlatform,+              drvBuilder = builder,+              drvArgs = args,+              drvEnv = Map.empty+            }+    buildTmp <- (</> "nova-nix-test-trivial-build-tmp") <$> getTemporaryDirectory+    forceRemoveIfExists buildTmp+    createDirectoryIfMissing True buildTmp+    let config = (defaultBuildConfig storeDir) {bcTmpDir = buildTmp}+    result <- buildDerivation config store drv+    forceRemoveIfExists buildTmp+    case result of+      BuildSuccess sp -> do+        let outFile = storePathToFilePath storeDir sp+        landed <- Dir.doesFileExist outFile+        content <- if landed then TIO.readFile outFile else pure ""+        narInfo <- queryPathInfo (stDB store) sp+        let realNarHash = case narInfo of+              Just info ->+                piNarSize info > 0+                  && T.isPrefixOf "sha256:" (piNarHash info)+                  && T.any (/= '0') (T.drop 7 (piNarHash info))+              Nothing -> False+        sequence+          [ runTest "trivial build succeeds" Pass,+            runTest "trivial build output landed in store" $+              if landed then Pass else Fail ("missing output file: " <> T.pack outFile),+            runTest "trivial build actually ran the command" $+              if "hi" `T.isInfixOf` content+                then Pass+                else Fail ("unexpected output content: " <> T.pack (show content)),+            runTest "trivial build records a real NAR hash + size" $+              if realNarHash+                then Pass+                else Fail ("NAR hash/size not real: " <> T.pack (show narInfo))+          ]+      BuildFailure msg code ->+        sequence+          [ runTest "trivial build succeeds" $+              Fail ("build failed (exit " <> T.pack (show code) <> "): " <> msg)+          ]++-- | Zstd compression level for test archives (zstd's own default).+testZstdCompressionLevel :: Int+testZstdCompressionLevel = 3++-- | Build a @.tar.zst@ archive from tar entries, MSYS2-package style.+compressArchive :: [TarEntry.Entry] -> BL.ByteString+compressArchive = ZstdL.compress testZstdCompressionLevel . Tar.write++-- | Test-setup helpers: the paths and link targets below are short literals,+-- so encoding them cannot fail; 'error' marks setup bugs, not test failures.+tarPathOrDie :: Bool -> FilePath -> TarEntry.TarPath+tarPathOrDie isDir p =+  either (error . ("test tar path: " ++)) id (TarEntry.toTarPath isDir p)++linkOrDie :: FilePath -> TarEntry.LinkTarget+linkOrDie t =+  fromMaybe (error ("test link target: " ++ t)) (TarEntry.toLinkTarget t)++tarDir :: FilePath -> TarEntry.Entry+tarDir p = TarEntry.directoryEntry (tarPathOrDie True p)++tarFile :: FilePath -> BL.ByteString -> TarEntry.Entry+tarFile p = TarEntry.fileEntry (tarPathOrDie False p)++tarExecFile :: FilePath -> BL.ByteString -> TarEntry.Entry+tarExecFile p content = (tarFile p content) {TarEntry.entryPermissions = 0o755}++tarHardLink :: FilePath -> FilePath -> TarEntry.Entry+tarHardLink p target =+  TarEntry.simpleEntry (tarPathOrDie False p) (Tar.HardLink (linkOrDie target))++tarSymLink :: FilePath -> FilePath -> TarEntry.Entry+tarSymLink p target =+  TarEntry.simpleEntry (tarPathOrDie False p) (Tar.SymbolicLink (linkOrDie target))++-- | Step 3 of the ladder: @builtin:unpack@, the stage-0 seed extractor.+-- Two zstd-compressed tar archives sharing a top-level prefix (the MSYS2+-- @mingw64\/@ shape) are extracted into ONE output: regular files, an+-- executable, a hardlink and a relative symlink (both materialized as+-- copies), with pacman metadata (@.PKGINFO@\/@.MTREE@) skipped.  A second+-- build proves cross-archive file collisions fail loudly, and a third that+-- path traversal is rejected.+testUnpackBuildIO :: IO [Bool]+testUnpackBuildIO = do+  putStrLn "builder/unpack-seed-archives"+  withTempStore $ \store -> do+    let storeDir = stDir store+    tmpBase <- getTemporaryDirectory+    let workDir = tmpBase </> "nova-nix-test-unpack-src"+    forceRemoveIfExists workDir+    createDirectoryIfMissing True workDir+    let archiveTools = workDir </> "pkg-tools.tar.zst"+        archiveData = workDir </> "pkg-data.tar.zst"+        archiveCollide = workDir </> "pkg-collide.tar.zst"+        archiveEscape = workDir </> "pkg-escape.tar.zst"+    BL.writeFile archiveTools $+      compressArchive+        [ tarDir "pkg",+          tarDir "pkg/bin",+          tarExecFile "pkg/bin/tool.exe" "tool-payload",+          tarHardLink "pkg/bin/tool-link.exe" "pkg/bin/tool.exe",+          tarSymLink "pkg/bin/sh.exe" "tool.exe",+          tarFile ".PKGINFO" "pkgname = tools"+        ]+    BL.writeFile archiveData $+      compressArchive+        [ tarDir "pkg",+          tarDir "pkg/share",+          tarFile "pkg/share/data.txt" "shared-data",+          tarFile ".MTREE" "mtree-bytes"+        ]+    BL.writeFile archiveCollide $+      compressArchive [tarFile "pkg/bin/tool.exe" "conflicting"]+    BL.writeFile archiveEscape $+      compressArchive [tarFile "../evil.txt" "escape"]+    let mkUnpackDrv outP srcFiles =+          Derivation+            { drvOutputs =+                [ DerivationOutput+                    { doName = "out",+                      doPath = outP,+                      doHashAlgo = "",+                      doHash = ""+                    }+                ],+              drvInputDrvs = Map.empty,+              drvInputSrcs = [],+              drvPlatform = currentPlatform,+              drvBuilder = builtinUnpackBuilder,+              drvArgs = [],+              drvEnv =+                Map.fromList+                  [(envSrcs, T.intercalate " " (map T.pack srcFiles))]+            }+        seedOut = StorePath (T.replicate 31 "0" <> "2") "unpack-seed"+        collideOut = StorePath (T.replicate 31 "0" <> "3") "unpack-collide"+        escapeOut = StorePath (T.replicate 31 "0" <> "4") "unpack-escape"+    buildTmp <- (</> "nova-nix-test-unpack-build-tmp") <$> getTemporaryDirectory+    forceRemoveIfExists buildTmp+    createDirectoryIfMissing True buildTmp+    let config = (defaultBuildConfig storeDir) {bcTmpDir = buildTmp}+    seedResult <- buildDerivation config store (mkUnpackDrv seedOut [archiveTools, archiveData])+    collideResult <- buildDerivation config store (mkUnpackDrv collideOut [archiveTools, archiveCollide])+    escapeResult <- buildDerivation config store (mkUnpackDrv escapeOut [archiveEscape])+    forceRemoveIfExists buildTmp+    forceRemoveIfExists workDir+    seedChecks <- case seedResult of+      BuildFailure msg code ->+        sequence+          [ runTest "unpack seed build succeeds" $+              Fail ("build failed (exit " <> T.pack (show code) <> "): " <> msg)+          ]+      BuildSuccess sp -> do+        let outRoot = storePathToFilePath storeDir sp+            readOut rel = do+              let path = outRoot </> rel+              exists <- Dir.doesFileExist path+              if exists then Just <$> TIO.readFile path else pure Nothing+        tool <- readOut ("pkg" </> "bin" </> "tool.exe")+        toolLink <- readOut ("pkg" </> "bin" </> "tool-link.exe")+        shLink <- readOut ("pkg" </> "bin" </> "sh.exe")+        dataFile <- readOut ("pkg" </> "share" </> "data.txt")+        pkgInfo <- Dir.doesFileExist (outRoot </> ".PKGINFO")+        mtree <- Dir.doesFileExist (outRoot </> ".MTREE")+        execBitOk <-+          if SI.os == "mingw32"+            then pure True -- NTFS has no exec bit; PATHEXT decides+            else Dir.executable <$> getPermissions (outRoot </> "pkg" </> "bin" </> "tool.exe")+        narInfo <- queryPathInfo (stDB store) sp+        let realNarHash = case narInfo of+              Just info ->+                piNarSize info > 0 && T.isPrefixOf "sha256:" (piNarHash info)+              Nothing -> False+        sequence+          [ runTest "unpack seed build succeeds" Pass,+            runTest "unpack: file extracted with content" $+              assertEqual "tool.exe" (Just "tool-payload") tool,+            runTest "unpack: hardlink materialized as copy" $+              assertEqual "tool-link.exe" (Just "tool-payload") toolLink,+            runTest "unpack: relative symlink materialized as copy" $+              assertEqual "sh.exe" (Just "tool-payload") shLink,+            runTest "unpack: second archive merged into shared prefix" $+              assertEqual "data.txt" (Just "shared-data") dataFile,+            runTest "unpack: pacman metadata skipped" $+              if pkgInfo || mtree+                then Fail ".PKGINFO/.MTREE leaked into the output"+                else Pass,+            runTest "unpack: executable bit materialized (unix)" $+              if execBitOk then Pass else Fail "tool.exe not executable",+            runTest "unpack: real NAR hash registered" $+              if realNarHash then Pass else Fail (T.pack (show narInfo))+          ]+    collideChecks <-+      sequence+        [ runTest "unpack: cross-archive file collision fails loudly" $+            case collideResult of+              BuildFailure msg _+                | "file collision" `T.isInfixOf` msg -> Pass+                | otherwise -> Fail ("wrong failure: " <> msg)+              BuildSuccess _ -> Fail "collision build unexpectedly succeeded"+        ]+    escapeChecks <-+      sequence+        [ runTest "unpack: path traversal rejected" $+            case escapeResult of+              BuildFailure msg _+                | "escapes archive root" `T.isInfixOf` msg -> Pass+                | otherwise -> Fail ("wrong failure: " <> msg)+              BuildSuccess _ -> Fail "escape build unexpectedly succeeded"+        ]+    pure (seedChecks ++ collideChecks ++ escapeChecks)++-- | Step 2 of the ladder: a dependency-aware build end-to-end.  A root+-- derivation depends on a leaf; both @.drv@ files are pre-written to the store+-- (as the build driver's closure-writing does), and 'buildWithDeps' must read+-- the closure, topologically order it, build the leaf first, then the root —+-- whose 'validateInputs' requires the leaf's realized output to be valid.+--+-- This is the regression guard for the input-@.drv@-closure fix: before it, no+-- derivation with a non-empty @drvInputDrvs@ could be realized (the closure was+-- never on disk, so the dependency graph could not be read).+testDependentBuildIO :: IO [Bool]+testDependentBuildIO = do+  putStrLn "builder/dependent-native-build"+  withTempStore $ \store -> do+    let storeDir = stDir store+        depOut = StorePath (T.replicate 31 "c" <> "0") "dep"+        depDrvSP = StorePath (T.replicate 31 "c" <> "1") "dep.drv"+        rootOut = StorePath (T.replicate 31 "a" <> "0") "root"+        rootDrvSP = StorePath (T.replicate 31 "a" <> "1") "root.drv"+        mkBuilder word+          | SI.os == "mingw32" = ("cmd.exe", ["/c", "echo " <> word <> ">%out%"])+          | otherwise = ("/bin/sh", ["-c", "echo " <> word <> " > $out"])+        (depBuilder, depArgs) = mkBuilder "leaf"+        (rootBuilder, rootArgs) = mkBuilder "root"+        depDrv =+          Derivation+            { drvOutputs = [DerivationOutput {doName = "out", doPath = depOut, doHashAlgo = "", doHash = ""}],+              drvInputDrvs = Map.empty,+              drvInputSrcs = [],+              drvPlatform = currentPlatform,+              drvBuilder = depBuilder,+              drvArgs = depArgs,+              drvEnv = Map.singleton "name" "dep"+            }+        rootDrv =+          Derivation+            { drvOutputs = [DerivationOutput {doName = "out", doPath = rootOut, doHashAlgo = "", doHash = ""}],+              drvInputDrvs = Map.singleton depDrvSP ["out"],+              drvInputSrcs = [],+              drvPlatform = currentPlatform,+              drvBuilder = rootBuilder,+              drvArgs = rootArgs,+              drvEnv = Map.singleton "name" "root"+            }+    -- The build driver writes the full .drv closure before building; emulate+    -- that here by writing both recipes to the store.+    writeDrv store depDrv depDrvSP+    writeDrv store rootDrv rootDrvSP+    buildTmp <- (</> "nova-nix-test-dep-build-tmp") <$> getTemporaryDirectory+    forceRemoveIfExists buildTmp+    createDirectoryIfMissing True buildTmp+    let config = (defaultBuildConfig storeDir) {bcTmpDir = buildTmp}+    result <- buildWithDeps config store rootDrv rootDrvSP+    depValid <- isValid store depOut+    rootValid <- isValid store rootOut+    forceRemoveIfExists buildTmp+    case result of+      BuildSuccess sp ->+        sequence+          [ runTest "dependent build succeeds with root output" $+              assertEqual "root output path" rootOut sp,+            runTest "leaf dependency built and registered first" $+              if depValid then Pass else Fail "leaf output not valid after build",+            runTest "root output built and registered" $+              if rootValid then Pass else Fail "root output not valid after build"+          ]+      BuildFailure msg code ->+        sequence+          [ runTest "dependent build succeeds with root output" $+              Fail ("dependency-aware build failed (exit " <> T.pack (show code) <> "): " <> msg)+          ]++-- | Regression tests for the 2026-06-09 audit eval-fidelity fixes.  All are+-- parity-safe — none affects a derivation or store-path hash.+testEvalFidelity :: IO [Bool]+testEvalFidelity = do+  putStrLn "eval/audit-fidelity"+  sequence+    [ -- builtins.match: a non-participating capture group is null, not ""+      runTest "match null capture group" $+        assertEval "match-null" "builtins.isNull (builtins.elemAt (builtins.match \"(a)(b)?\" \"a\") 1)" (VBool True),+      runTest "match participating group value" $+        assertEval "match-val" "builtins.elemAt (builtins.match \"(a)(b)\" \"ab\") 1" (mkStr "b"),+      -- builtins.split: a non-participating group is null too+      runTest "split null capture group" $+        assertEval "split-null" "builtins.isNull (builtins.elemAt (builtins.elemAt (builtins.split \"(a)|(b)\" \"a\") 1) 1)" (VBool True),+      -- builtins.toJSON honors __toString, preferring it over outPath+      runTest "toJSON __toString" $+        assertEval "tojson-tostr" "builtins.toJSON { __toString = self: \"x\"; }" (mkStr "\"x\""),+      runTest "toJSON __toString beats outPath" $+        assertEval "tojson-tostr-out" "builtins.toJSON { __toString = self: \"a\"; outPath = \"/b\"; }" (mkStr "\"a\""),+      runTest "toJSON outPath still works" $+        assertEval "tojson-out" "builtins.toJSON { outPath = \"/b\"; }" (mkStr "\"/b\""),+      -- String interpolation is strict (coerceMore = False): scalars error+      runTest "interp rejects int" $+        assertEvalFail "interp-int" "\"v${1}\"",+      runTest "interp rejects bool" $+        assertEvalFail "interp-bool" "\"${true}\"",+      runTest "interp rejects null" $+        assertEvalFail "interp-null" "\"${null}\"",+      runTest "concatStringsSep rejects int" $+        assertEvalFail "ccs-int" "builtins.concatStringsSep \",\" [ 1 2 ]",+      -- builtins.toString stays permissive+      runTest "toString still coerces int" $+        assertEval "tostr-int" "builtins.toString 1" (mkStr "1"),+      runTest "interp via toString works" $+        assertEval "interp-tostr" "\"v${builtins.toString 1}\"" (mkStr "v1"),+      -- builtins.substring rejects a negative start position+      runTest "substring negative start errors" $+        assertEvalFail "substr-neg" "builtins.substring (0 - 1) 3 \"hello\"",+      -- an integer literal that overflows Int64 is a parse error, not a wrap+      runTest "integer literal overflow errors" $+        assertEvalFail "int-overflow" "99999999999999999999",+      -- float exponents and leading-dot floats lex per the Nix grammar+      runTest "float exponent lexes as float" $+        assertEval "float-exp-type" "builtins.typeOf 6.022e23" (mkStr "float"),+      runTest "float negative exponent lexes" $+        assertEval "float-negexp-type" "builtins.typeOf 1.0e-3" (mkStr "float"),+      runTest "float exponent value" $+        assertEval "float-exp-val" "1.5e1 == 15.0" (VBool True),+      runTest "leading-dot float lexes as float" $+        assertEval "leading-dot-type" "builtins.typeOf .5" (mkStr "float"),+      runTest "leading-dot float value" $+        assertEval "leading-dot-val" ".5 == 0.5" (VBool True),+      -- PureEval tryEval must NOT catch abort — it propagates (matches C++ Nix)+      runTest "tryEval does not catch abort" $+        assertEvalFail "tryeval-abort" "(builtins.tryEval (builtins.abort \"x\")).success",+      -- every builtin in the registry (the source of builtinNames/builtinArity)+      -- is actually exposed in the builtins set — guards against builtinRegistry+      -- drifting from the exposed builtins+      runTest "every registered builtin is exposed" $+        let missing = [n | n <- builtinNames, evalNix ("builtins ? \"" <> n <> "\"") /= Right (VBool True)]+         in if null missing then Pass else Fail ("not exposed: " <> T.intercalate ", " missing)+    ]++-- | Pure hash decode/digest helpers shared by eval (fixed-output paths) and+-- the builder (builtin:fetchurl verification).+testHashHelpers :: IO [Bool]+testHashHelpers = do+  putStrLn "hash/decode-helpers"+  sequence+    [ runTest "hexToBytes empty" $ assertEqual "empty" (Just BS.empty) (Hash.hexToBytes ""),+      runTest "hexToBytes deadbeef" $+        assertEqual "deadbeef" (Just (BS.pack [0xde, 0xad, 0xbe, 0xef])) (Hash.hexToBytes "deadbeef"),+      runTest "hexToBytes uppercase" $+        assertEqual "DEADBEEF" (Just (BS.pack [0xde, 0xad, 0xbe, 0xef])) (Hash.hexToBytes "DEADBEEF"),+      runTest "hexToBytes odd length rejected" $ assertEqual "odd" Nothing (Hash.hexToBytes "abc"),+      runTest "hexToBytes non-hex rejected" $ assertEqual "nonhex" Nothing (Hash.hexToBytes "zz"),+      runTest "rawHashWithAlgo sha256 empty == known vector" $+        assertEqual+          "sha256-empty"+          (Hash.hexToBytes "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")+          (Hash.rawHashWithAlgo "sha256" BS.empty),+      runTest "rawHashWithAlgo unknown algo rejected" $+        assertEqual "unknown" Nothing (Hash.rawHashWithAlgo "sha3-256" (BS.pack [1, 2, 3]))+    ]+ testStoreOps :: IO [Bool] testStoreOps = do   putStrLn "store/ops"@@ -3670,7 +4124,7 @@         a1 <- cbcArg1 idx         a2 <- cbcArg2 idx         pure-          ( if op == opLitInt && a1 == 42 && a2 == 0+          ( if op == OpLitInt && a1 == 42 && a2 == 0               then Pass               else Fail ("op=" <> T.pack (show op) <> " a1=" <> T.pack (show a1))           ),@@ -3681,7 +4135,7 @@         a2 <- cbcArg2 idx         -- -1 as uint64 = 0xFFFFFFFFFFFFFFFF, lo=0xFFFFFFFF, hi=0xFFFFFFFF         pure-          ( if op == opLitInt && a1 == 0xFFFFFFFF && a2 == 0xFFFFFFFF+          ( if op == OpLitInt && a1 == 0xFFFFFFFF && a2 == 0xFFFFFFFF               then Pass               else                 Fail@@ -3699,21 +4153,21 @@         saT <- cbcShortArg idxT         saF <- cbcShortArg idxF         pure-          ( if opT == opLitBool && saT == 1 && opF == opLitBool && saF == 0+          ( if opT == OpLitBool && saT == 1 && opF == OpLitBool && saF == 0               then Pass               else Fail "bool encoding mismatch"           ),       runTestM "compile ELit NixNull" $ do         idx <- compileExpr (ELit NixNull)         op <- cbcOpcode idx-        pure (assertEqual "opcode" opLitNull op),+        pure (assertEqual "opcode" OpLitNull op),       runTestM "compile EResolvedVar" $ do         idx <- compileExpr (EResolvedVar 3 7)         op <- cbcOpcode idx         a1 <- cbcArg1 idx         a2 <- cbcArg2 idx         pure-          ( if op == opResolvedVar && a1 == 3 && a2 == 7+          ( if op == OpResolvedVar && a1 == 3 && a2 == 7               then Pass               else                 Fail@@ -3731,7 +4185,7 @@         sym <- cbcArg1 idx         let symText = symbolText (Symbol sym)         pure-          ( if op == opVar && symText == "hello"+          ( if op == OpVar && symText == "hello"               then Pass               else Fail ("op=" <> T.pack (show op) <> " sym=" <> symText)           ),@@ -3743,7 +4197,7 @@         funcOp <- cbcOpcode funcIdx         argOp <- cbcOpcode argIdx         pure-          ( if op == opApp && funcOp == opVar && argOp == opLitInt+          ( if op == OpApp && funcOp == OpVar && argOp == OpLitInt               then Pass               else                 Fail@@ -3767,7 +4221,7 @@         thenA1 <- cbcArg1 thenIdx         elseA1 <- cbcArg1 elseIdx         pure-          ( if op == opIf && condOp == opLitBool && thenA1 == 1 && elseA1 == 2+          ( if op == OpIf && condOp == OpLitBool && thenA1 == 1 && elseA1 == 2               then Pass               else Fail "if structure mismatch"           ),@@ -3782,7 +4236,7 @@         leftA1 <- cbcArg1 leftIdx         rightA1 <- cbcArg1 rightIdx         pure-          ( if op == opBinary && fl == binaryAdd && leftA1 == 10 && rightA1 == 20+          ( if op == OpBinary && fl == binaryAdd && leftA1 == 10 && rightA1 == 20               then Pass               else Fail "binary structure mismatch"           ),@@ -3793,7 +4247,7 @@         operandIdx <- cbcArg1 idx         operandA1 <- cbcArg1 operandIdx         pure-          ( if op == opUnary && fl == unaryNegate && operandA1 == 5+          ( if op == OpUnary && fl == unaryNegate && operandA1 == 5               then Pass               else Fail "unary structure mismatch"           ),@@ -3811,7 +4265,7 @@         c1a1 <- cbcArg1 c1         c2a1 <- cbcArg1 c2         pure-          ( if op == opList && count == 3 && c0a1 == 1 && c1a1 == 2 && c2a1 == 3+          ( if op == OpList && count == 3 && c0a1 == 1 && c1a1 == 2 && c2a1 == 3               then Pass               else                 Fail@@ -3837,12 +4291,12 @@         -- tag0=0(lit), tag1=1(interp), tag2=0(lit)         interpOp <- cbcOpcode val1         pure-          ( if op == opStr+          ( if op == OpStr               && count == 3               && tag0 == strpartLit               && tag1 == strpartInterp               && tag2 == strpartLit-              && interpOp == opVar+              && interpOp == OpVar               then Pass               else                 Fail@@ -3864,7 +4318,7 @@         scopeOp <- cbcOpcode scopeIdx         bodyOp <- cbcOpcode bodyIdx         pure-          ( if op == opWith && scopeOp == opVar && bodyOp == opVar+          ( if op == OpWith && scopeOp == OpVar && bodyOp == OpVar               then Pass               else Fail "with structure mismatch"           ),@@ -3876,7 +4330,7 @@         condOp <- cbcOpcode condIdx         bodyOp <- cbcOpcode bodyIdx         pure-          ( if op == opAssert && condOp == opLitBool && bodyOp == opLitInt+          ( if op == OpAssert && condOp == OpLitBool && bodyOp == OpLitInt               then Pass               else Fail "assert structure mismatch"           ),@@ -3889,7 +4343,7 @@         bodyIdx <- cbcArg2 idx         bodyOp <- cbcOpcode bodyIdx         pure-          ( if op == opLambda && fl == formalName && bodyOp == opResolvedVar+          ( if op == OpLambda && fl == formalName && bodyOp == OpResolvedVar               then Pass               else                 Fail@@ -3912,7 +4366,7 @@         bodyIdx <- cbcArg2 idx         bodyOp <- cbcOpcode bodyIdx         pure-          ( if op == opLet && count == 1 && bodyOp == opResolvedVar+          ( if op == OpLet && count == 1 && bodyOp == OpResolvedVar               then Pass               else                 Fail@@ -3934,7 +4388,7 @@         fl <- cbcFlags idx         count <- cbcShortArg idx         pure-          ( if op == opAttrs && fl == 0 && count == 1+          ( if op == OpAttrs && fl == 0 && count == 1               then Pass               else Fail "attrs structure mismatch"           ),@@ -3952,7 +4406,7 @@         fl <- cbcFlags idx         count <- cbcShortArg idx         pure-          ( if op == opAttrs && fl == 1 && count == 2+          ( if op == OpAttrs && fl == 1 && count == 2               then Pass               else                 Fail@@ -3969,7 +4423,7 @@         op <- cbcOpcode idx         fl <- cbcFlags idx         pure-          ( if op == opSelect && fl == 0+          ( if op == OpSelect && fl == 0               then Pass               else Fail ("flags=" <> T.pack (show fl))           ),@@ -3982,7 +4436,7 @@         defIdx <- cbcArg3 idx         defOp <- cbcOpcode defIdx         pure-          ( if op == opSelect && fl == 1 && defOp == opLitNull+          ( if op == OpSelect && fl == 1 && defOp == OpLitNull               then Pass               else Fail ("flags=" <> T.pack (show fl))           ),@@ -3993,7 +4447,7 @@         op <- cbcOpcode idx         pathLen <- cbcShortArg idx         pure-          ( if op == opHasAttr && pathLen == 2+          ( if op == OpHasAttr && pathLen == 2               then Pass               else                 Fail@@ -4009,9 +4463,9 @@         idx <- compileExpr (EApp (EApp (EVar "__findFile") (EVar "__nixPath")) (EStr [StrLit "nixpkgs"]))         op <- cbcOpcode idx         pure-          ( if op == opApp+          ( if op == OpApp               then Pass-              else Fail ("expected opApp, got op=" <> T.pack (show op))+              else Fail ("expected OpApp, got op=" <> T.pack (show op))           ),       runTestM "op_count grows after compilation" $ do         before <- cbcOpCount@@ -4031,13 +4485,13 @@       runTestM "compile ELit NixFloat" $ do         idx <- compileExpr (ELit (NixFloat 3.14))         op <- cbcOpcode idx-        pure (assertEqual "opcode" opLitFloat op),+        pure (assertEqual "opcode" OpLitFloat op),       runTestM "compile ELit NixUri" $ do         idx <- compileExpr (ELit (NixUri "https://example.com"))         op <- cbcOpcode idx         sym <- cbcArg1 idx         pure-          ( if op == opLitUri && symbolText (Symbol sym) == "https://example.com"+          ( if op == OpLitUri && symbolText (Symbol sym) == "https://example.com"               then Pass               else Fail "uri mismatch"           ),@@ -4046,7 +4500,7 @@         op <- cbcOpcode idx         sym <- cbcArg1 idx         pure-          ( if op == opLitPath && symbolText (Symbol sym) == "/nix/store/foo"+          ( if op == OpLitPath && symbolText (Symbol sym) == "/nix/store/foo"               then Pass               else Fail "path mismatch"           ),@@ -4061,7 +4515,7 @@         op <- cbcOpcode idx         count <- cbcShortArg idx         pure-          ( if op == opAttrs && count == 1+          ( if op == OpAttrs && count == 1               then Pass               else Fail "inherit binding mismatch"           ),@@ -4076,7 +4530,7 @@         op <- cbcOpcode idx         fl <- cbcFlags idx         pure-          ( if op == opLambda && fl == formalSet+          ( if op == OpLambda && fl == formalSet               then Pass               else Fail ("flags=" <> T.pack (show fl))           ),@@ -4091,7 +4545,7 @@         op <- cbcOpcode idx         fl <- cbcFlags idx         pure-          ( if op == opLambda && fl == formalNamedSet+          ( if op == OpLambda && fl == formalNamedSet               then Pass               else Fail ("flags=" <> T.pack (show fl))           ),@@ -4112,7 +4566,7 @@         capL1 <- cbcData (capOff + 4)         capI1 <- cbcData (capOff + 5)         pure-          ( if op == opLambda+          ( if op == OpLambda               && capTag == captureSlots               && capCount == 2               && capL0 == 1@@ -4144,7 +4598,7 @@         op <- cbcOpcode idx         sym <- cbcArg1 idx         pure-          ( if op == opWithVar && symbolText (Symbol sym) == "dynamic"+          ( if op == OpWithVar && symbolText (Symbol sym) == "dynamic"               then Pass               else Fail "withvar mismatch"           ),@@ -4153,7 +4607,7 @@         op <- cbcOpcode idx         count <- cbcShortArg idx         pure-          ( if op == opIndStr && count == 1+          ( if op == OpIndStr && count == 1               then Pass               else Fail "indstr mismatch"           )@@ -4174,6 +4628,11 @@         [ testExprTypes,           testStorePaths,           testDerivation,+          testTrivialBuildIO,+          testUnpackBuildIO,+          testDependentBuildIO,+          testEvalFidelity,+          testHashHelpers,           testEvalLiterals,           testEvalVariables,           testEvalArithmetic,