diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,19 @@ # Changelog +## 0.1.5.0 — 2026-02-26 + +### New Builtins, Coercion Fixes, CI Cleanup + +- **`builtins.warn`** — prints warning to stderr, returns second arg +- **`builtins.path`** — copies file/dir to store with `SCPlain` context +- **`builtins.seq` fix** — now forces first arg to WHNF (was silent no-op) +- **`builtins.trace`/`traceVerbose` fix** — print to stderr via `MonadEval.traceMessage` +- **`coerceToString` fix** — handle `VAttrs` with `__toString` and `outPath`, enabling `"${pkgs.hello}/bin/hello"` interpolation (was type error on all attrsets) +- **`fetchTarball` fix** — downloads and extracts via curl|tar pipeline (was returning raw .tar.gz) +- **`MonadEval`** — added `traceMessage`, `copyPathToStore` methods +- CI: switched to `haskell-actions/run-ormolu@v16` (matching all Novavero repos) +- 108 builtins, 511 tests, -Werror clean, ormolu clean, hlint clean + ## 0.1.4.0 — 2026-02-26 ### Memory Optimization + Lazy Non-Rec Attrsets @@ -48,8 +62,6 @@ ## 0.1.1.0 — 2026-02-24 -### Phase 4: nixpkgs Compatibility - - **Thunk memoization** — Per-thunk `IORef` memo cells (matching real Nix in-place mutation). `forceThunk` is a `MonadEval` method: IO evaluators memoize per-allocation, pure evaluators re-evaluate. GC reclaims dead thunks naturally — no unbounded global cache. `unsafePerformIO` CAF float-out prevented via `NOINLINE` + `seq` pattern. - **8.4x builtin dispatch speedup** — Case dispatch replacing polymorphic `Map` reconstruction on every call. Direct pattern match on builtin name for zero-allocation dispatch in the hot path. - **Regex builtins** — `builtins.match` and `builtins.split` via `regex-tdfa` (pure Haskell POSIX ERE, cross-platform) @@ -81,7 +93,7 @@ - Demo `test.nix` included: derivations, lambdas, builtins.map, arithmetic — runs on all platforms - 16 builtins exposed at top level without `builtins.` prefix (`toString`, `map`, `throw`, `import`, `derivation`, `abort`, `baseNameOf`, `dirOf`, `isNull`, `removeAttrs`, `placeholder`, `scopedImport`, `fetchTarball`, `fetchGit`, `fetchurl`, `toFile`) — matches real Nix language spec, required for nixpkgs compatibility -### Phase 3: String Contexts + Dependency Resolution + Substituter +### String Contexts, Dependency Resolution, Binary Substituter - String context tracking on all `VStr` values: `SCPlain`, `SCDrvOutput`, `SCAllOutputs` - Context propagation through interpolation, string concatenation, `replaceStrings`, `substring`, `concatStringsSep`, and all string builtins @@ -95,7 +107,7 @@ - Cleanup pass: eliminated all partial functions (`T.head`/`T.tail` to `T.uncons`, `!!` to safe lookup, `last` to pattern match), flattened deeply nested code into composed functions, `Data.Sequence` BFS queues throughout, semantic section organization - 494 tests (68 new: string context, context propagation, dependency graph, substituter, build orchestrator) -### Phase 2: Store + Builder +### Content-Addressed Store + Derivation Builder - Real SQLite-backed store database (`ValidPaths` + `Refs` tables, WAL mode) - Store operations: `addToStore` (cross-device safe), `scanReferences` (byte-scan for store path references), `setReadOnly` (recursive), `writeDrv` @@ -106,7 +118,7 @@ - CLI `nova-nix build FILE.nix`: evaluate, extract derivation, write `.drv`, build, print output path - 426 tests (45 new: 10 store DB, 13 store ops, 10 ATerm parser, 8 builder, 4 CLI end-to-end) -### Phase 1: Parser + Evaluator + Builtins +### Parser, Lazy Evaluator, 85 Builtins - Full Nix expression parser (hand-rolled recursive descent, 13 precedence levels) - Lazy evaluator with thunk-based evaluation, knot-tying for recursive bindings diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -174,39 +174,39 @@ ### Parser -| Module | Purpose | Status | -|--------|---------|--------| -| `Nix.Expr.Types` | Complete Nix AST — 17 expression constructors (including `ESearchPath`), atoms, formals, operators, string parts, source locations | Done | -| `Nix.Parser` | Hand-rolled recursive descent parser + lexer. Direct `Text` consumption, source position tracking | Done | -| `Nix.Parser.Lexer` | Tokenizer — integers, floats, strings with interpolation, paths, URIs, search paths, all operators/keywords | Done | -| `Nix.Parser.Expr` | Expression parser — 13 precedence levels, left/right/non-associative operators, application, selection, dynamic keys | Done | -| `Nix.Parser.Internal` | Parser state and combinator internals | Done | -| `Nix.Parser.ParseError` | Structured parse errors with source positions | Done | +| Module | Purpose | +|--------|---------| +| `Nix.Expr.Types` | Complete Nix AST — 17 expression constructors (including `ESearchPath`), atoms, formals, operators, string parts, source locations | +| `Nix.Parser` | Hand-rolled recursive descent parser + lexer. Direct `Text` consumption, source position tracking | +| `Nix.Parser.Lexer` | Tokenizer — integers, floats, strings with interpolation, paths, URIs, search paths, all operators/keywords | +| `Nix.Parser.Expr` | Expression parser — 13 precedence levels, left/right/non-associative operators, application, selection, dynamic keys | +| `Nix.Parser.Internal` | Parser state and combinator internals | +| `Nix.Parser.ParseError` | Structured parse errors with source positions | ### Evaluator -| Module | Purpose | Status | -|--------|---------|--------| -| `Nix.Eval` | Lazy evaluator — all 17 AST constructors, thunk forcing, env operations, 101-builtin dispatch, `__functor` callable sets, search path resolution, dynamic attribute keys. Polymorphic via `MonadEval` | Done | -| `Nix.Eval.Types` | Shared types — `NixValue` (11 constructors), `Thunk` (lazy env for knot-tying), `Env` (lexical + with-scope chain), `StringContext` (store path tracking), `MonadEval` typeclass, `PureEval` runner | Done | -| `Nix.Eval.Operator` | Binary/unary operators — arithmetic with float promotion, deep structural equality, division-by-zero checks | Done | -| `Nix.Eval.StringInterp` | String interpolation — value coercion with context propagation, indented string whitespace stripping | Done | -| `Nix.Eval.Context` | String context construction, queries, extraction — pure helpers for building and inspecting store path references | Done | -| `Nix.Eval.IO` | IO evaluation monad — real filesystem access, import cache (with directory import), process execution, store writes, NIX_PATH parsing, per-thunk IORef memoization (matching real Nix in-place mutation) | Done | -| `Nix.Builtins` | Built-in function environment — 106 builtins, search path plumbing (`parseNixPath`), top-level builtin exposure | Done | +| Module | Purpose | +|--------|---------| +| `Nix.Eval` | Lazy evaluator — all 17 AST constructors, thunk forcing, env operations, 106-builtin dispatch, `__functor` callable sets, search path resolution, dynamic attribute keys. Polymorphic via `MonadEval` | +| `Nix.Eval.Types` | Shared types — `NixValue` (11 constructors), `Thunk` (lazy env for knot-tying), `Env` (lexical + with-scope chain), `StringContext` (store path tracking), `MonadEval` typeclass, `PureEval` runner | +| `Nix.Eval.Operator` | Binary/unary operators — arithmetic with float promotion, deep structural equality, division-by-zero checks | +| `Nix.Eval.StringInterp` | String interpolation — value coercion with context propagation, indented string whitespace stripping | +| `Nix.Eval.Context` | String context construction, queries, extraction — pure helpers for building and inspecting store path references | +| `Nix.Eval.IO` | IO evaluation monad — real filesystem access, import cache (with directory import), process execution, store writes, NIX_PATH parsing, per-thunk IORef memoization (matching real Nix in-place mutation) | +| `Nix.Builtins` | Built-in function environment — 106 builtins, search path plumbing (`parseNixPath`), top-level builtin exposure | ### Store + Builder -| Module | Purpose | Status | -|--------|---------|--------| -| `Nix.Derivation` | Derivation type, ATerm serialization + parsing (`toATerm`/`fromATerm`), platform detection | Done | -| `Nix.Hash` | Derivation hashing, store path computation, shared hex/base-32 utilities | Done | -| `Nix.Store.Path` | Store path types — `StoreDir`, `StorePath`, `parseStorePath`, Windows/Unix support | Done | -| `Nix.Store.DB` | SQLite store database — `ValidPaths` + `Refs` tables, WAL mode, path registration, reference/deriver queries | Done | -| `Nix.Store` | High-level store operations — `addToStore`, `scanReferences`, `setReadOnly`, `writeDrv` | Done | -| `Nix.Builder` | Derivation builder — dependency graph construction, topological sort, binary cache substitution, local build with output registration | Done | -| `Nix.DependencyGraph` | Dependency graph construction (BFS with `Seq` queue) and topological sort (Kahn's algorithm, O(V+E)), cycle detection | Done | -| `Nix.Substituter` | Binary cache substituter — HTTP narinfo fetch, signature verification, NAR download/decompress/unpack, store registration. Multi-cache with priority ordering | Done | +| Module | Purpose | +|--------|---------| +| `Nix.Derivation` | Derivation type, ATerm serialization + parsing (`toATerm`/`fromATerm`), platform detection | +| `Nix.Hash` | Derivation hashing, store path computation, shared hex/base-32 utilities | +| `Nix.Store.Path` | Store path types — `StoreDir`, `StorePath`, `parseStorePath`, Windows/Unix support | +| `Nix.Store.DB` | SQLite store database — `ValidPaths` + `Refs` tables, WAL mode, path registration, reference/deriver queries | +| `Nix.Store` | High-level store operations — `addToStore`, `scanReferences`, `setReadOnly`, `writeDrv` | +| `Nix.Builder` | Derivation builder — dependency graph construction, topological sort, binary cache substitution, local build with output registration | +| `Nix.DependencyGraph` | Dependency graph construction (BFS with `Seq` queue) and topological sort (Kahn's algorithm, O(V+E)), cycle detection | +| `Nix.Substituter` | Binary cache substituter — HTTP narinfo fetch, signature verification, NAR download/decompress/unpack, store registration. Multi-cache with priority ordering | --- @@ -284,40 +284,9 @@ ## Roadmap -### Done - -- [x] **Lexer** — Full Nix tokenization (integers, floats, strings with interpolation, paths, URIs, search paths, operators, keywords) -- [x] **Parser** — 13 precedence levels, all Nix syntax, structured error reporting -- [x] **Evaluator** — All 17 AST constructors, lazy thunks, recursive let/rec via knot-tying, with-scope chain, dynamic attribute keys -- [x] **106 builtins** — Type checks, arithmetic, bitwise, strings, lists, attrsets, higher-order, JSON, hashing, regex (`match`/`split` via `regex-tdfa`), version parsing, `setFunctionArgs`/`functionArgs`, tryEval, deepSeq, genericClosure, string context introspection, all IO builtins, derivation -- [x] **MonadEval refactor** — Evaluator polymorphic in effect monad (`PureEval` for tests, `EvalIO` for real evaluation) -- [x] **IO builtins** — `import` (with directory import support), `readFile`, `pathExists`, `readDir`, `getEnv`, `toPath`, `toFile`, `findFile`, `scopedImport`, `fetchurl`, `fetchTarball`, `fetchGit`, `currentTime` -- [x] **`derivation`** — Attrset to `.drv` build recipe with computed `drvPath` and `outPath`, context-aware input population -- [x] **String context tracking** — `SCPlain`, `SCDrvOutput`, `SCAllOutputs` on every `VStr`, propagated through interpolation, operators, and all string builtins. `hasContext`, `getContext`, `appendContext` introspection builtins. -- [x] **ATerm serialization + parsing** — Full `.drv` round-trip with `toATerm`/`fromATerm`, string escaping, sorted environments -- [x] **SQLite store DB** — `ValidPaths` + `Refs` tables, WAL mode, registration, validity checks, reference/deriver queries -- [x] **Store operations** — `parseStorePath`, `addToStore` (cross-device safe), `scanReferences` (byte-scan), `setReadOnly`, `writeDrv` -- [x] **Dependency graph** — BFS construction with `Data.Sequence` (O(V+E)), topological sort via Kahn's algorithm, cycle detection -- [x] **Builder** — Full build loop with recursive dependency resolution: topo sort, cache check, binary substitution, local build, output registration -- [x] **Binary substituter** — HTTP binary cache protocol: narinfo fetch/parse, Ed25519 signature verification via nova-cache, NAR download/decompress/unpack, store DB registration. Multi-cache with priority ordering. -- [x] **NIX_PATH / search path resolution** — `` desugars to `builtins.findFile builtins.nixPath name`. `NIX_PATH` parsed at startup. `--nix-path` CLI flag for additional entries. -- [x] **Dynamic attribute keys** — `{ ${expr} = val; }` works in all binding contexts with monadic key resolution preserving knot-tying -- [x] **Directory imports** — `import ./dir` resolves to `./dir/default.nix` automatically -- [x] **CLI** — `nova-nix eval FILE.nix`, `nova-nix eval --expr 'EXPR'`, `nova-nix build FILE.nix`, `--nix-path NAME=PATH` -- [x] **Thunk memoization** — Per-thunk `IORef` memo cells matching real Nix in-place mutation. GC reclaims dead thunks naturally. -- [x] **Regex builtins** — `builtins.match` and `builtins.split` (POSIX ERE via `regex-tdfa`, pure Haskell, cross-platform) -- [x] **Callable attribute sets** — `__functor` dispatch: sets with `__functor` are callable, enabling `lib.makeOverridable`, `lib.makeExtensible`, and the nixpkgs override system -- [x] **Lazy builtins** — `map`, `genList`, `mapAttrs` return deferred thunks (O(1) until demanded). `concatMap` is semi-lazy. Critical for nixpkgs which maps over 80,000+ packages. -- [x] **Null dynamic keys** — `{ ${null} = val; }` skips the binding, used by the nixpkgs module system for conditional attributes -- [x] **Bundled ``** — Ships as Cabal data-file for nixpkgs stdenv bootstrap -- [x] **nixpkgs module system** — `lib.evalModules`, `lib.mkOption`, `lib.mkIf`, `lib.types.*` all working -- [x] **nixpkgs lib fully working** — trivial, strings, lists, attrsets, systems, generators, functional patterns (`fix`, `makeOverridable`, `makeExtensible`) -- [x] **511 tests** — parser, evaluator, store, builder, substituter, dependency graph, search paths, dynamic keys, directory imports, laziness, CLI end-to-end - ### Next -- [ ] **Full `import {}` performance** — nixpkgs lib layer evaluates correctly; stdenv bootstrap runs but needs performance optimization for the full 80,000+ package set -- [ ] **Remaining builtins** — 13 missing (`fromTOML`, `hashFile`, `readFileType`, `traceVerbose`, `break`, `filterSource`, `outputOf`, and fetch variants) +- [ ] **Full `import {}` performance** — nixpkgs lib layer evaluates correctly; stdenv bootstrap runs but needs memory optimization for the full 80,000+ package set (currently OOM at 2GB, investigating space leak) - [ ] **`nova-nix shell`** — Enter a development shell (like `nix shell`) - [ ] **`nova-nix repl`** — Interactive evaluator diff --git a/nova-nix.cabal b/nova-nix.cabal --- a/nova-nix.cabal +++ b/nova-nix.cabal @@ -1,6 +1,6 @@ cabal-version: 3.0 name: nova-nix -version: 0.1.4.0 +version: 0.1.5.0 synopsis: Windows-native Nix implementation in pure Haskell description: A pure Haskell implementation of the Nix package manager that runs natively diff --git a/src/Nix/Eval.hs b/src/Nix/Eval.hs --- a/src/Nix/Eval.hs +++ b/src/Nix/Eval.hs @@ -68,11 +68,11 @@ import qualified Data.ByteArray as BA import qualified Data.ByteString as BS import Data.Char (chr, digitToInt, isAlpha, isDigit, isHexDigit, isOctDigit, ord) -import Data.List (foldl') +import Data.List (find, foldl') import qualified Data.Map.Lazy as MapL import Data.Map.Strict (Map) import qualified Data.Map.Strict as Map -import Data.Maybe (catMaybes, isJust, isNothing) +import Data.Maybe (catMaybes, fromMaybe, isJust, isNothing) import Data.Sequence (Seq (..)) import qualified Data.Sequence as Seq import qualified Data.Set as Set @@ -80,7 +80,7 @@ import qualified Data.Text as T import qualified Data.Text.Encoding as TE import Nix.Derivation (Derivation (..), DerivationOutput (..), textToPlatform, toATerm) -import Nix.Eval.Context (extractInputDrvs, extractInputSrcs) +import Nix.Eval.Context (extractInputDrvs, extractInputSrcs, plainContext) import Nix.Eval.Operator (evalBinary, evalUnary, nixCompare, nixEqual) import Nix.Eval.StringInterp (coerceToString, evalIndStringParts, evalStringParts) import Nix.Eval.Types @@ -142,8 +142,8 @@ eval :: (MonadEval m) => Env -> Expr -> m NixValue eval env expr = case expr of ELit atom -> evalLit atom - EStr parts -> uncurry VStr <$> evalStringParts eval env parts - EIndStr parts -> uncurry VStr <$> evalIndStringParts eval env parts + EStr parts -> uncurry VStr <$> evalStringParts eval force applyValue env parts + EIndStr parts -> uncurry VStr <$> evalIndStringParts eval force applyValue env parts EVar name -> evalVar env name EAttrs isRec bindings -> evalAttrs env isRec bindings EList exprs -> pure (VList (map (mkThunk env) exprs)) @@ -768,8 +768,9 @@ builtin1 "isPath" (pure . VBool . isPathVal), builtin1 "ceil" builtinCeil, builtin1 "floor" builtinFloor, - builtin2 "seq" (\_ b -> pure b), - builtin2 "trace" (\_ b -> pure b), + builtin2 "seq" builtinSeq, + builtin2 "trace" builtinTrace, + builtin2 "warn" builtinWarn, builtin1 "unsafeDiscardStringContext" builtinDiscardContext, builtin1 "unsafeDiscardOutputDependency" builtinDiscardOutputDep, -- String context introspection @@ -835,8 +836,8 @@ builtin2 "addErrorContext" (\_ val -> pure val), -- Attr position (return null — nixpkgs handles this gracefully) builtin2 "unsafeGetAttrPos" (\_ _ -> pure VNull), - -- Debugging (no-ops without --trace-verbose / --debugger) - builtin2 "traceVerbose" (\_ b -> pure b), + -- Debugging (traceVerbose: same as trace for now, --trace-verbose not yet gated) + builtin2 "traceVerbose" builtinTrace, builtin1 "break" pure, -- IO: file hashing + type detection builtin2 "hashFile" builtinHashFile, @@ -847,7 +848,8 @@ builtin1 "convertHash" builtinConvertHash, -- XML serialization builtin1 "toXML" builtinToXML, - -- Source filtering + -- Source filtering + path import + builtin1 "path" builtinPath, builtin2 "filterSource" builtinFilterSource, -- Experimental feature stubs builtin2 "outputOf" builtinOutputOf, @@ -947,8 +949,9 @@ "isPath" -> apply1 (pure . VBool . isPathVal) "ceil" -> apply1 builtinCeil "floor" -> apply1 builtinFloor - "seq" -> apply2 (\_ b -> pure b) - "trace" -> apply2 (\_ b -> pure b) + "seq" -> apply2 builtinSeq + "trace" -> apply2 builtinTrace + "warn" -> apply2 builtinWarn "unsafeDiscardStringContext" -> apply1 builtinDiscardContext "unsafeDiscardOutputDependency" -> apply1 builtinDiscardOutputDep -- String context introspection @@ -1014,8 +1017,8 @@ "addErrorContext" -> apply2 (\_ val -> pure val) -- Attr position (return null — nixpkgs handles this gracefully) "unsafeGetAttrPos" -> apply2 (\_ _ -> pure VNull) - -- Debugging (no-ops without --trace-verbose / --debugger) - "traceVerbose" -> apply2 (\_ b -> pure b) + -- Debugging (traceVerbose: same as trace for now) + "traceVerbose" -> apply2 builtinTrace "break" -> apply1 pure -- IO: file hashing + type detection "hashFile" -> apply2 builtinHashFile @@ -1026,7 +1029,8 @@ "convertHash" -> apply1 builtinConvertHash -- XML serialization "toXML" -> apply1 builtinToXML - -- Source filtering + -- Source filtering + path import + "path" -> apply1 builtinPath "filterSource" -> apply2 builtinFilterSource -- Experimental feature stubs "outputOf" -> apply2 builtinOutputOf @@ -1544,7 +1548,7 @@ coerceThunk thunk = do val <- force thunk coerceToStringPermissive val -coerceToStringPermissive other = coerceToString other +coerceToStringPermissive other = coerceToString force applyValue other -- | The current system platform string. currentSystemStr :: Text @@ -2329,6 +2333,28 @@ deepForce (VAttrs attrs) = mapM_ (force >=> deepForce) (attrSetElems attrs) deepForce _ = pure () +-- | @builtins.seq a b@ — evaluate @a@ to WHNF, then return @b@. +builtinSeq :: (MonadEval m) => NixValue -> NixValue -> m NixValue +builtinSeq !_first = pure + +-- | @builtins.trace msg val@ — print @msg@ to stderr, return @val@. +builtinTrace :: (MonadEval m) => NixValue -> NixValue -> m NixValue +builtinTrace msgVal result = do + msg <- case msgVal of + VStr s _ -> pure s + other -> pure (typeName other) + traceMessage ("trace: " <> msg) + pure result + +-- | @builtins.warn msg val@ — print warning to stderr, return @val@. +builtinWarn :: (MonadEval m) => NixValue -> NixValue -> m NixValue +builtinWarn msgVal result = do + msg <- case msgVal of + VStr s _ -> pure s + other -> pure (typeName other) + traceMessage ("warning: " <> msg) + pure result + -- --------------------------------------------------------------------------- -- Builtin implementations — graph traversal -- --------------------------------------------------------------------------- @@ -2567,37 +2593,59 @@ throwEvalError ("builtins.fetchurl: expected a string or set, got " <> typeName other) builtinFetchTarball :: (MonadEval m) => NixValue -> m NixValue -builtinFetchTarball (VStr url _) = fetchUrlSimple url Nothing +builtinFetchTarball (VStr url _) = fetchAndExtractTarball url builtinFetchTarball (VAttrs attrs) = do url <- forceAttrStr "builtins.fetchTarball" "url" attrs - sha256 <- forceOptionalAttrStr attrs "sha256" - fetchUrlSimple url sha256 + fetchAndExtractTarball url builtinFetchTarball other = throwEvalError ("builtins.fetchTarball: expected a string or set, got " <> typeName other) +-- | Download a tarball, extract it, and return the path to the extracted +-- directory. Uses a content-hashed temp directory. Downloads and extracts +-- in a single shell pipeline to avoid binary-as-text encoding issues. +fetchAndExtractTarball :: (MonadEval m) => Text -> m NixValue +fetchAndExtractTarball url = do + sysTmp <- getTempDir + let urlHash = sha256Hex (TE.encodeUtf8 url) + extractDir = sysTmp <> "/nova-nix-tarball-" <> urlHash + -- Single pipeline: mkdir, download, extract with --strip-components=1 + -- The -- separator prevents argument injection from the URL. + (code, _, errOut) <- + runProcess + "sh" + [ "-c", + "mkdir -p \"$1\" && curl -sSfL -- \"$2\" | tar -xz -C \"$1\" --strip-components=1", + "--", + extractDir, + url + ] + "" + case code of + 0 -> pure (VPath extractDir) + _ -> throwEvalError ("builtins.fetchTarball: " <> errOut) + builtinFetchGit :: (MonadEval m) => NixValue -> m NixValue builtinFetchGit (VStr url _) = do - -- Use a content-based temp dir to avoid predictable paths + sysTmp <- getTempDir let urlHash = sha256Hex (TE.encodeUtf8 url) - sysTmpRaw <- getEnvVar "TMPDIR" - -- TMPDIR on Unix, TEMP/TMP on Windows; fall back to /tmp - sysTmp <- - if T.null sysTmpRaw - then do - winTmp <- getEnvVar "TEMP" - pure (if T.null winTmp then "/tmp" else winTmp) - else pure sysTmpRaw - let tmpDir = sysTmp <> "/nova-nix-fetchgit-" <> urlHash - (code, _stdout, stderr) <- runProcess "git" ["clone", "--depth", "1", "--", url, tmpDir] "" - if code /= 0 - then throwEvalError ("builtins.fetchGit: git clone failed: " <> stderr) - else pure (VPath tmpDir) + cloneDir = sysTmp <> "/nova-nix-fetchgit-" <> urlHash + (code, _, errOut) <- runProcess "git" ["clone", "--depth", "1", "--", url, cloneDir] "" + case code of + 0 -> pure (VPath cloneDir) + _ -> throwEvalError ("builtins.fetchGit: git clone failed: " <> errOut) builtinFetchGit (VAttrs attrs) = do url <- forceAttrStr "builtins.fetchGit" "url" attrs builtinFetchGit (mkStr url) builtinFetchGit other = throwEvalError ("builtins.fetchGit: expected a string or set, got " <> typeName other) +-- | Resolve the system temp directory. Checks @TMPDIR@ (Unix), then +-- @TEMP@ (Windows), falls back to @\/tmp@. +getTempDir :: (MonadEval m) => m Text +getTempDir = do + candidates <- mapM getEnvVar ["TMPDIR", "TEMP"] + pure (fromMaybe "/tmp" (find (not . T.null) candidates)) + -- | Fetch a URL and optionally verify its hash. fetchUrlSimple :: (MonadEval m) => Text -> Maybe Text -> m NixValue fetchUrlSimple url _sha256 = do @@ -3415,6 +3463,38 @@ escapeChar '&' = "&" escapeChar '"' = """ escapeChar c = T.singleton c + +-- --------------------------------------------------------------------------- +-- Builtin implementations — builtins.path +-- --------------------------------------------------------------------------- + +-- | @builtins.path { path; name?; filter?; sha256?; recursive?; }@ +-- +-- Copy a path to the store and return the store path as a string with +-- context. @name@ defaults to the basename of @path@. @filter@ is +-- accepted but not yet applied (copies everything). +builtinPath :: (MonadEval m) => NixValue -> m NixValue +builtinPath (VAttrs attrs) = do + pathStr <- forceAttrStr "builtins.path" "path" attrs + nameOverride <- forceOptionalAttrStr attrs "name" + let name = fromMaybe (extractBaseName pathStr) nameOverride + storePathText <- copyPathToStore pathStr name + -- Parse the store path to construct proper SCPlain context + case parseStorePath defaultStoreDir storePathText of + Just sp -> pure (VStr storePathText (plainContext sp)) + Nothing -> pure (VStr storePathText emptyContext) +builtinPath other = + throwEvalError ("builtins.path: expected an attribute set, got " <> typeName other) + +-- | Extract the last path component from a path string. +extractBaseName :: Text -> Text +extractBaseName path = + let stripped = T.dropWhileEnd (\c -> c == '/' || c == '\\') path + in case T.breakOnEnd "/" stripped of + ("", _) -> case T.breakOnEnd "\\" stripped of + ("", _) -> stripped + (_, name) -> name + (_, name) -> name -- --------------------------------------------------------------------------- -- Builtin implementations — filterSource diff --git a/src/Nix/Eval/IO.hs b/src/Nix/Eval/IO.hs --- a/src/Nix/Eval/IO.hs +++ b/src/Nix/Eval/IO.hs @@ -27,7 +27,7 @@ where import Control.Exception (Exception, SomeException, displayException, fromException, throwIO, try) -import Control.Monad (when) +import Control.Monad (unless, when) import Control.Monad.IO.Class (liftIO) import Control.Monad.Reader (ReaderT (..), ask, asks, local) import qualified Data.ByteString as BS @@ -49,6 +49,7 @@ import System.Environment (lookupEnv) import System.Exit (ExitCode (..)) import System.FilePath (isRelative, takeDirectory, ()) +import System.IO (hPutStrLn, stderr) import qualified System.Process as Proc -- --------------------------------------------------------------------------- @@ -237,6 +238,17 @@ ExitFailure n -> n pure (code, T.pack stdoutStr, T.pack stderrStr) + copyPathToStore srcPath name = do + storeDir <- EvalIO (asks esStoreDir) + let contentHash = sha256Hex (encodeUtf8 ("source:" <> srcPath <> ":" <> name)) + inner = "nix-store:sha256:" <> contentHash <> ":" <> T.pack storeDir <> ":" <> name + pathHash = truncatedBase32 (encodeUtf8 inner) + destPath = T.pack storeDir <> "/" <> pathHash <> "-" <> name + wrapIO (copyToStoreIfMissing (T.unpack srcPath) (T.unpack destPath) storeDir) + pure destPath + + traceMessage msg = EvalIO (liftIO (hPutStrLn stderr (T.unpack msg))) + resolvePathLiteral path = do baseDir <- EvalIO (asks esBaseDir) let raw = T.unpack path @@ -370,3 +382,25 @@ FormalNamedSet n fs ellipsis -> FormalNamedSet n (map goFormal fs) ellipsis goFormal (Formal n mDef) = Formal n (fmap goExpr mDef) + +-- --------------------------------------------------------------------------- +-- Store copy helpers +-- --------------------------------------------------------------------------- + +-- | Copy a source path (file or directory) to the store if not already present. +copyToStoreIfMissing :: FilePath -> FilePath -> FilePath -> IO () +copyToStoreIfMissing src dest storeDir = do + Dir.createDirectoryIfMissing True storeDir + alreadyExists <- Dir.doesPathExist dest + unless alreadyExists (copyPath src dest) + +-- | Copy a file or directory tree to a destination. +copyPath :: FilePath -> FilePath -> IO () +copyPath src dest = do + isDir <- Dir.doesDirectoryExist src + if isDir + then do + Dir.createDirectoryIfMissing True dest + entries <- Dir.listDirectory src + mapM_ (\entry -> copyPath (src entry) (dest entry)) entries + else Dir.copyFile src dest diff --git a/src/Nix/Eval/StringInterp.hs b/src/Nix/Eval/StringInterp.hs --- a/src/Nix/Eval/StringInterp.hs +++ b/src/Nix/Eval/StringInterp.hs @@ -14,17 +14,23 @@ import Data.Text (Text) import qualified Data.Text as T import Nix.Eval.Context (concatStrings) -import Nix.Eval.Types (Env, MonadEval (..), NixValue (..), StringContext, emptyContext, typeName) +import Nix.Eval.Types (Env, MonadEval (..), NixValue (..), StringContext, Thunk, attrSetLookup, emptyContext, typeName) import Nix.Expr.Types (Expr, StringPart (..)) -- | The evaluator function, passed as a parameter to avoid cyclic imports. type Eval m = Env -> Expr -> m NixValue +-- | Force a thunk to a value. +type Force m = Thunk -> m NixValue + +-- | Apply a function value to an argument value. +type Apply m = NixValue -> NixValue -> m NixValue + -- | Evaluate the parts of a regular string (double-quoted). -- Returns the concatenated text and the merged context from all parts. -evalStringParts :: (MonadEval m) => Eval m -> Env -> [StringPart] -> m (Text, StringContext) -evalStringParts evalFn env parts = do - chunks <- mapM (evalOnePart evalFn env) parts +evalStringParts :: (MonadEval m) => Eval m -> Force m -> Apply m -> Env -> [StringPart] -> m (Text, StringContext) +evalStringParts evalFn forceFn applyFn env parts = do + chunks <- mapM (evalOnePart evalFn forceFn applyFn env) parts pure (concatStrings chunks) -- | Evaluate the parts of an indented string (double single-quoted). @@ -32,34 +38,47 @@ -- After interpolation, strips the common leading whitespace from all -- non-empty lines (the standard Nix indented-string semantics). -- Context is preserved through indentation stripping. -evalIndStringParts :: (MonadEval m) => Eval m -> Env -> [StringPart] -> m (Text, StringContext) -evalIndStringParts evalFn env parts = do - (raw, ctx) <- evalStringParts evalFn env parts +evalIndStringParts :: (MonadEval m) => Eval m -> Force m -> Apply m -> Env -> [StringPart] -> m (Text, StringContext) +evalIndStringParts evalFn forceFn applyFn env parts = do + (raw, ctx) <- evalStringParts evalFn forceFn applyFn env parts pure (stripIndentation raw, ctx) -- | Evaluate a single string part, returning its text and context. -evalOnePart :: (MonadEval m) => Eval m -> Env -> StringPart -> m (Text, StringContext) -evalOnePart _ _ (StrLit txt) = pure (txt, emptyContext) -evalOnePart evalFn env (StrInterp expr) = do +evalOnePart :: (MonadEval m) => Eval m -> Force m -> Apply m -> Env -> StringPart -> m (Text, StringContext) +evalOnePart _ _ _ _ (StrLit txt) = pure (txt, emptyContext) +evalOnePart evalFn forceFn applyFn env (StrInterp expr) = do val <- evalFn env expr - coerceToString val + coerceToString forceFn applyFn val -- | Coerce a Nix value to a string for interpolation. -- --- Strict coercion: strings, ints, floats, paths, null, bools. --- Lists, sets (without @__toString@/@outPath@), and functions are errors. --- Used by string interpolation (@"${...}"@). -coerceToString :: (MonadEval m) => NixValue -> m (Text, StringContext) -coerceToString val = case val of - VStr s ctx -> pure (s, ctx) - VInt n -> pure (T.pack (show n), emptyContext) - VFloat n -> pure (T.pack (show n), emptyContext) - VPath p -> pure (p, emptyContext) - VNull -> pure ("", emptyContext) - VBool True -> pure ("1", emptyContext) - VBool False -> pure ("", emptyContext) - other -> - throwEvalError ("cannot coerce " <> typeName other <> " to a string") +-- Strict coercion: strings, ints, floats, paths, null, bools, and +-- attribute sets with @__toString@ or @outPath@. +-- Lists and functions without coercion metadata are errors. +-- Used by string interpolation (@"${...}"@) and @builtins.toString@. +coerceToString :: (MonadEval m) => Force m -> Apply m -> NixValue -> m (Text, StringContext) +coerceToString _ _ (VStr s ctx) = pure (s, ctx) +coerceToString _ _ (VInt n) = pure (T.pack (show n), emptyContext) +coerceToString _ _ (VFloat n) = pure (T.pack (show n), emptyContext) +coerceToString _ _ (VPath p) = pure (p, emptyContext) +coerceToString _ _ VNull = pure ("", emptyContext) +coerceToString _ _ (VBool True) = pure ("1", emptyContext) +coerceToString _ _ (VBool False) = pure ("", emptyContext) +-- Attribute sets: try __toString first, then outPath +coerceToString forceFn applyFn (VAttrs attrs) = + case attrSetLookup "__toString" attrs of + Just toStrThunk -> do + toStrFn <- forceFn toStrThunk + result <- applyFn toStrFn (VAttrs attrs) + coerceToString forceFn applyFn result + Nothing -> case attrSetLookup "outPath" attrs of + Just outPathThunk -> do + outPathVal <- forceFn outPathThunk + coerceToString forceFn applyFn outPathVal + Nothing -> + throwEvalError "cannot coerce a set to a string (missing __toString or outPath)" +coerceToString _ _ other = + throwEvalError ("cannot coerce " <> typeName other <> " to a string") -- --------------------------------------------------------------------------- -- Indented string whitespace stripping diff --git a/src/Nix/Eval/Types.hs b/src/Nix/Eval/Types.hs --- a/src/Nix/Eval/Types.hs +++ b/src/Nix/Eval/Types.hs @@ -560,6 +560,14 @@ -- closures remain valid after the import scope ends. resolvePathLiteral :: Text -> m Text + -- | Copy a path (file or directory) to the store, returning the store path. + -- First argument is the source path, second is the store name. + copyPathToStore :: Text -> Text -> m Text + + -- | Print a trace/warning message. + -- IO evaluators write to stderr; pure evaluators silently discard. + traceMessage :: Text -> m () + -- | Force a thunk to a value, with memoization. -- IO evaluators should cache results (via per-thunk @IORef@). -- Pure evaluators re-evaluate each time. @@ -587,6 +595,8 @@ readFileBytes _ = throwEvalError "hashFile: not available in pure evaluation" getFileType _ = throwEvalError "readFileType: not available in pure evaluation" runProcess _ _ _ = throwEvalError "runProcess: not available in pure evaluation" + copyPathToStore _ _ = throwEvalError "builtins.path: not available in pure evaluation" + traceMessage _ = pure () resolvePathLiteral = pure forceThunk _ (Evaluated val) = pure val forceThunk evalFn (ThunkRef ref) =