diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,131 @@ --- +## [0.17.5] - 2026-08-19 + +Point release on the 0.17.x line. Two language-surface changes lead: sum-type eliminators unify on +`match` (scrutinee-first), and the primitive set is tentatively finalized — `strings.lines`/`unlines` +are removed in favour of `hydra.formatting` term helpers. A symlink-aware filesystem family lands +across all ten hosts, and Hydra's own build system continues its migration into Hydra. + +**Backward-incompatible.** The `cases` -> `match` rename and the `strings.lines`/`unlines` removal +both change the published kernel surface. Code written against 0.17.4 that uses `Optionals.cases` +or `strings.lines`/`strings.unlines` must be updated. + +### Highlights + +- **Sum-type eliminators unify on `match`** ([#615](https://github.com/CategoricalData/hydra/issues/615)): + the `cases`/`match` helper pair is flipped to a single scrutinee-first `match` convention across the + Haskell, Java, Python, and Scala DSLs, the `optionals.cases` -> `optionals.match` kernel primitive, + and the corresponding implementations in TypeScript and all four Lisp dialects. +- **Primitive set tentatively finalized** ([#417](https://github.com/CategoricalData/hydra/issues/417)): + `strings.lines` and `strings.unlines` are removed as primitives; their semantics are preserved as + `hydra.formatting.lines`/`unlines` term helpers, with overlay implementations dropped across all ten + hosts and callers repointed. +- **Symlink-aware filesystem primitives** ([#666](https://github.com/CategoricalData/hydra/issues/666)): + `status(followLinks)`, `createSymlink`, and `readSymlink` land with native implementations for every + host — Haskell, Java, Python, Scala, TypeScript, Clojure, Common Lisp, Scheme, and Emacs Lisp. +- **Build system promotion continues** ([#416](https://github.com/CategoricalData/hydra/issues/416)): + adds `hydra.build.walk` extension-glob helpers (`extensionOf`/`filterByExtension`/`matchesExtension`) + and `hydra.build.comparereportlogic`, a set of pure snapshot-compare decision helpers. + +### Bug fixes + +- **`math.range` was inclusive of both bounds** ([#647](https://github.com/CategoricalData/hydra/issues/647)), + silently fabricating an extra element in index arithmetic. +- **Java coder TCO-`cases` codegen** dropped enclosing-method type parameters from union-variant casts + (found via [#666](https://github.com/CategoricalData/hydra/issues/666)). +- **`ShaclRdf` promoted to a DSL module** ([#652](https://github.com/CategoricalData/hydra/issues/652)) + so the SHACL pipeline works outside Haskell. +- **Cold-seed shim portability**: the R22 bootstrap shim used GNU-only `\+` and `\b`, which fail under + BSD `sed` on macOS ([#417](https://github.com/CategoricalData/hydra/issues/417)). +- **Bootstrap Haskell target missing `unix`**: the demo's static `package.yaml` was not updated when + `System.Posix.Files` entered the kernel overlay, breaking every `*-to-haskell` bootstrap cell + ([#670](https://github.com/CategoricalData/hydra/issues/670)). +- **Bootstrap comparison scored Scala and TypeScript cells as passing without comparing anything**: + `compare_output`'s target switch was missing both languages, so the baseline path was malformed, + every file counted as "no baseline found", and a zero-file comparison reported `pass`. A Scala or + TypeScript *target* cell could never fail ([#671](https://github.com/CategoricalData/hydra/issues/671)). + +### Known issues + +- A pre-#630 post-generation text pass rewrites `hydra.lib.` -> `hydra.overlay..lib.` + in generated **doc comments** as well as code, because its quote-prefix guard does not recognize + mid-sentence prose references. Affects two kernel doc strings in `src/main` + (`hydra.error.system`, `hydra.error.packaging`) plus ~20 generated docstrings in the test tree; + cosmetic only, no behavioral impact. Tracked as part of + [#633](https://github.com/CategoricalData/hydra/issues/633). +- Bootstrap comparison diff detail is not persisted to the run directory + ([#671](https://github.com/CategoricalData/hydra/issues/671)). + +--- + +## [0.17.4] - 2026-08-09 + +Point release on the 0.17.x line. Fixes an inference bug that mis-typed generated Java and Python for +`readFile`, adds the `divide` primitive, and restores `hydra-ext` to the published package set. Release +validation also surfaced a family of pre-existing cross-host code-generation defects — none of which the +per-host CI jobs can catch, because only the bootstrap exercises one host generating *another*. + +### Highlights + +- **Output-only nominal generalization fixed** ([#637](https://github.com/CategoricalData/hydra/issues/637), + with [#638](https://github.com/CategoricalData/hydra/issues/638) / + [#639](https://github.com/CategoricalData/hydra/issues/639) / + [#640](https://github.com/CategoricalData/hydra/issues/640)): a nominal type appearing only in the + *output* position of a primitive's signature was generalized to a free type variable instead of resolving + as nominal, mis-typing the generated Java and Python for `readFile`. Also widens universe closure and adds + a `kernelTypeUniverse` helper, an effect DSL builder, and `effects`/`files`/`text`/`system` lib wrappers. +- **`divide` primitive** ([#317](https://github.com/CategoricalData/hydra/issues/317)): now present in the + published kernel. This also clears the cold-clone dangle, since a cold seed against published 0.17.4 + resolves `DefMath.divide`. +- **`hydra-ext` republished** ([#636](https://github.com/CategoricalData/hydra/issues/636)): back in the + published set after its absence from 0.17.1–0.17.3, shipping to Hackage and PyPI. + +### Bug fixes + +- **Overlay-lib redirection skipped for the Haskell target** + ([#630](https://github.com/CategoricalData/hydra/issues/630)): + the Java, Python, Scheme, Clojure and Common Lisp heads resolved the overlay lib directory using an + all-lowercase path and did not lower-case the names they scanned. Haskell's overlay tree is capitalized + (`Hydra/Overlay/Haskell/Lib/`), so every `hydra.lib.*` redirect silently failed and the generated Haskell + imported `Hydra.Lib.*` instead of `Hydra.Overlay.Haskell.Lib.*`, producing hundreds of type errors. The + defect was invisible on case-sensitive filesystems, where the failed directory check falls back to a + hard-coded list that happens to be correct. The Python head carried *two* independent copies of this scan; + only the one in `generation.py` feeds the Haskell coder. +- **Lisp-as-host bootstrap** ([#520](https://github.com/CategoricalData/hydra/issues/520), + [#630](https://github.com/CategoricalData/hydra/issues/630)): the four Lisp head decode drivers called + `from_json` with a stale 4-argument signature, and the Lisp bootstrap-driver coders were not pre-bound with + `overlaySubs`. +- **Python-as-host dropped the `Graph` argument** ([#630](https://github.com/CategoricalData/hydra/issues/630)): + `overlaySubs` was not threaded into the Python head's `write_*` wrappers. +- **Clojure `resolve-coder` and `decimal-to-float32`** + ([#630](https://github.com/CategoricalData/hydra/issues/630)): a stray parenthesis closed a `case` early, + making an unsupported-default block run unconditionally; and `(float (.doubleValue ...))` threw on + `Float.MAX_VALUE`, now saturating via `(.floatValue (bigdec x))`. +- **Cold-seeder failed on macOS** ([#630](https://github.com/CategoricalData/hydra/issues/630)): the #376 + cold seeder used a bare `sed -i -e`, which is GNU-only and aborts immediately under BSD `sed`. + +### Release engineering + +- **Review checkpoint before Maven Central publishes** + ([#591](https://github.com/CategoricalData/hydra/issues/591)): `HYDRA_JAVA_PUBLISH_HOLD=1` uploads the + aggregated Java deployment as `USER_MANAGED`, so it validates and then holds for review instead of + publishing automatically. This mirrors `HYDRA_SCALA_PUBLISH_HOLD` and makes an upload reversible — a held + deployment can be dropped, whereas an automatic one cannot be undone. + +### Known issues + +- **`hydra-ext` is not published to Maven Central** + ([#643](https://github.com/CategoricalData/hydra/issues/643)): it does not compile for the Java target, + because the visitor-pattern inner interface the Java coder emits collides with the enclosing `Visitor` + type in `hydra.cpp.syntax`. Only Java is affected; the Python coder emits sibling classes rather than + nested ones. ext ships to Hackage and PyPI at 0.17.4 and is expected to rejoin the Java set once fixed. +- **TypeScript no longer self-hosts** + ([#642](https://github.com/CategoricalData/hydra/issues/642)): the TypeScript host's JSON decoder fails to + decode most of the kernel, so `typescript-as-host` code generation is broken. No published artifact is + affected — `dist/typescript` is generated by the Haskell host and passes its test suite. + ## [0.17.3] - 2026-08-01 Point release on the 0.17.x line, focused on **release-artifact integrity**. The 0.17.2 release shipped diff --git a/hydra-lisp.cabal b/hydra-lisp.cabal --- a/hydra-lisp.cabal +++ b/hydra-lisp.cabal @@ -1,11 +1,11 @@ cabal-version: 1.12 --- This file has been generated from package.yaml by hpack version 0.39.1. +-- This file has been generated from package.yaml by hpack version 0.38.1. -- -- see: https://github.com/sol/hpack name: hydra-lisp -version: 0.17.4 +version: 0.17.5 synopsis: Hydra's Lisp coder: emit Clojure/Scheme/Common-Lisp/Emacs-Lisp source description: Hydra is an implementation of the LambdaGraph data model, which takes advantage of an isomorphism between labeled hypergraphs and typed lambda calculus: in Hydra, "graphs are programs, and programs are graphs". Lisp support for Hydra (shared across Clojure, Scheme, Common Lisp, and Emacs Lisp) category: Data @@ -39,6 +39,6 @@ build-depends: base >=4.19.0 && <4.22 , containers >=0.6.7 && <0.8 - , hydra-kernel ==0.17.4 + , hydra-kernel ==0.17.5 , scientific >=0.3.7 && <0.4 default-language: Haskell2010 diff --git a/src/main/haskell/Hydra/Dsl/Lisp/Syntax.hs b/src/main/haskell/Hydra/Dsl/Lisp/Syntax.hs --- a/src/main/haskell/Hydra/Dsl/Lisp/Syntax.hs +++ b/src/main/haskell/Hydra/Dsl/Lisp/Syntax.hs @@ -7,7 +7,7 @@ import qualified Hydra.Core as Core import qualified Hydra.Lisp.Syntax as Syntax import qualified Hydra.Typed as Typed -import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, map, pure, sum) +import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, lines, map, pure, sum, unlines) import qualified Data.Scientific as Sci -- | DSL constructor for hydra.lisp.syntax.AndExpression diff --git a/src/main/haskell/Hydra/Lisp/Coder.hs b/src/main/haskell/Hydra/Lisp/Coder.hs --- a/src/main/haskell/Hydra/Lisp/Coder.hs +++ b/src/main/haskell/Hydra/Lisp/Coder.hs @@ -56,7 +56,7 @@ import qualified Hydra.Validation as Validation import qualified Hydra.Variables as Variables import qualified Hydra.Variants as Variants -import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, map, pure, sum) +import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, lines, map, pure, sum, unlines) import qualified Data.Scientific as Sci import qualified Data.Set as S @@ -267,7 +267,7 @@ let fname = Formatting.convertCaseCamelToLowerSnake (Core.unName (Core.projectionFieldName proj)) tname = qualifiedSnakeName (Core.projectionTypeName proj) - in (Optionals.cases marg (Right (lispLambdaExpr [ + in (Optionals.match marg (Right (lispLambdaExpr [ "__rec"] (Syntax.ExpressionFieldAccess (Syntax.FieldAccess { Syntax.fieldAccessRecordType = (Syntax.Symbol tname), Syntax.fieldAccessField = (Syntax.Symbol fname), @@ -304,7 +304,7 @@ Syntax.mapEntryKey = k, Syntax.mapEntryValue = v})))) (Maps.toList v0)) (\pairs -> Right (Syntax.ExpressionMap (Syntax.MapLiteral { Syntax.mapLiteralEntries = pairs}))) - Core.TermOptional v0 -> Optionals.cases v0 (Right (lispApp (lispVar "list") [ + Core.TermOptional v0 -> Optionals.match v0 (Right (lispApp (lispVar "list") [ lispKeyword "none"])) (\val -> Eithers.bind (encodeTerm dialect overlaySubs cx g val) (\sval -> Right (lispApp (lispVar "list") [ lispKeyword "given", sval]))) @@ -470,7 +470,7 @@ Core.applicationFunction = cfterm, Core.applicationArgument = (Core.TermVariable (Core.Name "match_value"))}))) (\bodyExpr -> Right (Syntax.CondClause { Syntax.condClauseCondition = condExpr, - Syntax.condClauseBody = bodyExpr})))) caseFields) (\clauses -> Eithers.bind (Optionals.cases defCase (Right Nothing) (\dt -> Eithers.bind (encodeTerm dialect overlaySubs cx g dt) (\defBody -> Right (Just defBody)))) (\defExpr -> + Syntax.condClauseBody = bodyExpr})))) caseFields) (\clauses -> Eithers.bind (Optionals.match defCase (Right Nothing) (\dt -> Eithers.bind (encodeTerm dialect overlaySubs cx g dt) (\defBody -> Right (Just defBody)))) (\defExpr -> let condExpr = Syntax.ExpressionCond (Syntax.CondExpression { Syntax.condExpressionClauses = clauses, @@ -480,7 +480,7 @@ "match_value"] condExpr) [ lispApp (lispVar (dialectCadr dialect)) [ lispVar "match_target"]] - in (Optionals.cases marg (Right (lispLambdaExpr [ + in (Optionals.match marg (Right (lispLambdaExpr [ "match_target"] innerExpr)) (\arg -> Eithers.bind (encodeTerm dialect overlaySubs cx g arg) (\sarg -> Right (lispApp (lispLambdaExpr [ "match_target"] innerExpr) [ sarg]))))))) @@ -488,7 +488,7 @@ -- | Encode a Hydra wrap elimination (unwrap) as a Lisp expression, with an optional argument for applied unwraps encodeUnwrapElim :: Syntax.Dialect -> S.Set String -> t0 -> Graph.Graph -> Core.Name -> Maybe Core.Term -> Either t1 Syntax.Expression encodeUnwrapElim dialect overlaySubs cx g name marg = - Optionals.cases marg (Right (lispLambdaExpr [ + Optionals.match marg (Right (lispLambdaExpr [ "__rec"] (lispVar "__rec"))) (\arg -> encodeTerm dialect overlaySubs cx g arg) -- | Check whether a term is a reference to a specific primitive, stripping type applications, type lambdas, and annotations @@ -504,7 +504,7 @@ -- | The per-parameter isLazy flags of the primitive a head term refers to, in order; empty if the head is not a registered primitive lazyFlagsForPrimitiveTerm :: Graph.Graph -> Core.Term -> [Bool] lazyFlagsForPrimitiveTerm g headTerm = - Optionals.cases (primHeadName headTerm) [] (\name -> Optionals.cases (Maps.lookup name (Graph.graphPrimitives g)) [] (\prim -> Lists.map (\p -> Typing.parameterIsLazy p) (Typing.termSignatureParameters (Packaging.primitiveDefinitionSignature (Graph.primitiveDefinition prim))))) + Optionals.match (primHeadName headTerm) [] (\name -> Optionals.match (Maps.lookup name (Graph.graphPrimitives g)) [] (\prim -> Lists.map (\p -> Typing.parameterIsLazy p) (Typing.termSignatureParameters (Packaging.primitiveDefinitionSignature (Graph.primitiveDefinition prim))))) -- | A Lisp function application expression lispApp :: Syntax.Expression -> [Syntax.Expression] -> Syntax.Expression diff --git a/src/main/haskell/Hydra/Lisp/Language.hs b/src/main/haskell/Hydra/Lisp/Language.hs --- a/src/main/haskell/Hydra/Lisp/Language.hs +++ b/src/main/haskell/Hydra/Lisp/Language.hs @@ -36,7 +36,7 @@ import qualified Hydra.Util as Util import qualified Hydra.Validation as Validation import qualified Hydra.Variants as Variants -import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, map, pure, sum) +import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, lines, map, pure, sum, unlines) import qualified Data.Scientific as Sci import qualified Data.Set as S diff --git a/src/main/haskell/Hydra/Lisp/Serde.hs b/src/main/haskell/Hydra/Lisp/Serde.hs --- a/src/main/haskell/Hydra/Lisp/Serde.hs +++ b/src/main/haskell/Hydra/Lisp/Serde.hs @@ -44,7 +44,7 @@ import qualified Hydra.Util as Util import qualified Hydra.Validation as Validation import qualified Hydra.Variants as Variants -import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, map, pure, sum) +import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, lines, map, pure, sum, unlines) import qualified Data.Scientific as Sci -- | Serialize an and expression: (and expr1 expr2 ...) @@ -88,7 +88,7 @@ Serialization.parens (Serialization.spaceSepAdaptive (Lists.map (expressionToExpr d) (Syntax.caseClauseKeys c))), (expressionToExpr d (Syntax.caseClauseBody c))])) clauses defaultPart = - Optionals.cases dflt [] (\e -> [ + Optionals.match dflt [] (\e -> [ Serialization.parens (Serialization.spaceSepAdaptive [ Serialization.cst "else", (expressionToExpr d e)])]) @@ -119,7 +119,7 @@ expressionToExpr d (Syntax.condClauseCondition c), (expressionToExpr d (Syntax.condClauseBody c))]) clauses) defaultPart = - Optionals.cases dflt [] (\e -> [ + Optionals.match dflt [] (\e -> [ Serialization.cst ":else", (expressionToExpr d e)]) in (Serialization.parens (Serialization.spaceSepAdaptive (Lists.concat [ @@ -133,7 +133,7 @@ expressionToExpr d (Syntax.condClauseCondition c), (expressionToExpr d (Syntax.condClauseBody c))])) clauses defaultPart = - Optionals.cases dflt [] (\e -> [ + Optionals.match dflt [] (\e -> [ Serialization.parens (Serialization.spaceSepAdaptive [ Serialization.cst "t", (expressionToExpr d e)])]) @@ -148,7 +148,7 @@ expressionToExpr d (Syntax.condClauseCondition c), (expressionToExpr d (Syntax.condClauseBody c))])) clauses defaultPart = - Optionals.cases dflt [] (\e -> [ + Optionals.match dflt [] (\e -> [ Serialization.parens (Serialization.spaceSepAdaptive [ Serialization.cst "t", (expressionToExpr d e)])]) @@ -163,7 +163,7 @@ expressionToExpr d (Syntax.condClauseCondition c), (expressionToExpr d (Syntax.condClauseBody c))])) clauses defaultPart = - Optionals.cases dflt [] (\e -> [ + Optionals.match dflt [] (\e -> [ Serialization.parens (Serialization.spaceSepAdaptive [ Serialization.cst "else", (expressionToExpr d e)])]) @@ -431,7 +431,7 @@ let cond = expressionToExpr d (Syntax.ifExpressionCondition ifExpr) then_ = expressionToExpr d (Syntax.ifExpressionThen ifExpr) else_ = Syntax.ifExpressionElse ifExpr - elsePart = Optionals.cases else_ [] (\e -> [ + elsePart = Optionals.match else_ [] (\e -> [ expressionToExpr d e]) in (Serialization.parens (Serialization.spaceSepAdaptive (Lists.concat [ [ @@ -472,7 +472,7 @@ Syntax.DialectScheme -> Serialization.noSep [ Serialization.cst "'", (Serialization.cst name)] - _ -> Serialization.cst (Optionals.cases ns (Strings.concat2 ":" name) (\n -> Strings.concat [ + _ -> Serialization.cst (Optionals.match ns (Strings.concat2 ":" name) (\n -> Strings.concat [ n, "/:", name])) @@ -495,7 +495,7 @@ mname = Syntax.lambdaName lam kw = lambdaKeyword d in case d of - Syntax.DialectClojure -> Optionals.cases mname (Serialization.parens (Serialization.spaceSepAdaptive (Lists.concat [ + Syntax.DialectClojure -> Optionals.match mname (Serialization.parens (Serialization.spaceSepAdaptive (Lists.concat [ [ Serialization.cst kw], [ @@ -850,7 +850,7 @@ importNames = Lists.map (\idecl -> Syntax.unNamespaceName (Syntax.importDeclarationModule idecl)) imports exportSyms = Lists.concat (Lists.map (\edecl -> Lists.map symbolToExpr (Syntax.exportDeclarationSymbols edecl)) exports) in case d of - Syntax.DialectClojure -> Optionals.cases modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> + Syntax.DialectClojure -> Optionals.match modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> let nameStr = Syntax.unNamespaceName (Syntax.moduleDeclarationName m) requireClauses = Lists.map (\imp -> Serialization.brackets Serialization.squareBrackets Serialization.inlineStyle (Serialization.spaceSep [ @@ -886,7 +886,7 @@ nsForm], declareForm, formPart]))) - Syntax.DialectEmacsLisp -> Optionals.cases modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> + Syntax.DialectEmacsLisp -> Optionals.match modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> let nameStr = Syntax.unNamespaceName (Syntax.moduleDeclarationName m) requireClLib = Serialization.parens (Serialization.spaceSepAdaptive [ @@ -914,7 +914,7 @@ formPart, [ provideForm]]))) - Syntax.DialectCommonLisp -> Optionals.cases modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> + Syntax.DialectCommonLisp -> Optionals.match modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> let nameStr = Syntax.unNamespaceName (Syntax.moduleDeclarationName m) colonName = Strings.concat2 ":" nameStr useClause = @@ -946,7 +946,7 @@ defpkgForm, inpkgForm], formPart]))) - Syntax.DialectScheme -> Optionals.cases modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> + Syntax.DialectScheme -> Optionals.match modDecl (Serialization.doubleNewlineSep (Lists.concat2 warning formPart)) (\m -> let nameStr = Syntax.unNamespaceName (Syntax.moduleDeclarationName m) nameParts = Lists.map (\p -> Formatting.convertCaseCamelToLowerSnake p) (Strings.splitOn "." nameStr) nameExpr = Serialization.parens (Serialization.spaceSepAdaptive (Lists.map (\p -> Serialization.cst p) nameParts)) @@ -1086,9 +1086,9 @@ let mdoc = Syntax.topLevelFormWithCommentsDoc fwc mcomment = Syntax.topLevelFormWithCommentsComment fwc form = Syntax.topLevelFormWithCommentsForm fwc - docPart = Optionals.cases mdoc [] (\ds -> [ + docPart = Optionals.match mdoc [] (\ds -> [ docstringToExpr ds]) - commentPart = Optionals.cases mcomment [] (\c -> [ + commentPart = Optionals.match mcomment [] (\c -> [ commentToExpr c]) formExpr = topLevelFormToExpr d form in (Serialization.newlineSep (Lists.concat [ diff --git a/src/main/haskell/Hydra/Lisp/Syntax.hs b/src/main/haskell/Hydra/Lisp/Syntax.hs --- a/src/main/haskell/Hydra/Lisp/Syntax.hs +++ b/src/main/haskell/Hydra/Lisp/Syntax.hs @@ -5,7 +5,7 @@ module Hydra.Lisp.Syntax where import qualified Hydra.Core as Core -import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, map, pure, sum) +import Prelude hiding (Enum, Ordering, decodeFloat, encodeFloat, fail, lines, map, pure, sum, unlines) import qualified Data.Scientific as Sci -- | Logical and: (and expr1 expr2 ...)