diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,88 @@ --- +## [0.17.7] - 2026-09-14 + +Correctness-hardening point release on the 0.17.x line, consolidating the 0.17.6 breaking batch. It +closes a family of silent cross-host correctness bugs in the equality, ordering, integer-narrowing +and string primitives, and adds the systematic test coverage that exposed them. No kernel-surface +breaking changes. + +### Highlights + +- **Silent-wrong primitives corrected across hosts** + ([#745](https://github.com/CategoricalData/hydra/issues/745), + [#742](https://github.com/CategoricalData/hydra/issues/742)): new shared cross-host fixtures + uncovered ~a dozen primitives (`equal`/`compare` on collections, float equality, `bigintToIntN` + narrowing, `parseUint*` bounds, astral-plane string ops) that returned wrong results on one or more + non-Haskell hosts while appearing to pass. All are now correct and verified per-host against the + Haskell reference oracle. +- **Systematic test-coverage hardening** + ([#749](https://github.com/CategoricalData/hydra/issues/749)): new kernel coverage for previously + untested primitive behavior (equality/ordering collection depth, `notEqual`, `functions.const`/`flip`), + which drove wiring those primitives into every host's default-implementation fallback. +- **Scale-preserving decimals across every lossy-double host** + ([#727](https://github.com/CategoricalData/hydra/issues/727), + [#719](https://github.com/CategoricalData/hydra/issues/719)): `Literal.decimal` gains a real + scale-preserving representation (coefficient + scale) in the Scheme, Emacs Lisp, Common Lisp and + TypeScript runtimes and coders, and the generated `Literal.Decimal` `equals`/`compareTo` on Java, + Python and Scala stop ignoring scale. The `scaleDistinctTestNames` skip list — which had been + masking the gap on five hosts — is deleted, so decimal scale is now verified everywhere rather + than excused. + +### Bug fixes + +- **[#745](https://github.com/CategoricalData/hydra/issues/745)** silent-wrong equality/ordering, + `bigintToIntN` narrowing, `parseUint32/64` bounds, and astral-plane `charAt`/`splitOn`/`compare` + across the eight non-Haskell hosts. +- **[#742](https://github.com/CategoricalData/hydra/issues/742)** structural `equal`/`compare` on + maps, sets and nested decimals no longer falls back to scale-blind or print-based comparison + (including a hand-inlined Scala primitive that shadowed the overlay fix). +- **[#743](https://github.com/CategoricalData/hydra/issues/743)** kernel JSON decoder now rejects an + `Either` object carrying both `left` and `right` keys instead of silently resolving one arm. +- **[#740](https://github.com/CategoricalData/hydra/issues/740)** the recursive parametric union decoder + now emits an explicit type application instead of an untyped lambda, so recursive parametric-union + types generate and compile on the Java host; also fixes `update-json-kernel` reconcile to build a + complete inference universe so drifted kernel modules re-infer. +- **[#720](https://github.com/CategoricalData/hydra/issues/720)** Python's + `hydra.lib.equality`/`ordering` did not satisfy IEEE `totalOrder` for floats, mis-ordering `NaN` + and signed zero. +- **[#722](https://github.com/CategoricalData/hydra/issues/722)** two `kernelDefaultCoreProfile` + correctness defects: rule ids that referred to rules no longer present, and a stubbed + `isValidName` that accepted every name. + +### Improvements + +- **[#749](https://github.com/CategoricalData/hydra/issues/749)** `notEqual`/`const`/`flip` wired + into each host's `#609` default-implementation fallback; orphaned Avro coder test suite recovered. +- **[#730](https://github.com/CategoricalData/hydra/issues/730)** a shared `~/.stack` build slot + (`bin/with-stack-slot.sh`) replaces the pgrep slot mechanic, serializing concurrent GHC builds + across the fleet behind a real mutex. The wrapper is now portable to macOS: `flock(1)` is + util-linux and absent there, so it is emulated via Perl's `flock()` where missing, and the + GNU-only `date -Is`/`date -d` calls are replaced with portable equivalents. Its bypass path also + no longer re-execs forever. Without these, every entry point that routes through the slot — + `bin/sync.sh`, `bin/test.sh`, `bin/run-bootstrapping-demo.sh` and `bin/prepare-release.sh` — + failed immediately on macOS. +- **[#600](https://github.com/CategoricalData/hydra/issues/600)** the five published TypeScript + packages are subpath-import-only (uniform export surface). +- **[#716](https://github.com/CategoricalData/hydra/issues/716)** `hydra.paths` reshaped onto the + term-graph model (`TermGraph`/`TypeGraph`), aligning path steps with the term and type grammars + and adding the link view of a graph; `mapKey`/`mapValue` collapse into a single `mapEntry` step. +- **[#701](https://github.com/CategoricalData/hydra/issues/701)** module cache freshness is now a + Merkle hash over each module's dependency closure, so a change deep in the closure correctly + invalidates dependents. +- **[#559](https://github.com/CategoricalData/hydra/issues/559)** host-agnostic build: the + remaining hand-coded host build logic is promoted into `hydra.build.*`, with Java as the + definitive second host. +- **[#721](https://github.com/CategoricalData/hydra/issues/721)** `checkNestedModuleNames` packaging + check enforces the module-namespace no-prefix rule. + +### Documentation + +- **[#508](https://github.com/CategoricalData/hydra/issues/508)** de-Claude-speak documentation pass: + style-guide section added, stale paths and dead links repaired, agent-operations wiki pages moved + into `claude/`, and a `getting-started.md` guardrail against re-adding a TypeScript root export. + ## [0.17.6] - 2026-08-30 Point release on the 0.17.x line, and the last backward-incompatible batch before the 1.0 surface diff --git a/hydra-python.cabal b/hydra-python.cabal --- a/hydra-python.cabal +++ b/hydra-python.cabal @@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack name: hydra-python -version: 0.17.6 +version: 0.17.7 synopsis: Hydra's Python coder: emit Python source from Hydra modules description: Hydra is an implementation of the LambdaGraph data model, which takes advantage of an isomorphism between labeled hypergraphs and typed lambda calculus: in Hydra, "graphs are programs, and programs are graphs". Python support for Hydra category: Data @@ -47,6 +47,6 @@ build-depends: base >=4.19.0 && <4.22 , containers >=0.6.7 && <0.8 - , hydra-kernel ==0.17.6 + , hydra-kernel ==0.17.7 , scientific >=0.3.7 && <0.4 default-language: Haskell2010 diff --git a/src/main/haskell/Hydra/Dsl/Python/Utils.hs b/src/main/haskell/Hydra/Dsl/Python/Utils.hs --- a/src/main/haskell/Hydra/Dsl/Python/Utils.hs +++ b/src/main/haskell/Hydra/Dsl/Python/Utils.hs @@ -94,6 +94,10 @@ Core.applicationFunction = (Core.TermVariable (Core.Name "hydra.python.utils.commentStatement")), Core.applicationArgument = (Typed.unTypedTerm arg0)})) +-- | DSL reference to hydra.python.utils.decimalVariantMethods +decimalVariantMethods :: Typed.TypedTerm [Syntax.Statement] +decimalVariantMethods = Typed.TypedTerm (Core.TermVariable (Core.Name "hydra.python.utils.decimalVariantMethods")) + -- | DSL reference to hydra.python.utils.decodePyComparisonToPyAwaitPrimary decodePyComparisonToPyAwaitPrimary :: Typed.TypedTerm Syntax.Comparison -> Typed.TypedTerm (Maybe Syntax.Primary) decodePyComparisonToPyAwaitPrimary arg0 = diff --git a/src/main/haskell/Hydra/Python/Coder.hs b/src/main/haskell/Hydra/Python/Coder.hs --- a/src/main/haskell/Hydra/Python/Coder.hs +++ b/src/main/haskell/Hydra/Python/Coder.hs @@ -1320,13 +1320,15 @@ ftype = Core.fieldTypeType fieldType in (Eithers.bind (Annotations.getTypeDescription cx (pythonEnvironmentGetGraph env) ftype) (\fcomment -> let isUnit = Equality.equal (Strip.deannotateType ftype) Core.TypeUnit + isDecimal = Equality.equal (Strip.deannotateType ftype) (Core.TypeLiteral Core.LiteralTypeDecimal) varName = deconflictVariantName False env unionName fname (PythonEnvironment.pythonEnvironmentGraph env) tparamNames = findTypeParams env ftype tparamPyNames = Lists.map PythonNames.encodeTypeVariable tparamNames fieldParams = Lists.map Utils.pyNameToPyTypeParameter tparamPyNames body = Logic.ifElse isUnit (Utils.indentedBlock fcomment [ - Utils.unitVariantMethods varName]) (Utils.indentedBlock fcomment []) + Utils.unitVariantMethods varName]) (Logic.ifElse isDecimal (Utils.indentedBlock fcomment [ + Utils.decimalVariantMethods]) (Utils.indentedBlock fcomment [])) in (Eithers.bind (Logic.ifElse isUnit (Right Nothing) (Eithers.bind (encodeTypeQuoted env ftype) (\quotedType -> Right (Just (variantArgs quotedType []))))) (\margs -> Right (Utils.pyClassDefinitionToPyStatement (Syntax.ClassDefinition { Syntax.classDefinitionDecorators = Nothing, Syntax.classDefinitionName = varName, @@ -1954,6 +1956,11 @@ (condImportSymbol "Lazy" True), (condImportSymbol "PersistentMap" (PythonEnvironment.pythonModuleMetadataUsesFrozenDict meta)), (condImportSymbol "PersistentSet" (PythonEnvironment.pythonModuleMetadataUsesFrozenSet meta))]), + ( + "hydra.overlay.python.util._decimal", + [ + condImportSymbol "decimal_node_eq" (PythonEnvironment.pythonModuleMetadataUsesDecimal meta), + (condImportSymbol "decimal_node_hash" (PythonEnvironment.pythonModuleMetadataUsesDecimal meta))]), ( "typing", [ diff --git a/src/main/haskell/Hydra/Python/Utils.hs b/src/main/haskell/Hydra/Python/Utils.hs --- a/src/main/haskell/Hydra/Python/Utils.hs +++ b/src/main/haskell/Hydra/Python/Utils.hs @@ -83,6 +83,109 @@ commentStatement :: String -> Syntax.Statement commentStatement s = pyExpressionToPyStatement (tripleQuotedString s) +-- | Generate __eq__ and __hash__ methods for the Decimal-typed union variant, delegating to hydra.overlay.python.util._decimal (scale-aware per docs/specification/ordering-and-equality.md; Node's generic derivation delegates to Decimal's own scale-blind equality/hash) +decimalVariantMethods :: [Syntax.Statement] +decimalVariantMethods = + [ + eqMethod, + hashMethod] + where + eqBody = + pySimpleStatementToPyStatement (Syntax.SimpleStatementReturn (Syntax.ReturnStatement [ + Syntax.StarExpressionSimple (functionCall (Syntax.PrimarySimple (Syntax.AtomName (Syntax.Name "decimal_node_eq"))) [ + Syntax.ExpressionSimple (Syntax.Disjunction [ + Syntax.Conjunction [ + Syntax.InversionSimple (Syntax.Comparison { + Syntax.comparisonLhs = Syntax.BitwiseOr { + Syntax.bitwiseOrLhs = Nothing, + Syntax.bitwiseOrRhs = Syntax.BitwiseXor { + Syntax.bitwiseXorLhs = Nothing, + Syntax.bitwiseXorRhs = Syntax.BitwiseAnd { + Syntax.bitwiseAndLhs = Nothing, + Syntax.bitwiseAndRhs = Syntax.ShiftExpression { + Syntax.shiftExpressionLhs = Nothing, + Syntax.shiftExpressionRhs = Syntax.Sum { + Syntax.sumLhs = Nothing, + Syntax.sumRhs = Syntax.Term { + Syntax.termLhs = Nothing, + Syntax.termRhs = (Syntax.FactorSimple (Syntax.Power { + Syntax.powerLhs = Syntax.AwaitPrimary { + Syntax.awaitPrimaryAwait = False, + Syntax.awaitPrimaryPrimary = (Syntax.PrimarySimple (Syntax.AtomName (Syntax.Name "self")))}, + Syntax.powerRhs = Nothing}))}}}}}}, + Syntax.comparisonRhs = []})]]), + (Syntax.ExpressionSimple (Syntax.Disjunction [ + Syntax.Conjunction [ + Syntax.InversionSimple (Syntax.Comparison { + Syntax.comparisonLhs = Syntax.BitwiseOr { + Syntax.bitwiseOrLhs = Nothing, + Syntax.bitwiseOrRhs = Syntax.BitwiseXor { + Syntax.bitwiseXorLhs = Nothing, + Syntax.bitwiseXorRhs = Syntax.BitwiseAnd { + Syntax.bitwiseAndLhs = Nothing, + Syntax.bitwiseAndRhs = Syntax.ShiftExpression { + Syntax.shiftExpressionLhs = Nothing, + Syntax.shiftExpressionRhs = Syntax.Sum { + Syntax.sumLhs = Nothing, + Syntax.sumRhs = Syntax.Term { + Syntax.termLhs = Nothing, + Syntax.termRhs = (Syntax.FactorSimple (Syntax.Power { + Syntax.powerLhs = Syntax.AwaitPrimary { + Syntax.awaitPrimaryAwait = False, + Syntax.awaitPrimaryPrimary = (Syntax.PrimarySimple (Syntax.AtomName (Syntax.Name "other")))}, + Syntax.powerRhs = Nothing}))}}}}}}, + Syntax.comparisonRhs = []})]]))])])) + eqMethod = + Syntax.StatementCompound (Syntax.CompoundStatementFunction (Syntax.FunctionDefinition { + Syntax.functionDefinitionDecorators = Nothing, + Syntax.functionDefinitionRaw = Syntax.FunctionDefRaw { + Syntax.functionDefRawAsync = False, + Syntax.functionDefRawName = (Syntax.Name "__eq__"), + Syntax.functionDefRawTypeParams = [], + Syntax.functionDefRawParams = (Just selfOtherParams), + Syntax.functionDefRawReturnType = Nothing, + Syntax.functionDefRawFuncTypeComment = Nothing, + Syntax.functionDefRawBlock = (indentedBlock Nothing [ + [ + eqBody]])}})) + hashBody = + pySimpleStatementToPyStatement (Syntax.SimpleStatementReturn (Syntax.ReturnStatement [ + Syntax.StarExpressionSimple (functionCall (Syntax.PrimarySimple (Syntax.AtomName (Syntax.Name "decimal_node_hash"))) [ + Syntax.ExpressionSimple (Syntax.Disjunction [ + Syntax.Conjunction [ + Syntax.InversionSimple (Syntax.Comparison { + Syntax.comparisonLhs = Syntax.BitwiseOr { + Syntax.bitwiseOrLhs = Nothing, + Syntax.bitwiseOrRhs = Syntax.BitwiseXor { + Syntax.bitwiseXorLhs = Nothing, + Syntax.bitwiseXorRhs = Syntax.BitwiseAnd { + Syntax.bitwiseAndLhs = Nothing, + Syntax.bitwiseAndRhs = Syntax.ShiftExpression { + Syntax.shiftExpressionLhs = Nothing, + Syntax.shiftExpressionRhs = Syntax.Sum { + Syntax.sumLhs = Nothing, + Syntax.sumRhs = Syntax.Term { + Syntax.termLhs = Nothing, + Syntax.termRhs = (Syntax.FactorSimple (Syntax.Power { + Syntax.powerLhs = Syntax.AwaitPrimary { + Syntax.awaitPrimaryAwait = False, + Syntax.awaitPrimaryPrimary = (Syntax.PrimarySimple (Syntax.AtomName (Syntax.Name "self")))}, + Syntax.powerRhs = Nothing}))}}}}}}, + Syntax.comparisonRhs = []})]])])])) + hashMethod = + Syntax.StatementCompound (Syntax.CompoundStatementFunction (Syntax.FunctionDefinition { + Syntax.functionDefinitionDecorators = Nothing, + Syntax.functionDefinitionRaw = Syntax.FunctionDefRaw { + Syntax.functionDefRawAsync = False, + Syntax.functionDefRawName = (Syntax.Name "__hash__"), + Syntax.functionDefRawTypeParams = [], + Syntax.functionDefRawParams = (Just selfOnlyParams), + Syntax.functionDefRawReturnType = Nothing, + Syntax.functionDefRawFuncTypeComment = Nothing, + Syntax.functionDefRawBlock = (indentedBlock Nothing [ + [ + hashBody]])}})) + -- | Decode a Comparison to a Primary if possible decodePyComparisonToPyAwaitPrimary :: Syntax.Comparison -> Maybe Syntax.Primary decodePyComparisonToPyAwaitPrimary c =