hydra-pg 0.17.0 → 0.17.1

raw patch · 4 files changed

+286/−4 lines, 4 filesdep ~hydra-kerneldep ~hydra-rdfPVP ok

version bump matches the API change (PVP)

Dependency ranges changed: hydra-kernel, hydra-rdf

API changes (from Hackage documentation)

Files

• CHANGELOG.md +139/−0
• hydra-pg.cabal +6/−4
• src/main/haskell/Hydra/Overlay/Haskell/Dsl/Pg/Mappings.hs +65/−0
• src/main/haskell/Hydra/Overlay/Haskell/Dsl/Pg/Schemas.hs +76/−0

@@ -15,6 +15,145 @@  --- +## [0.17.1] - unreleased++Development version. Opens the 0.17.x line — the first release built entirely against+published 0.17.0 hosts (Hackage / PyPI / Maven Central), with the `hostOverrides` shims+that were needed while 0.16.1 lacked the effect type and I/O primitives removed.++### New features++- **Generated term-reference DSLs** ([#467](https://github.com/CategoricalData/hydra/issues/467)):+  kernel *term* modules now project generated DSL surfaces, completing the story begun with+  type-module DSLs and the generated primitive wrappers (`hydra.dsl.lib.*`, 0.17.0). A+  signature read-back in the DSL pass reuses the main pass's inference, and a demand-curated+  set of 19 kernel term modules (plus the json/test/extract modules) emits one typed,+  rename-safe reference per definition into every target language — e.g. Haskell+  `Hydra.Dsl.Strip.deannotateType`, Java `hydra.dsl.Strip.deannotateType(...)`, Python+  `hydra.dsl.strip.deannotate_type(...)`. The Java and Python coder sources were pivoted onto+  the generated references, retiring ~700 stringly-typed inline `var("hydra....")` references+  and shrinking Python's hand registry to the non-generatable remainder (byte-identical+  generated output before and after). Follow-ups:+  [#555](https://github.com/CategoricalData/hydra/issues/555) (typed references to derived+  encode/decode/show functions via `TypedName`),+  [#556](https://github.com/CategoricalData/hydra/issues/556) (non-kernel packages).+- **Package validation** ([#574](https://github.com/CategoricalData/hydra/issues/574),+  [#575](https://github.com/CategoricalData/hydra/issues/575)): a translingual `hydra.validate.*`+  framework carries per-package `ValidationProfile`s (error/warning rules, error/warning caps) and a+  catalog of checks — undeclared-package-dependency detection (now a fatal gate, #574),+  definition-name convention (PascalCase), definition ordering, documentation completeness, and+  conflicting-variant-name detection — layered on the existing core term/type well-formedness rules.+  A kernel-strict / others-relaxed policy keeps `/sync` green fleet-wide while the documentation+  backlog drains. See the new [Validation](https://github.com/CategoricalData/hydra/wiki/Validation)+  wiki page for the full per-package constraint specification.+- **DSL-authoring registration guards** ([#554](https://github.com/CategoricalData/hydra/issues/554)):+  fail-fast authoring-time validation for the host-native DSL sources — every DSL-authoring module in+  Java/`hydra-jvm`, Python, and Scala now asserts its definitions are fully registered+  (`Defs.checkComplete` / `check_complete`), backed by a Haskell kernel-source registration-completeness+  scanner, so a definition added to a source file but omitted from its `definitions` list fails loudly+  instead of silently vanishing from the generated output.+- **`hydra-build` package** ([#546](https://github.com/CategoricalData/hydra/issues/546),+  [#529](https://github.com/CategoricalData/hydra/issues/529),+  [#530](https://github.com/CategoricalData/hydra/issues/530)): the build-orchestration logic that had+  lived only in host scripts is now translingual Hydra. Pure module-list utilities became+  `hydra.build.modules` (#529) and the orphan-reconcile decision became `hydra.build.reconcile` (#530),+  each with a translingual test suite; the `hydra.build.*` modules were then extracted into a dedicated+  `hydra-build` package (#546), separate from the kernel.+- **Metadata sidecar + content/version-addressed freshness**+  ([#415](https://github.com/CategoricalData/hydra/issues/415)): dist manifests now carry a+  `moduleFormatVersion` field, and the generator's freshness basis is content/version-addressed — the+  groundwork for reproducible, cache-correct incremental generation.+- **Structured generation-provenance record**+  ([#523](https://github.com/CategoricalData/hydra/issues/523)): each generation run now records a+  structured `Generation` provenance entry (digest v2), replacing the ad-hoc shell threading of+  generation metadata.+- **Worker + issue hierarchy** ([#557](https://github.com/CategoricalData/hydra/issues/557)): a+  principled agent/worker framework — roles and parentage mirroring the GitHub issue tree, a+  cross-worktree message transport with a verify-after-copy protocol, a blocking-aware design-question+  (`decisions/`) channel, an issue-proposal queue with the `/issues` command, an autonomy-dial banner,+  and cross-machine staging coordination. Contributor-facing process only; no change to generated code.++### API and usability++- **DSL usability passes** ([#430](https://github.com/CategoricalData/hydra/issues/430),+  [#553](https://github.com/CategoricalData/hydra/issues/553)): reviewed the host-native Java, Python,+  and Scala DSL surfaces for ergonomics and cross-host parity (documented def shapes, added a Scala+  projection helper), bringing the three authoring dialects into closer alignment.++### Fixed++Bug fixes landed this cycle, grouped by area (see each issue for detail):++- **TypeScript** — reactivated `hydra.lib.*` primitive tests surfaced encoder gaps: lazy show-calling+  let bindings and compact bare-string `LiteralType` expansion in the test converter+  ([#564](https://github.com/CategoricalData/hydra/issues/564)),+  `binaryToBytes` registration + base64 binary decode in `decodeUtf8`+  ([#561](https://github.com/CategoricalData/hydra/issues/561)),+  `hydra.test.build.*` emission after the #546 split+  ([#570](https://github.com/CategoricalData/hydra/issues/570)), and `HYDRA_DEFAULT_IMPLS` support in+  the test runner ([#536](https://github.com/CategoricalData/hydra/issues/536)).+- **Tests and CI** — restored the default-primitive-implementation testing solution+  ([#388](https://github.com/CategoricalData/hydra/issues/388),+  [#549](https://github.com/CategoricalData/hydra/issues/549)), reactivated previously-ignored kernel+  test cases ([#513](https://github.com/CategoricalData/hydra/issues/513),+  [#550](https://github.com/CategoricalData/hydra/issues/550)), wired the regression harnesses into CI+  and split them out of the monolithic Haskell job for timeout headroom+  ([#535](https://github.com/CategoricalData/hydra/issues/535),+  [#563](https://github.com/CategoricalData/hydra/issues/563)), added Scala/Go/Lisp+  self-containment gates to `verify-distribution.sh`+  ([#537](https://github.com/CategoricalData/hydra/issues/537)), a manifest-JSON-tracked CI check+  ([#541](https://github.com/CategoricalData/hydra/issues/541)), a per-package test-digest freshness+  fix ([#551](https://github.com/CategoricalData/hydra/issues/551)), overlay-file restoration in+  `test-stale-output-prune.sh` ([#558](https://github.com/CategoricalData/hydra/issues/558)), and a+  sync-freshness fingerprint that now sees host-native DSL sources+  ([#562](https://github.com/CategoricalData/hydra/issues/562)).+- **Kernel and coders** — Java binary-literal signed-byte emission+  ([#528](https://github.com/CategoricalData/hydra/issues/528)), `hydra.json.Decode.fromJson`+  parametric-type (Application/Forall) support+  ([#531](https://github.com/CategoricalData/hydra/issues/531)), stale 2-param `Coder` shape in+  Java/Scala test bootstrap ([#552](https://github.com/CategoricalData/hydra/issues/552)), and a+  kernel dead-code/TODO cleanup ([#538](https://github.com/CategoricalData/hydra/issues/538)).+- **Lib and build** — `hydra.lib.files` copy/removeDirectory/status overlays across all nine hosts+  ([#525](https://github.com/CategoricalData/hydra/issues/525)), `hydra.build.*` regeneration into the+  Lisp dialects ([#544](https://github.com/CategoricalData/hydra/issues/544)), Scheme maps+  last-writer-wins in `union` ([#545](https://github.com/CategoricalData/hydra/issues/545)),+  standalone-kernel SHA dependency ([#542](https://github.com/CategoricalData/hydra/issues/542)),+  the lexicon's empty Types section ([#539](https://github.com/CategoricalData/hydra/issues/539)), and+  a duplicated generated-file header in `generate-head-haskell-build.py`+  ([#540](https://github.com/CategoricalData/hydra/issues/540)).++- **TypeScript overlay-import rewrite** ([#565](https://github.com/CategoricalData/hydra/issues/565)):+  the TypeScript coder's lib overlay-import rewrite no longer captures `hydra.lib.defaults`, which is a+  generated (not overlaid) module; excluding it fixes spurious import references in generated TypeScript.++- **Lexicon generation** (`bin/regenerate-lexicon.sh`): the generator had been failing+  silently since the encoder/decoder modules moved to in-memory synthesis+  ([#448](https://github.com/CategoricalData/hydra/issues/448)) — the script masked the+  failure and left a stale `docs/hydra-lexicon.txt` missing the four effectful primitive+  libraries (`hydra.lib.{effects,files,system,text}`). The synthesized coder modules are+  now included in the lexicon universe, the script fails loudly, and the regenerated+  lexicon covers all 267 primitives across 17 library modules.+- **`bin/prepare-release.sh`**: the non-blocking Java quality check invoked Gradle+  subprojects (`:hydra-rdf4j:test` etc.) that were removed in+  [#511](https://github.com/CategoricalData/hydra/issues/511), so it always warned without+  checking anything; it now runs the `dist/java/hydra-pg` package tests (the current home+  of the former binding tests).+- **`.gitignore`**: the ignore rules for the overlaid hand-written Haskell runtime still+  named pre-[#501](https://github.com/CategoricalData/hydra/issues/501) paths, so 41+  duplicate copies under `dist/haskell/` had become tracked; the rules now match the+  `Hydra/Overlay/` location and the duplicates are untracked.++### Documentation++- Corrected the 0.17.0 Maven coordinates throughout (Java artifacts publish under group+  `net.fortytwo.hydra.java`, Scala under `net.fortytwo.hydra.scala`,+  [#519](https://github.com/CategoricalData/hydra/issues/519)); added the npm and Scala+  release channels to the README; added TypeScript/Scala/Lisp sections to Getting started;+  repointed stale pre-#418/#501 DSL paths in the architecture docs; added GitHub issue and+  PR templates.++---+ ## [0.17.0] - 2026-07-01  Major themes: first-class support for effects and I/O in the kernel, TypeScript and

@@ -1,11 +1,11 @@ cabal-version: 1.12 --- This file has been generated from package.yaml by hpack version 0.39.1.+-- This file has been generated from package.yaml by hpack version 0.38.1. -- -- see: https://github.com/sol/hpack  name:           hydra-pg-version:        0.17.0+version:        0.17.1 synopsis:       Hydra's property-graph (TinkerPop/Gremlin) model and coder support description:    Hydra is an implementation of the LambdaGraph data model, which takes advantage of an isomorphism between labeled hypergraphs and typed lambda calculus: in Hydra, "graphs are programs, and programs are graphs". Property graph support for Hydra category:       Data@@ -46,6 +46,8 @@       Hydra.Graphviz.Serde       Hydra.Neo4j.Model       Hydra.Neo4j.Pg+      Hydra.Overlay.Haskell.Dsl.Pg.Mappings+      Hydra.Overlay.Haskell.Dsl.Pg.Schemas       Hydra.Pg.Coder       Hydra.Pg.Graphson.Coder       Hydra.Pg.Graphson.Construct@@ -75,7 +77,7 @@       base >=4.19.0 && <4.22     , bytestring >=0.11.5 && <0.13     , containers >=0.6.7 && <0.8-    , hydra-kernel ==0.17.0-    , hydra-rdf ==0.17.0+    , hydra-kernel ==0.17.1+    , hydra-rdf ==0.17.1     , scientific >=0.3.7 && <0.4   default-language: Haskell2010

@@ -0,0 +1,65 @@+-- | A DSL for defining mappings between relational data and property graphs in Hydra.+module Hydra.Overlay.Haskell.Dsl.Pg.Mappings where++import Hydra.Core+import Hydra.Typed+import Hydra.Rewriting+import Hydra.Pg.Model+import Hydra.Overlay.Haskell.Dsl.Typed.Phantoms++import qualified Data.Map as M++-- * Helpers++temporaryTableName = "(((temp)))"++replaceTableName tableName = rewriteTerm $ \recurse term -> case term of+  TermProject (Projection tname fname) -> if unName tname == temporaryTableName+    then TermProject $ Projection (Name tableName) fname+    else term+  _ -> recurse term++-- * Graph construction functions++-- | Create a mapping function that selects a specified column from a table+column :: String -> TypedTerm (r -> Maybe a)+column sourceColumn = project (Name temporaryTableName) (Name sourceColumn)++-- | Create an edge with the given label, ID, source vertex, target vertex, and properties+-- Example: edge "social_network.csv" "knows" (constant unit) (column "person_1") (column "person_2")+--            [property "since" $ column "knows_since"]+edge :: String -> String -> TypedTerm e -> TypedTerm vo -> TypedTerm vi -> [Property Term] -> Edge Term+edge tableName label (TypedTerm idValue) (TypedTerm outValue) (TypedTerm inValue) propertiesList = Edge+  (EdgeLabel label)+  (replaceTableName tableName idValue)+  (replaceTableName tableName outValue)+  (replaceTableName tableName inValue)+  (M.fromList $ fmap (\(Property k v) -> (k, replaceTableName tableName v)) propertiesList)++-- | Create an edge with a unit ID (when the edge id isn't significant)+-- Example: simpleEdge "social_network.csv" "knows" (column "person_1") (column "person_2")+--            [property "since" $ column "knows_since"]+edgeNoId :: String -> String -> TypedTerm vo -> TypedTerm vi -> [Property Term] -> Edge Term+edgeNoId tableName label = edge tableName label (constant unit)++-- | Create a lazy graph from vertices and edges, without enforcing id uniqueness+-- Example: graph [person1, person2] [knows]+-- Where:+--   person1 = vertex "Person" (string "p1") [property "name" (string "John")]+--   person2 = vertex "Person" (string "p2") [property "name" (string "Jane")]+--   knows = edge "knows" (string "e1") person1 person2 [property "since" (int32 2020)]+graph :: [Vertex v] -> [Edge v] -> LazyGraph v+graph = LazyGraph++-- | Create a property with the given key and value+-- Example: property "name" (string "John")+property :: String -> TypedTerm v -> Property Term+property key (TypedTerm value) = Property (PropertyKey key) value++-- | Create a vertex derived from the given table with the given label, ID, and properties+-- Example: vertex "people.csv" "Person" personId [property "name" (column "full_name"), property "age" (column "age)]+vertex :: String -> String -> TypedTerm v -> [Property Term] -> Vertex Term+vertex tableName label (TypedTerm idValue) propertiesList = Vertex+  (VertexLabel label)+  (replaceTableName tableName idValue)+  (M.fromList $ fmap (\(Property k v) -> (k, replaceTableName tableName v)) propertiesList)

@@ -0,0 +1,76 @@+-- | A DSL for constructing property graph schemas and instances in Hydra.+module Hydra.Overlay.Haskell.Dsl.Pg.Schemas where++import Hydra.Pg.Model+import Hydra.Core+import qualified Hydra.Overlay.Haskell.Dsl.Types as Types+import qualified Hydra.Overlay.Haskell.Dsl.Terms as Terms++import qualified Data.Map as M++-- * Type-level functions (for schema definitions)++-- | Create a property type definition with the given key and type+-- Example: propertyType "name" Types.string+propertyType :: String -> t -> PropertyType t+propertyType key typ = PropertyType (PropertyKey key) typ False++-- | Mark a property type as required+-- Example: required (propertyType "name" Types.string)+required :: PropertyType t -> PropertyType t+required (PropertyType key typ _) = PropertyType key typ True++-- | Define a vertex type with the given label, ID type, and property types+-- Example: vertexType "Person" Types.string [required (propertyType "name" Types.string), propertyType "age" Types.int32]+vertexType :: String -> t -> [PropertyType t] -> VertexType t+vertexType label id properties = VertexType (VertexLabel label) id properties++-- | Define an edge type with the given label, ID type, source vertex label, target vertex label, and property types+-- Example: edgeType "KNOWS" Types.string "Person" "Person" [propertyType "since" Types.int32]+edgeType :: String -> t -> String -> String -> [PropertyType t] -> EdgeType t+edgeType label id outLabel inLabel properties = EdgeType (EdgeLabel label) id (VertexLabel outLabel) (VertexLabel inLabel) properties++-- | Define a simple edge type with a unit ID type+-- Example: simpleEdgeType "KNOWS" "Person" "Person" [propertyType "since" Types.int32]+simpleEdgeType :: String -> String -> String -> [PropertyType Type] -> EdgeType Type+simpleEdgeType label = edgeType label Types.unit++-- | Create a graph schema from vertex types and edge types+-- Example: schema [personType] [knowsType]+-- Where:+--   personType = vertexType "Person" Types.string [required (propertyType "name" Types.string)]+--   knowsType = edgeType "KNOWS" Types.string "Person" "Person" [propertyType "since" Types.int32]+schema :: [VertexType t] -> [EdgeType t] -> GraphSchema t+schema vtypes etypes = GraphSchema+  (M.fromList $ fmap (\vt -> (vertexTypeLabel vt, vt)) vtypes)+  (M.fromList $ fmap (\et -> (edgeTypeLabel et, et)) etypes)++-- * Term-level functions (for graph instances)++-- | Create a property with the given key and value+-- Example: property "name" (Terms.string "John")+property :: String -> v -> Property v+property key value = Property (PropertyKey key) value++-- | Create a vertex with the given label, ID, and properties+-- Example: vertex "Person" (Terms.string "p1") [property "name" (Terms.string "John"), property "age" (Terms.int32 42)]+vertex :: String -> v -> [Property v] -> Vertex v+vertex label idValue propertiesList = Vertex (VertexLabel label) idValue $+  M.fromList $ fmap (\(Property k v) -> (k, v)) propertiesList++-- | Create an edge with the given label, ID, source vertex, target vertex, and properties+-- Example: edge "KNOWS" (Terms.string "e1") person1 person2 [property "since" (Terms.int32 2020)]+edge :: String -> v -> v -> v -> [Property v] -> Edge v+edge label idValue outValue inValue propertiesList = Edge (EdgeLabel label) idValue outValue inValue $+  M.fromList $ fmap (\(Property k v) -> (k, v)) propertiesList++-- | Create a graph from vertices and edges+-- Example: graph [person1, person2] [knows]+-- Where:+--   person1 = vertex "Person" (Terms.string "p1") [property "name" (Terms.string "John")]+--   person2 = vertex "Person" (Terms.string "p2") [property "name" (Terms.string "Jane")]+--   knows = edge "KNOWS" (Terms.string "e1") person1 person2 [property "since" (Terms.int32 2020)]+graph :: Ord v => [Vertex v] -> [Edge v] -> Graph v+graph vertices edges = Graph+  (M.fromList $ fmap (\v -> (vertexId v, v)) vertices)+  (M.fromList $ fmap (\e -> (edgeId e, e)) edges)