shikumi-compile 0.2.1.0 → 0.2.1.1

raw patch · 11 files changed

+87/−73 lines, 11 filesdep ~baikaidep ~effectfuldep ~shikumi-compilePVP ok

version bump matches the API change (PVP)

Dependency ranges changed: baikai, effectful, shikumi-compile

API changes (from Hackage documentation)

Files

• CHANGELOG.md +4/−0
• shikumi-compile.cabal +49/−40
• src/Shikumi/Compile/ChainOfThought.hs +10/−10
• src/Shikumi/Compile/FewShot.hs +2/−2
• src/Shikumi/Compile/RAG.hs +4/−4
• src/Shikumi/Compile/Serialize.hs +1/−1
• src/Shikumi/Compile/Types.hs +9/−9
• src/Shikumi/Compile/ZeroShot.hs +1/−1
• test/Main.hs +4/−3
• test/Test/Capture.hs +2/−2
• test/Test/Fixtures.hs +1/−1

@@ -2,6 +2,10 @@  ## Unreleased +## 0.2.1.1 — 2026-10-05++- Move the dependency on `mori://shinzui/baikai/packages/baikai` to `>=0.7.1.0 && <0.8` and widen the `effectful` bound to `>=2.6 && <2.8`, so both effectful 2.6 and 2.7 are supported (effectful 2.7 needs `baikai-effectful` 0.4.0.2, effectful 2.6 needs 0.4.0.1). Bounds only; no source changed.+ ## 0.2.1.0 — 2026-09-08  - Raise the internal `shikumi` bound to `^>=0.4.0.0` for the breaking core release.

@@ -1,8 +1,8 @@-cabal-version:   3.4-name:            shikumi-compile-version:         0.2.1.0-synopsis:        The compiler layer for shikumi LM programs (EP-9)-category:        AI+cabal-version: 3.4+name: shikumi-compile+version: 0.2.1.1+synopsis: The compiler layer for shikumi LM programs (EP-9)+category: AI description:   The compiler layer for shikumi: pure @Program -> Program@ transformations that   bake a prompting strategy into a program's per-node parameters or structure@@ -14,20 +14,26 @@   acceptance assertion is about the prompt the model would have seen, observed   offline through a capturing stub, so the whole package tests with no network. -license:         BSD-3-Clause-author:          Nadeem Bitar-maintainer:      nadeem@gmail.com-build-type:      Simple+license: BSD-3-Clause+author: Nadeem Bitar+maintainer: nadeem@gmail.com+build-type: Simple extra-doc-files: CHANGELOG.md  common common-options   ghc-options:-    -Wall -Wcompat -Widentities -Wincomplete-uni-patterns-    -Wincomplete-record-updates -Wredundant-constraints-    -fhide-source-paths -Wmissing-export-lists -Wpartial-fields+    -Wall+    -Wcompat+    -Widentities+    -Wincomplete-uni-patterns+    -Wincomplete-record-updates+    -Wredundant-constraints+    -fhide-source-paths+    -Wmissing-export-lists+    -Wpartial-fields     -Wmissing-deriving-strategies -  default-language:   GHC2024+  default-language: GHC2024   default-extensions:     DeriveAnyClass     DuplicateRecordFields@@ -35,8 +41,8 @@     OverloadedStrings  library-  import:          common-options-  hs-source-dirs:  src+  import: common-options+  hs-source-dirs: src   exposed-modules:     Shikumi.Compile     Shikumi.Compile.ChainOfThought@@ -50,37 +56,40 @@     Shikumi.Compile.ZeroShot    build-depends:-    , aeson         >=2.2      && <2.3-    , base          >=4.20     && <5-    , bytestring    >=0.11     && <0.13-    , effectful     >=2.5      && <2.7-    , generic-lens  >=2.2      && <2.4-    , lens          ^>=5.3-    , shikumi       ^>=0.4.0.0-    , text          ^>=2.1+    aeson >=2.2 && <2.3,+    base >=4.20 && <5,+    bytestring >=0.11 && <0.13,+    effectful >=2.6 && <2.8,+    generic-lens >=2.2 && <2.4,+    lens ^>=5.3,+    shikumi ^>=0.4.0.0,+    text ^>=2.1,  test-suite shikumi-compile-test-  import:         common-options-  type:           exitcode-stdio-1.0+  import: common-options+  type: exitcode-stdio-1.0   hs-source-dirs: test-  main-is:        Main.hs-  ghc-options:    -threaded -with-rtsopts=-N+  main-is: Main.hs+  ghc-options:+    -threaded+    -with-rtsopts=-N+   other-modules:     StructureSpec     Test.Capture     Test.Fixtures    build-depends:-    , aeson-    , baikai           >=0.7.0.0  && <0.8-    , base-    , bytestring-    , effectful-    , generic-lens-    , lens-    , shikumi          ^>=0.4.0.0-    , shikumi-compile  ^>=0.2.1.0-    , tasty-    , tasty-hunit-    , text-    , vector+    aeson,+    baikai >=0.7.1.0 && <0.8,+    base,+    bytestring,+    effectful,+    generic-lens,+    lens,+    shikumi ^>=0.4.0.0,+    shikumi-compile ^>=0.2.1.1,+    tasty,+    tasty-hunit,+    text,+    vector,

@@ -5,23 +5,23 @@ -- before answering. -- -- __Implementation: a structural rewrite (the plan's preferred form).__ EP-4 has--- no @reasoning@ flag on 'Shikumi.Program.Params' to flip; instead, reasoning is a+-- no @reasoning@ flag on t'Shikumi.Program.Params' to flip; instead, reasoning is a -- /structural/ property — 'Shikumi.Module.chainOfThought' augments a signature's -- output with a leading @reasoning@ field and amends its instruction to "think step -- by step", then projects the answer back out with @FMap@. So this compiler walks -- the program and replaces each @Predict sig ps@ leaf with exactly that shape:--- @FMap value (chainOfThoughtRaw sig)@, carrying the node's existing 'Params'+-- @FMap value (chainOfThoughtRaw sig)@, carrying the node's existing t'Shikumi.Program.Params' -- across. The rewrite is type-preserving (a @Program i o@ stays a @Program i o@) -- because the augmented @Program i (WithReasoning o)@ is mapped back through -- 'Shikumi.Module.value'. ----- This is feasible — and faithful — only because EP-4 /exports the GADT--- constructors/, so a downstream package can pattern-match and rebuild nodes. EP-4--- does not ship a node-level @rewriteNodes@ helper, so the recursion is spelled out--- here over every constructor (the EP-4/EP-5 rule: every function over the GADT--- must match every constructor).+-- This is feasible — and faithful — only because EP-4+-- /exports the GADT constructors/, so a downstream package can pattern-match and+-- rebuild nodes. EP-4 does not ship a node-level @rewriteNodes@ helper, so the+-- recursion is spelled out here over every constructor (the EP-4/EP-5 rule: every+-- function over the GADT must match every constructor). ----- __Caveat (recorded in the plan's Decision Log).__ The node's existing 'Params'+-- __Caveat (recorded in the plan's Decision Log).__ The node's existing t'Shikumi.Program.Params' -- are preserved verbatim. For the common case — compiling a base (uncompiled) -- program whose nodes carry 'Shikumi.Program.emptyParams' — this reproduces -- 'Shikumi.Module.chainOfThought' exactly. If a node already carries an@@ -73,8 +73,8 @@  -- | The structural rewrite, polymorphic in @i@/@o@ so it threads through the GADT's -- existential children. At a 'Predict' leaf the constructor's captured dictionaries--- ('Shikumi.Schema.FromModel' @i@/@o@, 'Shikumi.Schema.ToSchema' @o@, the--- 'Shikumi.Adapter.ToPrompt's) are exactly what 'chainOfThoughtRaw' needs.+-- ('Shikumi.Schema.FromModel' @i@/@o@, t'Shikumi.Schema.ToSchema' @o@, the+-- t'Shikumi.Adapter.ToPrompt's) are exactly what 'chainOfThoughtRaw' needs. cot :: Program i o -> Program i o cot (PredictCaptured codec sig ps) =   case chainOfThoughtRaw sig of

@@ -2,7 +2,7 @@ -- LM-call node's parameters. This is DSPy's @LabeledFewShot@ and the headline -- compiler of the plan. ----- The injected 'Demo's are stored as type-agnostic JSON (EP-4's+-- The injected t'Demo's are stored as type-agnostic JSON (EP-4's -- 'Shikumi.Program.Demo' carries @input@/@output@ as aeson @Value@s), so a -- single demo pool attaches uniformly to every node regardless of its signature. -- At run time each node's adapter (EP-3) renders the fields it recognizes; a demo@@ -29,7 +29,7 @@ fewShot ds = Compiler $ mapParams (\ps -> ps & #demos .~ ds)  -- | Build a few-shot compiler from typed input/output pairs, serializing each to a--- JSON 'Demo'. This is the recommended path when demos must line up with a node's+-- JSON t'Demo'. This is the recommended path when demos must line up with a node's -- record fields: the JSON keys are the record field names, so the adapter renders -- them precisely. fewShotTyped :: (ToJSON i, ToJSON o) => [(i, o)] -> Compiler

@@ -4,13 +4,13 @@ -- | The retrieval-augmented (RAG) compiler: install retrieved context so that the -- rendered prompt at every LM-call node carries the passages a retriever found. ----- __Purity vs. retrieval (how the plan's question is resolved here).__ 'compile' is+-- __Purity vs. retrieval (how the plan's question is resolved here).__ 'Shikumi.Compile.Types.compile' is -- pure, but retrieval fetches data. EP-4 ships /no/ effectful escape-hatch node -- (no @embed@ / @Embed@ constructor that would let an @i -> Eff es o@ become a -- @Program@ node), so the plan's "approach 1" (install a runtime retrieval step -- keyed on the actual input) is not available. This is the plan's documented -- /fallback/: retrieve at /compile time/ against a fixed sample query, then inject--- the top passages into every node's signature instruction. 'compile' stays pure+-- the top passages into every node's signature instruction. 'Shikumi.Compile.Types.compile' stays pure -- because the trivial 'Shikumi.Compile.Retriever.inMemoryRetriever' performs no -- effect — it is run via 'runPureEff'. The limitation is that retrieval is -- query-independent of the actual program input; wiring true per-input retrieval@@ -22,8 +22,8 @@ -- signature's base instruction) plus the retrieved context. This means RAG state -- survives 'Shikumi.Compile.Serialize.encodeCompiled' / -- 'Shikumi.Compile.Serialize.decodeCompiledOnto'. Composition order still matters:--- applying 'zeroShot' after 'rag' replaces the whole override and drops the--- context, while applying 'rag' after 'zeroShot' appends context to the zero-shot+-- applying 'Shikumi.Compile.ZeroShot.zeroShot' after 'rag' replaces the whole override and drops the+-- context, while applying 'rag' after 'Shikumi.Compile.ZeroShot.zeroShot' appends context to the zero-shot -- instruction. module Shikumi.Compile.RAG   ( rag,

@@ -1,4 +1,4 @@--- | Serialization of a 'CompiledProgram'.+-- | Serialization of a t'CompiledProgram'. -- -- __Parameter-state plus shape fingerprint, not whole-structure.__ A @Program@'s -- executable structure cannot be serialized in general (its @FMap@ nodes hold

@@ -2,16 +2,16 @@  -- | The two types this plan (EP-9) owns and the verbs over them. ----- A 'Compiler' is a /pure/ @Program -> Program@ rewrite that bakes a prompting+-- A t'Compiler' is a /pure/ @Program -> Program@ rewrite that bakes a prompting -- strategy into a program — it sets per-node parameters (zero-shot, few-shot) or -- rewrites structure (chain-of-thought, RAG). It is emphatically /not/ a search: -- it never calls the LM to try variations and keep the best (that is the optimizer,--- @docs/plans/10-optimizer-framework.md@).+-- @docs\/plans\/10-optimizer-framework.md@). ----- A 'CompiledProgram' is the result of compilation — a phantom-marked 'Program'+-- A t'CompiledProgram' is the result of compilation — a phantom-marked t'Program' -- that says "these parameters are intentional, this program is ready to--- run/serialize/optimize". It is MasterPlan integration point #6: the optimizer--- (EP-10) emits one and the CLI (EP-12) loads/runs/saves one.+-- run\/serialize\/optimize". It is MasterPlan integration point #6: the optimizer+-- (EP-10) emits one and the CLI (EP-12) loads\/runs\/saves one. -- -- __Why a newtype, not a side table of frozen parameters.__ EP-4 -- (@Shikumi.Program@) already stores each node's 'Shikumi.Program.Params' /inside/@@ -38,14 +38,14 @@ -- | A pure rewrite of a program. The @forall i o@ is the key design move: a -- compiler must apply to /any/ program regardless of its input/output types -- (few-shot demos are type-agnostic JSON, an instruction string is type-agnostic,--- the chain-of-thought rewrite is uniform), so a single 'Compiler' value+-- the chain-of-thought rewrite is uniform), so a single t'Compiler' value -- ('identity', 'Shikumi.Compile.ZeroShot.zeroShot', …) is usable everywhere. -- @RankNTypes@ (on by default in GHC2024) makes the rank-2 field legal. newtype Compiler = Compiler   { runCompiler :: forall i o. Program i o -> Program i o   } --- | A program that has been through a 'Compiler'. A newtype over 'Program' — the+-- | A program that has been through a t'Compiler'. A newtype over t'Program' — the -- parameters live on the nodes (see the module header). newtype CompiledProgram i o = CompiledProgram   { compiledProgram :: Program i o@@ -58,10 +58,10 @@ compile :: Compiler -> Program i o -> CompiledProgram i o compile (Compiler f) = CompiledProgram . f --- | Run a compiled program. Identical to running the wrapped 'Program', so it+-- | Run a compiled program. Identical to running the wrapped t'Program', so it -- inherits EP-4's @runProgram@ constraint exactly (MasterPlan integration point -- #4): @(LLM :> es, Error ShikumiError :> es)@ — the decode path throws typed--- 'ShikumiError's.+-- t'ShikumiError's. runCompiled ::   (LLM :> es, Error ShikumiError :> es) =>   CompiledProgram i o ->

@@ -15,7 +15,7 @@  -- | Override every node's instruction with @instr@ and remove all demos. Reaches -- every 'Shikumi.Program.Predict' node in the program — including nodes nested--- inside @Compose@/@Parallel@/@Retry@/etc. — because 'mapParams' visits them all.+-- inside @Compose@\/@Parallel@\/@Retry@\/etc. — because 'mapParams' visits them all. zeroShot :: Text -> Compiler zeroShot instr =   Compiler $

@@ -1,6 +1,7 @@--- | The EP-9 acceptance suite. Every assertion is about the /prompt the model would--- have seen/ (captured offline via "Test.Capture") or the /parameters now stored on--- the nodes/ (a pure @foldParams@ read) — never merely "a type was added".+-- | The EP-9 acceptance suite. Every assertion is about the+-- /prompt the model would have seen/ (captured offline via "Test.Capture") or the+-- /parameters now stored on the nodes/ (a pure @foldParams@ read) — never merely+-- "a type was added". module Main (main) where  import Data.Aeson qualified as Aeson

@@ -9,8 +9,8 @@ -- key; the whole suite is a deterministic @cabal test@. -- -- The captured text is the JSON encoding of the whole 'Baikai.Context' (system--- prompt + messages + tools). The instruction/reasoning cue/retrieved context live--- in the system prompt; few-shot demos render as user/assistant /messages/ — both+-- prompt + messages + tools). The instruction\/reasoning cue\/retrieved context live+-- in the system prompt; few-shot demos render as user\/assistant /messages/ — both -- are present in the JSON, so a substring assertion finds either. module Test.Capture   ( runWithCapture,

@@ -5,7 +5,7 @@ -- -- The record instances mirror @ProgramFixtures@ in the @shikumi@ test tree (which -- lives there and is not importable here): each record derives 'Generic' and gets--- empty 'ToSchema' / 'FromModel' / 'ToPrompt' / 'Validatable' bodies (Generic+-- empty 'ToSchema' \/ 'FromModel' \/ 'ToPrompt' \/ 'Validatable' bodies (Generic -- defaults), plus 'ToJSON' / 'FromJSON' so 'Shikumi.Compile.fewShotTyped' can build -- demos from typed pairs. module Test.Fixtures