packages feed

shikumi 0.4.1.0 → 0.4.1.1

raw patch · 21 files changed

+178/−165 lines, 21 filesdep ~baikaidep ~baikai-claudedep ~baikai-effectfulPVP ok

version bump matches the API change (PVP)

Dependency ranges changed: baikai, baikai-claude, baikai-effectful, baikai-openai, shikumi

API changes (from Hackage documentation)

Files

CHANGELOG.md view
@@ -1,5 +1,17 @@ # Changelog +## Unreleased++## 0.4.1.1 — 2026-10-16++### Changed++- Upgrade `mori://shinzui/baikai/packages/baikai` to `>=0.8.0.0 && <0.9`, `mori://shinzui/baikai/packages/baikai-claude` to `>=0.7.1.1 && <0.8`, `mori://shinzui/baikai/packages/baikai-openai` to `>=0.7.1.1 && <0.8`, `mori://shinzui/baikai/packages/baikai-effectful` to `>=0.4.0.3 && <0.5`.++### Fixed++- Correct Haddock markup and identifier links; no runtime or public API changes.+ ## 0.4.1.0 — 2026-09-30  - Route subscription-CLI models through the native schema adapter. `capabilityFor` now starts from Baikai's `declaredStructuredOutput` for the model's `api`: `AnthropicMessagesCli` (`claude -p --json-schema`) and `OpenAICompletionsCli` (`codex exec --output-schema`) are `NativeSchema`, so their requests carry the strict derived schema and a list-of-records output decodes. First-party HTTP routing is unchanged; third-party hosts over a first-party wire format (`deepseek`, `openrouter` over Chat Completions) and `Custom` hosts stay `PromptFallback`. Previously cached CLI responses miss once, because the request now carries a `responseFormat` and the native prompt. Implements `mori://shinzui/shikumi/okf/improvement-requests/concepts/IR-4`.
shikumi.cabal view
@@ -1,6 +1,6 @@ cabal-version: 3.4 name: shikumi-version: 0.4.1.0+version: 0.4.1.1 synopsis: Typed, structured, evaluable LM programs over baikai category: AI description:@@ -63,10 +63,10 @@   other-modules: Shikumi.Adapter.Xml   build-depends:     aeson >=2.2 && <2.3,-    baikai >=0.7.2.0 && <0.8,-    baikai-claude >=0.7.1.0 && <0.8,-    baikai-effectful >=0.4.0.1 && <0.5,-    baikai-openai >=0.7.1.0 && <0.8,+    baikai >=0.8.0.0 && <0.9,+    baikai-claude >=0.7.1.1 && <0.8,+    baikai-effectful >=0.4.0.3 && <0.5,+    baikai-openai >=0.7.1.1 && <0.8,     base >=4.20 && <5,     base64-bytestring >=1.2 && <1.3,     bytestring >=0.11 && <0.13,@@ -128,10 +128,10 @@   build-depends:     QuickCheck,     aeson,-    baikai >=0.7.2.0 && <0.8,-    baikai-claude >=0.7.1.0 && <0.8,-    baikai-effectful >=0.4.0.1 && <0.5,-    baikai-openai >=0.7.1.0 && <0.8,+    baikai >=0.8.0.0 && <0.9,+    baikai-claude >=0.7.1.1 && <0.8,+    baikai-effectful >=0.4.0.3 && <0.5,+    baikai-openai >=0.7.1.1 && <0.8,     base,     base64-bytestring,     bytestring,@@ -140,7 +140,7 @@     effectful,     generic-lens,     lens,-    shikumi ^>=0.4.1.0,+    shikumi ^>=0.4.1.1,     stm,     streamly-core,     tasty,
src/Shikumi/Adapter.hs view
@@ -4,7 +4,7 @@ {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE UndecidableInstances #-} --- | The 'Adapter' seam between a typed 'Signature' + input and the wire.+-- | The t'Adapter' seam between a typed 'Signature' + input and the wire. -- @render@ builds a baikai @Context@+@Options@; @parse@ decodes a baikai -- @Response@ into the typed output via "Shikumi.Schema". --@@ -16,7 +16,7 @@ -- bounded nested decoding and offer legacy or structured demonstration rendering. -- -- Native structured output is wired through a private /metadata channel/ (EP-14).--- Because a 'Program' renders before the ambient model is known (the model is+-- Because a t'Shikumi.Program.Program' renders before the ambient model is known (the model is -- supplied by an interpreter below 'Shikumi.Program.runProgram' — see -- "Shikumi.Routing"), 'render' cannot itself decide native-vs-fallback or build the -- final @responseFormat@. Instead, 'attachSchema' stamps the derived JSON schema@@ -171,7 +171,7 @@ instance (PromptValue a) => PromptValue (Maybe a) where   promptValue = maybe "" promptValue --- | A 'Constrained' field renders as its inner value (the constraint is a+-- | A t'Constrained' field renders as its inner value (the constraint is a -- compile-time/decode-time concern, not a prompt concern). EP-26. instance (PromptValue a) => PromptValue (Constrained cs a) where   promptValue = promptValue . unConstrained@@ -190,7 +190,7 @@ data ModelCapability = NativeSchema | PromptFallback   deriving stock (Eq, Show) --- | A pure capability check over a baikai 'Model', derived from baikai's+-- | A pure capability check over a baikai t'Model', derived from baikai's -- per-transport declaration ('declaredStructuredOutput'). -- -- A model is 'NativeSchema' when baikai declares 'NativeJsonSchema' for its@@ -238,7 +238,7 @@   NativeSchema -> nativeAdapter   PromptFallback -> fallbackAdapter --- | The reserved 'Options.metadata' key under which 'attachSchema' stamps the+-- | The reserved 'Baikai.Options.metadata' key under which 'attachSchema' stamps the -- derived JSON schema 'Value'. "Shikumi.Routing".@routeLLM@ reads it, turns it into -- @Options.responseFormat@ for native-capable models, and strips it before -- transport. Defined here (the lowest module both the adapter and the router share)@@ -246,22 +246,22 @@ metaResponseSchemaKey :: Text metaResponseSchemaKey = "shikumi.responseSchema" --- | The reserved 'Options.metadata' key under which 'stampTemperature' stamps a+-- | The reserved 'Baikai.Options.metadata' key under which 'stampTemperature' stamps a -- per-sample temperature (a JSON number). The router reads it, sets -- @Options.temperature@, and strips it before transport. metaTemperatureKey :: Text metaTemperatureKey = "shikumi.temperature" --- | Reserved 'Options.metadata' key carrying the full native-format system prompt+-- | Reserved 'Baikai.Options.metadata' key carrying the full native-format system prompt -- (instruction + native output guide) as a JSON string. Stamped by -- 'Shikumi.Program.runPredict'; "Shikumi.Routing".@routeLLM@ swaps it into the--- 'Context' for native-capable models and strips it before transport. Because+-- t'Context' for native-capable models and strips it before transport. Because -- @runPredict@ renders model-agnostically (the marker prompt), this carries the -- native alternative so the router can install it once the real model is known. metaNativePromptKey :: Text metaNativePromptKey = "shikumi.native.systemPrompt" --- | Reserved 'Options.metadata' key carrying the native-format demo assistant+-- | Reserved 'Baikai.Options.metadata' key carrying the native-format demo assistant -- turns, in order, as a JSON array of strings. Same lifecycle as -- 'metaNativePromptKey'. metaNativeDemosKey :: Text@@ -279,7 +279,7 @@ -- | Stamp the native-format system prompt and demo assistant turns onto a -- request's private metadata channel, under 'metaNativePromptKey' and -- 'metaNativeDemosKey'. Mirrors 'attachSchema': the router swaps these into the--- 'Context' for native-capable models and strips the keys before transport.+-- t'Context' for native-capable models and strips the keys before transport. attachNativeRender :: Text -> [Text] -> Options -> Options attachNativeRender sys demos opts =   opts@@ -300,7 +300,7 @@  -- | Stamp a per-sample temperature onto a request's private metadata channel under -- 'metaTemperatureKey'. Used by 'Shikumi.Program.runProgram' to thread a--- 'Shikumi.Program.MajorityVote' sample's temperature down to its 'Predict' nodes;+-- 'Shikumi.Program.MajorityVote' sample's temperature down to its 'Shikumi.Program.Predict' nodes; -- the router turns it into @Options.temperature@. stampTemperature :: Double -> Options -> Options stampTemperature t opts =@@ -352,9 +352,9 @@ -- ("Shikumi.Routing".@routeLLM@) picks between the native and fallback wire shapes -- from the model's detected 'ModelCapability' (native vs. prompt-fallback); XML is a -- caller choice, not a detectable capability, so 'adapterFor' never returns it. To--- use it, hold the 'Adapter' value directly and render/parse with it inside an+-- use it, hold the t'Adapter' value directly and render/parse with it inside an -- 'Shikumi.Program.embed' node — the same way 'Shikumi.Module.twoStep' drives--- 'fallbackAdapter' by hand. A per-node adapter selector through the 'Program' GADT+-- 'fallbackAdapter' by hand. A per-node adapter selector through the t'Shikumi.Program.Program' GADT -- was considered and deferred (see the parent MasterPlan's scope), as it is -- disproportionate to the need. xmlAdapter ::@@ -371,7 +371,7 @@     }  -- | Nested XML guides and faithful structured demonstrations (EP-55). Requires--- 'Aeson.ToJSON' for outputs; inputs still use 'ToPrompt'. Supports generated+-- [ToJSON]("Data.Aeson#t:ToJSON") for outputs; inputs still use t'ToPrompt'. Supports generated -- records, arrays, scalars and nullable schemas; other schema forms render an -- escaped JSON fallback. Parsing is shared with 'xmlAdapter'. Opt-in only. nestedXmlAdapter ::@@ -473,8 +473,8 @@  -- | A compact, JSON-like rendering of a derived schema for the fallback guide: -- scalars by type name, string enums as their quoted values joined by @|@,--- arrays as @[<item>, ...]@, objects as @{"k": <shape>, …}@ (keys in @required@--- order, any others sorted after), and nullable values as @<shape> | null@.+-- arrays as @[\<item\>, ...]@, objects as @{"k": \<shape\>, …}@ (keys in @required@+-- order, any others sorted after), and nullable values as @\<shape\> | null@. renderShape :: Value -> Text renderShape v = case nullableInner v of   Just s -> renderShape s <> " | null"
src/Shikumi/Adapter/Xml.hs view
@@ -218,7 +218,7 @@         | otherwise -> escape t       _ -> escape (json value) --- | Guide for generated record/array/scalar/nullable schemas. Other hand-written+-- | Guide for generated record\/array\/scalar\/nullable schemas. Other hand-written -- schema forms use escaped JSON values and are checked by the typed decoder. xmlSchemaGuide :: [Text] -> Value -> Text xmlSchemaGuide names schema =
src/Shikumi/Combinator.hs view
@@ -1,5 +1,5 @@ -- | The ergonomic combinator surface (EP-5): operators and smart constructors--- that assemble small typed 'Program's into larger ones. The runtime behaviour+-- that assemble small typed t'Program's into larger ones. The runtime behaviour -- lives in the GADT constructors owned by "Shikumi.Program" (so every node stays -- runnable, traversable by the optimizer, and serializable); this module is the -- thin, user-facing layer over them.@@ -23,7 +23,7 @@ -- built here runs sequentially under 'Shikumi.Program.runProgram' and -- concurrently (honouring 'mapP' widths) under -- 'Shikumi.Program.runProgramConc'. See the plan's Decision Log for why--- 'Concurrent' is kept off @runProgram@'s constraint (MasterPlan integration+-- 'Effectful.Concurrent.Concurrent' is kept off @runProgram@'s constraint (MasterPlan integration -- point #4). module Shikumi.Combinator   ( -- * Pipeline@@ -83,9 +83,9 @@ -- @pipeline@\/@Compose@; this n-ary helper is restricted to a single carried type -- because the GADT has no identity node to seed an empty fold.) ----- The argument is a 'NonEmpty' — there is no identity 'Program' to seed an empty+-- The argument is a 'NonEmpty' — there is no identity t'Program' to seed an empty -- fold, so emptiness is unrepresentable rather than a runtime crash. Write--- @chain (a ':|' [b, c])@ (or @chain (NE.fromList [a, b, c])@).+-- @chain (a :| [b, c])@ (or @chain (NE.fromList [a, b, c])@). chain :: NonEmpty (Program a a) -> Program a a chain ps = foldr1 (>>>) (NE.toList ps) @@ -122,7 +122,7 @@ -- ---------------------------------------------------------------------------  -- | Re-run a program up to @n@ total attempts, returning the first success or--- the last error. Retries on any 'ShikumiError'.+-- the last error. Retries on any t'ShikumiError'. retry :: Int -> Program i o -> Program i o retry = Retry @@ -159,7 +159,7 @@  -- | Sample a program @K@ times and fold the @K@ (non-empty) outputs with a custom -- reducer, for outputs that are not usefully 'Eq'. Like 'majorityVote' it applies--- the 'TempSchedule' per sample and exposes the sub-program's 'Params' once (not+-- the 'TempSchedule' per sample and exposes the sub-program's t'Shikumi.Program.Params' once (not -- @K@ times) — the samples are runs of a single node, not @K@ distinct 'ensemble' -- members. majorityVoteBy :: Int -> TempSchedule -> ([o] -> o) -> Program i o -> Program i o
src/Shikumi/Error.hs view
@@ -3,7 +3,7 @@ -- -- This module owns part of the MasterPlan's integration point #1: every later -- ExecPlan (signatures, caching, tracing, tools) MUST surface failures through--- 'ShikumiError' rather than inventing its own error type.+-- t'ShikumiError' rather than inventing its own error type. module Shikumi.Error   ( ShikumiError (..),     fromBaikaiError,
src/Shikumi/LLM.hs view
@@ -6,7 +6,7 @@ -- the @baikai-effectful@ package. -- -- The effect exposes two operations ('Complete', 'Stream'). The bare interpreters--- 'runLLM' / 'runLLMWith' map baikai's 'BaikaiError' into 'ShikumiError' and do+-- 'runLLM' / 'runLLMWith' map baikai's 'BaikaiError' into t'ShikumiError' and do -- validate continuation compatibility before transport. The resilient interpreter 'runLLMResilient' adds the production -- features baikai deliberately omits: retries with exponential backoff, an -- in-flight rate limit, and a US-dollar budget ceiling.@@ -93,9 +93,9 @@ -- Stream-error contract (all shikumi interpreters): the returned event list -- never terminates with an @EventError@. A provider failure — which the -- policy-free @Baikai@ transport surfaces in-band as a terminal @EventError@ — is--- converted to an out-of-band 'ShikumiError' thrown through @Error ShikumiError@,+-- converted to an out-of-band t'ShikumiError' thrown through @Error ShikumiError@, -- so 'stream' failures are transient-retryable and reported exactly like--- 'complete' failures.+-- 'Shikumi.LLM.complete' failures. data LLM :: Effect where   Complete :: Model -> Context -> Options -> LLM m Response   Stream :: Model -> Context -> Options -> LLM m [AssistantMessageEvent]@@ -116,7 +116,7 @@ -- ---------------------------------------------------------------------------  -- | Bare interpreter over baikai's process-global registry. Maps 'BaikaiError'--- into 'ShikumiError' and adds no policy. Use 'runLLMWith' for an isolated+-- into t'ShikumiError' and adds no policy. Use 'runLLMWith' for an isolated -- registry (tests do this). runLLM ::   (IOE :> es, Error ShikumiError :> es) =>@@ -142,7 +142,7 @@ -- in-band failure (an error-shaped 'Response') through 'raiseResponseError' — plus -- a defensive 'try' for any residual thrown 'BaikaiError' — and remaps it; the -- streaming path collects the events and, via 'raiseStreamError', converts a--- terminal @EventError@ into the same out-of-band 'ShikumiError' — so callers of+-- terminal @EventError@ into the same out-of-band t'ShikumiError' — so callers of -- 'stream' never receive an in-band error terminal and resilience/decoding treat -- both operations identically. bareHandler ::@@ -237,7 +237,7 @@ defaultRetryPolicy = RetryPolicy {maxAttempts = 3, baseDelayMs = 200, maxDelayMs = 5000}  -- | A simple in-flight rate limiter: a counter of available permits. Build it--- once with 'newRateLimiter' and store it in the 'LLMConfig' (not per call).+-- once with 'newRateLimiter' and store it in the t'LLMConfig' (not per call). newtype RateLimiter = RateLimiter (TVar Int)  -- | Create a rate limiter that allows at most @n@ concurrent calls.@@ -358,11 +358,11 @@   (tp : _) -> throwError (streamTerminalError tp)   [] -> pure evs --- | Enforce the blocking-error posture, the 'complete' analogue of+-- | Enforce the blocking-error posture, the 'Shikumi.LLM.complete' analogue of -- 'raiseStreamError'. Since baikai 0.3, 'BE.complete' does not throw on--- provider/registry/CLI failure: it returns an error-shaped 'Response' whose+-- provider\/registry\/CLI failure: it returns an error-shaped 'Response' whose -- 'responseError' is populated. Convert that in-band failure into the same--- out-of-band 'ShikumiError' the rest of shikumi consumes, so an error response+-- out-of-band t'ShikumiError' the rest of shikumi consumes, so an error response -- never masquerades as success and 'runLLMResilient' still retries transient -- failures (a classified error thrown /inside/ the retry loop). A success -- passes through unchanged.@@ -372,7 +372,7 @@   Just be -> throwError (fromBaikaiError be)   Nothing -> pure resp --- | Map a terminal 'EventError' payload to a 'ShikumiError'.+-- | Map a terminal 'EventError' payload to a t'ShikumiError'. streamTerminalError :: TerminalPayload -> ShikumiError streamTerminalError tp = case tp ^. #errorInfo of   Just err -> fromBaikaiError err
src/Shikumi/Module.hs view
@@ -3,9 +3,9 @@ {-# LANGUAGE ScopedTypeVariables #-}  -- | The two foundational /modules/ of shikumi (EP-4): @predict@ and--- @chainOfThought@. These are ordinary functions that build 'Program' values out+-- @chainOfThought@. These are ordinary functions that build t'Program' values out -- of the three core constructors — not new constructors. This is the pattern the--- combinators in @docs/plans/5-module-combinators-and-control-flow.md@ follow.+-- combinators in @docs\/plans\/5-module-combinators-and-control-flow.md@ follow. -- -- @chainOfThought@ extends the output signature with a leading @reasoning@ field -- (DSPy's @ChainOfThought@), producing a @Program i (WithReasoning o)@, then@@ -14,7 +14,7 @@ -- node, so its instruction and demos are visible to @paramsTraversal@ like any -- other node — the optimizer tunes a chain-of-thought node with no special casing. ----- @WithReasoning@'s schema/decode/prompt instances are hand-written rather than+-- @WithReasoning@'s schema\/decode\/prompt instances are hand-written rather than -- @Generic@-derived: its @value@ field is polymorphic in @o@, and the schema -- classes' overlappable per-field instances cannot be resolved for an abstract -- type variable. EP-3 exposes no @withReasoningField@, so the (nested) augmentation@@ -112,12 +112,12 @@         <*> getField path "value" o     _ -> Left (SchemaMismatch (renderPath path <> ": expected object")) --- Render without leaning on @o@'s @Show@: reuse @o@'s own 'ToPrompt'.+-- Render without leaning on @o@'s @Show@: reuse @o@'s own t'ToPrompt'. instance (ToPrompt o) => ToPrompt (WithReasoning o) where   toPromptFields wr = ("reasoning", reasoning wr) : toPromptFields (value wr)    -- @value@ is polymorphic in @o@, so the generic image walk cannot resolve; a-  -- 'WithReasoning' is a text-only output wrapper and carries no image fields.+  -- t'WithReasoning' is a text-only output wrapper and carries no image fields.   imageFields _ = []   imageFieldNames _ = [] @@ -151,7 +151,7 @@ -- | Augment a signature's output with a leading @reasoning@ field and amend its -- instruction to ask for step-by-step reasoning before the answer. The input -- field metadata is carried over from the source signature; the output metadata is--- the two fields of 'WithReasoning' (matching its hand-written schema).+-- the two fields of t'WithReasoning' (matching its hand-written schema). -- -- Note: signature-level demos are /not/ carried onto the reasoning-augmented -- node (@demos = []@), because a @Demo i o@ cannot be turned into a
src/Shikumi/Multimodal.hs view
@@ -8,12 +8,12 @@ -- | Multimodal /input/ field types for signatures (EP-24). -- -- Shikumi's V1 is text-in, text-out: every input field renders to prompt text.--- This module adds a typed 'Image' field that lowers to baikai's native inline+-- This module adds a typed t'Image' field that lowers to baikai's native inline -- image block ('Baikai.Content.ImageContent' / 'Baikai.Content.UserImage'), so a -- provider actually /sees/ the picture instead of receiving base64 buried in the -- running prose. ----- 'Image' stores /decoded/ bytes plus a MIME type. base64 is purely a wire+-- t'Image' stores /decoded/ bytes plus a MIME type. base64 is purely a wire -- concern that baikai handles itself (its @ToJSON ImageContent@ base64-encodes -- under a @data@ key), so 'imageToContent' is a direct field copy — no encoding -- happens here.@@ -21,8 +21,8 @@ -- Scope (honesty about provider limits): baikai's @UserContent@ models exactly -- @UserText@ and @UserImage@ today, so this module ships __image only__. Audio and -- document fields are upstream-gated on a new @Baikai.Content@ constructor; see the--- EP-24 plan's "Audio and document: upstream-gated future work" section. An 'Image'--- is __input-only__: it has no 'Shikumi.Schema.ToSchema' instance, so putting one in+-- EP-24 plan's "Audio and document: upstream-gated future work" section. An t'Image'+-- is __input-only__: it has no t'Shikumi.Schema.ToSchema' instance, so putting one in -- an /output/ record is a clean compile error (a model cannot emit raw image bytes -- through the structured-decode path). module Shikumi.Multimodal@@ -35,7 +35,7 @@     -- * Lowering to baikai     imageToContent, -    -- * Generic image-field discovery (drives "Shikumi.Adapter"\'s 'Shikumi.Adapter.ToPrompt' image methods)+    -- * Generic image-field discovery (drives "Shikumi.Adapter"\'s t'Shikumi.Adapter.ToPrompt' image methods)     GImageFields (..),     GImageFieldNames (..),   )@@ -86,7 +86,7 @@   imageFromBytes mime     <$> first (\e -> InvalidJSON ("image base64 decode: " <> T.pack e)) (Base64.decode (encodeUtf8 b64)) --- | Lower an 'Image' into baikai's wire image block. Bytes pass through decoded;+-- | Lower an t'Image' into baikai's wire image block. Bytes pass through decoded; -- baikai base64-encodes them only when serialising to the wire. imageToContent :: Image -> ImageContent imageToContent img = ImageContent {imageData = bytes img, mimeType = mime img}@@ -106,18 +106,18 @@ -- Generic image-field discovery -- --------------------------------------------------------------------------- ----- These generic walks back the 'Shikumi.Adapter.ToPrompt' image methods+-- These generic walks back the t'Shikumi.Adapter.ToPrompt' image methods -- ('Shikumi.Adapter.imageFields' / 'Shikumi.Adapter.imageFieldNames'): the former--- collects an input record's 'Image' values (so the adapter lowers them to baikai+-- collects an input record's t'Image' values (so the adapter lowers them to baikai -- 'Baikai.Content.UserImage' blocks), the latter names those fields (so the text -- renderer drops them from the prompt body). A record with no image fields yields -- @[]@ from both, which is exactly the regression-safe text-only path. The methods--- live on 'Shikumi.Adapter.ToPrompt' (not a separate class) so that the single--- @ToPrompt i@ constraint already threaded through @predict@/@Predict@/@adapterFor@+-- live on t'Shikumi.Adapter.ToPrompt' (not a separate class) so that the single+-- @ToPrompt i@ constraint already threaded through @predict@\/@Predict@\/@adapterFor@ -- carries them with no new constraint anywhere; the generic defaults mean any -- @Generic@ input record gets both for free. --- | A leaf field's images: a bare 'Image' is one image, a 'Field'-wrapped 'Image'+-- | A leaf field's images: a bare t'Image' is one image, a t'Field'-wrapped t'Image' -- unwraps, and every other leaf type contributes none. This single check drives -- both the value collector and the field-name collector. class ImageLeaf t where@@ -132,7 +132,7 @@ instance (ImageLeaf a) => ImageLeaf (Field d a) where   imageLeaf (Field a) = imageLeaf a --- | Collect the 'Image' values of a record's generic representation, in field+-- | Collect the t'Image' values of a record's generic representation, in field -- order. class GImageFields (f :: Type -> Type) where   gImageFields :: f p -> [Image]@@ -158,7 +158,7 @@   gImageFields _ = []  -- | Collect the /names/ of a record's image fields, in field order. Emits a--- selector's name iff its field type is an 'Image' (bare or 'Field'-wrapped).+-- selector's name iff its field type is an t'Image' (bare or t'Field'-wrapped). class GImageFieldNames (f :: Type -> Type) where   gImageFieldNames :: f p -> [Text]
src/Shikumi/Program.hs view
@@ -3,30 +3,30 @@ {-# LANGUAGE TypeApplications #-}  -- | The keystone of shikumi (EP-4): a typed /deep embedding/ of an LM program as--- inspectable data. A 'Program' @i o@ is a tree of three constructors that can be+-- inspectable data. A t'Program' @i o@ is a tree of three constructors that can be -- done three different things with at once: -- --   * __run__ as a typed function — 'runProgram' interprets the tree as an 'Eff' --     computation that issues @LLM@ calls and returns a typed @o@ (or throws a---     typed 'ShikumiError');---   * __rewritten as data__ — 'paramsTraversal' / 'foldParams' / 'mapParams' /---     'mapParamsAt' read and replace each node's optimizable 'Params' (its+--     typed t'ShikumiError');+--   * __rewritten as data__ — 'paramsTraversal' \/ 'foldParams' \/ 'mapParams' \/+--     'mapParamsAt' read and replace each node's optimizable t'Params' (its --     instruction override and few-shot demos) without running the program and --     without runtime reflection, which is what the optimizer---     (@docs/plans/10-optimizer-framework.md@) needs;+--     (@docs\/plans\/10-optimizer-framework.md@) needs; --   * __serialized__ — 'programShape' captures the closure-free structure and --     'programParams' / 'setProgramParams' move the JSON-serializable parameter --     vector, so an optimized program's state can be saved and replayed. -- -- The constructor set is deliberately minimal (three): richer modules -- (@chainOfThought@; the combinators in--- @docs/plans/5-module-combinators-and-control-flow.md@) are /derived/ functions+-- @docs\/plans\/5-module-combinators-and-control-flow.md@) are /derived/ functions -- that build these constructors, not new constructors. -- -- This module consumes EP-1 (@Shikumi.LLM@, @Shikumi.Error@) and EP-3 -- (@Shikumi.Signature@, @Shikumi.Adapter@, @Shikumi.Schema@). See the plan's -- Decision Log for the reconciliations with the delivered EP-3 surface (which--- exposes @render@/@parse@/@adapterFor@ rather than a single @runSignature@).+-- exposes @render@\/@parse@\/@adapterFor@ rather than a single @runSignature@). module Shikumi.Program   ( -- * The representation     Program@@ -125,9 +125,9 @@ -- | The optimizable overlay of a single node: an optional instruction override -- (@Nothing@ = use the signature's default) and an ordered list of few-shot -- demonstrations. This is the /uniform, serializable/ handle the compiler--- (@docs/plans/9-compiler-layer.md@) and optimizer--- (@docs/plans/10-optimizer-framework.md@) manipulate regardless of a node's--- @i@/@o@ — hence demos are stored as type-agnostic JSON (see 'Demo').+-- (@docs\/plans\/9-compiler-layer.md@) and optimizer+-- (@docs\/plans\/10-optimizer-framework.md@) manipulate regardless of a node's+-- @i@/@o@ — hence demos are stored as type-agnostic JSON (see t'Demo'). data Params = Params   { instructionOverride :: !(Maybe Text),     demos :: ![Demo]@@ -141,7 +141,7 @@ -- | A worked input/output example, stored as JSON so it is uniform across nodes -- of differing types. At run time each demo is decoded back into the node's typed -- @Sig.Demo i o@ and spliced into the prompt by EP-3's adapter; a demo whose JSON--- does not decode surfaces as a 'ShikumiError'.+-- does not decode surfaces as a t'ShikumiError'. data Demo = Demo   { input :: !Value,     output :: !Value@@ -182,7 +182,7 @@ -- ---------------------------------------------------------------------------  -- | A typed LM program. 'Predict' is a single signature-backed LM call carrying--- its 'Params'; 'Compose' sequences two programs (its intermediate type is+-- its t'Params'; 'Compose' sequences two programs (its intermediate type is -- existential); 'FMap' applies a pure post-processing function (no LM call). -- -- 'Predict' captures the adapter/decode dictionaries existentially so that@@ -214,7 +214,7 @@   Map :: Int -> Program a b -> Program [a] [b]   -- | Run two programs on the /same/ input and pair their outputs.   Parallel :: Program i a -> Program i b -> Program i (a, b)-  -- | Re-run a program up to @n@ total attempts on any 'ShikumiError'.+  -- | Re-run a program up to @n@ total attempts on any t'ShikumiError'.   Retry :: Int -> Program i o -> Program i o   -- | Like 'Retry' but only retries errors satisfying the predicate; a   -- non-matching error propagates after a single attempt.@@ -229,7 +229,7 @@   -- reducer. Each sample is run with its 'TempSchedule' temperature applied to   -- the wire (see 'TempSchedule'). The reducer is opaque to the parameter   -- traversal and to 'ProgramShape' (omitted, exactly like 'Ensemble'\'s), so a-  -- @MajorityVote@ exposes the sub-program's 'Params' once. The @Eq o@ that+  -- @MajorityVote@ exposes the sub-program's t'Params' once. The @Eq o@ that   -- 'modal' needs now lives on the smart constructor, not the GADT.   MajorityVote :: Int -> TempSchedule -> (NonEmpty o -> o) -> Program i o -> Program i o   -- | Run several programs on the same input, collect their (homogeneous)@@ -240,10 +240,10 @@   -- (@LLM@ + @Error ShikumiError@) so an 'Embed' node runs under the ordinary   -- 'runProgram'/'runProgramConc' without widening integration point #4\'s   -- constraint. This is the constructor multi-step agents (ReAct,-  -- @docs/plans/11-typed-tools-and-react-agents.md@) are built on: the agent-  -- loop is one 'Embed' node, so the agent is a real, composable 'Program' that+  -- @docs\/plans\/11-typed-tools-and-react-agents.md@) are built on: the agent+  -- loop is one 'Embed' node, so the agent is a real, composable t'Program' that   -- is runnable, structurally inspectable ('ShapeEmbed'), and serializable-  -- (it carries no 'Params', like 'FMap'). The body is a closure, so it is+  -- (it carries no t'Params', like 'FMap'). The body is a closure, so it is   -- opaque to the parameter traversal — exactly as 'FMap'\'s function is.   Embed :: (forall es. (LLM :> es, Error ShikumiError :> es) => i -> Eff es o) -> Program i o @@ -253,9 +253,9 @@ pipeline :: Program a b -> Program b c -> Program a c pipeline = Compose --- | Embed an opaque effectful step as a 'Program' node. The smart constructor for+-- | Embed an opaque effectful step as a t'Program' node. The smart constructor for -- 'Embed': lift an @(i -> Eff es o)@ — runnable in 'runProgram'\'s effect row — into--- a @Program i o@. Used by ReAct (@docs/plans/11-typed-tools-and-react-agents.md@) to+-- a @Program i o@. Used by ReAct (@docs\/plans\/11-typed-tools-and-react-agents.md@) to -- make a multi-step agent loop a first-class, composable program. embed :: (forall es. (LLM :> es, Error ShikumiError :> es) => i -> Eff es o) -> Program i o embed = Embed@@ -275,9 +275,9 @@ placeholderModel = emptyModel  -- | Interpret a program as a typed @Eff@ computation. A 'Predict' node overlays--- its 'Params' onto the signature (effective instruction + decoded demos), renders--- the request via EP-3's adapter, issues the 'LLM' call, and parses the response--- back into a typed @o@ — throwing a 'ShikumiError' on a parse or demo-decode+-- its t'Params' onto the signature (effective instruction + decoded demos), renders+-- the request via EP-3's adapter, issues the t'LLM' call, and parses the response+-- back into a typed @o@ — throwing a t'ShikumiError' on a parse or demo-decode -- failure. 'Compose' threads the intermediate value; 'FMap' maps the result purely. runProgram ::   (LLM :> es, Error ShikumiError :> es) =>@@ -326,8 +326,8 @@ -- internally via its own 'Concurrent' handler. runProgramConc (Embed f) i = f i --- | Interpret a single 'Predict' node: overlay its 'Params', render via EP-3's--- adapter, issue the 'LLM' call, parse back to a typed @o@. Shared by both+-- | Interpret a single 'Predict' node: overlay its t'Params', render via EP-3's+-- adapter, issue the t'LLM' call, parse back to a typed @o@. Shared by both -- executors so the wire behaviour is defined once. runPredict ::   forall i o es.@@ -462,13 +462,13 @@     pickBest (z : zs) =       fst (foldl' (\best cur -> if snd cur > snd best then cur else best) z zs) --- | Overlay a node's 'Params' onto its signature: substitute the instruction+-- | Overlay a node's t'Params' onto its signature: substitute the instruction -- override (when present) and decode the JSON demos into the signature's typed -- demo channel. A demo whose JSON does not decode is reported as the located--- 'ShikumiError' from 'fromModel'.+-- t'ShikumiError' from 'fromModel'. -- -- Exported as an execution internal: "Shikumi.Stream".@streamPredict@ reuses it--- (with 'parseResponse') so the streamed and blocking paths overlay 'Params' and+-- (with 'parseResponse') so the streamed and blocking paths overlay t'Params' and -- decode the reply through one definition, not two divergent copies. effectiveSignature ::   (FromModel i, FromModel o, Error ShikumiError :> es) =>@@ -486,11 +486,11 @@ -- Parameter interface -- --------------------------------------------------------------------------- --- | The source-of-truth traversal: focuses every 'Params' in a program in+-- | The source-of-truth traversal: focuses every t'Params' in a program in -- /left-to-right depth-first/ order (for @Compose f g@ all of @f@'s come before--- @g@'s). Composite nodes ('Compose', 'FMap') carry no 'Params' of their own — so+-- @g@'s). Composite nodes ('Compose', 'FMap') carry no t'Params' of their own — so -- a program's parameter count equals its number of 'Predict' nodes. Obeys the--- @lens@ @Traversal'@ laws; use it directly with @toListOf@/@over@/@set@.+-- @lens@ @Traversal'@ laws; use it directly with @toListOf@\/@over@\/@set@. paramsTraversal :: (Applicative f) => (Params -> f Params) -> Program i o -> f (Program i o) paramsTraversal h (Predict sig ps) = Predict sig <$> h ps paramsTraversal h (PredictCaptured codec sig ps) = PredictCaptured codec sig <$> h ps@@ -503,11 +503,11 @@ paramsTraversal h (Validate v p) = Validate v <$> paramsTraversal h p paramsTraversal h (MajorityVote k sched reduce p) = MajorityVote k sched reduce <$> paramsTraversal h p paramsTraversal h (Ensemble ps reduce) = Ensemble <$> traverse (paramsTraversal h) ps <*> pure reduce--- 'Embed' carries no 'Params' (its body is an opaque closure, like 'FMap'\'s+-- 'Embed' carries no t'Params' (its body is an opaque closure, like 'FMap'\'s -- function), so it is a traversal leaf — preserved untouched. paramsTraversal _ (Embed f) = pure (Embed f) --- | Read every node's 'Params', in traversal order.+-- | Read every node's t'Params', in traversal order. foldParams :: Program i o -> [Params] foldParams = getConst . paramsTraversal (\ps -> Const [ps]) @@ -515,7 +515,7 @@ -- structurally. A 'Predict' hides its @i@\/@o@ types existentially, so a typed -- @Signature@ cannot escape the GADT — but the field /names/ are plain 'Text' and -- can. This is what the EP-16 node-correlated trace and the grounded instruction--- proposer (@docs/plans/19-grounded-instruction-proposer.md@) consume.+-- proposer (@docs\/plans\/19-grounded-instruction-proposer.md@) consume. data NodeFields = NodeFields   { inputFieldNames :: ![Text],     outputFieldNames :: ![Text]@@ -548,7 +548,7 @@ -- Index-aligned with 'nodeFieldsIndexed' and 'foldParams': entry @n@ describes the -- same node those functions address, so the two lists can be zipped. Composite -- nodes carry no instruction and 'Embed' is opaque, so neither contributes an--- entry. This reads the /signature's/ base instruction (not any per-node 'Params'+-- entry. This reads the /signature's/ base instruction (not any per-node t'Params' -- instruction override), which is the stable task description suitable for -- documentation (consumed by @shikumi-okf@'s program-doc renderer). nodeInstructionsIndexed :: Program i o -> [Text]@@ -568,11 +568,11 @@     go (Ensemble ps _) = concatMap go ps     go (Embed _) = [] --- | Apply a function to every node's 'Params', preserving structure and types.+-- | Apply a function to every node's t'Params', preserving structure and types. mapParams :: (Params -> Params) -> Program i o -> Program i o mapParams f = runIdentity . paramsTraversal (Identity . f) --- | Apply a function to the 'Params' at a single 0-based index in traversal order;+-- | Apply a function to the t'Params' at a single 0-based index in traversal order; -- an out-of-range index leaves the program unchanged. The optimizer's primary edit -- primitive: "replace node @n@'s instruction/demos". The index it addresses is the -- same index 'foldParams' produces (the ordering law).@@ -686,14 +686,14 @@ sigLabel sig = T.intercalate "," (map fieldName (outputFields sig))  -- | The ordered parameter vector, in 'foldParams' order — JSON-serializable--- because 'Params'/'Demo' are. Saving an optimized program = write+-- because t'Params'/t'Demo' are. Saving an optimized program = write -- @(programShape p, programParams p)@; loading = read the @[Params]@, reconstruct -- @p@ in code, then 'setProgramParams'. programParams :: Program i o -> [Params] programParams = foldParams  -- | Apply a saved parameter vector onto a program of the matching shape, replacing--- each node's 'Params' in 'foldParams' order. The vector must have exactly one+-- each node's t'Params' in 'foldParams' order. The vector must have exactly one -- entry per 'Predict' node; a length mismatch is a 'ParamCountMismatch' 'Left'. setProgramParams :: [Params] -> Program i o -> Either ProgramShapeError (Program i o) setProgramParams ps prog
src/Shikumi/Refine.hs view
@@ -4,21 +4,21 @@ {-# LANGUAGE TypeApplications #-}  -- | Reward-driven self-refinement modules (EP-18): three ways to wrap an existing--- 'Program' so its re-runs are /steered by how good the answer is/.+-- t'Program' so its re-runs are /steered by how good the answer is/. -- --   * 'bestOfN' — run the inner program @N@ times at spread temperatures, score---     each output with a 'Reward', and keep the best (short-circuiting on a pass).+--     each output with a t'Reward', and keep the best (short-circuiting on a pass). --   * 'refine' — run the inner program; on a sub-threshold output, ask an LM to --     write a textual critique ("advice") and feed it into the next attempt. --   * 'multiChainComparison' — run @M@ reasoning attempts, then make one final LM --     call shown all @M@ candidates and asked to synthesize a consensus answer. ----- Each module is itself an ordinary 'Program' built from the existing 'embed'+-- Each module is itself an ordinary t'Program' built from the existing 'embed' -- ('Shikumi.Program.Embed') leaf — /no new GADT constructor/ — so it runs under the--- unchanged 'runProgram'/'runProgramConc', composes with every combinator, holds no+-- unchanged 'runProgram'/'Shikumi.Program.runProgramConc', composes with every combinator, holds no -- optimizable 'Shikumi.Program.Params' of its own (it serializes as -- 'Shikumi.Program.ShapeEmbed', exactly like @FMap@), and needs no edit to--- @paramsTraversal@/@programShape@/@setProgramParams@. This is the same pattern V1's+-- @paramsTraversal@\/@programShape@\/@setProgramParams@. This is the same pattern V1's -- ReAct agent uses. The cost, accepted: the inner program's per-node params are not -- visible to an optimizer /through/ the wrapper (the @Embed@ body is opaque) — fine -- for inference-time selection\/retry modules.@@ -94,7 +94,7 @@ defaultSpread :: TempSchedule defaultSpread = TempSpread 0.7 0.6 --- | Run an effect that may throw a 'ShikumiError', recovering the error as a+-- | Run an effect that may throw a t'ShikumiError', recovering the error as a -- 'Left' instead of propagating it (so a single failed attempt does not abort the -- whole loop). tryShikumi :: (Error ShikumiError :> es) => Eff es a -> Eff es (Either ShikumiError a)@@ -302,7 +302,7 @@ -- | The synthesis input: the original input plus the @M@ candidate -- @(reasoning, answer)@ attempts. Its instances are hand-written (not -- @Generic@-derived) because @attempts@ is polymorphic in @o@ — the same reason--- 'WithReasoning'\'s instances are hand-written.+-- t'WithReasoning'\'s instances are hand-written. data MultiChainInput i o = MultiChainInput   { original :: !i,     -- | length @M@; each rendered as "Student Attempt #k"@@ -310,7 +310,7 @@   }  -- | Render the original input, then each attempt as a "Student Attempt #k" line:--- @"I tried to <reasoning>; my answer is <answer>"@ (mirroring DSPy's rendering).+-- @"I tried to \<reasoning\>; my answer is \<answer\>"@ (mirroring DSPy's rendering). instance (ToPrompt i, ToPrompt o) => ToPrompt (MultiChainInput i o) where   toPromptFields mci =     toPromptFields (original mci)@@ -325,7 +325,7 @@   imageFields _ = []   imageFieldNames _ = [] --- | A 'MultiChainInput' is only ever an /input/ to the synthesis 'predict' node,+-- | A t'MultiChainInput' is only ever an /input/ to the synthesis 'predict' node, -- so this decoder exists to satisfy the 'Shikumi.Program.Predict' constraint (used -- when decoding demos, of which the synthesis node has none). instance (FromModel i, FromModel o) => FromModel (MultiChainInput i o) where@@ -345,7 +345,7 @@   Just v -> fromModelP (pushField nm path) v  -- | Build a synthesis signature for 'multiChainComparison'. The input is rendered--- via 'MultiChainInput'\'s 'ToPrompt' (so the input-field metadata is empty); the+-- via t'MultiChainInput'\'s t'ToPrompt' (so the input-field metadata is empty); the -- output-field metadata is derived from @o2@ so the output guide and demo -- rendering name the right fields. multiChainSig ::
src/Shikumi/Reward.hs view
@@ -2,21 +2,21 @@ -- function that scores a single program output, plus the smart constructors used -- to build one. ----- A 'Reward' returns a plain 'Double' (higher is better; by convention in @[0, 1]@+-- A t'Reward' returns a plain 'Double' (higher is better; by convention in @[0, 1]@ -- but not clamped — the self-refinement modules only ever /compare/ rewards, so any -- totally-ordered range works). It lives in the base @shikumi@ package — not in--- @shikumi-eval@ alongside @Score@/@Metric@/@Prediction@ — because the modules that--- consume it ('Shikumi.Refine') are first-class 'Shikumi.Program.Program' values+-- @shikumi-eval@ alongside @Score@\/@Metric@\/@Prediction@ — because the modules that+-- consume it ("Shikumi.Refine") are first-class t'Shikumi.Program.Program' values -- and @Program@ lives here; reusing @shikumi-eval@'s @Metric@ would invert the -- one-way @shikumi-eval -> shikumi@ dependency. A caller who already has a--- @shikumi-eval@ @Metric@ derives a 'Reward' from it with the one-line+-- @shikumi-eval@ @Metric@ derives a t'Reward' from it with the one-line -- @rewardFromMetric@ adapter that lives in @shikumi-eval@ (where both types are in -- scope), not here. -- -- A reward at inference time only ever sees the /one/ output it is scoring (not a -- held-out expected value and not a multi-sample prediction), so the simple -- @o -> Double@ shape is the honest one. EP-22 (GEPA,--- @docs/plans/22-gepa-reflective-optimizer.md@) reuses this same vocabulary for its+-- @docs\/plans\/22-gepa-reflective-optimizer.md@) reuses this same vocabulary for its -- reflective feedback and must not define a parallel reward type. module Shikumi.Reward   ( Reward (..),
src/Shikumi/Routing.hs view
@@ -9,11 +9,11 @@ -- The mechanism is approach (a) from the parent MasterPlan: a re-interpreting -- router installed /below/ 'Shikumi.Program.runProgram' on the existing @LLM@ -- effect — exactly the seam @cachedLLM@ ("Shikumi.Cache") and @tracedLLM@--- ("Shikumi.Trace") already use. 'runProgram' renders each 'Predict' node+-- ("Shikumi.Trace") already use. 'Shikumi.Program.runProgram' renders each 'Shikumi.Program.Predict' node -- model-agnostically against the inert placeholder model and stamps its intentions -- (the derived JSON schema, any per-sample temperature, and the native-format render -- alternative — system prompt + JSON demos) onto the private request-metadata--- channel ('Shikumi.Adapter.attachSchema' / 'Shikumi.Adapter.stampTemperature' /+-- channel ('Shikumi.Adapter.attachSchema' \/ 'Shikumi.Adapter.stampTemperature' \/ -- 'Shikumi.Adapter.attachNativeRender'). 'routeLLM' then reads the /ambient/ model -- supplied by 'runRouting', overwrites the placeholder with it, and calls -- 'translateForWire', which for a native-capable model sets@@ -26,7 +26,7 @@ -- -- Install order (mirroring @runTrace . runKeyedLLM . tracedLLM@): 'runRouting' is -- /outer/ of the real @LLM@ interpreter, which is /outer/ of 'routeLLM' (the--- innermost, closest to 'runProgram'). For example:+-- innermost, closest to 'Shikumi.Program.runProgram'). For example: -- -- @ -- runEff@@ -70,7 +70,7 @@ import Shikumi.LLM.Continuation (validateRequestContinuation)  -- | The ambient model-routing effect. Its single operation reads the model every--- 'Predict' node should dispatch against. It is supplied by an interpreter at the+-- 'Shikumi.Program.Predict' node should dispatch against. It is supplied by an interpreter at the -- bottom of the stack ('runRouting'), exactly as @LLM@ and @Error@ already are, so -- it never appears in 'Shikumi.Program.runProgram'\'s constraint row. data Routing :: Effect where@@ -91,7 +91,7 @@ -- 'Complete' and 'Stream' — is dispatched against the ambient model (overwriting -- the placeholder model the model-agnostic 'Shikumi.Program.runProgram' / -- 'Shikumi.Stream.streamProgram' passes), with the private metadata channel--- translated to real wire options (and the marker 'Context' swapped for the native+-- translated to real wire options (and the marker t'Context' swapped for the native -- one for native-capable models) and stripped. The two operations are rewritten -- identically through the single 'translateForWire', so a streamed call gets the -- same real model id and wire options a blocking call does.@@ -110,11 +110,11 @@  -- | Realize the private request-metadata channel against the real model. For a -- native-capable model: attach the native @responseFormat@ from the stamped--- schema, and swap the marker-format 'Context' (system prompt + demo assistant+-- schema, and swap the marker-format t'Context' (system prompt + demo assistant -- turns) for the stamped native-format alternative. In all cases: set -- @temperature@ when one was stamped, and strip the four rendering keys. -- Continuation expectations remain available for later boundary checks. Fallback-capability models keep the--- marker 'Context' unchanged; non-'Predict' @Complete@ calls (no stamps) are never+-- marker t'Context' unchanged; non-'Shikumi.Program.Predict' @Complete@ calls (no stamps) are never -- rewritten. translateForWire :: Model -> Context -> Options -> (Context, Options) translateForWire m ctx opts =@@ -144,7 +144,7 @@    in (ctx', opts')  -- | Install the native-format system prompt and demo assistant turns into a--- 'Context'. The assistant turns of a 'Predict'-rendered context are exactly the+-- t'Context'. The assistant turns of a 'Shikumi.Program.Predict'-rendered context are exactly the -- demo outputs, in order; each is replaced (via baikai's deterministic 'assistant' -- constructor) with the corresponding stamped native text. Defensive: if the count -- of assistant turns differs from the stamped list length the messages are left
src/Shikumi/Schema.hs view
@@ -7,13 +7,13 @@ {-# LANGUAGE UndecidableInstances #-}  -- | The schema engine: derive a provider JSON Schema from a Haskell record--- ('ToSchema' / 'deriveSchema'), and totally decode a provider JSON reply back+-- ([ToSchema]("Shikumi.Schema#t:ToSchema") / 'deriveSchema'), and totally decode a provider JSON reply back -- into a record ('FromModel' / 'parseOutput'), where every failure becomes a--- precise, located 'Shikumi.Error.ShikumiError'.+-- precise, located t'Shikumi.Error.ShikumiError'. ----- Both directions are hand-rolled on 'GHC.Generics' (the schema we emit is the+-- Both directions are hand-rolled on "GHC.Generics" (the schema we emit is the -- small subset OpenAI/Anthropic structured-output endpoints accept). Per-field--- descriptions flow from the 'Field' wrapper's type-level 'Symbol'.+-- descriptions flow from the t'Field' wrapper's type-level 'Symbol'. module Shikumi.Schema   ( -- * Forward: record -> JSON Schema     ToSchema (..),@@ -123,7 +123,7 @@  -- | Per-field schema and whether the field is required. The general -- (overlappable) case is "required, no description"; 'Maybe' is--- required-but-nullable (see below); 'Field' attaches the description and+-- required-but-nullable (see below); t'Field' attaches the description and -- preserves the inner required-ness. -- -- A 'Maybe' field is /required/ in the JSON Schema and expresses its optionality@@ -146,7 +146,7 @@     let (s, req) = fieldSchema (Proxy @a)      in (withDescription (T.pack (symbolVal (Proxy @d))) s, req) --- | A 'Constrained' field emits its constraints' JSON-Schema keywords onto the+-- | A t'Constrained' field emits its constraints' JSON-Schema keywords onto the -- inner type's schema, preserving the inner required-ness (EP-26). instance forall cs a. (ReflectConstraints cs a, FieldSchema a) => FieldSchema (Constrained cs a) where   fieldSchema _ =@@ -228,7 +228,7 @@ instance (FromModel a) => FromModel (Field d a) where   fromModelP path v = Field <$> fromModelP path v --- | A 'Constrained' field decodes its inner value, then enforces the constraints+-- | A t'Constrained' field decodes its inner value, then enforces the constraints -- (EP-26). A violation becomes a 'ValidationFailure' located at the field, so the -- check composes with any record-level 'Validatable' rule with no change to -- 'fromModelChecked'. (@FromField (Constrained cs a)@ is covered by the@@ -276,7 +276,7 @@      in M1 . K1 <$> fromField (pushField nm path) nm o  -- | Look a field up in the JSON object. The general (overlappable) case requires--- the field; 'Maybe' and a missing key succeed as 'Nothing'; 'Field' delegates.+-- the field; 'Maybe' and a missing key succeed as 'Nothing'; t'Field' delegates. class FromField t where   fromField :: FieldPath -> Text -> Object -> Either ShikumiError t @@ -340,7 +340,7 @@ -- 'Shikumi.Schema.Types.Constrained') to (a) a schema-decorating function that -- inserts the matching JSON-Schema keywords and (b) a runtime validator over the -- decoded leaf value. The vocabulary is a small, closed set--- ('Shikumi.Schema.Types.Constraint'); instances cover the five constructors over+-- ([Constraint]("Shikumi.Schema.Types#t:Constraint")); instances cover the five constructors over -- 'Text' and the numeric leaves. class ReflectConstraints (cs :: [Constraint]) a where   -- | Compose all keyword inserts onto the inner type's schema.@@ -413,7 +413,7 @@ -- @MinVal "abc"@) no longer crashes the process at schema-derivation time; instead -- the schema keyword is silently omitted and every decode of the field fails with a -- located 'ValidationFailure' naming the bad symbol (see the @MinVal@/@MaxVal@--- 'ReflectConstraints' instances).+-- t'ReflectConstraints' instances). parseBound :: forall s. (KnownSymbol s) => Proxy s -> Maybe Sci.Scientific parseBound _ = readMaybe (symbolVal (Proxy @s)) @@ -424,8 +424,8 @@ -- Shared field-metadata traversal (also used by Shikumi.Signature) -- --------------------------------------------------------------------------- --- | Collect 'FieldMeta' (name + optional description) for every record field of a--- type, recovering descriptions from 'Field' wrappers.+-- | Collect t'FieldMeta' (name + optional description) for every record field of a+-- type, recovering descriptions from t'Field' wrappers. fieldMetasOf :: forall a. (GFieldMetas (Rep a)) => [FieldMeta] fieldMetasOf = gFieldMetas (Proxy @(Rep a)) @@ -453,7 +453,7 @@         }     ] --- | The description of a field's type, if it is a 'Field' wrapper.+-- | The description of a field's type, if it is a t'Field' wrapper. class FieldDoc t where   fieldDoc :: Proxy t -> Maybe Text
src/Shikumi/Schema/Types.hs view
@@ -2,12 +2,12 @@ {-# LANGUAGE KindSignatures #-}  -- | Shared vocabulary for the schema engine: the per-field description wrapper--- 'Field', the derived 'FieldMeta' record, JSON-Schema smart constructors (the+-- t'Field', the derived t'FieldMeta' record, JSON-Schema smart constructors (the -- small subset OpenAI/Anthropic structured-output endpoints accept), and the -- field-path breadcrumb used to locate decode errors. -- -- This module is owned by EP-3 (integration point #3). The user-facing error--- type is 'Shikumi.Error.ShikumiError' (owned by EP-1); decode errors render a+-- type is t'Shikumi.Error.ShikumiError' (owned by EP-1); decode errors render a -- 'FieldPath' breadcrumb into that type's @Text@ payloads. module Shikumi.Schema.Types   ( -- * Field description wrapper@@ -56,7 +56,7 @@ import GHC.TypeLits (Nat, Symbol)  -- | A field carrying a compile-time description as a type-level 'Symbol'. A--- single 'GHC.Generics' traversal recovers both the field name (from the record+-- single "GHC.Generics" traversal recovers both the field name (from the record -- selector) and its description (from @desc@), so the two cannot drift. The -- wrapper is opt-in per field: a bare @Text@ field simply has no description. newtype Field (desc :: Symbol) a = Field {unField :: a}@@ -71,7 +71,7 @@ -- string-length rules); the signed/decimal numeric bounds 'MinVal'\/'MaxVal' carry -- the bound as a 'Symbol' (e.g. @MinVal "0"@, @MaxVal "100"@, @MinVal "-3.5"@) so -- both the schema emitter and the validator parse it to a 'Scientific'. Used--- promoted, as the @cs@ of 'Constrained'.+-- promoted, as the @cs@ of t'Constrained'. data Constraint   = -- | string minimum length -> @"minLength"@     MinLen Nat@@ -85,9 +85,9 @@     EnumOneOf [Symbol]  -- | A field value carrying a compile-time list of constraints (EP-26). Parallel to--- 'Field'; the two compose: @Field "desc" (Constrained '[MinLen 10] Text)@. The+-- t'Field'; the two compose: @Field "desc" (Constrained '[MinLen 10] Text)@. The -- reflection of @cs@ to schema keywords and a post-decode validator lives in--- "Shikumi.Schema" ('Shikumi.Schema.ReflectConstraints').+-- "Shikumi.Schema" ([ReflectConstraints]("Shikumi.Schema#t:ReflectConstraints")). newtype Constrained (cs :: [Constraint]) a = Constrained {unConstrained :: a}   deriving stock (Eq, Show) @@ -95,7 +95,7 @@ constrained :: a -> Constrained cs a constrained = Constrained --- | Per-field metadata recovered by the generic walk (used by 'Shikumi.Signature'+-- | Per-field metadata recovered by the generic walk (used by "Shikumi.Signature" -- and the prompt adapters). data FieldMeta = FieldMeta   { fieldName :: !Text,@@ -104,7 +104,7 @@   deriving stock (Eq, Show, Generic)  -- | A breadcrumb trail to a location inside a decoded value, e.g.--- @["bullets", "[2]"]@. Rendered into 'Shikumi.Error.ShikumiError' messages.+-- @["bullets", "[2]"]@. Rendered into t'Shikumi.Error.ShikumiError' messages. type FieldPath = [Text]  -- | Extend a path with a named record field.@@ -147,7 +147,7 @@ boolSchema :: Value boolSchema = object ["type" .= ("boolean" :: Text)] --- | @{"type":"array","items":<items>}@.+-- | @{"type":"array","items":\<items\>}@. arraySchema :: Value -> Value arraySchema items = object ["type" .= ("array" :: Text), "items" .= items] @@ -158,7 +158,7 @@ enumSchema names = object ["type" .= ("string" :: Text), "enum" .= names]  -- | Permit JSON @null@ in addition to the wrapped schema. The @anyOf@ form is--- used (more portable across providers than @"type":["T","null"]@).+-- used (more portable across providers than @"type":[\"T\","null"]@). nullableSchema :: Value -> Value nullableSchema s = object ["anyOf" .= [s, object ["type" .= ("null" :: Text)]]]
src/Shikumi/Signature.hs view
@@ -1,12 +1,12 @@--- | A 'Signature' bundles a task's natural-language instruction, its worked--- demonstrations, and the input/output field metadata derived from the @i@/@o@+-- | A t'Signature' bundles a task's natural-language instruction, its worked+-- demonstrations, and the input\/output field metadata derived from the @i@\/@o@ -- record types. -- -- The @instruction@ and @demos@ are the framework's /optimizable parameters/ -- (integration point #3): they are first-class, replaceable values that the -- compiler (EP-9) and optimizer (EP-10) rewrite to improve quality. The--- @inputFields@/@outputFields@ are /derived metadata/ (not optimizable). Because--- 'Signature' derives 'Generic', the @#instruction@ / @#demos@ @generic-lens@+-- @inputFields@\/@outputFields@ are /derived metadata/ (not optimizable). Because+-- t'Signature' derives 'Generic', the @#instruction@ / @#demos@ @generic-lens@ -- optics are available to downstream plans for free. module Shikumi.Signature   ( Demo (..),
src/Shikumi/Stream.hs view
@@ -3,7 +3,7 @@ {-# LANGUAGE TypeApplications #-}  -- | Program-level streaming (EP-25). A new, additive entry point 'streamProgram'--- runs a 'Program' and delivers a sequence of 'StreamEvent's to a caller-supplied+-- runs a t'Program' and delivers a sequence of 'StreamEvent's to a caller-supplied -- callback as execution proceeds, while still returning the fully-decoded typed -- output — identical to what 'Shikumi.Program.runProgram' would return for the same -- input. 'runProgram'\'s blocking contract (MasterPlan integration point #4) is@@ -18,7 +18,7 @@ --     the fallback adapter. Native whole-JSON field chunking is /not/ promised (a --     JSON blob is only parseable once whole). --   * a __status message__ — a human-readable signal that a phase started or ended---     ("LM call started/finished", "tool started/finished", "node started/finished").+--     ("LM call started\/finished", "tool started\/finished", "node started\/finished"). --     Status brackets every node, including composites; aggregating combinators --     ('Shikumi.Program.Map', 'Shikumi.Program.Parallel', --     'Shikumi.Program.MajorityVote', 'Shikumi.Program.Ensemble') and opaque@@ -178,8 +178,8 @@ -- -- The 'EventError' branch is retained for robustness against third-party @LLM@ -- interpreters, but is /unreachable through shikumi's own interpreters/: they now--- convert a terminal 'EventError' to an out-of-band 'ShikumiError' before the event--- list ever reaches here (see 'Shikumi.LLM' — @raiseStreamError@), so a streamed+-- convert a terminal 'EventError' to an out-of-band t'ShikumiError' before the event+-- list ever reaches here (see "Shikumi.LLM" — @raiseStreamError@), so a streamed -- program fails with the real transport error, not a decode of a partial body. reassemble :: [AssistantMessageEvent] -> Response reassemble evs = case terminalPayloads of@@ -205,10 +205,10 @@ -- program runs under the same interpreters as a blocking one. -- -- A leaf 'Predict' streams its first output field's chunks bracketed by--- @LmStart@/@LmEnd@. Composites are bracketed by @NodeStart@/@NodeEnd@:+-- @LmStart@\/@LmEnd@. Composites are bracketed by @NodeStart@\/@NodeEnd@: -- 'Compose'/'FMap' and the single-result control nodes--- ('Retry'/'RetryWhen'/'Validate') recurse so their leaf predicts still stream;--- the aggregating combinators ('Map'/'Parallel'/'MajorityVote'/'Ensemble') and the+-- ('Retry'\/'RetryWhen'\/'Validate') recurse so their leaf predicts still stream;+-- the aggregating combinators ('Map'\/'Parallel'\/'MajorityVote'\/'Shikumi.Program.Ensemble') and the -- opaque 'Embed' body run blocking and emit node-boundary status only (field chunks -- come from leaf LM calls on the single-result path). streamProgram ::
test/ConstraintSpec.hs view
@@ -58,7 +58,7 @@  instance Validatable BadBound --- | Drill into @properties.<field>@ of a record schema 'Value'.+-- | Drill into @properties.\<field\>@ of a record schema 'Value'. propOf :: Text -> Value -> Maybe Value propOf name (Object o) = do   Object props <- KM.lookup "properties" o
test/ProgramSpec.hs view
@@ -1,5 +1,5 @@ -- | EP-4 M1 (run a node through the LLM effect) and M2 (the parameter interface:--- @foldParams@ / @mapParams@ / @mapParamsAt@ and the ordering law).+-- @foldParams@ \/ @mapParams@ \/ @mapParamsAt@ and the ordering law). module ProgramSpec (tests) where  import Control.Lens ((&), (.~))
test/RefineStub.hs view
@@ -3,10 +3,11 @@ -- | Hermetic, network-free fixtures and deterministic stub @LLM@ interpreters for -- the EP-18 self-refinement spec ('Shikumi.Refine'). ----- The whole point of a self-refinement test is that __a later attempt scores--- better than the first__. So the stub is not a constant: it inspects the rendered--- request and the per-sample temperature ('Options.temperature', made live by the--- router installed above it) and answers by a rule:+-- The whole point of a self-refinement test is that+-- __a later attempt scores better than the first__. So the stub is not a constant:+-- it inspects the rendered request and the per-sample temperature+-- ('Options.temperature', made live by the router installed above it) and answers by+-- a rule: -- --   * a /classify/ call answers a label chosen by the request's temperature --     (@< 0.5@ → @low@, @< 1.0@ → @mid@, else @high@), so 'Shikumi.Refine.bestOfN'
test/XmlAdapterSpec.hs view
@@ -58,7 +58,7 @@       "</note>"     ] --- | The same body with the @<bullets>@ tag omitted (required field missing).+-- | The same body with the @\<bullets\>@ tag omitted (required field missing). xmlBodyNoBullets :: Text xmlBodyNoBullets =   T.intercalate