shikumi-trace 0.3.0.0 → 0.3.0.1
raw patch · 11 files changed
+114/−97 lines, 11 filesdep ~baikaidep ~effectfuldep ~shikumi-cachePVP ok
version bump matches the API change (PVP)
Dependency ranges changed: baikai, effectful, shikumi-cache, shikumi-trace
API changes (from Hackage documentation)
Files
- CHANGELOG.md +4/−0
- shikumi-trace.cabal +69/−57
- src/Shikumi/Trace.hs +5/−5
- src/Shikumi/Trace/Feedback.hs +3/−3
- src/Shikumi/Trace/Internal/Spike.hs +9/−8
- src/Shikumi/Trace/Node.hs +7/−7
- src/Shikumi/Trace/Program.hs +5/−5
- src/Shikumi/Trace/Replay.hs +5/−5
- src/Shikumi/Trace/ResponseJSON.hs +2/−2
- src/Shikumi/Trace/Store.hs +4/−4
- test/Main.hs +1/−1
CHANGELOG.md view
@@ -2,6 +2,10 @@ ## Unreleased +## 0.3.0.1 — 2026-10-05++- Move the dependency on `mori://shinzui/baikai/packages/baikai` to `>=0.7.1.0 && <0.8` and widen the `effectful` bound to `>=2.6 && <2.8`, so both effectful 2.6 and 2.7 are supported (effectful 2.7 needs `baikai-effectful` 0.4.0.2, effectful 2.6 needs 0.4.0.1). Bounds only; no source changed.+ ## 0.3.0.0 — 2026-09-08 - Raise the internal `shikumi` bound to `^>=0.4.0.0` and `shikumi-cache` to `^>=0.2.0.0`.
shikumi-trace.cabal view
@@ -1,10 +1,10 @@-cabal-version: 3.4-name: shikumi-trace-version: 0.3.0.0+cabal-version: 3.4+name: shikumi-trace+version: 0.3.0.1 synopsis: Hierarchical tracing, observability, and deterministic replay for shikumi (EP-7) -category: AI+category: AI description: Hierarchical (nested) tracing and deterministic offline replay for shikumi. Running a program inside the @Trace@ effect produces a /tree/ of spans (program@@ -15,20 +15,26 @@ re-runs the same program with the network disabled, serving every call from the recording (and raising a precise @ReplayDivergence@ for any unrecorded call). -license: BSD-3-Clause-author: Nadeem Bitar-maintainer: nadeem@gmail.com-build-type: Simple+license: BSD-3-Clause+author: Nadeem Bitar+maintainer: nadeem@gmail.com+build-type: Simple extra-doc-files: CHANGELOG.md common common-options ghc-options:- -Wall -Wcompat -Widentities -Wincomplete-uni-patterns- -Wincomplete-record-updates -Wredundant-constraints- -fhide-source-paths -Wmissing-export-lists -Wpartial-fields+ -Wall+ -Wcompat+ -Widentities+ -Wincomplete-uni-patterns+ -Wincomplete-record-updates+ -Wredundant-constraints+ -fhide-source-paths+ -Wmissing-export-lists+ -Wpartial-fields -Wmissing-deriving-strategies - default-language: GHC2024+ default-language: GHC2024 default-extensions: DeriveAnyClass DuplicateRecordFields@@ -36,8 +42,8 @@ OverloadedStrings library- import: common-options- hs-source-dirs: src+ import: common-options+ hs-source-dirs: src exposed-modules: Shikumi.Trace Shikumi.Trace.Demo@@ -51,58 +57,64 @@ Shikumi.Trace.Store build-depends:- , aeson >=2.2 && <2.3- , baikai >=0.7.0.0 && <0.8- , base >=4.20 && <5- , bytestring >=0.11 && <0.13- , containers >=0.6 && <0.9- , directory >=1.3 && <1.4- , effectful >=2.5 && <2.7- , generic-lens >=2.2 && <2.4- , lens ^>=5.3- , scientific >=0.3 && <0.4- , shikumi ^>=0.4.0.0- , shikumi-cache ^>=0.2.0.0- , text ^>=2.1- , time >=1.12 && <1.17- , vector >=0.13 && <0.14+ aeson >=2.2 && <2.3,+ baikai >=0.7.1.0 && <0.8,+ base >=4.20 && <5,+ bytestring >=0.11 && <0.13,+ containers >=0.6 && <0.9,+ directory >=1.3 && <1.4,+ effectful >=2.6 && <2.8,+ generic-lens >=2.2 && <2.4,+ lens ^>=5.3,+ scientific >=0.3 && <0.4,+ shikumi ^>=0.4.0.0,+ shikumi-cache ^>=0.2.0.1,+ text ^>=2.1,+ time >=1.12 && <1.17,+ vector >=0.13 && <0.14, executable shikumi-trace-demo- import: common-options+ import: common-options hs-source-dirs: app- main-is: Main.hs- ghc-options: -threaded -with-rtsopts=-N+ main-is: Main.hs+ ghc-options:+ -threaded+ -with-rtsopts=-N+ build-depends:- , base- , shikumi-trace ^>=0.3.0.0+ base,+ shikumi-trace ^>=0.3.0.1, test-suite shikumi-trace-test- import: common-options- type: exitcode-stdio-1.0+ import: common-options+ type: exitcode-stdio-1.0 hs-source-dirs: test- main-is: Main.hs- ghc-options: -threaded -with-rtsopts=-N+ main-is: Main.hs+ ghc-options:+ -threaded+ -with-rtsopts=-N+ other-modules: ObservationSpec TraceFixtures build-depends:- , aeson- , baikai >=0.7.0.0 && <0.8- , base- , bytestring- , containers- , effectful- , generic-lens- , lens ^>=5.3- , QuickCheck- , shikumi ^>=0.4.0.0- , shikumi-cache ^>=0.2.0.0- , shikumi-trace ^>=0.3.0.0- , tasty- , tasty-hunit- , tasty-quickcheck- , temporary- , text ^>=2.1- , time- , vector+ QuickCheck,+ aeson,+ baikai >=0.7.1.0 && <0.8,+ base,+ bytestring,+ containers,+ effectful,+ generic-lens,+ lens ^>=5.3,+ shikumi ^>=0.4.0.0,+ shikumi-cache ^>=0.2.0.1,+ shikumi-trace ^>=0.3.0.1,+ tasty,+ tasty-hunit,+ tasty-quickcheck,+ temporary,+ text ^>=2.1,+ time,+ vector,
src/Shikumi/Trace.hs view
@@ -15,10 +15,10 @@ -- on each 'Shikumi.LLM.complete' it opens an 'LlmCallSpan' under the active span -- and fills its attributes — model, provider, latency, tokens, cost, tool calls, -- the recorded response JSON, and the EP-6 content-addressed 'Shikumi.Cache.Key.cacheKey'--- (integration point #7) — from the returned 'Baikai.Response'.+-- (integration point #7) — from the returned t'Baikai.Response.Response'. ----- 'renderTree' pretty-prints the tree; 'Shikumi.Trace.Store' serializes it and--- 'Shikumi.Trace.Replay' replays it offline.+-- 'renderTree' pretty-prints the tree; "Shikumi.Trace.Store" serializes it and+-- "Shikumi.Trace.Replay" replays it offline. module Shikumi.Trace ( -- * Span and tree types SpanKind (..),@@ -261,7 +261,7 @@ -- The building state lives in 'IORef's reached through the 'Prim' effect (so no -- open-ended @IOE@ is needed here — only in-process mutation), and span -- timestamps come from shikumi's own 'Time' effect. Both are discharged at the--- program edge by 'runPrim' and 'runTime'.+-- program edge by 'Effectful.Prim.runPrim' and 'Shikumi.Effect.Time.runTime'. -- -- The span stack is sequential: use 'Shikumi.Program.runProgram' with -- 'tracedLLM', not 'Shikumi.Program.runProgramConc'. State writes are atomic so a@@ -339,7 +339,7 @@ (sid : _) -> atomicModifyIORef' (st ^. #spans) (\m -> (m & ix sid . #attrs %~ f, ())) [] -> pure () --- | Freeze the building state into an immutable 'TraceTree'.+-- | Freeze the building state into an immutable t'TraceTree'. freezeTree :: (Prim :> es) => TraceState -> Eff es TraceTree freezeTree st = do sp <- readIORef (st ^. #spans)
src/Shikumi/Trace/Feedback.hs view
@@ -3,12 +3,12 @@ -- | The per-node feedback channel (EP-16, M3). ----- A 'FeedbackLog' is a sibling of the trace (deliberately /not/ part of+-- A t'FeedbackLog' is a sibling of the trace (deliberately /not/ part of -- 'Shikumi.Trace.TraceTree', whose serialized format is pinned and consumed by -- replay): it maps a node's 'NodePath' to the textual critiques attached to it. A -- metric or LM-judge writes a critique with 'attachFeedback' during evaluation; an -- optimizer reads all critiques for a node with 'feedbackFor' during a proposal--- step. GEPA (@docs/plans/22-gepa-reflective-optimizer.md@) is the headline+-- step. GEPA (@docs\/plans\/22-gepa-reflective-optimizer.md@) is the headline -- consumer. module Shikumi.Trace.Feedback ( FeedbackLog (..),@@ -37,7 +37,7 @@ deriving stock (Eq, Show) -- | Serialized as a list of @(NodePath, [Text])@ pairs, so 'NodePath' needs only--- 'ToJSON'\/'FromJSON' (not a text 'ToJSONKey').+-- t'ToJSON'\/t'FromJSON' (not a text t'Data.Aeson.ToJSONKey'). instance ToJSON FeedbackLog where toJSON (FeedbackLog m) = toJSON (Map.toList m)
src/Shikumi/Trace/Internal/Spike.hs view
@@ -1,12 +1,12 @@ {-# LANGUAGE GADTs #-} -- | M0 de-risking spike for EP-7--- (@docs/plans/7-hierarchical-tracing-observability-and-replay.md@).+-- (@docs\/plans\/7-hierarchical-tracing-observability-and-replay.md@). ----- The one genuinely novel piece of the tracing plan is /how the span hierarchy is--- formed/, since neither baikai nor EP-1's @LLM@ effect carries a parent\/child--- relationship. This spike proves the mechanism the production effect (M1) is--- built on:+-- The one genuinely novel piece of the tracing plan is+-- /how the span hierarchy is formed/, since neither baikai nor EP-1's @LLM@ effect+-- carries a parent/child relationship. This spike proves the mechanism the+-- production effect (M1) is built on: -- -- * a manually-maintained __stack of span ids__ (an @IORef [SpanId]@), and -- * an __interpose__ over EP-1's @LLM@ effect that, on each 'Shikumi.LLM.complete',@@ -14,13 +14,14 @@ -- delegating to the real handler. -- -- This is the same @interpose@ seam EP-6's @cachedLLM@ already uses, so the risk--- is low; the spike confirms that the id on top of the stack /at the moment the--- call is sent/ is the enclosing span, which is exactly what M1 needs.+-- is low; the spike confirms that the id on top of the stack+-- /at the moment the call is sent/ is the enclosing span, which is exactly what+-- M1 needs. -- -- Decision (EP-7, 2026-06-08): capture LM calls by interposing on the @LLM@ -- effect rather than installing a baikai 'Baikai.Trace.Sink.TraceSink'. EP-1's -- interpreters expose no sink parameter, and @LLM.complete@ returns the full--- baikai 'Baikai.Response' (latency, usage, cost, tool blocks) — strictly more+-- baikai t'Baikai.Response.Response' (latency, usage, cost, tool blocks) — strictly more -- than baikai's flat @TraceEvent@. See the plan's Decision Log. module Shikumi.Trace.Internal.Spike ( SpanId (..),
src/Shikumi/Trace/Node.hs view
@@ -2,7 +2,7 @@ -- | Node identity for the trace (EP-16, M1). ----- A 'NodePath' names a 'Shikumi.Program.Program' node's structural position as the+-- A t'NodePath' names a t'Shikumi.Program.Program' node's structural position as the -- list of branch steps from the program root. 'programNodePaths' enumerates the -- path of every @Predict@ node in /exactly/ the left-to-right depth-first order -- 'Shikumi.Program.foldParams' yields their @Params@ — so @programNodePaths p !! n@@@ -43,7 +43,7 @@ ) -- | One structural step from a parent node to a child, naming which branch was--- taken. The labels mirror the 'Shikumi.Program.Program' constructors so a path is+-- taken. The labels mirror the t'Shikumi.Program.Program' constructors so a path is -- human-readable and shape-stable: two programs of the same shape yield identical -- paths. data NodeStep@@ -72,14 +72,14 @@ deriving stock (Eq, Ord, Show, Generic) deriving anyclass (ToJSON, FromJSON) --- | The structural position of a node within a 'Shikumi.Program.Program', as the+-- | The structural position of a node within a t'Shikumi.Program.Program', as the -- list of steps from the program root to that node, outermost first. A bare -- root @Predict@ has the empty path. newtype NodePath = NodePath [NodeStep] deriving stock (Eq, Ord, Show, Generic) deriving newtype (ToJSON, FromJSON) --- | Enumerate the 'NodePath' of every @Predict@ node, in the same left-to-right+-- | Enumerate the t'NodePath' of every @Predict@ node, in the same left-to-right -- depth-first order as 'Shikumi.Program.foldParams'. Reuses that identical descent -- so length and node order agree by construction. programNodePaths :: Program i o -> [NodePath]@@ -100,7 +100,7 @@ go prefix (Ensemble ps _) = concat (zipWith (\i p -> go (StepEnsemble i : prefix) p) [0 ..] ps) go _ (Embed _) = [] --- | Render a 'NodePath' to a short, stable string (e.g. @compose.0/predict@-style+-- | Render a t'NodePath' to a short, stable string (e.g. @compose.0/predict@-style -- slash-joined steps), suitable as a trace/OTel attribute value. The empty path -- renders as @\"root\"@. renderNodePath :: NodePath -> Text@@ -120,8 +120,8 @@ StepMajorityVote -> "majorityVote" StepEnsemble i -> "ensemble." <> T.pack (show i) --- | Associate each @Predict@ node's 'NodePath' with its input/output field names.--- A consumer with a 'NodePath' from a trace span zips against this to map+-- | Associate each @Predict@ node's t'NodePath' with its input/output field names.+-- A consumer with a t'NodePath' from a trace span zips against this to map -- path → field metadata; one with a @foldParams@ index uses -- 'Shikumi.Program.nodeFieldsIndexed' directly. nodeFields :: Program i o -> [(NodePath, NodeFields)]
src/Shikumi/Trace/Program.hs view
@@ -5,10 +5,10 @@ -- | Node-correlated program execution for the trace (EP-16, M2). -- -- 'runProgramTraced' is an /additive/ entry point: it runs a--- 'Shikumi.Program.Program' exactly like 'Shikumi.Program.runProgram' — reusing+-- t'Shikumi.Program.Program' exactly like 'Shikumi.Program.runProgram' — reusing -- its render\/parse\/retry\/vote semantics by delegating each @Predict@ leaf to -- @runProgram@ — but it also opens a trace span per node and threads the active--- node's 'NodePath' so that each model-call span is tagged with the structural+-- node's t'NodePath' so that each model-call span is tagged with the structural -- position of the node that issued it. 'Shikumi.Program.runProgram' / -- 'Shikumi.Program.runProgramConc' are untouched (MasterPlan integration point #4). --@@ -89,7 +89,7 @@ -- The current-node effect -- --------------------------------------------------------------------------- --- | Carries the 'NodePath' of the node currently executing. 'localNode' sets it+-- | Carries the t'NodePath' of the node currently executing. 'localNode' sets it -- for the duration of an inner action; 'askNode' reads it. A dedicated effect -- (rather than @Effectful.Reader@) keeps the value dynamically scoped as execution -- descends the program tree without the caller threading it, and keeps@@ -135,7 +135,7 @@ -- Node-aware capture -- --------------------------------------------------------------------------- --- | Like 'Shikumi.Trace.tracedLLM' but also stamps the active 'NodePath' onto each+-- | Like 'Shikumi.Trace.tracedLLM' but also stamps the active t'NodePath' onto each -- model-call span. The capture (model\/prompt\/response\/cost) and the node tag are -- written inside the /same/ 'withSpan', so the tag always lands on the LM-call span -- — no dependence on interpose ordering relative to a separate capture layer.@@ -153,7 +153,7 @@ -- --------------------------------------------------------------------------- -- | Run a program like 'Shikumi.Program.runProgram', opening a span per node and--- tagging each model-call span with the issuing node's 'NodePath'. The step prefix+-- tagging each model-call span with the issuing node's t'NodePath'. The step prefix -- accumulated as it descends is the same one 'Shikumi.Trace.Node.programNodePaths' -- builds, so a @Predict@ leaf's path here equals the path that enumeration assigns -- it. Each @Predict@ leaf delegates to @runProgram@ (reusing its exact semantics);
src/Shikumi/Trace/Replay.hs view
@@ -7,13 +7,13 @@ -- 'Shikumi.LLM.complete' from a recorded trace. It computes the EP-6 -- content-addressed 'Shikumi.Cache.Key.cacheKey' of the request, looks it up in a -- replay index (built by 'Shikumi.Trace.Store.replayIndex'), and returns the--- recorded 'Baikai.Response' — decoded via the 'Shikumi.Trace.ResponseJSON'+-- recorded t'Baikai.Response.Response' — decoded via the "Shikumi.Trace.ResponseJSON" -- instances. Because it replaces the @LLM@ effect at the /same boundary/, the rest -- of the program (modules, combinators, decoding) runs exactly as in live mode, so -- the typed outputs are identical; only the leaf LM calls are redirected. -- -- Divergence is __fail-closed and loud__: a request whose key is not in the trace--- raises a typed 'ReplayDivergence' carrying the key, the model id, and a redacted+-- raises a typed t'ReplayDivergence' carrying the key, the model id, and a redacted -- prompt summary. It never falls through to the network and never fabricates a -- response. (There is no registry in this interpreter at all, so "zero provider -- calls" is structural, not merely policy.)@@ -53,10 +53,10 @@ deriving anyclass (Exception) -- | Interpret the @LLM@ effect by lookup in a replay index instead of calling a--- provider. A hit returns the recorded response; a miss raises 'ReplayDivergence'.+-- provider. A hit returns the recorded response; a miss raises t'ReplayDivergence'. -- Streaming completions are not replayable and raise a divergence naming the key. ----- No @IOE@ is required: the lookup and decode are pure, and 'ReplayDivergence' is+-- No @IOE@ is required: the lookup and decode are pure, and t'ReplayDivergence' is -- raised with @effectful@'s pure-in-@Eff@ 'throwIO'. The contract the plan sketched -- as @(IOE :> es)@ is therefore satisfied with a strictly weaker constraint. runLLMReplay :: Map CacheKey Value -> Eff (LLM : es) a -> Eff es a@@ -74,7 +74,7 @@ throwIO $ (divergence (cacheKey m c o) m c) & #promptSummary .~ "replay does not support streaming completions" --- | Build a 'ReplayDivergence' for a request.+-- | Build a t'ReplayDivergence' for a request. divergence :: CacheKey -> Model -> Context -> ReplayDivergence divergence key m c = ReplayDivergence
src/Shikumi/Trace/ResponseJSON.hs view
@@ -1,7 +1,7 @@--- | A faithful JSON round-trip for baikai's 'Baikai.Response.Response' graph.+-- | A faithful JSON round-trip for baikai's t'Baikai.Response.Response' graph. -- -- This module exists for backward compatibility: it re-exports the baikai--- 'Response'-graph orphan instances ('ToJSON'/'FromJSON' for @Response@,+-- t'Baikai.Response.Response'-graph orphan instances ([ToJSON]("Data.Aeson#t:ToJSON")/t'Data.Aeson.FromJSON' for @Response@, -- @AssistantPayload@, @Usage@, @Cost@, @CostBreakdown@) from their single home, -- "Shikumi.Cache.ResponseJSON" in @shikumi-cache@ (EP-6 owns the cache key that -- @shikumi-trace@ already depends on, so the dependency direction is natural).
src/Shikumi/Trace/Store.hs view
@@ -1,8 +1,8 @@--- | Persisting a 'TraceTree' to a stable on-disk JSON format, and deriving the+-- | Persisting a t'TraceTree' to a stable on-disk JSON format, and deriving the -- replay index from it (EP-7, M2). ----- A trace file is a single JSON document — a 'TraceFile' carrying a--- @formatVersion@ integer and the whole 'TraceTree'. Persisting the tree as a+-- A trace file is a single JSON document — a t'TraceFile' carrying a+-- @formatVersion@ integer and the whole t'TraceTree'. Persisting the tree as a -- tree (rather than a flat event log) keeps replay a simple key lookup and keeps -- the file inspectable with @jq@ (e.g. @jq '.tree.spans | length'@). --@@ -12,7 +12,7 @@ -- does not understand rather than producing wrong outputs. -- -- 'replayIndex' projects the tree into the @Map CacheKey Value@ the replay--- interpreter ('Shikumi.Trace.Replay') consults: each LM-call span's EP-6+-- interpreter ("Shikumi.Trace.Replay") consults: each LM-call span's EP-6 -- 'Shikumi.Cache.Key.CacheKey' mapped to its recorded response JSON. module Shikumi.Trace.Store ( TraceFile (..),
test/Main.hs view
@@ -481,7 +481,7 @@ firstSpan (s : _) = s firstSpan [] = error "genTree: empty" --- | Flatten a shape into spans, assigning ids/parents/times from a running+-- | Flatten a shape into spans, assigning ids\/parents\/times from a running -- counter. Returns the produced spans (root first) and the next free counter. flattenShape :: Maybe SpanId -> Int -> Shape -> ([Span], Int) flattenShape par n (Shape k kids) =