baikai-trace-otel 0.4.0.0 → 0.4.0.1
raw patch · 4 files changed
+252/−13 lines, 4 filesdep ~baikaiPVP ok
version bump matches the API change (PVP)
Dependency ranges changed: baikai
API changes (from Hackage documentation)
Files
- CHANGELOG.md +200/−0
- baikai-trace-otel.cabal +4/−3
- src/Baikai/Trace/Sink/OpenTelemetry.hs +19/−10
- test/Main.hs +29/−0
CHANGELOG.md view
@@ -7,6 +7,206 @@ ## [Unreleased] +## [baikai 0.7.0.0] - 2026-09-08++### Added++- `BaikaiError.refusalCategory` preserves an Anthropic refusal's+ provider category. JSON adds `refusal_category`; older errors still decode.+ Evidence schema 2.5 records the addition without changing digest inputs.+ __Breaking__ to construct a `BaikaiError` from its full field list.++- `Speed`, `Options.speed`, catalog-owned fast rates and+ `computeCostAtSpeed`. Anthropic gates fast mode by model capability, adds the+ beta header and records unsupported drops. Terminal pricing uses observed+ speed, including cache duration; unreported speed is an explicit estimate.+ Older `Model` JSON defaults the new fields safely. __Breaking__: public+ records and sum types gain fields and constructors.++- API usage now records observed service tiers, inference speed and server-tool+ use in optional billing facts covered by evidence schema 2.2. Missing service+ information and uncurated products produce explicit standard-rate estimates.+ `computeCostForService` separates requested and observed service, while+ `computeCostAtRates` prices a resolved rate set once for future speed policies.+ Empty billing facts preserve legacy availability JSON; a CLI-reported zero+ cost retains its reported-total source. __Breaking__: the public vocabulary+ and records gain members.++- Failed trace terminals now retain partial response token counts, cost basis,+ usage availability and USD totals. Synthetic aborts leave unreported billing+ absent; legacy failed trace JSON still decodes. __Breaking__: `TraceEvent`'s+ `CallFailed` gains fields. See `baikai-trace-otel 0.4.0.1` for the export.++- Successful trace terminals and call-log records carry optional cost basis and+ usage availability; call logs also carry cache-write counts. Old JSON decodes+ with absent metadata and empty additive-zero bases stay omitted from traces.+ __Breaking__: `CallFinished` and the call-log record gain fields.++- Optional `Usage.availability` and shared inclusive/exclusive billing+ normalization, in the new `Baikai.Usage.Normalize`. OpenAI Chat/Responses and+ Claude preserve missing cache counters as explicit estimation reasons,+ distinguish reported zeroes, and merge cumulative usage without+ double-counting. Schema 2.2 commits provider availability while preserving+ legacy usage digests. __Breaking__: `Usage` gains a field.++- Optional `Model.pricingPolicy`, exact whole-request context tiers,+ and an explicit cache-duration rate resolver. Generated Astra pricing changes+ above 272000 input tokens; Fable exposes its one-hour write price. `Cost.basis`+ preserves calculation sources and estimation reasons when summed. Evidence+ schema 2.2 serializes the local basis without including local pricing metadata+ in provider commitments. __Breaking__: `Model` and `Cost` gain fields.++- Separate `OpenAIResponses` dispatch and compatibility types, and+ optional provider/model-scoped `ThinkingContent.replayState` with opaque+ diagnostic output and backward-compatible JSON decoding. Evidence schema 2.2+ includes replay state and optional billing facts in commitments while preserving+ legacy encodings when those fields are absent. __Breaking__ for a `case` over+ `Api` that is exhaustive without a wildcard.++- `Baikai.Evidence.ThinkingTranslation` gains `displayText` and+ `ThinkingAdjustment` gains `ThinkingSummaryUnavailable`, so a transport can+ record the thinking display setting it asked for and diagnose a successful+ response whose thinking blocks carry no readable summary. `Baikai.Compat`+ gains `supportsForcedToolChoice`; legacy JSON defaults it to True.+ __Breaking__ for an exhaustive `case` over `ThinkingAdjustment`.++- GPT-6 Astra and Claude Fable 5.1 catalog bindings, with verified+ pricing, token limits, and Anthropic thinking/sampling compatibility.++- Repository `update-models` skill for verifying provider releases and refreshing+ the curated JSON and generated Haskell catalog.++### Fixed++- Preserve OpenAI endpoint capability facts through catalog refreshes.++- Chat and Claude reject provider-scoped reasoning replay they cannot encode.++- Widened the `http-client-tls` bound to admit 0.4 (carried forward from the+ tagged but never-published 0.6.0.1).++## [baikai-claude 0.7.0.0] - 2026-09-08++### Changed++- Refusal messages include the reported category and explanation,+ retaining the original message when neither exists. Classification remains+ non-retryable `ContentFiltered`. Server-side fallbacks remain deliberately+ unsupported, as recorded in ADR 0005.++- Adaptive reasoning requests explicitly ask for summarized+ thinking. Evidence schema 2.4 records the display setting and diagnoses+ successful responses whose thinking blocks contain no readable summary.+ Budget and absent-thinking request shapes, signed empty blocks, redacted+ content and multi-turn replay are preserved.++- Fast mode is gated by the generated model capability: it adds the Anthropic+ beta header for a model that advertises it and records an evidence adjustment+ for one that does not.++### Fixed++- Price Fable cache writes using the TTL in the shaped request,+ including compatibility downgrades. Missing write-duration context is explicit+ in the cost basis.++- Reject forced tool choices locally on Fable 5.1, using the+ generated `supportsForcedToolChoice` capability. Automatic tool rounds retain+ signed empty/visible thinking, redacted blocks and prior-message order.++- Widened the `http-client-tls` bound to admit 0.4 (carried forward from the+ tagged but never-published 0.6.0.1).++### Changed (dependencies)++- Requires `baikai ^>=0.7.0`.++## [baikai-openai 0.7.0.0] - 2026-09-08++### Added++- Explicit `Baikai.Provider.OpenAI.Responses` registration and+ stream/complete provider with stateless reasoning replay, function tool turns,+ structured output and bounded worker cleanup, across the new+ `Baikai.Provider.OpenAI.Responses.{Request,Stream,Assembler}` modules. Astra+ now selects this provider through a per-model catalog override; callers must+ register it explicitly. Cache writes, billing availability and context pricing+ are integrated.++- `Baikai.Provider.OpenAI.Internal.Usage`, the shared usage mapping both the+ Chat Completions and Responses transports read.++### Fixed++- Reject tools locally for models whose Chat Completions endpoint+ disallows them, including GPT-6 Astra. Respect generated effort policies and+ sampling restrictions, with matching translation evidence and strict refusal.++- Validate Responses terminals and enforce the stream contracts.++- Widened the `http-client-tls` bound to admit 0.4 (carried forward from the+ tagged but never-published 0.6.0.1).++### Changed (dependencies)++- Requires `baikai ^>=0.7.0`.++## [baikai-trace-otel 0.4.0.1] - 2026-09-08++### Added++- Successful and failed spans export `baikai.cost.basis` and+ `baikai.usage.availability` as canonically encoded JSON. A failed span now+ also carries the input/output token counts and USD total that+ `baikai 0.7.0.0` retains on `CallFailed`, alongside its error status.++### Changed (dependencies)++- Requires `baikai ^>=0.7.0`, and now depends on `aeson ^>=2.2` to encode the+ two new attributes.++## [baikai-effectful 0.4.0.1] - 2026-09-08++### Changed (dependencies)++- Requires `baikai ^>=0.7.0`. No API change.++## [baikai-kit 0.2.0.1] - 2026-09-08++### Changed (dependencies)++- Requires `baikai ^>=0.7.0`. No API change.++## [baikai-agent 0.2.0.1] - 2026-09-08++### Changed (dependencies)++- Requires `baikai ^>=0.7.0`, `baikai-claude ^>=0.7` and+ `baikai-openai ^>=0.7`. No API change.++## [baikai 0.6.0.1] - 2026-08-30++### Fixed++- widened the `http-client-tls` bound to admit 0.4. The 0.4 API retains the+ manager functions this package uses and belongs to the same TLS 2.x / Crypton+ 1.1 dependency cohort as baikai 0.6; the old `^>=0.3` cap made baikai 0.6+ impossible to solve in applications that require Crypton 1.1.++## [baikai-claude 0.6.0.1] - 2026-08-30++### Fixed++- widened the `http-client-tls` bound to admit 0.4, allowing applications that+ require Crypton 1.1 to solve the dependency set.++## [baikai-openai 0.6.0.1] - 2026-08-30++### Fixed++- widened the `http-client-tls` bound to admit 0.4, allowing applications that+ require Crypton 1.1 to solve the dependency set.+ ## [baikai 0.6.0.0] - 2026-08-28 ### Added
baikai-trace-otel.cabal view
@@ -1,6 +1,6 @@ cabal-version: 3.4 name: baikai-trace-otel-version: 0.4.0.0+version: 0.4.0.1 synopsis: OpenTelemetry TraceSink for baikai. description: Provides an opt-in OpenTelemetry adapter for the baikai 'TraceSink'@@ -52,7 +52,8 @@ hs-source-dirs: src exposed-modules: Baikai.Trace.Sink.OpenTelemetry build-depends:- , baikai ^>=0.6.0+ , aeson ^>=2.2+ , baikai ^>=0.7.0 , base >=4.20 && <5 , containers ^>=0.7 , hs-opentelemetry-api >=1.0 && <1.1@@ -72,7 +73,7 @@ ghc-options: -threaded -with-rtsopts=-N build-depends: , aeson ^>=2.2- , baikai ^>=0.6.0+ , baikai ^>=0.7.0 , baikai-trace-otel , base , generic-lens
src/Baikai/Trace/Sink/OpenTelemetry.hs view
@@ -33,12 +33,14 @@ import Baikai.Trace.Event (TraceEvent (..)) import Baikai.Trace.Sink (TraceSink (..)) import Control.Monad (forM_)+import Data.Aeson qualified as Aeson import Data.HashMap.Strict qualified as HashMap import Data.Int (Int64) import Data.Map.Strict qualified as Map import Data.Maybe (fromMaybe) import Data.Scientific qualified as Scientific import Data.Text (Text)+import Data.Text.Encoding qualified as Text import Data.Time (UTCTime) import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds) import Data.Word (Word64)@@ -144,7 +146,7 @@ -- evidence, and, because 'Otel.addAttributes' replaces an existing key -- and evidence is pushed before the terminal, overwrote the genuinely -- observed value on every call that did.- CallFinished {eventId, timestamp, latencyMs, inputTokens, outputTokens, usd} ->+ CallFinished {eventId, timestamp, latencyMs, inputTokens, outputTokens, usd, costBasis, usageAvailability} -> case Map.lookup eventId m of -- A terminal without a live span is unreachable for normal withTraceStream -- usage because each traced call drives a fresh fold. Keep the silent drop so@@ -155,15 +157,17 @@ attrs = maybe id (\n -> AttrMap.insertByKey SC.genAi_usage_inputTokens (fromIntegral n :: Int64)) inputTokens $ maybe id (\n -> AttrMap.insertByKey SC.genAi_usage_outputTokens (fromIntegral n :: Int64)) outputTokens $- maybe id (\s -> HashMap.insert "baikai.cost.usd" (Attr.toAttribute (Scientific.toRealFloat s :: Double))) usd $- HashMap.fromList- [ ("baikai.latency_ms", Attr.toAttribute latencyMs)- ]+ maybe id (\b -> HashMap.insert "baikai.cost.basis" (Attr.toAttribute (Text.decodeUtf8 (Ev.canonicalEncode (Aeson.toJSON b))))) costBasis $+ maybe id (\a -> HashMap.insert "baikai.usage.availability" (Attr.toAttribute (Text.decodeUtf8 (Ev.canonicalEncode (Aeson.toJSON a))))) usageAvailability $+ maybe id (\s -> HashMap.insert "baikai.cost.usd" (Attr.toAttribute (Scientific.toRealFloat s :: Double))) usd $+ HashMap.fromList+ [ ("baikai.latency_ms", Attr.toAttribute latencyMs)+ ] Otel.addAttributes sp attrs Otel.setStatus sp Otel.Ok Otel.endSpan sp (Just (utcToTimestamp timestamp)) pure (Map.delete eventId m)- CallFailed {eventId, timestamp, latencyMs, errorMessage} ->+ CallFailed {eventId, timestamp, latencyMs, errorMessage, inputTokens, outputTokens, usd, costBasis, usageAvailability} -> case Map.lookup eventId m of -- A terminal without a live span is unreachable for normal withTraceStream -- usage because each traced call drives a fresh fold. Keep the silent drop so@@ -171,10 +175,15 @@ Nothing -> pure m Just sp -> do Otel.addAttributes sp $- HashMap.fromList- [ ("baikai.latency_ms", Attr.toAttribute latencyMs),- ("baikai.error", Attr.toAttribute errorMessage)- ]+ maybe id (\n -> AttrMap.insertByKey SC.genAi_usage_inputTokens (fromIntegral n :: Int64)) inputTokens $+ maybe id (\n -> AttrMap.insertByKey SC.genAi_usage_outputTokens (fromIntegral n :: Int64)) outputTokens $+ maybe id (\s -> HashMap.insert "baikai.cost.usd" (Attr.toAttribute (Scientific.toRealFloat s :: Double))) usd $+ maybe id (\b -> HashMap.insert "baikai.cost.basis" (Attr.toAttribute (Text.decodeUtf8 (Ev.canonicalEncode (Aeson.toJSON b))))) costBasis $+ maybe id (\a -> HashMap.insert "baikai.usage.availability" (Attr.toAttribute (Text.decodeUtf8 (Ev.canonicalEncode (Aeson.toJSON a))))) usageAvailability $+ HashMap.fromList+ [ ("baikai.latency_ms", Attr.toAttribute latencyMs),+ ("baikai.error", Attr.toAttribute errorMessage)+ ] Otel.setStatus sp (Otel.Error errorMessage) Otel.endSpan sp (Just (utcToTimestamp timestamp)) pure (Map.delete eventId m)
test/Main.hs view
@@ -5,12 +5,14 @@ import Baikai.Api (Api (..)) import Baikai.Content (AssistantContent (..), TextContent (..)) import Baikai.Context (Context (..), emptyContext)+import Baikai.Cost qualified as Cost import Baikai.Error (BaikaiError, providerError) import Baikai.Evidence ( CallStatus (..), EvidenceRequest, Observed (..), TransportKind (..),+ canonicalEncode, evidenceRequest, noThinkingRequested, )@@ -41,6 +43,7 @@ import Data.IORef (IORef, readIORef) import Data.Text (Text) import Data.Text qualified as Text+import Data.Text.Encoding qualified as TextEncoding import Data.Time (getCurrentTime) import Data.Vector qualified as V import OpenTelemetry.Attributes qualified as Attr@@ -62,6 +65,7 @@ [ successSpanTest, observedModelSpanTest, failureSpanTest,+ partialBillingSpanTest, abortSpanTest, evidenceSpanTest, liveEvidenceSpanTest,@@ -93,6 +97,7 @@ & #inputTokens .~ 12 & #outputTokens .~ 3 & #totalTokens .~ 15+ & #cost .~ Cost.estimateCost [Cost.ServiceTierNotReported] Cost.zeroCost stubResponse :: Api -> Response stubResponse a =@@ -178,6 +183,7 @@ (not (HashMap.member "gen_ai.response.model" attrs)) assertBool "has gen_ai.usage.input_tokens" (HashMap.member "gen_ai.usage.input_tokens" attrs) assertBool "has gen_ai.usage.output_tokens" (HashMap.member "gen_ai.usage.output_tokens" attrs)+ HashMap.lookup "baikai.cost.basis" attrs @?= Just (Attr.toAttribute (TextEncoding.decodeUtf8 (canonicalEncode (Aeson.toJSON (sampleUsage ^. #cost . #basis))))) assertBool "has baikai.event_id" (HashMap.member "baikai.event_id" attrs) assertBool "has baikai.latency_ms" (HashMap.member "baikai.latency_ms" attrs) assertBool "does not emit deprecated GenAI system key" (not (HashMap.member deprecatedGenAiSystemKey attrs))@@ -244,6 +250,29 @@ "the call span records a parent" (maybe False (const True) (Otel.spanParent sp)) other -> assertFailure ("expected one baikai.call span, got " <> show (length other))++partialBillingSpanTest :: TestTree+partialBillingSpanTest =+ testCase "failed span retains partial usage and calculation basis" $ do+ let a = Custom "baikai-otel-partial-cost"+ partial = stubResponse a & #message . #stopReason .~ ErrorReason & #message . #errorMessage .~ Just "connection reset" & #message . #usage . #cost . #usd .~ (3 / 1000000) & #message . #usage . #cost . #breakdown . #inputUsd .~ (3 / 1000000)+ handler _ _ _ = pure partial+ registerApiProvider (apiProviderWith a (liftCompleteToStream handler) handler)+ (tracer, getSpans) <- newTracerWithInMemory+ response <- withTrace (otelSink tracer) (stubModel a) stubContext stubOptions+ spans <- getSpans+ case spans of+ [sp] -> do+ hot <- spanHotSnapshot sp+ let attrs = Attr.getAttributeMap (Otel.hotAttributes hot)+ usage = response ^. #message . #usage+ HashMap.lookup "baikai.cost.basis" attrs @?= Just (Attr.toAttribute (TextEncoding.decodeUtf8 (canonicalEncode (Aeson.toJSON (usage ^. #cost . #basis)))))+ HashMap.lookup "baikai.cost.usd" attrs @?= Just (Attr.toAttribute (fromRational (usage ^. #cost . #usd) :: Double))+ assertBool "partial input count reaches span" (HashMap.member "gen_ai.usage.input_tokens" attrs)+ case Otel.hotStatus hot of+ Otel.Error _ -> pure ()+ other -> assertFailure ("expected error status: " <> show other)+ other -> assertFailure ("expected one span: " <> show (length other)) failureSpanTest :: TestTree failureSpanTest =