baikai-openai 0.7.0.0 → 0.7.1.0
raw patch · 8 files changed
+624/−81 lines, 8 filesdep ~baikaidep ~directoryPVP ok
version bump matches the API change (PVP)
Dependency ranges changed: baikai, directory
API changes (from Hackage documentation)
+ Baikai.Provider.OpenAI.Cli: codexCliCommandWith :: CodexCliConfig -> Maybe FilePath -> Model -> Context -> Options -> (FilePath, [String])
Files
- CHANGELOG.md +159/−0
- baikai-openai.cabal +78/−69
- src/Baikai/Provider/OpenAI/Api.hs +2/−0
- src/Baikai/Provider/OpenAI/Cli.hs +90/−10
- src/Baikai/Provider/OpenAI/Responses.hs +2/−0
- test/Main.hs +2/−0
- test/ResponsesSpec.hs +20/−2
- test/StructuredCliSpec.hs +271/−0
CHANGELOG.md view
@@ -7,6 +7,165 @@ ## [Unreleased] +## [baikai 0.7.2.0] - 2026-09-30++### Added++- Curated GPT-6.1 Sol on OpenAI Responses and Claude Sonnet 5.5 on Anthropic+ Messages (`openai_gpt_6_1_sol`, `anthropic_claude_sonnet_5_5`), with endpoint+ compatibility facts, standard prices, GPT-6.1 Sol's 272K-token context tier,+ and Sonnet 5.5's one-hour cache-write rate. Sonnet 5.5 rejects forced tool+ choice locally and has no fast mode. Both passed live text and function-tool+ acceptance on 2026-09-30 through `/v1/responses` and `/v1/messages`+ ([record](docs/validation/plan-85/2026-09-30-complete.json),+ [plan 85](docs/plans/85-prove-gpt-6-1-sol-and-claude-sonnet-5-5-live-compatibility.md)).+ The focused smoke runner gains the `sol61-*` and `sonnet55-*` cases.+- `Baikai.ResponseFormat.StructuredOutputSupport`+ (`NativeJsonSchema | NoStructuredOutput`), `declaredStructuredOutput :: Api ->+ StructuredOutputSupport`, and a `structuredOutput` field on `ApiProvider`+ (default `NoStructuredOutput` in `apiProviderWith`), so a caller can ask+ whether a transport enforces a schema without calling it. Every built-in+ provider declares `NativeJsonSchema`.++## [baikai-claude 0.7.1.0] - 2026-09-30++Requires `baikai >=0.7.2`.++### Added++- `claude -p` honours a `JsonSchema` response format (IR-11): it receives+ `--json-schema '<schema>'` and the response text is the tool's validated+ `structured_output`. A missing `structured_output` is a `DecodeFailure`; a+ `claude` too old for the flag yields an `InvalidRequest` error with the exit+ code rather than unconstrained text. `JsonObject`, `name` and `strict` are not+ forwarded; requests without a schema render the same argument vector as+ before. Both Claude providers declare `structuredOutput = NativeJsonSchema`.++### Fixed++- Anthropic adaptive `ThinkingHigh` now sends+ `output_config.effort: "high"` instead of omitting the field. Claude Opus 5.5+ defaults to `medium`, so it previously ran `ThinkingHigh` at medium effort;+ other adaptive models default to `high` and behave as before. Evidence for+ these calls records `effortText = "high"` and no longer carries+ `effort_omitted`, so strict evidence mode no longer refuses them.+ `Baikai.Evidence.EffortOmitted` stays exported, and older records still decode.++## [baikai-openai 0.7.1.0] - 2026-09-30++Requires `baikai >=0.7.2`.++### Added++- `codex exec` honours a `JsonSchema` response format (IR-11): the schema is+ written to a temporary file passed as `--output-schema <file>` and deleted+ however the call ends. A `codex` too old for the flag yields an+ `InvalidRequest` error with the exit code rather than unconstrained text.+ `JsonObject`, `name` and `strict` are not forwarded; requests without a schema+ render the same argument vector as before. All three OpenAI providers declare+ `structuredOutput = NativeJsonSchema`.+- `codexCliCommandWith`, which renders the `codex exec`+ vector with a given `--output-schema` file. `codexCliCommand` is unchanged+ and never renders the flag.++## [baikai 0.7.1.0] - 2026-09-23++### Added++- Curated GPT-6 Sol and Luna on OpenAI Responses and Claude Opus 5.5 on+ Anthropic Messages (`openai_gpt_6_sol`, `openai_gpt_6_luna`,+ `anthropic_claude_opus_5_5`), with endpoint compatibility and standard,+ long-context, cache-duration, and fast-mode prices where applicable. All+ three passed live acceptance on 2026-09-23. Sol and Luna dispatch to+ `OpenAIResponses`, so calling them requires the+ `Baikai.Provider.OpenAI.Responses.register` call that `baikai-openai 0.7.0.0`+ introduced.++## [baikai-kit 0.3.0.0] - 2026-09-23++Closes four improvement requests from the tools that ship `baikai-kit` as their+`kit` command: project scope resolves from a configurable root (IR-8),+`kit status` reports local edits separately from upstream drift (IR-7),+`kit install` without a name asks a tool-supplied chooser (IR-6), and `list`,+`status` and `update` print versioned JSON (IR-9). A consumer raising its bound+builds `KitConfig` with `kitConfig`, passes it to `kitCommandParser`, and+matches on `StatusRow.conditions`; each break is at a call site the compiler+names.++### Added++- `baikai-kit`: `KitConfig.projectRoot :: IO FilePath` says where project scope+ lives. Install, status, update, uninstall and `agentDirsForSession` all derive+ project-scope paths from it, so they agree whichever subdirectory a command+ runs from. `kitConfig` builds a configuration with every optional field at its+ default (project scope is the current directory, as before);+ `projectRootByMarkers [".git", ".mytool"]` is a ready-made resolver that walks+ up to the nearest marker and falls back to the current directory, and+ `findProjectRoot` is the underlying walk. Resolves IR-8.++- `baikai-kit`: `kit status` reports local edits. It runs the same+ installed-file check `kit update` uses to skip an item, and shows+ `modified` for an edited copy and `edits-unknown` for one whose sidecar+ predates the installed-file hash. The check is exported as+ `checkLocalEdits`, returning `LocalEdits` (`Unedited`, `Edited`,+ `EditsUnknown`). Resolves IR-7.++- `baikai-kit`: `kit install` with no name asks a chooser the tool supplies in+ the new `KitConfig.chooseItem :: Maybe (KitManifest -> IO (Maybe Text))`+ field. The engine refreshes the kit, passes the whole manifest, and installs+ what the chooser returns; a cancelled choice prints+ `No item chosen; nothing installed.` and exits 0. With no chooser (the+ `kitConfig` default) the command fails with the new `KitItemNameRequired`+ error, which tells the user to pass `NAME`. The engine ships no picker.+ `KitCommand` derives `Eq`, and `kit install --help` names the tool's+ `.<tool>/agents` directory. Resolves IR-6.++- `baikai-kit`: `kit list`, `kit status` and `kit update` accept `--json` and+ print exactly one versioned JSON document on stdout+ (`{"formatVersion": 1, "document": "kit-list" | "kit-status" | "kit-update", …}`);+ warnings and the first-clone notice go to stderr, and a failed command writes+ nothing to stdout. The shapes are written by explicit encoders —+ `Baikai.Kit.Json.listDocument`, `statusDocument`, `updateDocument`, and+ `kitJsonFormatVersion` — so library callers get the same values, and they are+ pinned by golden tests. `Baikai.Kit.Command.OutputFormat` selects the mode,+ and `Baikai.Kit.Status.InstalledCopy` / `installedCopies` report where each+ item is installed. Resolves IR-9.++### Changed++- `baikai-kit`: `KitConfig` gains the strict field `projectRoot`, so a record+ literal must set it; build the configuration with+ `kitConfig toolName repoUrl providers` instead and override fields with record+ update syntax. `KitConfig`'s `Show` instance is now hand-written and prints+ `<IO FilePath>` for the resolver. __Breaking__.++- `baikai-kit`: `kit status` conditions compose, and `dirty` is renamed+ `changed-upstream` (it meant the upstream sources changed without a version+ bump, not local edits); `dirty+outdated` now reads+ `outdated+changed-upstream`. `StatusRow.state :: KitState` is replaced by+ `StatusRow.conditions :: [KitCondition]` (sorted; empty means up to date),+ `renderState` by `conditionLabel` and `renderConditions`, and `classify`+ returns `[KitCondition]`. `KitUpToDate`, `KitDirty` and `KitDirtyOutdated`+ are gone; match on the list instead. __Breaking__.++- `baikai-kit`: `KitInstall` takes `Maybe Text` (`Nothing` asks the chooser),+ `kitCommandParser` takes the `KitConfig` (migration: `kitCommandParser`+ becomes `kitCommandParser myKitConfig`), `KitConfig` gains the `chooseItem`+ field (set by `kitConfig`), and `KitError` gains `KitItemNameRequired`.+ __Breaking__.++- `baikai-kit`: `KitList`, `KitStatus` and `KitUpdate` gain a trailing+ `OutputFormat` field (`HumanOutput` for the previous behaviour).+ __Breaking__.++## [baikai-effectful 0.4.0.2] - 2026-09-15++### Changed (dependencies)++- Requires `effectful-core ^>=2.7` (was `^>=2.6`). No API change: none of the+ 2.7 breaking APIs (`LocalEnv`'s second type parameter, `SharedSuffix`,+ `KnownEffects`, the ticked strict modules) are used.+ ## [baikai 0.7.0.0] - 2026-09-08 ### Added
baikai-openai.cabal view
@@ -1,27 +1,33 @@-cabal-version: 3.4-name: baikai-openai-version: 0.7.0.0-synopsis: OpenAI providers for the baikai abstraction+cabal-version: 3.4+name: baikai-openai+version: 0.7.1.0+synopsis: OpenAI providers for the baikai abstraction description: OpenAI-compatible backends for baikai: Chat Completions over SSE against OpenAI and any host that emulates it, the codex exec batch provider, a launcher for interactive Codex sessions, and the renderer for unattended codex runs driven by baikai-agent. -category: AI-license: BSD-3-Clause-license-file: LICENSE-author: Nadeem Bitar-maintainer: nadeem@gmail.com-copyright: (c) 2026 Nadeem Bitar-build-type: Simple-tested-with: GHC ==9.12.4+category: AI+license: BSD-3-Clause+license-file: LICENSE+author: Nadeem Bitar+maintainer: nadeem@gmail.com+copyright: (c) 2026 Nadeem Bitar+build-type: Simple+tested-with: ghc ==9.12.4 extra-doc-files: CHANGELOG.md common common-options ghc-options:- -Wall -Wcompat -Widentities -Wincomplete-uni-patterns- -Wincomplete-record-updates -Wredundant-constraints- -fhide-source-paths -Wmissing-export-lists -Wpartial-fields+ -Wall+ -Wcompat+ -Widentities+ -Wincomplete-uni-patterns+ -Wincomplete-record-updates+ -Wredundant-constraints+ -fhide-source-paths+ -Wmissing-export-lists+ -Wpartial-fields -Wmissing-deriving-strategies -- Exhaustiveness is an error, not a warning. A non-exhaustive match@@ -36,10 +42,11 @@ -- fail the build on warnings that are stylistic or that a future GHC -- invents, and would push people toward blanket suppression. ghc-options:- -Werror=incomplete-patterns -Werror=incomplete-uni-patterns+ -Werror=incomplete-patterns+ -Werror=incomplete-uni-patterns -Werror=incomplete-record-updates - default-language: GHC2024+ default-language: GHC2024 default-extensions: DeriveAnyClass DuplicateRecordFields@@ -47,8 +54,8 @@ OverloadedStrings library- import: common-options- hs-source-dirs: src+ import: common-options+ hs-source-dirs: src exposed-modules: Baikai.Provider.OpenAI.Agent Baikai.Provider.OpenAI.Api@@ -66,36 +73,37 @@ Baikai.Provider.OpenAI.Sse Baikai.Provider.OpenAI.Transport - other-modules: Paths_baikai_openai+ other-modules: Paths_baikai_openai autogen-modules: Paths_baikai_openai build-depends:- , aeson ^>=2.2- , baikai ^>=0.7.0- , base >=4.20 && <5- , base64-bytestring ^>=1.2- , bytestring ^>=0.12- , case-insensitive ^>=1.2- , containers ^>=0.7- , generic-lens ^>=2.3- , http-client ^>=0.7- , http-client-tls >=0.3 && <0.5- , http-types ^>=0.12- , lens ^>=5.3- , openai ^>=2.5- , process ^>=1.6- , scientific ^>=0.3- , servant-client ^>=0.20- , streamly >=0.11 && <0.13- , streamly-core >=0.3 && <0.5- , text ^>=2.1- , time ^>=1.14- , vector ^>=0.13+ aeson ^>=2.2,+ baikai ^>=0.7.2,+ base >=4.20 && <5,+ base64-bytestring ^>=1.2,+ bytestring ^>=0.12,+ case-insensitive ^>=1.2,+ containers ^>=0.7,+ directory ^>=1.3,+ generic-lens ^>=2.3,+ http-client ^>=0.7,+ http-client-tls >=0.3 && <0.5,+ http-types ^>=0.12,+ lens ^>=5.3,+ openai ^>=2.5,+ process ^>=1.6,+ scientific ^>=0.3,+ servant-client ^>=0.20,+ streamly >=0.11 && <0.13,+ streamly-core >=0.3 && <0.5,+ text ^>=2.1,+ time ^>=1.14,+ vector ^>=0.13, test-suite baikai-openai-test- import: common-options- type: exitcode-stdio-1.0+ import: common-options+ type: exitcode-stdio-1.0 hs-source-dirs: test- main-is: Main.hs+ main-is: Main.hs other-modules: BillingSpec CliEvidenceSpec@@ -114,31 +122,32 @@ ResponsesTransportSpec ShapeSpec SseSpec+ StructuredCliSpec TransportSpec build-depends:- , aeson- , baikai ^>=0.7.0- , baikai-openai- , base >=4.20 && <5- , bytestring- , case-insensitive- , containers- , directory- , filepath- , generic-lens- , http-client- , http-types- , lens ^>=5.3- , network- , openai- , servant-client- , stm- , streamly-core >=0.3 && <0.5- , tasty- , tasty-hunit- , temporary- , text ^>=2.1- , time- , tls- , vector+ aeson,+ baikai ^>=0.7.2,+ baikai-openai,+ base >=4.20 && <5,+ bytestring,+ case-insensitive,+ containers,+ directory,+ filepath,+ generic-lens,+ http-client,+ http-types,+ lens ^>=5.3,+ network,+ openai,+ servant-client,+ stm,+ streamly-core >=0.3 && <0.5,+ tasty,+ tasty-hunit,+ temporary,+ text ^>=2.1,+ time,+ tls,+ vector,
src/Baikai/Provider/OpenAI/Api.hs view
@@ -37,6 +37,7 @@ import Baikai.Provider.OpenAI.Internal.Stream (liveSseDriver, openaiChatStreamWith) import Baikai.Provider.OpenAI.Shape (describeThinkingShape) import Baikai.Provider.Registry (registerApiProvider)+import Baikai.ResponseFormat (declaredStructuredOutput) import Baikai.Stream.Event (AssistantMessageEvent) import Control.Lens ((&), (.~), (^.)) import Data.Generics.Labels ()@@ -59,6 +60,7 @@ describeThinkingShape (openaiCompletionsCompatFor m) (m ^. #reasoning) opts ) & #strengthCeiling .~ Ev.declaredStrength OpenAIChatCompletions+ & #structuredOutput .~ declaredStructuredOutput OpenAIChatCompletions -- | Streaming producer for the OpenAI Chat Completions API. --
src/Baikai/Provider/OpenAI/Cli.hs view
@@ -18,6 +18,18 @@ -- it ran, not which model served the request, so a successful exit -- never raises the recorded 'Baikai.Evidence.EvidenceStrength' — see -- 'Baikai.Provider.Cli.Internal.subprocessStrength'.+--+-- Structured output: a 'Baikai.ResponseFormat.JsonSchema' in+-- 'Baikai.Options.responseFormat' is written, as UTF-8 JSON bytes, to a+-- temporary file passed as @--output-schema <file>@; the file is removed+-- when the call ends, however it ends. The response text is the final+-- agent message, which the tool has constrained to the schema. Codex+-- submits the schema in strict mode, so it must satisfy OpenAI's strict+-- rules (for example @additionalProperties: false@). An installed+-- @codex@ too old to know the flag yields an 'Baikai.Error.InvalidRequest'+-- error carrying the exit code, never unconstrained text.+-- 'Baikai.ResponseFormat.JsonObject' and the schema's @name@ and+-- @strict@ have no CLI analogue and are not forwarded. module Baikai.Provider.OpenAI.Cli ( CodexCliConfig, executable,@@ -26,6 +38,7 @@ skipGitRepoCheck, ephemeral, codexCliCommand,+ codexCliCommandWith, codexCliPrompt, codexCliThinking, defaultCodexCliConfig,@@ -50,17 +63,21 @@ registerApiProvider, ) import Baikai.Response qualified as Resp+import Baikai.ResponseFormat (ResponseFormat (..), declaredStructuredOutput) import Baikai.StopReason (StopReason (..)) import Baikai.Stream (liftCompleteToStream) import Baikai.ThinkingLevel (ThinkingLevel, renderThinkingLevel) import Baikai.Usage (Usage, zeroUsage) import Control.Concurrent (forkIO) import Control.Concurrent.MVar (newEmptyMVar, putMVar, takeMVar)-import Control.Exception (SomeException, displayException, fromException, try)+import Control.Exception (IOException, SomeException, bracket, displayException, fromException, try) import Control.Lens ((&), (.~), (^.))+import Control.Monad (void)+import Data.Aeson qualified as Aeson import Data.ByteString qualified as BS+import Data.ByteString.Lazy qualified as LBS import Data.Generics.Labels ()-import Data.Maybe (fromMaybe)+import Data.Maybe (fromMaybe, isJust) import Data.Text (Text) import Data.Text qualified as Text import Data.Time.Clock (UTCTime, diffUTCTime, getCurrentTime)@@ -68,8 +85,9 @@ import GHC.Generics (Generic) import Streamly.Data.Stream (Stream) import Streamly.Data.Stream qualified as Stream+import System.Directory (getTemporaryDirectory, removeFile) import System.Exit (ExitCode (..))-import System.IO (Handle)+import System.IO (Handle, hClose, openBinaryTempFile) import System.Process qualified as P -- | Configuration for the @codex exec --json@ subprocess.@@ -114,6 +132,7 @@ -- control is a command-line flag derived from Options alone. & #describeThinking .~ (\_ opts -> codexCliThinking opts) & #strengthCeiling .~ Ev.declaredStrength OpenAICompletionsCli+ & #structuredOutput .~ declaredStructuredOutput OpenAICompletionsCli modelArgs :: Model -> [String] modelArgs m = case Text.strip (m ^. #modelId) of@@ -140,8 +159,21 @@ -- | Render the executable and arguments for a @codex exec --json@ -- batch call. The prompt is preceded by @--@ so dash-leading prompts -- cannot be parsed as options.+--+-- This never renders @--output-schema@, even when+-- 'Baikai.Options.responseFormat' carries a schema: the schema file+-- exists only while a call runs, so a pure renderer has no path to+-- name. Use 'codexCliCommandWith' to see the vector a schema call+-- spawns. codexCliCommand :: CodexCliConfig -> Model -> Context -> Options -> (FilePath, [String])-codexCliCommand cfg m ctx opts =+codexCliCommand cfg = codexCliCommandWith cfg Nothing++-- | 'codexCliCommand' with the schema file to pass as+-- @--output-schema@, rendered after the reasoning-effort override and+-- before 'extraArgs'. 'Nothing' renders exactly 'codexCliCommand'.+codexCliCommandWith ::+ CodexCliConfig -> Maybe FilePath -> Model -> Context -> Options -> (FilePath, [String])+codexCliCommandWith cfg schemaFile m ctx opts = ( cfg ^. #executable, ["exec"] <> modelArgs m@@ -149,6 +181,7 @@ <> ["--skip-git-repo-check" | cfg ^. #skipGitRepoCheck] <> ["--ephemeral" | cfg ^. #ephemeral] <> effortArgs opts+ <> maybe [] (\path -> ["--output-schema", path]) schemaFile <> fmap Text.unpack (cfg ^. #extraArgs) <> ["--", Text.unpack (codexCliPrompt ctx)] )@@ -192,11 +225,44 @@ adjustments = [Ev.EffortClamped lvl wire | wire /= renderThinkingLevel lvl] } +-- | The schema a caller asked the tool to enforce, if any.+requestedSchema :: Options -> Maybe Aeson.Value+requestedSchema opts = case opts ^. #responseFormat of+ Just (JsonSchema f) -> Just (f ^. #schema)+ _ -> Nothing++-- | Write the schema to a fresh temporary file for the duration of one+-- action, and remove it afterwards however the action ends.+--+-- The bytes come from 'Aeson.encode', which is UTF-8 by construction;+-- nothing here goes through a locale-dependent text handle. A failure+-- to remove the file is ignored: the call's outcome is already decided,+-- and a stray file in the temporary directory is harmless.+withSchemaFile :: Aeson.Value -> (FilePath -> IO a) -> IO a+withSchemaFile schemaDoc k = do+ dir <- getTemporaryDirectory+ bracket+ (openBinaryTempFile dir "baikai-codex-schema.json")+ (\(path, h) -> hClose h >> void (try (removeFile path) :: IO (Either IOException ())))+ ( \(path, h) -> do+ LBS.hPut h (Aeson.encode schemaDoc)+ hClose h+ k path+ )++compactJson :: Aeson.Value -> String+compactJson = Text.unpack . Internal.decodeUtf8Lenient . LBS.toStrict . Aeson.encode+ runCodexCli :: CodexCliConfig -> Model -> Context -> Options -> IO Resp.Response runCodexCli cfg m ctx opts = do- let (exe, args) = codexCliCommand cfg m ctx opts- procSpec =- (P.proc exe args)+ let schema = requestedSchema opts+ -- The envelope names the schema itself where the spawned vector+ -- names its temporary file: the random path would make the+ -- request commitment unreproducible, and the schema is what+ -- actually crossed the boundary.+ (exe, args) = codexCliCommandWith cfg (compactJson <$> schema) m ctx opts+ procSpec schemaFile =+ (P.proc exe (snd (codexCliCommandWith cfg schemaFile m ctx opts))) { P.std_in = P.NoStream, P.std_out = P.CreatePipe, P.std_err = P.CreatePipe,@@ -220,7 +286,15 @@ st mErr traverse (observeCodexCli exe mReport st) prepared- result <- Internal.trySync (P.withCreateProcess procSpec (consume start mkEv m))+ launch schemaFile =+ P.withCreateProcess (procSpec schemaFile) (consume start mkEv (isJust schema) m)+ -- Writing the schema file sits inside 'Internal.trySync' too, so a+ -- temporary directory that cannot be written becomes an error-shaped+ -- response rather than an exception escaping the provider.+ result <-+ Internal.trySync $ case schema of+ Nothing -> launch Nothing+ Just schemaDoc -> withSchemaFile schemaDoc (launch . Just) case result of Right resp -> pure resp Left ex -> do@@ -238,13 +312,15 @@ Maybe BaikaiError -> IO (Maybe Ev.ModelCallEvidence) ) ->+ -- | Whether @--output-schema@ was sent.+ Bool -> Model -> Maybe Handle -> Maybe Handle -> Maybe Handle -> P.ProcessHandle -> IO Resp.Response-consume start mkEv m _ mOut mErr ph = do+consume start mkEv schemaSent m _ mOut mErr ph = do case (mOut, mErr) of (Nothing, _) -> errorNow (providerError "codex: stdout handle missing") (_, Nothing) -> errorNow (providerError "codex: stderr handle missing")@@ -260,7 +336,11 @@ end <- getCurrentTime case exitCode of ExitFailure n -> do- let err = processError n (Internal.decodeUtf8Lenient errBytes)+ let stderr = Internal.decodeUtf8Lenient errBytes+ flagRejected+ | schemaSent = Internal.unsupportedFlagError "codex" "--output-schema" n stderr+ | otherwise = Nothing+ err = fromMaybe (processError n stderr) flagRejected -- The event stream was drained before the exit status was -- known, so a failed run may still have named its thread and -- its token counts. Those are genuine observations and are
src/Baikai/Provider/OpenAI/Responses.hs view
@@ -16,6 +16,7 @@ import Baikai.Provider.OpenAI.Responses.Request (describeThinking) import Baikai.Provider.OpenAI.Responses.Stream (liveResponsesDriver, openaiResponsesStreamWith) import Baikai.Provider.Registry (registerApiProvider)+import Baikai.ResponseFormat (declaredStructuredOutput) import Baikai.Stream.Event (AssistantMessageEvent) import Control.Lens ((&), (.~)) import Data.Generics.Labels ()@@ -30,6 +31,7 @@ apiProvider OpenAIResponses openaiResponsesStream & #describeThinking .~ describeThinking & #strengthCeiling .~ Ev.declaredStrength OpenAIResponses+ & #structuredOutput .~ declaredStructuredOutput OpenAIResponses openaiResponsesStream :: Model -> Context -> Options -> Stream IO AssistantMessageEvent openaiResponsesStream = openaiResponsesStreamWith liveResponsesDriver
test/Main.hs view
@@ -56,6 +56,7 @@ import ShapeSpec qualified import SseSpec qualified import Streamly.Data.Stream qualified as Stream+import StructuredCliSpec qualified import System.Directory (getPermissions, getTemporaryDirectory, setOwnerExecutable, setPermissions) import System.Environment (lookupEnv, setEnv, unsetEnv) import System.FilePath ((</>))@@ -113,6 +114,7 @@ ResponsesTransportSpec.tests, ShapeSpec.tests, SseSpec.tests,+ StructuredCliSpec.tests, TransportSpec.tests ]
test/ResponsesSpec.hs view
@@ -3,7 +3,7 @@ module ResponsesSpec (tests) where import Baikai hiding (model, schema)-import Baikai.Models.Generated (openai_gpt_6_astra)+import Baikai.Models.Generated (openai_gpt_6_1_sol, openai_gpt_6_astra, openai_gpt_6_luna, openai_gpt_6_sol) import Baikai.Provider.OpenAI.Responses.Request qualified as R import Control.Lens ((&), (.~)) import Control.Monad (forM_)@@ -21,7 +21,25 @@ tests = testGroup "Responses request mapping"- [ testCase "stateless request carries text, image, system, cap and metadata" $ do+ [ testCase "Sol, GPT-6.1 Sol, and Luna catalog bindings retain Responses shaping and replay identity" $+ forM_ [openai_gpt_6_sol, openai_gpt_6_1_sol, openai_gpt_6_luna] $ \m -> do+ m.api @?= OpenAIResponses+ let opts = emptyOptions & #thinking .~ Just ThinkingLow & #temperature .~ Just 0.5 & #topP .~ Just 0.8+ thoughtForModel = emptyThinkingContent & #replayState .~ Just (ThinkingReplay OpenAIResponses m.modelId (V.singleton reasoningItem))+ response = emptyResponse & #message . #content .~ V.fromList [AssistantThinking thoughtForModel, AssistantToolCall (ToolCall "call_7" "lookup" (object ["x" .= (1 :: Int)]))] & #message . #stopReason .~ ToolUse+ ctx = contextOf [user "go"] & #tools .~ V.singleton tool+ first <- mapped m ctx opts+ field "model" first.requestBody @?= Just (String m.modelId)+ field "reasoning" first.requestBody @?= Just (object ["effort" .= ("low" :: Text)])+ field "temperature" first.requestBody @?= Nothing+ field "top_p" first.requestBody @?= Nothing+ next <- appendToolResult ctx response (\_ -> pure (toolResultText "found"))+ second <- mapped m next opts+ let items = inputItems second+ items !! 1 @?= reasoningItem+ field "call_id" (items !! 2) @?= Just (String "call_7")+ field "call_id" (items !! 3) @?= Just (String "call_7"),+ testCase "stateless request carries text, image, system, cap and metadata" $ do let ctx = (systemUser "system instruction" "hello") & #messages .~ V.singleton (UserMessage UserPayload {content = V.fromList [UserText (TextContent "hello"), UserImage (ImageContent "abc" "image/png")], timestamp = Nothing})
+ test/StructuredCliSpec.hs view
@@ -0,0 +1,271 @@+-- | Structured-output passthrough for the @codex exec --json@+-- subprocess provider.+--+-- The process cases run a real child process: a few lines of @sh@+-- written into a temporary directory that record the argument vector,+-- copy the @--output-schema@ file before the provider can delete it,+-- print a canned event stream and stderr, and exit with a chosen code.+module StructuredCliSpec (tests) where++import Baikai+import Baikai.Provider.OpenAI.Cli qualified as CodexCli+import Control.Lens ((&), (.~), (^.))+import Data.Aeson (Value (..), (.=))+import Data.Aeson qualified as Aeson+import Data.Aeson.KeyMap qualified as KeyMap+import Data.ByteString qualified as BS+import Data.ByteString.Lazy qualified as LBS+import Data.Generics.Labels ()+import Data.List (isPrefixOf)+import Data.Text (Text)+import Data.Text qualified as Text+import Data.Text.Encoding qualified as Text+import Data.Text.IO qualified as TextIO+import Data.Vector qualified as Vector+import System.Directory (doesFileExist, getPermissions, setOwnerExecutable, setPermissions)+import System.FilePath ((</>))+import System.IO.Temp (withSystemTempDirectory)+import Test.Tasty (TestTree, testGroup)+import Test.Tasty.HUnit (assertBool, assertFailure, testCase, (@?=))++tests :: TestTree+tests =+ testGroup+ "StructuredCliSpec: codex exec --output-schema passthrough"+ [ renderingTests,+ processTests,+ testCase "the provider declares NativeJsonSchema without spawning anything" $+ CodexCli.codexCliProvider CodexCli.defaultCodexCliConfig ^. #structuredOutput+ @?= NativeJsonSchema+ ]++-- ============================================================+-- Pure argument rendering+-- ============================================================++renderingTests :: TestTree+renderingTests =+ testGroup+ "argument rendering"+ [ testCase "JsonObject renders the same vector as no response format" $+ CodexCli.codexCliCommand cfg testModel testContext (emptyOptions & #responseFormat .~ Just JsonObject)+ @?= CodexCli.codexCliCommand cfg testModel testContext emptyOptions,+ testCase "codexCliCommand renders no --output-schema even for a JsonSchema request" $+ assertBool+ "no schema flag"+ ("--output-schema" `notElem` snd (CodexCli.codexCliCommand cfg testModel testContext schemaOptions)),+ testCase "codexCliCommandWith Nothing is codexCliCommand" $+ CodexCli.codexCliCommandWith cfg Nothing testModel testContext schemaOptions+ @?= CodexCli.codexCliCommand cfg testModel testContext schemaOptions,+ testCase "codexCliCommandWith places --output-schema <file> before the extra args" $ do+ let (_, args) = CodexCli.codexCliCommandWith cfg (Just "/tmp/s.json") testModel testContext schemaOptions+ case break (== "--output-schema") args of+ (_, "--output-schema" : "/tmp/s.json" : rest) ->+ assertBool ("extra args follow the schema: " <> show rest) (["--color", "never"] `isPrefixOf` rest)+ _ -> assertFailure ("no --output-schema in " <> show args)+ ]+ where+ cfg = CodexCli.defaultCodexCliConfig {CodexCli.extraArgs = ["--color", "never"]}++-- ============================================================+-- A real child process+-- ============================================================++processTests :: TestTree+processTests =+ testGroup+ "fake codex process"+ [ testCase "the schema file carries the schema, the reply is returned unchanged, the file is removed" $+ withFakeCodex $ \run -> do+ (resp, captured) <- run schemaOptions (Fake conformingEvents "" 0)+ responseError resp @?= Nothing+ flattenAssistantText (resp ^. #message . #content) @?= conformingText+ case captured of+ Nothing -> assertFailure "the tool received no --output-schema"+ Just (path, bytes) -> do+ Aeson.decodeStrict bytes @?= Just itemsSchema+ doesFileExist (Text.unpack path) >>= (@?= False),+ testCase "the schema file is removed after a failed run too" $+ withFakeCodex $ \run -> do+ (resp, captured) <- run schemaOptions (Fake [] "the tool refused\n" 3)+ fmap (^. #category) (responseError resp) @?= Just ProcessFailure+ case captured of+ Nothing -> assertFailure "the tool received no --output-schema"+ Just (path, _) -> doesFileExist (Text.unpack path) >>= (@?= False),+ testCase "no response format sends no --output-schema" $+ withFakeCodex $ \run -> do+ (resp, captured) <- run emptyOptions (Fake conformingEvents "" 0)+ responseError resp @?= Nothing+ fmap fst captured @?= Nothing,+ testCase "codex rejecting --output-schema is InvalidRequest with the exit code" $+ withFakeCodex $ \run -> do+ (resp, _) <- run schemaOptions (Fake [] "error: unexpected argument '--output-schema' found\n" 2)+ case responseError resp of+ Nothing -> assertFailure "expected an error-shaped response"+ Just e -> do+ e ^. #category @?= InvalidRequest+ e ^. #exitCode @?= Just 2,+ testCase "the same stderr without a schema request stays a ProcessFailure" $+ withFakeCodex $ \run -> do+ (resp, _) <- run emptyOptions (Fake [] "error: unexpected argument '--output-schema' found\n" 2)+ fmap (^. #category) (responseError resp) @?= Just ProcessFailure,+ -- The spawned vector names a fresh random path each call; the+ -- evidence envelope names the schema instead, so the commitment+ -- is reproducible and still changes with the schema.+ testCase "the request commitment is reproducible and commits to the schema" $+ withFakeCodex $ \run -> do+ (first, _) <- run (schemaOptions & #evidence .~ Just (evidenceRequest "run-87")) (Fake conformingEvents "" 0)+ (second, _) <- run (schemaOptions & #evidence .~ Just (evidenceRequest "run-87")) (Fake conformingEvents "" 0)+ (plain, _) <- run (emptyOptions & #evidence .~ Just (evidenceRequest "run-87")) (Fake conformingEvents "" 0)+ let c1 = encodedCommitment first+ assertBool ("a commitment was recorded: " <> show c1) (c1 /= Nothing)+ c1 @?= encodedCommitment second+ assertBool "the schema is part of the commitment" (c1 /= encodedCommitment plain)+ ]++-- | What the fake prints and how it exits.+data Fake = Fake+ { eventLines :: [Text],+ stderrBody :: Text,+ code :: Int+ }++-- | Write one fake @codex@ and hand back a function that runs a call+-- against it and returns the response with the schema path and bytes+-- the tool received.+--+-- Calls whose commitments are compared must share one fake, because+-- the argument vector the commitment digests begins with the+-- executable's path.+withFakeCodex :: ((Options -> Fake -> IO (Response, Maybe (Text, BS.ByteString))) -> IO a) -> IO a+withFakeCodex k =+ withSystemTempDirectory "baikai-codex-structured" $ \dir -> do+ let outPath = dir </> "stdout"+ errPath = dir </> "stderr"+ codePath = dir </> "code"+ pathPath = dir </> "schema-path"+ copyPath = dir </> "schema-copy"+ exe <-+ writeFakeExecutable dir "codex" $+ unlines+ [ "#!/bin/sh",+ "if [ \"$1\" = \"--version\" ]; then echo 'codex-cli 9.9.9'; exit 0; fi",+ "rm -f '" <> pathPath <> "' '" <> copyPath <> "'",+ "prev=",+ "for a in \"$@\"; do",+ " if [ \"$prev\" = \"--output-schema\" ]; then",+ " printf '%s' \"$a\" > '" <> pathPath <> "'",+ " cp \"$a\" '" <> copyPath <> "'",+ " fi",+ " prev=\"$a\"",+ "done",+ "cat '" <> outPath <> "'",+ "cat '" <> errPath <> "' >&2",+ "exit $(cat '" <> codePath <> "')"+ ]+ reg <- newProviderRegistry+ registerApiProviderWith+ reg+ (CodexCli.codexCliProvider CodexCli.defaultCodexCliConfig {CodexCli.executable = exe})+ k $ \opts fake -> do+ TextIO.writeFile outPath (Text.unlines (eventLines fake))+ TextIO.writeFile errPath (stderrBody fake)+ writeFile codePath (show (code fake))+ resp <- completeRequestWith reg testModel testContext opts+ sent <- doesFileExist pathPath+ captured <-+ if sent+ then do+ path <- TextIO.readFile pathPath+ bytes <- BS.readFile copyPath+ pure (Just (path, bytes))+ else pure Nothing+ pure (resp, captured)++writeFakeExecutable :: FilePath -> String -> String -> IO FilePath+writeFakeExecutable dir name body = do+ let path = dir </> name+ writeFile path body+ perms <- getPermissions path+ setPermissions path (setOwnerExecutable True perms)+ pure path++-- | The encoded @request_commitment@ of a response's evidence record.+encodedCommitment :: Response -> Maybe Value+encodedCommitment resp = case Aeson.toJSON <$> (resp ^. #evidence) of+ Just (Object o) -> KeyMap.lookup "request_commitment" o+ _ -> Nothing++-- ============================================================+-- Fixtures+-- ============================================================++testModel :: Model+testModel =+ emptyModel+ & #modelId .~ "gpt-5.6"+ & #api .~ OpenAICompletionsCli+ & #provider .~ "openai"++testContext :: Context+testContext = emptyContext & #messages .~ Vector.singleton (user "List two fruits.")++schemaOptions :: Options+schemaOptions =+ emptyOptions & #responseFormat .~ Just (JsonSchema (jsonSchemaFormat "fruits" itemsSchema))++-- | An object holding an array of objects, one field an enum, in the+-- strict shape codex requires.+itemsSchema :: Value+itemsSchema =+ Aeson.object+ [ "type" .= ("object" :: Text),+ "properties"+ .= Aeson.object+ [ "items"+ .= Aeson.object+ [ "type" .= ("array" :: Text),+ "items"+ .= Aeson.object+ [ "type" .= ("object" :: Text),+ "properties"+ .= Aeson.object+ [ "label" .= Aeson.object ["type" .= ("string" :: Text)],+ "level"+ .= Aeson.object+ [ "type" .= ("string" :: Text),+ "enum" .= (["low", "high"] :: [Text])+ ]+ ],+ "required" .= (["label", "level"] :: [Text]),+ "additionalProperties" .= False+ ]+ ]+ ],+ "required" .= (["items"] :: [Text]),+ "additionalProperties" .= False+ ]++conformingText :: Text+conformingText =+ "{\"items\":[{\"label\":\"Strawberry\",\"level\":\"low\"},{\"label\":\"Mango\",\"level\":\"high\"}]}"++-- | The event stream @codex-cli 0.159.0@ emits for an @--output-schema@+-- run: the structured reply is the text of the ordinary final+-- @agent_message@.+conformingEvents :: [Text]+conformingEvents =+ [ "{\"type\":\"thread.started\",\"thread_id\":\"019fd471-4a48-7c83-be67-6b7c49646e87\"}",+ "{\"type\":\"turn.started\"}",+ Text.decodeUtf8 . LBS.toStrict . Aeson.encode $+ Aeson.object+ [ "type" .= ("item.completed" :: Text),+ "item"+ .= Aeson.object+ [ "id" .= ("item_0" :: Text),+ "type" .= ("agent_message" :: Text),+ "text" .= conformingText+ ]+ ],+ "{\"type\":\"turn.completed\",\"usage\":{\"input_tokens\":10,\"cached_input_tokens\":0,\"output_tokens\":20}}"+ ]