diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,173 @@ ## [Unreleased] +## [baikai 0.8.0.0] - 2026-10-10 + +### Added + +- Catalog: `anthropic_claude_haiku_5_5` (Claude Haiku 5.5, released + 2026-10-07). It uses Messages with adaptive thinking, drops sampling options, + forwards forced tool choice, and has a 1,000,000-token context and + 128,000-token output limit. Prompts over 100,000 tokens, counting cache reads + and writes, are priced at $0.50/$0.05/$0.625/$2.50 per million input, cache + read, cache write and output tokens instead of $0.10/$0.01/$0.125/$0.50, and + one-hour cache writes at $1 instead of $0.20. Live text and tool cases + passed on 2026-10-10, and a live request confirmed forced tool choice with + adaptive thinking. + +### Changed (breaking) + +- `Baikai.Model.InputPriceTier` gains a third field, `longCacheWriteCost :: + Maybe Rational`: the one-hour cache-write rate for requests that cross that + tier. `resolveRates` now takes the long rate from the selected tier rather + than overriding every tier with the policy-level rate, and validation rejects + a policy whose policy-level long rate leaves any tier without one. Tier JSON + without the key decodes as `Nothing`. __Breaking__: positional constructions + of `InputPriceTier` need the new argument. + +### Changed + +- Catalog: the 2026-10-10 model refresh lowers `anthropic_claude_sonnet_5_5`'s + cache-read rate from $0.20 to $0.10 per million tokens, following Anthropic's + 2026-10-07 price cut, and corrects `anthropic_claude_sonnet_4_5`'s context + window from 1,000,000 to 200,000 tokens (its 1M beta ended on 2026-04-30). + Claude Haiku 5.5 was curated separately (see Added). No OpenAI model was + added. + +- Documentation: the 2026-10-02 model refresh found no new OpenAI or Anthropic + models; the catalog is unchanged. `docs/user/models-and-providers.md` notes + Anthropic's retirement of Claude Sonnet 4.5 on 2026-11-30 (deprecation of + `anthropic_claude_sonnet_4_5` is planned in plan 89 for a future release; + it remains in 0.8.0.0), + why the access-gated Claude Mythos models are not curated, and how GPT-6 + Astra's new ultrafast service tier is costed. + +### Fixed + +- The shared batch subprocess scope now owns POSIX process groups, joins pipe + readers, and synchronously reaps direct children before acknowledging + cancellation. Executable-version probes use the same scope. Darwin/Linux + coverage excludes processes that deliberately escape the group and adopted + descendant reaping; Windows retains direct-child cleanup. The provider fixes + ship in `baikai-claude 0.7.1.1` and `baikai-openai 0.7.1.1`; see + [plan 90](docs/plans/90-terminate-owned-batch-cli-process-groups-before-acknowledging-cancellation.md) + and [BUG-1](docs/bug-reports/batch-cli-cancellation-leaves-child-alive.md). + +- `baikai`'s source distribution now ships `data/models/*.json` and the test + suite's fixtures under `test/fixtures/`. Without them, running the suite from + the Hackage tarball failed 27 of 794 tests on missing files: the fixtures + themselves, plus the catalog round-trip test, whose `baikai-gen-models` run + had no model data to read. + +## [baikai-claude 0.7.1.1] - 2026-10-10 + +### Fixed + +- Batch `claude -p` calls stop owned POSIX process-group members and join pipe + readers before acknowledging cancellation, including active synthetic streams + and evidence probes. Preserves the original asynchronous exception. Resolves + [BUG-1](docs/bug-reports/batch-cli-cancellation-leaves-child-alive.md). + +### Changed + +- Requires `baikai ^>=0.8.0` for shared subprocess ownership and the new pricing + tier field. No provider API change. Haiku 5.5 request contracts are covered + by offline tests. + +## [baikai-openai 0.7.1.1] - 2026-10-10 + +### Fixed + +- Batch `codex exec` calls stop owned POSIX process-group members and join pipe + readers before acknowledging cancellation, including active synthetic streams, + repeated cancellation, and evidence probes. The temporary schema survives + until subprocess cleanup finishes. Resolves + [BUG-1](docs/bug-reports/batch-cli-cancellation-leaves-child-alive.md). + +### Changed + +- Requires `baikai ^>=0.8.0` for shared subprocess ownership and the new pricing + tier field. No provider API change. + +## [baikai-trace-otel 0.4.0.2] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. + +## [baikai-effectful 0.4.0.3] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. + +## [baikai-kit 0.4.0.1] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. + +## [baikai-agent 0.2.0.2] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. +- Raises the provider bounds to `baikai-claude ^>=0.7.1.1` and + `baikai-openai ^>=0.7.1.1`, so installing the tool selects the fixed batch + adapters. Binary regressions use the shared process fixture and cover both + cancellation and a 1MiB output flood. + +## [baikai-kit 0.4.0.0] - 2026-10-02 + +### Added + +- Per-item `visibility` (`tool-only` by default, or `shared`) for manifest + versions 1 and 2, with `kit install --shared`, `--tool-only`, and + `--accept-shared-codex`. Shared Claude skills and agents use tracked links; + tool-only Codex skills use tracked disabled entries in `config.toml`. +- `codexSessionArgs`, required in the `extraArgs` of a consumer's own Codex + launches to re-enable its tool-only skills. This is the one launcher step + beyond a dependency bump; the standard `kitConfig`/`kitCommandParser`/`runKit` + integration otherwise compiles unchanged. Codex custom agents cannot be + isolated and require explicit acceptance, or shared visibility. +- Requested and effective visibility in status (table and JSON), the + `visibility-broken` condition, and a migration note for legacy shared Codex + skills. Update repairs visibility even for items skipped for local edits; + uninstall removes only owned links and config entries. + +### Changed (breaking) + +- `SkillEntry` and `AgentEntry` gain `visibility`; `SidecarMeta` gains + `visibility`, `visibilitySource`, `sharedLinks`, and `codexDisabledSkills`; + `newSidecarMeta` takes these values. `RemovalOutcome` gains `linksRemoved` + and `configEntriesRemoved`; `StatusRow` gains `requestedVisibility` and + `effectiveVisibility`; `KitConfig` gains `confirmSharedCodex` (default + `Nothing` in `kitConfig`). +- `KitInstall` gains `InstallOptions`; `installItem` and `installFrom` take + it as their last argument. Use `defaultInstallOptions` to follow the + manifest. Explicit install flags persist across updates, while new + manifest-driven installs follow updated defaults. Legacy placement is kept. +- JSON adds list `visibility`, status `requestedVisibility` and + `effectiveVisibility`, and the new condition value. `formatVersion` stays + 1 because all additions preserve existing keys and their meaning. + +### Fixed + +- Codex installs refuse destinations without this tool's sidecar instead of + overwriting user or other-tool assets. Uninstall also preserves foreign + Codex assets. Shared Claude names are checked before any provider write. +- Codex config edits preserve UTF-8 text, comments and permissions, refuse + symlinks and invalid/conflicting config, and verify the semantic change + before atomic rename. Reused user-owned disabled entries survive uninstall. + Shared visibility refuses a user-owned entry that would keep the skill hidden. +- The source distribution ships the test suite's manifest fixtures and JSON + goldens, so the test suite passes when run from the Hackage tarball. Since + 0.3.0.0 it had failed there, because those files were missing. + ## [baikai 0.7.2.0] - 2026-09-30 ### Added diff --git a/baikai-openai.cabal b/baikai-openai.cabal --- a/baikai-openai.cabal +++ b/baikai-openai.cabal @@ -1,6 +1,6 @@ cabal-version: 3.4 name: baikai-openai -version: 0.7.1.0 +version: 0.7.1.1 synopsis: OpenAI providers for the baikai abstraction description: OpenAI-compatible backends for baikai: Chat Completions over SSE against OpenAI and @@ -77,7 +77,7 @@ autogen-modules: Paths_baikai_openai build-depends: aeson ^>=2.2, - baikai ^>=0.7.2, + baikai ^>=0.8.0, base >=4.20 && <5, base64-bytestring ^>=1.2, bytestring ^>=0.12, @@ -101,12 +101,15 @@ test-suite baikai-openai-test import: common-options + ghc-options: -threaded type: exitcode-stdio-1.0 hs-source-dirs: test main-is: Main.hs other-modules: BillingSpec + CliCancellationSpec CliEvidenceSpec + CliProcessFixture Contract EndpointModels ErrorClassSpec @@ -127,7 +130,7 @@ build-depends: aeson, - baikai ^>=0.7.2, + baikai ^>=0.8.0, baikai-openai, base >=4.20 && <5, bytestring, @@ -141,6 +144,7 @@ lens ^>=5.3, network, openai, + process, servant-client, stm, streamly-core >=0.3 && <0.5, @@ -151,3 +155,6 @@ time, tls, vector, + + if !os(windows) + build-depends: unix ^>=2.8 diff --git a/src/Baikai/Provider/OpenAI/Cli.hs b/src/Baikai/Provider/OpenAI/Cli.hs --- a/src/Baikai/Provider/OpenAI/Cli.hs +++ b/src/Baikai/Provider/OpenAI/Cli.hs @@ -57,6 +57,7 @@ import Baikai.Model (Model) import Baikai.Options (Options) import Baikai.Provider.Cli.Internal qualified as Internal +import Baikai.Provider.Cli.Process.Internal (withOwnedProcess, withOwnedWorker) import Baikai.Provider.Registry ( ApiProvider (..), apiProviderWith, @@ -68,8 +69,6 @@ import Baikai.Stream (liftCompleteToStream) import Baikai.ThinkingLevel (ThinkingLevel, renderThinkingLevel) import Baikai.Usage (Usage, zeroUsage) -import Control.Concurrent (forkIO) -import Control.Concurrent.MVar (newEmptyMVar, putMVar, takeMVar) import Control.Exception (IOException, SomeException, bracket, displayException, fromException, try) import Control.Lens ((&), (.~), (^.)) import Control.Monad (void) @@ -287,7 +286,7 @@ mErr traverse (observeCodexCli exe mReport st) prepared launch schemaFile = - P.withCreateProcess (procSpec schemaFile) (consume start mkEv (isJust schema) m) + withOwnedProcess (procSpec schemaFile) (consume start mkEv (isJust schema) m) -- Writing the schema file sits inside 'Internal.trySync' too, so a -- temporary directory that cannot be written becomes an error-shaped -- response rather than an exception escaping the provider. @@ -324,14 +323,9 @@ case (mOut, mErr) of (Nothing, _) -> errorNow (providerError "codex: stdout handle missing") (_, Nothing) -> errorNow (providerError "codex: stderr handle missing") - (Just hOut, Just hErr) -> do - errVar <- newEmptyMVar - _ <- - forkIO $ do - result <- try (BS.hGetContents hErr) :: IO (Either SomeException BS.ByteString) - putMVar errVar (either (const BS.empty) id result) + (Just hOut, Just hErr) -> withOwnedWorker (BS.hGetContents hErr) $ \joinErr -> do report <- Internal.parseCodexJsonlStream (handleStream hOut) - errBytes <- takeMVar errVar + errBytes <- joinErr exitCode <- P.waitForProcess ph end <- getCurrentTime case exitCode of diff --git a/test/CliCancellationSpec.hs b/test/CliCancellationSpec.hs new file mode 100644 --- /dev/null +++ b/test/CliCancellationSpec.hs @@ -0,0 +1,50 @@ +{-# LANGUAGE CPP #-} + +module CliCancellationSpec (tests) where + +import Baikai +import Baikai.Provider.OpenAI.Cli qualified as Cli +import CliProcessFixture +import Control.Lens ((&), (.~), (^.)) +import Control.Monad (void) +import Data.Aeson qualified as Aeson +import Streamly.Data.Stream qualified as Stream +import System.Directory (doesFileExist) +import System.FilePath (()) +import Test.Tasty (TestTree, testGroup) +import Test.Tasty.HUnit (testCase, (@?=)) + +tests :: TestTree +#ifndef mingw32_HOST_OS +tests = testGroup "CliCancellationSpec: Codex owned batch cancellation" + [ testCase "simple descendant" $ run False False False, + testCase "SIGKILL-resistant descendant" $ run True False False, + testCase "repeated cancellation" $ run True True False, + testCase "active synthetic stream" $ run False False True, + testCase "schema lives until cleanup, then is removed" $ + withFixture False False schemaPrefix $ \fixture -> do + let provider = makeProvider fixture + opts = emptyOptions & #responseFormat .~ Just (JsonSchema (jsonSchemaFormat "fixture" (Aeson.object []))) + cancelAndJoin False ((provider ^. #complete) model emptyContext opts) fixture $ \pids -> do + assertStopped pids + path <- init <$> readFile (directory fixture "schema") + doesFileExist path >>= (@?= False) + readFile (directory fixture "schema-at-termination") >>= (@?= "present\n"), + testCase "evidence version-probe descendant" $ withFixture False False versionPrefix $ \fixture -> do + let provider = makeProvider fixture + opts = emptyOptions & #evidence .~ Just (evidenceRequest "cancellation-probe") + cancelAndJoin False ((provider ^. #complete) model emptyContext opts) fixture assertStopped + ] + where + run resistant repeated streaming = withFixture resistant False "" $ \fixture -> do + let provider = makeProvider fixture + action | streaming = void (Stream.toList ((provider ^. #stream) model emptyContext emptyOptions)) + | otherwise = void ((provider ^. #complete) model emptyContext emptyOptions) + cancelAndJoin repeated action fixture assertStopped + makeProvider fixture = Cli.codexCliProvider (Cli.defaultCodexCliConfig & #executable .~ executable fixture & #workingDir .~ Just (directory fixture)) + model = emptyModel & #api .~ OpenAICompletionsCli & #modelId .~ "fixture" + schemaPrefix = "while [ \"$#\" -gt 0 ]; do if [ \"$1\" = \"--output-schema\" ]; then schema=\"$2\"; echo \"$schema\" > schema; test -f \"$schema\" || exit 7; fi; shift; done\ntrap 'test -f \"$schema\" && echo present > schema-at-termination; exit 0' INT TERM" + versionPrefix = "if [ \"$1\" != \"--version\" ]; then printf '%s\\n' '{\"type\":\"item.completed\",\"item\":{\"type\":\"agent_message\",\"text\":\"ok\"}}'; exit 0; fi" +#else +tests = testGroup "CliCancellationSpec: POSIX fixtures require Darwin/Linux" [] +#endif diff --git a/test/CliProcessFixture.hs b/test/CliProcessFixture.hs new file mode 100644 --- /dev/null +++ b/test/CliProcessFixture.hs @@ -0,0 +1,158 @@ +{-# LANGUAGE CPP #-} + +-- | Real, offline subprocess fixtures shared by core and adapter regressions. +-- Keep the copies in each package identical: each source distribution must +-- contain its own test dependencies. +module CliProcessFixture + ( Fixture (..), + withFixture, + awaitReady, + assertStopped, + cancelAndJoin, + processState, + writeExecutable, + ) +where + +import Control.Concurrent (forkFinally, forkIO, killThread, threadDelay) +import Control.Concurrent.MVar (readMVar) +import Control.Concurrent.MVar qualified +import Control.Exception (SomeAsyncException, bracket, finally, fromException, try) +import Control.Monad (forM_, unless, void) +import Data.List (isInfixOf) +import System.Directory (doesFileExist, getPermissions, setOwnerExecutable, setPermissions) +import System.Exit (ExitCode (..)) +import System.FilePath (()) +import System.IO.Temp (withSystemTempDirectory) +import System.Process qualified as P +import System.Timeout (timeout) +import Test.Tasty.HUnit (assertBool, assertFailure) +#ifndef mingw32_HOST_OS +import System.Posix.Signals (sigKILL, signalProcess) +#endif + +data Fixture = Fixture + { directory :: FilePath, + executable :: FilePath, + parentFile :: FilePath, + childFile :: FilePath, + readyFile :: FilePath + } + +-- The optional prefix can implement a successful normal invocation and hang +-- only in --version. Every fixture sleeps 30s so success cannot be natural expiry. +withFixture :: Bool -> Bool -> String -> (Fixture -> IO a) -> IO a +withFixture resistant early prefix use = withSystemTempDirectory "baikai-cli-cancellation" $ \dir -> do + let sleeper = dir "sleeper" + parent = dir "parent" + child = dir "child" + ready = dir "ready" + body = + unlines $ + ["#!/bin/sh", prefix] + <> ["trap '' INT TERM" | resistant] + <> [ "echo $$ > '" <> parent <> "'", + "sh -c 'echo $$ > \"" <> child <> "\"; exec \"" <> sleeper <> "\" 30' &", + "while [ ! -s '" <> child <> "' ]; do sleep 0.01; done", + "echo ready > '" <> ready <> "'" + ] + <> [if early then "exit 0" else "wait"] + P.callProcess "/bin/ln" ["-s", "/bin/sleep", sleeper] + exe <- writeExecutable dir "vendor" body + let fixture = Fixture dir exe parent child ready + -- Capture process identities before use returns/fails; identity comparison in + -- emergency cleanup avoids signalling a PID reused after a fixture exits. + bracket (pure fixture) cleanupFixture use + +cleanupFixture :: Fixture -> IO () +cleanupFixture fixture = forM_ [childFile fixture, parentFile fixture] $ \file -> do + exists <- doesFileExist file + if not exists + then pure () + else do + pid <- read <$> readFile file + state <- processState pid + -- Both the script and sleeper executable have this invocation's unique + -- directory in their command line, including after exec. A reused PID + -- belonging to a different invocation cannot match that identity. + (_, identity, _) <- P.readProcessWithExitCode "/bin/ps" ["-p", show pid, "-o", "lstart=,command="] "" + let ours = (directory fixture <> "/") `isInfixOf` identity + unless (null state || 'Z' `elem` state || not ours) $ do + (_, current, _) <- P.readProcessWithExitCode "/bin/ps" ["-p", show pid, "-o", "lstart=,command="] "" + if current /= identity + then pure () + else do + killFixturePid pid + +killFixturePid :: Int -> IO () +#ifndef mingw32_HOST_OS +killFixturePid pid = void (try (signalProcess sigKILL (fromIntegral pid)) :: IO (Either IOError ())) +#else +killFixturePid _ = pure () +#endif + +writeExecutable :: FilePath -> String -> String -> IO FilePath +writeExecutable dir name body = do + let path = dir name + writeFile path body + perms <- getPermissions path + setPermissions path (setOwnerExecutable True perms) + pure path + +awaitReady :: Fixture -> IO (Int, Int) +awaitReady fixture = do + ready <- timeout 2000000 poll + case ready of + Nothing -> assertFailure "fixture did not publish readiness within two seconds" + Just () -> (,) <$> (read <$> readFile (parentFile fixture)) <*> (read <$> readFile (childFile fixture)) + where + poll = do + exists <- doesFileExist (readyFile fixture) + if exists then pure () else threadDelay 10000 >> poll + +processState :: Int -> IO String +processState pid = do + (code, out, err) <- P.readProcessWithExitCode "/bin/ps" ["-p", show pid, "-o", "stat="] "" + case code of + ExitSuccess -> pure out + ExitFailure 1 | null out && null err -> pure "" + _ -> assertFailure ("ps observation failed: " <> show (code, out, err)) + +assertStopped :: (Int, Int) -> IO () +assertStopped (parent, child) = do + parentState <- processState parent + assertBool ("direct child remains: " <> parentState) (null parentState) + childState <- processState child + assertBool ("descendant is running: " <> childState) (null childState || 'Z' `elem` childState) + -- Removal of an adopted zombie is diagnostic, distinct from termination. + unless (null childState) $ do + removed <- timeout 200000 (let poll = processState child >>= \s -> if null s then pure () else threadDelay 10000 >> poll in poll) + case removed of + Nothing -> putStrLn "fixture descendant stopped; adoptive parent has not yet reaped zombie" + Just () -> pure () + +cancelAndJoin :: Bool -> IO a -> Fixture -> ((Int, Int) -> IO ()) -> IO () +cancelAndJoin repeated action fixture after = do + done <- Control.Concurrent.MVar.newEmptyMVar + tid <- forkFinally (void action) (Control.Concurrent.MVar.putMVar done) + let stop = void (forkIO (killThread tid)) + ( do + pids <- awaitReady fixture + stop + if repeated then void (forkIO (threadDelay 40000 >> killThread tid)) else pure () + -- Keep the original 200ms observation without using it as acknowledgement. + threadDelay 200000 + diagnostic <- processState (snd pids) + putStrLn ("descendant state at 200ms: " <> show diagnostic) + terminal <- timeout 2800000 (readMVar done) + case terminal of + Just (Left e) | Just _ <- (fromException e :: Maybe SomeAsyncException) -> after pids + other -> assertFailure ("expected acknowledged async cancellation within 3s, got " <> show other) + ) + `finally` do + stop + cleanupFixture fixture + joined <- timeout 3000000 (readMVar done) + case joined of + Nothing -> assertFailure "fixture worker failed to finish after emergency cleanup" + Just _ -> pure () diff --git a/test/Main.hs b/test/Main.hs --- a/test/Main.hs +++ b/test/Main.hs @@ -30,6 +30,7 @@ ) import Baikai.Provider.OpenAI.Shape (describeThinkingShape) import BillingSpec qualified +import CliCancellationSpec qualified import CliEvidenceSpec qualified import Contract (assertErrorContract, assertOneErrorTerminal) import Control.Exception (bracket) @@ -100,6 +101,7 @@ responseFormatMappingTest, optionsMappingTest, BillingSpec.tests, + CliCancellationSpec.tests, CliEvidenceSpec.tests, ErrorClassSpec.tests, EvidenceSpec.tests,