diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,332 @@ ## [Unreleased] +## [baikai 0.8.0.0] - 2026-10-10 + +### Added + +- Catalog: `anthropic_claude_haiku_5_5` (Claude Haiku 5.5, released + 2026-10-07). It uses Messages with adaptive thinking, drops sampling options, + forwards forced tool choice, and has a 1,000,000-token context and + 128,000-token output limit. Prompts over 100,000 tokens, counting cache reads + and writes, are priced at $0.50/$0.05/$0.625/$2.50 per million input, cache + read, cache write and output tokens instead of $0.10/$0.01/$0.125/$0.50, and + one-hour cache writes at $1 instead of $0.20. Live text and tool cases + passed on 2026-10-10, and a live request confirmed forced tool choice with + adaptive thinking. + +### Changed (breaking) + +- `Baikai.Model.InputPriceTier` gains a third field, `longCacheWriteCost :: + Maybe Rational`: the one-hour cache-write rate for requests that cross that + tier. `resolveRates` now takes the long rate from the selected tier rather + than overriding every tier with the policy-level rate, and validation rejects + a policy whose policy-level long rate leaves any tier without one. Tier JSON + without the key decodes as `Nothing`. __Breaking__: positional constructions + of `InputPriceTier` need the new argument. + +### Changed + +- Catalog: the 2026-10-10 model refresh lowers `anthropic_claude_sonnet_5_5`'s + cache-read rate from $0.20 to $0.10 per million tokens, following Anthropic's + 2026-10-07 price cut, and corrects `anthropic_claude_sonnet_4_5`'s context + window from 1,000,000 to 200,000 tokens (its 1M beta ended on 2026-04-30). + Claude Haiku 5.5 was curated separately (see Added). No OpenAI model was + added. + +- Documentation: the 2026-10-02 model refresh found no new OpenAI or Anthropic + models; the catalog is unchanged. `docs/user/models-and-providers.md` notes + Anthropic's retirement of Claude Sonnet 4.5 on 2026-11-30 (deprecation of + `anthropic_claude_sonnet_4_5` is planned in plan 89 for a future release; + it remains in 0.8.0.0), + why the access-gated Claude Mythos models are not curated, and how GPT-6 + Astra's new ultrafast service tier is costed. + +### Fixed + +- The shared batch subprocess scope now owns POSIX process groups, joins pipe + readers, and synchronously reaps direct children before acknowledging + cancellation. Executable-version probes use the same scope. Darwin/Linux + coverage excludes processes that deliberately escape the group and adopted + descendant reaping; Windows retains direct-child cleanup. The provider fixes + ship in `baikai-claude 0.7.1.1` and `baikai-openai 0.7.1.1`; see + [plan 90](docs/plans/90-terminate-owned-batch-cli-process-groups-before-acknowledging-cancellation.md) + and [BUG-1](docs/bug-reports/batch-cli-cancellation-leaves-child-alive.md). + +- `baikai`'s source distribution now ships `data/models/*.json` and the test + suite's fixtures under `test/fixtures/`. Without them, running the suite from + the Hackage tarball failed 27 of 794 tests on missing files: the fixtures + themselves, plus the catalog round-trip test, whose `baikai-gen-models` run + had no model data to read. + +## [baikai-claude 0.7.1.1] - 2026-10-10 + +### Fixed + +- Batch `claude -p` calls stop owned POSIX process-group members and join pipe + readers before acknowledging cancellation, including active synthetic streams + and evidence probes. Preserves the original asynchronous exception. Resolves + [BUG-1](docs/bug-reports/batch-cli-cancellation-leaves-child-alive.md). + +### Changed + +- Requires `baikai ^>=0.8.0` for shared subprocess ownership and the new pricing + tier field. No provider API change. Haiku 5.5 request contracts are covered + by offline tests. + +## [baikai-openai 0.7.1.1] - 2026-10-10 + +### Fixed + +- Batch `codex exec` calls stop owned POSIX process-group members and join pipe + readers before acknowledging cancellation, including active synthetic streams, + repeated cancellation, and evidence probes. The temporary schema survives + until subprocess cleanup finishes. Resolves + [BUG-1](docs/bug-reports/batch-cli-cancellation-leaves-child-alive.md). + +### Changed + +- Requires `baikai ^>=0.8.0` for shared subprocess ownership and the new pricing + tier field. No provider API change. + +## [baikai-trace-otel 0.4.0.2] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. + +## [baikai-effectful 0.4.0.3] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. + +## [baikai-kit 0.4.0.1] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. + +## [baikai-agent 0.2.0.2] - 2026-10-10 + +### Changed + +- Requires `baikai ^>=0.8.0` for compatibility with the core pricing tier + change. No package API change. +- Raises the provider bounds to `baikai-claude ^>=0.7.1.1` and + `baikai-openai ^>=0.7.1.1`, so installing the tool selects the fixed batch + adapters. Binary regressions use the shared process fixture and cover both + cancellation and a 1MiB output flood. + +## [baikai-kit 0.4.0.0] - 2026-10-02 + +### Added + +- Per-item `visibility` (`tool-only` by default, or `shared`) for manifest + versions 1 and 2, with `kit install --shared`, `--tool-only`, and + `--accept-shared-codex`. Shared Claude skills and agents use tracked links; + tool-only Codex skills use tracked disabled entries in `config.toml`. +- `codexSessionArgs`, required in the `extraArgs` of a consumer's own Codex + launches to re-enable its tool-only skills. This is the one launcher step + beyond a dependency bump; the standard `kitConfig`/`kitCommandParser`/`runKit` + integration otherwise compiles unchanged. Codex custom agents cannot be + isolated and require explicit acceptance, or shared visibility. +- Requested and effective visibility in status (table and JSON), the + `visibility-broken` condition, and a migration note for legacy shared Codex + skills. Update repairs visibility even for items skipped for local edits; + uninstall removes only owned links and config entries. + +### Changed (breaking) + +- `SkillEntry` and `AgentEntry` gain `visibility`; `SidecarMeta` gains + `visibility`, `visibilitySource`, `sharedLinks`, and `codexDisabledSkills`; + `newSidecarMeta` takes these values. `RemovalOutcome` gains `linksRemoved` + and `configEntriesRemoved`; `StatusRow` gains `requestedVisibility` and + `effectiveVisibility`; `KitConfig` gains `confirmSharedCodex` (default + `Nothing` in `kitConfig`). +- `KitInstall` gains `InstallOptions`; `installItem` and `installFrom` take + it as their last argument. Use `defaultInstallOptions` to follow the + manifest. Explicit install flags persist across updates, while new + manifest-driven installs follow updated defaults. Legacy placement is kept. +- JSON adds list `visibility`, status `requestedVisibility` and + `effectiveVisibility`, and the new condition value. `formatVersion` stays + 1 because all additions preserve existing keys and their meaning. + +### Fixed + +- Codex installs refuse destinations without this tool's sidecar instead of + overwriting user or other-tool assets. Uninstall also preserves foreign + Codex assets. Shared Claude names are checked before any provider write. +- Codex config edits preserve UTF-8 text, comments and permissions, refuse + symlinks and invalid/conflicting config, and verify the semantic change + before atomic rename. Reused user-owned disabled entries survive uninstall. + Shared visibility refuses a user-owned entry that would keep the skill hidden. +- The source distribution ships the test suite's manifest fixtures and JSON + goldens, so the test suite passes when run from the Hackage tarball. Since + 0.3.0.0 it had failed there, because those files were missing. + +## [baikai 0.7.2.0] - 2026-09-30 + +### Added + +- Curated GPT-6.1 Sol on OpenAI Responses and Claude Sonnet 5.5 on Anthropic + Messages (`openai_gpt_6_1_sol`, `anthropic_claude_sonnet_5_5`), with endpoint + compatibility facts, standard prices, GPT-6.1 Sol's 272K-token context tier, + and Sonnet 5.5's one-hour cache-write rate. Sonnet 5.5 rejects forced tool + choice locally and has no fast mode. Both passed live text and function-tool + acceptance on 2026-09-30 through `/v1/responses` and `/v1/messages` + ([record](docs/validation/plan-85/2026-09-30-complete.json), + [plan 85](docs/plans/85-prove-gpt-6-1-sol-and-claude-sonnet-5-5-live-compatibility.md)). + The focused smoke runner gains the `sol61-*` and `sonnet55-*` cases. +- `Baikai.ResponseFormat.StructuredOutputSupport` + (`NativeJsonSchema | NoStructuredOutput`), `declaredStructuredOutput :: Api -> + StructuredOutputSupport`, and a `structuredOutput` field on `ApiProvider` + (default `NoStructuredOutput` in `apiProviderWith`), so a caller can ask + whether a transport enforces a schema without calling it. Every built-in + provider declares `NativeJsonSchema`. + +## [baikai-claude 0.7.1.0] - 2026-09-30 + +Requires `baikai >=0.7.2`. + +### Added + +- `claude -p` honours a `JsonSchema` response format (IR-11): it receives + `--json-schema ''` and the response text is the tool's validated + `structured_output`. A missing `structured_output` is a `DecodeFailure`; a + `claude` too old for the flag yields an `InvalidRequest` error with the exit + code rather than unconstrained text. `JsonObject`, `name` and `strict` are not + forwarded; requests without a schema render the same argument vector as + before. Both Claude providers declare `structuredOutput = NativeJsonSchema`. + +### Fixed + +- Anthropic adaptive `ThinkingHigh` now sends + `output_config.effort: "high"` instead of omitting the field. Claude Opus 5.5 + defaults to `medium`, so it previously ran `ThinkingHigh` at medium effort; + other adaptive models default to `high` and behave as before. Evidence for + these calls records `effortText = "high"` and no longer carries + `effort_omitted`, so strict evidence mode no longer refuses them. + `Baikai.Evidence.EffortOmitted` stays exported, and older records still decode. + +## [baikai-openai 0.7.1.0] - 2026-09-30 + +Requires `baikai >=0.7.2`. + +### Added + +- `codex exec` honours a `JsonSchema` response format (IR-11): the schema is + written to a temporary file passed as `--output-schema ` and deleted + however the call ends. A `codex` too old for the flag yields an + `InvalidRequest` error with the exit code rather than unconstrained text. + `JsonObject`, `name` and `strict` are not forwarded; requests without a schema + render the same argument vector as before. All three OpenAI providers declare + `structuredOutput = NativeJsonSchema`. +- `codexCliCommandWith`, which renders the `codex exec` + vector with a given `--output-schema` file. `codexCliCommand` is unchanged + and never renders the flag. + +## [baikai 0.7.1.0] - 2026-09-23 + +### Added + +- Curated GPT-6 Sol and Luna on OpenAI Responses and Claude Opus 5.5 on + Anthropic Messages (`openai_gpt_6_sol`, `openai_gpt_6_luna`, + `anthropic_claude_opus_5_5`), with endpoint compatibility and standard, + long-context, cache-duration, and fast-mode prices where applicable. All + three passed live acceptance on 2026-09-23. Sol and Luna dispatch to + `OpenAIResponses`, so calling them requires the + `Baikai.Provider.OpenAI.Responses.register` call that `baikai-openai 0.7.0.0` + introduced. + +## [baikai-kit 0.3.0.0] - 2026-09-23 + +Closes four improvement requests from the tools that ship `baikai-kit` as their +`kit` command: project scope resolves from a configurable root (IR-8), +`kit status` reports local edits separately from upstream drift (IR-7), +`kit install` without a name asks a tool-supplied chooser (IR-6), and `list`, +`status` and `update` print versioned JSON (IR-9). A consumer raising its bound +builds `KitConfig` with `kitConfig`, passes it to `kitCommandParser`, and +matches on `StatusRow.conditions`; each break is at a call site the compiler +names. + +### Added + +- `baikai-kit`: `KitConfig.projectRoot :: IO FilePath` says where project scope + lives. Install, status, update, uninstall and `agentDirsForSession` all derive + project-scope paths from it, so they agree whichever subdirectory a command + runs from. `kitConfig` builds a configuration with every optional field at its + default (project scope is the current directory, as before); + `projectRootByMarkers [".git", ".mytool"]` is a ready-made resolver that walks + up to the nearest marker and falls back to the current directory, and + `findProjectRoot` is the underlying walk. Resolves IR-8. + +- `baikai-kit`: `kit status` reports local edits. It runs the same + installed-file check `kit update` uses to skip an item, and shows + `modified` for an edited copy and `edits-unknown` for one whose sidecar + predates the installed-file hash. The check is exported as + `checkLocalEdits`, returning `LocalEdits` (`Unedited`, `Edited`, + `EditsUnknown`). Resolves IR-7. + +- `baikai-kit`: `kit install` with no name asks a chooser the tool supplies in + the new `KitConfig.chooseItem :: Maybe (KitManifest -> IO (Maybe Text))` + field. The engine refreshes the kit, passes the whole manifest, and installs + what the chooser returns; a cancelled choice prints + `No item chosen; nothing installed.` and exits 0. With no chooser (the + `kitConfig` default) the command fails with the new `KitItemNameRequired` + error, which tells the user to pass `NAME`. The engine ships no picker. + `KitCommand` derives `Eq`, and `kit install --help` names the tool's + `./agents` directory. Resolves IR-6. + +- `baikai-kit`: `kit list`, `kit status` and `kit update` accept `--json` and + print exactly one versioned JSON document on stdout + (`{"formatVersion": 1, "document": "kit-list" | "kit-status" | "kit-update", …}`); + warnings and the first-clone notice go to stderr, and a failed command writes + nothing to stdout. The shapes are written by explicit encoders — + `Baikai.Kit.Json.listDocument`, `statusDocument`, `updateDocument`, and + `kitJsonFormatVersion` — so library callers get the same values, and they are + pinned by golden tests. `Baikai.Kit.Command.OutputFormat` selects the mode, + and `Baikai.Kit.Status.InstalledCopy` / `installedCopies` report where each + item is installed. Resolves IR-9. + +### Changed + +- `baikai-kit`: `KitConfig` gains the strict field `projectRoot`, so a record + literal must set it; build the configuration with + `kitConfig toolName repoUrl providers` instead and override fields with record + update syntax. `KitConfig`'s `Show` instance is now hand-written and prints + `` for the resolver. __Breaking__. + +- `baikai-kit`: `kit status` conditions compose, and `dirty` is renamed + `changed-upstream` (it meant the upstream sources changed without a version + bump, not local edits); `dirty+outdated` now reads + `outdated+changed-upstream`. `StatusRow.state :: KitState` is replaced by + `StatusRow.conditions :: [KitCondition]` (sorted; empty means up to date), + `renderState` by `conditionLabel` and `renderConditions`, and `classify` + returns `[KitCondition]`. `KitUpToDate`, `KitDirty` and `KitDirtyOutdated` + are gone; match on the list instead. __Breaking__. + +- `baikai-kit`: `KitInstall` takes `Maybe Text` (`Nothing` asks the chooser), + `kitCommandParser` takes the `KitConfig` (migration: `kitCommandParser` + becomes `kitCommandParser myKitConfig`), `KitConfig` gains the `chooseItem` + field (set by `kitConfig`), and `KitError` gains `KitItemNameRequired`. + __Breaking__. + +- `baikai-kit`: `KitList`, `KitStatus` and `KitUpdate` gain a trailing + `OutputFormat` field (`HumanOutput` for the previous behaviour). + __Breaking__. + +## [baikai-effectful 0.4.0.2] - 2026-09-15 + +### Changed (dependencies) + +- Requires `effectful-core ^>=2.7` (was `^>=2.6`). No API change: none of the + 2.7 breaking APIs (`LocalEnv`'s second type parameter, `SharedSuffix`, + `KnownEffects`, the ticked strict modules) are used. + ## [baikai 0.7.0.0] - 2026-09-08 ### Added diff --git a/baikai-agent.cabal b/baikai-agent.cabal --- a/baikai-agent.cabal +++ b/baikai-agent.cabal @@ -1,7 +1,7 @@ -cabal-version: 3.4 -name: baikai-agent -version: 0.2.0.1 -synopsis: Unattended coding-agent runs for the Baikai abstraction +cabal-version: 3.4 +name: baikai-agent +version: 0.2.0.2 +synopsis: Unattended coding-agent runs for the Baikai abstraction description: Runs a local coding-agent command-line tool with no terminal and no human present: delivers the prompt on standard input, drains both @@ -10,21 +10,27 @@ vocabulary lives in @Baikai.Agent@ in the core package, and vendor packages own the translation into each tool's argument vector. -category: AI -license: BSD-3-Clause -license-file: LICENSE -author: Nadeem Bitar -maintainer: nadeem@gmail.com -copyright: (c) 2026 Nadeem Bitar -build-type: Simple -tested-with: GHC ==9.12.4 +category: AI +license: BSD-3-Clause +license-file: LICENSE +author: Nadeem Bitar +maintainer: nadeem@gmail.com +copyright: (c) 2026 Nadeem Bitar +build-type: Simple +tested-with: ghc ==9.12.4 extra-doc-files: CHANGELOG.md common common-options ghc-options: - -Wall -Wcompat -Widentities -Wincomplete-uni-patterns - -Wincomplete-record-updates -Wredundant-constraints - -fhide-source-paths -Wmissing-export-lists -Wpartial-fields + -Wall + -Wcompat + -Widentities + -Wincomplete-uni-patterns + -Wincomplete-record-updates + -Wredundant-constraints + -fhide-source-paths + -Wmissing-export-lists + -Wpartial-fields -Wmissing-deriving-strategies -- Exhaustiveness is an error, not a warning. A non-exhaustive match @@ -39,10 +45,11 @@ -- fail the build on warnings that are stylistic or that a future GHC -- invents, and would push people toward blanket suppression. ghc-options: - -Werror=incomplete-patterns -Werror=incomplete-uni-patterns + -Werror=incomplete-patterns + -Werror=incomplete-uni-patterns -Werror=incomplete-record-updates - default-language: GHC2024 + default-language: GHC2024 default-extensions: DeriveAnyClass DuplicateRecordFields @@ -50,75 +57,74 @@ OverloadedStrings library - import: common-options - hs-source-dirs: src + import: common-options + hs-source-dirs: src exposed-modules: Baikai.Agent.Cli Baikai.Agent.Config Baikai.Agent.Run build-depends: - , aeson ^>=2.2 - , baikai ^>=0.7.0 - , baikai-claude ^>=0.7 - , baikai-openai ^>=0.7 - , base >=4.20 && <5 - , bytestring ^>=0.12 - , containers ^>=0.7 - , directory ^>=1.3 - , filepath ^>=1.5 - , generic-lens ^>=2.3 - , lens ^>=5.3 - , optparse-applicative ^>=0.19 - , process ^>=1.6 - , settei ^>=0.2 - , settei-env ^>=0.2 - , settei-kdl ^>=0.2 - , settei-optparse-applicative ^>=0.2 - , streamly-core >=0.3 && <0.5 - , text ^>=2.1 - , time ^>=1.14 + aeson ^>=2.2, + baikai ^>=0.8.0, + baikai-claude ^>=0.7.1.1, + baikai-openai ^>=0.7.1.1, + base >=4.20 && <5, + bytestring ^>=0.12, + containers ^>=0.7, + directory ^>=1.3, + filepath ^>=1.5, + generic-lens ^>=2.3, + lens ^>=5.3, + optparse-applicative ^>=0.19, + process ^>=1.6, + settei ^>=0.2, + settei-env ^>=0.2, + settei-kdl ^>=0.2, + settei-optparse-applicative ^>=0.2, + streamly-core >=0.3 && <0.5, + text ^>=2.1, + time ^>=1.14, -- The process package offers a group-wide interrupt but no group-wide -- terminate, which a timeout needs to reach a coding agent's own -- children. Where POSIX signals exist, use them. if !os(windows) build-depends: unix ^>=2.8 - cpp-options: -DBAIKAI_POSIX_SIGNALS + cpp-options: -DBAIKAI_POSIX_SIGNALS executable baikai - import: common-options + import: common-options hs-source-dirs: app - main-is: Main.hs - + main-is: Main.hs -- The runner waits on the child with waitForProcess under -- System.Timeout and drains the child's pipes on forked threads. -- Without the threaded runtime that wait blocks every Haskell thread, -- so the configured timeout can never fire and a chatty child -- deadlocks the run on a full pipe. BinaryTests proves it against this -- executable rather than against the suite's own runtime. - ghc-options: -threaded - + ghc-options: -threaded -- Deliberately thin: everything real lives in Baikai.Agent.Cli so the -- whole command-line surface is reachable from the test suite without -- spawning the built binary. build-depends: - , baikai-agent - , base >=4.20 && <5 - , bytestring ^>=0.12 - , generic-lens ^>=2.3 - , lens ^>=5.3 - , optparse-applicative ^>=0.19 - , settei-env ^>=0.2 - , text ^>=2.1 + baikai-agent, + base >=4.20 && <5, + bytestring ^>=0.12, + generic-lens ^>=2.3, + lens ^>=5.3, + optparse-applicative ^>=0.19, + settei-env ^>=0.2, + text ^>=2.1, test-suite baikai-agent-test - import: common-options - type: exitcode-stdio-1.0 - hs-source-dirs: test - main-is: Main.hs + import: common-options + type: exitcode-stdio-1.0 + hs-source-dirs: test + main-is: Main.hs other-modules: BinaryTests + CliProcessFixture CliTests ConfigTests EvidenceTests @@ -126,7 +132,9 @@ -- -threaded is not optional here: the runner forks threads to drain -- pipes and relies on System.Timeout interrupting a blocking wait. - ghc-options: -threaded -with-rtsopts=-N + ghc-options: + -threaded + -with-rtsopts=-N -- BinaryTests spawns the shipped executable rather than exercising the -- library in this process. Naming it here makes cabal build it before @@ -134,24 +142,27 @@ -- runs the binary a user installs instead of this suite's own runtime. build-tool-depends: baikai-agent:baikai build-depends: - , aeson - , baikai - , baikai-agent - , base - , bytestring - , containers - , directory - , filepath - , generic-lens - , lens - , optparse-applicative - , process - , settei - , settei-env - , settei-kdl - , settei-optparse-applicative - , tasty - , tasty-hunit - , temporary - , text - , time + aeson, + baikai, + baikai-agent, + base, + bytestring, + containers, + directory, + filepath, + generic-lens, + lens, + optparse-applicative, + process, + settei, + settei-env, + settei-kdl, + settei-optparse-applicative, + tasty, + tasty-hunit, + temporary, + text, + time, + + if !os(windows) + build-depends: unix ^>=2.8 diff --git a/test/BinaryTests.hs b/test/BinaryTests.hs --- a/test/BinaryTests.hs +++ b/test/BinaryTests.hs @@ -299,11 +299,11 @@ contents <- readFile path pure [line | line <- lines contents, not (null line)] --- | Fail unless every recorded process is gone. +-- | Fail unless every recorded process has stopped. -- -- Polled rather than checked once: the runner returns as soon as it has -- reaped the group's leader, and the kernel may take a moment longer to --- finish reaping a grandchild. +-- stop a grandchild. Adopted zombies need not disappear in that interval. awaitAllGone :: [String] -> IO () awaitAllGone pids = go (40 :: Int) where @@ -320,12 +320,15 @@ (_, _) -> threadDelay 50000 >> go (n - 1) filterM' p xs = concat <$> mapM (\x -> (\keep -> [x | keep]) <$> p x) xs --- | Whether a process exists, asked with the null signal: @kill -0@ --- delivers nothing and fails when no such process is there. +-- | A zombie is stopped even if its adoptive parent has not reaped it. +-- Null signals include zombies and would incorrectly fail on container PID 1. processAlive :: String -> IO Bool processAlive pid = do - (code, _, _) <- P.readProcessWithExitCode "kill" ["-0", pid] "" - pure (code == ExitSuccess) + (code, state, err) <- P.readProcessWithExitCode "/bin/ps" ["-p", pid, "-o", "stat="] "" + case code of + ExitSuccess -> pure (not (null state) && 'Z' `notElem` state) + ExitFailure 1 | null state && null err -> pure False + _ -> assertFailure ("ps observation failed: " <> show (code, state, err)) -- -------------------------------------------------------------------- -- Small helpers diff --git a/test/CliProcessFixture.hs b/test/CliProcessFixture.hs new file mode 100644 --- /dev/null +++ b/test/CliProcessFixture.hs @@ -0,0 +1,158 @@ +{-# LANGUAGE CPP #-} + +-- | Real, offline subprocess fixtures shared by core and adapter regressions. +-- Keep the copies in each package identical: each source distribution must +-- contain its own test dependencies. +module CliProcessFixture + ( Fixture (..), + withFixture, + awaitReady, + assertStopped, + cancelAndJoin, + processState, + writeExecutable, + ) +where + +import Control.Concurrent (forkFinally, forkIO, killThread, threadDelay) +import Control.Concurrent.MVar (readMVar) +import Control.Concurrent.MVar qualified +import Control.Exception (SomeAsyncException, bracket, finally, fromException, try) +import Control.Monad (forM_, unless, void) +import Data.List (isInfixOf) +import System.Directory (doesFileExist, getPermissions, setOwnerExecutable, setPermissions) +import System.Exit (ExitCode (..)) +import System.FilePath (()) +import System.IO.Temp (withSystemTempDirectory) +import System.Process qualified as P +import System.Timeout (timeout) +import Test.Tasty.HUnit (assertBool, assertFailure) +#ifndef mingw32_HOST_OS +import System.Posix.Signals (sigKILL, signalProcess) +#endif + +data Fixture = Fixture + { directory :: FilePath, + executable :: FilePath, + parentFile :: FilePath, + childFile :: FilePath, + readyFile :: FilePath + } + +-- The optional prefix can implement a successful normal invocation and hang +-- only in --version. Every fixture sleeps 30s so success cannot be natural expiry. +withFixture :: Bool -> Bool -> String -> (Fixture -> IO a) -> IO a +withFixture resistant early prefix use = withSystemTempDirectory "baikai-cli-cancellation" $ \dir -> do + let sleeper = dir "sleeper" + parent = dir "parent" + child = dir "child" + ready = dir "ready" + body = + unlines $ + ["#!/bin/sh", prefix] + <> ["trap '' INT TERM" | resistant] + <> [ "echo $$ > '" <> parent <> "'", + "sh -c 'echo $$ > \"" <> child <> "\"; exec \"" <> sleeper <> "\" 30' &", + "while [ ! -s '" <> child <> "' ]; do sleep 0.01; done", + "echo ready > '" <> ready <> "'" + ] + <> [if early then "exit 0" else "wait"] + P.callProcess "/bin/ln" ["-s", "/bin/sleep", sleeper] + exe <- writeExecutable dir "vendor" body + let fixture = Fixture dir exe parent child ready + -- Capture process identities before use returns/fails; identity comparison in + -- emergency cleanup avoids signalling a PID reused after a fixture exits. + bracket (pure fixture) cleanupFixture use + +cleanupFixture :: Fixture -> IO () +cleanupFixture fixture = forM_ [childFile fixture, parentFile fixture] $ \file -> do + exists <- doesFileExist file + if not exists + then pure () + else do + pid <- read <$> readFile file + state <- processState pid + -- Both the script and sleeper executable have this invocation's unique + -- directory in their command line, including after exec. A reused PID + -- belonging to a different invocation cannot match that identity. + (_, identity, _) <- P.readProcessWithExitCode "/bin/ps" ["-p", show pid, "-o", "lstart=,command="] "" + let ours = (directory fixture <> "/") `isInfixOf` identity + unless (null state || 'Z' `elem` state || not ours) $ do + (_, current, _) <- P.readProcessWithExitCode "/bin/ps" ["-p", show pid, "-o", "lstart=,command="] "" + if current /= identity + then pure () + else do + killFixturePid pid + +killFixturePid :: Int -> IO () +#ifndef mingw32_HOST_OS +killFixturePid pid = void (try (signalProcess sigKILL (fromIntegral pid)) :: IO (Either IOError ())) +#else +killFixturePid _ = pure () +#endif + +writeExecutable :: FilePath -> String -> String -> IO FilePath +writeExecutable dir name body = do + let path = dir name + writeFile path body + perms <- getPermissions path + setPermissions path (setOwnerExecutable True perms) + pure path + +awaitReady :: Fixture -> IO (Int, Int) +awaitReady fixture = do + ready <- timeout 2000000 poll + case ready of + Nothing -> assertFailure "fixture did not publish readiness within two seconds" + Just () -> (,) <$> (read <$> readFile (parentFile fixture)) <*> (read <$> readFile (childFile fixture)) + where + poll = do + exists <- doesFileExist (readyFile fixture) + if exists then pure () else threadDelay 10000 >> poll + +processState :: Int -> IO String +processState pid = do + (code, out, err) <- P.readProcessWithExitCode "/bin/ps" ["-p", show pid, "-o", "stat="] "" + case code of + ExitSuccess -> pure out + ExitFailure 1 | null out && null err -> pure "" + _ -> assertFailure ("ps observation failed: " <> show (code, out, err)) + +assertStopped :: (Int, Int) -> IO () +assertStopped (parent, child) = do + parentState <- processState parent + assertBool ("direct child remains: " <> parentState) (null parentState) + childState <- processState child + assertBool ("descendant is running: " <> childState) (null childState || 'Z' `elem` childState) + -- Removal of an adopted zombie is diagnostic, distinct from termination. + unless (null childState) $ do + removed <- timeout 200000 (let poll = processState child >>= \s -> if null s then pure () else threadDelay 10000 >> poll in poll) + case removed of + Nothing -> putStrLn "fixture descendant stopped; adoptive parent has not yet reaped zombie" + Just () -> pure () + +cancelAndJoin :: Bool -> IO a -> Fixture -> ((Int, Int) -> IO ()) -> IO () +cancelAndJoin repeated action fixture after = do + done <- Control.Concurrent.MVar.newEmptyMVar + tid <- forkFinally (void action) (Control.Concurrent.MVar.putMVar done) + let stop = void (forkIO (killThread tid)) + ( do + pids <- awaitReady fixture + stop + if repeated then void (forkIO (threadDelay 40000 >> killThread tid)) else pure () + -- Keep the original 200ms observation without using it as acknowledgement. + threadDelay 200000 + diagnostic <- processState (snd pids) + putStrLn ("descendant state at 200ms: " <> show diagnostic) + terminal <- timeout 2800000 (readMVar done) + case terminal of + Just (Left e) | Just _ <- (fromException e :: Maybe SomeAsyncException) -> after pids + other -> assertFailure ("expected acknowledged async cancellation within 3s, got " <> show other) + ) + `finally` do + stop + cleanupFixture fixture + joined <- timeout 3000000 (readMVar done) + case joined of + Nothing -> assertFailure "fixture worker failed to finish after emergency cleanup" + Just _ -> pure () diff --git a/test/Main.hs b/test/Main.hs --- a/test/Main.hs +++ b/test/Main.hs @@ -15,9 +15,9 @@ import Baikai.Agent.Run (runAgentCommand, timeoutMicros) import Baikai.Evidence (noThinkingRequested) import BinaryTests (binaryTests) +import CliProcessFixture qualified as Fixture import CliTests (cliTests) import ConfigTests (configTests) -import Control.Concurrent (threadDelay) import Control.Lens ((&), (.~), (^.)) import Data.ByteString qualified as BS import Data.ByteString.Char8 qualified as BS8 @@ -28,8 +28,7 @@ import EvidenceTests (evidenceTests) import PublicSurfaceSpec qualified import System.Directory - ( doesFileExist, - findExecutable, + ( findExecutable, getPermissions, setOwnerExecutable, setPermissions, @@ -37,6 +36,7 @@ import System.Exit (ExitCode (..)) import System.FilePath (()) import System.IO.Temp (withSystemTempDirectory) +import System.Process qualified as P import Test.Tasty (TestTree, defaultMain, testGroup) import Test.Tasty.HUnit (assertBool, assertFailure, testCase, (@?=)) @@ -239,25 +239,17 @@ processGroupTest :: TestTree processGroupTest = - testCase "kills grandchildren when the group is terminated" - $ - -- The most valuable test here and the easiest to omit: it is what - -- proves a coding agent's own child processes die with it. - withFakeExecutable - "spawns-a-child" - "#!/bin/sh\n(sleep 3; touch \"$1\") &\nsleep 5\n" - $ \dir exe -> do - let marker = dir "grandchild-survived" - req = capturingRequest dir "ignored" & #timeout .~ Just 1 - outcome <- runPlain req (stdinCommand exe [marker] "ignored") + testCase "kills grandchildren when the group is terminated" $ + Fixture.withFixture False False "" $ \fixture -> do + let req = capturingRequest (Fixture.directory fixture) "ignored" & #timeout .~ Just 1 + outcome <- runPlain req (stdinCommand (Fixture.executable fixture) [] "ignored") case outcome of Left (RunTimedOut _) -> pure () other -> assertFailure ("expected RunTimedOut, got: " <> show other) - -- Wait past the grandchild's delay before checking, or the file - -- would be absent merely because it is early. - waitSeconds 4 - survived <- doesFileExist marker - assertBool "the grandchild was terminated with its group" (not survived) + -- A 30-second child cannot finish within the runner's timeout and + -- signal grace periods. Check stopped state immediately on return; + -- adopted zombies have stopped even when PID 1 has not reaped them. + Fixture.awaitReady fixture >>= Fixture.assertStopped -- | A timed-out run reports what it drained before the kill. -- @@ -388,16 +380,16 @@ -- No shipped renderer selects this transport, so a fixture is the -- only place the two-sided contract can be observed. The script -- echoes its argument and appends whatever standard input it can - -- read, which must be nothing. + -- read, which must not include a second copy of that prompt. withFakeExecutable "echo-arg" -- Shift past the -- separator the way a real tool's own argument -- parser would, then echo the prompt and append whatever standard - -- input can be read, which must be nothing. Reading fails outright - -- because this transport gives the child no standard input at all, - -- and that failure is tolerated so the script's own exit code - -- still reports success. - "#!/bin/sh\n[ \"$1\" = \"--\" ] && shift\nprintf '%s' \"$1\"\ncat 2>/dev/null || true\n" + -- input contains that prompt. NoStream closes stdin; on Rosetta a + -- spawned reader can reopen its own ELF on descriptor zero, so + -- assert the transport contract (no duplicate prompt) directly. + -- A read failure is tolerated because closed stdin is expected. + "#!/bin/sh\n[ \"$1\" = \"--\" ] && shift\nprintf '%s' \"$1\"\nif grep -qF -- \"$1\" 2>/dev/null; then printf duplicate-prompt; fi\nexit 0\n" $ \dir exe -> do let promptBody = "the prompt is an argument" :: Text.Text cmd = @@ -407,6 +399,9 @@ promptTransport = PromptAsArgument, promptText = promptBody } + -- Verify the fixture detects a prompt incorrectly sent through stdin. + (_, duplicated, _) <- P.readProcessWithExitCode exe ["--", Text.unpack promptBody] (Text.unpack promptBody) + duplicated @?= Text.unpack promptBody <> "duplicate-prompt" result <- runPlain (capturingRequest dir promptBody) cmd >>= expectRan capturedBytes (result ^. #stdout) @?= Just (Text.encodeUtf8 promptBody) @@ -434,6 +429,3 @@ basename :: FilePath -> FilePath basename = reverse . takeWhile (/= '/') . reverse - -waitSeconds :: Int -> IO () -waitSeconds seconds = threadDelay (seconds * 1000000)