llm-simple 0.4.2.0 → 0.4.3.0

raw patch · 86 files changed

+6215/−926 lines, 86 filesPVP: major bump suggested

API removals or changes: PVP suggests a major version bump

API changes (from Hackage documentation)

Files

• CHANGELOG.md +91/−0
• Readme.md +2/−2
• internal/LLM/Tools/Internal/TextFile.hs +164/−0
• llm-simple.cabal +24/−2
• model-catalog.json +0/−1
• src/LLM.hs +3/−3
• src/LLM/Agent.hs +4/−4
• src/LLM/Agent/Events.hs +1/−1
• src/LLM/Agent/Generate.hs +18/−11
• src/LLM/Agent/GenerateObject.hs +1/−1
• src/LLM/Agent/ToolUtils.hs +12/−12
• src/LLM/Agent/Tools/HistoryTool.hs +48/−4
• src/LLM/Agent/Types.hs +7/−7
• src/LLM/Core.hs +1/−0
• src/LLM/Core/Abort.hs +1/−1
• src/LLM/Core/Internal/Body.hs +8/−6
• src/LLM/Core/Internal/Schema.hs +237/−0
• src/LLM/Core/Internal/Transport.hs +36/−0
• src/LLM/Core/LLMProvider.hs +7/−7
• src/LLM/Core/ProviderUtils.hs +51/−56
• src/LLM/Core/SSE.hs +16/−5
• src/LLM/Core/Types.hs +62/−15
• src/LLM/Core/Usage.hs +2/−2
• src/LLM/Core/Utils.hs +1/−1
• src/LLM/Generate/Generate.hs +7/−188
• src/LLM/Generate/GenerateObject.hs +5/−1
• src/LLM/Generate/GenerateUtils.hs +11/−286
• src/LLM/Generate/Internal/Generate.hs +223/−0
• src/LLM/Generate/Internal/GenerateUtils.hs +334/−0
• src/LLM/Generate/ModelConfig.hs +6/−6
• src/LLM/Generate/Types.hs +6/−6
• src/LLM/Load/FsTools.hs +1/−1
• src/LLM/Load/LoadGateways.hs +1/−1
• src/LLM/Providers/Claude.hs +7/−18
• src/LLM/Providers/DeepSeek.hs +5/−16
• src/LLM/Providers/Gemini.hs +12/−26
• src/LLM/Providers/Ollama.hs +5/−14
• src/LLM/Providers/OpenAI.hs +7/−17
• src/LLM/Tools/FindFiles.hs +2/−2
• src/LLM/Tools/FsConfig.hs +2/−2
• src/LLM/Tools/FsLimits.hs +18/−2
• src/LLM/Tools/Grep.hs +66/−26
• src/LLM/Tools/Internal/TextFile.hs +0/−124
• src/LLM/Tools/Internal/WalkBudget.hs +20/−1
• src/LLM/Tools/MultiReplaceInFile.hs +4/−1
• src/LLM/Tools/ReadFilePaginated.hs +16/−34
• src/LLM/Tools/Readdir.hs +1/−1
• test/LLM/ChatSpec.hs +34/−0
• test/LLM/FsToolsSpec.hs +32/−2
• test/LLM/GeminiSpec.hs +2/−1
• test/LLM/GenerateObjectSpec.hs +65/−2
• test/LLM/HistoryToolSpec.hs +40/−0
• test/LLM/HttpConfigSpec.hs +48/−7
• test/LLM/LoadSpec.hs +10/−0
• test/LLM/OpenAISpec.hs +112/−0
• test/LLM/OptionalFields.hs +76/−0
• test/LLM/TextFileSpec.hs +86/−0
• test/LLM/TypesSpec.hs +43/−0
• test/fixtures/claude-conversation-generated.json +364/−0
• test/fixtures/claude-conversation-streamed.json +332/−0
• test/fixtures/claude-max-tokens-tool.sse +18/−0
• test/fixtures/claude-midstream-error.sse +12/−0
• test/fixtures/claude-text.json +17/−0
• test/fixtures/claude-thinking-tool-use.json +28/−0
• test/fixtures/claude-thinking-tool-use.sse +38/−0
• test/fixtures/claude-tool-use.json +23/−0
• test/fixtures/deepseek-conversation-generated.json +414/−0
• test/fixtures/deepseek-conversation-streamed.json +367/−0
• test/fixtures/gemini-conversation-generated.json +466/−0
• test/fixtures/gemini-conversation-streamed.json +420/−0
• test/fixtures/gemini-text.json +20/−0
• test/fixtures/gemini-tool-use.json +23/−0
• test/fixtures/ollama-catalog.json +32/−0
• test/fixtures/ollama-conversation-generated.json +360/−0
• test/fixtures/ollama-conversation-streamed.json +338/−0
• test/fixtures/ollama-multi-tool-chunk.sse +6/−0
• test/fixtures/openai-conversation-generated.json +400/−0
• test/fixtures/openai-conversation-streamed.json +338/−0
• test/fixtures/openai-length-tool.sse +8/−0
• test/fixtures/openai-midstream-error.sse +6/−0
• test/fixtures/openai-multi-tool-chunk.sse +8/−0
• test/fixtures/openai-text.json +20/−0
• test/fixtures/openai-tool-use.json +30/−0
• test/fixtures/providers-ollama-only.json +7/−0
• test/fixtures/providers-openrouter-only.json +8/−0
• test/fixtures/providers-openrouter.json +8/−0

@@ -5,6 +5,96 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.4.3.0] - 2026-11-04++### Added++- `LLM.Core.Types.validateConversation` (also in `LLM.Core`): problems in+  the tool-call structure of a conversation, by turn index.+- `LLM.Tools.FsLimits.maxSearchBytes` (64 MiB): the file bytes one `grep`+  call may read.+- `LLM.Tools.MultiReplaceInFile.Replacement`, the type of one edit in+  `MultiReplaceInFileToolArgs`, exported without its constructor. Before,+  the exported field `_mrifReplacements :: [Replacement]` named a type+  that could not be imported.++### Fixed++- `grep` reads at most `maxSearchBytes` (64 MiB) of files per call. Only+  the text files it searches count; binary files and files over the 1 MiB+  per-file cap are not read and do not count. A search that runs out+  stops before the next file and says so in the header ("stopped after+  reading N bytes; narrow the path or pattern"), with the file count shown+  as `N+`. Before, a search with few matches could read 20,000 files of+  1 MiB, about 20 GiB, in one call. Each file is now read only up to the+  size charged for it, so a file that grows during the search is not read+  past the cap. **Behaviour change** for searches over more than 64 MiB.+- A `get_history` page is at most `maxOutputBytes` (100 KiB). Before,+  user and assistant text was rendered in full, so one page could be+  larger than the context window that `agContextWindow` protects. A page+  over the cap keeps its newest turns and ends with a line saying how many+  older turns were left out; a newest turn that is over the cap on its own+  is cut and marked (`[turn cut: N bytes, first M shown]`). The paging is+  unchanged: the turns left out are not moved to the next chunk.+  **Behaviour change** for pages over 100 KiB.+- A streamed response is at most 32 MiB in total, the same cap as a+  non-streaming body. Before, only one event (8 MiB) was bounded, so an+  endpoint sending small events forever kept the call reading, and the+  parsers keeping every delta, without end. Past the cap the stream ends+  with `ParseError "response body exceeds 33554432 bytes"`, as for an+  oversized event. **Behaviour change** for streams over 32 MiB.+- Every generation entry point also checks the tool-call structure of the+  request's messages (`validateConversation`): a tool turn that does not+  directly follow an assistant turn with tool calls, tool calls not+  followed by a tool turn, a call without a result, a call answered twice,+  a result for an unknown call ID, or a call ID used twice in one turn.+  Each problem is logged like an invalid turn+  (`onLog Warn "turn <n>: <reason>"`, in turn order) and the request is+  still sent. Providers rejected such histories or dropped parts of them,+  each differently. **0.5 will reject them** with an error.+- An agent run (`LLM.Agent.generateText`, `streamText`) checks the input+  conversation once, at the start of the run, instead of on every model+  call, so an invalid turn is logged once per run instead of once per+  round. `n` in `turn <n>: <reason>` now counts in the input conversation,+  also when `agContextWindow` sends only part of it. The turns the agent+  adds come from providers and are not checked.++- The source distribution (the Hackage tarball) includes the test+  fixtures (`test/fixtures/*.json`, `test/fixtures/*.sse`). Before, the+  test suite run from it failed 41 tests on missing files. CI now also+  runs the tests from the unpacked tarball.+- Structured output with OpenAI (and Ollama, which shares the schema):+  strict mode lists every property as required, so a field from+  `optionalField`, `optionalFieldWithDefault` or+  `optionalFieldWithOmittedDefault` had to be filled in by the model.+  `normalizeSchemaOpenAI` now makes such a property nullable+  (`anyOf: [<schema>, {"type": "null"}]`), in nested objects, array+  items, union branches and `$defs` too. `genObject` and+  `LLM.Agent.generateObject` drop a `null` for a property the codec has+  optional and not nullable before decoding, for every provider, so the+  field gets its default; before, that `null` was a decode error.+  Nullable fields (`Maybe` with `requiredField`, `optionalFieldOrNull`)+  are unchanged. `genObjectUntyped` and `generateObjectUntyped` return+  the provider's JSON as before, `null`s included. **Behaviour change**+  for structured-output types with optional fields.+- Structured output with OpenAI failed with 400 ("'oneOf' is not+  permitted") for any type with an `Either` field or another disjoint+  union, since strict mode refuses `oneOf`. `normalizeSchemaOpenAI` now+  turns `oneOf` into `anyOf`; the branches cannot both match, so the+  meaning is the same. It also changes only schema keywords now: before,+  it added `required` and `additionalProperties` to objects inside+  literal values such as `default` or `enum`.+- Structured output with OpenAI or Ollama failed with 400 ("reference to+  component '#/$defs/<Name>' which was not found in the schema") for any+  type using a `named` codec below the root: autodocodec puts the+  `$defs` next to the first use, but `#/$defs/<Name>` points from the+  root. `normalizeSchemaOpenAI` now moves every `$defs` entry to the+  root.+- The bundled `model-catalog.json` set `temperature: 0.5` for+  `gpt_5_6_terra`, which the model refuses ("Only the default (1) value+  is supported"), so every call with it failed with 400. The entry now+  leaves the temperature unset.+ ## [0.4.2.0] - 2026-11-03  ### Added@@ -740,6 +830,7 @@   DeepSeek), single-shot generation with fallbacks, agent tool loops, structured   output, JSON model catalog loading, and workspace-scoped filesystem tools. +[0.4.3.0]: https://github.com/aische/llm-simple/compare/v0.4.2.0...v0.4.3.0 [0.4.2.0]: https://github.com/aische/llm-simple/compare/v0.4.1.0...v0.4.2.0 [0.4.1.0]: https://github.com/aische/llm-simple/compare/v0.4.0.1...v0.4.1.0 [0.4.0.1]: https://github.com/aische/llm-simple/compare/v0.4.0.0...v0.4.0.1

@@ -227,7 +227,7 @@  Set `rtReadonly = True` to omit mutating tools from the schema sent to the model and to block them at execution time. -Every request's messages are checked with `validateTurn` (for example, no tool-call parts in a user turn); an invalid turn is logged as a warning and the request is still sent. From 0.5 it will be an error.+Every request's messages are checked with `validateTurn` (for example, no tool-call parts in a user turn) and `validateConversation` (tool results follow the assistant turn whose calls they answer, each call answered exactly once); each problem is logged as a warning and the request is still sent. From 0.5 it will be an error.  Log messages of an agent run (`onLog`) start with `[<agName>] `; leave `agName` empty for no prefix. @@ -289,7 +289,7 @@ - The sandbox is **path-scoped**, not kernel-level isolation. Use a dedicated, disposable workspace directory — not your home directory or a production tree. - There is no denylist for sensitive in-workspace files (`.env`, `.git/config`, etc.). Anything inside the workspace is accessible to tools the model can call. - Read tools enforce byte caps: whole-file and edit tools use a 1 MiB limit; `read_file_paginated` accepts source files up to 10 MiB and returns at most 100 KiB per page.-- Search and listing tools bound their work: `find_files`, `grep` and `directory_tree` look at no more than 20,000 entries per call and say so when they stop; tool output is capped at 100 KiB.+- Search and listing tools bound their work: `find_files`, `grep` and `directory_tree` look at no more than 20,000 entries per call, `grep` reads no more than 64 MiB of files per call, and each says so when it stops; tool output is capped at 100 KiB. - `rtReadonly` omits mutating tools from the model schema and blocks them at execution time if they are invoked anyway. - `writefile` and the replace tools replace the file as a whole (a temporary file renamed over it), so a failed write leaves the old content. The file keeps its mode bits but gets a new inode, so hard links to it keep the old content. Concurrent edits of one file from the same process are serialized; edits from other processes are not. 

@@ -0,0 +1,164 @@+-- | Text reads on one open 'Handle': the binary sniff, the size check and+-- the read itself all use the same open file, instead of reopening the+-- path for each step. Shared by 'LLM.Tools.FsLimits' and the tools.+module LLM.Tools.Internal.TextFile+  ( Opener,+    binarySniffBytes,+    withTextHandle,+    CappedRead (..),+    readCapped,+    readCappedWith,+    readBoundedBytesWith,+    readBoundedBytesWithSize,+    readBoundedTextWith,+    readBoundedUtf8With,+    readLenient,+    writeUtf8,+  )+where++import Control.Exception (IOException, throwIO, try)+import GHC.IO.Exception (IOException (ioe_filename))+import Data.ByteString qualified as BS+import Data.Maybe (fromMaybe)+import Data.Text (Text)+import Data.Text qualified as T+import Data.Text.Encoding (decodeUtf8', decodeUtf8With, encodeUtf8)+import Data.Text.Encoding.Error (lenientDecode)+import System.IO (Handle, SeekMode (AbsoluteSeek), hFileSize, hSeek)++-- | Opens a file for reading and runs an action on it, e.g.+-- @withBinaryFile path ReadMode@.+type Opener = forall a. (Handle -> IO a) -> IO a++-- | Number of leading bytes inspected when sniffing for binary content.+-- Matches the heuristic used by @git diff@ and @grep@: any NUL byte in+-- the prefix means binary.+binarySniffBytes :: Int+binarySniffBytes = 8192++-- | Open a file, sniff its first 'binarySniffBytes' for a NUL byte, and run+-- the action from the start of the file if there is none.+--+-- 'Nothing' if the file looks binary, or cannot be opened or sniffed: the+-- caller fails closed rather than read an unreadable file as text. An+-- 'IOException' from the action itself is rethrown.+withTextHandle :: Opener -> (Handle -> IO r) -> IO (Maybe r)+withTextHandle open act = do+  r <- try $ open $ \h -> do+    sniff <- try (BS.hGet h binarySniffBytes)+    case sniff of+      Left (_ :: IOException) -> pure (Right Nothing)+      Right bs+        | BS.elem 0 bs -> pure (Right Nothing)+        | otherwise -> do+            hSeek h AbsoluteSeek 0+            fmap Just <$> try (act h)+  case r of+    Left (_ :: IOException) -> pure Nothing+    Right (Left (e :: IOException)) -> throwIO e+    Right (Right x) -> pure x++-- | What 'readCapped' found.+data CappedRead+  = -- | The file is over the cap at the size check; its size.+    CappedTooLarge Integer+  | -- | The file was within the cap at the size check but went past it+    -- before the read ended.+    CappedGrew+  | -- | The size at the check, and the file's bytes.+    CappedBytes Integer BS.ByteString+  deriving (Eq, Show)++-- | Check the size of an open file, then read it, refusing more than @cap@+-- bytes. Reads one byte past the cap, so a file that grows past it after+-- the size check is reported as 'CappedGrew' instead of looking like it+-- ended at the cap.+readCapped :: Integer -> Handle -> IO CappedRead+readCapped = readCappedWith hFileSize++-- | 'readCapped' with the size check given, so a test can stand in for a+-- file that grows between the check and the read.+readCappedWith :: (Handle -> IO Integer) -> Integer -> Handle -> IO CappedRead+readCappedWith sizeOf cap h = do+  size <- sizeOf h+  if size > cap+    then pure (CappedTooLarge size)+    else do+      bytes <- BS.hGet h (fromIntegral cap + 1)+      pure $+        if fromIntegral (BS.length bytes) > cap+          then CappedGrew+          else CappedBytes size bytes++-- | Read a file up to @cap@ bytes. Refuses binary files, checks the size+-- before reading, and reads at most @cap + 1@ bytes so a file that grows+-- during the read is caught ('readCapped').+readBoundedBytesWith :: Opener -> Text -> Int -> IO (Either Text BS.ByteString)+readBoundedBytesWith = readBoundedBytesWithSize hFileSize++-- | 'readBoundedBytesWith' with the size check given, so a test can stand+-- in for a file that grows between the check and the read. An+-- 'IOException' from the size check or the read becomes an error naming+-- that step.+readBoundedBytesWithSize :: (Handle -> IO Integer) -> Opener -> Text -> Int -> IO (Either Text BS.ByteString)+readBoundedBytesWithSize sizeOf open display cap = do+  r <- withTextHandle open $ \h -> do+    sizeRes <- try (sizeOf h)+    case sizeRes of+      Left (e :: IOException) ->+        pure $ Left $ "could not stat " <> display <> ": " <> describeIOError e+      Right sz -> do+        readRes <- try (readCappedWith (const (pure sz)) (fromIntegral cap) h)+        pure $ case readRes of+          Left (e :: IOException) ->+            Left $ "could not read " <> display <> ": " <> describeIOError e+          Right (CappedTooLarge _) ->+            Left $+              display+                <> " is "+                <> T.pack (show sz)+                <> " bytes, which exceeds the "+                <> T.pack (show cap)+                <> "-byte cap."+          Right CappedGrew ->+            Left $+              display+                <> " grew past the "+                <> T.pack (show cap)+                <> "-byte cap during the read."+          Right (CappedBytes _ bs) -> Right bs+  pure $+    fromMaybe+      (Left (display <> " appears to be a binary file; only text files are supported."))+      r++-- | An 'IOException' without its file name, which for a handle is the+-- absolute path it was opened with. Callers name the file themselves.+describeIOError :: IOException -> Text+describeIOError e = T.pack (show e {ioe_filename = Nothing})++-- | 'readBoundedBytesWith', decoded for display. Invalid UTF-8 bytes become+-- U+FFFD, so never write the result back.+readBoundedTextWith :: Opener -> Text -> Int -> IO (Either Text Text)+readBoundedTextWith open display cap =+  fmap (decodeUtf8With lenientDecode) <$> readBoundedBytesWith open display cap++-- | 'readBoundedBytesWith', for editing. Refuses files that are not valid+-- UTF-8, since writing them back would destroy their bytes.+readBoundedUtf8With :: Opener -> Text -> Int -> IO (Either Text Text)+readBoundedUtf8With open display cap = do+  r <- readBoundedBytesWith open display cap+  pure $+    r >>= \bs -> case decodeUtf8' bs of+      Right t -> Right t+      Left _ -> Left $ display <> " is not valid UTF-8; refusing to edit it."++-- | Read the rest of the handle as UTF-8, replacing invalid bytes with+-- U+FFFD. Independent of the locale.+readLenient :: Handle -> IO Text+readLenient h = decodeUtf8With lenientDecode <$> BS.hGetContents h++-- | Write text as UTF-8, independent of the locale.+writeUtf8 :: Handle -> Text -> IO ()+writeUtf8 h = BS.hPut h . encodeUtf8

@@ -2,7 +2,7 @@  name:               llm-simple -version:            0.4.2.0+version:            0.4.3.0  synopsis:     Multi-provider LLM library with agent tool loops and filesystem tools@@ -41,6 +41,8 @@ extra-source-files:     model-catalog.json     providers.json+    test/fixtures/*.json+    test/fixtures/*.sse  tested-with:     GHC ==9.6.7@@ -75,6 +77,18 @@         -Wunused-imports         -Werror=missing-fields +-- Leaf modules shared by the library and the test suite. Private: other+-- packages cannot depend on it.+library internal+    import: opts+    exposed-modules:+        LLM.Tools.Internal.TextFile+    hs-source-dirs:     internal+    build-depends:+        base           >=4.18  && <4.22,+        bytestring     >=0.11  && <0.13,+        text           >=1.2   && <2.2+ library     import: opts     exposed-modules:@@ -140,13 +154,17 @@      other-modules:         LLM.Core.Internal.Body+        LLM.Core.Internal.Schema+        LLM.Core.Internal.Transport         LLM.Core.SSE+        LLM.Generate.Internal.Generate+        LLM.Generate.Internal.GenerateUtils         LLM.Tools.Internal.PathLock         LLM.Tools.Internal.TextEdit-        LLM.Tools.Internal.TextFile         LLM.Tools.Internal.WalkBudget     hs-source-dirs:     src     build-depends:+        llm-simple:internal,         aeson          >=2.0   && <2.3,         aeson-pretty   >=0.8   && <0.9,         autodocodec    >=0.5 && < 0.6,@@ -241,6 +259,8 @@                         LLM.HttpConfigSpec                         LLM.LLMProviderSpec                         LLM.GenerateObjectSpec+                        LLM.OptionalFields+                        LLM.TextFileSpec                         LLM.ToolUtilsSpec                         LLM.CrossProviderSpec                         LLM.ScaleSpec@@ -248,6 +268,7 @@                         LLM.WeatherTool     hs-source-dirs:     test     build-depends:+        llm-simple:internal,         base           >=4.18  && <4.22,         containers     >=0.6.7 && <0.8,         directory      >=1.3   && <1.4,@@ -257,6 +278,7 @@         aeson          >=2.0   && <2.3,         aeson-pretty   >=0.8   && <0.9,         autodocodec    >=0.5 && < 0.6,+        autodocodec-schema >=0.2 && <0.3,         bytestring     >=0.11  && <0.13,         hspec          >=2.10  && <2.12,         heptapod       >=1.1   && <1.2,

@@ -26,7 +26,6 @@             "pricePerMillionOutput": 12.0         },         "maxTokens": 4096,-        "temperature": 0.5,         "requestTimeout": 10000,         "throttleDelay": 1000,         "retryCount": 3,

@@ -5,9 +5,9 @@ -- A typical agent application wires four pieces together: -- -- 1. __Models__ — load a 'ModelConfig' (and optional fallbacks) from a JSON---    catalog via 'loadModelOrThrow' or 'loadModelsOrThrow'. See 'LLM.Load' for+--    catalog via 'loadModelOrThrow' or 'loadModelsOrThrow'. See "LLM.Load" for --    catalog format, provider names, and environment variables.--- 2. __Tools__ — build a 'ToolMap' of implementations the model may call. Use+-- 2. __Tools__ — build a t'LLM.Agent.Types.ToolMap' of implementations the model may call. Use --    'fsTools' for the bundled workspace-scoped filesystem tools, or define --    custom tools with 'TypedTool' and 'toTool'. -- 3. __Agent__ — configure an 'Agent' (system prompt, tool allow-list, optional@@ -37,7 +37,7 @@ -- * __Agent layer__ ('LLM.Agent') — multi-round tool loop, context windowing, --   automatic @get_history@ injection, lifecycle events, and readonly enforcement. --   Prefer this for chat agents and coding assistants.--- * __Generate layer__ ('LLM.Generate') — single provider request with retries+-- * __Generate layer__ ("LLM.Generate") — single provider request with retries --   and model fallbacks ('generateTextWithFallbacks', 'streamTextWithFallbacks', --   'genObject'). Use directly when you control the conversation loop yourself, --   or via 'createGenRequest' to share agent configuration.

@@ -1,11 +1,11 @@ -- | Multi-round agent loop with tool execution. -- -- 'generateText' and 'streamText' call the model, execute any returned tool--- calls, append 'ToolTurn' results, and repeat until the model replies with--- plain text or a limit is hit ('Agent.agMaxToolRounds', abort signal).+-- calls, append v'LLM.Core.Types.ToolTurn' results, and repeat until the model replies with+-- plain text or a limit is hit ('LLM.Agent.Types.agMaxToolRounds', abort signal). ----- Tool implementations live in a separate 'ToolMap'; 'Agent.agTools' is the--- allow-list of names exposed to the model. When 'Agent.agContextWindow' is+-- Tool implementations live in a separate 'ToolMap'; 'LLM.Agent.Types.agTools' is the+-- allow-list of names exposed to the model. When 'LLM.Agent.Types.agContextWindow' is -- set, older turns are dropped from the provider request and a @get_history@ -- tool is injected automatically so the model can page through hidden history. --

@@ -13,7 +13,7 @@   ) import LLM.Core.Utils (trySync) --- | Emit a lifecycle event, attaching 'rtGenerationId' from runtime.+-- | Emit a lifecycle event, attaching 'LLM.Agent.Types.rtGenerationId' from runtime. -- Synchronous observer exceptions are swallowed so observers cannot abort -- the loop; asynchronous ones (timeout, cancellation) propagate. emitEvent :: RuntimeArgs -> GenerateEventDetail -> IO ()

@@ -31,10 +31,11 @@   ) import LLM.Core.Usage (Usage (..), emptyUsage) import LLM.Core.Utils (getToolCalls, toolResult)-import LLM.Generate.Generate-  ( generateTextWithFallbacksRotated,-    streamTextWithFallbacksRotated,+import LLM.Generate.Internal.Generate+  ( generateTextWithFallbacksRotatedWith,+    streamTextWithFallbacksRotatedWith,   )+import LLM.Generate.Internal.GenerateUtils (TurnCheck (SkipTurnCheck), logInvalidTurns) import LLM.Generate.Logger (Hooks (..), LogLevel (Warn)) import LLM.Generate.ModelConfig   ( ModelWithFallbacks (..),@@ -49,15 +50,18 @@  -- | Run the agent loop until the model returns final text or the run fails. ----- On success, 'gtrNewMessages' contains only the turns produced during this+-- On success, 'LLM.Generate.Types.gtrNewMessages' contains only the turns produced during this -- call (assistant and tool turns), not the input conversation. -- Tool rounds continue until the model responds without tool calls or--- 'Agent.agMaxToolRounds' rounds have run. Then one final model call asks+-- 'LLM.Agent.Types.agMaxToolRounds' rounds have run. Then one final model call asks -- for a text answer ('toolBudgetUsedUpPrompt'); tool calls in that reply -- are not run, and the run ends with 'GErrToolExceeded'. ----- The warnings of 'LLM.Agent.ToolUtils.validateAgent' go to 'onLog' at--- the start of the run.+-- The warnings of 'LLM.Agent.ToolUtils.validateAgent' go to 'LLM.Generate.Logger.onLog' at+-- the start of the run, and so do the problems of the input conversation+-- ('LLM.Core.Types.validateTurns', 'LLM.Core.Types.validateConversation'),+-- once per run, as @turn <n>: <reason>@ with @n@ counting from 0 in the+-- input conversation. generateText ::   Agent ->   ModelWithFallbacks ->@@ -67,7 +71,7 @@   IO (Either GenerateErrorResult GenerateTextResult) generateText agent models toolMap rt initialTurns =   agentLoop-    (\a m r t -> generateTextWithFallbacksRotated (createGenRequest id a toolMap r t) m)+    (\a m r t -> generateTextWithFallbacksRotatedWith SkipTurnCheck (createGenRequest id a toolMap r t) m)     agent     models     toolMap@@ -89,7 +93,7 @@   IO (Either GenerateErrorResult GenerateTextResult) streamText onChunk agent models toolMap rt initialTurns =   agentLoop-    (\a m r t -> streamTextWithFallbacksRotated onChunk (createGenRequest id a toolMap r t) m)+    (\a m r t -> streamTextWithFallbacksRotatedWith SkipTurnCheck onChunk (createGenRequest id a toolMap r t) m)     agent     models     toolMap@@ -97,7 +101,7 @@     initialTurns  -- | The line appended to the system prompt for the final model call, made--- once 'Agent.agMaxToolRounds' tool rounds have run.+-- once 'LLM.Agent.Types.agMaxToolRounds' tool rounds have run. toolBudgetUsedUpPrompt :: Text toolBudgetUsedUpPrompt =   "The tool budget for this run is used up. Answer in text; do not call tools."@@ -106,7 +110,9 @@ toolNotRunText :: Text toolNotRunText = "Not run: tool round limit reached" --- | The tool loop. @call@ returns the model chain rotated so the model that+-- | The tool loop. The caller's turns are checked once, here; @call@ must+-- not check them again (the turns the loop adds come from providers).+-- @call@ returns the model chain rotated so the model that -- answered comes first; the next round starts with that chain, so a run that -- fell back stays on the fallback instead of retrying a failing primary every -- round. The next 'generateText' / 'streamText' call starts from the caller's@@ -121,6 +127,7 @@   IO (Either GenerateErrorResult GenerateTextResult) agentLoop call agent models toolMap rt0 initialTurns = do   mapM_ (rt.rtHooks.onLog Warn) (validateAgent agent toolMap)+  logInvalidTurns rt.rtHooks initialTurns   emitEvent rt GenerationStarted   go models initialTurns [] emptyUsage 0   where

@@ -22,7 +22,7 @@  -- | Generate a typed Haskell value from the model via Autodocodec. ----- Tools are never sent: @grTools@ is always empty. 'agContextWindow' still+-- Tools are never sent: @grTools@ is always empty. 'LLM.Agent.Types.agContextWindow' still -- truncates messages, but does not inject @get_history@. generateObject ::   (GeneratableObject t) =>

@@ -61,7 +61,7 @@   fromMaybe (toolResult tc "Aborted") <$> executeToolInterruptible Nothing hooks ctx tools tc  -- | Like 'executeTool', but the tool is interrupted when the signal fires.--- Returns 'Nothing' if it was; 'onToolError' then reports @\"Aborted\"@.+-- Returns 'Nothing' if it was; 'LLM.Generate.Logger.onToolError' then reports @\"Aborted\"@. executeToolInterruptible :: Maybe AbortSignal -> Hooks -> ToolContext -> [Tool Text] -> ToolCall -> IO (Maybe ToolResult) executeToolInterruptible mSig hooks ctx tools tc = case lookup tc.tcName toolMap of   Nothing -> do@@ -172,7 +172,7 @@ windowOffset Nothing _ = 0 windowOffset (Just n) conv = findNthUserFromEnd n conv --- | Find the index of the Nth 'UserTurn' from the end of a conversation.+-- | Find the index of the Nth v'LLM.Core.Types.UserTurn' from the end of a conversation. -- Returns 0 if there are fewer than @n@ user messages, or if @n <= 0@. findNthUserFromEnd :: Int -> [Turn] -> Int findNthUserFromEnd n conv@@ -183,7 +183,7 @@ -- -- Applies context windowing ('windowOffset'), resolves tools from the -- 'ToolMap' (including auto-injected @get_history@ when windowed), and--- respects 'RuntimeArgs.rtReadonly' when selecting tool definitions.+-- respects 'LLM.Agent.Types.rtReadonly' when selecting tool definitions. createGenRequest :: (Text -> result) -> Agent -> ToolMap result -> RuntimeArgs -> [Turn] -> GenRequest createGenRequest embed agent toolMap rt messages =   let offset = windowOffset agent.agContextWindow messages@@ -199,8 +199,8 @@  -- | Like 'createGenRequest', but never attaches tool definitions. ----- Still applies 'agContextWindow' truncation to 'grMessages'. Use this for--- structured-output paths ('generateObject' / 'generateObjectUntyped'), which+-- Still applies 'LLM.Agent.Types.agContextWindow' truncation to 'LLM.Generate.Types.grMessages'. Use this for+-- structured-output paths ('LLM.Agent.GenerateObject.generateObject' / 'LLM.Agent.GenerateObject.generateObjectUntyped'), which -- cannot execute tools and must not advertise @get_history@. createGenRequestNoTools :: Agent -> RuntimeArgs -> [Turn] -> GenRequest createGenRequestNoTools agent rt messages =@@ -214,9 +214,9 @@           grHooks = rt.rtHooks         } --- | The tools sent to the model: the 'Agent.agTools' names found in the+-- | The tools sent to the model: the 'LLM.Agent.Types.agTools' names found in the -- 'ToolMap', each once (the first occurrence counts), read-only ones only--- when 'rtReadonly' is set, followed by the built-in @get_history@ when+-- when 'LLM.Agent.Types.rtReadonly' is set, followed by the built-in @get_history@ when -- windowing is on. A caller tool named @get_history@ is dropped while -- windowing is on. Names missing from the map are skipped; -- 'validateAgent' reports them.@@ -252,14 +252,14 @@  -- | Warnings about an agent configuration that the agent loop works around: ----- * a name listed more than once in 'agTools' (only the first counts);--- * a name in 'agTools' that the 'ToolMap' lacks (it is not sent);+-- * a name listed more than once in 'LLM.Agent.Types.agTools' (only the first counts);+-- * a name in 'LLM.Agent.Types.agTools' that the 'ToolMap' lacks (it is not sent); -- * a caller tool named @get_history@ while windowing is on (the built-in --   one is sent instead);--- * @'agContextWindow' = 'Just' n@ with @n <= 0@ (no windowing).+-- * @'LLM.Agent.Types.agContextWindow' = 'Just' n@ with @n <= 0@ (no windowing). -- -- 'LLM.Agent.generateText' and 'LLM.Agent.streamText' send each warning--- to 'onLog' at 'LLM.Generate.Logger.Warn' once at the start of a run.+-- to 'LLM.Generate.Logger.onLog' at 'LLM.Generate.Logger.Warn' once at the start of a run. -- An empty list means none apply. validateAgent :: Agent -> ToolMap result -> [Text] validateAgent agent toolMap =@@ -320,7 +320,7 @@ -- | The runtime arguments the agent entry points ('LLM.Agent.generateText', -- 'LLM.Agent.streamText', 'LLM.Agent.generateObject', -- 'LLM.Agent.generateObjectUntyped') run with: 'safeRuntimeArgs', and every--- 'onLog' message prefixed with @[\<agName\>] @. An empty 'agName' adds+-- 'LLM.Generate.Logger.onLog' message prefixed with @[\<agName\>] @. An empty 'LLM.Agent.Types.agName' adds -- no prefix. agentRuntimeArgs :: Agent -> RuntimeArgs -> RuntimeArgs agentRuntimeArgs agent rt = safeRuntimeArgs rt {rtHooks = named rt.rtHooks}

@@ -18,6 +18,8 @@ import Data.Text qualified as T import GHC.Generics (Generic) import LLM.Agent.Types (ToolContext (..))+import LLM.Tools.FsLimits (maxOutputBytes)+import LLM.Tools.Internal.WalkBudget (textPrefixBytes, utf8Length) import LLM.Core.Types   ( ToolCall (tcName),     ToolResult (trContent, trName),@@ -53,7 +55,7 @@           chunks = chunkBackward nUserMessages hidden       if chunkIdx < 0 || chunkIdx >= length chunks         then pure "(no more history)"-        else pure $ formatChunk (chunks !! chunkIdx)+        else pure $ formatChunkWithin maxOutputBytes (chunks !! chunkIdx)  historyToolTyped :: TypedTool ToolContext HistoryToolArgs historyToolTyped =@@ -63,12 +65,15 @@         "Retrieve earlier conversation history that is not in your current context window. "           <> "Pass chunk=0 for the most recent hidden history, chunk=1 for the one before that, etc. "           <> "Returns \"(no earlier history)\" when nothing is hidden, and \"(no more history)\" "-          <> "when the chunk index is out of range.",+          <> "when the chunk index is out of range. A page is at most "+          <> T.pack (show maxOutputBytes)+          <> " bytes: past that its oldest turns are left out (the last line says how many), "+          <> "and a single turn over the cap is cut and marked.",       ttoolReadonly = True,       ttoolExecute = getHistoryExecTyped     } --- | Count the number of 'UserTurn's in a conversation.+-- | Count the number of v'LLM.Core.Types.UserTurn's in a conversation. countUserTurns :: [Turn] -> Int countUserTurns = length . filter isUserTurn @@ -77,7 +82,7 @@ isUserTurn _ = False  -- | Split a conversation into pages of @n@ user messages each, working--- backward from the end. Each page starts at a 'UserTurn'.+-- backward from the end. Each page starts at a v'LLM.Core.Types.UserTurn'. -- Chunk 0 is the most recent page, chunk 1 the one before, etc. -- The oldest chunk (highest index) may contain fewer than @n@ user messages. --@@ -124,6 +129,45 @@ -- | Format a chunk of conversation turns as readable text. formatChunk :: [Turn] -> Text formatChunk = T.intercalate "\n" . map formatTurn++-- | 'formatChunk' within @cap@ UTF-8 bytes. A page over the cap keeps+-- its newest turns and ends with a line saying how many older turns of+-- the page were left out. If the newest turn alone is over the cap, its+-- text is cut and marked instead, so the page is never empty.+formatChunkWithin :: Int -> [Turn] -> Text+formatChunkWithin cap turns+  | utf8Length full <= cap = full+  | otherwise = T.intercalate "\n" (reverse keptNewestFirst ++ note)+  where+    full = formatChunk turns+    rendered = map formatTurn turns+    -- Room for the note and the cut marker.+    budget = max 0 (cap - 256)+    keptNewestFirst = case keepNewest 0 (reverse rendered) of+      [] -> [cutTurn (last rendered)]+      kept -> kept+    dropped = length turns - length keptNewestFirst+    note =+      [ "["+          <> T.pack (show dropped)+          <> " earlier turn(s) of this page left out: the page is over "+          <> T.pack (show cap)+          <> " bytes]"+        | dropped > 0+      ]+    keepNewest _ [] = []+    keepNewest used (l : ls)+      | used + size > budget = []+      | otherwise = l : keepNewest (used + size) ls+      where+        size = utf8Length l + 1+    cutTurn l =+      textPrefixBytes budget l+        <> " [turn cut: "+        <> T.pack (show (utf8Length l))+        <> " bytes, first "+        <> T.pack (show budget)+        <> " shown]"  formatTurn :: Turn -> Text formatTurn (UserMessage parts) =

@@ -26,7 +26,7 @@  -- | Static agent configuration shared across requests. data Agent = Agent-  { -- | Name of the agent. The agent entry points prefix every 'onLog'+  { -- | Name of the agent. The agent entry points prefix every 'LLM.Generate.Logger.onLog'     -- message of a run with @[\<agName\>] @; an empty name adds no prefix.     -- It is not sent to the model.     agName :: Text,@@ -43,7 +43,7 @@     -- answer ('LLM.Agent.Generate.toolBudgetUsedUpPrompt'). A text reply     -- ends the run successfully. Tool calls in that reply are not run: each     -- gets the result @\"Not run: tool round limit reached\"@ and the run-    -- returns 'GErrToolExceeded'. So every tool that runs has its result+    -- returns v'LLM.Generate.Types.GErrToolExceeded'. So every tool that runs has its result     -- seen by the model, and a run makes at most @agMaxToolRounds + 1@     -- model calls. A value @<= 0@ makes only the final call.     agMaxToolRounds :: Int,@@ -69,20 +69,20 @@     -- | Observer for 'GenerateEvent' lifecycle notifications (UI, telemetry).     rtOnEvent :: EventObserver,     -- | When 'True', mutating tools are rejected at execution time even if-    -- they appear in 'Agent.agTools'. Read-only tools still run.+    -- they appear in 'LLM.Agent.Types.agTools'. Read-only tools still run.     rtReadonly :: Bool   }  -- | A tool: its definition (sent to the model) paired with its implementation. ----- 'toolExecute' receives a 'ToolContext' (full conversation + usage) and+-- 'LLM.Agent.Types.toolExecute' receives a 'ToolContext' (full conversation + usage) and -- the JSON arguments from the model. data Tool result = Tool   { toolDef :: ToolDef,     toolExecute :: ToolContext -> Value -> IO result   } --- | Map from tool name to implementation. Built manually or via 'fsTools'.+-- | Map from tool name to implementation. Built manually or via 'LLM.Load.FsTools.fsTools'. type ToolMap result = Map Text (Tool result)  -- | Context passed to tool implementations during execution.@@ -91,14 +91,14 @@     tcConversation :: [Turn],     -- | Accumulated token usage so far     tcUsage :: Usage,-    -- | Index into 'tcConversation' where the visible window starts.+    -- | Index into 'LLM.Agent.Types.tcConversation' where the visible window starts.     -- Everything before this index is hidden from the model.     -- The @get_history@ tool uses this to serve paginated history.     tcWindowOffset :: Int,     tcRuntimeArgs :: RuntimeArgs   } --- | Generation lifecycle event tagged with the run's 'rtGenerationId'.+-- | Generation lifecycle event tagged with the run's 'LLM.Agent.Types.rtGenerationId'. data GenerateEvent = GenerateEvent   { geGenerationId :: UUID,     geDetail :: GenerateEventDetail

@@ -33,6 +33,7 @@     coalesceAdjacentTextParts,     validateTurn,     validateTurns,+    validateConversation,     ToolCall (..),     ToolResult (..),     LLMHooks (..),

@@ -11,7 +11,7 @@ import Control.Concurrent.STM (TVar, atomically, check, newTVarIO, readTVar, readTVarIO, writeTVar)  -- | A cancellation signal.--- Create one with 'newAbortSignal', pass it into 'RuntimeArgs' or 'GenRequest', and call+-- Create one with 'newAbortSignal', pass it into t'LLM.Agent.Types.RuntimeArgs' or t'LLM.Generate.Types.GenRequest', and call -- 'abort' from any thread. A model call or tool that is running when the signal -- fires is interrupted. newtype AbortSignal = AbortSignal (TVar Bool)

@@ -1,9 +1,10 @@ -- | Bounded reading of provider response bodies. ----- A 200 body larger than 'maxResponseBodyBytes' (or a stream event larger--- than 'maxStreamEventBytes') is signalled with 'ResponseTooLarge', which+-- A 200 body larger than 'maxResponseBodyBytes' (a stream larger than it+-- in total, or a stream event larger than 'maxStreamEventBytes') is+-- signalled with 'ResponseTooLarge', which -- is not exported from any public module. The @generic*@ functions in--- 'LLM.Core.LLMProvider' catch it and return a 'ParseError'; the type of+-- 'LLM.Core.LLMProvider' catch it and return a v'LLM.Core.Types.ParseError'; the type of -- 'LLM.Core.LLMProvider.sendRequest' stays unchanged. module LLM.Core.Internal.Body   ( ResponseTooLarge (..),@@ -28,16 +29,17 @@  instance Exception ResponseTooLarge --- | The 'ParseError' text for 'ResponseTooLarge'.+-- | The v'LLM.Core.Types.ParseError' text for 'ResponseTooLarge'. responseTooLargeMessage :: ResponseTooLarge -> Text responseTooLargeMessage (ResponseTooLarge n) =   "response body exceeds " <> T.pack (show n) <> " bytes" --- | Bytes of a non-200 body kept for the 'HttpError' message.+-- | Bytes of a non-200 body kept for the v'LLM.Core.Types.HttpError' message. maxErrorBodyBytes :: Int maxErrorBodyBytes = 64 * 1024 --- | Largest 200 body a non-streaming call accepts.+-- | Largest 200 body a non-streaming call accepts, and the largest total+-- size of a stream. maxResponseBodyBytes :: Int maxResponseBodyBytes = 32 * 1024 * 1024 

@@ -0,0 +1,237 @@+-- | JSON Schema helpers for structured output.+--+-- OpenAI's strict mode needs every property listed in @required@; an+-- optional property is emulated by making it nullable+-- ('LLM.Core.ProviderUtils.normalizeSchemaOpenAI'). The model then answers+-- @null@ for a field it leaves out, which the decoder of an+-- @optionalField*@ codec does not accept. 'dropOptionalNulls' removes those+-- @null@s again before decoding, using the schema the codec emitted.+module LLM.Core.Internal.Schema+  ( SchemaDefs,+    schemaDefs,+    isNullableSchema,+    optionalProperties,+    dropOptionalNulls,+    mapSubschemas,+    hoistDefs,+  )+where++import Data.Aeson (Value (..))+import Data.Aeson.Key qualified as K+import Data.Aeson.KeyMap qualified as KM+import Data.Foldable (toList)+import Data.Maybe (mapMaybe)+import Data.Text (Text)+import Data.Text qualified as T++-- | Named sub-schemas (@$defs@, @definitions@), and the root for @#@.+data SchemaDefs = SchemaDefs+  { sdRoot :: Value,+    sdNamed :: KM.KeyMap Value+  }++-- | The definitions anywhere in a schema. autodocodec puts the @$defs@ of a+-- named codec next to its first use, not at the root, so the whole schema+-- is searched. On a name clash the first one found wins.+schemaDefs :: Value -> SchemaDefs+schemaDefs root = SchemaDefs root (collectDefs "$defs" root `KM.union` collectDefs "definitions" root)++-- | The entries of one definitions keyword (@$defs@ or @definitions@)+-- anywhere in a schema, following sub-schema positions only; the node's+-- own entries first, then those below it in order.+collectDefs :: K.Key -> Value -> KM.KeyMap Value+collectDefs k (Object o) =+  foldr KM.union KM.empty $+    [d | Just (Object d) <- [KM.lookup k o]] <> map (collectDefs k) (subschemas o)+collectDefs _ _ = KM.empty++-- | Move every @$defs@ and @definitions@ entry to the root of the schema.+-- References like @#/$defs/<name>@ are pointers from the root, but+-- autodocodec puts the @$defs@ of a named codec next to its first use,+-- where OpenAI and Ollama do not find them. On a name clash the first one+-- found wins, as in 'schemaDefs'.+hoistDefs :: Value -> Value+hoistDefs root@(Object o) =+  Object (put "$defs" (put "definitions" (mapSubschemas strip o)))+  where+    put k obj+      | KM.null defs = KM.delete k obj+      | otherwise = KM.insert k (Object defs) obj+      where+        defs = fmap strip (collectDefs k root)+    -- A sub-schema without its definitions, at every depth.+    strip (Object x) = Object (mapSubschemas strip (KM.delete "$defs" (KM.delete "definitions" x)))+    strip v = v+hoistDefs v = v++-- | The schema a @$ref@ points to, for @#@, @#/$defs/<name>@ and+-- @#/definitions/<name>@. Other references are not resolved.+resolveRef :: SchemaDefs -> Text -> Maybe Value+resolveRef defs ref+  | ref == "#" = Just defs.sdRoot+  | Just name <- T.stripPrefix "#/$defs/" ref = named name+  | Just name <- T.stripPrefix "#/definitions/" ref = named name+  | otherwise = Nothing+  where+    named name = KM.lookup (K.fromText name) defs.sdNamed++-- | Whether a schema accepts @null@. A schema we cannot judge (an+-- unresolved @$ref@, a schema with no type constraint) counts as nullable,+-- so it is left alone.+isNullableSchema :: SchemaDefs -> Value -> Bool+isNullableSchema defs = go []+  where+    go seen (Object o)+      | Just (String ref) <- KM.lookup "$ref" o =+          ref `elem` seen || maybe True (go (ref : seen)) (resolveRef defs ref)+      | hasType "null" o = True+      | Just (Array vs) <- KM.lookup "enum" o = Null `elem` vs+      | Just c <- KM.lookup "const" o = c == Null+      | any (branches o) ["anyOf", "oneOf"] = True+      | Just (Array bs) <- KM.lookup "allOf" o = all (go seen) bs+      | any (`KM.member` o) constraining = False+      | otherwise = True+      where+        branches obj k = case KM.lookup k obj of+          Just (Array bs) -> any (go seen) bs+          _ -> False+    go _ (Bool b) = b+    go _ _ = True+    constraining = ["type", "anyOf", "oneOf", "allOf", "not", "properties", "items"]++-- | The properties of an object schema that are not in its @required@ list.+optionalProperties :: KM.KeyMap Value -> [K.Key]+optionalProperties o = case KM.lookup "properties" o of+  Just (Object props) -> filter (`notElem` requiredKeys o) (KM.keys props)+  _ -> []++requiredKeys :: KM.KeyMap Value -> [K.Key]+requiredKeys o = case KM.lookup "required" o of+  Just (Array ks) -> [K.fromText k | String k <- toList ks]+  _ -> []++hasType :: Text -> KM.KeyMap Value -> Bool+hasType t o = case KM.lookup "type" o of+  Just (String s) -> s == t+  Just (Array ts) -> String t `elem` ts+  _ -> False++-- | Remove the @null@ values of properties that the schema has optional and+-- not nullable, in nested objects and arrays too, so that the codec's+-- decoder sees them as absent. Everything else is kept as it is.+--+-- In a union (@anyOf@, @oneOf@) the walk continues only into the one branch+-- that fits the value (an object with the branch's required keys and no+-- unknown ones, or an array); with no or several such branches the value+-- is left alone.+dropOptionalNulls :: Value -> Value -> Value+dropOptionalNulls schema = walk [] schema+  where+    defs = schemaDefs schema++    -- @seen@: references followed since the last step into the value, so a+    -- reference cycle that does not consume the value stops.+    walk :: [Text] -> Value -> Value -> Value+    walk seen (Object s) v+      | Just (String ref) <- KM.lookup "$ref" s =+          case resolveRef defs ref of+            Just target | ref `notElem` seen -> walk (ref : seen) target v+            _ -> v+      | Just bs <- unionBranches s = case filter (fits seen v) bs of+          [b] -> walk seen b v+          _ -> v+    walk _ (Object s) (Object v) = Object (KM.mapMaybeWithKey (field s) v)+    walk _ (Object s) (Array vs)+      | Just items@(Object _) <- KM.lookup "items" s = Array (fmap (walk [] items) vs)+    walk _ _ v = v++    field s k x = case propertySchema s k of+      Just ps+        | x == Null,+          k `notElem` requiredKeys s,+          not (isNullableSchema defs ps) ->+            Nothing+        | otherwise -> Just (walk [] ps x)+      Nothing -> case KM.lookup "additionalProperties" s of+        Just aps@(Object _) -> Just (walk [] aps x)+        _ -> Just x++    propertySchema s k = case KM.lookup "properties" s of+      Just (Object props) -> KM.lookup k props+      _ -> Nothing++    unionBranches s =+      case mapMaybe (`KM.lookup` s) ["anyOf", "oneOf"] of+        [Array bs] -> Just (toList bs)+        _ -> Nothing++    -- Whether the value has the shape of this branch; only objects and+    -- arrays matter, since nothing is dropped from other values.+    fits seen v (Object b)+      | Just (String ref) <- KM.lookup "$ref" b =+          case resolveRef defs ref of+            Just target | ref `notElem` seen -> fits (ref : seen) v target+            _ -> False+    fits _ (Object v) (Object b) =+      (hasType "object" b || KM.member "properties" b)+        && all (`KM.member` v) (requiredKeys b)+        && ( case (KM.lookup "properties" b, KM.lookup "additionalProperties" b) of+               (_, Just (Object _)) -> True+               (_, Just (Bool True)) -> True+               (Just (Object props), _) -> all (`KM.member` props) (KM.keys v)+               _ -> KM.null v+           )+    fits _ (Array _) (Object b) = hasType "array" b || KM.member "items" b+    fits _ _ _ = False++-- | Apply a function to every direct sub-schema of a schema node: the+-- values of the keywords that hold a sub-schema, a list of them, or a map+-- of them. Everything else (property names, @enum@, @const@, @default@+-- and other literal values) is kept as it is.+mapSubschemas :: (Value -> Value) -> KM.KeyMap Value -> KM.KeyMap Value+mapSubschemas f = KM.mapWithKey sub+  where+    sub k v+      | k `elem` schemaMapKeywords = mapObject v+      | k `elem` schemaKeywords = schemaOrList v+      | otherwise = v+    mapObject (Object o) = Object (fmap f o)+    mapObject v = v+    schemaOrList (Array a) = Array (fmap f a)+    schemaOrList v = f v++-- | The direct sub-schemas of a schema node, in the positions+-- 'mapSubschemas' visits.+subschemas :: KM.KeyMap Value -> [Value]+subschemas o = concat [sub k v | (k, v) <- KM.toList o]+  where+    sub k v+      | k `elem` schemaMapKeywords, Object m <- v = KM.elems m+      | k `elem` schemaKeywords, Array a <- v = toList a+      | k `elem` schemaKeywords = [v]+      | otherwise = []++-- | Keywords whose value maps names to sub-schemas.+schemaMapKeywords :: [KM.Key]+schemaMapKeywords = ["properties", "patternProperties", "$defs", "definitions", "dependentSchemas"]++-- | Keywords whose value is a sub-schema or a list of sub-schemas.+schemaKeywords :: [KM.Key]+schemaKeywords =+  [ "items",+    "prefixItems",+    "additionalItems",+    "additionalProperties",+    "unevaluatedProperties",+    "unevaluatedItems",+    "propertyNames",+    "contains",+    "anyOf",+    "oneOf",+    "allOf",+    "not",+    "if",+    "then",+    "else"+  ]

@@ -0,0 +1,36 @@+-- | The HTTP calls every built-in provider makes: a JSON @POST@ with+-- 'lenientConfig', read with 'readResponseBody' or handed to a stream+-- parser through 'handleStreamResponse'. Each provider keeps its own URL,+-- auth options and body preparation.+module LLM.Core.Internal.Transport+  ( postJson,+    postJsonStream,+  )+where++import Data.Aeson (Value)+import Data.ByteString (ByteString)+import LLM.Core.ProviderUtils (handleStreamResponse, lenientConfig, readResponseBody)+import LLM.Core.Types (LLMTextResult)+import Network.HTTP.Req (Option, POST (POST), ReqBodyJson (ReqBodyJson), Url, reqBr, runReq)++-- | Send a non-streaming request; the result is+-- 'LLM.Core.LLMProvider.sendRequest''s @(status, body)@.+postJson :: Url scheme -> Option scheme -> Value -> IO (Int, ByteString)+postJson url opts body =+  runReq lenientConfig $+    reqBr POST url (ReqBodyJson body) opts readResponseBody++-- | Send a streaming request, for 'LLM.Core.LLMProvider.openStream'. On a+-- 200 response the continuation gets the body reader; any other status+-- becomes an 'LLM.Core.Types.HttpError' without calling it.+postJsonStream ::+  Url scheme ->+  Option scheme ->+  Value ->+  (IO ByteString -> IO LLMTextResult) ->+  IO LLMTextResult+postJsonStream url opts body k =+  runReq lenientConfig $+    reqBr POST url (ReqBodyJson body) opts $ \resp ->+      handleStreamResponse resp k

@@ -32,15 +32,15 @@ data LLMProvider = LLMProvider   { -- | Provider name for logging/hooks (e.g. "claude", "gemini", "openai")     providerName :: Text,-    -- | MIME types of inline images the provider accepts (see 'gwImageMediaTypes').+    -- | MIME types of inline images the provider accepts (see 'LLM.Core.Types.gwImageMediaTypes').     providerImageMediaTypes :: [Text],-    -- | Whether the provider accepts URL images (see 'gwImageUrlSupport').+    -- | Whether the provider accepts URL images (see 'LLM.Core.Types.gwImageUrlSupport').     providerImageUrlSupport :: Bool,     -- | Build the JSON request body. Bool indicates whether streaming is     -- requested. A top-level @_model@ key is routing data for-    -- 'sendRequest' and 'openStream' (the Gemini provider puts the model+    -- 'LLM.Core.LLMProvider.sendRequest' and 'LLM.Core.LLMProvider.openStream' (the Gemini provider puts the model     -- name there, since its URL contains it); they must remove it before-    -- sending, and 'onLLMRequest' sees the body without it+    -- sending, and 'LLM.Core.Types.onLLMRequest' sees the body without it     -- ('withoutRoutingFields').     buildBody :: Bool -> ChatRequest -> Value,     -- | Make a non-streaming HTTP call, returning (status code, raw response@@ -108,7 +108,7 @@     Right resp -> handleResponse p hooks p.parseObjectResponse resp  -- | The request body as it is sent: without the top-level @_model@ routing--- key (see 'buildBody'). Other values are returned unchanged.+-- key (see 'LLM.Core.LLMProvider.buildBody'). Other values are returned unchanged. withoutRoutingFields :: Value -> Value withoutRoutingFields (Object o) = Object (KM.delete "_model" o) withoutRoutingFields v = v@@ -140,7 +140,7 @@  -- | Run a provider's HTTP call. An 'HttpException' becomes a 'NetworkError' -- and a response body over the size cap a 'ParseError'; both are reported--- to 'onLLMResponseError'. Other exceptions propagate.+-- to 'LLM.Core.Types.onLLMResponseError'. Other exceptions propagate. providerCall :: LLMProvider -> LLMHooks -> IO a -> IO (Either LLMError a) providerCall p hooks act = do   r <-@@ -160,7 +160,7 @@ -- | Turn a non-streaming (status, raw body) into a result. A non-200 status -- is an 'HttpError' carrying the body as text, whether or not it is JSON -- (an HTML 502 from a proxy keeps its status). A 200 body that is not JSON--- is a 'ParseError'. 'onLLMResponse' sees every body that decodes as JSON.+-- is a 'ParseError'. 'LLM.Core.Types.onLLMResponse' sees every body that decodes as JSON. handleResponse :: LLMProvider -> LLMHooks -> (Value -> IO (LLMResult a)) -> (Int, ByteString) -> IO (LLMResult a) handleResponse p hooks parse (status, raw) = do   let decoded = eitherDecodeStrict' raw

@@ -16,7 +16,7 @@  import Control.Exception (throwIO) import Control.Retry (limitRetries)-import Data.Aeson (ToJSON (toJSON), Value (..), decodeStrict', eitherDecodeStrict', encode, object)+import Data.Aeson (ToJSON (toJSON), Value (..), decodeStrict', eitherDecodeStrict', encode, object, (.=)) import Data.Aeson.KeyMap qualified as KM import Data.ByteString qualified as BS import Data.Text (Text)@@ -31,6 +31,7 @@     readBodyPrefix,     readErrorBody,   )+import LLM.Core.Internal.Schema (hoistDefs, isNullableSchema, mapSubschemas, optionalProperties, schemaDefs) import LLM.Core.Types (LLMError (EmptyResponse, HttpError, ParseError), LLMResult, LLMTextResult) import Network.HTTP.Client qualified as HC import Network.HTTP.Client.TLS (tlsManagerSettings)@@ -59,7 +60,7 @@ -- | Manager shared by all provider requests. A non-streaming call receives -- its response headers only after generation finishes, so http-client's -- default 30 s response timeout would cut off long calls. The only response--- deadline is the one the application configures ('mcRequestTimeout').+-- deadline is the one the application configures ('LLM.Generate.ModelConfig.mcRequestTimeout'). sharedManager :: HC.Manager sharedManager =   unsafePerformIO $@@ -126,23 +127,53 @@       pure $ Left $ HttpError status (decodeUtf8With lenientDecode (BS.concat chunks))     else handler (HC.responseBody resp) --- | Recursively enforce OpenAI structured output constraints:---   every object node gets additionalProperties:false and all keys in required.+-- | Recursively enforce OpenAI structured output constraints: every object+-- node gets @additionalProperties: false@ and all its properties in+-- @required@. Strict mode allows no optional properties, so a property+-- that was optional and does not accept @null@ becomes nullable+-- (@anyOf: [<schema>, {"type": "null"}]@); the model answers @null@ to+-- leave it out, and 'LLM.Generate.genObject' removes that @null@ before+-- decoding.+--+-- Strict mode refuses @oneOf@, so it becomes @anyOf@ (unless the node has+-- an @anyOf@ already). autodocodec emits @oneOf@ for disjoint unions such+-- as 'Either', whose branches cannot both match, so the meaning is the+-- same.+--+-- Every @$defs@ entry moves to the root: autodocodec puts the @$defs@ of a+-- named codec next to its first use, but @#/$defs/<name>@ points from the+-- root, and strict mode refuses a reference it cannot find there.+--+-- Only schema keywords are changed: the walk follows sub-schema positions,+-- so a property called @oneOf@ or a literal value is kept. normalizeSchemaOpenAI :: Value -> Value-normalizeSchemaOpenAI (Object o) =-  let o' = fmap normalizeSchemaOpenAI o-      fixed = case KM.lookup "type" o' of-        Just (String "object") ->-          let propKeys = case KM.lookup "properties" o' of-                Just (Object p) -> KM.keys p-                _ -> []-              withAP = KM.insert "additionalProperties" (Bool False) o'-              withReq = KM.insert "required" (toJSON propKeys) withAP-           in withReq-        _ -> o'-   in Object fixed-normalizeSchemaOpenAI (Array a) = Array (fmap normalizeSchemaOpenAI a)-normalizeSchemaOpenAI v = v+normalizeSchemaOpenAI schema = hoistDefs (go schema)+  where+    defs = schemaDefs schema+    go (Object o) =+      let o' = oneOfToAnyOf (mapSubschemas go o)+          fixed = case KM.lookup "type" o' of+            Just (String "object") ->+              let propKeys = case KM.lookup "properties" o' of+                    Just (Object p) -> KM.keys p+                    _ -> []+                  optional = optionalProperties o+                  withNull k p+                    | k `elem` optional && not (isNullableSchema defs p) =+                        object ["anyOf" .= [p, object ["type" .= ("null" :: Text)]]]+                    | otherwise = p+                  withNullable = case KM.lookup "properties" o' of+                    Just (Object p) -> KM.insert "properties" (Object (KM.mapWithKey withNull p)) o'+                    _ -> o'+                  withAP = KM.insert "additionalProperties" (Bool False) withNullable+                  withReq = KM.insert "required" (toJSON propKeys) withAP+               in withReq+            _ -> o'+       in Object fixed+    go v = v+    oneOfToAnyOf o = case KM.lookup "oneOf" o of+      Just bs | not (KM.member "anyOf" o) -> KM.insert "anyOf" bs (KM.delete "oneOf" o)+      _ -> o  stripJsonFences :: Text -> Text stripJsonFences t =@@ -195,42 +226,6 @@ stripBoundsAndComments :: Value -> Value stripBoundsAndComments (Object obj) =   Object $-    KM.fromList-      [ (k, stripKeyword k v)-        | (k, v) <- KM.toList obj,-          k `notElem` ["minimum", "maximum", "$comment"]-      ]-  where-    stripKeyword k v-      | k `elem` schemaMapKeywords = mapObject stripBoundsAndComments v-      | k `elem` schemaKeywords = stripSchemaOrList v-      | otherwise = v-    mapObject f (Object o) = Object (fmap f o)-    mapObject _ v = v-    stripSchemaOrList (Array a) = Array (fmap stripBoundsAndComments a)-    stripSchemaOrList v = stripBoundsAndComments v+    mapSubschemas stripBoundsAndComments $+      KM.filterWithKey (\k _ -> k `notElem` ["minimum", "maximum", "$comment"]) obj stripBoundsAndComments other = other---- | Keywords whose value maps names to sub-schemas.-schemaMapKeywords :: [KM.Key]-schemaMapKeywords = ["properties", "patternProperties", "$defs", "definitions", "dependentSchemas"]---- | Keywords whose value is a sub-schema or a list of sub-schemas.-schemaKeywords :: [KM.Key]-schemaKeywords =-  [ "items",-    "prefixItems",-    "additionalItems",-    "additionalProperties",-    "unevaluatedProperties",-    "unevaluatedItems",-    "propertyNames",-    "contains",-    "anyOf",-    "oneOf",-    "allOf",-    "not",-    "if",-    "then",-    "else"-  ]

@@ -16,7 +16,7 @@ import Data.Text (Text) import Data.Text qualified as T import Data.Text.Encoding qualified as TE-import LLM.Core.Internal.Body (ResponseTooLarge (..), maxStreamEventBytes)+import LLM.Core.Internal.Body (ResponseTooLarge (..), maxResponseBodyBytes, maxStreamEventBytes) import LLM.Core.Types (LLMError (..))  data SSEEvent = SSEEvent@@ -32,13 +32,24 @@ -- continues so the caller can still tell whether the stream ended properly. -- The result describes the first such event, if any. ----- An event (or a line) larger than 'maxStreamEventBytes' throws--- 'ResponseTooLarge', which the @generic*@ functions turn into a--- 'ParseError'.+-- An event (or a line) larger than 'maxStreamEventBytes', or a stream+-- longer than 'maxResponseBodyBytes' in total, throws 'ResponseTooLarge',+-- which the @generic*@ functions turn into a 'ParseError'. readSSEEvents :: IO ByteString -> (SSEEvent -> IO ()) -> IO (Maybe Text)-readSSEEvents readChunk callback = do+readSSEEvents readRaw callback = do   bufRef <- newIORef BS.empty   badRef <- newIORef Nothing+  totalRef <- newIORef (0 :: Int)+  let -- Count every byte read, so a stream of small events cannot go on+      -- forever.+      readChunk :: IO ByteString+      readChunk = do+        chunk <- readRaw+        total <- (+ BS.length chunk) <$> readIORef totalRef+        when (total > maxResponseBodyBytes) $+          throwIO (ResponseTooLarge maxResponseBodyBytes)+        writeIORef totalRef total+        pure chunk   let readLine :: IO (Maybe ByteString)       readLine = do         buf <- readIORef bufRef

@@ -28,6 +28,7 @@     coalesceAdjacentTextParts,     validateTurn,     validateTurns,+    validateConversation,     opaqueForProvider,     stripForeignOpaque, @@ -71,6 +72,7 @@   ) import Data.Aeson.Types (Parser) import Data.Char (isSpace, isAsciiUpper, isAsciiLower, isDigit)+import Data.List (nub, sortOn, zip4) import Data.Maybe (mapMaybe) import Data.Text (Text) import Data.Text qualified as T@@ -80,7 +82,7 @@ -- | Provider adapter for one LLM backend. -- -- Implement the three callbacks to add a custom provider, or use the--- bundled constructors in 'LLM.Providers'.+-- bundled constructors in "LLM.Providers". data LLMGateway = LLMGateway   { -- | Short name used in logs and hooks (e.g. @openai@).     gwName :: Text,@@ -108,7 +110,7 @@ -- | Hooks for observing raw provider request and response JSON on the wire. -- -- Distinct from 'LLM.Generate.Logger.Hooks', which covers application-level--- logging and tool tracing. Bridge to this type via 'llmHooks'.+-- logging and tool tracing. Bridge to this type via 'LLM.Generate.GenerateUtils.llmHooks'. data LLMHooks = LLMHooks   { onLLMRequest :: Text -> Value -> IO (),     onLLMResponse :: Text -> Value -> IO (),@@ -128,7 +130,7 @@     meoIncludeReasoning :: Bool,     -- | Send @reasoning_content@ on every assistant message, @""@ when the     -- turn has no thinking text (DeepSeek thinking mode rejects assistant-    -- messages without it). Only takes effect with 'meoIncludeReasoning'.+    -- messages without it). Only takes effect with 'LLM.Core.Types.meoIncludeReasoning'.     meoRequireReasoning :: Bool   }   deriving (Show, Eq)@@ -137,8 +139,8 @@ defaultMessageEncodeOptions =   MessageEncodeOptions {meoIncludeReasoning = False, meoRequireReasoning = False} --- | DeepSeek encoding with thinking off. 'deepSeekBuildBodyPairs' sets--- 'meoRequireReasoning' when the request enables thinking.+-- | DeepSeek encoding with thinking off. @deepSeekBuildBodyPairs@ (in "LLM.Providers.DeepSeek") sets+-- 'LLM.Core.Types.meoRequireReasoning' when the request enables thinking. deepSeekMessageEncodeOptions :: MessageEncodeOptions deepSeekMessageEncodeOptions =   MessageEncodeOptions {meoIncludeReasoning = True, meoRequireReasoning = False}@@ -146,7 +148,7 @@ -- | Opaque, provider-owned payload that must round-trip for replay. -- -- Used for Claude thinking signatures, Gemini thought signatures, and any--- similar provider-bound state. Consumers must treat 'poPayload' as opaque.+-- similar provider-bound state. Consumers must treat 'LLM.Core.Types.poPayload' as opaque. data ProviderOpaque = ProviderOpaque   { poProvider :: Text,     poModel :: Maybe Text,@@ -214,7 +216,7 @@   ]  -- | MIME types accepted by 'mkImageBase64' / 'imageBase64Part'. Not every--- provider accepts all of them; see 'gwImageMediaTypes'.+-- provider accepts all of them; see 'LLM.Core.Types.gwImageMediaTypes'. supportedImageMediaTypes :: [Text] supportedImageMediaTypes =   commonImageMediaTypes ++ ["image/heic", "image/heif"]@@ -393,7 +395,7 @@ -- | Migration helper: emit thinking (if any), then text, then tool calls. -- -- Does not recover arbitrary provider block order; use 'AssistantMessage'--- with 'respContent' when replaying authoritative ordered parts.+-- with 'LLM.Core.Types.respContent' when replaying authoritative ordered parts. assistantTurn :: Text -> Maybe Text -> [ToolCall] -> Turn assistantTurn text mReasoning calls =   AssistantMessage $@@ -499,6 +501,51 @@ validateTurns :: [Turn] -> [(Int, Text)] validateTurns turns = [(i, reason) | (i, turn) <- zip [0 ..] turns, Left reason <- [validateTurn turn]] +-- | Problems in the tool-call structure of a conversation, as (index from+-- 0, reason), in index order. Empty when the structure is valid:+--+-- * a 'ToolTurn' directly follows an assistant turn with tool calls;+-- * the call IDs of an assistant turn are distinct, and that turn is+--   directly followed by a 'ToolTurn';+-- * the 'ToolTurn' answers each of those calls exactly once, and answers+--   no other call ID.+--+-- Providers reject such conversations or drop parts of them, each in its+-- own way. Every generation entry point logs these problems as warnings,+-- like 'validateTurns', and sends the request unchanged. From 0.5 they+-- will be errors.+--+-- @since 0.4.3.0+validateConversation :: [Turn] -> [(Int, Text)]+validateConversation turns =+  sortOn+    fst+    [ problem+      | (i, prev, turn, next) <- zip4 [0 ..] (Nothing : map Just turns) turns (map Just (drop 1 turns) ++ [Nothing]),+        problem <- check i prev turn next+    ]+  where+    check :: Int -> Maybe Turn -> Turn -> Maybe Turn -> [(Int, Text)]+    check i prev turn next = case turn of+      ToolTurn results -> case prev of+        Just (AssistantMessage parts)+          | calls@(_ : _) <- turnToolCalls parts ->+              answers i (map (.tcId) calls) (map (.trCallId) results)+        _ -> [(i, "tool results do not follow an assistant turn with tool calls")]+      AssistantMessage parts+        | ids@(_ : _) <- map (.tcId) (turnToolCalls parts) ->+            [(i, "duplicate tool call id " <> quote d) | d <- duplicates ids]+              ++ case next of+                Just (ToolTurn _) -> []+                _ -> [(i, "tool calls are not followed by tool results")]+      _ -> []+    answers i callIds resultIds =+      [(i, "no result for tool call " <> quote c) | c <- nub callIds, c `notElem` resultIds]+        ++ [(i, "tool call " <> quote r <> " is answered more than once") | r <- duplicates resultIds, r `elem` callIds]+        ++ [(i, "tool result for unknown call id " <> quote r) | r <- nub resultIds, r `notElem` callIds]+    duplicates xs = nub [x | x <- xs, length (filter (== x) xs) > 1]+    quote = T.pack . show+ -- | Keep opaque metadata only when it belongs to @provider@. opaqueForProvider :: Text -> Maybe ProviderOpaque -> Maybe ProviderOpaque opaqueForProvider provider (Just o)@@ -534,15 +581,15 @@     toolDescription :: Text,     toolParameters :: Value, -- JSON Schema object     -- | When 'True', the tool is advertised as read-only and remains available-    -- when 'RuntimeArgs.rtReadonly' is set. Mutating tools should use 'False'.+    -- when 'LLM.Agent.Types.rtReadonly' is set. Mutating tools should use 'False'.     toolReadonly :: Bool   }   deriving (Show, Eq) --- | Typed tool definition before conversion to 'Tool' via 'toTool'.+-- | Typed tool definition before conversion to t'LLM.Agent.Types.Tool' via 'LLM.Agent.ToolUtils.toTool'. ----- Define one with an Autodocodec argument type; 'toTool' derives the JSON--- Schema for 'ToolDef.toolParameters' automatically.+-- Define one with an Autodocodec argument type; 'LLM.Agent.ToolUtils.toTool' derives the JSON+-- Schema for 'LLM.Core.Types.toolParameters' automatically. data TypedTool c a = TypedTool   { ttoolName :: Text,     ttoolDescription :: Text,@@ -552,7 +599,7 @@  -- | A tool invocation returned by the model. ----- 'tcProviderMeta' is opaque, provider-owned state that must round-trip back+-- 'LLM.Core.Types.tcProviderMeta' is opaque, provider-owned state that must round-trip back -- to the same provider (and usually the same model) when this tool call is -- replayed. Gemini thought signatures are the current example. data ToolCall = ToolCall@@ -622,8 +669,8 @@  -- | A response from an LLM provider. ----- 'respContent' is authoritative ordered content. 'respText' and--- 'respReasoning' are convenience projections and may be lossy.+-- 'LLM.Core.Types.respContent' is authoritative ordered content. 'LLM.Core.Types.respText' and+-- 'LLM.Core.Types.respReasoning' are convenience projections and may be lossy. data ChatResponse = ChatResponse   { respText :: Text,     respContent :: [ContentPart],

@@ -17,7 +17,7 @@  -- | Token usage from a single API call. ----- 'usageInputTokens' is the total input token count, including tokens that were+-- 'LLM.Core.Usage.usageInputTokens' is the total input token count, including tokens that were -- cache reads or cache writes. Provider parsers normalize differing wire -- semantics so this total is never double-counted. --@@ -100,7 +100,7 @@  -- | Pricing in dollars per million tokens. ----- Optional cache rates fall back to 'pricePerMillionInput' when absent.+-- Optional cache rates fall back to 'LLM.Core.Usage.pricePerMillionInput' when absent. data PricingInfo = PricingInfo   { pricePerMillionInput :: Double,     pricePerMillionOutput :: Double,

@@ -187,7 +187,7 @@     (const action)  -- | Build a synthetic JSON summary from a streamed ChatResponse,--- used by providers to fire 'onResponse' after streaming completes.+-- used by providers to fire 'LLM.Generate.Logger.onResponse' after streaming completes. streamResponseJson :: ChatResponse -> Value streamResponseJson r =   object

@@ -9,193 +9,12 @@   ) where -import Control.Exception (Exception, handle, throwIO)-import Control.Monad (unless, when)-import Data.IORef (IORef, newIORef, readIORef, writeIORef)-import Data.Maybe (fromMaybe)-import LLM.Core.Abort (isAbortedMaybe)-import LLM.Core.Types-  ( ChatResponse (..),-    LLMError (Aborted),-    LLMGateway (..),-    LLMTextResult,-    StreamEvent (..),-    Turn (..),-  )-import LLM.Core.Usage (emptyUsage)-import LLM.Core.Utils (withFirstEventTimeout)-import LLM.Generate.GenerateUtils-  ( callWithRetry,-    callWithRetryTimeout,-    mkRequest,+import LLM.Generate.Internal.Generate+  ( generateTextLLM,+    generateTextWithFallbacks,+    generateTextWithFallbacksRotated,+    streamTextLLM,+    streamTextWithFallbacks,+    streamTextWithFallbacksRotated,     usageWithModelCost,-    safeGenRequest,-    withModelFallbacksRotated,   )-import LLM.Generate.ModelConfig-  ( ModelConfig (..),-    ModelWithFallbacks (..),-  )-import LLM.Generate.Types-  ( GenRequest (..),-    GenerateResult,-    RoundTextRole (..),-    StreamChunk (..),-  )---- | Generate text with retries and model fallbacks.------ Tries 'mwfModel' first, then each fallback in order. Returns a provider--- 'ChatResponse' on success; does not execute tools.-generateTextWithFallbacks ::-  GenRequest ->-  ModelWithFallbacks ->-  IO (GenerateResult ChatResponse)-generateTextWithFallbacks gr models =-  fmap snd <$> generateTextWithFallbacksRotated (safeGenRequest gr) models---- | Like 'generateTextWithFallbacks', but also returns the chain rotated so--- the model that answered comes first (see 'withModelFallbacksRotated').-generateTextWithFallbacksRotated ::-  GenRequest ->-  ModelWithFallbacks ->-  IO (GenerateResult (ModelWithFallbacks, ChatResponse))-generateTextWithFallbacksRotated gr0 models =-  withModelFallbacksRotated gr models $ \mc -> do-    r <- generateTextLLM gr mc-    case r of-      Left err -> pure $ Left err-      Right resp ->-        let usage = usageWithModelCost mc (fromMaybe emptyUsage resp.respUsage)-         in pure $ Right resp {respUsage = Just usage}-  where-    gr = safeGenRequest gr0---- | Like 'generateTextWithFallbacks', but streams 'StreamChunk' values--- through @onChunk@ as the provider responds.-streamTextWithFallbacks ::-  (StreamChunk -> IO ()) ->-  GenRequest ->-  ModelWithFallbacks ->-  IO (GenerateResult ChatResponse)-streamTextWithFallbacks onChunk gr models =-  fmap snd <$> streamTextWithFallbacksRotated onChunk (safeGenRequest gr) models---- | Like 'streamTextWithFallbacks', but also returns the rotated chain--- (see 'withModelFallbacksRotated').-streamTextWithFallbacksRotated ::-  (StreamChunk -> IO ()) ->-  GenRequest ->-  ModelWithFallbacks ->-  IO (GenerateResult (ModelWithFallbacks, ChatResponse))-streamTextWithFallbacksRotated onChunk gr0 models = do-  let gr = safeGenRequest gr0-  emittedRef <- newIORef False-  withModelFallbacksRotated gr models $ \mc -> do-    r <- streamTextAttempts emittedRef onChunk gr mc-    case r of-      Left err -> pure $ Left err-      Right resp ->-        let usage = usageWithModelCost mc (fromMaybe emptyUsage resp.respUsage)-         in pure $ Right resp {respUsage = Just usage}--generateTextLLM :: GenRequest -> ModelConfig -> IO LLMTextResult-generateTextLLM gr0 mc =-  callWithRetryTimeout gr mc $-    let request = mkRequest gr mc-     in mc.mcGateway.gwGenerateText gr.grLLMHooks request-  where-    gr = safeGenRequest gr0---- | Stream one model call with retries. 'mcRequestTimeout' bounds the time--- until the first stream event of each attempt, not the whole stream.------ If an attempt fails after streaming chunks, the next attempt first emits--- 'StreamAttemptReset'.-streamTextLLM :: (StreamChunk -> IO ()) -> GenRequest -> ModelConfig -> IO LLMTextResult-streamTextLLM onChunk gr mc = do-  emittedRef <- newIORef False-  streamTextAttempts emittedRef onChunk (safeGenRequest gr) mc---- | 'streamTextLLM' with the \"chunks emitted\" flag supplied by the caller,--- so it is shared across retries and model fallbacks of one model call.-streamTextAttempts ::-  IORef Bool ->-  (StreamChunk -> IO ()) ->-  GenRequest ->-  ModelConfig ->-  IO LLMTextResult-streamTextAttempts emittedRef onChunk gr mc =-  callWithRetry gr mc $-    let request = mkRequest gr mc-        onChunk' c = writeIORef emittedRef True >> onChunk c-     in do-          emitted <- readIORef emittedRef-          when emitted $ do-            onChunk StreamAttemptReset-            writeIORef emittedRef False-          ProviderStreamCallback {pscOnEvent, pscFinalize} <--            mkProviderStreamCallback gr onChunk'-          -- Deliver no event after the abort signal fired. ('callWithRetry'-          -- also interrupts the attempt, but asynchronously, so an event-          -- could otherwise slip through first.)-          let onEvent e = do-                aborted <- isAbortedMaybe gr.grAbortSignal-                if aborted then throwIO StreamAborted else pscOnEvent e-          result <--            handle (\StreamAborted -> pure (Left Aborted)) $-              withFirstEventTimeout-                mc.mcRequestTimeout-                (mc.mcGateway.gwStreamText gr.grLLMHooks request)-                onEvent-          case result of-            Right _ -> pscFinalize >> pure result-            Left err -> pure (Left err)---- | Thrown from the stream callback once the abort signal has fired.-data StreamAborted = StreamAborted-  deriving (Show)--instance Exception StreamAborted--data RoundPhase = Unclassified | Preamble | Answer-  deriving (Eq)--data ProviderStreamCallback = ProviderStreamCallback-  { pscOnEvent :: StreamEvent -> IO (),-    pscFinalize :: IO ()-  }--initialPhase :: [Turn] -> RoundPhase-initialPhase turns =-  case reverse turns of-    ToolTurn _ : _ -> Answer-    _ -> Unclassified--mkProviderStreamCallback ::-  GenRequest ->-  (StreamChunk -> IO ()) ->-  IO ProviderStreamCallback-mkProviderStreamCallback gr onChunk = do-  phaseRef <- newIORef (initialPhase gr.grMessages)-  pure-    ProviderStreamCallback-      { pscOnEvent = \case-          StreamReasoningDelta txt -> onChunk (ReasoningDelta txt)-          StreamDelta txt -> do-            phase <- readIORef phaseRef-            case phase of-              Answer -> onChunk (AnswerDelta txt)-              Preamble -> onChunk (PreambleDelta txt)-              Unclassified -> onChunk (TextDelta txt)-          StreamToolCall tc -> do-            phase <- readIORef phaseRef-            unless (phase == Preamble) $-              onChunk (RoundTextRoleCommitted PreambleRole)-            writeIORef phaseRef Preamble-            onChunk (StreamToolCallChunk tc),-        pscFinalize = do-          phase <- readIORef phaseRef-          when (phase == Unclassified) $-            onChunk (RoundTextRoleCommitted AnswerRole)-      }

@@ -11,6 +11,7 @@ import Data.Maybe (fromMaybe) import Data.Text qualified as T import LLM.Core.Abort (isAbortedMaybe)+import LLM.Core.Internal.Schema (dropOptionalNulls) import LLM.Core.ProviderUtils (stripBoundsAndComments) import LLM.Core.Types (LLMGateway (..), LLMObjectResult) import LLM.Core.Usage (Usage (..), emptyUsage)@@ -52,7 +53,10 @@   case res of     Left e -> pure (Left e)     Right (v, u) -> do-      case AE.fromJSON v of+      -- A provider in strict mode answers null for an optional field+      -- (see normalizeSchemaOpenAI); the codec reads it as absent only+      -- without the null.+      case AE.fromJSON (dropOptionalNulls jsonschema v) of         AE.Error e ->           pure $             Left $

@@ -12,290 +12,15 @@   ) where -import Control.Concurrent (threadDelay)-import Control.Monad (when)-import Data.Text (Text)-import Data.Text qualified as T-import Data.List (nub)-import Data.Maybe (fromMaybe)-import LLM.Core.Abort (isAbortedMaybe)-import LLM.Core.Types-  ( ChatRequest (..),-    ContentPart (..),-    ImageSource (..),-    LLMError (..),-    LLMGateway (gwImageMediaTypes, gwImageUrlSupport, gwName),-    PartBody (..),-    LLMHooks (..),-    ThinkingMode (..),-    Turn (..),-    conversationHasImages,-    validateTurns,-  )-import LLM.Core.Usage (Usage (..), estimateCost)-import LLM.Core.Utils (interruptOnAbort, withRetry, withTimeout)-import LLM.Generate.Logger (Hooks (..), LogLevel (..), safeHooks, safeLLMHooks)-import LLM.Generate.ModelConfig-  ( ModelCapabilities (..),-    ModelConfig (..),-    ModelWithFallbacks (..),-    mfwToModelConfigs,-    modelRetryPolicy,+import LLM.Generate.Internal.GenerateUtils+  ( callWithRetry,+    callWithRetryTimeout,+    llmHooks,+    maybeThrottle,+    mkRequest,+    safeGenRequest,+    usageWithModelCost,+    validateModelCapabilities,+    withModelFallbacks,+    withModelFallbacksRotated,   )-import LLM.Generate.Types (GenRequest (..), GenerateError (..), GenerateResult)--maybeThrottle :: Maybe Int -> IO a -> IO a-maybeThrottle Nothing io = io-maybeThrottle (Just ms) io = threadDelay (ms * 1000) >> io--mkRequest :: GenRequest -> ModelConfig -> ChatRequest-mkRequest gr mc =-  ChatRequest-    { reqModel = mc.mcModel,-      reqConversation = gr.grMessages,-      reqSystem = gr.grSystemPrompt,-      reqMaxTokens = mc.mcMaxTokens,-      reqTemperature = mc.mcTemperature,-      reqTools = gr.grTools,-      reqThinking = mc.mcThinking-    }--usageWithModelCost :: ModelConfig -> Usage -> Usage-usageWithModelCost mc u = u {usageTotalCost = estimateCost mc.mcPricing u}---- | Check that a candidate model declares every capability the request needs.------ Unsupported vision is a candidate failure (do not drop images), and so is--- an inline image whose media type the gateway does not accept--- ('gwImageMediaTypes') or a URL image on a gateway without--- 'gwImageUrlSupport'. URL images have no known media type, so their type--- is not checked.--- Thinking configuration requires a declared thinking capability. Cache--- hints are not validated here; unsupported providers ignore them.-validateModelCapabilities :: ModelConfig -> [Turn] -> Either LLMError ()-validateModelCapabilities mc turns = do-  whenNeedsVision-  whenNeedsMediaTypes-  whenNeedsImageUrls-  whenNeedsThinking-  where-    caps = mc.mcCapabilities-    whenNeedsVision-      | conversationHasImages turns,-        not caps.capVision =-          Left $-            UnsupportedCapability $-              "model "-                <> mc.mcModel-                <> " does not support vision"-      | otherwise = Right ()-    whenNeedsMediaTypes =-      case filter (`notElem` mc.mcGateway.gwImageMediaTypes) (inlineImageMediaTypes turns) of-        [] -> Right ()-        unsupported ->-          Left $-            UnsupportedCapability $-              "model "-                <> mc.mcModel-                <> " via "-                <> mc.mcGateway.gwName-                <> " does not accept image media type "-                <> T.intercalate ", " unsupported-    whenNeedsImageUrls-      | conversationHasImageUrls turns,-        not mc.mcGateway.gwImageUrlSupport =-          Left $-            UnsupportedCapability $-              "model "-                <> mc.mcModel-                <> " via "-                <> mc.mcGateway.gwName-                <> " does not accept image URLs"-      | otherwise = Right ()-    whenNeedsThinking =-      case mc.mcThinking of-        Just ThinkingMode {tmEnabled = True}-          | not caps.capThinking ->-              Left $-                UnsupportedCapability $-                  "model "-                    <> mc.mcModel-                    <> " does not support thinking"-        _ -> Right ()---- | Distinct media types of the inline ('ImageBase64') images in a conversation.-inlineImageMediaTypes :: [Turn] -> [Text]-inlineImageMediaTypes turns =-  nub [mt | ImageBase64 mt _ <- imageSources turns]--conversationHasImageUrls :: [Turn] -> Bool-conversationHasImageUrls turns =-  not (null [() | ImageUrl _ <- imageSources turns])--imageSources :: [Turn] -> [ImageSource]-imageSources turns =-  [src | parts <- map turnParts turns, ContentPart (ImagePart src) _ <- parts]-  where-    turnParts (UserMessage parts) = parts-    turnParts (AssistantMessage parts) = parts-    turnParts (ToolTurn _) = []---- | Run one model call with the model's throttle delay and retry policy.--- @attempt@ is run once per try; any per-attempt timeout must be inside it.------ 'mcThrottleDelay' is slept before every try, retries included. Before a--- retry it comes after the retry backoff, so the two delays add up.------ When 'grAbortSignal' fires, the call is interrupted wherever it is (in an--- attempt, or in the throttle or backoff sleep) and returns 'Aborted'. An--- interrupted attempt is cancelled, which closes its HTTP connection.-callWithRetry ::-  GenRequest ->-  ModelConfig ->-  IO (Either LLMError a) ->-  IO (Either LLMError a)-callWithRetry gr mc attempt =-  fmap (fromMaybe (Left Aborted)) $-    interruptOnAbort gr.grAbortSignal $-      withRetry-        (modelRetryPolicy mc)-        (gr.grHooks.onLog Warn)-        (maybeThrottle mc.mcThrottleDelay (unlessAborted gr attempt))---- | Run @act@ unless 'grAbortSignal' has fired, else return 'Aborted'--- (which is not retried). Checked before every attempt, so also after--- throttle and backoff sleeps.-unlessAborted :: GenRequest -> IO (Either LLMError a) -> IO (Either LLMError a)-unlessAborted gr act = do-  aborted <- isAbortedMaybe gr.grAbortSignal-  if aborted then pure (Left Aborted) else act---- | 'callWithRetry' where each attempt is limited by 'mcRequestTimeout'.------ The timeout applies to each attempt separately, not to the whole--- sequence of retries and backoff sleeps. A timed-out attempt yields--- 'TimeoutError', which is retryable. For streaming, see--- 'LLM.Core.Utils.withFirstEventTimeout' (used by 'streamTextLLM').-callWithRetryTimeout ::-  GenRequest ->-  ModelConfig ->-  IO (Either LLMError a) ->-  IO (Either LLMError a)-callWithRetryTimeout gr mc invoke =-  callWithRetry gr mc (withTimeout mc.mcRequestTimeout invoke)--withModelFallbacks ::-  GenRequest ->-  ModelWithFallbacks ->-  (ModelConfig -> IO (Either LLMError a)) ->-  IO (GenerateResult a)-withModelFallbacks gr models invokePerModel =-  fmap (\(_, _, a) -> a) <$> runModelFallbacks gr models invokePerModel---- | Like 'withModelFallbacks', but also returns the chain rotated so that the--- model that answered comes first. The models before it (the ones that--- failed) move to the end, so they stay a last resort:--- @xs ++ [m] ++ ys@ becomes @ModelWithFallbacks m (ys ++ xs)@.------ Used to keep an agent run on the model that answered. Logs at 'Info' when--- the chain changes.-withModelFallbacksRotated ::-  GenRequest ->-  ModelWithFallbacks ->-  (ModelConfig -> IO (Either LLMError a)) ->-  IO (GenerateResult (ModelWithFallbacks, a))-withModelFallbacksRotated gr models invokePerModel = do-  r <- runModelFallbacks gr models invokePerModel-  case r of-    Left err -> pure $ Left err-    Right (rotated, chain, a) -> do-      when rotated $-        gr.grHooks.onLog Info (formatContinuingOn chain.mwfModel)-      pure $ Right (chain, a)---- | Shared loop of 'withModelFallbacks' and 'withModelFallbacksRotated'.--- On success, returns whether the chain was rotated, the rotated chain and--- the result.------ Before the first model is tried, each invalid turn in 'grMessages'--- ('validateTurns') is logged at 'Warn' as @turn <n>: <reason>@ (@n@--- counts from 0). The request is sent unchanged.-runModelFallbacks ::-  GenRequest ->-  ModelWithFallbacks ->-  (ModelConfig -> IO (Either LLMError a)) ->-  IO (GenerateResult (Bool, ModelWithFallbacks, a))-runModelFallbacks gr models invokePerModel = do-  mapM_-    (\(i, reason) -> gr.grHooks.onLog Warn ("turn " <> T.pack (show i) <> ": " <> reason))-    (validateTurns gr.grMessages)-  case mfwToModelConfigs models of-    [] -> pure $ Left GErrAllModelsFailed-    modelConfigs -> loop [] modelConfigs Nothing-  where-    loop _ [] mLast =-      pure $-        Left $-          maybe GErrAllModelsFailed GErrLLM mLast-    loop failed (mc : rest) mLast = do-      aborted <- isAbortedMaybe gr.grAbortSignal-      if aborted then pure (Left GErrAborted) else tryModel failed mc rest mLast-    tryModel failed mc rest _ = do-      gr.grHooks.onLog Info (formatTryingModel mc)-      case validateModelCapabilities mc gr.grMessages of-        Left err ->-          case rest of-            [] -> pure $ Left (GErrLLM err)-            _ -> do-              gr.grHooks.onLog Warn (formatModelFallback mc err)-              loop (failed ++ [mc]) rest (Just err)-        Right () -> do-          r <- invokePerModel mc-          case r of-            Left Aborted -> pure $ Left GErrAborted-            Left err ->-              case rest of-                [] -> pure $ Left (GErrLLM err)-                _ -> do-                  gr.grHooks.onLog Warn (formatModelFallback mc err)-                  loop (failed ++ [mc]) rest (Just err)-            Right a -> pure $ Right (not (null failed), ModelWithFallbacks mc (rest ++ failed), a)--formatTryingModel :: ModelConfig -> Text-formatTryingModel mc =-  "Trying model: "-    <> mc.mcModel-    <> " via "-    <> mc.mcGateway.gwName--formatContinuingOn :: ModelConfig -> Text-formatContinuingOn mc =-  "Continuing run on "-    <> mc.mcModel-    <> " via "-    <> mc.mcGateway.gwName--formatModelFallback :: ModelConfig -> LLMError -> Text-formatModelFallback mc err =-  "Falling back from "-    <> mc.mcModel-    <> ": "-    <> T.pack (show err)--llmHooks :: Hooks -> LLMHooks-llmHooks hooks =-  LLMHooks-    { onLLMRequest = hooks.onRequest,-      onLLMResponse = hooks.onResponse,-      onLLMResponseError = hooks.onResponseError-    }---- | The request with its hooks wrapped by 'safeHooks' and 'safeLLMHooks',--- so an exception thrown by a hook cannot fail a call whose side effects--- already happened. Applied by every public generation entry point.-safeGenRequest :: GenRequest -> GenRequest-safeGenRequest gr =-  gr-    { grHooks = safeHooks gr.grHooks,-      grLLMHooks = safeLLMHooks gr.grLLMHooks-    }

@@ -0,0 +1,223 @@+-- | Implementation of "LLM.Generate.Generate", plus the variants that skip+-- the turn check ('TurnCheck') for the agent loop.+module LLM.Generate.Internal.Generate+  ( usageWithModelCost,+    generateTextLLM,+    streamTextLLM,+    generateTextWithFallbacks,+    streamTextWithFallbacks,+    generateTextWithFallbacksRotated,+    streamTextWithFallbacksRotated,+    generateTextWithFallbacksRotatedWith,+    streamTextWithFallbacksRotatedWith,+  )+where++import Control.Exception (Exception, handle, throwIO)+import Control.Monad (unless, when)+import Data.IORef (IORef, newIORef, readIORef, writeIORef)+import Data.Maybe (fromMaybe)+import LLM.Core.Abort (isAbortedMaybe)+import LLM.Core.Types+  ( ChatResponse (..),+    LLMError (Aborted),+    LLMGateway (..),+    LLMTextResult,+    StreamEvent (..),+    Turn (..),+  )+import LLM.Core.Usage (emptyUsage)+import LLM.Core.Utils (withFirstEventTimeout)+import LLM.Generate.Internal.GenerateUtils+  ( callWithRetry,+    callWithRetryTimeout,+    mkRequest,+    usageWithModelCost,+    safeGenRequest,+    TurnCheck (..),+    withModelFallbacksRotatedWith,+  )+import LLM.Generate.ModelConfig+  ( ModelConfig (..),+    ModelWithFallbacks (..),+  )+import LLM.Generate.Types+  ( GenRequest (..),+    GenerateResult,+    RoundTextRole (..),+    StreamChunk (..),+  )++-- | Generate text with retries and model fallbacks.+--+-- Tries 'LLM.Generate.ModelConfig.mwfModel' first, then each fallback in order. Returns a provider+-- 'ChatResponse' on success; does not execute tools.+generateTextWithFallbacks ::+  GenRequest ->+  ModelWithFallbacks ->+  IO (GenerateResult ChatResponse)+generateTextWithFallbacks gr models =+  fmap snd <$> generateTextWithFallbacksRotated (safeGenRequest gr) models++-- | Like 'generateTextWithFallbacks', but also returns the chain rotated so+-- the model that answered comes first (see 'LLM.Generate.GenerateUtils.withModelFallbacksRotated').+generateTextWithFallbacksRotated ::+  GenRequest ->+  ModelWithFallbacks ->+  IO (GenerateResult (ModelWithFallbacks, ChatResponse))+generateTextWithFallbacksRotated = generateTextWithFallbacksRotatedWith CheckTurns++-- | 'generateTextWithFallbacksRotated' with the choice of 'TurnCheck'.+generateTextWithFallbacksRotatedWith ::+  TurnCheck ->+  GenRequest ->+  ModelWithFallbacks ->+  IO (GenerateResult (ModelWithFallbacks, ChatResponse))+generateTextWithFallbacksRotatedWith check gr0 models =+  withModelFallbacksRotatedWith check gr models $ \mc -> do+    r <- generateTextLLM gr mc+    case r of+      Left err -> pure $ Left err+      Right resp ->+        let usage = usageWithModelCost mc (fromMaybe emptyUsage resp.respUsage)+         in pure $ Right resp {respUsage = Just usage}+  where+    gr = safeGenRequest gr0++-- | Like 'generateTextWithFallbacks', but streams 'StreamChunk' values+-- through @onChunk@ as the provider responds.+streamTextWithFallbacks ::+  (StreamChunk -> IO ()) ->+  GenRequest ->+  ModelWithFallbacks ->+  IO (GenerateResult ChatResponse)+streamTextWithFallbacks onChunk gr models =+  fmap snd <$> streamTextWithFallbacksRotated onChunk (safeGenRequest gr) models++-- | Like 'streamTextWithFallbacks', but also returns the rotated chain+-- (see 'LLM.Generate.GenerateUtils.withModelFallbacksRotated').+streamTextWithFallbacksRotated ::+  (StreamChunk -> IO ()) ->+  GenRequest ->+  ModelWithFallbacks ->+  IO (GenerateResult (ModelWithFallbacks, ChatResponse))+streamTextWithFallbacksRotated = streamTextWithFallbacksRotatedWith CheckTurns++-- | 'streamTextWithFallbacksRotated' with the choice of 'TurnCheck'.+streamTextWithFallbacksRotatedWith ::+  TurnCheck ->+  (StreamChunk -> IO ()) ->+  GenRequest ->+  ModelWithFallbacks ->+  IO (GenerateResult (ModelWithFallbacks, ChatResponse))+streamTextWithFallbacksRotatedWith check onChunk gr0 models = do+  let gr = safeGenRequest gr0+  emittedRef <- newIORef False+  withModelFallbacksRotatedWith check gr models $ \mc -> do+    r <- streamTextAttempts emittedRef onChunk gr mc+    case r of+      Left err -> pure $ Left err+      Right resp ->+        let usage = usageWithModelCost mc (fromMaybe emptyUsage resp.respUsage)+         in pure $ Right resp {respUsage = Just usage}++generateTextLLM :: GenRequest -> ModelConfig -> IO LLMTextResult+generateTextLLM gr0 mc =+  callWithRetryTimeout gr mc $+    let request = mkRequest gr mc+     in mc.mcGateway.gwGenerateText gr.grLLMHooks request+  where+    gr = safeGenRequest gr0++-- | Stream one model call with retries. 'LLM.Generate.ModelConfig.mcRequestTimeout' bounds the time+-- until the first stream event of each attempt, not the whole stream.+--+-- If an attempt fails after streaming chunks, the next attempt first emits+-- 'StreamAttemptReset'.+streamTextLLM :: (StreamChunk -> IO ()) -> GenRequest -> ModelConfig -> IO LLMTextResult+streamTextLLM onChunk gr mc = do+  emittedRef <- newIORef False+  streamTextAttempts emittedRef onChunk (safeGenRequest gr) mc++-- | 'streamTextLLM' with the \"chunks emitted\" flag supplied by the caller,+-- so it is shared across retries and model fallbacks of one model call.+streamTextAttempts ::+  IORef Bool ->+  (StreamChunk -> IO ()) ->+  GenRequest ->+  ModelConfig ->+  IO LLMTextResult+streamTextAttempts emittedRef onChunk gr mc =+  callWithRetry gr mc $+    let request = mkRequest gr mc+        onChunk' c = writeIORef emittedRef True >> onChunk c+     in do+          emitted <- readIORef emittedRef+          when emitted $ do+            onChunk StreamAttemptReset+            writeIORef emittedRef False+          ProviderStreamCallback {pscOnEvent, pscFinalize} <-+            mkProviderStreamCallback gr onChunk'+          -- Deliver no event after the abort signal fired. ('callWithRetry'+          -- also interrupts the attempt, but asynchronously, so an event+          -- could otherwise slip through first.)+          let onEvent e = do+                aborted <- isAbortedMaybe gr.grAbortSignal+                if aborted then throwIO StreamAborted else pscOnEvent e+          result <-+            handle (\StreamAborted -> pure (Left Aborted)) $+              withFirstEventTimeout+                mc.mcRequestTimeout+                (mc.mcGateway.gwStreamText gr.grLLMHooks request)+                onEvent+          case result of+            Right _ -> pscFinalize >> pure result+            Left err -> pure (Left err)++-- | Thrown from the stream callback once the abort signal has fired.+data StreamAborted = StreamAborted+  deriving (Show)++instance Exception StreamAborted++data RoundPhase = Unclassified | Preamble | Answer+  deriving (Eq)++data ProviderStreamCallback = ProviderStreamCallback+  { pscOnEvent :: StreamEvent -> IO (),+    pscFinalize :: IO ()+  }++initialPhase :: [Turn] -> RoundPhase+initialPhase turns =+  case reverse turns of+    ToolTurn _ : _ -> Answer+    _ -> Unclassified++mkProviderStreamCallback ::+  GenRequest ->+  (StreamChunk -> IO ()) ->+  IO ProviderStreamCallback+mkProviderStreamCallback gr onChunk = do+  phaseRef <- newIORef (initialPhase gr.grMessages)+  pure+    ProviderStreamCallback+      { pscOnEvent = \case+          StreamReasoningDelta txt -> onChunk (ReasoningDelta txt)+          StreamDelta txt -> do+            phase <- readIORef phaseRef+            case phase of+              Answer -> onChunk (AnswerDelta txt)+              Preamble -> onChunk (PreambleDelta txt)+              Unclassified -> onChunk (TextDelta txt)+          StreamToolCall tc -> do+            phase <- readIORef phaseRef+            unless (phase == Preamble) $+              onChunk (RoundTextRoleCommitted PreambleRole)+            writeIORef phaseRef Preamble+            onChunk (StreamToolCallChunk tc),+        pscFinalize = do+          phase <- readIORef phaseRef+          when (phase == Unclassified) $+            onChunk (RoundTextRoleCommitted AnswerRole)+      }

@@ -0,0 +1,334 @@+-- | Implementation of "LLM.Generate.GenerateUtils", plus the variants of+-- the fallback loop that the agent loop uses ('TurnCheck').+module LLM.Generate.Internal.GenerateUtils+  ( maybeThrottle,+    mkRequest,+    usageWithModelCost,+    callWithRetry,+    callWithRetryTimeout,+    withModelFallbacks,+    withModelFallbacksRotated,+    TurnCheck (..),+    withModelFallbacksRotatedWith,+    logInvalidTurns,+    validateModelCapabilities,+    llmHooks,+    safeGenRequest,+  )+where++import Control.Concurrent (threadDelay)+import Control.Monad (when)+import Data.Text (Text)+import Data.Text qualified as T+import Data.List (nub, sortOn)+import Data.Maybe (fromMaybe)+import LLM.Core.Abort (isAbortedMaybe)+import LLM.Core.Types+  ( ChatRequest (..),+    ContentPart (..),+    ImageSource (..),+    LLMError (..),+    LLMGateway (gwImageMediaTypes, gwImageUrlSupport, gwName),+    PartBody (..),+    LLMHooks (..),+    ThinkingMode (..),+    Turn (..),+    conversationHasImages,+    validateConversation,+    validateTurns,+  )+import LLM.Core.Usage (Usage (..), estimateCost)+import LLM.Core.Utils (interruptOnAbort, withRetry, withTimeout)+import LLM.Generate.Logger (Hooks (..), LogLevel (..), safeHooks, safeLLMHooks)+import LLM.Generate.ModelConfig+  ( ModelCapabilities (..),+    ModelConfig (..),+    ModelWithFallbacks (..),+    mfwToModelConfigs,+    modelRetryPolicy,+  )+import LLM.Generate.Types (GenRequest (..), GenerateError (..), GenerateResult)++maybeThrottle :: Maybe Int -> IO a -> IO a+maybeThrottle Nothing io = io+maybeThrottle (Just ms) io = threadDelay (ms * 1000) >> io++mkRequest :: GenRequest -> ModelConfig -> ChatRequest+mkRequest gr mc =+  ChatRequest+    { reqModel = mc.mcModel,+      reqConversation = gr.grMessages,+      reqSystem = gr.grSystemPrompt,+      reqMaxTokens = mc.mcMaxTokens,+      reqTemperature = mc.mcTemperature,+      reqTools = gr.grTools,+      reqThinking = mc.mcThinking+    }++usageWithModelCost :: ModelConfig -> Usage -> Usage+usageWithModelCost mc u = u {usageTotalCost = estimateCost mc.mcPricing u}++-- | Check that a candidate model declares every capability the request needs.+--+-- Unsupported vision is a candidate failure (do not drop images), and so is+-- an inline image whose media type the gateway does not accept+-- ('LLM.Core.Types.gwImageMediaTypes') or a URL image on a gateway without+-- 'LLM.Core.Types.gwImageUrlSupport'. URL images have no known media type, so their type+-- is not checked.+-- Thinking configuration requires a declared thinking capability. Cache+-- hints are not validated here; unsupported providers ignore them.+validateModelCapabilities :: ModelConfig -> [Turn] -> Either LLMError ()+validateModelCapabilities mc turns = do+  whenNeedsVision+  whenNeedsMediaTypes+  whenNeedsImageUrls+  whenNeedsThinking+  where+    caps = mc.mcCapabilities+    whenNeedsVision+      | conversationHasImages turns,+        not caps.capVision =+          Left $+            UnsupportedCapability $+              "model "+                <> mc.mcModel+                <> " does not support vision"+      | otherwise = Right ()+    whenNeedsMediaTypes =+      case filter (`notElem` mc.mcGateway.gwImageMediaTypes) (inlineImageMediaTypes turns) of+        [] -> Right ()+        unsupported ->+          Left $+            UnsupportedCapability $+              "model "+                <> mc.mcModel+                <> " via "+                <> mc.mcGateway.gwName+                <> " does not accept image media type "+                <> T.intercalate ", " unsupported+    whenNeedsImageUrls+      | conversationHasImageUrls turns,+        not mc.mcGateway.gwImageUrlSupport =+          Left $+            UnsupportedCapability $+              "model "+                <> mc.mcModel+                <> " via "+                <> mc.mcGateway.gwName+                <> " does not accept image URLs"+      | otherwise = Right ()+    whenNeedsThinking =+      case mc.mcThinking of+        Just ThinkingMode {tmEnabled = True}+          | not caps.capThinking ->+              Left $+                UnsupportedCapability $+                  "model "+                    <> mc.mcModel+                    <> " does not support thinking"+        _ -> Right ()++-- | Distinct media types of the inline ('ImageBase64') images in a conversation.+inlineImageMediaTypes :: [Turn] -> [Text]+inlineImageMediaTypes turns =+  nub [mt | ImageBase64 mt _ <- imageSources turns]++conversationHasImageUrls :: [Turn] -> Bool+conversationHasImageUrls turns =+  not (null [() | ImageUrl _ <- imageSources turns])++imageSources :: [Turn] -> [ImageSource]+imageSources turns =+  [src | parts <- map turnParts turns, ContentPart (ImagePart src) _ <- parts]+  where+    turnParts (UserMessage parts) = parts+    turnParts (AssistantMessage parts) = parts+    turnParts (ToolTurn _) = []++-- | Run one model call with the model's throttle delay and retry policy.+-- @attempt@ is run once per try; any per-attempt timeout must be inside it.+--+-- 'LLM.Generate.ModelConfig.mcThrottleDelay' is slept before every try, retries included. Before a+-- retry it comes after the retry backoff, so the two delays add up.+--+-- When 'LLM.Generate.Types.grAbortSignal' fires, the call is interrupted wherever it is (in an+-- attempt, or in the throttle or backoff sleep) and returns 'Aborted'. An+-- interrupted attempt is cancelled, which closes its HTTP connection.+callWithRetry ::+  GenRequest ->+  ModelConfig ->+  IO (Either LLMError a) ->+  IO (Either LLMError a)+callWithRetry gr mc attempt =+  fmap (fromMaybe (Left Aborted)) $+    interruptOnAbort gr.grAbortSignal $+      withRetry+        (modelRetryPolicy mc)+        (gr.grHooks.onLog Warn)+        (maybeThrottle mc.mcThrottleDelay (unlessAborted gr attempt))++-- | Run @act@ unless 'LLM.Generate.Types.grAbortSignal' has fired, else return 'Aborted'+-- (which is not retried). Checked before every attempt, so also after+-- throttle and backoff sleeps.+unlessAborted :: GenRequest -> IO (Either LLMError a) -> IO (Either LLMError a)+unlessAborted gr act = do+  aborted <- isAbortedMaybe gr.grAbortSignal+  if aborted then pure (Left Aborted) else act++-- | 'callWithRetry' where each attempt is limited by 'LLM.Generate.ModelConfig.mcRequestTimeout'.+--+-- The timeout applies to each attempt separately, not to the whole+-- sequence of retries and backoff sleeps. A timed-out attempt yields+-- 'TimeoutError', which is retryable. For streaming, see+-- 'LLM.Core.Utils.withFirstEventTimeout' (used by 'LLM.Generate.Generate.streamTextLLM').+callWithRetryTimeout ::+  GenRequest ->+  ModelConfig ->+  IO (Either LLMError a) ->+  IO (Either LLMError a)+callWithRetryTimeout gr mc invoke =+  callWithRetry gr mc (withTimeout mc.mcRequestTimeout invoke)++withModelFallbacks ::+  GenRequest ->+  ModelWithFallbacks ->+  (ModelConfig -> IO (Either LLMError a)) ->+  IO (GenerateResult a)+withModelFallbacks gr models invokePerModel =+  fmap (\(_, _, a) -> a) <$> runModelFallbacks CheckTurns gr models invokePerModel++-- | Like 'withModelFallbacks', but also returns the chain rotated so that the+-- model that answered comes first. The models before it (the ones that+-- failed) move to the end, so they stay a last resort:+-- @xs ++ [m] ++ ys@ becomes @ModelWithFallbacks m (ys ++ xs)@.+--+-- Used to keep an agent run on the model that answered. Logs at 'Info' when+-- the chain changes.+withModelFallbacksRotated ::+  GenRequest ->+  ModelWithFallbacks ->+  (ModelConfig -> IO (Either LLMError a)) ->+  IO (GenerateResult (ModelWithFallbacks, a))+withModelFallbacksRotated = withModelFallbacksRotatedWith CheckTurns++-- | Whether the fallback loop checks the request's messages first.+data TurnCheck+  = -- | Log the problems of @grMessages@ before the first model+    -- ('logInvalidTurns'). What every public entry point does.+    CheckTurns+  | -- | Do not check. For the agent loop, which checks the caller's turns+    -- once per run; the turns it adds come from providers.+    SkipTurnCheck++-- | 'withModelFallbacksRotated' with the choice of 'TurnCheck'.+withModelFallbacksRotatedWith ::+  TurnCheck ->+  GenRequest ->+  ModelWithFallbacks ->+  (ModelConfig -> IO (Either LLMError a)) ->+  IO (GenerateResult (ModelWithFallbacks, a))+withModelFallbacksRotatedWith check gr models invokePerModel = do+  r <- runModelFallbacks check gr models invokePerModel+  case r of+    Left err -> pure $ Left err+    Right (rotated, chain, a) -> do+      when rotated $+        gr.grHooks.onLog Info (formatContinuingOn chain.mwfModel)+      pure $ Right (chain, a)++-- | Shared loop of 'withModelFallbacks' and 'withModelFallbacksRotated'.+-- On success, returns whether the chain was rotated, the rotated chain and+-- the result.+--+-- With 'CheckTurns', 'logInvalidTurns' runs before the first model is+-- tried. The request is sent unchanged.+runModelFallbacks ::+  TurnCheck ->+  GenRequest ->+  ModelWithFallbacks ->+  (ModelConfig -> IO (Either LLMError a)) ->+  IO (GenerateResult (Bool, ModelWithFallbacks, a))+runModelFallbacks check gr models invokePerModel = do+  case check of+    CheckTurns -> logInvalidTurns gr.grHooks gr.grMessages+    SkipTurnCheck -> pure ()+  case mfwToModelConfigs models of+    [] -> pure $ Left GErrAllModelsFailed+    modelConfigs -> loop [] modelConfigs Nothing+  where+    loop _ [] mLast =+      pure $+        Left $+          maybe GErrAllModelsFailed GErrLLM mLast+    loop failed (mc : rest) mLast = do+      aborted <- isAbortedMaybe gr.grAbortSignal+      if aborted then pure (Left GErrAborted) else tryModel failed mc rest mLast+    tryModel failed mc rest _ = do+      gr.grHooks.onLog Info (formatTryingModel mc)+      case validateModelCapabilities mc gr.grMessages of+        Left err ->+          case rest of+            [] -> pure $ Left (GErrLLM err)+            _ -> do+              gr.grHooks.onLog Warn (formatModelFallback mc err)+              loop (failed ++ [mc]) rest (Just err)+        Right () -> do+          r <- invokePerModel mc+          case r of+            Left Aborted -> pure $ Left GErrAborted+            Left err ->+              case rest of+                [] -> pure $ Left (GErrLLM err)+                _ -> do+                  gr.grHooks.onLog Warn (formatModelFallback mc err)+                  loop (failed ++ [mc]) rest (Just err)+            Right a -> pure $ Right (not (null failed), ModelWithFallbacks mc (rest ++ failed), a)++-- | Log each invalid turn ('validateTurns') and each problem in the+-- tool-call structure ('validateConversation') at 'Warn' as+-- @turn <n>: <reason>@ (@n@ counts from 0), in turn order.+logInvalidTurns :: Hooks -> [Turn] -> IO ()+logInvalidTurns hooks turns =+  mapM_+    (\(i, reason) -> hooks.onLog Warn ("turn " <> T.pack (show i) <> ": " <> reason))+    (sortOn fst (validateTurns turns ++ validateConversation turns))++formatTryingModel :: ModelConfig -> Text+formatTryingModel mc =+  "Trying model: "+    <> mc.mcModel+    <> " via "+    <> mc.mcGateway.gwName++formatContinuingOn :: ModelConfig -> Text+formatContinuingOn mc =+  "Continuing run on "+    <> mc.mcModel+    <> " via "+    <> mc.mcGateway.gwName++formatModelFallback :: ModelConfig -> LLMError -> Text+formatModelFallback mc err =+  "Falling back from "+    <> mc.mcModel+    <> ": "+    <> T.pack (show err)++llmHooks :: Hooks -> LLMHooks+llmHooks hooks =+  LLMHooks+    { onLLMRequest = hooks.onRequest,+      onLLMResponse = hooks.onResponse,+      onLLMResponseError = hooks.onResponseError+    }++-- | The request with its hooks wrapped by 'safeHooks' and 'safeLLMHooks',+-- so an exception thrown by a hook cannot fail a call whose side effects+-- already happened. Applied by every public generation entry point.+safeGenRequest :: GenRequest -> GenRequest+safeGenRequest gr =+  gr+    { grHooks = safeHooks gr.grHooks,+      grLLMHooks = safeLLMHooks gr.grLLMHooks+    }

@@ -58,7 +58,7 @@     mcGateway :: LLMGateway,     -- | Provider-specific model identifier (e.g. @gpt-4.1-2025-04-14@).     mcModel :: Text,-    -- | Per-million-token pricing used to populate 'Usage.usageTotalCost'.+    -- | Per-million-token pricing used to populate 'LLM.Core.Usage.usageTotalCost'.     mcPricing :: PricingInfo,     -- | Maximum tokens requested from the provider.     mcMaxTokens :: Int,@@ -75,7 +75,7 @@     -- | Delay in milliseconds before each API call attempt, retries     -- included (rate limiting). Before a retry it is added to the backoff.     mcThrottleDelay :: Maybe Int,-    -- | Number of retries on retryable 'LLMError's before giving up on this model.+    -- | Number of retries on retryable t'LLM.Core.Types.LLMError's before giving up on this model.     mcRetryCount :: Int,     -- | Base backoff in milliseconds between retries (jittered). The     -- server's @Retry-After@ header is not honoured, so for a provider that@@ -85,8 +85,8 @@  -- | Primary model plus an ordered fallback chain. ----- On failure the library tries 'mwfModel' first, then each entry in--- 'mwfFallbacks' in order. Aborts ('GErrAborted') do not fall through.+-- On failure the library tries 'LLM.Generate.ModelConfig.mwfModel' first, then each entry in+-- 'LLM.Generate.ModelConfig.mwfFallbacks' in order. Aborts (v'LLM.Generate.Types.GErrAborted') do not fall through. data ModelWithFallbacks = ModelWithFallbacks   { mwfModel :: ModelConfig,     mwfFallbacks :: [ModelConfig]@@ -95,8 +95,8 @@ mfwToModelConfigs :: ModelWithFallbacks -> [ModelConfig] mfwToModelConfigs mwf = mwf.mwfModel : mwf.mwfFallbacks --- | 'mcRetryCount' retries with full-jitter exponential backoff from--- 'mcJitterBackoff', each delay capped at 30 s. @Retry-After@ is ignored.+-- | 'LLM.Generate.ModelConfig.mcRetryCount' retries with full-jitter exponential backoff from+-- 'LLM.Generate.ModelConfig.mcJitterBackoff', each delay capped at 30 s. @Retry-After@ is ignored. modelRetryPolicy :: ModelConfig -> RetryPolicyM IO modelRetryPolicy mc =   capDelay 30_000_000 $

@@ -25,11 +25,11 @@ import LLM.Core.Usage (Usage) import LLM.Generate.Logger (Hooks) --- | A single provider request. Built manually or via 'createGenRequest'.+-- | A single provider request. Built manually or via 'LLM.Agent.ToolUtils.createGenRequest'. ----- For structured output ('genObject', 'genObjectUntyped'), leave 'grTools' empty.+-- For structured output ('LLM.Generate.GenerateObject.genObject', 'LLM.Generate.GenerateObject.genObjectUntyped'), leave 'LLM.Generate.Types.grTools' empty. data GenRequest = GenRequest-  { -- | System prompt override (also available on 'Agent.agSystemPrompt').+  { -- | System prompt override (also available on 'LLM.Agent.Types.agSystemPrompt').     grSystemPrompt :: Maybe Text,     -- | Conversation turns sent to the provider (may already be windowed).     grMessages :: [Turn],@@ -51,7 +51,7 @@  -- | Successful result from the agent loop or a single-shot text generation. data GenerateTextResult = GenerateTextResult-  { -- | Matches 'RuntimeArgs.rtGenerationId' for this run.+  { -- | Matches 'LLM.Agent.Types.rtGenerationId' for this run.     gtrGenerationId :: UUID,     -- | New turns produced during this call (assistant and tool turns only).     gtrNewMessages :: [Turn],@@ -79,9 +79,9 @@ data GenerateError   = -- | Provider-level failure (HTTP, network, parse, etc.).     GErrLLM LLMError-  | -- | 'Agent.agMaxToolRounds' exhausted, and the model still called tools+  | -- | 'LLM.Agent.Types.agMaxToolRounds' exhausted, and the model still called tools     -- in the final call that followed. Those calls were not run;-    -- 'gerPartialNewMessages' ends with that assistant turn and a tool turn+    -- 'LLM.Generate.Types.gerPartialNewMessages' ends with that assistant turn and a tool turn     -- answering each call with a placeholder, so it can be sent again as is.     GErrToolExceeded   | -- | Every model in the fallback chain failed.

@@ -29,7 +29,7 @@ -- All paths are resolved inside a canonical sandbox ('LLM.Tools.FsConfig'). -- Mutating tools refuse paths that escape the workspace; reads are bounded -- (1 MiB for edits, 10 MiB for paginated reads). Include desired names in--- 'Agent.agTools' to expose them to the model.+-- 'LLM.Agent.Types.agTools' to expose them to the model. fsTools :: FilePath -> IO (ToolMap Text) fsTools filePath = do   cfg <- mkFsConfig filePath

@@ -209,7 +209,7 @@ -- yourself or use 'loadGatewaysWithDotenv'. -- -- For custom providers, place a @providers.json@ next to your model catalog;--- 'loadModelOrThrow' loads that file automatically.+-- 'LLM.Load.LoadModels.loadModelOrThrow' loads that file automatically. loadGateways :: IO GatewayMap loadGateways = loadGatewaysFromCatalog defaultProviderCatalogMap 

@@ -39,8 +39,9 @@ import Data.Text.Encoding (encodeUtf8) import Data.Text.Lazy qualified as TL import Data.Text.Lazy.Encoding (decodeUtf8)+import LLM.Core.Internal.Transport (postJson, postJsonStream) import LLM.Core.LLMProvider (LLMProvider (..), toGateway)-import LLM.Core.ProviderUtils (decodeObjectText, decodeToolArguments, handleStreamResponse, invalidToolArgumentsError, lenientConfig, readResponseBody, unexpectedResponse)+import LLM.Core.ProviderUtils (decodeObjectText, decodeToolArguments, invalidToolArgumentsError, unexpectedResponse) import LLM.Core.SSE (SSEEvent (sseData, sseEvent), malformedEvent, readSSEEvents, streamFailure) import LLM.Core.Types   ( commonImageMediaTypes,@@ -78,16 +79,11 @@     pattern UserTurn,   ) import LLM.Core.Usage (Usage (..), emptyUsage)-import Network.HTTP.Client qualified as HC import Network.HTTP.Req   ( Option,-    POST (POST),-    ReqBodyJson (ReqBodyJson),     Url,     header,     https,-    reqBr,-    runReq,     (/:),   ) @@ -132,12 +128,7 @@       providerImageUrlSupport = True,       buildBody = claudeBuildBody,       sendRequest = sendRequest,-      openStream = \body k ->-        runReq lenientConfig $ do-          let url = baseUrl /: "v1" /: "messages"-              opts = baseOpts <> claudeAuthOpts apiKey-          reqBr POST url (ReqBodyJson body) opts $ \resp ->-            handleStreamResponse resp (k . HC.brRead),+      openStream = postJsonStream url opts,       parseStream = \body readChunk callback ->         let modelHint = fromMaybe "" (parseMaybe (withObject "body" (.: "model")) body)          in parseClaudeStream modelHint readChunk callback,@@ -151,11 +142,9 @@       parseObjectResponse = parseClaudeObjectResponse     }   where-    sendRequest body =-      runReq lenientConfig $ do-        let url = baseUrl /: "v1" /: "messages"-            opts = baseOpts <> claudeAuthOpts apiKey-        reqBr POST url (ReqBodyJson body) opts readResponseBody+    sendRequest = postJson url opts+    url = baseUrl /: "v1" /: "messages"+    opts = baseOpts <> claudeAuthOpts apiKey  claudeAuthOpts :: Text -> Option scheme claudeAuthOpts apiKey =@@ -447,7 +436,7 @@  -- | The JSON text of the first text block. No text block is -- 'EmptyResponse'; invalid JSON text or a body without @content@ is a--- 'ParseError' (see 'decodeObjectText' and 'unexpectedResponse').+-- v'LLM.Core.Types.ParseError' (see 'decodeObjectText' and 'unexpectedResponse'). parseClaudeObjectResponse :: Value -> IO (LLMResult (Value, Maybe Usage)) parseClaudeObjectResponse v = pure $ case parseEither go v of   Left err -> Left (unexpectedResponse err v)

@@ -17,8 +17,8 @@ import Data.Text (Text) import Data.Text.Lazy qualified as TL import Data.Text.Lazy.Encoding qualified as TLE+import LLM.Core.Internal.Transport (postJson, postJsonStream) import LLM.Core.LLMProvider (LLMProvider (..), toGateway)-import LLM.Core.ProviderUtils (handleStreamResponse, lenientConfig, readResponseBody) import LLM.Core.Types   ( commonImageMediaTypes,     ChatRequest@@ -45,12 +45,8 @@   ) import Network.HTTP.Req   ( Option,-    POST (POST),-    ReqBodyJson (ReqBodyJson),     Url,     https,-    reqBr,-    runReq,     (/:),   ) @@ -75,12 +71,7 @@       providerImageUrlSupport = True,       buildBody = deepSeekBuildBody,       sendRequest = sendRequest,-      openStream = \body k ->-        runReq lenientConfig $ do-          let url = baseUrl /: "v1" /: "chat" /: "completions"-              opts = baseOpts <> authHeader apiKey-          reqBr POST url (ReqBodyJson body) opts $ \resp ->-            handleStreamResponse resp k,+      openStream = postJsonStream url opts,       parseStream = const parseOpenAIStream,       parseResponse = pure . parseOpenAIResponse,       buildObjectBody = \r schema ->@@ -97,11 +88,9 @@       parseObjectResponse = pure . parseOpenAIObjectResponse     }   where-    sendRequest body =-      runReq lenientConfig $ do-        let url = baseUrl /: "v1" /: "chat" /: "completions"-            opts = baseOpts <> authHeader apiKey-        reqBr POST url (ReqBodyJson body) opts readResponseBody+    sendRequest = postJson url opts+    url = baseUrl /: "v1" /: "chat" /: "completions"+    opts = baseOpts <> authHeader apiKey  -- | OpenAI-compatible body; DeepSeek uses @max_tokens@ (not @max_completion_tokens@). deepSeekBuildBody :: Bool -> ChatRequest -> Value

@@ -35,8 +35,9 @@ import Data.Text.Encoding (encodeUtf8) import Data.UUID.Types (UUID) import Data.UUID.Types qualified as UUID+import LLM.Core.Internal.Transport (postJson, postJsonStream) import LLM.Core.LLMProvider (LLMProvider (..), toGateway, withoutRoutingFields)-import LLM.Core.ProviderUtils (decodeObjectText, handleStreamResponse, lenientConfig, readResponseBody, stripBoundsAndComments)+import LLM.Core.ProviderUtils (decodeObjectText, stripBoundsAndComments) import LLM.Core.SSE (SSEEvent (sseData), malformedEvent, readSSEEvents, streamFailure) import LLM.Core.Types   ( supportedImageMediaTypes,@@ -75,13 +76,9 @@ import Network.HTTP.Client qualified as HC import Network.HTTP.Req   ( Option,-    POST (POST),-    ReqBodyJson (ReqBodyJson),     Url,     header,     https,-    reqBr,-    runReq,     (/:),     (=:),   )@@ -113,17 +110,11 @@       providerImageUrlSupport = True,       buildBody = const geminiBuildBody,       sendRequest = sendRequest,-      openStream = \body k ->-        runReq lenientConfig $ do-          let model = extractModel body-              url =-                baseUrl-                  /: "v1beta"-                  /: "models"-                  /: (model <> ":streamGenerateContent")-              opts = baseOpts <> geminiAuthOpts apiKey <> "alt" =: ("sse" :: Text)-          reqBr POST url (ReqBodyJson (withoutRoutingFields body)) opts $ \resp ->-            handleStreamResponse resp k,+      openStream = \body ->+        postJsonStream+          (modelUrl body ":streamGenerateContent")+          (opts <> "alt" =: ("sse" :: Text))+          (withoutRoutingFields body),       parseStream = const parseGeminiStream,       parseResponse = parseGeminiResponse,       buildObjectBody = \r schema ->@@ -153,15 +144,10 @@     }   where     sendRequest body =-      runReq lenientConfig $ do-        let model = extractModel body-            url =-              baseUrl-                /: "v1beta"-                /: "models"-                /: (model <> ":generateContent")-            opts = baseOpts <> geminiAuthOpts apiKey-        reqBr POST url (ReqBodyJson (withoutRoutingFields body)) opts readResponseBody+      postJson (modelUrl body ":generateContent") opts (withoutRoutingFields body)+    -- The model is in the path, taken from the body's routing field.+    modelUrl body method = baseUrl /: "v1beta" /: "models" /: (extractModel body <> method)+    opts = baseOpts <> geminiAuthOpts apiKey  geminiAuthOpts :: Text -> Option scheme geminiAuthOpts apiKey = header "x-goog-api-key" (encodeUtf8 apiKey)@@ -562,7 +548,7 @@             }     } --- | Pull a thought signature out of 'tcProviderMeta' iff it was emitted by+-- | Pull a thought signature out of 'LLM.Core.Types.tcProviderMeta' iff it was emitted by -- the model we're currently calling. signatureForModel :: Text -> Maybe ProviderOpaque -> Maybe Text signatureForModel currentModel (Just o)

@@ -12,8 +12,9 @@   ) import Data.Aeson.Types (Pair) import Data.Text (Text)+import LLM.Core.Internal.Transport (postJson, postJsonStream) import LLM.Core.LLMProvider (LLMProvider (..), toGateway)-import LLM.Core.ProviderUtils (handleStreamResponse, lenientConfig, readResponseBody, normalizeSchemaOpenAI)+import LLM.Core.ProviderUtils (normalizeSchemaOpenAI) import LLM.Core.Types   ( commonImageMediaTypes,     ChatRequest@@ -28,13 +29,9 @@ import LLM.Providers.OpenAI (buildMessages, encodeToolDef, parseOpenAIObjectResponse, parseOpenAIResponse, parseOpenAIStream) import Network.HTTP.Req   ( Option,-    POST (POST),-    ReqBodyJson (ReqBodyJson),     Url,     http,     port,-    reqBr,-    runReq,     (/:),   ) @@ -60,11 +57,7 @@       providerImageUrlSupport = False,       buildBody = \stream r -> object (ollamaBuildBodyPairs stream r),       sendRequest = sendRequest,-      openStream = \body k ->-        runReq lenientConfig $ do-          let url = baseUrl /: "v1" /: "chat" /: "completions"-          reqBr POST url (ReqBodyJson body) baseOpts $ \resp ->-            handleStreamResponse resp k,+      openStream = postJsonStream url baseOpts,       parseStream = const parseOpenAIStream,       parseResponse = pure . parseOpenAIResponse,       buildObjectBody = \r schema ->@@ -85,10 +78,8 @@       parseObjectResponse = pure . parseOpenAIObjectResponse     }   where-    sendRequest body =-      runReq lenientConfig $ do-        let url = baseUrl /: "v1" /: "chat" /: "completions"-        reqBr POST url (ReqBodyJson body) baseOpts readResponseBody+    sendRequest = postJson url baseOpts+    url = baseUrl /: "v1" /: "chat" /: "completions"  -- | Request body for Ollama's OpenAI-compatible endpoint. Like OpenAI's, but -- the token limit is @max_tokens@: Ollama ignores @max_completion_tokens@

@@ -44,8 +44,9 @@ import Data.Text (Text) import Data.Text qualified as T import Data.Text.Encoding (decodeUtf8, encodeUtf8)+import LLM.Core.Internal.Transport (postJson, postJsonStream) import LLM.Core.LLMProvider (LLMProvider (..), toGateway)-import LLM.Core.ProviderUtils (decodeObjectText, decodeToolArguments, handleStreamResponse, invalidToolArgumentsError, lenientConfig, normalizeSchemaOpenAI, readResponseBody, unexpectedResponse)+import LLM.Core.ProviderUtils (decodeObjectText, decodeToolArguments, invalidToolArgumentsError, normalizeSchemaOpenAI, unexpectedResponse) import LLM.Core.SSE (SSEEvent (sseData), malformedEvent, readSSEEvents, streamFailure) import LLM.Core.Types   ( commonImageMediaTypes,@@ -86,13 +87,9 @@ import Network.HTTP.Client qualified as HC import Network.HTTP.Req   ( Option,-    POST (POST),-    ReqBodyJson (ReqBodyJson),     Url,     header,     https,-    reqBr,-    runReq,     (/:),   ) @@ -126,12 +123,7 @@       providerImageUrlSupport = True,       buildBody = openAIBuildBody,       sendRequest = sendRequest,-      openStream = \body k ->-        runReq lenientConfig $ do-          let url = baseUrl /: "v1" /: "chat" /: "completions"-              opts = baseOpts <> authHeader apiKey-          reqBr POST url (ReqBodyJson body) opts $ \resp ->-            handleStreamResponse resp k,+      openStream = postJsonStream url opts,       parseStream = const parseOpenAIStream,       parseResponse = pure . parseOpenAIResponse,       buildObjectBody = \r schema ->@@ -152,11 +144,9 @@       parseObjectResponse = pure . parseOpenAIObjectResponse     }   where-    sendRequest body =-      runReq lenientConfig $ do-        let url = baseUrl /: "v1" /: "chat" /: "completions"-            opts = baseOpts <> authHeader apiKey-        reqBr POST url (ReqBodyJson body) opts readResponseBody+    sendRequest = postJson url opts+    url = baseUrl /: "v1" /: "chat" /: "completions"+    opts = baseOpts <> authHeader apiKey  authHeader :: Text -> Option scheme authHeader apiKey@@ -352,7 +342,7 @@ -- -- No choice, or a missing or null content, is 'EmptyResponse'; so is -- content that is empty once code fences are stripped. Content that is not--- valid JSON, and a body without the expected fields, are 'ParseError's+-- valid JSON, and a body without the expected fields, are v'LLM.Core.Types.ParseError's -- that quote the decoder's message and the start of the text or body. parseOpenAIObjectResponse :: Value -> LLMResult (Value, Maybe Usage) parseOpenAIObjectResponse v = case parseEither content v of

@@ -229,8 +229,8 @@ matchGlob (p : ps) (s : ss) = matchSegment (T.unpack p) (T.unpack s) && matchGlob ps ss  -- | Match a single glob segment (no '/' allowed in either side).--- Supports '*' (any run of chars, possibly empty) and '?' (one char).--- On a mismatch only the most recent '*' takes one more char, which is+-- Supports @*@ (any run of chars, possibly empty) and @?@ (one char).+-- On a mismatch only the most recent @*@ takes one more char, which is -- enough for this glob syntax and keeps matching O(pattern * name) instead -- of exponential in the number of stars. matchSegment :: String -> String -> Bool

@@ -28,7 +28,7 @@ import System.FilePath (addTrailingPathSeparator, dropTrailingPathSeparator, equalFilePath, isAbsolute, isRelative, joinPath, normalise, splitDirectories, splitFileName, takeDirectory, (</>))  -- | Configuration for file-system tools.--- 'fsBasePath' must be a canonical absolute path (use 'mkFsConfig').+-- 'LLM.Tools.FsConfig.fsBasePath' must be a canonical absolute path (use 'mkFsConfig'). -- -- The @FsConfig@ data constructor is deprecated: build the configuration -- with 'mkFsConfig', which canonicalizes the path. From 0.5 the constructor@@ -99,7 +99,7 @@ -- dangling link is resolved to that target and judged by where it -- points rather than appended as if it were an ordinary new file. -- Throws 'SandboxViolation' on escape attempts, and an 'IOException' for a--- @..@ that would step back out of a symbolic link (see 'collapseDots').+-- @..@ that would step back out of a symbolic link (see @collapseDots@). sandboxPath :: FsConfig -> FilePath -> IO FilePath sandboxPath cfg relPath = do   let base = cfg.fsBasePath

@@ -23,6 +23,7 @@     maxTreeEntries,     maxVisitedEntries,     maxOutputBytes,+    maxSearchBytes,     binarySniffBytes,     detectBinary,     readBoundedTextFile,@@ -40,10 +41,10 @@ import Data.Text.Encoding (decodeUtf8With, encodeUtf8) import Data.Text.Encoding.Error (lenientDecode) import LLM.Tools.Internal.TextFile-  ( binarySniffBytes,-    readBoundedTextWith,+  ( readBoundedTextWith,     readBoundedUtf8With,   )+import LLM.Tools.Internal.TextFile qualified as TextFile import System.Directory (copyPermissions, doesFileExist, removeFile, renameFile) import System.FilePath (splitFileName) import System.IO@@ -94,6 +95,21 @@ -- is cut and the cut is reported. maxOutputBytes :: Int maxOutputBytes = 100 * 1024 -- 100 KiB++-- | Budget of file bytes one @grep@ call may read. Only the text files it+-- searches count; binary files and files over the per-file size cap are+-- skipped without being read. A search that runs out stops before the+-- next file and says so in its output.+--+-- @since 0.4.3.0+maxSearchBytes :: Int+maxSearchBytes = 64 * 1024 * 1024 -- 64 MiB++-- | Number of leading bytes inspected when sniffing for binary content.+-- Matches the heuristic used by @git diff@ and @grep@: any NUL byte in+-- the prefix means binary.+binarySniffBytes :: Int+binarySniffBytes = TextFile.binarySniffBytes  -- | A NUL byte in the first 'binarySniffBytes' is a robust, cheap binary -- heuristic. On I/O failure the file is treated as binary (the caller

@@ -7,16 +7,20 @@ import Autodocodec qualified as AC import Control.Exception (IOException, try) import Control.Monad (unless)-import Data.IORef (IORef, modifyIORef', newIORef, readIORef)+import Data.IORef (IORef, atomicModifyIORef', modifyIORef', newIORef, readIORef) import Data.Aeson (FromJSON) import Data.List (sort) import Data.Text (Text)+import Data.ByteString qualified as BS+import Data.Maybe (catMaybes) import Data.Text qualified as T+import Data.Text.Encoding (decodeUtf8With)+import Data.Text.Encoding.Error (lenientDecode) import GHC.Generics (Generic) import LLM.Core.Types (TypedTool (..)) import LLM.Tools.FsConfig (FsConfig, isFileHidden, isSymlink, sandboxPath, withWorkspacePaths)-import LLM.Tools.FsLimits (expensiveWalkDepth, maxOutputBytes, maxVisitedEntries)-import LLM.Tools.Internal.TextFile (readLenient, withTextHandle)+import LLM.Tools.FsLimits (expensiveWalkDepth, maxOutputBytes, maxSearchBytes, maxVisitedEntries)+import LLM.Tools.Internal.TextFile (withTextHandle) import LLM.Tools.Internal.WalkBudget (WalkBudget, budgetExhausted, budgetNote, capLines, newWalkBudget, takeEntries) import System.Directory   ( doesDirectoryExist,@@ -27,6 +31,7 @@ import System.FilePath (takeExtension, takeFileName, (</>)) import System.IO   ( IOMode (ReadMode),+    hFileSize,     withBinaryFile,   ) @@ -104,7 +109,9 @@           <> "the header reports whether results were truncated and how many files were scanned. "           <> "A search looks at most "           <> T.pack (show maxVisitedEntries)-          <> " entries and stops at maxResults; then the file count is shown as 'N+'.",+          <> " entries, reads at most "+          <> T.pack (show (maxSearchBytes `div` (1024 * 1024)))+          <> " MiB of files, and stops at maxResults; then the file count is shown as 'N+'.",       ttoolReadonly = True,       ttoolExecute = const (withWorkspacePaths cfg . execute cfg)     }@@ -139,34 +146,49 @@                   <> "-byte cap for grep."             else do               budget <- newWalkBudget maxVisitedEntries-              st <- newIORef (GrepState [] 0 0 False)+              st <- newIORef (GrepState [] 0 0 False 0 False)               let search = searchFile st pat ci cap               if isFile                 then search resolved (takeFileName resolved)                 else walkDir budget st search exts resolved "" 0-              GrepState matches _ scanned capHit <- readIORef st+              GrepState matches _ scanned capHit bytesRead bytesOut <- readIORef st               stopped <- budgetExhausted budget+              let notes =+                    catMaybes+                      [ if stopped then Just (budgetNote budget) else Nothing,+                        if bytesOut then Just (bytesNote bytesRead) else Nothing+                      ]               pure $                 renderResults                   args._grepPath                   pat                   (reverse matches)                   scanned-                  ((capHit && not isFile) || stopped)+                  ((capHit && not isFile) || stopped || bytesOut)                   capHit-                  (if stopped then Just (budgetNote budget) else Nothing)+                  (if null notes then Nothing else Just (T.intercalate " | " notes))  -- | Lowercase and strip leading dots so callers can pass "hs" or ".hs". normalizeExtensions :: [Text] -> [Text] normalizeExtensions = map (T.toLower . T.dropWhile (== '.')) --- | Matches so far (newest first), their count, files searched, and--- whether the walk stopped because 'maxResults' was reached.-data GrepState = GrepState [Match] Int Int Bool+-- | Matches so far (newest first), their count, files searched, whether+-- the walk stopped because @maxResults@ was reached, the file bytes read,+-- and whether the walk stopped because 'maxSearchBytes' ran out.+data GrepState = GrepState [Match] Int Int Bool Int Bool +-- | True once the search should look at no more files.+searchDone :: GrepState -> Bool+searchDone (GrepState _ _ _ capHit _ bytesOut) = capHit || bytesOut++-- | The note added to the output when 'maxSearchBytes' ran out.+bytesNote :: Int -> Text+bytesNote n = "stopped after reading " <> T.pack (show n) <> " bytes; narrow the path or pattern"+ -- | Walk a directory and search each candidate file as it is found, -- respecting depth, hidden-file, extension and size limits and the entry--- budget. Stops once 'maxResults' is reached. Order is stable (entries+-- budget. Stops once @maxResults@ is reached or 'maxSearchBytes' runs+-- out. Order is stable (entries -- sorted) so output is deterministic. Each file is shown by its path -- relative to the search root, built from the names walked, so no host -- path is shown.@@ -188,8 +210,8 @@           allowed <- takeEntries budget (filter (not . isFileHidden) entries)           mapM_ (visit dir display depth) allowed     visit parent display depth name = do-      GrepState _ _ _ capHit <- readIORef st-      unless capHit $ do+      done <- searchDone <$> readIORef st+      unless done $ do         let full = parent </> name             fullDisplay = display </> name         -- Skip symlinks to prevent traversal out of the sandbox.@@ -230,22 +252,40 @@     Left _ -> pure []  -- | Search one file and add its matches, up to the cap. Binary and--- unreadable files count as searched but give no matches.+-- unreadable files count as searched but give no matches. A text file is+-- charged to 'maxSearchBytes' before it is read; a file that does not fit+-- is not read or counted, and ends the search. searchFile :: IORef GrepState -> Text -> Bool -> Int -> FilePath -> FilePath -> IO () searchFile st pat ci cap f display = do   r <--    try (withTextHandle (withBinaryFile f ReadMode) readLenient) ::-      IO (Either IOException (Maybe Text))-  let ms = case r of-        Right (Just content) -> matchLines display content needle ci-        _ -> []-  modifyIORef' st $ \(GrepState acc n scanned _) ->-    let remaining = cap - n-        taken = take remaining ms-        n' = n + length taken-     in GrepState (reverse taken ++ acc) n' (scanned + 1) (n' >= cap)+    try (withTextHandle (withBinaryFile f ReadMode) readCharged) ::+      IO (Either IOException (Maybe (Maybe Text)))+  case r of+    Right (Just Nothing) -> pure ()+    _ -> do+      let ms = case r of+            Right (Just (Just content)) -> matchLines display content needle ci+            _ -> []+      modifyIORef' st $ \(GrepState acc n scanned _ used out) ->+        let remaining = cap - n+            taken = take remaining ms+            n' = n + length taken+         in GrepState (reverse taken ++ acc) n' (scanned + 1) (n' >= cap) used out   where     needle = if ci then T.toLower pat else pat+    -- Charge the file's size (at most the per-file cap) and read that+    -- many bytes, or refuse it when the budget cannot cover it.+    readCharged h = do+      sz <- min maxFileSizeBytes <$> hFileSize h+      let n = fromIntegral sz+      fits <- charge n+      if fits+        then Just . decodeUtf8With lenientDecode <$> BS.hGet h n+        else pure Nothing+    charge n = atomicModifyIORef' st $ \(GrepState acc k scanned capHit used out) ->+      if used + n <= maxSearchBytes+        then (GrepState acc k scanned capHit (used + n) out, True)+        else (GrepState acc k scanned capHit used True, False)  data Match = Match   { mPath :: FilePath,@@ -267,7 +307,7 @@   | otherwise = T.take maxLineLength t <> "…"  -- | Render the results. @partial@: the walk stopped early, so the file--- count is a lower bound. @capHit@: 'maxResults' was reached.+-- count is a lower bound. @capHit@: @maxResults@ was reached. renderResults :: Text -> Text -> [Match] -> Int -> Bool -> Bool -> Maybe Text -> Text renderResults rootDisplay pat allMatches scanned partial capHit note =   let (body, cut) = capLines maxOutputBytes (map fmtMatch allMatches)

@@ -1,124 +0,0 @@--- | Text reads on one open 'Handle': the binary sniff, the size check and--- the read itself all use the same open file, instead of reopening the--- path for each step. Shared by 'LLM.Tools.FsLimits' and the tools.-module LLM.Tools.Internal.TextFile-  ( Opener,-    binarySniffBytes,-    withTextHandle,-    readBoundedBytesWith,-    readBoundedTextWith,-    readBoundedUtf8With,-    readLenient,-    writeUtf8,-  )-where--import Control.Exception (IOException, throwIO, try)-import GHC.IO.Exception (IOException (ioe_filename))-import Data.ByteString qualified as BS-import Data.Maybe (fromMaybe)-import Data.Text (Text)-import Data.Text qualified as T-import Data.Text.Encoding (decodeUtf8', decodeUtf8With, encodeUtf8)-import Data.Text.Encoding.Error (lenientDecode)-import System.IO (Handle, SeekMode (AbsoluteSeek), hFileSize, hSeek)---- | Opens a file for reading and runs an action on it, e.g.--- @withBinaryFile path ReadMode@.-type Opener = forall a. (Handle -> IO a) -> IO a---- | Number of leading bytes inspected when sniffing for binary content.--- Matches the heuristic used by @git diff@ and @grep@: any NUL byte in--- the prefix means binary.-binarySniffBytes :: Int-binarySniffBytes = 8192---- | Open a file, sniff its first 'binarySniffBytes' for a NUL byte, and run--- the action from the start of the file if there is none.------ 'Nothing' if the file looks binary, or cannot be opened or sniffed: the--- caller fails closed rather than read an unreadable file as text. An--- 'IOException' from the action itself is rethrown.-withTextHandle :: Opener -> (Handle -> IO r) -> IO (Maybe r)-withTextHandle open act = do-  r <- try $ open $ \h -> do-    sniff <- try (BS.hGet h binarySniffBytes)-    case sniff of-      Left (_ :: IOException) -> pure (Right Nothing)-      Right bs-        | BS.elem 0 bs -> pure (Right Nothing)-        | otherwise -> do-            hSeek h AbsoluteSeek 0-            fmap Just <$> try (act h)-  case r of-    Left (_ :: IOException) -> pure Nothing-    Right (Left (e :: IOException)) -> throwIO e-    Right (Right x) -> pure x---- | Read a file up to @cap@ bytes. Refuses binary files, checks the size--- before reading, and reads at most @cap + 1@ bytes so a file that grows--- during the read is caught.-readBoundedBytesWith :: Opener -> Text -> Int -> IO (Either Text BS.ByteString)-readBoundedBytesWith open display cap = do-  r <- withTextHandle open $ \h -> do-    sizeRes <- try (hFileSize h)-    case sizeRes of-      Left (e :: IOException) ->-        pure $ Left $ "could not stat " <> display <> ": " <> describeIOError e-      Right sz-        | sz > fromIntegral cap ->-            pure $-              Left $-                display-                  <> " is "-                  <> T.pack (show sz)-                  <> " bytes, which exceeds the "-                  <> T.pack (show cap)-                  <> "-byte cap."-        | otherwise -> do-            bsRes <- try (BS.hGet h (cap + 1))-            pure $ case bsRes of-              Left (e :: IOException) ->-                Left $ "could not read " <> display <> ": " <> describeIOError e-              Right bs-                | BS.length bs > cap ->-                    Left $-                      display-                        <> " grew past the "-                        <> T.pack (show cap)-                        <> "-byte cap during the read."-                | otherwise -> Right bs-  pure $-    fromMaybe-      (Left (display <> " appears to be a binary file; only text files are supported."))-      r---- | An 'IOException' without its file name, which for a handle is the--- absolute path it was opened with. Callers name the file themselves.-describeIOError :: IOException -> Text-describeIOError e = T.pack (show e {ioe_filename = Nothing})---- | 'readBoundedBytesWith', decoded for display. Invalid UTF-8 bytes become--- U+FFFD, so never write the result back.-readBoundedTextWith :: Opener -> Text -> Int -> IO (Either Text Text)-readBoundedTextWith open display cap =-  fmap (decodeUtf8With lenientDecode) <$> readBoundedBytesWith open display cap---- | 'readBoundedBytesWith', for editing. Refuses files that are not valid--- UTF-8, since writing them back would destroy their bytes.-readBoundedUtf8With :: Opener -> Text -> Int -> IO (Either Text Text)-readBoundedUtf8With open display cap = do-  r <- readBoundedBytesWith open display cap-  pure $-    r >>= \bs -> case decodeUtf8' bs of-      Right t -> Right t-      Left _ -> Left $ display <> " is not valid UTF-8; refusing to edit it."---- | Read the rest of the handle as UTF-8, replacing invalid bytes with--- U+FFFD. Independent of the locale.-readLenient :: Handle -> IO Text-readLenient h = decodeUtf8With lenientDecode <$> BS.hGetContents h---- | Write text as UTF-8, independent of the locale.-writeUtf8 :: Handle -> Text -> IO ()-writeUtf8 h = BS.hPut h . encodeUtf8

@@ -9,6 +9,8 @@     budgetNote,     capLines,     utf8Length,+    utf8Prefix,+    textPrefixBytes,   ) where @@ -16,7 +18,7 @@ import Data.IORef (IORef, atomicModifyIORef', newIORef, readIORef) import Data.Text (Text) import Data.Text qualified as T-import Data.Text.Encoding (encodeUtf8)+import Data.Text.Encoding (decodeUtf8, encodeUtf8)  -- | How many more directory entries a walk may look at, and whether it was -- denied any.@@ -60,3 +62,20 @@ -- | Size of a text in UTF-8 bytes. utf8Length :: Text -> Int utf8Length = BS.length . encodeUtf8++-- | At most @n@ bytes from the start, without splitting a UTF-8 sequence.+utf8Prefix :: Int -> BS.ByteString -> BS.ByteString+utf8Prefix n bs+  | BS.length bs <= n = bs+  | otherwise = BS.take (boundary n) bs+  where+    -- Back up while the byte at the cut is a continuation byte (10xxxxxx),+    -- so the cut falls before the lead byte of its sequence.+    boundary i+      | i > 0 && BS.index bs i >= 0x80 && BS.index bs i < 0xC0 = boundary (i - 1)+      | otherwise = i++-- | At most @n@ UTF-8 bytes from the start of a text, ending on a whole+-- character.+textPrefixBytes :: Int -> Text -> Text+textPrefixBytes n = decodeUtf8 . utf8Prefix n . encodeUtf8

@@ -1,4 +1,4 @@-module LLM.Tools.MultiReplaceInFile (multiReplaceInFileToolTyped, MultiReplaceInFileToolArgs (..)) where+module LLM.Tools.MultiReplaceInFile (multiReplaceInFileToolTyped, MultiReplaceInFileToolArgs (..), Replacement) where  import Autodocodec qualified as AC import Data.Aeson (FromJSON)@@ -11,6 +11,9 @@ import LLM.Tools.Internal.PathLock (withPathLock) import LLM.Tools.Internal.TextEdit (countOccurrences, replaceFirst) +-- | One edit of @multi_replace_in_file@: the @old@ text, which must occur+-- exactly once, and the @new@ text that replaces it. Decoded from the+-- tool's JSON arguments. data Replacement = Replacement   { _repOld :: Text,     _repNew :: Text

@@ -15,14 +15,14 @@ import LLM.Core.Types (TypedTool (..)) import LLM.Tools.FsConfig (FsConfig, sandboxPath, withWorkspacePaths) import LLM.Tools.FsLimits (maxOutputBytes, maxPaginatedFileBytes)-import LLM.Tools.Internal.TextFile (withTextHandle)+import LLM.Tools.Internal.TextFile (CappedRead (..), readCapped, withTextHandle)+import LLM.Tools.Internal.WalkBudget (utf8Prefix) import System.IO   ( IOMode (ReadMode),-    hFileSize,     withBinaryFile,   ) --- | Default number of lines returned per page when the caller omits 'limit'.+-- | Default number of lines returned per page when the caller omits @limit@. defaultLimit :: Int defaultLimit = 200 @@ -89,31 +89,25 @@           capText = T.pack (show maxPaginatedFileBytes)       resolved <- sandboxPath cfg rel       result <- withTextHandle (withBinaryFile resolved ReadMode) $ \h -> do-        size <- hFileSize h-        if size > maxPaginatedFileBytes-          then-            pure . Left $+        capped <- readCapped maxPaginatedFileBytes h+        pure $ case capped of+          CappedTooLarge size ->+            Left $               path                 <> " is "                 <> T.pack (show size)                 <> " bytes, which exceeds the "                 <> capText                 <> "-byte cap for read_file_paginated."-          else do-            -- Read one byte past the cap, so a file that grew past it since-            -- the size check is reported instead of looking like it ended.-            bytes <- BS.hGet h (fromIntegral maxPaginatedFileBytes + 1)-            pure $-              if fromIntegral (BS.length bytes) > maxPaginatedFileBytes-                then-                  Left $-                    path-                      <> " grew past the "-                      <> capText-                      <> "-byte cap for read_file_paginated during the read."-                else-                  let (lns, hasMore) = takePage offset limit bytes-                   in Right (size, lns, hasMore)+          CappedGrew ->+            Left $+              path+                <> " grew past the "+                <> capText+                <> "-byte cap for read_file_paginated during the read."+          CappedBytes size bytes ->+            let (lns, hasMore) = takePage offset limit bytes+             in Right (size, lns, hasMore)       pure $ case result of         Nothing ->           "Error: "@@ -168,18 +162,6 @@      decode = decodeUtf8With lenientDecode     newline = 10---- | At most @n@ bytes from the start, without splitting a UTF-8 sequence.-utf8Prefix :: Int -> BS.ByteString -> BS.ByteString-utf8Prefix n bs-  | BS.length bs <= n = bs-  | otherwise = BS.take (boundary n) bs-  where-    -- Back up while the byte at the cut is a continuation byte (10xxxxxx),-    -- so the cut falls before the lead byte of its sequence.-    boundary i-      | i > 0 && BS.index bs i >= 0x80 && BS.index bs i < 0xC0 = boundary (i - 1)-      | otherwise = i  renderPage :: Text -> Integer -> Int -> [Text] -> Bool -> Text renderPage path size offset lns hasMore =

@@ -62,7 +62,7 @@   pure $ T.intercalate "\n" (shown ++ note)  -- | Annotate a directory entry without following symlinks.--- Symbolic links are tagged with '@' and never treated as directories,+-- Symbolic links are tagged with @\@@ and never treated as directories, -- even if they happen to point at one (possibly outside the sandbox). annotateEntry :: FilePath -> FilePath -> IO Text annotateEntry parent name = do

@@ -791,6 +791,24 @@       filter ("no_such_tool" `T.isInfixOf`) ws         `shouldBe` ["[test] agTools lists no_such_tool, which is not in the tool map; it is not sent to the model"] +    it "logs an invalid caller turn once per run, not once per round" $ do+      reqs <- newIORef []+      (rt, logs, _) <- recordingRuntime+      let -- Two tool rounds, then text: three model calls.+          twoRounds req =+            let toolTurns = length [() | ToolTurn _ <- req.reqConversation]+                tc = mkToolCall ("call_" <> T.pack (show toolTurns)) "get_weather" (object ["location" .= ("London" :: Text)])+             in pure . Right $+                  if toolTurns >= 2+                    then ChatResponse "done" [textPart "done"] Nothing Nothing+                    else ChatResponse "" [toolCallPart tc] Nothing Nothing+          history = [UserMessage [textPart "go", toolCallPart (mkToolCall "c1" "get_weather" (object []))]]+      result <- generateText defaultAgent {agTools = ["get_weather"]} (ModelWithFallbacks (mockModel (recordingGateway reqs twoRounds)) []) toolMap rt history+      fmap (.gtrText) result `shouldBe` Right "done"+      length <$> readIORef reqs `shouldReturn` 3+      ws <- warnings <$> readIORef logs+      filter ("turn " `T.isInfixOf`) ws `shouldBe` ["[test] turn 0: user messages may not contain tool-call parts"]+     it "uses the built-in get_history over a caller tool of that name when windowing is on" $ do       reqs <- newIORef []       (rt, logs, _) <- recordingRuntime@@ -880,6 +898,22 @@       readIORef calls `shouldReturn` ["a", "b"]       ws <- (\ls -> [m | (Warn, m) <- ls, "turn " `T.isPrefixOf` m]) <$> readIORef logs       ws `shouldBe` ["turn 2: user messages may not contain tool-call parts"]++    it "logs problems in the tool-call structure in turn order" $ do+      logs <- newIORef []+      calls <- newIORef []+      let history =+            invalidHistory+              ++ [ ToolTurn [ToolResult "c9" "get_weather" "sunny"],+                   AssistantMessage [textPart "done"]+                 ]+      _ <- generateTextWithFallbacks (request logs history) (ModelWithFallbacks (mockModel (loggingGateway calls "m" answer)) [])+      readIORef calls `shouldReturn` ["m"]+      ws <- (\ls -> [m | (Warn, m) <- ls]) <$> readIORef logs+      ws+        `shouldBe` [ "turn 2: user messages may not contain tool-call parts",+                     "turn 3: tool results do not follow an assistant turn with tool calls"+                   ]      it "logs nothing for a valid history" $ do       logs <- newIORef []

@@ -4,7 +4,7 @@ import Control.Concurrent.MVar (newEmptyMVar, putMVar, takeMVar) import Control.Exception (IOException, bracket_, evaluate, try) import Control.Monad (forM, forM_)-import Data.Aeson (Value, object, (.=))+import Data.Aeson (Result (..), Value, fromJSON, object, (.=)) import Data.ByteString qualified as BS import Data.Map qualified as Map import Data.Text (Text)@@ -19,7 +19,8 @@ import LLM.Generate.Logger (noHooks) import LLM.Load.FsTools (fsTools) import LLM.Tools.FsConfig (SandboxViolation (..))-import LLM.Tools.FsLimits (maxOutputBytes, maxTreeEntries, maxVisitedEntries)+import LLM.Tools.FsLimits (maxOutputBytes, maxSearchBytes, maxTreeEntries, maxVisitedEntries)+import LLM.Tools.MultiReplaceInFile (MultiReplaceInFileToolArgs (..), Replacement) import System.Directory   ( createDirectory,     createDirectoryLink,@@ -218,6 +219,16 @@       readBack <- runTool tm "read_file_paginated" (object ["path" .= ("multi.txt" :: Text)])       T.unpack readBack `shouldContain` "AAA BBB" +    it "exports the type of the replacements field" $ do+      let args =+            object+              [ "path" .= ("multi.txt" :: Text),+                "replacements" .= [object ["old" .= ("a" :: Text), "new" .= ("b" :: Text)]]+              ]+      case fromJSON args :: Result MultiReplaceInFileToolArgs of+        Success decoded -> length (decoded._mrifReplacements :: [Replacement]) `shouldBe` 1+        Error err -> expectationFailure err+   describe "create_directory" $ do     it "creates nested directories" $ withSandbox $ \sb -> do       tm <- fsTools sb@@ -463,6 +474,25 @@       result <- runTool tm "grep" (object ["pattern" .= ("needle" :: Text), "maxResults" .= (2 :: Int)])       T.unpack result `shouldContain` "2 match(es) across 2+ file(s)"       T.unpack result `shouldContain` "stopped at maxResults"++    it "grep stops before the file that would exceed maxSearchBytes" $ withSandbox $ \sb -> do+      tm <- fsTools sb+      let mib = 1024 * 1024+          perFile = mib -- the per-file cap, so each file is searched+          filesInBudget = maxSearchBytes `div` perFile+          name i = sb </> ("t" <> replicate (3 - length (show i)) '0' <> show i <> ".txt")+          withNeedle = BS.replicate (perFile - 7) 0x61 <> "needle\n"+      -- A binary file is not read, so it does not count against the budget.+      BS.writeFile (sb </> "a-binary.bin") (BS.cons 0 (BS.replicate (perFile - 1) 0x61))+      mapM_ (\i -> BS.writeFile (name i) (BS.replicate perFile 0x61)) [1 .. filesInBudget - 1]+      -- The last file that fits, and one past the budget.+      BS.writeFile (name filesInBudget) withNeedle+      BS.writeFile (name (filesInBudget + 1)) withNeedle+      result <- runTool tm "grep" (object ["pattern" .= ("needle" :: Text)])+      T.unpack result `shouldContain` ("1 match(es) across " <> show (filesInBudget + 1) <> "+ file(s)")+      T.unpack result `shouldContain` ("stopped after reading " <> show maxSearchBytes <> " bytes")+      T.unpack result `shouldContain` (drop (length sb + 1) (name filesInBudget) <> ":1:")+      T.unpack result `shouldNotContain` drop (length sb + 1) (name (filesInBudget + 1))      it "grep reports an exact file count when it searched everything" $ withSandbox $ \sb -> do       tm <- fsTools sb

@@ -102,7 +102,8 @@       let schema =             object               [ "type" .= ("object" :: Text),-                "properties" .= object ["type" .= object ["type" .= ("string" :: Text)]]+                "properties" .= object ["type" .= object ["type" .= ("string" :: Text)]],+                "required" .= (["type"] :: [Text])               ]       normalizeSchemaOpenAI schema         `shouldBe` object

@@ -1,8 +1,9 @@+{-# LANGUAGE TypeApplications #-} {-# OPTIONS_GHC -Wno-missing-fields #-}  module LLM.GenerateObjectSpec (spec) where -import Data.Aeson (Value, object, (.=))+import Data.Aeson (Value (Null), object, (.=)) import Data.IORef (IORef, newIORef, readIORef, writeIORef) import Data.Text (Text) import Data.Text qualified as T@@ -18,7 +19,8 @@     ModelWithFallbacks (..),     defaultModelCapabilities,   )-import LLM.Generate.Types (GenerateError (..), GenerateErrorResult (..))+import LLM.Generate.Types (GenerateError (..), GenerateErrorResult (..), GeneratableObject)+import LLM.OptionalFields (Inner (..), Node (..), Optionals (..), innerDefaults) import LLM.WeatherTool (WeatherToolArgs (..)) import Test.Hspec   ( Spec,@@ -110,6 +112,62 @@             Just req -> req.reqTools `shouldBe` []         Left err -> expectationFailure $ show err +  describe "generateObject with optional fields answered null (strict mode)" $ do+    let nullInner = object ["a" .= Null, "b" .= Null]+        answer =+          object+            [ "req_maybe" .= Null,+              "or_null" .= Null,+              "opt" .= Null,+              "def" .= Null,+              "omit" .= Null,+              "nested" .= nullInner,+              "named" .= nullInner,+              "list" .= [nullInner, object ["a" .= ("x" :: Text), "b" .= (7 :: Int)]],+              "union" .= object ["Left" .= nullInner],+              "plain" .= ("p" :: Text)+            ]++    it "reads null as absent for every optional field, in nested objects, arrays, unions and $refs" $ do+      result <- runTypedWith (objectGateway answer (mkUsage 1 1))+      case result of+        Right (v, _) ->+          v+            `shouldBe` Optionals+              { reqMaybe = Nothing,+                orNull = Nothing,+                opt = Nothing,+                withDefault = 5,+                omitted = 5,+                nested = Just innerDefaults,+                named = Just innerDefaults,+                list = [innerDefaults, Inner (Just "x") 7],+                union = Left innerDefaults,+                plain = "p"+              }+        Left err -> expectationFailure $ show err++    it "reads a null optional object as absent" $ do+      let answer' = object ["req_maybe" .= Null, "nested" .= Null, "named" .= Null, "list" .= ([] :: [Value]), "union" .= object ["Right" .= (1 :: Int)], "plain" .= ("p" :: Text)]+      result <- runTypedWith @Optionals (objectGateway answer' (mkUsage 1 1))+      case result of+        Right (v, _) -> (v.nested, v.named, v.union) `shouldBe` (Nothing, Nothing, Right 1)+        Left err -> expectationFailure $ show err++    it "still reports null for a required field that is not nullable" $ do+      let answer' = object ["req_maybe" .= Null, "list" .= ([] :: [Value]), "union" .= object ["Right" .= (1 :: Int)], "plain" .= Null]+      result <- runTypedWith @Optionals (objectGateway answer' (mkUsage 1 1))+      case result of+        Left GenerateErrorResult {gerError = GErrParseObjectError _} -> pure ()+        other -> expectationFailure $ "expected GErrParseObjectError, got: " <> show (fmap fst other)++    it "reads null as absent in a recursive type" $ do+      let answer' = object ["label" .= ("a" :: Text), "next" .= object ["label" .= ("b" :: Text), "next" .= Null]]+      result <- runTypedWith (objectGateway answer' (mkUsage 1 1))+      case result of+        Right (v, _) -> v `shouldBe` Node "a" (Just (Node "b" Nothing))+        Left err -> expectationFailure $ show err+ objectGateway :: Value -> Usage -> LLMGateway objectGateway value usage =   LLMGateway@@ -197,3 +255,8 @@ runTyped models = do   rt <- mkRuntime Nothing   generateObject defaultAgent models rt [UserTurn "weather?"]++runTypedWith :: (GeneratableObject t) => LLMGateway -> IO (Either GenerateErrorResult (t, Usage))+runTypedWith gw = do+  rt <- mkRuntime Nothing+  generateObject defaultAgent (ModelWithFallbacks (mockModel gw) []) rt [UserTurn "extract"]

@@ -4,7 +4,9 @@  import Data.Aeson (object, (.=)) import Data.Text (Text)+import Data.ByteString qualified as BS import Data.Text qualified as T+import Data.Text.Encoding (encodeUtf8) import Heptapod (generate) import LLM.Agent.ToolUtils (toTool, windowOffset) import LLM.Agent.Tools.HistoryTool (historyToolTyped)@@ -12,6 +14,7 @@ import LLM.Core.Types (Turn (..), assistantTurn, pattern UserTurn) import LLM.Generate.GenerateUtils (llmHooks) import LLM.Generate.Logger (noHooks)+import LLM.Tools.FsLimits (maxOutputBytes) import Test.Hspec   ( Spec,     describe,@@ -19,6 +22,7 @@     shouldBe,     shouldContain,     shouldNotContain,+    shouldSatisfy,   )  spec :: Spec@@ -72,6 +76,39 @@       T.unpack result `shouldContain` "[User] hidden question"       T.unpack result `shouldContain` "[Assistant] hidden answer" +  describe "get_history page size" $ do+    it "leaves out the oldest turns of a page over maxOutputBytes and says how many" $ do+      let answer i = assistantTurn ("answer-" <> T.pack (show i) <> " " <> T.replicate 5000 "x") Nothing []+          conv =+            [UserTurn "old question"]+              ++ map answer [0 .. 39 :: Int]+              ++ [UserTurn "visible question"]+      result <- runHistory windowedAgent conv 0+      utf8Size result `shouldSatisfy` (<= maxOutputBytes)+      T.unpack result `shouldContain` "answer-39 "+      T.unpack result `shouldNotContain` "answer-0 "+      T.unpack result `shouldNotContain` "old question"+      let keptAnswers = length (filter ("[Assistant] answer-" `T.isPrefixOf`) (T.lines result))+      last (T.lines result)+        `shouldBe` "["+        <> T.pack (show (41 - keptAnswers))+        <> " earlier turn(s) of this page left out: the page is over "+        <> T.pack (show maxOutputBytes)+        <> " bytes]"++    it "cuts the newest turn when it alone is over maxOutputBytes" $ do+      let conv =+            [ UserTurn "old question",+              assistantTurn ("huge " <> T.replicate 200000 "y") Nothing [],+              UserTurn "visible question"+            ]+      result <- runHistory windowedAgent conv 0+      utf8Size result `shouldSatisfy` (<= maxOutputBytes)+      T.unpack result `shouldContain` "[Assistant] huge yyy"+      T.unpack result `shouldContain` "[turn cut: "+      T.unpack result `shouldNotContain` "old question"+      T.unpack result `shouldContain` "[1 earlier turn(s) of this page left out"+ sampleConversation :: [Turn] sampleConversation =   [ UserTurn "first question",@@ -118,3 +155,6 @@           }       tool = toTool historyToolTyped   tool.toolExecute ctx (object ["chunk" .= chunk])++utf8Size :: Text -> Int+utf8Size = BS.length . encodeUtf8

@@ -5,6 +5,7 @@ import Control.Monad (forever, void) import Data.Aeson (Value, object) import Data.ByteString qualified as BS+import Data.ByteString.Char8 qualified as BC import Data.IORef (atomicModifyIORef', newIORef, readIORef) import Data.Text qualified as T import System.Timeout (timeout)@@ -67,6 +68,31 @@           Left (ParseError msg) -> T.unpack msg `shouldBe` "response body exceeds 8388608 bytes"           other -> expectationFailure ("expected ParseError, got " <> take 200 (show other)) +  describe "stream total size" $ do+    it "turns an endless stream of small events into a ParseError" $+      withRepeatingServer "200 OK" "" (BS.concat (replicate 1000 textEvent)) $ \p -> do+        r <- within $ genericStreamText (ollamaProviderWith (http "127.0.0.1") (port p)) (llmHooks noHooks) request (const (pure ()))+        case r of+          Left (ParseError msg) -> T.unpack msg `shouldBe` "response body exceeds 33554432 bytes"+          other -> expectationFailure ("expected ParseError, got " <> take 200 (show other))++    it "parses a stream just under the total cap" $ do+      -- 31 comment lines of 1 MiB (each under the per-line cap), then a+      -- short reply: about 31 MiB in all.+      let padding = BS.concat (replicate 31 (": " <> BS.replicate (1024 * 1024) 0x70 <> "\n"))+          body = padding <> textEvent <> finishEvent <> "data: [DONE]\n\n"+      BS.length body `shouldSatisfy` (< 32 * 1024 * 1024)+      withFixedServer "200 OK" body $ \p -> do+        r <- within $ genericStreamText (ollamaProviderWith (http "127.0.0.1") (port p)) (llmHooks noHooks) request (const (pure ()))+        case r of+          Right _ -> pure ()+          Left err -> expectationFailure ("expected a parsed stream, got " <> take 200 (show err))++-- | One OpenAI-style stream event with a text delta, and the final one.+textEvent, finishEvent :: BS.ByteString+textEvent = "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"x\"},\"finish_reason\":null}]}\n\n"+finishEvent = "data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n"+ -- | Fail instead of hanging when a body is read without a bound. within :: IO a -> IO a within act = timeout 20_000_000 act >>= maybe (fail "timed out: the body was read without a bound") pure@@ -87,19 +113,34 @@ -- given status line, then the prefix, then @x@ bytes until the client -- hangs up. withEndlessServer :: BS.ByteString -> BS.ByteString -> (Int -> IO a) -> IO a-withEndlessServer status prefix action =+withEndlessServer status prefix = withRepeatingServer status prefix (BS.replicate 65536 0x78)++-- | Like 'withEndlessServer', repeating the given chunk.+withRepeatingServer :: BS.ByteString -> BS.ByteString -> BS.ByteString -> (Int -> IO a) -> IO a+withRepeatingServer status prefix chunk =+  withServer $ \conn -> do+    NB.sendAll conn ("HTTP/1.1 " <> status <> "\r\nContent-Length: 1000000000000\r\n\r\n" <> prefix)+    -- Stops when the client closes the connection and a send fails.+    void (try @SomeException (forever (NB.sendAll conn chunk)))++-- | A TCP server on a free local port that answers every POST with the+-- given status line and body.+withFixedServer :: BS.ByteString -> BS.ByteString -> (Int -> IO a) -> IO a+withFixedServer status body =+  withServer $ \conn ->+    void . try @SomeException $+      NB.sendAll conn ("HTTP/1.1 " <> status <> "\r\nContent-Length: " <> BC.pack (show (BS.length body)) <> "\r\n\r\n" <> body)++-- | A TCP server on a free local port that runs the handler for each POST.+withServer :: (N.Socket -> IO ()) -> (Int -> IO a) -> IO a+withServer handler action =   bracket open N.close $ \sock -> do     p <- fromIntegral <$> N.socketPort sock     tid <- forkIO $ forever $ do       (conn, _) <- N.accept sock       void $ forkIO $ flip finally (N.close conn) $ do         chunk <- NB.recv conn 4096-        if "POST" `BS.isPrefixOf` chunk-          then do-            NB.sendAll conn ("HTTP/1.1 " <> status <> "\r\nContent-Length: 1000000000000\r\n\r\n" <> prefix)-            -- Stops when the client closes the connection and a send fails.-            void (try @SomeException (forever (NB.sendAll conn (BS.replicate 65536 0x78))))-          else pure ()+        if "POST" `BS.isPrefixOf` chunk then handler conn else pure ()     action p `finally` killThread tid   where     open = do

@@ -88,6 +88,16 @@               )         Left err -> expectationFailure err +    -- gpt-5.6-terra refuses any temperature but the default (1): with+    -- 0.5 every call failed with 400.+    it "sends no temperature for the bundled gpt_5_6_terra" $ do+      result <- eitherDecodeFileStrict' "./model-catalog.json"+      case result of+        Right (items :: [ModelCatalogItem]) ->+          fmap (.temperature) (find (\i -> i.modelConfigName == "gpt_5_6_terra") items)+            `shouldBe` Just Nothing+        Left err -> expectationFailure err+     it "loads catalogs without capabilities (defaults false)" $ do       cfg <- loadModelOrThrow ollamaCatalog "mistral"       cfg.mcCapabilities `shouldBe` defaultModelCapabilities

@@ -2,10 +2,13 @@  module LLM.OpenAISpec (spec) where +import Autodocodec qualified as AC+import Autodocodec.Schema (jsonSchemaVia) import Data.Aeson   ( Value (Array, Object, String),     eitherDecodeFileStrict',     object,+    toJSON,     (.=),   ) import Data.Aeson.Key qualified as K@@ -32,8 +35,10 @@     thinkingPart,     toolCallPart,   )+import LLM.Core.ProviderUtils (normalizeSchemaOpenAI, stripBoundsAndComments) import LLM.Core.Usage (Usage (..), mkUsage) import LLM.Core.Utils (getToolCalls, hasToolCalls, isRetryable)+import LLM.OptionalFields (Node, Optionals) import LLM.Providers.OpenAI (encodeTurn, parseOpenAIResponse, parseOpenAIStream, parseOpenAIUsage) import Test.Hspec   ( Spec,@@ -200,6 +205,94 @@           contentTypes parts `shouldBe` ["text", "image_url", "image_url"]           mapM_ (\p -> lookupKey "cache_control" p `shouldBe` Nothing) parts +  describe "normalizeSchemaOpenAI optional properties (strict mode)" $ do+    let schema = strictSchema (AC.codec :: AC.JSONCodec Optionals)+        prop k = schemaAt ["properties", k] schema++    it "lists every property as required" $+      schemaAt ["required"] schema+        `shouldBe` Just+          (toJSON (["def", "list", "named", "nested", "omit", "opt", "or_null", "plain", "req_maybe", "union"] :: [Text]))++    it "makes optionalField, optionalFieldWithDefault and optionalFieldWithOmittedDefault nullable" $ do+      prop "opt" `shouldBe` Just (nullable (typed "string"))+      prop "def" `shouldBe` Just (nullable (typed "integer"))+      prop "omit" `shouldBe` Just (nullable (typed "integer"))++    it "makes an optional nested object nullable, inline or by $ref" $ do+      fmap (const ()) (schemaAt ["properties", "nested", "anyOf", "0", "properties"] schema) `shouldBe` Just ()+      schemaAt ["properties", "nested", "anyOf", "1"] schema `shouldBe` Just (typed "null")+      schemaAt ["properties", "named", "anyOf", "0", "$ref"] schema `shouldBe` Just "#/$defs/NamedInner"+      schemaAt ["properties", "named", "anyOf", "1"] schema `shouldBe` Just (typed "null")++    it "leaves nullable and required properties as they are" $ do+      let nullOrString = object ["anyOf" .= [typed "null", typed "string"]]+      prop "req_maybe" `shouldBe` Just nullOrString+      prop "or_null" `shouldBe` Just nullOrString+      prop "plain" `shouldBe` Just (typed "string")+      schemaAt ["properties", "list", "type"] schema `shouldBe` Just "array"++    it "does the same inside nested objects, array items, union branches and definitions" $ do+      let inner path = do+            schemaAt (path <> ["properties", "a"]) schema `shouldBe` Just (nullable (typed "string"))+            schemaAt (path <> ["properties", "b"]) schema `shouldBe` Just (nullable (typed "integer"))+            schemaAt (path <> ["required"]) schema `shouldBe` Just (toJSON (["a", "b"] :: [Text]))+      inner ["properties", "nested", "anyOf", "0"]+      inner ["properties", "list", "items"]+      inner ["properties", "union", "anyOf", "0", "properties", "Left"]+      inner ["$defs", "NamedInner"]++    it "turns the oneOf of an Either into anyOf" $ do+      schemaAt ["properties", "union", "oneOf"] schema `shouldBe` Nothing+      schemaAt ["properties", "union", "anyOf", "0", "required"] schema `shouldBe` Just (toJSON (["Left"] :: [Text]))+      schemaAt ["properties", "union", "anyOf", "1", "required"] schema `shouldBe` Just (toJSON (["Right"] :: [Text]))++    it "changes only schema keywords, not property names or literal values" $ do+      let oneOfSchema = object ["oneOf" .= [typed "string", typed "integer"]]+          literal = object ["type" .= ("object" :: Text), "oneOf" .= [typed "string"]]+          input =+            object+              [ "type" .= ("object" :: Text),+                "properties" .= object ["oneOf" .= oneOfSchema],+                "required" .= (["oneOf"] :: [Text]),+                "default" .= object ["oneOf" .= literal]+              ]+      normalizeSchemaOpenAI input+        `shouldBe` object+          [ "type" .= ("object" :: Text),+            "properties" .= object ["oneOf" .= object ["anyOf" .= [typed "string", typed "integer"]]],+            "required" .= (["oneOf"] :: [Text]),+            "default" .= object ["oneOf" .= literal],+            "additionalProperties" .= False+          ]++    it "moves the $defs of a named codec to the root" $ do+      schemaAt ["properties", "named", "anyOf", "0"] schema+        `shouldBe` Just (object ["$ref" .= ("#/$defs/NamedInner" :: Text)])+      fmap (const ()) (schemaAt ["$defs", "NamedInner", "properties"] schema) `shouldBe` Just ()++    it "keeps $defs that are at the root already, and a property called $defs" $ do+      let node = strictSchema (AC.codec :: AC.JSONCodec Node)+      schemaAt ["$ref"] node `shouldBe` Just "#/$defs/Node"+      fmap (const ()) (schemaAt ["$defs", "Node", "properties"] node) `shouldBe` Just ()+      let input =+            object+              [ "type" .= ("object" :: Text),+                "properties" .= object ["$defs" .= typed "string"],+                "required" .= (["$defs"] :: [Text])+              ]+      normalizeSchemaOpenAI input+        `shouldBe` object+          [ "type" .= ("object" :: Text),+            "properties" .= object ["$defs" .= typed "string"],+            "required" .= (["$defs"] :: [Text]),+            "additionalProperties" .= False+          ]++    it "makes a recursive optional property nullable" $+      schemaAt ["$defs", "Node", "properties", "next"] (strictSchema (AC.codec :: AC.JSONCodec Node))+        `shouldBe` Just (nullable (object ["$ref" .= ("#/$defs/Node" :: Text)]))+   describe "parseOpenAIUsage" $ do     it "extracts token counts" $ do       Right val <- eitherDecodeFileStrict' "test/fixtures/openai-text.json"@@ -245,6 +338,25 @@                 _ -> expectationFailure "missing response"             _ -> expectationFailure "empty conversation array"         _ -> expectationFailure "expected conversation array"++-- | The schema 'genObject' sends for a codec, as OpenAI receives it.+strictSchema :: AC.JSONCodec a -> Value+strictSchema c = normalizeSchemaOpenAI (stripBoundsAndComments (toJSON (jsonSchemaVia c)))++-- | The sub-schema at a path of keys; a number indexes an array.+schemaAt :: [Text] -> Value -> Maybe Value+schemaAt [] v = Just v+schemaAt (k : rest) (Object o) = KM.lookup (K.fromText k) o >>= schemaAt rest+schemaAt (k : rest) (Array a) = case reads (T.unpack k) of+  [(i, "")] -> a V.!? i >>= schemaAt rest+  _ -> Nothing+schemaAt _ _ = Nothing++nullable :: Value -> Value+nullable s = object ["anyOf" .= [s, object ["type" .= ("null" :: Text)]]]++typed :: Text -> Value+typed t = object ["type" .= t]  userContent :: Value -> [Value] userContent (Object o) =

@@ -0,0 +1,76 @@+-- | A structured-output type with every kind of optional field+-- autodocodec offers, for the strict-schema tests (P3 in fixes-043).+module LLM.OptionalFields+  ( Optionals (..),+    Inner (..),+    Node (..),+    innerDefaults,+  )+where++import Autodocodec qualified as AC+import Data.Aeson (FromJSON)+import Data.Text (Text)++data Inner = Inner+  { innerA :: Maybe Text,+    innerB :: Int+  }+  deriving (Eq, Show)++-- | The value of an 'Inner' whose fields were all left out.+innerDefaults :: Inner+innerDefaults = Inner Nothing 3++instance AC.HasCodec Inner where+  codec =+    AC.object "Inner" $+      Inner+        <$> AC.optionalField "a" "optional" AC..= (.innerA)+        <*> AC.optionalFieldWithDefault "b" 3 "with default" AC..= (.innerB)++data Optionals = Optionals+  { reqMaybe :: Maybe Text,+    orNull :: Maybe Text,+    opt :: Maybe Text,+    withDefault :: Int,+    omitted :: Int,+    nested :: Maybe Inner,+    named :: Maybe Inner,+    list :: [Inner],+    union :: Either Inner Int,+    plain :: Text+  }+  deriving (Eq, Show)+  deriving (FromJSON) via (AC.Autodocodec Optionals)++instance AC.HasCodec Optionals where+  codec =+    AC.object "Optionals" $+      Optionals+        <$> AC.requiredField "req_maybe" "required, Maybe" AC..= (.reqMaybe)+        <*> AC.optionalFieldOrNull "or_null" "optionalFieldOrNull" AC..= (.orNull)+        <*> AC.optionalField "opt" "optionalField" AC..= (.opt)+        <*> AC.optionalFieldWithDefault "def" 5 "optionalFieldWithDefault" AC..= (.withDefault)+        <*> AC.optionalFieldWithOmittedDefault "omit" 5 "optionalFieldWithOmittedDefault" AC..= (.omitted)+        <*> AC.optionalField "nested" "nested object" AC..= (.nested)+        <*> AC.optionalFieldWith "named" (AC.named "NamedInner" AC.codec) "named object ($ref)" AC..= (.named)+        <*> AC.requiredField "list" "objects in an array" AC..= (.list)+        <*> AC.requiredField "union" "Either (oneOf)" AC..= (.union)+        <*> AC.requiredField "plain" "required, not nullable" AC..= (.plain)++-- | A recursive type: its schema refers to itself through @$ref@.+data Node = Node+  { label :: Text,+    next :: Maybe Node+  }+  deriving (Eq, Show)+  deriving (FromJSON) via (AC.Autodocodec Node)++instance AC.HasCodec Node where+  codec =+    AC.named "Node" $+      AC.object "Node" $+        Node+          <$> AC.requiredField "label" "label" AC..= (.label)+          <*> AC.optionalField "next" "next node" AC..= (.next)

@@ -0,0 +1,86 @@+module LLM.TextFileSpec (spec) where++import Control.Exception (throwIO)+import Data.ByteString qualified as BS+import Data.Text (Text)+import LLM.Tools.Internal.TextFile (CappedRead (..), readBoundedBytesWith, readBoundedBytesWithSize, readCapped, readCappedWith)+import System.FilePath ((</>))+import System.IO (Handle, IOMode (ReadMode), hFileSize, withBinaryFile)+import System.IO.Temp (withSystemTempDirectory)+import Test.Hspec (Spec, describe, it, shouldReturn)++spec :: Spec+spec = do+  readCappedSpec+  readBoundedBytesSpec++readCappedSpec :: Spec+readCappedSpec = describe "TextFile.readCapped" $ do+  it "reads a file of exactly the cap" $+    withFile (BS.replicate cap 97) $ \path ->+      withBinaryFile path ReadMode (readCapped (fromIntegral cap))+        `shouldReturn` CappedBytes (fromIntegral cap) (BS.replicate cap 97)++  it "refuses a file over the cap at the size check" $+    withFile (BS.replicate (cap + 1) 97) $ \path ->+      withBinaryFile path ReadMode (readCapped (fromIntegral cap))+        `shouldReturn` CappedTooLarge (fromIntegral cap + 1)++  -- C23: a file within the cap at the size check that grows past it+  -- before the read must be an error, not a page that ends at the cap.+  it "reports a file that grows past the cap after the size check" $+    withFile (BS.replicate (cap + 2) 97) $ \path ->+      withBinaryFile path ReadMode (readCappedWith (sizeBeforeGrowth 2) (fromIntegral cap))+        `shouldReturn` CappedGrew++  it "reads a file that grows but stays within the cap" $+    withFile (BS.replicate cap 97) $ \path ->+      withBinaryFile path ReadMode (readCappedWith (sizeBeforeGrowth 2) (fromIntegral cap))+        `shouldReturn` CappedBytes (fromIntegral cap - 2) (BS.replicate cap 97)+  where+    cap = 1000++readBoundedBytesSpec :: Spec+readBoundedBytesSpec = describe "TextFile.readBoundedBytesWith" $ do+  it "reads a file of exactly the cap" $+    withFile (BS.replicate cap 97) $ \path ->+      readBoundedBytesWith (withBinaryFile path ReadMode) name cap+        `shouldReturn` Right (BS.replicate cap 97)++  it "refuses a file over the cap at the size check" $+    withFile (BS.replicate (cap + 1) 97) $ \path ->+      readBoundedBytesWith (withBinaryFile path ReadMode) name cap+        `shouldReturn` Left "f.txt is 1001 bytes, which exceeds the 1000-byte cap."++  it "reports a file that grows past the cap after the size check" $+    withFile (BS.replicate (cap + 2) 97) $ \path ->+      readBoundedBytesWithSize (sizeBeforeGrowth 2) (withBinaryFile path ReadMode) name cap+        `shouldReturn` Left "f.txt grew past the 1000-byte cap during the read."++  it "reads a file that grows but stays within the cap" $+    withFile (BS.replicate cap 97) $ \path ->+      readBoundedBytesWithSize (sizeBeforeGrowth 2) (withBinaryFile path ReadMode) name cap+        `shouldReturn` Right (BS.replicate cap 97)++  it "names the size check when it fails" $+    withFile "text" $ \path ->+      readBoundedBytesWithSize (\_ -> throwIO (userError "no size")) (withBinaryFile path ReadMode) name cap+        `shouldReturn` Left "could not stat f.txt: user error (no size)"+  where+    cap = 1000+    name :: Text+    name = "f.txt"++-- | A size check that sees the file as it was @n@ bytes earlier: the file+-- grew by @n@ bytes between the check and the read. (Appending for real+-- from the check does not work: GHC's file locking refuses a write handle+-- while the read handle is open.)+sizeBeforeGrowth :: Integer -> Handle -> IO Integer+sizeBeforeGrowth n h = subtract n <$> hFileSize h++withFile :: BS.ByteString -> (FilePath -> IO a) -> IO a+withFile content act =+  withSystemTempDirectory "llm-simple-textfile" $ \dir -> do+    let path = dir </> "f.txt"+    BS.writeFile path content+    act path

@@ -19,6 +19,8 @@     textPart,     thinkingPart,     toolCallPart,+    ToolResult (..),+    validateConversation,     validateTurn,   ) import LLM.Core.Usage@@ -196,6 +198,47 @@       validateTurn         (AssistantMessage [imageUrlPart "https://example.com/a.png"])         `shouldBe` Left "assistant messages may not contain image parts"++  describe "validateConversation" $ do+    let call i = toolCallPart (mkToolCall i "get_weather" (object []))+        result i = ToolResult i "get_weather" "sunny"+    it "accepts a valid multi-round history" $+      validateConversation+        [ UserTurn "weather?",+          AssistantMessage [textPart "checking", call "c1", call "c2"],+          ToolTurn [result "c2", result "c1"],+          AssistantMessage [call "c3"],+          ToolTurn [result "c3"],+          AssistantMessage [textPart "sunny"],+          UserTurn "thanks"+        ]+        `shouldBe` []++    it "reports tool results that do not follow tool calls" $ do+      validateConversation [UserTurn "hi", ToolTurn [result "c1"]]+        `shouldBe` [(1, "tool results do not follow an assistant turn with tool calls")]+      validateConversation [UserTurn "hi", AssistantMessage [textPart "no calls"], ToolTurn [result "c1"]]+        `shouldBe` [(2, "tool results do not follow an assistant turn with tool calls")]++    it "reports tool calls without a following tool turn" $+      validateConversation [UserTurn "hi", AssistantMessage [call "c1"], UserTurn "again"]+        `shouldBe` [(1, "tool calls are not followed by tool results")]++    it "reports a call without a result" $+      validateConversation [UserTurn "hi", AssistantMessage [call "c1", call "c2"], ToolTurn [result "c1"]]+        `shouldBe` [(2, "no result for tool call \"c2\"")]++    it "reports a call answered twice" $+      validateConversation [UserTurn "hi", AssistantMessage [call "c1"], ToolTurn [result "c1", result "c1"]]+        `shouldBe` [(2, "tool call \"c1\" is answered more than once")]++    it "reports a result for an unknown call id" $+      validateConversation [UserTurn "hi", AssistantMessage [call "c1"], ToolTurn [result "c1", result "zz"]]+        `shouldBe` [(2, "tool result for unknown call id \"zz\"")]++    it "reports duplicate call ids in one assistant turn" $+      validateConversation [UserTurn "hi", AssistantMessage [call "c1", call "c1"], ToolTurn [result "c1"]]+        `shouldBe` [(1, "duplicate tool call id \"c1\"")]    describe "imageBase64Part" $ do     it "accepts valid png base64" $ do

@@ -0,0 +1,364 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": ["location"],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "caller": {+                        "type": "direct"+                    },+                    "id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                    "input": {+                        "location": "London"+                    },+                    "name": "get_weather",+                    "type": "tool_use"+                }+            ],+            "id": "msg_01JZsXgNBYvVB4gFbTAEHFzc",+            "model": "claude-haiku-4-5-20251001",+            "role": "assistant",+            "stop_details": null,+            "stop_reason": "tool_use",+            "stop_sequence": null,+            "type": "message",+            "usage": {+                "cache_creation": {+                    "ephemeral_1h_input_tokens": 0,+                    "ephemeral_5m_input_tokens": 0+                },+                "cache_creation_input_tokens": 0,+                "cache_read_input_tokens": 0,+                "inference_geo": "not_available",+                "input_tokens": 623,+                "output_tokens": 54,+                "service_tier": "standard"+            }+        }+    },+    {+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                            "input": {+                                "location": "London"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                            "tool_use_id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": ["location"],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's pleasant weather overall!",+                    "type": "text"+                }+            ],+            "id": "msg_01UXKDjyeAJFHXJXzsvVBNtE",+            "model": "claude-haiku-4-5-20251001",+            "role": "assistant",+            "stop_details": null,+            "stop_reason": "end_turn",+            "stop_sequence": null,+            "type": "message",+            "usage": {+                "cache_creation": {+                    "ephemeral_1h_input_tokens": 0,+                    "ephemeral_5m_input_tokens": 0+                },+                "cache_creation_input_tokens": 0,+                "cache_read_input_tokens": 0,+                "inference_geo": "not_available",+                "input_tokens": 708,+                "output_tokens": 40,+                "service_tier": "standard"+            }+        }+    },+    {+        "prompt": "and in paris?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                            "input": {+                                "location": "London"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                            "tool_use_id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's pleasant weather overall!",+                            "type": "text"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": ["location"],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "caller": {+                        "type": "direct"+                    },+                    "id": "toolu_01Q5J5oAGkvBUfM52fFfshJk",+                    "input": {+                        "location": "Paris"+                    },+                    "name": "get_weather",+                    "type": "tool_use"+                }+            ],+            "id": "msg_019hKACjpVyumUCpqRWHAcb8",+            "model": "claude-haiku-4-5-20251001",+            "role": "assistant",+            "stop_details": null,+            "stop_reason": "tool_use",+            "stop_sequence": null,+            "type": "message",+            "usage": {+                "cache_creation": {+                    "ephemeral_1h_input_tokens": 0,+                    "ephemeral_5m_input_tokens": 0+                },+                "cache_creation_input_tokens": 0,+                "cache_read_input_tokens": 0,+                "inference_geo": "not_available",+                "input_tokens": 755,+                "output_tokens": 54,+                "service_tier": "standard"+            }+        }+    },+    {+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                            "input": {+                                "location": "London"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                            "tool_use_id": "toolu_01DDtP6JvBHoWtxVLcVsz1JJ",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's pleasant weather overall!",+                            "type": "text"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01Q5J5oAGkvBUfM52fFfshJk",+                            "input": {+                                "location": "Paris"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in Paris is sunny, 23°C, no wind.",+                            "tool_use_id": "toolu_01Q5J5oAGkvBUfM52fFfshJk",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": ["location"],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The weather in Paris is **sunny** with a temperature of **23°C** and **no wind**. It's warmer and nicer than London at the moment!",+                    "type": "text"+                }+            ],+            "id": "msg_01KwfBSA7mE7KUiq6gRk19go",+            "model": "claude-haiku-4-5-20251001",+            "role": "assistant",+            "stop_details": null,+            "stop_reason": "end_turn",+            "stop_sequence": null,+            "type": "message",+            "usage": {+                "cache_creation": {+                    "ephemeral_1h_input_tokens": 0,+                    "ephemeral_5m_input_tokens": 0+                },+                "cache_creation_input_tokens": 0,+                "cache_read_input_tokens": 0,+                "inference_geo": "not_available",+                "input_tokens": 834,+                "output_tokens": 39,+                "service_tier": "standard"+            }+        }+    }+]

@@ -0,0 +1,332 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "stream": true,+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": [+                            "location"+                        ],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "arguments": {+                        "location": "London"+                    },+                    "id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                    "name": "get_weather",+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 623,+                "output_tokens": 54+            }+        },+        "sse": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"model\":\"claude-haiku-4-5-20251001\",\"id\":\"msg_011CfxsaAbH28pQFPzpsK9Bc\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":623,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"cache_creation\":{\"ephemeral_5m_input_tokens\":0,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":22,\"service_tier\":\"standard\",\"inference_geo\":\"not_available\"},\"diagnostics\":null}            }\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_01GvM5JAsPjxKLnk7qohqRSR\",\"name\":\"get_weather\",\"input\":{},\"caller\":{\"type\":\"direct\"}}              }\n\nevent: ping\ndata: {\"type\": \"ping\"}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"\"}  }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"locat\"}         }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"ion\\\": \\\"\"}  }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"London\\\"}\"} }\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0      }\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null,\"stop_details\":null,\"container\":null},\"usage\":{\"input_tokens\":623,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"output_tokens\":54}              }\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"              }\n\n"+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                            "input": {+                                "location": "London"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                            "tool_use_id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "stream": true,+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": [+                            "location"+                        ],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's a mild day, so you might want to bring a light jacket if you're heading out!",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's a mild day, so you might want to bring a light jacket if you're heading out!",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 708,+                "output_tokens": 55+            }+        },+        "sse": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"model\":\"claude-haiku-4-5-20251001\",\"id\":\"msg_011CfxsaDxCHnwCSKMbSgBYN\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":708,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"cache_creation\":{\"ephemeral_5m_input_tokens\":0,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":1,\"service_tier\":\"standard\",\"inference_geo\":\"not_available\"},\"diagnostics\":null}    }\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}     }\n\nevent: ping\ndata: {\"type\": \"ping\"}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"The\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" weather in London is\"}            }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" currently **\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"partly cloudy**\"}            }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" with a temperature of\"} }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" **\"}          }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"18°C**\"}             }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" and a **\"}              }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"light breeze from\"}    }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" the west**.\"}  }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" It\"}         }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"'s a mild\"}               }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" day\"}          }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\",\"}            }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" so\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" you might\"}    }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" want to bring\"}         }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" a light jacket if\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" you're heading\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" out!\"}     }\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0             }\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\",\"stop_sequence\":null,\"stop_details\":null,\"container\":null},\"usage\":{\"input_tokens\":708,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"output_tokens\":55}           }\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"               }\n\n"+    },+    {+        "prompt": "and in paris?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                            "input": {+                                "location": "London"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                            "tool_use_id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's a mild day, so you might want to bring a light jacket if you're heading out!",+                            "type": "text"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "stream": true,+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": [+                            "location"+                        ],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "arguments": {+                        "location": "Paris"+                    },+                    "id": "toolu_01Ut7JGiytus1NwfjxXuni3Z",+                    "name": "get_weather",+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 770,+                "output_tokens": 54+            }+        },+        "sse": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"model\":\"claude-haiku-4-5-20251001\",\"id\":\"msg_011CfxsaHwoLbCoepii2Rhw4\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":770,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"cache_creation\":{\"ephemeral_5m_input_tokens\":0,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":21,\"service_tier\":\"standard\",\"inference_geo\":\"not_available\"},\"diagnostics\":null}   }\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_01Ut7JGiytus1NwfjxXuni3Z\",\"name\":\"get_weather\",\"input\":{},\"caller\":{\"type\":\"direct\"}}     }\n\nevent: ping\ndata: {\"type\": \"ping\"}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"locat\"}          }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"ion\\\": \\\"\"}        }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"Paris\\\"}\"}     }\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0             }\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null,\"stop_details\":null,\"container\":null},\"usage\":{\"input_tokens\":770,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"output_tokens\":54}            }\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"        }\n\n"+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                            "input": {+                                "location": "London"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                            "tool_use_id": "toolu_01GvM5JAsPjxKLnk7qohqRSR",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "text": "The weather in London is currently **partly cloudy** with a temperature of **18°C** and a **light breeze from the west**. It's a mild day, so you might want to bring a light jacket if you're heading out!",+                            "type": "text"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                },+                {+                    "content": [+                        {+                            "id": "toolu_01Ut7JGiytus1NwfjxXuni3Z",+                            "input": {+                                "location": "Paris"+                            },+                            "name": "get_weather",+                            "type": "tool_use"+                        }+                    ],+                    "role": "assistant"+                },+                {+                    "content": [+                        {+                            "content": "Weather in Paris is sunny, 23°C, no wind.",+                            "tool_use_id": "toolu_01Ut7JGiytus1NwfjxXuni3Z",+                            "type": "tool_result"+                        }+                    ],+                    "role": "user"+                }+            ],+            "model": "claude-haiku-4-5-20251001",+            "stream": true,+            "system": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+            "tools": [+                {+                    "description": "Get the current weather for a given location",+                    "input_schema": {+                        "$comment": "WeatherToolArgs",+                        "properties": {+                            "location": {+                                "$comment": "City name, e.g. London",+                                "type": "string"+                            }+                        },+                        "required": [+                            "location"+                        ],+                        "type": "object"+                    },+                    "name": "get_weather"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The weather in Paris is **sunny** with a temperature of **23°C** and **no wind**. It's a beautiful day in Paris - warmer and sunnier than London!",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "The weather in Paris is **sunny** with a temperature of **23°C** and **no wind**. It's a beautiful day in Paris - warmer and sunnier than London!",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 849,+                "output_tokens": 42+            }+        },+        "sse": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"model\":\"claude-haiku-4-5-20251001\",\"id\":\"msg_011CfxsaM9HegK5K5qdRsJvE\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":849,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"cache_creation\":{\"ephemeral_5m_input_tokens\":0,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":4,\"service_tier\":\"standard\",\"inference_geo\":\"not_available\"},\"diagnostics\":null}           }\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}        }\n\nevent: ping\ndata: {\"type\": \"ping\"}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"The weather in Paris\"}         }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" is **\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"sunny** with a\"}              }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" temperature of **23\"}      }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"°C** and\"}  }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" **\"}       }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"no wind**.\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" It's a beautiful\"}              }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" day in Paris\"}  }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" -\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" war\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"mer and\"}      }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\" sunnier than London\"}     }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"!\"}    }\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0         }\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\",\"stop_sequence\":null,\"stop_details\":null,\"container\":null},\"usage\":{\"input_tokens\":849,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"output_tokens\":42}        }\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"               }\n\n"+    }+]

@@ -0,0 +1,18 @@+event: message_start+data: {"type":"message_start","message":{"id":"msg_trunc","type":"message","role":"assistant","model":"claude-haiku-4-5-20251001","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":10,"output_tokens":0}}}++event: content_block_start+data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_trunc_1","name":"get_weather","input":{}}}++event: content_block_delta+data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"location\": \"Lon"}}++event: content_block_stop+data: {"type":"content_block_stop","index":0}++event: message_delta+data: {"type":"message_delta","delta":{"stop_reason":"max_tokens","stop_sequence":null},"usage":{"output_tokens":16}}++event: message_stop+data: {"type":"message_stop"}+

@@ -0,0 +1,12 @@+event: message_start+data: {"type":"message_start","message":{"id":"msg_err","type":"message","role":"assistant","model":"claude-haiku-4-5-20251001","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":10,"output_tokens":0}}}++event: content_block_start+data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}++event: content_block_delta+data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Partial answer"}}++event: error+data: {"type":"error","error":{"type":"overloaded_error","message":"Overloaded"}}+

@@ -0,0 +1,17 @@+{+    "id": "msg_01XFDUDYJgAACzvnptvVoYEL",+    "type": "message",+    "role": "assistant",+    "content": [+        {+            "type": "text",+            "text": "Hello! How can I help you today?"+        }+    ],+    "model": "claude-haiku-4-5-20251001",+    "stop_reason": "end_turn",+    "usage": {+        "input_tokens": 25,+        "output_tokens": 10+    }+}

@@ -0,0 +1,28 @@+{+    "id": "msg_thinking_tool",+    "type": "message",+    "role": "assistant",+    "model": "claude-haiku-4-5-20251001",+    "content": [+        {+            "type": "thinking",+            "thinking": "I should call the weather tool.",+            "signature": "sig_thinking_abc123"+        },+        {+            "type": "text",+            "text": "Checking the weather."+        },+        {+            "type": "tool_use",+            "id": "toolu_weather_1",+            "name": "get_weather",+            "input": { "location": "London" }+        }+    ],+    "stop_reason": "tool_use",+    "usage": {+        "input_tokens": 40,+        "output_tokens": 60+    }+}

@@ -0,0 +1,38 @@+event: message_start+data: {"type":"message_start","message":{"id":"msg_thinking_tool","type":"message","role":"assistant","model":"claude-haiku-4-5-20251001","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":40,"output_tokens":0,"cache_read_input_tokens":0,"cache_creation_input_tokens":0}}}++event: content_block_start+data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}++event: content_block_delta+data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"I should call the weather tool."}}++event: content_block_delta+data: {"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"sig_thinking_abc123"}}++event: content_block_stop+data: {"type":"content_block_stop","index":0}++event: content_block_start+data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}}++event: content_block_delta+data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Checking the weather."}}++event: content_block_stop+data: {"type":"content_block_stop","index":1}++event: content_block_start+data: {"type":"content_block_start","index":2,"content_block":{"type":"tool_use","id":"toolu_weather_1","name":"get_weather","input":{}}}++event: content_block_delta+data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"{\"location\": \"London\"}"}}++event: content_block_stop+data: {"type":"content_block_stop","index":2}++event: message_delta+data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":60}}++event: message_stop+data: {"type":"message_stop"}

@@ -0,0 +1,23 @@+{+    "id": "msg_01Aq9w938a90dw8q",+    "type": "message",+    "role": "assistant",+    "content": [+        {+            "type": "text",+            "text": "Let me check the weather for you."+        },+        {+            "type": "tool_use",+            "id": "toolu_01A09q90qw90lq917835lq9",+            "name": "get_weather",+            "input": { "location": "London" }+        }+    ],+    "model": "claude-haiku-4-5-20251001",+    "stop_reason": "tool_use",+    "usage": {+        "input_tokens": 50,+        "output_tokens": 35+    }+}

@@ -0,0 +1,414 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                }+            ],+            "model": "deepseek-v4-flash",+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "tool_calls",+                    "index": 0,+                    "logprobs": null,+                    "message": {+                        "content": "",+                        "reasoning_content": "The user is asking about the weather in London. I can use the get_weather tool to answer this question.",+                        "role": "assistant",+                        "tool_calls": [+                            {+                                "function": {+                                    "arguments": "{\"location\": \"London\"}",+                                    "name": "get_weather"+                                },+                                "id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618",+                                "index": 0,+                                "type": "function"+                            }+                        ]+                    }+                }+            ],+            "created": 1783771054,+            "id": "86881337-9981-4093-b08b-c6bb828f51ef",+            "model": "deepseek-v4-flash",+            "object": "chat.completion",+            "system_fingerprint": "fp_8b330d02d0_prod0820_fp8_kvcache_20260402",+            "usage": {+                "completion_tokens": 68,+                "completion_tokens_details": {+                    "reasoning_tokens": 23+                },+                "prompt_cache_hit_tokens": 256,+                "prompt_cache_miss_tokens": 80,+                "prompt_tokens": 336,+                "prompt_tokens_details": {+                    "cached_tokens": 256+                },+                "total_tokens": 404+            }+        }+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "reasoning_content": "The user is asking about the weather in London. I can use the get_weather tool to answer this question.",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618"+                }+            ],+            "model": "deepseek-v4-flash",+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "stop",+                    "index": 0,+                    "logprobs": null,+                    "message": {+                        "content": "The weather in London is currently **partly cloudy** with a temperature of **18°C**. There's a **light breeze** coming from the west. Not too bad! 🌤️",+                        "reasoning_content": "I have the weather information for London. Let me share it with the user.",+                        "role": "assistant"+                    }+                }+            ],+            "created": 1783771055,+            "id": "aa2e01d5-7e2f-441d-a51e-67dba6def0f5",+            "model": "deepseek-v4-flash",+            "object": "chat.completion",+            "system_fingerprint": "fp_8b330d02d0_prod0820_fp8_kvcache_20260402",+            "usage": {+                "completion_tokens": 56,+                "completion_tokens_details": {+                    "reasoning_tokens": 16+                },+                "prompt_cache_hit_tokens": 384,+                "prompt_cache_miss_tokens": 49,+                "prompt_tokens": 433,+                "prompt_tokens_details": {+                    "cached_tokens": 384+                },+                "total_tokens": 489+            }+        }+    },+    {+        "prompt": "and in paris?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "reasoning_content": "The user is asking about the weather in London. I can use the get_weather tool to answer this question.",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618"+                },+                {+                    "content": "The weather in London is currently **partly cloudy** with a temperature of **18°C**. There's a **light breeze** coming from the west. Not too bad! 🌤️",+                    "reasoning_content": "I have the weather information for London. Let me share it with the user.",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                }+            ],+            "model": "deepseek-v4-flash",+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "tool_calls",+                    "index": 0,+                    "logprobs": null,+                    "message": {+                        "content": "",+                        "reasoning_content": "The user is asking about the weather in Paris. Let me get that information.",+                        "role": "assistant",+                        "tool_calls": [+                            {+                                "function": {+                                    "arguments": "{\"location\": \"Paris\"}",+                                    "name": "get_weather"+                                },+                                "id": "call_00_1PYplGFNtKGlcxKaWG299040",+                                "index": 0,+                                "type": "function"+                            }+                        ]+                    }+                }+            ],+            "created": 1783771056,+            "id": "87b805ea-a0bb-4a9d-8c98-5b2136a56fae",+            "model": "deepseek-v4-flash",+            "object": "chat.completion",+            "system_fingerprint": "fp_8b330d02d0_prod0820_fp8_kvcache_20260402",+            "usage": {+                "completion_tokens": 61,+                "completion_tokens_details": {+                    "reasoning_tokens": 16+                },+                "prompt_cache_hit_tokens": 384,+                "prompt_cache_miss_tokens": 114,+                "prompt_tokens": 498,+                "prompt_tokens_details": {+                    "cached_tokens": 384+                },+                "total_tokens": 559+            }+        }+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "reasoning_content": "The user is asking about the weather in London. I can use the get_weather tool to answer this question.",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_00_Rx8k7TxPUlsbOoGFTqVR6618"+                },+                {+                    "content": "The weather in London is currently **partly cloudy** with a temperature of **18°C**. There's a **light breeze** coming from the west. Not too bad! 🌤️",+                    "reasoning_content": "I have the weather information for London. Let me share it with the user.",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                },+                {+                    "reasoning_content": "The user is asking about the weather in Paris. Let me get that information.",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"Paris\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_1PYplGFNtKGlcxKaWG299040",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in Paris is sunny, 23°C, no wind.",+                    "role": "tool",+                    "tool_call_id": "call_00_1PYplGFNtKGlcxKaWG299040"+                }+            ],+            "model": "deepseek-v4-flash",+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "stop",+                    "index": 0,+                    "logprobs": null,+                    "message": {+                        "content": "The weather in Paris is **sunny** and **23°C** with no wind. ☀️ It's a bit warmer than London and a lovely day there!",+                        "reasoning_content": "Let me share the Paris weather info with the user.",+                        "role": "assistant"+                    }+                }+            ],+            "created": 1783771057,+            "id": "a40d09ab-3803-4018-b25b-b92f7938c3cb",+            "model": "deepseek-v4-flash",+            "object": "chat.completion",+            "system_fingerprint": "fp_8b330d02d0_prod0820_fp8_kvcache_20260402",+            "usage": {+                "completion_tokens": 47,+                "completion_tokens_details": {+                    "reasoning_tokens": 11+                },+                "prompt_cache_hit_tokens": 512,+                "prompt_cache_miss_tokens": 72,+                "prompt_tokens": 584,+                "prompt_tokens_details": {+                    "cached_tokens": 512+                },+                "total_tokens": 631+            }+        }+    }+]

@@ -0,0 +1,367 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                }+            ],+            "model": "deepseek-v4-flash",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "I'll check the current weather in London for you.",+                    "type": "text"+                },+                {+                    "arguments": {+                        "location": "London"+                    },+                    "id": "call_00_ET_untwD9piFDtwetSJEIiB1629",+                    "name": "get_weather",+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "I'll check the current weather in London for you.",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 352,+                "output_tokens": 49+            }+        },+        "sse": "data: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null,\"reasoning_content\":\"\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"I\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"'ll\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" check\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" the\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" current\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" weather\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" in\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" London\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" for\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" you\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\".\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_00_ET_untwD9piFDtwetSJEIiB1629\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"location\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\": \"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"London\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"bf126b7a-907f-4430-9507-c4d94db6891a\",\"object\":\"chat.completion.chunk\",\"created\":1791831612,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":352,\"completion_tokens\":49,\"total_tokens\":401,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0},\"prompt_cache_hit_tokens\":0,\"prompt_cache_miss_tokens\":352}}\n\ndata: [DONE]\n\n"+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": "I'll check the current weather in London for you.",+                    "reasoning_content": "",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_ET_untwD9piFDtwetSJEIiB1629",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_00_ET_untwD9piFDtwetSJEIiB1629"+                }+            ],+            "model": "deepseek-v4-flash",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "Here's the current weather in London:\n\n- **Conditions:** Partly cloudy\n- **Temperature:** 18°C (about 64°F)\n- **Wind:** Light breeze from the west\n\nFairly mild and pleasant — a good day to be out and about, though you might want a light layer just in case. Let me know if you'd like the forecast for another city!",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "Here's the current weather in London:\n\n- **Conditions:** Partly cloudy\n- **Temperature:** 18°C (about 64°F)\n- **Wind:** Light breeze from the west\n\nFairly mild and pleasant — a good day to be out and about, though you might want a light layer just in case. Let me know if you'd like the forecast for another city!",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 256,+                "input_tokens": 430,+                "output_tokens": 81+            }+        },+        "sse": "data: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null,\"reasoning_content\":\"\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Here\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"'s\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" the\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" current\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" weather\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" in\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" London\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":\\n\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"-\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" **\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Conditions\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":**\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Part\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"ly\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" cloudy\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"-\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" **\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Temperature\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":**\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" \",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"18\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"°\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"C\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" (\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"about\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" \",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"64\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"°\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"F\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\")\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"-\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" **\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Wind\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":**\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Light\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" breeze\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" from\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" the\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" west\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\\n\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Fair\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"ly\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" mild\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" and\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" pleasant\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" —\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" a\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" good\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" day\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" to\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" be\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" out\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" and\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" about\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\",\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" though\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" you\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" might\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" want\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" a\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" light\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" layer\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" just\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" in\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" case\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\".\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Let\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" me\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" know\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" if\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" you\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"'d\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" like\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" the\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" forecast\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" for\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" another\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" city\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"!\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"07fdc257-1774-4c36-a6ff-fcf2de1d950c\",\"object\":\"chat.completion.chunk\",\"created\":1791831613,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":430,\"completion_tokens\":81,\"total_tokens\":511,\"prompt_tokens_details\":{\"cached_tokens\":256},\"completion_tokens_details\":{\"reasoning_tokens\":0},\"prompt_cache_hit_tokens\":256,\"prompt_cache_miss_tokens\":174}}\n\ndata: [DONE]\n\n"+    },+    {+        "prompt": "and in paris?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": "I'll check the current weather in London for you.",+                    "reasoning_content": "",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_ET_untwD9piFDtwetSJEIiB1629",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_00_ET_untwD9piFDtwetSJEIiB1629"+                },+                {+                    "content": "Here's the current weather in London:\n\n- **Conditions:** Partly cloudy\n- **Temperature:** 18°C (about 64°F)\n- **Wind:** Light breeze from the west\n\nFairly mild and pleasant — a good day to be out and about, though you might want a light layer just in case. Let me know if you'd like the forecast for another city!",+                    "reasoning_content": "",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                }+            ],+            "model": "deepseek-v4-flash",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The user wants weather in Paris. Call the tool.",+                    "type": "thinking"+                },+                {+                    "arguments": {+                        "location": "Paris"+                    },+                    "id": "call_00_e8BJvsWyP3oJO3v2tKQx8381",+                    "name": "get_weather",+                    "type": "tool_call"+                }+            ],+            "reasoning": "The user wants weather in Paris. Call the tool.",+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 384,+                "input_tokens": 520,+                "output_tokens": 50+            }+        },+        "sse": "data: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null,\"reasoning_content\":\"\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\"The\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" user\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" wants\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" weather\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" in\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" Paris\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\".\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" Call\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" the\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\" tool\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":null,\"reasoning_content\":\".\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_00_e8BJvsWyP3oJO3v2tKQx8381\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"location\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\": \"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"Paris\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"d68e6d85-0ef2-44d4-bfe2-5955338a7ad9\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":520,\"completion_tokens\":50,\"total_tokens\":570,\"prompt_tokens_details\":{\"cached_tokens\":384},\"completion_tokens_details\":{\"reasoning_tokens\":11},\"prompt_cache_hit_tokens\":384,\"prompt_cache_miss_tokens\":136}}\n\ndata: [DONE]\n\n"+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "content": "I'll check the current weather in London for you.",+                    "reasoning_content": "",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_ET_untwD9piFDtwetSJEIiB1629",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_00_ET_untwD9piFDtwetSJEIiB1629"+                },+                {+                    "content": "Here's the current weather in London:\n\n- **Conditions:** Partly cloudy\n- **Temperature:** 18°C (about 64°F)\n- **Wind:** Light breeze from the west\n\nFairly mild and pleasant — a good day to be out and about, though you might want a light layer just in case. Let me know if you'd like the forecast for another city!",+                    "reasoning_content": "",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                },+                {+                    "reasoning_content": "The user wants weather in Paris. Call the tool.",+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"Paris\"}",+                                "name": "get_weather"+                            },+                            "id": "call_00_e8BJvsWyP3oJO3v2tKQx8381",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in Paris is sunny, 23°C, no wind.",+                    "role": "tool",+                    "tool_call_id": "call_00_e8BJvsWyP3oJO3v2tKQx8381"+                }+            ],+            "model": "deepseek-v4-flash",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "thinking": {+                "type": "enabled"+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "Here's the current weather in Paris:\n\n- **Conditions:** Sunny\n- **Temperature:** 23°C (about 73°F)\n- **Wind:** None\n\nParis is warmer and brighter than London right now — a great day for a walk along the Seine or a café terrace. Anything else you'd like to check?",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "Here's the current weather in Paris:\n\n- **Conditions:** Sunny\n- **Temperature:** 23°C (about 73°F)\n- **Wind:** None\n\nParis is warmer and brighter than London right now — a great day for a walk along the Seine or a café terrace. Anything else you'd like to check?",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 384,+                "input_tokens": 595,+                "output_tokens": 68+            }+        },+        "sse": "data: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null,\"reasoning_content\":\"\"},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Here\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"'s\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" the\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" current\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" weather\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" in\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Paris\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":\\n\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"-\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" **\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Conditions\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":**\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Sunny\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"-\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" **\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Temperature\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":**\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" \",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"23\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"°\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"C\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" (\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"about\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" \",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"73\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"°\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"F\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\")\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"-\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" **\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Wind\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\":**\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" None\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\\n\\n\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Paris\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" is\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" warmer\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" and\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" brighter\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" than\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" London\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" right\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" now\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" —\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" a\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" great\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" day\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" for\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" a\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" walk\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" along\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" the\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Seine\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" or\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" a\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" café\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" terrace\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\".\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" Anything\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" else\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" you\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"'d\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" like\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" to\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" check\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"?\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":null}],\"usage\":null}\n\ndata: {\"id\":\"3b6fbf1c-520b-4047-bc01-db9637f3b8fd\",\"object\":\"chat.completion.chunk\",\"created\":1791831615,\"model\":\"deepseek-flash\",\"system_fingerprint\":\"aeb56401ca74e127821c4f9126dcb669\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\",\"reasoning_content\":null},\"logprobs\":null,\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":595,\"completion_tokens\":68,\"total_tokens\":663,\"prompt_tokens_details\":{\"cached_tokens\":384},\"completion_tokens_details\":{\"reasoning_tokens\":0},\"prompt_cache_hit_tokens\":384,\"prompt_cache_miss_tokens\":211}}\n\ndata: [DONE]\n\n"+    }+]

@@ -0,0 +1,466 @@+[+  {+    "prompt": "how's the weather in london?",+    "request": {+      "_model": "gemini-2.5-flash",+      "contents": [+        {+          "parts": [+            {+              "text": "how's the weather in london?"+            }+          ],+          "role": "user"+        }+      ],+      "generationConfig": {+        "maxOutputTokens": 1024+      },+      "system_instruction": {+        "parts": [+          {+            "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+          }+        ]+      },+      "tools": [+        {+          "function_declarations": [+            {+              "description": "Get the current weather for a given location",+              "name": "get_weather",+              "parameters": {+                "properties": {+                  "location": {+                    "type": "string"+                  }+                },+                "required": [+                  "location"+                ],+                "type": "object"+              }+            }+          ]+        }+      ]+    },+    "response": {+      "candidates": [+        {+          "content": {+            "parts": [+              {+                "functionCall": {+                  "args": {+                    "location": "london"+                  },+                  "name": "get_weather"+                },+                "thoughtSignature": "CrYCAQw51sfPxRG9vnimuR22EaIvtoyyucxEJBQJ/X4Psewnt2wdz1be27Atf9YdHnB6/PBdAbq5RCk1Vh+08JCYgPOVpXLk9K7Bsou6wpFSyb3kyw7XWFQtw1R6hU5Igo3j/Fx+1R6qWkohByimQSOOFLpTZSsf/op6GzkeCC4vjamvIylc08JpA9/eBmJmt+54uU7JSaIzMVMHTs7Gyy8myV1cuJ4vLozakJf0ZwgC211Bcj4/YECr/z2B+mwGOtRlrNp6x1h/xKB2zbYlfqxi8UBG2VKCVPnFFJhlRwsGtoCyFtPHiyBYAp/g5Oa1g1TH1htteqR7DA8D2plfzcVpz1YP6YArNkCDuIX9Fo8diuDF558XLebgD3tibSLtoNP+mYR7Mz9Grelzx01YtCSlxKI6Y6auZQ=="+              }+            ],+            "role": "model"+          },+          "finishMessage": "Model generated function call(s).",+          "finishReason": "STOP",+          "index": 0+        }+      ],+      "modelVersion": "gemini-2.5-flash",+      "responseId": "MQf_aamWB5qDkdUPxoCDuAg",+      "usageMetadata": {+        "candidatesTokenCount": 15,+        "promptTokenCount": 80,+        "promptTokensDetails": [+          {+            "modality": "TEXT",+            "tokenCount": 80+          }+        ],+        "serviceTier": "standard",+        "thoughtsTokenCount": 67,+        "totalTokenCount": 162+      }+    }+  },+  {+    "request": {+      "_model": "gemini-2.5-flash",+      "contents": [+        {+          "parts": [+            {+              "text": "how's the weather in london?"+            }+          ],+          "role": "user"+        },+        {+          "parts": [+            {+              "functionCall": {+                "args": {+                  "location": "london"+                },+                "name": "get_weather"+              },+              "thoughtSignature": "CrYCAQw51sfPxRG9vnimuR22EaIvtoyyucxEJBQJ/X4Psewnt2wdz1be27Atf9YdHnB6/PBdAbq5RCk1Vh+08JCYgPOVpXLk9K7Bsou6wpFSyb3kyw7XWFQtw1R6hU5Igo3j/Fx+1R6qWkohByimQSOOFLpTZSsf/op6GzkeCC4vjamvIylc08JpA9/eBmJmt+54uU7JSaIzMVMHTs7Gyy8myV1cuJ4vLozakJf0ZwgC211Bcj4/YECr/z2B+mwGOtRlrNp6x1h/xKB2zbYlfqxi8UBG2VKCVPnFFJhlRwsGtoCyFtPHiyBYAp/g5Oa1g1TH1htteqR7DA8D2plfzcVpz1YP6YArNkCDuIX9Fo8diuDF558XLebgD3tibSLtoNP+mYR7Mz9Grelzx01YtCSlxKI6Y6auZQ=="+            }+          ],+          "role": "model"+        },+        {+          "parts": [+            {+              "functionResponse": {+                "name": "get_weather",+                "response": {+                  "result": "Weather in London is partly cloudy, 18°C, light breeze from the west."+                }+              }+            }+          ],+          "role": "user"+        }+      ],+      "generationConfig": {+        "maxOutputTokens": 1024+      },+      "system_instruction": {+        "parts": [+          {+            "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+          }+        ]+      },+      "tools": [+        {+          "function_declarations": [+            {+              "description": "Get the current weather for a given location",+              "name": "get_weather",+              "parameters": {+                "properties": {+                  "location": {+                    "type": "string"+                  }+                },+                "required": [+                  "location"+                ],+                "type": "object"+              }+            }+          ]+        }+      ]+    },+    "response": {+      "candidates": [+        {+          "content": {+            "parts": [+              {+                "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west. \n"+              }+            ],+            "role": "model"+          },+          "finishReason": "STOP",+          "index": 0+        }+      ],+      "modelVersion": "gemini-2.5-flash",+      "responseId": "Mgf_adrRNMW0nsEP0vmmiQ4",+      "usageMetadata": {+        "candidatesTokenCount": 22,+        "promptTokenCount": 128,+        "promptTokensDetails": [+          {+            "modality": "TEXT",+            "tokenCount": 128+          }+        ],+        "serviceTier": "standard",+        "totalTokenCount": 150+      }+    }+  },+  {+    "prompt": "and in paris?",+    "request": {+      "_model": "gemini-2.5-flash",+      "contents": [+        {+          "parts": [+            {+              "text": "how's the weather in london?"+            }+          ],+          "role": "user"+        },+        {+          "parts": [+            {+              "functionCall": {+                "args": {+                  "location": "london"+                },+                "name": "get_weather"+              },+              "thoughtSignature": "CrYCAQw51sfPxRG9vnimuR22EaIvtoyyucxEJBQJ/X4Psewnt2wdz1be27Atf9YdHnB6/PBdAbq5RCk1Vh+08JCYgPOVpXLk9K7Bsou6wpFSyb3kyw7XWFQtw1R6hU5Igo3j/Fx+1R6qWkohByimQSOOFLpTZSsf/op6GzkeCC4vjamvIylc08JpA9/eBmJmt+54uU7JSaIzMVMHTs7Gyy8myV1cuJ4vLozakJf0ZwgC211Bcj4/YECr/z2B+mwGOtRlrNp6x1h/xKB2zbYlfqxi8UBG2VKCVPnFFJhlRwsGtoCyFtPHiyBYAp/g5Oa1g1TH1htteqR7DA8D2plfzcVpz1YP6YArNkCDuIX9Fo8diuDF558XLebgD3tibSLtoNP+mYR7Mz9Grelzx01YtCSlxKI6Y6auZQ=="+            }+          ],+          "role": "model"+        },+        {+          "parts": [+            {+              "functionResponse": {+                "name": "get_weather",+                "response": {+                  "result": "Weather in London is partly cloudy, 18°C, light breeze from the west."+                }+              }+            }+          ],+          "role": "user"+        },+        {+          "parts": [+            {+              "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west. \n"+            }+          ],+          "role": "model"+        },+        {+          "parts": [+            {+              "text": "and in paris?"+            }+          ],+          "role": "user"+        }+      ],+      "generationConfig": {+        "maxOutputTokens": 1024+      },+      "system_instruction": {+        "parts": [+          {+            "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+          }+        ]+      },+      "tools": [+        {+          "function_declarations": [+            {+              "description": "Get the current weather for a given location",+              "name": "get_weather",+              "parameters": {+                "properties": {+                  "location": {+                    "type": "string"+                  }+                },+                "required": [+                  "location"+                ],+                "type": "object"+              }+            }+          ]+        }+      ]+    },+    "response": {+      "candidates": [+        {+          "content": {+            "parts": [+              {+                "functionCall": {+                  "args": {+                    "location": "paris"+                  },+                  "name": "get_weather"+                },+                "thoughtSignature": "CrQBAQw51sevrdIM6Bou7D32/R36TgB5hj/7FN/LL7N1gwPXNi8IJ4I8iTtpuFuP+nroxiSvtr35+QmtIQvPMcaEZZJHYC7shC9rSa2S2xzqQMrlU71f2I7guOxzBvT62Z/RW/M1423X5dcLBKBkJz+n0M219tR/6eJepndHtJLP2nsiPDlhTS4/QulhjP3iMN1GzEg20HhsY1Ri6GMS3BFyu+udYByLXiaCwaNyyquNwdApTNml"+              }+            ],+            "role": "model"+          },+          "finishMessage": "Model generated function call(s).",+          "finishReason": "STOP",+          "index": 0+        }+      ],+      "modelVersion": "gemini-2.5-flash",+      "responseId": "NAf_aaO0FsTknsEP3MvJqA0",+      "usageMetadata": {+        "candidatesTokenCount": 15,+        "promptTokenCount": 158,+        "promptTokensDetails": [+          {+            "modality": "TEXT",+            "tokenCount": 158+          }+        ],+        "serviceTier": "standard",+        "thoughtsTokenCount": 36,+        "totalTokenCount": 209+      }+    }+  },+  {+    "request": {+      "_model": "gemini-2.5-flash",+      "contents": [+        {+          "parts": [+            {+              "text": "how's the weather in london?"+            }+          ],+          "role": "user"+        },+        {+          "parts": [+            {+              "functionCall": {+                "args": {+                  "location": "london"+                },+                "name": "get_weather"+              },+              "thoughtSignature": "CrYCAQw51sfPxRG9vnimuR22EaIvtoyyucxEJBQJ/X4Psewnt2wdz1be27Atf9YdHnB6/PBdAbq5RCk1Vh+08JCYgPOVpXLk9K7Bsou6wpFSyb3kyw7XWFQtw1R6hU5Igo3j/Fx+1R6qWkohByimQSOOFLpTZSsf/op6GzkeCC4vjamvIylc08JpA9/eBmJmt+54uU7JSaIzMVMHTs7Gyy8myV1cuJ4vLozakJf0ZwgC211Bcj4/YECr/z2B+mwGOtRlrNp6x1h/xKB2zbYlfqxi8UBG2VKCVPnFFJhlRwsGtoCyFtPHiyBYAp/g5Oa1g1TH1htteqR7DA8D2plfzcVpz1YP6YArNkCDuIX9Fo8diuDF558XLebgD3tibSLtoNP+mYR7Mz9Grelzx01YtCSlxKI6Y6auZQ=="+            }+          ],+          "role": "model"+        },+        {+          "parts": [+            {+              "functionResponse": {+                "name": "get_weather",+                "response": {+                  "result": "Weather in London is partly cloudy, 18°C, light breeze from the west."+                }+              }+            }+          ],+          "role": "user"+        },+        {+          "parts": [+            {+              "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west. \n"+            }+          ],+          "role": "model"+        },+        {+          "parts": [+            {+              "text": "and in paris?"+            }+          ],+          "role": "user"+        },+        {+          "parts": [+            {+              "functionCall": {+                "args": {+                  "location": "paris"+                },+                "name": "get_weather"+              },+              "thoughtSignature": "CrQBAQw51sevrdIM6Bou7D32/R36TgB5hj/7FN/LL7N1gwPXNi8IJ4I8iTtpuFuP+nroxiSvtr35+QmtIQvPMcaEZZJHYC7shC9rSa2S2xzqQMrlU71f2I7guOxzBvT62Z/RW/M1423X5dcLBKBkJz+n0M219tR/6eJepndHtJLP2nsiPDlhTS4/QulhjP3iMN1GzEg20HhsY1Ri6GMS3BFyu+udYByLXiaCwaNyyquNwdApTNml"+            }+          ],+          "role": "model"+        },+        {+          "parts": [+            {+              "functionResponse": {+                "name": "get_weather",+                "response": {+                  "result": "Weather in Paris is sunny, 23°C, no wind."+                }+              }+            }+          ],+          "role": "user"+        }+      ],+      "generationConfig": {+        "maxOutputTokens": 1024+      },+      "system_instruction": {+        "parts": [+          {+            "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+          }+        ]+      },+      "tools": [+        {+          "function_declarations": [+            {+              "description": "Get the current weather for a given location",+              "name": "get_weather",+              "parameters": {+                "properties": {+                  "location": {+                    "type": "string"+                  }+                },+                "required": [+                  "location"+                ],+                "type": "object"+              }+            }+          ]+        }+      ]+    },+    "response": {+      "candidates": [+        {+          "content": {+            "parts": [+              {+                "text": "The weather in Paris is sunny, 23°C, with no wind.",+                "thoughtSignature": "CqgCAQw51seXOw7QjuKe6rjxSd4E2U2OFa1doN8TM5udn1WvUSWsA1vIVh7IoJf5CcegjMFjQMv2lpofnTtuSAWUwJcQpBbL9VR629g50rsReSvUqWXVTxahct+x/VthG9sP0VG2P2dxg037Ydzz7v1pBN4tXJUBM0qlA/Up3wG2mlhZf7cIQesxPoqBXww3t/XmUzvbjZmVlzCns6/gKPGcEVGkhu6P4F7jjf5V4MomL41paTZ4OwrK871YFKBzpWCbgzUnUBAMPWutw4D6DEJGhS6DcW6r/QqI0fvHXKTeGPkuU0iw0lje0+6i0sBwLn1qcej9yKjh4eJl1UMb1SNGnUmS1SPx16+NKkrcLrKRWLlZ31SvfEdrxyMbOTB89ware1RExK9E8hQ="+              }+            ],+            "role": "model"+          },+          "finishReason": "STOP",+          "index": 0+        }+      ],+      "modelVersion": "gemini-2.5-flash",+      "responseId": "Ngf_ac2NDPPknsEPo5al8Q8",+      "usageMetadata": {+        "candidatesTokenCount": 17,+        "promptTokenCount": 202,+        "promptTokensDetails": [+          {+            "modality": "TEXT",+            "tokenCount": 202+          }+        ],+        "serviceTier": "standard",+        "thoughtsTokenCount": 60,+        "totalTokenCount": 279+      }+    }+  }+]

@@ -0,0 +1,420 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "_model": "gemini-2.5-flash",+            "contents": [+                {+                    "parts": [+                        {+                            "text": "how's the weather in london?"+                        }+                    ],+                    "role": "user"+                }+            ],+            "generationConfig": {+                "maxOutputTokens": 1024+            },+            "system_instruction": {+                "parts": [+                    {+                        "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+                    }+                ]+            },+            "tools": [+                {+                    "function_declarations": [+                        {+                            "description": "Get the current weather for a given location",+                            "name": "get_weather",+                            "parameters": {+                                "properties": {+                                    "location": {+                                        "type": "string"+                                    }+                                },+                                "required": [+                                    "location"+                                ],+                                "type": "object"+                            }+                        }+                    ]+                }+            ]+        },+        "response": {+            "content": [+                {+                    "arguments": {+                        "location": "london"+                    },+                    "id": "call_3",+                    "name": "get_weather",+                    "provider_meta": {+                        "model": "gemini-2.5-flash",+                        "payload": {+                            "thoughtSignature": "CiQBaRR9E1bQKFDb1IdCRw/EwdzLL8d3nA9HW1uaE6faGtp/yJkKbgFpFH0THbbSxOnGPDEv9ClVdoqXwmLCFArKL60PLMXEWPW7sNdcYw8NpJ+AYt2PrSUqTUa3/CycUwxkaatMAZH+XwtkGk9fWK82f1kIyp/849p37fZlAGNVYetP7OoHqwzVtMfSVS92pTGO7Zy7CskBAWkUfRMyfjq19zfhRokIcvIu7DY+jDwoabfNpANlgtBuXPnDnKlCy2kvjMd9lQCAF9UC4Awi9r9NClqa7D4W5tbKdNswYMlvXfTbcRRcuSJODFQKgLqXtLAS0NLLs1j96KjQrGJKWeGHvUxiQ1NTIGnOk0DimDdsZ9sSm94RYNkPmCsQSbOZJH5/8GO20HhJA90WsqRtufz8KFenR9b+N1yBpEwGVNdzRWvQYNQ40z14yJ8ue0ZHTJFCGL1/k1Rcq798v6ZhKdHR"+                        },+                        "provider": "gemini"+                    },+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 80,+                "output_tokens": 15+            }+        },+        "sse": "data: {\"candidates\": [{\"content\": {\"parts\": [{\"functionCall\": {\"name\": \"get_weather\",\"args\": {\"location\": \"london\"}},\"thoughtSignature\": \"CiQBaRR9E1bQKFDb1IdCRw/EwdzLL8d3nA9HW1uaE6faGtp/yJkKbgFpFH0THbbSxOnGPDEv9ClVdoqXwmLCFArKL60PLMXEWPW7sNdcYw8NpJ+AYt2PrSUqTUa3/CycUwxkaatMAZH+XwtkGk9fWK82f1kIyp/849p37fZlAGNVYetP7OoHqwzVtMfSVS92pTGO7Zy7CskBAWkUfRMyfjq19zfhRokIcvIu7DY+jDwoabfNpANlgtBuXPnDnKlCy2kvjMd9lQCAF9UC4Awi9r9NClqa7D4W5tbKdNswYMlvXfTbcRRcuSJODFQKgLqXtLAS0NLLs1j96KjQrGJKWeGHvUxiQ1NTIGnOk0DimDdsZ9sSm94RYNkPmCsQSbOZJH5/8GO20HhJA90WsqRtufz8KFenR9b+N1yBpEwGVNdzRWvQYNQ40z14yJ8ue0ZHTJFCGL1/k1Rcq798v6ZhKdHR\"}],\"role\": \"model\"},\"finishReason\": \"STOP\",\"index\": 0,\"finishMessage\": \"Model generated function call(s).\"}],\"usageMetadata\": {\"promptTokenCount\": 80,\"candidatesTokenCount\": 15,\"totalTokenCount\": 154,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 80}],\"thoughtsTokenCount\": 59,\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"OS7Navm-Dv7_vdIPqLDGgAk\"}\r\n\r\n"+    },+    {+        "prompt": null,+        "request": {+            "_model": "gemini-2.5-flash",+            "contents": [+                {+                    "parts": [+                        {+                            "text": "how's the weather in london?"+                        }+                    ],+                    "role": "user"+                },+                {+                    "parts": [+                        {+                            "functionCall": {+                                "args": {+                                    "location": "london"+                                },+                                "name": "get_weather"+                            },+                            "thoughtSignature": "CiQBaRR9E1bQKFDb1IdCRw/EwdzLL8d3nA9HW1uaE6faGtp/yJkKbgFpFH0THbbSxOnGPDEv9ClVdoqXwmLCFArKL60PLMXEWPW7sNdcYw8NpJ+AYt2PrSUqTUa3/CycUwxkaatMAZH+XwtkGk9fWK82f1kIyp/849p37fZlAGNVYetP7OoHqwzVtMfSVS92pTGO7Zy7CskBAWkUfRMyfjq19zfhRokIcvIu7DY+jDwoabfNpANlgtBuXPnDnKlCy2kvjMd9lQCAF9UC4Awi9r9NClqa7D4W5tbKdNswYMlvXfTbcRRcuSJODFQKgLqXtLAS0NLLs1j96KjQrGJKWeGHvUxiQ1NTIGnOk0DimDdsZ9sSm94RYNkPmCsQSbOZJH5/8GO20HhJA90WsqRtufz8KFenR9b+N1yBpEwGVNdzRWvQYNQ40z14yJ8ue0ZHTJFCGL1/k1Rcq798v6ZhKdHR"+                        }+                    ],+                    "role": "model"+                },+                {+                    "parts": [+                        {+                            "functionResponse": {+                                "name": "get_weather",+                                "response": {+                                    "result": "Weather in London is partly cloudy, 18°C, light breeze from the west."+                                }+                            }+                        }+                    ],+                    "role": "user"+                }+            ],+            "generationConfig": {+                "maxOutputTokens": 1024+            },+            "system_instruction": {+                "parts": [+                    {+                        "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+                    }+                ]+            },+            "tools": [+                {+                    "function_declarations": [+                        {+                            "description": "Get the current weather for a given location",+                            "name": "get_weather",+                            "parameters": {+                                "properties": {+                                    "location": {+                                        "type": "string"+                                    }+                                },+                                "required": [+                                    "location"+                                ],+                                "type": "object"+                            }+                        }+                    ]+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west.",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west.",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 187,+                "output_tokens": 22+            }+        },+        "sse": "data: {\"candidates\": [{\"content\": {\"parts\": [{\"text\": \"The weather in London\"}],\"role\": \"model\"},\"index\": 0}],\"usageMetadata\": {\"promptTokenCount\": 187,\"candidatesTokenCount\": 4,\"totalTokenCount\": 191,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 187}],\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"Oi7NaujWAa2ExN8P89zcgQg\"}\r\n\r\ndata: {\"candidates\": [{\"content\": {\"parts\": [{\"text\": \" is partly cloudy, 18°C, with a light breeze from the west.\"}],\"role\": \"model\"},\"finishReason\": \"STOP\",\"index\": 0}],\"usageMetadata\": {\"promptTokenCount\": 187,\"candidatesTokenCount\": 22,\"totalTokenCount\": 209,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 187}],\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"Oi7NaujWAa2ExN8P89zcgQg\"}\r\n\r\n"+    },+    {+        "prompt": "and in paris?",+        "request": {+            "_model": "gemini-2.5-flash",+            "contents": [+                {+                    "parts": [+                        {+                            "text": "how's the weather in london?"+                        }+                    ],+                    "role": "user"+                },+                {+                    "parts": [+                        {+                            "functionCall": {+                                "args": {+                                    "location": "london"+                                },+                                "name": "get_weather"+                            },+                            "thoughtSignature": "CiQBaRR9E1bQKFDb1IdCRw/EwdzLL8d3nA9HW1uaE6faGtp/yJkKbgFpFH0THbbSxOnGPDEv9ClVdoqXwmLCFArKL60PLMXEWPW7sNdcYw8NpJ+AYt2PrSUqTUa3/CycUwxkaatMAZH+XwtkGk9fWK82f1kIyp/849p37fZlAGNVYetP7OoHqwzVtMfSVS92pTGO7Zy7CskBAWkUfRMyfjq19zfhRokIcvIu7DY+jDwoabfNpANlgtBuXPnDnKlCy2kvjMd9lQCAF9UC4Awi9r9NClqa7D4W5tbKdNswYMlvXfTbcRRcuSJODFQKgLqXtLAS0NLLs1j96KjQrGJKWeGHvUxiQ1NTIGnOk0DimDdsZ9sSm94RYNkPmCsQSbOZJH5/8GO20HhJA90WsqRtufz8KFenR9b+N1yBpEwGVNdzRWvQYNQ40z14yJ8ue0ZHTJFCGL1/k1Rcq798v6ZhKdHR"+                        }+                    ],+                    "role": "model"+                },+                {+                    "parts": [+                        {+                            "functionResponse": {+                                "name": "get_weather",+                                "response": {+                                    "result": "Weather in London is partly cloudy, 18°C, light breeze from the west."+                                }+                            }+                        }+                    ],+                    "role": "user"+                },+                {+                    "parts": [+                        {+                            "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west."+                        }+                    ],+                    "role": "model"+                },+                {+                    "parts": [+                        {+                            "text": "and in paris?"+                        }+                    ],+                    "role": "user"+                }+            ],+            "generationConfig": {+                "maxOutputTokens": 1024+            },+            "system_instruction": {+                "parts": [+                    {+                        "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+                    }+                ]+            },+            "tools": [+                {+                    "function_declarations": [+                        {+                            "description": "Get the current weather for a given location",+                            "name": "get_weather",+                            "parameters": {+                                "properties": {+                                    "location": {+                                        "type": "string"+                                    }+                                },+                                "required": [+                                    "location"+                                ],+                                "type": "object"+                            }+                        }+                    ]+                }+            ]+        },+        "response": {+            "content": [+                {+                    "arguments": {+                        "location": "paris"+                    },+                    "id": "call_8",+                    "name": "get_weather",+                    "provider_meta": {+                        "model": "gemini-2.5-flash",+                        "payload": {+                            "thoughtSignature": "CiQBaRR9EyXvkCduORckP6Gd3AwN7JRlQTOFepCIqTV3p1bf5YoKYQFpFH0T6odim/izeA1QBHQ4ooFrwCGXlJ8JzWewHhM6MS5qBnQjM9bUB2pJeoVGWC6wqJTNHp0kh2QP2P9kBvUWsccL/6IAgVZmVR9qOeWhun5Ye7XIo66bHi6ozVag5+sKMAFpFH0TFr+bKrK6F+Q0knAs07pXDXdHPATItqO1PJBM9pvJ9Y/mVOajK8JmoYv5Pgp3AWkUfRPFRL2DVEVug3eX8/UCTche9nghcBzrDhj0FLlEa+vfDqLiV7+xtEjv5ICphg3cq39asZClLQ/qAqIHzNo1XVrpxCQKaLVVBkB3YXJohSex+FQYMsowW1zOwKUycx2akiiaaWR2bCE44y4zJ8YW453nccMKVwFpFH0T4bYsGa/XUdVDKILzrvMYuqgai6XQ3yoOU6fwGnkWOaoqUNTXWOdCsxqE22HL1YsnBzuDlk9yc8QOBfcVcA3kmrJkZbzpdwrVdOLIp6x9FWN23Q=="+                        },+                        "provider": "gemini"+                    },+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 156,+                "output_tokens": 15+            }+        },+        "sse": "data: {\"candidates\": [{\"content\": {\"parts\": [{\"functionCall\": {\"name\": \"get_weather\",\"args\": {\"location\": \"paris\"}},\"thoughtSignature\": \"CiQBaRR9EyXvkCduORckP6Gd3AwN7JRlQTOFepCIqTV3p1bf5YoKYQFpFH0T6odim/izeA1QBHQ4ooFrwCGXlJ8JzWewHhM6MS5qBnQjM9bUB2pJeoVGWC6wqJTNHp0kh2QP2P9kBvUWsccL/6IAgVZmVR9qOeWhun5Ye7XIo66bHi6ozVag5+sKMAFpFH0TFr+bKrK6F+Q0knAs07pXDXdHPATItqO1PJBM9pvJ9Y/mVOajK8JmoYv5Pgp3AWkUfRPFRL2DVEVug3eX8/UCTche9nghcBzrDhj0FLlEa+vfDqLiV7+xtEjv5ICphg3cq39asZClLQ/qAqIHzNo1XVrpxCQKaLVVBkB3YXJohSex+FQYMsowW1zOwKUycx2akiiaaWR2bCE44y4zJ8YW453nccMKVwFpFH0T4bYsGa/XUdVDKILzrvMYuqgai6XQ3yoOU6fwGnkWOaoqUNTXWOdCsxqE22HL1YsnBzuDlk9yc8QOBfcVcA3kmrJkZbzpdwrVdOLIp6x9FWN23Q==\"}],\"role\": \"model\"},\"finishReason\": \"STOP\",\"index\": 0,\"finishMessage\": \"Model generated function call(s).\"}],\"usageMetadata\": {\"promptTokenCount\": 156,\"candidatesTokenCount\": 15,\"totalTokenCount\": 223,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 156}],\"thoughtsTokenCount\": 52,\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"Oi7NauCwGuTR28oPquKmuAk\"}\r\n\r\n"+    },+    {+        "prompt": null,+        "request": {+            "_model": "gemini-2.5-flash",+            "contents": [+                {+                    "parts": [+                        {+                            "text": "how's the weather in london?"+                        }+                    ],+                    "role": "user"+                },+                {+                    "parts": [+                        {+                            "functionCall": {+                                "args": {+                                    "location": "london"+                                },+                                "name": "get_weather"+                            },+                            "thoughtSignature": "CiQBaRR9E1bQKFDb1IdCRw/EwdzLL8d3nA9HW1uaE6faGtp/yJkKbgFpFH0THbbSxOnGPDEv9ClVdoqXwmLCFArKL60PLMXEWPW7sNdcYw8NpJ+AYt2PrSUqTUa3/CycUwxkaatMAZH+XwtkGk9fWK82f1kIyp/849p37fZlAGNVYetP7OoHqwzVtMfSVS92pTGO7Zy7CskBAWkUfRMyfjq19zfhRokIcvIu7DY+jDwoabfNpANlgtBuXPnDnKlCy2kvjMd9lQCAF9UC4Awi9r9NClqa7D4W5tbKdNswYMlvXfTbcRRcuSJODFQKgLqXtLAS0NLLs1j96KjQrGJKWeGHvUxiQ1NTIGnOk0DimDdsZ9sSm94RYNkPmCsQSbOZJH5/8GO20HhJA90WsqRtufz8KFenR9b+N1yBpEwGVNdzRWvQYNQ40z14yJ8ue0ZHTJFCGL1/k1Rcq798v6ZhKdHR"+                        }+                    ],+                    "role": "model"+                },+                {+                    "parts": [+                        {+                            "functionResponse": {+                                "name": "get_weather",+                                "response": {+                                    "result": "Weather in London is partly cloudy, 18°C, light breeze from the west."+                                }+                            }+                        }+                    ],+                    "role": "user"+                },+                {+                    "parts": [+                        {+                            "text": "The weather in London is partly cloudy, 18°C, with a light breeze from the west."+                        }+                    ],+                    "role": "model"+                },+                {+                    "parts": [+                        {+                            "text": "and in paris?"+                        }+                    ],+                    "role": "user"+                },+                {+                    "parts": [+                        {+                            "functionCall": {+                                "args": {+                                    "location": "paris"+                                },+                                "name": "get_weather"+                            },+                            "thoughtSignature": "CiQBaRR9EyXvkCduORckP6Gd3AwN7JRlQTOFepCIqTV3p1bf5YoKYQFpFH0T6odim/izeA1QBHQ4ooFrwCGXlJ8JzWewHhM6MS5qBnQjM9bUB2pJeoVGWC6wqJTNHp0kh2QP2P9kBvUWsccL/6IAgVZmVR9qOeWhun5Ye7XIo66bHi6ozVag5+sKMAFpFH0TFr+bKrK6F+Q0knAs07pXDXdHPATItqO1PJBM9pvJ9Y/mVOajK8JmoYv5Pgp3AWkUfRPFRL2DVEVug3eX8/UCTche9nghcBzrDhj0FLlEa+vfDqLiV7+xtEjv5ICphg3cq39asZClLQ/qAqIHzNo1XVrpxCQKaLVVBkB3YXJohSex+FQYMsowW1zOwKUycx2akiiaaWR2bCE44y4zJ8YW453nccMKVwFpFH0T4bYsGa/XUdVDKILzrvMYuqgai6XQ3yoOU6fwGnkWOaoqUNTXWOdCsxqE22HL1YsnBzuDlk9yc8QOBfcVcA3kmrJkZbzpdwrVdOLIp6x9FWN23Q=="+                        }+                    ],+                    "role": "model"+                },+                {+                    "parts": [+                        {+                            "functionResponse": {+                                "name": "get_weather",+                                "response": {+                                    "result": "Weather in Paris is sunny, 23°C, no wind."+                                }+                            }+                        }+                    ],+                    "role": "user"+                }+            ],+            "generationConfig": {+                "maxOutputTokens": 1024+            },+            "system_instruction": {+                "parts": [+                    {+                        "text": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available."+                    }+                ]+            },+            "tools": [+                {+                    "function_declarations": [+                        {+                            "description": "Get the current weather for a given location",+                            "name": "get_weather",+                            "parameters": {+                                "properties": {+                                    "location": {+                                        "type": "string"+                                    }+                                },+                                "required": [+                                    "location"+                                ],+                                "type": "object"+                            }+                        }+                    ]+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "The weather in Paris is sunny, 23°C, with no wind.",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "The weather in Paris is sunny, 23°C, with no wind.",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 252,+                "output_tokens": 16+            }+        },+        "sse": "data: {\"candidates\": [{\"content\": {\"parts\": [{\"text\": \"The weather in Paris\"}],\"role\": \"model\"},\"index\": 0}],\"usageMetadata\": {\"promptTokenCount\": 252,\"candidatesTokenCount\": 4,\"totalTokenCount\": 256,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 252}],\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"Oy7NauqyJtzAxN8Pho7W0A0\"}\r\n\r\ndata: {\"candidates\": [{\"content\": {\"parts\": [{\"text\": \" is sunny, \"}],\"role\": \"model\"},\"index\": 0}],\"usageMetadata\": {\"promptTokenCount\": 252,\"candidatesTokenCount\": 7,\"totalTokenCount\": 259,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 252}],\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"Oy7NauqyJtzAxN8Pho7W0A0\"}\r\n\r\ndata: {\"candidates\": [{\"content\": {\"parts\": [{\"text\": \"23°C, with no wind.\"}],\"role\": \"model\"},\"finishReason\": \"STOP\",\"index\": 0}],\"usageMetadata\": {\"promptTokenCount\": 252,\"candidatesTokenCount\": 16,\"totalTokenCount\": 268,\"promptTokensDetails\": [{\"modality\": \"TEXT\",\"tokenCount\": 252}],\"serviceTier\": \"standard\"},\"modelVersion\": \"gemini-2.5-flash\",\"responseId\": \"Oy7NauqyJtzAxN8Pho7W0A0\"}\r\n\r\n"+    }+]

@@ -0,0 +1,20 @@+{+    "candidates": [+        {+            "content": {+                "parts": [+                    {+                        "text": "Hello! How can I help you today?"+                    }+                ],+                "role": "model"+            },+            "finishReason": "STOP"+        }+    ],+    "usageMetadata": {+        "promptTokenCount": 20,+        "candidatesTokenCount": 8,+        "totalTokenCount": 28+    }+}

@@ -0,0 +1,23 @@+{+    "candidates": [+        {+            "content": {+                "parts": [+                    {+                        "functionCall": {+                            "name": "get_weather",+                            "args": { "location": "London" }+                        }+                    }+                ],+                "role": "model"+            },+            "finishReason": "STOP"+        }+    ],+    "usageMetadata": {+        "promptTokenCount": 40,+        "candidatesTokenCount": 12,+        "totalTokenCount": 52+    }+}

@@ -0,0 +1,32 @@+[+    {+        "modelConfigName": "llama_3_2",+        "providerName": "ollama",+        "modelName": "llama3.2:latest",+        "pricing": {+            "pricePerMillionInput": 0.0,+            "pricePerMillionOutput": 0.0+        },+        "maxTokens": 1024,+        "temperature": 0.5,+        "requestTimeout": 3000,+        "throttleDelay": 1000,+        "retryCount": 3,+        "jitterBackoff": 1000+    },+    {+        "modelConfigName": "mistral",+        "providerName": "ollama",+        "modelName": "mistral:latest",+        "pricing": {+            "pricePerMillionInput": 0.0,+            "pricePerMillionOutput": 0.0+        },+        "maxTokens": 1024,+        "temperature": 0.5,+        "requestTimeout": 3000,+        "throttleDelay": 1000,+        "retryCount": 3,+        "jitterBackoff": 1000+    }+]

@@ -0,0 +1,360 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                }+            ],+            "model": "llama3.2:latest",+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "tool_calls",+                    "index": 0,+                    "message": {+                        "content": "",+                        "role": "assistant",+                        "tool_calls": [+                            {+                                "function": {+                                    "arguments": "{\"location\":\"London\"}",+                                    "name": "get_weather"+                                },+                                "id": "call_4ysdmo61",+                                "index": 0,+                                "type": "function"+                            }+                        ]+                    }+                }+            ],+            "created": 1792089544,+            "id": "chatcmpl-279",+            "model": "llama3.2:latest",+            "object": "chat.completion",+            "system_fingerprint": "fp_ollama",+            "usage": {+                "completion_tokens": 17,+                "prompt_tokens": 188,+                "total_tokens": 205+            }+        },+        "sse": null+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_4ysdmo61",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_4ysdmo61"+                }+            ],+            "model": "llama3.2:latest",+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "stop",+                    "index": 0,+                    "message": {+                        "content": "The current weather conditions in London are mainly sunny with some cloudiness, and the temperature is around 18 degrees Celsius (64°F). There's a gentle breeze coming from the west at about 19 km/h (12 mph). Would you like to know more or check the forecast for a specific date?",+                        "role": "assistant"+                    }+                }+            ],+            "created": 1792089545,+            "id": "chatcmpl-901",+            "model": "llama3.2:latest",+            "object": "chat.completion",+            "system_fingerprint": "fp_ollama",+            "usage": {+                "completion_tokens": 62,+                "prompt_tokens": 137,+                "total_tokens": 199+            }+        },+        "sse": null+    },+    {+        "prompt": "and in paris?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_4ysdmo61",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_4ysdmo61"+                },+                {+                    "content": "The current weather conditions in London are mainly sunny with some cloudiness, and the temperature is around 18 degrees Celsius (64°F). There's a gentle breeze coming from the west at about 19 km/h (12 mph). Would you like to know more or check the forecast for a specific date?",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                }+            ],+            "model": "llama3.2:latest",+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "tool_calls",+                    "index": 0,+                    "message": {+                        "content": "",+                        "role": "assistant",+                        "tool_calls": [+                            {+                                "function": {+                                    "arguments": "{\"location\":\"Paris\"}",+                                    "name": "get_weather"+                                },+                                "id": "call_sahmucf9",+                                "index": 0,+                                "type": "function"+                            }+                        ]+                    }+                }+            ],+            "created": 1792089546,+            "id": "chatcmpl-768",+            "model": "llama3.2:latest",+            "object": "chat.completion",+            "system_fingerprint": "fp_ollama",+            "usage": {+                "completion_tokens": 17,+                "prompt_tokens": 306,+                "total_tokens": 323+            }+        },+        "sse": null+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_4ysdmo61",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_4ysdmo61"+                },+                {+                    "content": "The current weather conditions in London are mainly sunny with some cloudiness, and the temperature is around 18 degrees Celsius (64°F). There's a gentle breeze coming from the west at about 19 km/h (12 mph). Would you like to know more or check the forecast for a specific date?",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"Paris\"}",+                                "name": "get_weather"+                            },+                            "id": "call_sahmucf9",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in Paris is sunny, 23°C, no wind.",+                    "role": "tool",+                    "tool_call_id": "call_sahmucf9"+                }+            ],+            "model": "llama3.2:latest",+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "choices": [+                {+                    "finish_reason": "stop",+                    "index": 0,+                    "message": {+                        "content": "The current weather conditions in Paris are perfect for a warm day outdoors! It's a beautiful, sunny day with a high temperature of 23 degrees Celsius (73°F) and not a single cloud in the sky. There's no wind to speak of, making it an ideal day to enjoy the city's famous landmarks, cafes, or simply stroll along the Seine River.",+                        "role": "assistant"+                    }+                }+            ],+            "created": 1792089548,+            "id": "chatcmpl-34",+            "model": "llama3.2:latest",+            "object": "chat.completion",+            "system_fingerprint": "fp_ollama",+            "usage": {+                "completion_tokens": 76,+                "prompt_tokens": 251,+                "total_tokens": 327+            }+        },+        "sse": null+    }+]

@@ -0,0 +1,338 @@+[+    {+        "prompt": "how's the weather in london?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                }+            ],+            "model": "llama3.2:latest",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "arguments": {+                        "location": "London"+                    },+                    "id": "call_wucefmo8",+                    "name": "get_weather",+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 188,+                "output_tokens": 17+            }+        },+        "sse": "data: {\"id\":\"chatcmpl-793\",\"object\":\"chat.completion.chunk\",\"created\":1792089548,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"id\":\"call_wucefmo8\",\"index\":0,\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"location\\\":\\\"London\\\"}\"}}]},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-793\",\"object\":\"chat.completion.chunk\",\"created\":1792089548,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":\"tool_calls\"}]}\n\ndata: {\"id\":\"chatcmpl-793\",\"object\":\"chat.completion.chunk\",\"created\":1792089548,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[],\"usage\":{\"prompt_tokens\":188,\"completion_tokens\":17,\"total_tokens\":205}}\n\ndata: [DONE]\n\n"+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_wucefmo8",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_wucefmo8"+                }+            ],+            "model": "llama3.2:latest",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "Currently, the weather in London is partly cloudy with a temperature of 18°C and a light breeze blowing from the west.",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "Currently, the weather in London is partly cloudy with a temperature of 18°C and a light breeze blowing from the west.",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 137,+                "output_tokens": 26+            }+        },+        "sse": "data: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089548,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"Currently\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089548,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\",\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089548,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" the\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" weather\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" in\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" London\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" is\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" partly\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" cloudy\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" with\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" a\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" temperature\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" of\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" \"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"18\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"°C\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" and\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" a\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" light\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" breeze\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" blowing\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" from\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" the\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" west\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\".\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":\"stop\"}]}\n\ndata: {\"id\":\"chatcmpl-502\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[],\"usage\":{\"prompt_tokens\":137,\"completion_tokens\":26,\"total_tokens\":163}}\n\ndata: [DONE]\n\n"+    },+    {+        "prompt": "and in paris?",+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_wucefmo8",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_wucefmo8"+                },+                {+                    "content": "Currently, the weather in London is partly cloudy with a temperature of 18°C and a light breeze blowing from the west.",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                }+            ],+            "model": "llama3.2:latest",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "arguments": {+                        "location": "Paris"+                    },+                    "id": "call_9243whne",+                    "name": "get_weather",+                    "type": "tool_call"+                }+            ],+            "reasoning": null,+            "text": "",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 270,+                "output_tokens": 17+            }+        },+        "sse": "data: {\"id\":\"chatcmpl-72\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"id\":\"call_9243whne\",\"index\":0,\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"location\\\":\\\"Paris\\\"}\"}}]},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-72\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":\"tool_calls\"}]}\n\ndata: {\"id\":\"chatcmpl-72\",\"object\":\"chat.completion.chunk\",\"created\":1792089549,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[],\"usage\":{\"prompt_tokens\":270,\"completion_tokens\":17,\"total_tokens\":287}}\n\ndata: [DONE]\n\n"+    },+    {+        "prompt": null,+        "request": {+            "max_tokens": 1024,+            "messages": [+                {+                    "content": "You are a helpful assistant who answers questions and executes tools for the user. Always use tools when asked to, but use only the tools that are available.",+                    "role": "system"+                },+                {+                    "content": "how's the weather in london?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"London\"}",+                                "name": "get_weather"+                            },+                            "id": "call_wucefmo8",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in London is partly cloudy, 18°C, light breeze from the west.",+                    "role": "tool",+                    "tool_call_id": "call_wucefmo8"+                },+                {+                    "content": "Currently, the weather in London is partly cloudy with a temperature of 18°C and a light breeze blowing from the west.",+                    "role": "assistant"+                },+                {+                    "content": "and in paris?",+                    "role": "user"+                },+                {+                    "role": "assistant",+                    "tool_calls": [+                        {+                            "function": {+                                "arguments": "{\"location\":\"Paris\"}",+                                "name": "get_weather"+                            },+                            "id": "call_9243whne",+                            "type": "function"+                        }+                    ]+                },+                {+                    "content": "Weather in Paris is sunny, 23°C, no wind.",+                    "role": "tool",+                    "tool_call_id": "call_9243whne"+                }+            ],+            "model": "llama3.2:latest",+            "stream": true,+            "stream_options": {+                "include_usage": true+            },+            "tools": [+                {+                    "function": {+                        "description": "Get the current weather for a given location",+                        "name": "get_weather",+                        "parameters": {+                            "$comment": "WeatherToolArgs",+                            "properties": {+                                "location": {+                                    "$comment": "City name, e.g. London",+                                    "type": "string"+                                }+                            },+                            "required": [+                                "location"+                            ],+                            "type": "object"+                        }+                    },+                    "type": "function"+                }+            ]+        },+        "response": {+            "content": [+                {+                    "text": "Currently, the weather in Paris is sunny with a temperature of 23°C and no wind blowing.",+                    "type": "text"+                }+            ],+            "reasoning": null,+            "text": "Currently, the weather in Paris is sunny with a temperature of 23°C and no wind blowing.",+            "usage": {+                "cache_creation_tokens": 0,+                "cache_read_tokens": 0,+                "input_tokens": 215,+                "output_tokens": 21+            }+        },+        "sse": "data: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"Currently\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\",\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" the\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" weather\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" in\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" Paris\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" is\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" sunny\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" with\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" a\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" temperature\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" of\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" \"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"23\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"°C\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" and\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" no\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" wind\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\" blowing\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\".\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":\"stop\"}]}\n\ndata: {\"id\":\"chatcmpl-480\",\"object\":\"chat.completion.chunk\",\"created\":1792089550,\"model\":\"llama3.2:latest\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[],\"usage\":{\"prompt_tokens\":215,\"completion_tokens\":21,\"total_tokens\":236}}\n\ndata: [DONE]\n\n"+    }+]