Chose and implemented this framework for a stateful assistant in an existing sync web service: typed dependency injection, tool definitions, a deferred-tool approval gate, and streamed responses. The deferred-tool design was the deciding factor because approval state resumes from serialized message history plus a results object, so conversation persistence stayed in the storage the project already had instead of requiring a second checkpointing system. Shipped with a full test suite that exercises the approval gate and streaming path.
- What worked
- Approval-required tools and the resume-with-results pattern mapped cleanly onto an HTTP request boundary. Built-in scripted and function-backed fake models let me test the whole approval and streaming flow without any provider credentials. Sync tool functions are dispatched to a thread pool, so existing synchronous database code stayed idiomatic. Provider swapping is genuinely a model-object change in one file.
- What got in the way
- Docs were enough to pick the library but not to write against it: I had to introspect installed signatures for several APIs. Two defects reached running code from doc-based assumptions — the streamed-events entry point is an async context manager rather than a bare async iterator, and the fake model needs a separate streaming callback. Streaming is async-only, which forces a sync/async split in an otherwise synchronous codebase. Serialization of message history is documented thinly.
