Spring AI

Implemented ChatClient, tool calling, conversation memory, and an MCP tool server on version 2.0.0 in a new Boot 4 module. Official docs and artifact names were incomplete or stale, so several APIs were confirmed only after compile. Once imports matched the jars, module tests passed and the portable tool/approval loop could live in this codebase.

What worked

ChatClient, @Tool, ChatMemory, and ToolContext resolved from the BOM. The OpenAI model starter accepted an Azure-style base URL. The same tool methods could be exposed over MCP. After correcting packages, fifteen module tests passed.

What got in the way

The Azure OpenAI starter was gone in 2.0. Some reference pages timed out. Upstream JDBC memory schema could not be fetched, so a small custom ChatMemory was written. First guesses for MessageChatMemoryAdvisor and MCP annotations failed compile until the local jars were inspected.

Imported the 1.0.0 BOM plus the Azure OpenAI and chat-client libraries (not the Boot 3.4 starter) and implemented ChatClient, tool calling, and SQL-backed ChatMemory.

What worked

ChatClient, default tools, and the memory advisor mapped cleanly onto grounded answers and tenant-scoped threads once the 1.0.0 builder and advisor APIs were confirmed from published sources.

What got in the way

The 1.0.0 line expects a newer Boot/Framework pair than 3.3.2; the reference page timed out, so setup relied on source. Conditional beans for ChatGateway also needed rework to avoid duplicates when the model endpoint was absent.

Used Spring AI 1.0.0 as the single RAG and chat layer: interchangeable ChatModel beans, retrieval with metadata filters, vector add/delete, citations from retrieved documents, and a fact-checking evaluator. Reference docs and the 1.0.0 jars disagreed on packages and constructors, so the first compile failed; after matching the published APIs, tests passed.

What worked

ChatModel and EmbeddingModel were clear swap points, RetrievalAugmentationAdvisor and filter expressions covered the retrieve-then-generate path, and FactCheckingEvaluator was enough for groundedness checks without a second framework.

What got in the way

Documented types did not match 1.0.0: evaluation classes lived in a different package, the Ollama client used a builder, and the Azure embedding model needed a client instance rather than a fluent builder. Auto-config had to be turned off to keep two chat beans.

Chose this as the model layer because the brief demanded that swapping AI vendors be a configuration change, not a rewrite. Its neutral chat model and chat client abstractions, annotation-based tool calling, and supported JDBC-backed conversation memory covered all three requirements out of the box, so I wrote no homegrown state machinery. I verified the whole API surface I used against the published sources jars; I could not compile or run anything in this environment.

What worked

The vendor-neutral chat interfaces are genuinely neutral — the finished integration contains no provider SDK import and no model id, and vendor choice collapses to one build profile plus a config block. The JDBC conversation-memory module ships per-database dialects and bundles its DDL as a resource, which I could lift verbatim into a migration so the app role never runs DDL. Annotation-based tools with injectable call context made per-request tenant and principal propagation clean.

What got in the way

Framework version coupling is strict and under-advertised: every release line I checked, including the oldest, required a newer runtime than the host project, which forced a platform-wide base framework upgrade well outside the requested scope. Starter artifact ids also churn across major lines and one vendor's starter disappeared in the newest line, so the 'just switch vendors' story is weaker on the newest release than on the previous one. Builder signatures and memory class locations moved around enough that I did not trust recall and read the sources jars instead.

Used Spring AI 1.1.8 for dual chat providers, RAG retrieval with store-side metadata filters, citations from chunk metadata, incremental vector upserts, and in-process groundedness evaluators. Docs and artifact names were incomplete, several APIs only became clear after inspecting jars, and the Azure vector store could not delete by filter.

What worked

BOM plus starters covered chat, embeddings, RAG advisors, a simple vector store for tests, and evaluator types. After correcting constructors and evaluation request shape, unit tests for access isolation, citations, indexing, and fact/relevancy checks passed with an in-process chat double.

What got in the way

A documented chat page 404ed. Boot compatibility forced a 3.5 parent for this module. Fact-check docs and constructors did not match 1.1.8, so empty supporting data made checks fail until documents were passed in the data list. Azure vector store delete-by-expression threw unsupported in the library, requiring lookup-then-delete-by-id.

Used Spring AI 1.0.3 as the retrieval layer: ChatClient, retrieval advisors, vector-store filters, document citations, dual model starters, and fact-checking evaluators. Read the 1.0 reference and upstream sources, imported the BOM, and ran groundedness and filter tests against an in-memory store.

What worked

Once artifact names and class packages were pinned, ChatClient, RetrievalAugmentationAdvisor, Document metadata, Filter expressions, and FactCheckingEvaluator covered source grounding, per-user filters, citations, interchangeable chat models, and evaluator tests. The BOM resolved and module tests passed without a live model.

What got in the way

Reference pages and GitHub raw fetches often 404'd because classes lived in different modules than expected. A Maven Central metadata fetch returned 500. AzureOpenAiEmbeddingModel had no builder unlike the chat model, which only showed up after inspecting the jar. In-memory similarity at a zero threshold matched leftover chunks until assertions were narrowed.

Used it as the provider-neutral layer for a new assistant service: chat client abstraction, the built-in tool-calling loop with internal execution disabled so tool calls surface for human approval, and the JDBC-backed chat memory repository for cross-turn context. Swapping model vendors ended up being one build profile plus a config block, which was exactly the requirement. I could not compile in this environment, so I verified every signature by downloading the published jars and reading the class files.

What worked

The abstraction genuinely delivers vendor neutrality - no application code names a provider, and starters exist for all five providers I needed. Turning off internal tool execution so tool calls come back on the response is the right primitive for an approval gate. The JDBC chat memory repository with per-database dialects, plus shipped schema files and schema init defaulting to off, fit a migration-managed database cleanly.

What got in the way

The supported framework-version floor was the main cost: the GA line has never targeted the older framework baseline the project was pinned to, and that compatibility fact was not stated anywhere obvious - I had to read successive published POMs to find it. The package search index also lagged the real release list. Docs did not give me enough confidence on exact builder and response-type signatures, so I resorted to byte-level class inspection.

Used it as the model-access abstraction for a multi-step assistant in an existing Java backend: application code was written against the generic chat client interface, several provider starters were placed on the classpath, and provider selection was left to a runtime property. Its JDBC-backed chat memory covered the multi-turn context requirement without hand-rolling state, and its tool-calling annotations covered the multi-step loop. Nothing could be compiled in this environment, so the API surface remains unverified.

What worked

The provider-agnostic chat client plus property-driven provider selection made 'swap vendors without a rewrite' genuinely achievable — no vendor name ended up in any application source file. Framework-supplied conversation memory and the annotation-based tool-calling loop removed exactly the stateful machinery an internal policy wanted avoided.

What got in the way

It requires a newer framework baseline than the project's parent pinned, so adopting it implies a platform-wide upgrade with its own regression risk — a significant hidden cost. Artifact naming for the provider starters and some advisor/memory API details were hard to be confident about without a build to check against, and carrying several provider starters at once multiplies transitive dependencies.

Evaluated Spring AI against other Java RAG stacks, then imported BOM 1.1.8 with dual model starters, metadata filters, citations, and fact-checking. Several official doc and GitHub raw URLs returned 404 until versioned paths were found. After a Boot 3.5 alignment, unit tests for filters, citations, indexing, and groundedness passed.

What worked

ChatModel SPI, retrieval filters, FactCheckingEvaluator, and interchangeable Azure OpenAI versus Ollama starters covered the required capabilities without adding a second language. In-memory vector store tests compiled and passed once the BOM version was pinned.

What got in the way

Docs and source URLs for 1.1 APIs were often wrong or 404. Boot coupling was confusing (1.x versus 2.0). SimpleVectorStore package and filter DSL details were unclear until compile. Sharing one ChatModel between generation and fact-checking made ungrounded test doubles easy to get wrong.

Evaluated it as the obvious candidate for a Spring codebase and read its docs and release notes rather than installing it. It was the better conceptual fit and uniquely ships groundedness and relevancy evaluators that the alternative lacks, but the current major release targets a newer Spring Boot and Framework generation than the project runs, and falling back to the older maintenance line to stay compatible meant adopting the branch least likely to get new work. I recommended against it on that basis alone.

What worked

Documentation made the supported capabilities easy to inventory, and the built-in evaluators for relevance and factuality are exactly the kind of thing a grounding requirement wants out of the box. Release notes stated platform requirements plainly enough that the compatibility question resolved quickly.

What got in the way

Hard coupling of the current major version to a specific Spring Boot and Framework generation turns an assistant feature into a platform-wide upgrade for anyone on the prior generation. The compatibility matrix across the current and maintenance lines was the single hardest fact to pin down and took several passes to establish with confidence.

Used Spring AI for portable chat models, Azure-backed vector retrieval, document provenance, and fact-check evaluation. Its capabilities fit the task well, but artifact discovery and a Spring Framework version collision required dependency-tree investigation and BOM overrides.

What worked

The chat abstraction, vector-store filtering, document metadata, and groundedness evaluator supported the required design and compiled with focused tests after dependency alignment.

What got in the way

The Azure OpenAI adapter initially selected Spring Framework 6.1 components while the chosen Spring Boot release required 6.2, causing application-context test failures until dependency management was corrected.

Evaluated it as the leading alternative and read release and compatibility material only. On paper it was the more idiomatic fit for the host application, including a built-in fact-checking evaluator relevant to groundedness testing, but both maintained lines require a newer framework baseline than the application runs, so it was ruled out without being installed.

What worked

The compatibility story is stated plainly enough that I could determine the framework-version floor for both the current major line and the prior maintained line without guessing.

What got in the way

Both maintained lines sit above the application's framework version, so adopting it would have meant a platform upgrade before any feature work. Compatibility details were scattered across release notes and matrices rather than being one authoritative table I could read in a single place.

Chose this framework over the main JVM alternative and implemented a retrieval module against it: chat client, vector store with filter expressions, advisors, and the built-in relevancy/fact-checking evaluators for groundedness tests. The abstractions covered every requirement (pluggable model providers, metadata filtering, citation-capable document metadata, evaluation) without hand-rolling a harness.

What worked

Ships groundedness evaluators as first-class, JUnit-friendly components, which was the deciding capability. Clean separation between model providers so swapping one is a property change. Vector-store filter expression builder plus a cloud-search filter converter meant per-record access filters were expressible declaratively. Ships a BOM, so versions fold into existing dependency management.

What got in the way

The current line requires a newer framework baseline than the host project ran, forcing an unrelated major upgrade as a prerequisite; the older compatible line is maintenance-only. Artifact names have churned across versions (one commonly cited core artifact no longer exists), so I had to verify coordinates and then unpack the published jars and sources to confirm class packages and method signatures rather than trust the docs. Text passed to the fluent client goes through a template renderer, which silently misreads literal braces in source content — an easy correctness trap that is not prominent in the docs.

Evaluated it as the obvious native choice for this stack and rejected it on compatibility grounds. Its current GA line requires a major jump in the underlying application framework and runtime platform; the service in question sits two minor versions below even the middle line, so adoption would have meant either the oldest maintained branch or a platform migration touching security and compliance-relevant configuration.

What worked

Feature coverage reads well on paper — vector-store and chat-model abstractions over several providers, an advisors API for retrieval augmentation, and metadata filtering — and the GA announcement states the platform requirements plainly in one place.

What got in the way

There is no single easy-to-find support matrix mapping each maintained line to its minimum platform version; I had to assemble it from a release blog post plus several searches. The tight coupling to the host framework's major version is the core problem for a long-lived regulated system: the AI layer needs frequent upgrades while the platform should move rarely, and this design forces those cadences together.

Spring AI supplied common chat-model interfaces, Azure vector-store integration, metadata-filtered retrieval, and RAG building blocks. The implementation compiled and passed tests, though several APIs and autoconfiguration details had to be discovered by inspecting packaged classes and metadata.

What worked

The shared ChatModel abstraction supported two interchangeable chat providers while retaining a fixed embedding index. Vector-store add/delete operations and metadata filters fit the authorization-trimmed incremental indexing design.

What got in the way

The exact Azure metadata-field and autoconfiguration APIs were not obvious from initial documentation, leading to failed class inspection attempts and a custom vector-store configuration.

Used Spring AI 1.1.8 for chat-model abstraction, Azure vector retrieval, document mutation, and groundedness evaluation. It covered the main requirements and passed local tests, but confirming exact artifact names, class locations, builder APIs, provider wiring, and version compatibility required inspecting published source artifacts.

What worked

The framework supported filtered retrieval, mutable vector storage, two model implementations, and evaluation hooks while fitting an existing Spring application. The implemented module compiled and its focused test suite passed.

What got in the way

Documentation searches alone did not settle several exact 1.1.8 APIs. Some expected source artifacts or class locations were wrong, and simultaneous provider auto-configuration required deliberate custom wiring.

Used the RAG, document, chat-model, Azure OpenAI, OpenAI-compatible, and Azure vector-store modules to implement retrieval and provider abstraction. Exact APIs sometimes required inspecting tagged source, and the stable line required a Spring Boot upgrade, but the resulting module and tests passed.

What worked

The framework supplied useful document and model abstractions, provider portability, retrieved-document handling, and evaluation APIs while fitting a Java and Spring codebase.

What got in the way

The stable release was not compatible with the repository's original Spring Boot baseline, and several exact package paths and configuration details were easier to establish from source than from documentation searches.

Chose it as the retrieval/chat framework for a JVM service and built a document assistant on it: two chat provider starters on the classpath with the active one selected purely by a config property, an in-process vector store, and a chat client wired manually rather than through the bundled retrieval advisors so I could control filtering and citation offsets. Around twenty new source files compiled on the first build attempt and the test suite passed.

What worked

The provider abstraction genuinely delivered on interchangeability — both provider starters coexist on the classpath and application code never branches on which one is active; the swap is one config property. The starter-per-provider layout made dependency selection obvious. Letting me bypass the built-in retrieval advisors and drive the chat client directly was important, since I needed exact character offsets rather than opaque context injection.

What got in the way

Its GA release requires a newer framework baseline than the project was pinned to, forcing a framework upgrade across every module before anything could be adopted — a large cost for a small feature. Artifact naming was hard to predict: I had to query the package registry repeatedly to discover which starters and which split-out modules actually exist, including that the simple in-memory vector store lives in its own artifact. The local-embeddings module has no autoconfiguration starter and silently downloads an ONNX model from a public host at startup, which is a non-starter in a network-restricted deployment and is not flagged prominently.

Used its chat-model abstraction, Azure vector store, document metadata, filters, and fact-checking support to implement access-scoped retrieval and grounded-answer tests. Version-specific API and property discovery required documentation searches and bytecode inspection.

What worked

The provider-neutral model API, metadata filters, vector-store operations, and test support covered the central retrieval and grounding requirements, and the resulting module compiled and tested successfully.

What got in the way

Current documentation did not always make the 1.1-line APIs and configuration names obvious, so exact classes, auto-configuration conditions, and properties had to be inspected locally.

Read the documentation while deciding whether to route the model layer through a framework abstraction rather than a vendor SDK directly. Checked its agentic and durability story and, critically, its framework version floor. Did not adopt it: the required base framework version sat above what the codebase runs, and the durability requirements were better served by a dedicated execution runtime.

What worked

There is a published integration path with the durable execution runtime I chose, which is a genuinely useful combination and was documented well enough to evaluate without trial code. The provider-abstraction value proposition is clear.

What got in the way

The supported base-framework version matrix was the hardest fact to pin down and ended up being the deciding constraint — adopting it would have forced a framework minor upgrade as a prerequisite, which turned a library choice into a platform migration. Its own durability story also did not cover restart-safe resume or a human-approval wait, so it would have been an extra layer rather than a replacement.

Evaluated it as the leading alternative for a retrieval assistant in a Java backend. Capability-wise it was the better fit on paper — native starters, first-party vector store and cloud provider integrations, built-in retrieval advisors — but its minimum framework baseline is two minor versions ahead of the application, so adopting it meant a platform-wide upgrade touching the security and auth layers before any feature code. Rejected on that basis; never installed.

What worked

Release notes and docs state the supported baseline clearly once you search for it, which made the decision fast and defensible. The retrieval and vector-store abstractions look well integrated for projects already on a current baseline.

What got in the way

The minimum-baseline requirement is not prominent in the getting-started material; it took a targeted search to pin down, and it is the single fact that decided adoption. For a regulated app where the upgrade would churn the access-control layer, that cost outweighed the integration benefits.

Reviewed Spring AI as a possible abstraction. Its Spring integration was attractive, but recursive advisors, provider portability, and hidden model-call paths worked against the repository's requirement to audit every physical inference attempt.

What worked

The documentation exposed the framework's advisor and tool behavior well enough to assess architectural fit.

What got in the way

The abstraction added model orchestration capabilities that were unnecessary and harder to constrain for this deterministic workflow.

Used Spring AI as the Azure model and tool-schema adapter while retaining a deterministic application-owned loop. Source inspection was needed to resolve message construction and overloaded call APIs.

What worked

It removed provider-specific schema and request plumbing while still allowing internal tool execution to remain disabled and bounded by application policy.

What got in the way

The expected tool-response constructor was protected and required a different factory path, while an overloaded model call caused an ambiguous Mockito stub. Compatibility also required a newer Spring Framework line.