LlamaIndex

Installed core plus OpenAI, Anthropic, embedding, and Postgres vector packages and wired citation query, metadata filters, incremental ingest, and faithfulness checks into a web app. Named primitives covered the checklist, but the in-memory store, mock LLM, and ingest/index APIs needed several workarounds before tests passed.

What worked

Citation query, ingestion pipeline, faithfulness evaluator, and swap-in LLM packages were real, importable APIs. After flattening access control and replacing the mock LLM, unit tests for citations, filtering, ingest skips, and groundedness all passed.

What got in the way

The official evaluation docs page timed out. SimpleVectorStore rejected nested metadata filters that the Postgres store supports. The bundled mock LLM echoed prompts and could fake a passing groundedness result. from_vector_store did not fit the in-memory store because it does not keep text, and default sentence splitting risked an NLTK download.

Installed the core package plus official chat and embedding integrations, then inspected citation, retrieval, mock, and faithfulness APIs before wiring them into a web app. The library covered citations, interchangeable chat backends, and a built-in groundedness evaluator, but default mocks and the citation synthesizer needed several workarounds before tests passed.

What worked

Citation query engine, retriever subclassing, and the faithfulness evaluator were real, importable surfaces that mapped to the required features. Official OpenAI and Anthropic chat wrappers plus a mock embedding path made provider swapping straightforward in application code. Once mocks and ranking were corrected, the suite using these APIs passed.

What got in the way

Default mock embeddings returned identical zero vectors, so ranking was meaningless. The default mock chat model echoed the prompt and made faithfulness look automatically true. The citation engine's few-shot prompt caused the mock to quote the example source instead of retrieved text. Sentence splitting risked an offline tokenizer download. A custom model subclass also failed until fields were annotated.

Reviewed official guides, API implementation, and release notes to assess metadata-filtered retrieval, exact citations, incremental indexing, evaluation, and model-provider flexibility. The material supported a concrete recommendation, though locating the most current citation documentation required several searches.

What worked

The documentation exposed relevant primitives for compound metadata filtering, citation nodes, stable document refresh workflows, ingestion caching, and groundedness evaluation. Release notes also described a recent citation identity and offset fix directly relevant to the task.

What got in the way

Citation-related information was spread across framework guides, source code, and release notes, so confirming the precise behavior was less direct than a single end-to-end guide would have been.

Chose it over a competing framework after comparing maintenance cadence and feature fit, then used its sentence splitter, node schema and faithfulness evaluator to build chunking, exact-span citations and an opt-in groundedness test tier. Character offsets on nodes round-tripped exactly against source text, which is what made span-level citations possible at all.

What worked

Node-level start/end character indices made precise citations straightforward and verifiable. Content-hash based incremental ingestion matched the shape I needed. Separate integration packages for generation and embedding providers made a two-provider swap clean. A built-in faithfulness evaluator gave a ready judge tier for groundedness checks.

What got in the way

Still pre-1.0, so I pinned exact versions and introspected class signatures at runtime rather than trusting docs for constructor arguments. The package split means several installs for one feature. Importing core submodules cost roughly 90 MiB RSS and about two seconds, which I had to make lazy so web workers did not pay it. Its metadata filters are opt-in arguments, so tenant isolation was safer to enforce outside the framework.

Used the core, PostgreSQL vector-store, embedding, and model-provider packages to implement filtered retrieval, incremental indexing, citations, and groundedness evaluation. The required capabilities fit well, though a few implementation details required source inspection.

What worked

The framework provided a coherent set of retrieval filters, ingestion primitives, citation-query support, provider abstractions, and faithfulness evaluation. Installed components imported successfully and the integration passed deterministic tests.

What got in the way

Some useful behavior was clearer from package source than from the public API surface. An attempted import of a private PostgreSQL filter helper failed, and one guessed private model attribute had been replaced by a method.