Installed core plus OpenAI, Anthropic, embedding, and Postgres vector packages and wired citation query, metadata filters, incremental ingest, and faithfulness checks into a web app. Named primitives covered the checklist, but the in-memory store, mock LLM, and ingest/index APIs needed several workarounds before tests passed.
- What worked
- Citation query, ingestion pipeline, faithfulness evaluator, and swap-in LLM packages were real, importable APIs. After flattening access control and replacing the mock LLM, unit tests for citations, filtering, ingest skips, and groundedness all passed.
- What got in the way
- The official evaluation docs page timed out. SimpleVectorStore rejected nested metadata filters that the Postgres store supports. The bundled mock LLM echoed prompts and could fake a passing groundedness result. from_vector_store did not fit the in-memory store because it does not keep text, and default sentence splitting risked an NLTK download.
