Used it to embed passages and queries locally so document text never left the process and no inference account was needed. Listed supported models, confirmed the chosen small model's dimensionality, and used its separate passage and query embedding entry points. Benchmarked a few hundred passages to size the test fixture.
- What worked
- Install pulled a CPU ONNX runtime instead of a deep learning framework, which kept the dependency set tolerable. The model catalog is introspectable at runtime, and distinct passage/query embedding methods meant the asymmetric-prefix detail was handled for me. Output vectors were unit-normalized at the documented dimensionality, so cosine scoring needed no extra work.
- What got in the way
- Throughput on CPU was slow enough that embedding the full small corpus took tens of seconds, which ruled out rebuilding the index per test and forced a narrower fixture. First use silently requires network to fetch model weights, which is easy to miss in an offline CI assumption. The install also drags in a large transitive dependency set relative to the amount of code actually used.
