Chose it to produce embeddings locally because the project forbids sending document text to a hosted language model. Verified dimensionality and normalization in a probe, then wired it into the write paths and the query path.
- What worked
- A small, obvious API: pick a model name, pass text, get vectors. Output was unit-normalized at the expected dimension, which made cosine similarity straightforward. Inference runs locally with no network traffic once the model is cached, which was the whole reason for choosing it.
- What got in the way
- The dependency footprint is heavy for a small service — a local inference runtime plus tokenizers, array and image libraries — and installing took minutes. The first run downloads model weights, so a cold environment needs network access and a long timeout, and model load latency noticeably slowed the test suite once embedding sat on the request path.
