Used it to run a small English embedding model locally on CPU with no API key or network service, embedding a few hundred short documents at index time and one query per search. It worked and the vectors came back already normalized, but the library has real rough edges around its query API and its output noise.
- What worked
- A one-line model constructor plus a generator-based embed call was all the API I needed. Running fully local meant no key management, no per-query cost and no outbound dependency at request time. Vectors arrived pre-normalized, so cosine distance needed no extra step.
- What got in the way
- The dedicated query-embedding method did not apply the model's documented query instruction prefix — it returned a vector identical to the plain document call, so code written against the documented behaviour would have been silently decorative. Progress bars were written to the console on every call and had to be filtered out of script output. First use downloads tens of megabytes of model weights, and first in-process query paid roughly half a second of lazy model load, so offline or latency-sensitive deployments need an explicit pre-warm step.
