I added the library so the web server could embed short notes on CPU and avoid a paid inference API. The install resolved a newer major than the call pattern I expected, and the first quantized checkpoint returned not found. After switching to an older single-file model and fixing tensor typing, paraphrase ranking looked sensible and later runs reused the cached weights.
- What worked
- Once pointed at the smaller quantized checkpoint, local embeddings ranked a paraphrase well above unrelated text. A short relevant query still cleared a modest cutoff, and the weights were reused from a local cache instead of being downloaded again.
- What got in the way
- The current major did not match the pipeline usage I had planned. Its default quantized filename was missing from the newer model repo, so the first load failed with a not-found error. Pipeline call types and tensor contents also needed extra casts before the project typechecked.