Evaluated as the recommended embedding partner for the model provider in use, and initially planned around its smallest model because open weights would have satisfied a constraint that no document text leave the host. Inspecting the published artifact listing showed the small model is actually a several-hundred-million-parameter decoder-derived model whose full-precision export is well over a gigabyte, far too heavy for this deployment. Kept it as an opt-in backend but did not make it the default and never called the hosted API.
- What worked
- Open weights for a small model are genuinely useful, since they make a local-only deployment possible at all. A contextualized-chunk variant also looked well suited to short procedural documents.
- What got in the way
- The naming implies a lightweight encoder, but the artifact is much larger than that; this only became apparent after querying file sizes directly. Quantized variants exist but trade quality in ways the documentation does not quantify, and pooling behavior for the export was not documented well enough to implement confidently.