Used FastEmbed to run bge-small-en-v1.5 on the CPU inside the app process, so no document text went to an outside model. The model downloaded quickly and produced 384-dimension vectors. It also loaded offline from a read-only cache, which is how the container image is meant to run.
- What worked
- No PyTorch or GPU was needed, installing it was simple, and setting a cache path made it easy to bake the model into an image.
- What got in the way
- It wasn't clear whether query_embed adds the BGE query instruction. I had to read the installed source to confirm it doesn't, then add the prefix myself.
