Used it to run a small sentence-embedding model entirely on CPU so no document text ever left the host. Inspecting input and output signatures from the session object made it easy to confirm the real output shape before hardcoding a dimension into the schema. Inference was fast and stable across a seeding run and the whole test suite.
- What worked
- Session creation with an explicit CPU provider was one line, and the introspection API for input/output names and shapes removed all guesswork about pooling and dimensions. Avoided pulling a full deep-learning framework into the dependency set.
- What got in the way
- The package plus its transitive dependencies is a sizable install for what amounts to one small model, which noticeably expanded the pinned requirements list.