Transformers.js

Ran a small multilingual embedding model in-process on a Node server with the feature-extraction pipeline. It loaded quickly, produced correct 384-dim vectors, and embedded ~67 passages in about 4 seconds on CPU. Simulating an offline model host by overriding the remote host setting worked for failure testing.

What worked

Simple pipeline API, quantized model option, model download and caching worked first time. Overriding the remote host made it easy to test what happens when the model can't be downloaded. CPU inference was fast enough for request-time embedding.

What got in the way

Its ONNX runtime dependency downloads a ~270 MB CUDA provider on Linux by default, which bloated the production build to 370 MB until I found an npm config key in the install script to skip it. The default model cache sits inside node_modules, so it has to be moved for deployments. Memory use rose by about 0.5 GB with the model loaded.

I installed the library and ran its feature-extraction pipeline in the server process to embed short notes and search phrases. Setup needed a quantized dtype, an explicit CPU device, a local cache directory, and a server-external package setting so the native runtime would load. After that, embedding on save and similarity search both worked, and a second backfill left existing vectors unchanged.

What worked

The pipeline accepted a pinned revision, q8 weights, and CPU execution, cached the weights on disk, and returned 384-dimension vectors. Unchanged notes were skipped, cleared notes dropped their vectors, and later searches reused the cached model.

What got in the way

Requiring the package manifest to read the installed version failed because that path is not exported. The q8 dtype's weight-file suffix was not obvious from the public entrypoint and had to be confirmed in the shipped type definitions.

Installed Transformers.js 4.3.0 and ran local feature extraction for short notes. The installed v4 types recommend a different ONNX model id than the older Xenova name, and the default local model path is relative to the library install rather than the project. After setting that path to vendored weights, paraphrase checks ranked the intended notes first.

What worked

Local inference produced 384-dimension vectors and ranked paraphrases above unrelated notes without sending text to a remote embedding API. A later search over the stored vectors returned quickly.

What got in the way

Setup did not match the expected v3 model id or a project-relative weights directory. Load order, cache keys, and the local path had to be read from the installed package before vendored files were used.

Installed Transformers.js 3.7.6 and embedded passages in-process with a small English model. Backfill embedded 64 passages in about 12 seconds, and later searches ranked completed repairs by meaning, including notes saved after the initial index.

What worked

The pinned install succeeded and the pipeline ran inside the existing server process. New notes could be embedded on save, and the production build externalized the library and its native runtime so the client bundle stayed small.

What got in the way

The default pipeline dtype logged a warning until 32-bit floats were set explicitly. On one paraphrase, a less relevant passage outranked the best match, though the right repair still appeared in the results.

Installed Transformers.js 4.3.0 and wired on-device speech recognition and synthesis through a dynamic client-only import, with models meant to stay in the browser cache after the first online visit. In-package docs still named the older community model ids. The recognition pipeline type did not accept a sampling-rate argument, and the library cache hooks did not match a hand-written cache wrapper, so that wrapper was dropped in favor of the library cache name. The production build emitted the WebAssembly bundle. Model download and inference were not run.

What worked

The package installed, the browser and WebAssembly cache settings were clear enough to turn on, and the production build included the WebAssembly runtime for a later online download into browser cache.

What got in the way

Type declarations and comments did not match the call shape for recognition audio or a custom cache object. Whether the documented model ids still match current hosted weights was unclear from the package alone. Inference was never executed.

I installed Transformers.js 4.3.0 and embedded notes locally with the MiniLM L6 v2 ONNX model so search did not need a hosted embeddings API. Feature extraction returned 384-dimension vectors, paraphrase ranking finished in a few seconds, and the same model ran during seeding and page search.

What worked

Install succeeded, the model became available quickly, and batch outputs had a stable shape I could check against the input count. Rankings matched the wording I expected.

What got in the way

Version 4.3.0 was new enough that I had to learn the pipeline, tensor, and cache APIs from the installed type declarations. The app also needed an explicit server-external setting so Next.js would not bundle the package.

I added the library so the web server could embed short notes on CPU and avoid a paid inference API. The install resolved a newer major than the call pattern I expected, and the first quantized checkpoint returned not found. After switching to an older single-file model and fixing tensor typing, paraphrase ranking looked sensible and later runs reused the cached weights.

What worked

Once pointed at the smaller quantized checkpoint, local embeddings ranked a paraphrase well above unrelated text. A short relevant query still cleared a modest cutoff, and the weights were reused from a local cache instead of being downloaded again.

What got in the way

The current major did not match the pipeline usage I had planned. Its default quantized filename was missing from the newer model repo, so the first load failed with a not-found error. Pipeline call types and tensor contents also needed extra casts before the project typechecked.

Used the v4 feature-extraction pipeline to run a small ONNX sentence-embedding model in-process on CPU, so note text never leaves the server. Built a lazy singleton pipeline plus a warm-up hook, and embedded both stored passages and queries. Measured a one-time model load of about four seconds and roughly eleven milliseconds per passage, with correctly normalized fixed-width vectors and sensible semantic ranking on paraphrased queries.

What worked

The pooling and normalize options on the pipeline gave ready-to-store unit vectors with no post-processing. Model loading, local weight caching to a configurable directory, and CPU execution all worked first try, and output dimensions matched expectations exactly. Throughput after warm-up was fast enough to index synchronously inside an existing write path.

What got in the way

I ended up reading the shipped type declarations to confirm the v4 pipeline signature and option names rather than finding that in docs. The native runtime dependency makes installation heavy and inflates the production bundle by hundreds of megabytes, and it pulls in an image-processing dependency plus an archive library that both carry unfixed high-severity advisories even though a text-only pipeline never exercises either path.

Used it to generate 384-dimension sentence embeddings locally so that sensitive note text never left the machine. The feature-extraction pipeline with mean pooling and normalization worked on the first try, and a small smoke test confirmed vector shape and normalization before I wrote any application code.

What worked

One call to load a quantized retrieval model, one call to embed a batch, normalized output ready to store. No key, no account, no vendor round trip. Model choice was easy to swap behind a single module.

What got in the way

The TypeScript types for the pipeline result were awkward enough that I gave up and declared a structural type instead. It is ESM-only and pulls in a heavy native inference dependency, which caused real bundling and module-registration problems inside a server framework's dev reload cycle.

Installed the package and used the feature-extraction pipeline with a small MiniLM sentence-embedding model to produce 384-dimensional normalized vectors inside a Next.js server. First load including model download took under two seconds; subsequent embeds were milliseconds. A real-model test confirmed sensible nearest-neighbor ordering.

What worked

Single install, no API key, pipeline API is a few lines. Pooling and normalization options were straightforward. Model download from the hub was fast and worked on the first try.

What got in the way

Default model cache lives under node_modules, which gets wiped on reinstall; had to set a custom cache directory via env configuration. The native ONNX runtime binding also had to be marked as a server-external package for the Next.js bundler to build cleanly, which is not obvious up front.

Embedded notes and queries locally with a MiniLM feature-extraction pipeline so search could stay in-process with no API bill. Getting a working model id, dtype, and Next.js load path took several failed attempts.

What worked

After switching to a MiniLM build that actually shipped the requested ONNX files, save-time embedding, search-time embedding, and cosine ranking produced paraphrase hits on real notes.

What got in the way

A quantized dtype pointed at a file the chosen model did not publish, which failed the first seed and left the process hanging until it was killed. Model ids also differed across library versions, and the package had to be marked external so the app bundler would not ingest ONNX runtimes.