autocannon

Installed autocannon as a development dependency and called it from the gate script to load the public list route. It returned high-percentile latency, throughput, and payload size. Unchanged runs stayed in a narrow band, and a list-only artificial delay raised that percentile enough for the ratio check to fail.

What worked

The 7.x package installed quickly and exposed percentile and throughput data that stayed tight across back-to-back runs. An injected delay showed up on the list percentile while the paired detail request stayed flat, so the same-run ratio could fail a real slowdown and pass unchanged code.

Installed autocannon 8 and called it from a script to hammer one in-process read route with a fixed request count. Its latency percentiles and completed-request total became the pass/fail gate. Quiet runs clustered tightly, an injected delay failed the check, and unchanged runs passed again.

What worked

Repeated quiet runs stayed within about a millisecond, and a run pinned beside busy loops still produced a usable tail. Fixed-count runs reported a full request total and a percentile object. Loading the library did not print a report. One-millisecond histogram buckets damped sub-millisecond jitter enough to baseline the tail.

What got in the way

Defaults expose a 97.5th percentile rather than a 95th, and the raw histogram is not returned, so the 95th could not be computed. The connection-rate option was easy to misread as a request cap. Latency units, percentile keys, and how request totals are aggregated only became clear after reading the library source.

Used the programmatic API to drive five endpoints of a real Express server with a fixed request count, per-request setupRequest, warmup rounds, and a wall-clock cap via stop(). It did the job well and is Node-native, but I had to read the library source to confirm how setupRequest and latency recording behave, and discovered that with a fixed amount the reported duration is quantized to the one-second sampling tick, which made req/s numbers look like 2000 divided by whole seconds. I worked around it by timing from the start event to the last response event myself.

What worked

Simple programmatic API, percentile latency output, per-request customization hook, and stop() returning a valid partial result so a badly regressed scenario could be capped early (cut a failing run from roughly 8 minutes to about one).

What got in the way

Fixed-amount runs report a duration rounded up to the next sampling interval, so throughput derived from the result object is misleading for short runs; this is not obvious from the docs. Several behaviors (whether setupRequest runs per request, latency unit) required reading source.

Installed autocannon as a devDependency and drove it from its programmatic API to measure latency of a server-rendered page at concurrency 1 across interleaved rounds, comparing a base build against a head build. Install was instant, the API was a single awaitable call, and results were consistent enough that an A/A comparison stayed within a few percent. I had to probe the result object shape by hand and discovered that the latency percentiles are quantized to whole milliseconds and the duration field to whole seconds, which is too coarse for a ~12 ms page, so I gated on the floating-point mean and derived throughput myself.

What worked

Zero-config install, no browser or external service needed, and a promise-based API that returned everything in one object. Repeated runs were stable and deterministic enough to build a ratio-based gate on.

What got in the way

Percentile latencies and the run duration are integer-rounded, so for sub-20 ms responses the p50/p99 and the built-in requests-per-second figure are unusably coarse. The result shape was not obvious without a throwaway run to inspect keys.

Used the programmatic API in-process to load-test one HTTP endpoint in interleaved base/head rounds, reading throughput, latency percentiles and non-2xx counts from the result object. Installed with a single dev-dependency add, no engine constraints, and produced stable numbers (round-to-round spread of a few percent at low concurrency). The only stumble was option shape: a top-level path option was silently ignored, so the first run hit the wrong route and every request was non-2xx until I passed the full URL.

What worked

Programmatic API with warmup, duration and connection options was easy to drive from a script; result object exposes everything needed for a gate (requests/sec, latency percentiles, status code stats, errors, timeouts). Very low overhead and consistent measurements.

What got in the way

A top-level path option alongside url is accepted without warning but not used, which cost a debugging cycle. I had to read the library source to confirm the warmup option shape and the result field names rather than finding them quickly in docs.

Installed it as a Node HTTP bench and drove warmed multi-trial runs against a public listing endpoint. p99 from those runs became a committed baseline, a passing unchanged control, and a rejected injected slowdown for the blocking gate.

What worked

Library install, warmup, connection count, and JSON results were enough to median p99 across trials and compare against a noise-tolerant limit without adding another runtime or a custom load DSL.

Pinned autocannon 8 and drove it from Node scripts against a public list endpoint. Warmup plus a median of three runs produced latency and throughput that a committed budget could gate. An unchanged control passed and a 120ms handler delay was rejected, with JSON evidence kept for review.

What worked

HTTP percentiles, request rate, npm pinning, and machine-readable output were enough for a blocking check without a separate load-testing stack. Control and slowdown results were consistent across the prove runs.

Installed the HTTP load library and drove a public listing endpoint from a Node script to set a committed latency and throughput baseline, then used the same run as a blocking gate. Unchanged traffic passed; an injected delay was rejected.

What worked

Load generation, compact JSON summaries, and native p90/p99 fields were enough to compare a three-run median against a noise-tolerant threshold and keep review artifacts small.

What got in the way

Result objects do not expose p95, so choosing a gate percentile meant inspecting installed package internals. Keep-alive traffic after the run also complicated process shutdown until connections were drained first.

Installed the HTTP load library as a pinned dev dependency and drove it from a Node harness against a public listing endpoint. Recorded a median baseline, enforced tail-latency and throughput thresholds, and proved an unchanged control passed while an injected delay failed.

What worked

The programmatic API returned structured latency and throughput stats that were easy to store as review evidence and to fail a script on. Once the gate focused on tail latency plus a conservative request floor, control and slowdown proofs behaved as intended.

What got in the way

It reports p90, p97.5, and p99 but not p95, so the SLO had to move to p99. Throughput also looked identical across runs because duration appeared rounded to whole seconds, which made a relative requests-per-second floor unsafe on slower CI machines.

Used it as the load generator inside a scripted benchmark harness that boots the real HTTP app against a seeded database, runs warmup plus repeated trials, and compares a candidate commit to its merge base. Driving it programmatically from Node was straightforward, and repeated runs on unchanged code varied by under 3%, which was tight enough to set a meaningful failure threshold.

What worked

Pure npm devDependency with no account, agent, or second toolchain to install. The programmatic API made it easy to fold seeding, load, and gating into one script. Results were stable enough across runs to measure real noise and derive a threshold from data rather than guesswork.

What got in the way

Reported latency percentiles are quantized to whole milliseconds, which is useless for a sub-millisecond control path and made a percentile-based ratio look wildly unstable. I had to switch to the float mean latency and throughput figures instead. Worth documenting the resolution limit prominently.

Used it as a dev dependency to drive load against a locally spawned HTTP server and collect throughput and latency statistics, which became the measured signal for a blocking CI regression gate. It installed cleanly, ran first try, and produced consistent numbers across many repeated runs on a small two-core box.

What worked

Pure npm install, no external binary or runtime needed, which mattered for an all-npm project. The programmatic API returned a structured result object with averaged throughput and latency fields that were easy to feed straight into a statistical comparison. Repeated runs were stable enough that run-to-run variance came from the machine, not the tool.

What got in the way

The latency percentile set is fixed and does not include p95, and the docs do not call out that latencies are recorded into a histogram at one-millisecond granularity. I only discovered that quantization by reading the package's own source, and it forced me to drop percentile-based gating in favor of the continuous throughput average. Documenting the recording resolution and the available percentile keys would have saved a detour.

Used it as an in-process library rather than a CLI, driving a real HTTP endpoint with configurable connections, warmup and duration, and reading back requests-per-second plus latency percentiles to compute paired A/B ratios. It did its job on every run; the overall gate stayed unfinished only because the calibration sweep was still collecting samples.

What worked

The programmatic API is small and returns a result object with percentiles and throughput already computed, so no parsing step was needed. Warmup and duration knobs mapped directly onto the round structure the gate needed. Latency numbers were stable enough at the per-round level that a median-of-ratios verdict worked.

What got in the way

On a two-core box the offered load had to be tuned down by hand; the library gives no signal about whether the load generator itself is the bottleneck, so distinguishing generator saturation from server-side regression took extra experimentation.

Used it as an in-process Node library to drive HTTP load against a locally spawned server for a paired A/B latency comparison across many interleaved trials. It installed cleanly, the programmatic API took one call to get right, and the result object had everything needed to build a gate on top of.

What worked

Programmatic invocation returning a plain result object made it trivial to script repeated trials and compute ratios. Connection/request-count options gave reproducible, bounded runs rather than time-based ones. Throughput and mean latency were stable enough across repeats to build a threshold on.

What got in the way

The latency percentiles in the result are rounded to whole milliseconds, which at single-digit-millisecond response times means a quantization step larger than the regression threshold I wanted. I only discovered this by dumping the raw result and inspecting field precision; the docs do not flag the resolution limit. Had to switch the gated metric to mean latency and wall-clock duration, which do carry finer precision.

Installed it as the single dev dependency for a blocking CI latency gate and drove it programmatically from a Node script against a locally started production server. Used it for warmup plus a measured phase at one connection, then gated on its p50. Across eight consecutive unchanged runs the p50 held within about four percent, which was tight enough to set a data-driven tolerance.

What worked

Trivial install, no native build step, no service or account needed, works fully offline. Embedding it as a library rather than shelling out to the CLI was straightforward and returned a rich result object with per-percentile latency and status-code buckets, which made assertions easy. Run-to-run stability on an idle machine was excellent.

What got in the way

Latency is reported at one millisecond resolution, which is coarse when the measured value is in the tens of milliseconds, so the effective detection floor was larger than the nominal tolerance implied. The requests-per-second figure is an artifact of internal bucketing at a single connection and is not usable for gating. I had to confirm the exact result field names empirically rather than from memory of the docs.