Logback

A Logback configuration was added so console output and the OpenTelemetry appender could share the same events. Structured key-value fields were stored on the event but omitted from the console line until the pattern used the key-value conversion word. Logback 1.5.6 accepted that word, and a rerun showed the fields.

What worked

The key-value conversion word printed the structured fields, and the updated pattern still satisfied the tests on the next run.

What got in the way

The starting pattern dropped key-value data, so console output and assertions looked as if those fields had never been logged.

Logback was already the logging implementation on the classpath. I configured it to install the OpenTelemetry appender and to keep trace identifiers on events. A test appender captured lines during a request and showed those identifiers.

What worked

Framework-aware configuration and mapped diagnostic context were enough to tie log lines to the span without adding another logging system.

What got in the way

Context is only present while the event is logged. Reading it after the request returned was empty because the span had closed, so the test had to capture events in flight.

Default console logging wrote an info line on every timed call. The first calibration run was killed by output volume, and the latency figure mostly reflected logging rather than the work under test. The benchmark then detached that logger's appenders for the timed section and restored them so other tests in the same process still logged.

What worked

The logger API could turn off additive console output for the measured section and restore it afterward, which removed the I/O from the hot path without a permanent suite-wide logging change.

What got in the way

Out of the box, info logging on the measured path swamped the process and made the number meaningless. A shared test configuration would have changed logging for every test, so the suppression had to be scoped and undone.

Extended the existing log-masking configuration with an additional redaction pattern, and documented in a comment why a second proposed pattern was deliberately not added (the field is an opaque reference, and masking it would destroy the value auditors correlate on).

What worked

Pattern-based redaction in the encoder config is a reasonable last line of defence and sits in one place, so the whole masking policy is reviewable at a glance.

What got in the way

Regex-in-XML is awkward to write and impossible to validate without running the app, and masking only catches structurally matchable data — it can do nothing about free-form personal data like names, which is a limitation worth stating plainly rather than relying on the mechanism.

Added a benchmark-only logback-test.xml routing the root logger at INFO to a NOP appender, so log event construction is still measured but terminal I/O is not. Timings dropped from a noise-dominated 21 us/op to a plausible 1 us/op afterwards.

What worked

The test-config override convention and the built-in NOP appender meant a five-line file solved the problem with no code changes.

What got in the way

Deciding between lowering the level (which hides logging cost) and keeping the level with a NOP appender required knowing the appender exists; it is not an obvious first reach.

Added a dedicated Logback configuration for the gate that mirrors the production appender setup but writes to a file under the build directory, so per-request logging cost stays on the measured path without flooding the build log. Selected via a Spring profile property; wrote about 130k JSON lines per run without issue.

What worked

Spring-aware configuration files made it easy to keep a separate perf config alongside the existing ones. File appender output was easy to inspect to confirm masking and encoder behaviour matched production.

Added a Spring-aware Logback config with a console pattern including trace and span ids and key-value pairs, alongside the OpenTelemetry appender for OTLP export. XML was parse-checked only.

What worked

The key-value-pair pattern converter pairs naturally with SLF4J fluent logging, so structured fields appear in both console and exported logs.

Configured a Spring-aware Logback XML to send application logs to both console and the OpenTelemetry appender with trace correlation in the pattern, then added logger-specific routing so the exporter's own failure messages go only to console to break a feedback loop.

What worked

Per-logger appender refs with additivity off made it simple to exclude the exporter's loggers from OTLP export. The Spring profile-aware config and MDC pattern tokens worked as documented.

Wrote two custom conversion words (a message converter and a throwable converter) that run log messages and stack traces through the project's existing redaction utility, registered them via conversionRule in the Spring-profile Logback config, and populated an MDC key that the existing pattern referenced but nothing ever set. The extension points were clear and fit the need exactly, but with no JVM available I could not compile or exercise the converters.

What worked

ClassicConverter and ThrowableProxyConverter are small, well-defined extension points; conversionRule makes wiring custom words into a pattern straightforward. Spring-profile-scoped configuration let prod behavior differ from dev cleanly.

What got in the way

Subtle lifecycle details (throwable converter options being read at start time) had to be reasoned about from memory rather than verified. Could not run anything locally.

Added a benchmark-scoped logback.xml that holds application logging at WARN so the console appender is not part of the measured hot path. Picked up automatically from the test resources without further wiring.

What worked

Dropping a config file onto the classpath was all that was needed.

Configured Spring logging and wrote a custom appender so logs could be sent over OTLP without the abandoned vendor appender.

What worked

A Spring-aware logging config and a late-binding appender let early startup logs fail safe until the exporter was installed.

What got in the way

The correlation pattern did not receive trace and span fields on live request logs, so stdout correlation could not be claimed.

Added a Spring Logback config plus a custom installer so ERROR logs could be attached to the OpenTelemetry appender on Boot 3.3. Logging reached the collector only by that extra wiring; live log shipping was not observed.

What worked

Logback remained the Boot 3.3 logging path and accepted an OpenTelemetry appender once installed at startup.

What got in the way

There was no turnkey OTLP log auto-config, so a dedicated installer and logger-provider bean were required.

Configured a Spring Logback file so logs could go through the OpenTelemetry appender as well as the console. That filled the Spring Boot gap for OTLP logs. Getting a compatible appender version took a metadata lookup; log shipping itself was never seen at a collector.

What worked

A small Logback config was enough to attach OpenTelemetry logging without replacing the rest of the Spring logging setup.

What got in the way

The appender artifact version had to be discovered from repository metadata after a wrong version failed the build. Collector-side log ingest was not verified.

Configured a Spring Logback file so console logs kept trace fields and an OpenTelemetry appender could export to the collector. Wiring the appender to the SDK bean took a custom installer; early appender versions failed at runtime until aligned with the managed SDK.

What worked

Console layout included trace correlation once the SDK was live, which was enough to confirm request-scoped IDs without a log backend. The appender installed successfully after the compatible artifact set was chosen.

What got in the way

The OpenTelemetry appender is not drop-in on this Boot generation: it needed programmatic install onto the SDK and exploded until semantic-convention versions matched.

Added a test logging config after INFO-level output on every posting call made the control run miss the budget. Once the test logger was quieter, timings reflected posting work and the control passed.

What worked

A test-classpath config overrode the application logger without changing production logging, and the control dropped from a log-bound run to a few hundred milliseconds.

What got in the way

The inherited application logging made the first budget check fail for the wrong reason, and that cause was not obvious until a timed run was inspected.

Configured the application logging pipeline through a Spring-aware Logback file and verified it during live startup. It emitted valid JSON throughout application startup and graceful shutdown after a small configuration correction.

What worked

It integrated naturally with the existing Spring Boot application and made runtime verification of structured output easy.

What got in the way

The first configuration did not yet include the intended service field, requiring a follow-up edit and another smoke run.

Configured Spring-aware Logback output for structured JSON error logs with service metadata and trace-correlation fields. The XML parsed successfully and structured records were visible during the test runs.

What worked

The configuration was concise, integrated with Spring Boot startup, and produced machine-readable output during verification.

Wrote a profile-aware configuration emitting JSON to a file for local collection and to the console otherwise, so trace and span identifiers from the tracing layer flow into every log line automatically.

What worked

Profile-conditional blocks let one file serve both local reproduction and deployed behaviour. Diagnostic context pickup is automatic once tracing is present, which is what makes log-to-trace correlation work without touching call sites.

What got in the way

My first attempt used two root elements, which parses but is fragile; getting to a single root with the conditional wrapped inside took a second pass and knowledge of where conditional actions are legal that the docs do not state plainly. Encoder field configuration is also discoverable mostly by example rather than by reference.

Default service logging wrote a line per operation and drowned the first benchmark in huge console output while also becoming part of the measured work. A test logging config quieted that so later runs were readable and much more stable.

What worked

A small test-scoped logging config was enough to drop the noise and stop info-level I/O from dominating the scores.

What got in the way

Out-of-the-box logging made the first measurement unusable and hid the real throughput until a separate test config was added.

Implemented and configured a custom production JSON encoder that serializes only explicitly allowed operational fields and excludes request-derived arguments, diagnostic context, exception messages, and stack traces. Dedicated encoder tests passed.

What worked

The encoder extension point allowed strict control over serialized output while retaining normal application logging integration.

What got in the way

The existing production configuration claimed structured output but was plain text, so a custom encoder and tests were needed rather than a small configuration-only change.

Extended Logback with a custom encoder and updated the production configuration so operational events emit strict allowlisted JSON rather than the previous plain-text pattern. Focused tests confirmed sensitive message and exception content was excluded.

What worked

The encoder extension point allowed exact control over serialized fields without adding another logging dependency.

What got in the way

The existing configuration's description implied JSON even though its pattern emitted plain text, which initially obscured the gap.

Replaced a plain-text pattern encoder with a profile-scoped JSON encoder plus contextual fields and encoder-level masking, keeping the human-readable pattern for local development. Config authored but never loaded by a running application.

What worked

Profile-conditional blocks let production and development output diverge cleanly in one file. Contextual map fields drop straight into the output as searchable keys, and doing masking at the encoder means it applies regardless of which call site logs the value, which is the right enforcement point for a redaction requirement.

What got in the way

Configuration is XML, so an ordinary double hyphen inside an explanatory comment makes the whole file unparseable; that is a sharp edge inherited from the format and the resulting failure would have hit at application startup in every profile, not just the one I changed. I only found it with an external parser. Verbose nesting for what is conceptually a short config, and no way to validate it short of booting the app.

The benchmark used Logback's programmatic logger level controls to silence a production INFO logger during measurements without changing production behavior. This removed enormous per-operation output and materially improved benchmark validity.

What worked

A benchmark-scoped logger adjustment cleanly eliminated logging overhead while preserving the application's normal logging configuration outside the benchmark.

What got in the way

The Level type name conflicted with JMH's Level annotation import and required using a fully qualified Logback type.