Elasticsearch

Integrated the managed Elasticsearch-compatible search service as the typo-tolerant operator search backend, with an outbox-backed bulk indexer and a read-only search endpoint using fuzzy matching. Local query shaping and parsing tests passed, but no live deployment or live query was run in the record.

What worked

API concepts for index mapping, fuzzy matching, and bulk indexing were clear enough to implement with standard HTTP and JSON handling plus configuration for URL, key, and index name.

What got in the way

Live provisioning, authentication, indexing, and query behavior could not be observed because no live service account was used.

Integrated the hosted search service for typo-tolerant vehicle and trip lookup at large scale, using versioned indices, bulk updates, and a search endpoint with database fallback. Query building and bulk formatting were covered by unit tests, but live indexing was not exercised because service credentials were unavailable.

What worked

REST indexing and fuzzy query concepts were clear enough to implement without adding an external SDK dependency.

What got in the way

Live indexing behavior, relevance tuning, and update latency could not be confirmed without service access.

Used the official JavaScript client for index setup, document indexing, search queries, and liveness checks. All operations against the live single-node instance succeeded, including idempotent overwrites and filtered search.

What worked

Index management, single-document writes, search calls, and ping checks were straightforward and behaved consistently during live verification.

Chose a dedicated search service to handle misspellings and large result sets without loading the primary database. Implemented fuzzy multi-field queries and best-effort write-through indexing against its REST API, verified only with a fake server.

What worked

API concepts for fuzzy matching and per-document upserts were clear enough to implement with plain HTTP and no extra SDK. Design cleanly separates system of record from search traffic.

What got in the way

No live cluster was exercised in the task, so ranking quality, latency at scale, and historical backfill remain unverified.

Ran a real single-node instance from the downloaded distribution to verify indexing and querying of tens of millions style order identifier records, keeping search load off the primary transactional database.

What worked

Startup with single-node discovery and disabled security worked for local verification. HTTP health, count and search APIs responded consistently and confirmed indexed documents were queryable.

What got in the way

Default clustered and secured settings needed overrides and memory tuning for a small verification host, plus a wait for readiness before first request.

Integrated managed Elasticsearch as the operator search backend for typo-tolerant queries, with single-document upserts and bulk indexing for backfills. Local query building, hit parsing, and endpoint tests passed, but no live cluster was available for end-to-end verification.

What worked

API shape was clear for fuzzy matching, boosts, single-document writes, and bulk NDJSON payloads. Configuration through URL, key, and index name kept wiring simple.

What got in the way

Live search behavior against a real cluster was not exercised in this task, so ranking quality and bulk throughput still need a live check.

I designed an index for vehicle and trip lookup with in-place updates, a one-second refresh, fuzzy and ngram fields for misspellings, and external versions so an older write cannot overwrite a newer one. Local tests checked query JSON and bulk versioning only. I did not install an official client or run a cluster, so ranking and refresh behavior were not observed.

What worked

The published field types and query controls covered exact identifiers, prefixes, fuzzy names, and ngram matches on mistyped plates, with boosts and a recency tie-break, without a separate spellcheck pass or a full reindex on each change.

What got in the way

Index-time and search-time analyzers are easy to set inconsistently. An early mapping analyzed plate and VIN queries with a different analyzer than the ngram index, which would have stopped query grams from overlapping indexed grams. I caught that from the mapping rules before shipping, not from a cluster error.

Implemented search over vehicle and trip records using typo-tolerant queries, background indexing after database writes, and a paged backfill job. Configuration was environment based with graceful degradation when unconfigured. Verified only against local HTTP stubs, not a live cluster.

What worked

Query and bulk-index API shapes were clear enough to implement with only the standard library, including index setup and paged backfill.

What got in the way

No live cluster was available, so relevance, performance, and managed hosting behavior at large scale were not observed.

Recommended and then implemented a dedicated search cluster for tens of millions of order and subscriber records, with async indexing from existing order event topics to keep search load off the primary database.

What worked

Conceptual fit for scale, latency and isolation was clear, and the final manifest covered replicas, pinned image, persistent storage, restart policy and health probes.

What got in the way

No live cluster was available, so storage sizing, probe paths and version compatibility were fixed from prior project context and left for confirmation on a build host.

I used the hosted deployment API reference to specify a named Elasticsearch deployment in one GCP region, including create-or-update, a health wait, and retrieval of the endpoint and an API key. Template and hot-tier instance-configuration identifiers took several searches to pin down. I encoded that into a provision script and deployment spec, then only compile-checked the script. I never called the live API.

What worked

The deployment CRUD reference was enough to describe creating or updating a deployment by name, waiting until health is green or yellow, and reading the Elasticsearch endpoint plus an API key for later secret storage.

What got in the way

The right deployment template and CPU-optimized hot-tier instance configuration IDs were not obvious from the first examples, so the spec stayed uncertain until several follow-up searches. The HTML form of the deployment reference did not load on the first fetch; the markdown copy of the same page did. No live organization was available to confirm the calls.

Evaluated only through documentation while comparing self-hosted search options for a small single-database app. Docs made memory needs, JVM tuning, and cluster operations clear enough to rule it out for this size of deployment.

What worked

Documentation clearly described operational weight versus a lightweight single-binary alternative.

The Elasticsearch Java client 8.10.4 arrived with the Spring Boot 3.2.11 Elasticsearch starter and was used to build an index template, upsert order documents, apply partial updates, and run term filters. A unit test loaded the template JSON through the client and the search module tests passed. No Elasticsearch cluster was available, so live indexing and query latency were not measured.

What worked

After the right builder and mapping members were located, the template JSON parsed into a put-index-template request, and the update and search call shapes compiled. The module tests passed once the mapping assertion used the keyword type check.

What got in the way

withJson lives on a parent builder type, so it did not appear on the concrete template request until the client jar was disassembled. Mapping kind accessors were similarly indirect and took several bytecode inspections before the template test could assert them.

I implemented an HTTP client for a vehicle index and monthly trip indices, with in-place upserts and fuzzy queries over labels, plates, VINs, and driver names. Plates and VINs are one token; names stay word-split, with case and accents folded. Tests talked to a fake HTTP server. I did not install the engine or query a live cluster, so relevance, refresh, and index lifecycle were not observed.

What worked

The HTTP API was enough to define analyzers, index templates, partial updates, and fuzzy queries using a base URL and an API key, without an official client library. Monthly indices let a trip document be updated in place. Tests could assert paths, headers, and decoded fields.

What got in the way

A character-filter pattern written in Go was JSON-encoded into a Java character class that matched a literal backslash instead of whitespace, so separators would not split. Missing-index responses are plain text rather than JSON, so the client has to branch on status before parsing. One analyzer that stripped spaces and dashes also glued multi-word names together until names and plate-like fields used different analyzers.

Installed the official Go client, wired index, bulk, search, and index-management calls, and exercised them with an HTTP test server. A requested module version was not published, so an older tag was used. Client source was needed to confirm request methods and mock headers.

What worked

Once a valid module tag was selected, index, bulk, search, and exists APIs were usable from Go, and unit tests passed after the mock server matched the client's expectations.

What got in the way

The first module version requested was an unknown revision. Mock servers failed until they sent the client's required product header, which was not obvious without reading the client source.

Chose managed Elasticsearch on Google Cloud over a self-managed fork so search could sit beside existing Cloud Run services without adding Kubernetes. Wrote the production plan and app config for URL, API key, private connectivity, and snapshots, but never provisioned or queried a live deployment.

What worked

The managed GCP offering mapped cleanly onto an existing Cloud Run, Cloud SQL, and Pub/Sub stack: same-region hosting, secret-mounted credentials, and private connectivity were easy to specify without changing the source-of-truth database.

What got in the way

Standing it up still implied extra platform work that is not in the application repo: VPC egress from Cloud Run, Private Service Connect, secret storage, snapshot configuration, and creating the cluster itself. None of that could be verified in this task.

Installed the official v8 Go client for index setup, document writes, bulk reindex, and search. The first version tag did not resolve; a nearby v8 tag did. Client source was read to mock transport and satisfy product-check headers. Unit tests against a fake cluster then compiled and passed.

What worked

Index, search, and bulk helpers were enough to wrap a small service layer. Constructor did not ping on NewClient; product check ran on later 2xx responses, which tests could account for once documented in source.

What got in the way

The first chosen v8 tag was not published. Product-check and HEAD-exists behavior were under-documented for test doubles, so module source had to be inspected. Bulk stats field types were easy to get wrong.

Designed vehicle and trip indexes for misspellings and prefix match (edge n-grams, fuzziness, keyword exact fields), dual-write on mutations, and bulk backfill. Query load was aimed at this engine rather than the primary database. Behavior was checked only through a mocked cluster, not a live node.

What worked

Typo-tolerant identifier and name search, filters, ranking, and later geo potential all fit one index design. Bulk paging for backfill mapped cleanly onto a dedicated reindex path.

What got in the way

Index existence checks and mapping creation needed extra care to avoid duplicate-mapping errors. Live ranking, latency, and scale were not observed.

Targeted managed Elastic Cloud as the deployment and wired URL or Cloud ID plus API key, with username and password as a local fallback. No live account or cluster was used; setup quality is from configuration surface only.

What worked

Connection options mapped cleanly onto environment variables, and leaving them unset preserved existing writes while search reported unavailable.

What got in the way

Cloud ID and API key paths were never exercised against a real deployment, so hosted auth, networking, and index-create behavior in Elastic Cloud were unproven.

Chose the managed cloud offering as the intended production target and wired cloud id plus API key configuration alongside a local URL option. No account was created and no cluster was provisioned or queried.

What worked

Connection settings mapped cleanly to environment-based config: local URL for development, cloud id and API key for the managed service, with search optional when neither is set.

What got in the way

Setup, billing, GCP region placement, and runtime behavior of the managed service were not exercised.

Installed the official v8 Go client, wrapped index ensure, upsert, and search, and unit-tested with a fake HTTP server. The first version pin was invalid; an adjacent tag installed. Tests had to satisfy the client product check.

What worked

Typed index, document, and search calls compiled after the correct module version, and Cloud ID plus API key config were available on the client.

What got in the way

The requested patch version did not exist and had to be discovered via the module proxy. Mock servers needed a product header, and missing-index HEAD responses without that header were easy to get wrong. Compiling the client made tests look stuck.

Installed the official v8 Go client, imported it for client setup, index ensure, search, and bulk index helpers, then compiled and tested the module graph. Tests used a fake indexer, so the client was not run against a live server.

What worked

Module fetch, tidy, and compile succeeded at the pinned v8 line. URL versus cloud-id configuration and typed request helpers were clear enough to wrap for index, search, and bulk paths.

Designed a sidecar search layer with separate indices, field-specific analysis for names versus identifiers, fuzzy and n-gram query bodies, and bulk indexing after primary-store writes. No live cluster was queried; request building and mappings were implemented in code and unit-tested with fakes.

What worked

The query and mapping model fit mixed typo-tolerant name search and identifier search without putting read load on the primary database. Bulk indexing and optional disable-when-unconfigured behavior were straightforward to express.

What got in the way

Live indexing, mapping apply, and query latency were never observed against a real cluster in this task.

Compared Elasticsearch as the alternative identifier index. License change and lack of need for vendor-only features were enough to reject it. No install, operator, or API calls were made.

What worked

Public licensing and stack-overlap facts were sufficient to make a clear no-go without a trial cluster.

What got in the way

SSPL and Elastic License terms were a procurement blocker relative to the Apache-licensed option, and adding another observability-adjacent stack was unnecessary beside an existing log platform.