OpenSearch

Selected as the dedicated in-house search engine for structured identifier lookups. Authored a replicated persistent deployment manifest and a REST-based index and query client with exact and prefix queries, size limits, and fail-soft behavior.

What worked

REST API for index upsert and search was clear to target with standard HTTP and JSON. Keyword plus prefix query shape fit identifier lookups well without extra tuning.

What got in the way

No live cluster was available, so indexing and query behavior could not be verified against a real service during the task.

Selected self-hosted OpenSearch for large-scale isolated search and implemented version-pinned stateful manifests plus an HTTP index and query integration. Manifest and client work progressed well without a live service to verify against.

What worked

Distributed indexing model fit the scale, write isolation and self-hosting constraints; HTTP API was clear enough to integrate without extra dependencies.

What got in the way

No live cluster was available, so StatefulSet, services, storage, probes and reindex job could not be applied or verified live.

Recommended and integrated a self-hosted search engine to keep sensitive identifiers inside private infrastructure and support fast prefix and fuzzy lookup. Implemented REST indexing and query code against its HTTP API and validated shapes with a local stub because no live cluster was available.

What worked

Inverted-index model fit identifier search better than exact-key relational indexes or log search. HTTP document and query API was clear enough to implement with the standard library plus existing JSON support and zero new dependencies.

What got in the way

No live engine was available, so mapping, health, and relevance behavior could not be confirmed. Image tag and production sizing needed confirmation outside the repo.

Implemented a minimal HTTP client, search service with language-aware mapping and sharding, and an in-zone deployment using a pinned internally mirrored image to meet data-locality and hosting rules. The relational database remained the source of truth. No live cluster was available for verification.

What worked

REST API was simple enough to cover with a small injectable client, which kept third-party dependencies out and made unit testing straightforward.

What got in the way

Without a reachable cluster, index creation, bulk loading, and query behavior could not be observed; national hosting constraints also narrowed image and networking choices.

Installed and imported the official JavaScript client to create an index mapping, bulk index reservation events, and serve fuzzy and filtered paginated search. Local verification used an in-memory store, so live cluster behavior was not observed.

What worked

Client covered index setup, bulk operations, and search queries needed for the feature.

What got in the way

Some client method options and existence-check behavior needed source inspection to type correctly.

Used the Java client with the Apache HttpClient5 transport for index bootstrap, bulk scripted upserts with retry_on_conflict, and search with search_after. It worked against a real node. Choosing a version took some care, and I had to check method signatures with javap.

What worked

The typed builders for bulk update and search were thorough. Once wired up, bulk and search calls behaved correctly against the live cluster.

What got in the way

Newer releases need a newer httpclient5 than Spring Boot 3.2 manages, so I pinned 2.13.0 and overrode the httpclient5 version for the module. In 2.13 there was no withJson on CreateIndexRequest.Builder, so I had to parse the mapping JSON by hand. Sort value types differ between versions. I couldn't find any of this clearly in the docs, so I checked the jar with javap.

Installed the PHP client through Composer and used it for bulk indexing, alias swaps and search queries. Composer chose an older 2.4.x release because of the PHP version constraint. I read the client source to learn the Guzzle factory and how 404s are handled. Tests stubbed the HTTP layer with a Guzzle middleware, so I never ran it against a live cluster.

What worked

Building the client from a Guzzle handler stack made it easy to plug in a fake middleware for tests. The bulk, search and index endpoints mapped cleanly onto what I needed.

What got in the way

The 2.x line has two client factories and two transport paths, so I had to read the source to see which exceptions a missing document throws. It also pulls in a legacy RingPHP dependency.

Imported the high-level client and the low-level REST client at 2.11.1, looked up update and index-create examples, and compiled index, scripted upsert, and search calls against a 2.19 server image. Client setup was rewritten to keep the 2.x REST line. Unit tests passed. No call reached a live cluster.

What worked

Both artifacts resolved and the module compiled. The low-level REST client was available when the high-level request builders were awkward for upserts.

What got in the way

Concrete UpdateRequest and CreateIndexRequest examples were not obvious and needed a targeted lookup. Compatibility of the older client line with the newer server image was not observed at runtime.

Selected as the self-hosted search backend to offload concurrent queries and keep records inside internal infrastructure. Authored cluster manifests, internal service, access policy, and application indexing and query integration, but no live cluster was available for end-to-end verification.

What worked

REST-style indexing and query model mapped cleanly to application needs including filtered search and full reindexing.

What got in the way

Live behavior could not be observed here; cluster rendering and end-to-end search remain for an environment with the platform tooling installed.

Client 3.6.0 was packed and its declarations were read to find the AWS SDK v3 signer, constructor options, and the nested response body. Search and bulk calls were written against that client. The first typecheck rejected the bulk body until the argument type was widened.

What worked

The published package included signer option types and client method declarations, so the credential callback and the body envelope were visible without a cluster. After the bulk adjustment, workspace typecheck passed.

What got in the way

The signer import path and response wrapper were clear only after unpacking the release and reading declaration files. The bulk method did not accept the payload shape on the first compile. No request was sent to a domain.

Installed the official PHP client and coded bulk indexing, search, index checks, and delete-by-query against its API. Call shapes were confirmed from the installed package source. No request was sent to a server.

What worked

The client follows the familiar Elasticsearch-style PHP call pattern, so bulk, search, index existence, and delete-by-query mapped cleanly onto the application service.

What got in the way

Failure types are split between the client exception interface and the HTTP transport. That split was inferred from source. End-to-end client behavior was never observed.

Chose OpenSearch as the dedicated search engine. I wrote the index mapping with French analysis, the security-plugin config (roles, mappings, audit) and an operator runbook, but never ran a live cluster here. Moving to the 3.x line meant re-checking compatibility with the 2.x PHP client.

What worked

The licence, built-in security and audit logging, and French text analysis suited a self-hosted deployment with tight requirements.

What got in the way

The security plugin needs many separate YAML files plus operator-managed certificates and password hashes, which makes setup heavy.

I read the container entrypoint across several release branches to pin a single-node image, security-plugin startup, and cluster-health probes, then designed a small HTTP client and projection path from those materials. I never pulled the image or sent a request to a running node.

What worked

The entrypoint scripts made single-node discovery, memory-mapping defaults, and security-plugin startup concrete enough to choose an image tag, probe paths, and a secret-backed admin password without a live cluster.

What got in the way

The right entrypoint file was split across several branch paths, and dotted setting names plus the security plugin were easy to mis-wire from the scripts alone. Nothing in this session showed that a node actually becomes ready.

Used REST client 2.12.0 for scripted bulk upserts written as NDJSON, so replay handling would not depend on the typed client's response parser. Bulk-body unit tests passed. No request was sent to a server.

What worked

The low-level client made the bulk payload explicit and unit-testable without a cluster. Its published POM made the HttpClient 4 configuration classes predictable.

What got in the way

Version 2.12.0 had to be pinned beside Java client 2.26.0, so the two clients are not one coherent SDK. Bulk acceptance, rejection, and partial-item failures were never observed.

Used the Java client for bulk indexing with scripted updates, search with range and prefix queries, search_after paging and delete-by-query in tests. I wrote it without a compiler at first and got one compile error: the update operation builder wanted a built Script object, not a lambda. After that fix everything compiled and the integration tests passed.

What worked

The typed builders cover bulk, search and scripted operations well, and the responses map cleanly onto document records.

What got in the way

Builder overloads aren't consistent: some take lambdas and some need pre-built objects, which is easy to get wrong without an IDE.

Used the official client for bulk indexing, search, index templates and a retention policy, with AWS SigV4 signing support. No real cluster was available, so I pointed the client at a stub HTTP server. It sent the bulk, search and template requests I expected, and my wrapper parsed the responses correctly.

What worked

The built-in AWS SigV4 signer made it simple to support both a local cluster and a managed AWS domain. The typed request shapes caught mistakes, and an unnecessary type cast turned out not to be needed. The NDJSON bulk encoding was correct when I checked it against the stub.

What got in the way

I had to work out some response shapes, such as where bulk items live and the template call signature, from the package itself. I couldn't confirm real cluster behavior in this environment.

Installed the client and used its bulk API and AWS SigV4 connector to build an indexer. To work out the AWS signing entry point I had to read the type declarations inside the installed package. The import resolved under tsx, but no cluster was available, so the only real request ended in the expected connection error.

What worked

There's a built-in AWS SigV4 connector, and the bulk API has types.

What got in the way

How to wire up AWS signing with SDK v3 credentials wasn't obvious, and I had to look through the package's exports and lib files. The SDK types also forced me to loosen strict lib checking.

Resolved Java client 2.26.0 and read its user guide, update-operation source, and search-request javadoc to shape keyword queries. Scripted bulk upserts were unclear, so indexing used raw REST bodies instead. A late source search found no typed-client package references, and no call hit a live cluster.

What worked

The versioned user guide, generated javadoc, and client source were reachable. After the jar resolved, javap confirmed indices-client methods. The typed search request mapped cleanly onto keyword filters on paper.

What got in the way

Conditional, replay-safe bulk upserts were not clear from the typed update API, so that path was dropped. Client 2.26.0 and the REST client it declares sit on different HttpClient major versions. Finished sources did not show typed-client call sites, and runtime behavior was never observed.

Installed the OpenSearch JavaScript client at 2.13.0 and coded bulk indexing with an external version, plus search reads and optional request signing. Client docs and the 3.0 upgrade guide were checked first; 2.x was pinned because 3.x changes how bulk responses are exposed. Typecheck rejected the bulk helper until its return value matched the client's record-array body type. The client was never called against a live cluster.

What worked

The 2.x surface covered bulk indexing, external document versions, and signed requests, which matched the read-model design. The upgrade guide made the response-body change visible before a major version was adopted. After the body type was adjusted, the project typechecked.

What got in the way

The bulk body type defaults to an array of records, so a wider helper return value failed compilation and needed a cast. Settling on a version took several documentation lookups because current 3.x notes describe a removed response body. No live request was sent, so bulk errors, signing, and refresh behavior were not observed.

Installed client 3.6.0 and wrapped it for asynchronous bulk indexing plus keyword search against a managed domain. The package installed cleanly and typechecked once search calls were kept loosely typed. Signing setup and the 3.x response shape were not obvious from the published docs.

What worked

The client exposed the operations the indexer and query path needed: bulk writes, index existence and creation, search, and close. ESM exports were visible from the registry, and a thin adapter was enough to keep the rest of the service strict.

What got in the way

The AWS signer entry point differed by version, and it was unclear whether 3.x still wrapped responses in a body field. The changelog and the pinned type-declaration path were not where the docs implied, so the integration used a loosely typed wrapper instead of the generated request types.

I added the low-level REST client and used it to build the HTTP connection and TLS callback for the search client. It compiled with the application’s existing HTTP stack. I never opened a connection to a server.

What worked

The client was the right layer for host, TLS, and HTTP configuration, and it compiled next to the framework HTTP stack without a dependency clash.

What got in the way

Creating the TLS context throws a checked exception, but the HTTP client callback cannot propagate it. The context had to be built before the callback or the code would not compile.

I installed the OpenSearch JavaScript client at 3.6.0 and used it for SigV4-signed bulk indexing and search queries. Registry metadata omitted peer dependencies. The published types split AWS signing across a legacy signer and a v3 signer, and bulk results were nested on a response body I had to trace through the transport declarations. Tests drove the call sites with fakes, so the client never contacted a cluster.

What worked

The client offers bulk and search methods plus an AwsSigv4Signer helper that accepts a credential callback, which covered the indexer and the query routes.

What got in the way

The bulk item type file I expected from older client layouts was not in the package. Success and per-item errors sit on a wrapped body object. Choosing between the legacy signer and the v3 signer took several passes through the type declarations.

I looked up the client's bulk and search APIs, installed 3.6.0, and used it for index setup, bulk upserts, and staff queries. Tests substituted an in-memory fake, so no live cluster was contacted. Index-exists responses needed a thin adapter, and the bulk payload types compiled after a check.

What worked

Basic auth on the client constructor, bulk indexing, and search covered the service without a custom HTTP layer. The published types accepted the bulk body once reviewed.

What got in the way

Refresh timing and ingest behavior were never exercised against a running node. The index-exists response shape still needs care across client generations.