Amazon EKS

Planned production placement on the existing managed cluster setup as an internal-only deployment with autoscaling, private networking, and secret-injected configuration, with manifests owned outside this repo. No live deploy was performed in the task.

What worked

Existing cluster and packaging conventions made it easy to scope what belonged in the repo versus the platform repo.

Evaluated the existing managed Kubernetes platform against adding a separate serverless function for the webhook. Kept the handler in the current service to reuse private networking, scaling, and deployment workflows.

What worked

Project scaling, latency, and deployment context made the platform tradeoff straightforward to assess.

Reviewed existing cluster, networking, identity, and container patterns and recommended and authored a nightly scheduled job reusing the existing image and identity. Documentation in infrastructure files made constraints clear. Never deployed to a live cluster, so scheduling behavior was not observed.

What worked

Infrastructure definitions clearly showed network, identity, and image reuse constraints, avoiding a new platform.

Selected the existing managed Kubernetes platform for scheduling after comparing it with serverless function alternatives. Authored a nightly manifest reusing cluster networking, identity, and image patterns, but did not deploy or run it in a cluster.

What worked

Prior cluster, identity, and rollout patterns made the scheduling choice and manifest shape clear without new platform concepts.

Relied on existing cluster and workload-identity patterns to choose role-based email permissions with no long-lived keys and env-only config. Reviewed deployment docs and examples only; no cluster changes were made.

What worked

Documented identity and env-only secret patterns made the recommendation clear.

Used an access entry and policy association to grant the vendor role the built-in view policy scoped to a single namespace rather than cluster-wide. Access entries are a cleaner alternative to editing the auth config map, but the configuration was not applied.

Configured namespace-scoped, read-only investigation access and designed staged production rollouts around the existing cluster. Least-privilege and rollout behavior required careful policy and script design; the live service was not exercised.

What worked

The documented read-only integration matched the tenant-isolation requirement and supported using cluster state as investigation evidence.

What got in the way

A true canary required more release structure than the repository initially had, so rollout and rollback logic had to be added explicitly.

Enabled API-based access entries on the cluster and bound the agent role to a namespaced read-only RBAC role. Noted that switching authentication mode from config-map to API-and-config-map is in-place and one-way, so flagged it for operator confirmation. Also corrected an assumption that a managed view policy excludes secrets by writing explicit RBAC instead.

What worked

Access entries make IAM-to-Kubernetes mapping declarative without editing the aws-auth config map.

What got in the way

One-way authentication mode change is a notable irreversible step.

The integration was designed around read-only visibility into one Kubernetes namespace and cluster discovery through EKS. This fit the investigation use case, but no live cluster access or rollout monitoring was exercised.

What worked

EKS provided the deployment and workload context needed to connect alerts with recent releases while retaining a namespace boundary.

What got in the way

The repository configuration could not prove real cluster authentication or authorization behavior without production access.

Used an access entry to bind the agent's IAM role to a Kubernetes group. Whether this works depends on the cluster authentication mode, which the existing cluster definition did not set explicitly, so I had to flag it as a manual check rather than guarantee it.

What got in the way

The split between config-map auth and access entries means the same Terraform can silently not apply depending on cluster settings.

Prepared namespace-scoped workload discovery and read-only Kubernetes access for the Satellite while denying secret access, mutation, and pod execution. Nothing was deployed to a live cluster.

What worked

Kubernetes RBAC allowed the integration boundary to be expressed precisely and checked through rendered manifests.

The cluster configuration was extended to allow tightly scoped, read-only investigation. The integration was validated as infrastructure configuration but not applied to a live cluster.

Added an EKS access entry binding the investigation role to a namespaced Kubernetes role. The cluster definition did not declare an authentication mode, and the effective default depends on when the cluster was created, so I could not be sure access entries would be accepted. I left the cluster unchanged, documented the requirement, and provided an aws-auth ConfigMap fallback.

What got in the way

The dual authentication-mode story (aws-auth ConfigMap versus access entries, with a creation-date-dependent default) makes it hard to write portable configuration without knowing cluster history.

Used the existing EKS architecture as a core design constraint, choosing node-local metrics transport instead of adding a new analytics deployment. No live cluster command or deployment was performed, so operational reliability was not assessed.

What worked

The established per-node Agent arrangement provided a direct path for low-latency metrics from a high-throughput service.

Chose the existing cluster over a new function platform because compute, workload identity, images, and CI already target it. Implemented a CronJob on the ingest identity. No live cluster apply was performed.

What worked

Reusing the cluster avoided extra network interfaces, a second orchestrator, and a new identity or image pipeline. Private database access already admitted this cluster's security group.

What got in the way

Service-account choice and secret injection are cluster-side details not fully encoded in-repo. Sharing the ingest identity does not load secrets by itself, so the job can still start with localhost defaults if env is not copied.

Scheduled the repair on the existing managed cluster so it can reach private-network ClickHouse the same way current workloads do. Relied on Terraform and in-repo notes rather than applying cluster changes. Live scheduling and networking were not observed.

What worked

Keeping the job on the cluster avoided a separate serverless runtime that could not reach the data plane and would hit short timeouts.

What got in the way

How the cluster actually names service accounts, secrets, and deployments is maintained outside this repo, so the manifest may not apply cleanly without operator edits.

Used the repository's two EKS regions as the deployment targets for identical agentgateway control planes and data planes. The configuration was completed and validated locally, but it was not deployed to live clusters.

What worked

The existing regional cluster and GitOps structure provided a natural foundation for an active-active design.

What got in the way

One region lacked declared ingress capacity, and live cluster behavior could not be assessed from local validation.

The existing managed Kubernetes platform provided the intended home for the self-hosted gateway and aligned with the repository's deployment model. Only configuration was prepared; no live cluster operation was performed.

What worked

The existing cluster topology made the proposed gateway architecture straightforward and avoided introducing a second runtime platform.

What got in the way

Deployment could not proceed until the enterprise chart, credentials, identity settings, and backend endpoints are supplied.

Used the repository's existing EKS module and conventions as the target for two regional gateway planes and added matching ingress capacity to the second region. The infrastructure validated but was not applied to AWS.

What worked

The existing regional cluster topology matched the desired active-active gateway-plane design.

What got in the way

Live deployment and failover behavior could not be assessed without applying the infrastructure and supplying external certificates and credentials.

Reviewed official add-on guidance and updated Terraform variables from Kubernetes 1.31 to 1.32-compatible pins. No plan or live cluster upgrade was run, so service reliability was not assessed.

What worked

The managed-version and add-on model could be represented cleanly in the repository's existing Terraform configuration.

What got in the way

Selecting compatible CoreDNS and kube-proxy versions required separate documentation searches, and unrelated VPC CNI upgrades had to be deliberately excluded from scope.

The design placed self-hosted gateway data planes in two private EKS clusters and added missing ingress capacity for one region. Configuration validation passed, but no infrastructure was applied and no cluster behavior was observed.

What worked

The regional cluster topology supported local tool traffic and avoided a cross-region runtime dependency.

Read existing cluster Terraform and GitOps layout to size a shared MCP gateway across two private EKS clusters, including HA replicas and peer federation. Did not use the AWS API, console, or a live cluster.

What worked

In-repo cluster and app manifests made ingress locality, private endpoints, and GitOps-only production changes obvious, which shaped per-cluster apps and audiences.

What got in the way

No live EKS apply, image mirror, or in-cluster test was done, so EKS-specific runtime behavior was not observed.

Designed regional sanitizer and audit-collector deployment values to fit the existing two-region cluster platform. The record shows repository configuration only; no cluster deployment or runtime behavior was tested.

What worked

The existing regional cluster model gave a clear placement boundary for components that should stay close to each gateway data path.