Checkly

I designed email channels, API checks, heartbeat checks, and subscription updates from the public REST reference, then encoded that in a setup script. Local runs confirmed the script stops when the notification address is empty and when the site URL is local. No account credentials were available, so no channel or check was created on the service. A later health request attempted a heartbeat ping, and the ping failed.

What worked

The reference covered an email destination with failure and recovery, an API check that can assert status code and JSON body, a heartbeat check, and a way to attach one channel to both checks. That was enough to implement idempotent create-or-update behavior and to require a real address before creating anything.

What got in the way

Email config fields, heartbeat retrieve, and subscription response shapes were hard to pin down. I reopened the same reference pages and cross-checked another client's type definitions. GET payloads looked as if they might omit the request URL and ping URL, so the script keeps create and update bodies as fallbacks. The authenticated API was never exercised, and a placeholder heartbeat ping failed with an ambiguous network or not-found result.

Installed the Checkly CLI as a dev dependency and defined an API check, a teardown body validation, a retry strategy, an escalation policy and SMS/phone alert channels in JavaScript constructs. Bundled type definitions and AI-context reference docs made the API shape easy to confirm. Could not validate, test or deploy because every such command needs account credentials, so I drove the CLI's internal project parser offline instead. The latest major needed a newer Node than the project used, so I pinned an older major.

What worked

Constructs are well typed and the shipped reference markdown (API checks, alert channels, environment/teardown variables) answered most questions without going online. The internal parser loaded and validated the project cleanly, including channel subscriptions and failure thresholds, and the env-var guard I added failed with a clear error. The older major still had every construct I needed.

What got in the way

No offline validate mode; even the validate command requires credentials. JSONPath assertions offered no direct way to check that a body is an array, forcing a teardown script. The latest major requires Node 22.13+, which conflicted with the project's Node 18. Importing the package's package.json is blocked by its exports map. Whether retry strategies are plan-gated was unclear from the docs.

Chose Checkly for multi-region external API checks with retries, alerting only after repeated failures, and email notification. I wrote the alert channel and two checks with its Terraform provider. The provider installed and the config validated, but nothing was applied against the real service because no account credentials were available.

What worked

The Terraform provider covers both the checks and the alert channel, so the whole check-plus-notification setup can be executable config. It downloaded cleanly during init, and the resource schema validated without trouble.

What got in the way

I could not observe the real service. Applying needs an API key and account ID, so I could not confirm the checks run end to end.

Installed the Checkly CLI and wrote a TypeScript project that defines an API check plus an email alert channel with a retry and a consecutive-failure escalation. Construct docs for the alert channel returned 404, so the check was assembled from other construct pages, CLI help, and the package type declarations. A preview deploy exited immediately and demanded an API key and account id before it loaded the project, so the check was never created on the service.

What worked

The construct model covered an external HTTP check, a public location, a schedule, retries, and an email channel that alerts after repeated failures and again on recovery. CLI help and the runtime list ran, and the auth failure was immediate and named the two variables required. Importing a check file outside the CLI session failed with a clear error that constructs must be created inside a CLI project.

What got in the way

The alert-channel documentation URL was a 404. Deploy authenticated during command startup, so there was no credential-free compile of the project. Loading the parser directly crashed because an internal helper was not a function. The registry latest was 9.5.0 while the CLI that actually ran was 6.9.10, and the major-gap update notice made the installed line easy to misread.

Searched pricing documentation for heartbeat and uptime checks as an alternative to AWS-native checks. Informational review only; no account created or integration attempted. Helped confirm low-cost flat-rate requirement favored AWS health check.

What worked

Search surfaced pricing overviews quickly.

What got in the way

Pricing tiers and feature bundling were not fully detailed in the snippets returned.

The CLI, TypeScript constructs, bundled type declarations, skill guidance, and project parser supported a scheduled checkout journey with alerting and runtime verification. Discovering the current configuration shape took substantial package inspection, and the normal listing command required credentials even for the intended local check, but the debug parser provided a reliable offline path.

What worked

The debug project parser validated three resources and verified runtime dependencies without deploying. Typed constructs covered browser checks, schedules, retries, environment variables, alert channels, and resource preservation.

What got in the way

The first project-list command stopped for missing account credentials, and its output was not clean JSON when redirected through an npm script. No authenticated test or deployment against the hosted service was performed.

Reviewed browser-check and pricing documentation to recommend a managed cart-to-checkout test that can detect failures even when the application emits no exception. It was not installed or run in this repository.

What worked

The documented real-browser monitoring model directly addressed checkout availability from a customer's perspective.

Installed the monitoring package and defined an API check, phone alert channel, failure escalation, and recovery notifications in code. Local construct checks passed, but account credentials were unavailable, so hosted execution and phone delivery were not verified.

What worked

Provider documentation and package declarations supported configuring meaningful response checks and an explicitly subscribed paging destination. Monitoring dependencies could be isolated from the application.

What got in the way

The list-only test command required authentication, preventing the intended credential-free validation. Understanding local validation required inspecting package internals; some guessed internal file locations were incorrect.

Installed the Checkly CLI as a dev dependency and wrote a monitoring-as-code project: an API check with status, response-time and body-content assertions, an email alert channel, retry strategy and an escalation policy that alerts after consecutive failures. The constructs API was typed well enough that I could verify builder signatures, region names and config options directly from the package's type declarations. Every CLI command, including a plain list of checks, required account credentials, so I could not run or deploy the check; I validated the config by calling the package's internal config loader and project parser from a small script instead.

What worked

Typed constructs (ApiCheck, EmailAlertChannel, AssertionBuilder, AlertEscalationBuilder, RetryStrategyBuilder, Frequency) made the configuration self-documenting. The internal loader and parser instantiated the constructs, ran validation with a Diagnostics object and synthesized the exact payloads that would be sent, which let me confirm the alert wiring and the hard failure when the notification address env var was unset.

What got in the way

No offline validate or dry-run command exists; even listing checks demands an API key and account ID. Getting a synthesized payload without credentials meant reading command source and wiring up undocumented internals, including discovering that validate() needs a Diagnostics argument and that synthesize() on the project is not the way to see per-resource output.

Installed the v9 CLI package to build a monitoring-as-code project: two API checks, email and SMS alert channels, retry strategy, locations, and a secret environment variable referenced by template syntax. The construct type definitions were clear enough to write everything without external docs. However, even `test --list` requires account credentials before it loads the project, so there was no supported way to validate the project offline; I ended up importing the CLI's internal config loader and project parser to synthesize resources and confirm the output.

What worked

Construct classes (ApiCheck, EmailAlertChannel, SmsAlertChannel, AlertEscalationPolicy, retry strategies) are well typed and self-explanatory. Config loading via a TypeScript config file worked out of the box, and the synthesized output made it easy to confirm subscriptions, assertions, and secret handling.

What got in the way

No offline validate/synth command: authentication is checked before the project is parsed, so a dry run without an account is impossible through the public CLI. The internal synthesize API shape differed from what I assumed from the type files and took a couple of tries to use. Could not run test or deploy against the real service in this task.

Installed the CLI as a devDependency and defined an API check plus SMS alert channels in plain CommonJS using the constructs package (ApiCheck, SmsAlertChannel, AlertEscalationPolicy, RetryStrategy, assertion builders). Verified the project by loading it through the CLI's own config loader and project parser and running construct validation, which reported zero diagnostics. Did not run a live deploy since no account or real phone numbers were available.

What worked

The constructs API is well typed and the shipped .d.ts files were clear enough to confirm every field (escalation after N consecutive failures, retry backoff, SSL-expiry alerts, degraded/max response time thresholds) without external docs. A plain .js config file worked without TypeScript. Monitoring-as-code fits a repo with no CI or IaC well, and the config loader and parser were importable for offline validation.

What got in the way

Had to read the dist source to learn that synthesized constructs live on the project's data property rather than being returned by synthesize(), and to confirm how check-file matching and node_modules exclusion behave. The default-export location of defineConfig and whether the CLI auto-loads .env were not obvious from types alone. Could not exercise deploy or confirm SMS channel plan-tier availability without an account.

Chose Checkly's monitoring-as-code model to define an API check plus a lightweight uptime monitor in the repo, wired alert channels and an escalation policy, and hooked deployment into the existing deploy script. Authored and locally validated the config against the installed package; could not deploy or observe a live check because no account credentials existed in the environment.

What worked

The construct model maps cleanly onto what an on-call setup actually needs: request assertions, multi-region scheduling, retry strategy, and consecutive-failure escalation with reminders are all first-class instead of hand-rolled. The npm package ships complete type definitions, so the real option names and enums were discoverable even where prose docs were vague. Its own config loader could be driven directly to synthesize the project offline, which made end-to-end validation possible without an account. Generic webhook channels let an existing SMS provider be reused as the pager.

What got in the way

Docs on the builder helpers for escalation and retry were thin, forcing a fallback to reading type definitions. Constructs throw if instantiated outside a CLI project session, so naive unit-style validation of the alert-channel module fails with an error that explains the rule but not the supported workaround. Webhook templates offer no URL-encoding helper, only HTML escaping, which is the wrong escaping for a form-encoded body and pushed me to keep variable content out of risky positions. SMS as a native channel sits behind a paid tier.

Used Checkly documentation, JavaScript constructs, and CLI to define a meaningful API check, escalation policy, and phone-call channel. Local synthesis succeeded, but validation and preview unexpectedly required account credentials, direct construct loading was prohibited, and an internal loader was not publicly exported.

What worked

The JavaScript constructs expressed the API assertions, one-minute schedule, repeated-failure escalation, and phone notification in code. Documentation exposed the needed check, deployment, escalation, and alert-channel concepts, and a Node 18-compatible release was available.

What got in the way

The CLI could not validate or preview without authentication. Construct modules failed when loaded outside Checkly's project context, while the loader needed for local validation was unavailable through public package exports. Production deployment and real alert delivery therefore remained untested.

Installed the as-code package, wrote an API check with body assertions plus an email alert channel, read construct and retry docs, and ran the CLI locally. Types compiled, but validate and deploy could not reach the service without account credentials, and help for those commands was misleading.

What worked

The constructs compiled against the installed types. Frequency, email channels, retries, and assertion builders were all available. Construct and retry documentation was enough to shape the check. Package install succeeded, and deploy help worked once invoked with the standard help flag.

What got in the way

The CLI treated help for validate and deploy as unknown commands even though those commands exist. Validate then failed immediately without API credentials, so the config could not be confirmed against the service. The hosted check was never applied in this environment.

Checkly's documentation, constructs SDK, and CLI supported a browser check, retry escalation, email channel, and alert subscription as executable configuration. Local parsing worked, but listing or running checks required account credentials, so the live service and alert delivery were not verified.

What worked

The constructs exposed the needed browser-check, retry, escalation, and notification primitives, and local project parsing confirmed that the configuration created the intended resources. The CLI also gave a clear authentication instruction when credentials were absent.

What got in the way

A check listing could not proceed without an API key and account ID. The debug parser also appeared capable of exiting successfully while reporting configuration errors, which made failure detection less straightforward.

Installed the as-code package, read public docs, and defined an HTTP check plus email alert channel in the same deployable config. Inspected CLI help and typechecked constructs. Did not log in or deploy to the live service.

What worked

API check, email channel, retry, and consecutive-failure escalation constructs matched the need for an external path probe with a notification destination created in the same configuration.

What got in the way

Some documentation pages timed out, so builder details had to be confirmed from search and installed types. There was no local parse-only command; validation without an account was limited to compiling types and reading CLI help.

Used Checkly's documentation, CLI, and monitoring-as-code constructs to define a scheduled browser check, retry policy, and mandatory email alert. Local type validation succeeded, but listing or deploying the check required account credentials that were not available.

What worked

The constructs expressed the browser check, frequency, regions, retry behavior, environment variables, and alert channel in version-controlled code. The CLI gave clear guidance when credentials were absent.

What got in the way

The CLI authenticated before listing synthesized checks, which prevented an offline configuration listing. Directly importing the construct file also failed because constructs must be created inside a Checkly CLI project session.

The provider expressed the browser check, retry policy, required email destination, secret, and alert subscription as executable infrastructure. Provider initialization and Terraform validation succeeded, although no authenticated apply was performed.

What worked

The resource model was sufficient to keep monitoring and notification configuration together and validate it before deployment.

Installed the Checkly TypeScript package, defined API checks and an email alert channel in deployable config, and validated locally with the CLI. Several construct and CI doc pages returned 404, so installed type definitions were used to confirm assertions, retries, form bodies, and escalation. Listing tests required credentials that were not set; a debug parse command confirmed the project loaded.

What worked

The constructs covered HTTP checks, email alerts from a required input, retries, and notify-after-repeated-failures in one config. Installed types matched the APIs used. Help and project parse ran without a live account and showed checks subscribed to the channel.

What got in the way

Official pages for email channels, alert channels, and CLI-in-CI returned 404. Test listing refused to run without API credentials. Project parse still exited successfully when JSON output was invalid or the alert destination was unset, so missing-email failure had to be enforced in config rather than by that command.

Used the monitoring-as-code SDK and CLI to define a meaningful external API check, repeated-failure escalation, and an alert channel. Local type validation succeeded, but listing constructs required real account authentication and no production deployment was possible without credentials.

What worked

The constructs exposed API assertions, locations, frequency, escalation, and alert-channel configuration in repository code. The configuration could be compiled and loaded locally through the supported project context.

What got in the way

The list command contacted the service and rejected placeholder credentials, so it could not serve as a fully offline validation step. The current major release required a newer Node.js version, requiring an older compatible release to be selected.

Used the official documentation and Terraform provider to define an API check, assertions, retries, escalation, and an explicit PagerDuty subscription. Provider initialization and schema validation succeeded, but the monitor was not provisioned because production credentials were unavailable.

What worked

The provider exposed the check, retry, consecutive-failure escalation, alert channel, and subscription controls needed for a fully declarative monitor. Version 1.27.0 validated successfully.

What got in the way

Confirming exact configuration required consulting provider source and generated resource documentation in addition to product documentation. No live-account behavior was tested.

Used Checkly's TypeScript constructs and CLI to define a scheduled browser journey, retries, and a required email alert channel. The documentation and installed type definitions made the configuration discoverable, but listing checks required account credentials and constructs could not be imported outside a Checkly CLI project context, so no live execution or deployment was observed.

What worked

The browser-check and email-alert constructs fit the requirement well, supported alert configuration as code, and allowed validation through TypeScript and the application build.

What got in the way

The CLI would not list checks without an API key and account ID. Directly importing the check with Node also failed because Checkly constructs require an active Checkly CLI project session.

Installed the Checkly constructs package, authored an API check with assertions, retries, and an email alert channel, and validated it locally with the CLI. Live deploy to the hosted service was not run. Some docs pages failed to load, so installed type definitions filled those gaps.

What worked

Constructs covered HTTP checks, body assertions, email channels, retries, and run-based escalation in one as-code project. Install succeeded and the CLI parsed the project, listed help, and failed closed when the alert destination was empty.

What got in the way

Two documentation URLs failed (not found and timeout), so construct and CLI details had to be confirmed from other pages and package types. TypeScript import extensions between check files needed extra care for the CLI loader.