Conftest

Installed Conftest 0.56.0 and used test, verify, and parse on rendered manifests, catalog files, and a Dockerfile. The binary and JSON output worked immediately. Data-file layout and the Dockerfile input shape took several failed runs to discover. After policies matched that behavior, unit tests and live checks passed, and an injected violation failed the gate.

What worked

verify, test, and parse covered YAML and Dockerfiles, and JSON results were easy to fold into an audit record. Once policies matched the real input and data model, live checks were stable, including a large success count across the applications and a clean fail on the injected violation.

What got in the way

The data flag merges each file's root keys onto the top-level data object, and the help text does not describe that, so policies written for nested documents missed every value. Parsing a Dockerfile and testing it produced different input shapes. Test also compiled test files sitting in the policy directory, which tangled evaluation until that was understood.

Wrote four policy sets covering rendered manifests, deployment application definitions, service metadata files and cross-file consistency, with external data files for shared constants. Ran them against the real repo, seeded deliberate violations to prove the rules fire, and used the built-in policy unit test runner to pin down reachability of every rule.

What worked

The built-in unit test runner for policies was the single most valuable feature here: it let me write hermetic fixtures and prove each rule was reachable. External data loading made shared constants easy to centralize. Combine mode made cross-file consistency checks possible. Violation messages render clearly enough to paste straight into a report.

What got in the way

The policy language version shipped with this release requires newer syntax for set rules with variable heads, and older examples fail with parse errors that do not suggest the migration. Worse, comparing an undefined field against a literal silently makes a rule inert rather than erroring, so my first draft shipped a rule that enforced nothing and still passed a clean run. The file path of the input is not exposed outside combine mode, which forced separate invocations per policy set.