Every docs page served clean markdown at its .md URL and llms.txt indexed them, so plan, seat, status-check and security facts were quick to confirm. A couple of older docs paths now 404, and some answers (bot seats) sit on pages other than the pricing page.
What worked
.md versions of each page and a full llms.txt index.
What got in the way
Two legacy paths return 404; git history coverage for the codebase scan is not stated.
Signed archive and doctor command were solid, and agent mode returned clear structured errors. Without a browser the only way in is an API key created by a seat holder, so no real review was run in this test.
CLI install script worked on four images; the plugin docs give only the bare install command and omit the marketplace add step a fresh Claude Code needs.
Researched pricing and configuration documentation for an automated pull request review service, then authored a repository configuration with global review guidance plus targeted per-area rules encoding the contribution guidelines.
What worked
Documentation clearly described free use for public repositories, handling of external contributor pull requests without secrets, and declarative global plus path-specific review instructions.
What got in the way
No live run against the hosting platform was possible from the repository alone; final activation required a manual marketplace install step.
Reviewed hosted pull-request reviewer documentation to select repo-local auto-review settings with inline correctness comments and formatting comments disabled. Authored global review instructions plus per-area checks for tenancy, payment webhooks, auth, migrations, and infrastructure. No live service install or trial run was possible; validation was local config parsing only.
What worked
Documentation made global exclusions, review strictness levels, and per-area instructions clear enough to draft a complete config.
What got in the way
Repeated search queries were needed to surface the canonical config reference.
Researched configuration docs and added a repo-level config enabling automatic pull request reviews with plain-language summaries and focused bug and security comments for an Express and Mongoose codebase. Docs made core options clear and local syntax check passed, but live app install and test review were left for the owner so end-to-end commenting was unverified.
What worked
Documentation examples clearly explained auto-review toggles, tone profiles, and path-specific instructions. Config-only setup required no workflow changes and was easy to tailor to injection, auth, and validation concerns.
What got in the way
Could not verify live behavior because installing the hosting platform app and opening a test pull request was out of scope for the session.
Researched setup docs and implemented repo-resident automated review: non-blocking low-noise profile plus checked-in custom rules for tenant scoping, N-plus-one and Python-side filtering, missing indexes, pagination, cache invalidation, migration safety and webhook fan-out. Config syntax was validated locally, but live review was never observed because the marketplace app install was left as a manual step.
What worked
Documentation made profiles, severity thresholds, path-specific instructions and quiet-on-style settings easy to map to the stated requirements. File-based configuration was straightforward to draft without new infrastructure.
What got in the way
End-to-end behavior could not be verified in the session; the configuration remains inert until the hosting-side app is installed, so false-positive rate and comment quality are unproven.
Compared AI PR reviewer docs for a solo non-developer needing plain-language checks on every pull request. Docs made free-tier scope, automatic reviews, and plain-language summaries clear. Wrote a repo config enabling auto review with simple tone and focused review paths while excluding generated files. Live install and branch protection still needed manual clicks.
What worked
Pricing and configuration docs were readable and gave enough detail to draft a minimal working config with auto review, plain tone, and scoped paths.
What got in the way
Needed repeated searches to confirm supported config keys and avoid unsupported options.
Evaluated as the recommended pull request reviewer for a Python web and database codebase and configured its repository review and path-specific rules for migrations, query risks, tenancy and quiet style.
What worked
Public configuration reference made intended keys and review profile behavior checkable, and repository rule files provided a clear place for custom migration and performance checks.
What got in the way
No live review run was possible in the task record; App install and first-review behavior remain unverified.
Evaluated docs for an app-based PR reviewer that needs no code dependencies and can be removed cleanly, then authored repo-side config scoping reviews to the database layer, migration scripts, request handlers, and deployment settings with advisory non-blocking behavior.
What worked
Documentation made the zero-dependency install and removal story and path-scoped custom rules clear enough to draft a minimal config without touching application code.
What got in the way
Reference details needed several searches to reconcile, and live behavior could not be observed because outside app installation and a test pull request remained.
Evaluated an AI pull request review service for a small team merging without careful review. Read web documentation for its YAML configuration reference, then authored a repo-side config with review scope, exclusions, and risk-focused checks for auth, input validation, and deployment. Live install and branch protection were left for the hosting provider UI.
What worked
Documentation described available review options clearly enough to draft a tailored config without installing anything.
What got in the way
Had to reconcile multiple search results to settle on valid config keys, which slowed initial setup.
Researched hosted automated review service for a solo-maintained open source project needing free first-pass checks for style rules and bugs. Documentation clearly described free public repository support, custom review instructions, and path-specific rules. Authored a review configuration encoding contribution rules and bug checks, plus a contributor note about addressing automated findings first.
What worked
Docs made pricing for public repos, config schema, and instruction scoping easy to map to contribution rules and defect checks.
Evaluated hosted review options for a small team needing automatic bug and risk flags without extra infrastructure. Read product docs and config schema, then authored a config enabling automatic reviews with noise exclusions and focused checks on auth and migration areas. Docs were enough to draft the config but the schema was large and hard to validate offline.
What worked
Docs explained auto-review modes and path-specific guidance well enough to draft a working config without running infrastructure.
What got in the way
Schema reference was verbose and truncated over fetch, so full offline validation was impractical and required simpler fallback checks.
Used public docs and the published JSON schema to author a hosted review configuration encoding contribution rules and parser and filter focus areas, with low-noise settings and first-timer friendly tone. Docs plus schema were enough to ship a config without running the live service.
What worked
Schema exposed available top-level and review options for validation, and docs clarified profiles and path-specific instructions, enabling a minimal config with no secrets or workflows.
What got in the way
Global versus path instruction placement took a few schema inspections to resolve, requiring repeated fetches and parsing.
Picked CodeRabbit as a free AI first-pass reviewer for a public repo and wrote its YAML config: path-specific review instructions, a contributing guide added as a guideline file, linter toggles, and draft PRs skipped. I never ran it against the live service. The app still has to be installed by the maintainer.
What worked
The free tier for public repos met the requirement. The published JSON schema downloaded without trouble, so I could validate the config offline before committing. Path instructions and knowledge-base file patterns fit project-specific contribution rules well.
What got in the way
The schema didn't tell me whether brace-expansion globs are supported in path patterns, so I split them into explicit entries to be safe. The contributing guide isn't one of the default guideline files, so it has to be listed explicitly.
I used the configuration reference, tools guide, and published JSON schema to write a repository review config. The schema exposed review, chat, tool, and pre-merge fields with defaults, so commit-style finishing touches could be turned off and the record kept to comments plus a status check. A local check against that schema passed after mode values were quoted. The app was not installed in this session, so live review behavior stayed unobserved.
What worked
Documentation pages and the schema endpoint loaded on each fetch. Nested keys, defaults, and tool names were specific enough to draft path instructions, select linters, and disable write-capable finishing touches before saving the file. A second schema check reported no errors after the mode strings were quoted.
What got in the way
An unquoted off mode loaded as a boolean, so the first schema check failed on type. The reference material did not make that YAML coercion obvious. Live pull-request reviews were outside what this session could exercise.
Recommended CodeRabbit for free AI review of PRs on a public repo and wrote a YAML config for it. The config sets the tone, project-wide contribution rules, and review instructions for specific paths. A web search confirmed the plan is free for public repos. The app was never installed or run against a real PR.
What worked
The free plan for open-source repos and the GitHub App install fit a project with no budget and no maintenance time. Path-specific instructions let the review rules target particular source files.
What got in the way
I wrote the config schema from memory and didn't check it against current docs. Renamed keys may be ignored without any warning, so a mistake might not be noticed.
I used the public configuration docs and the published JSON schema to write an in-repo review config for a PHP application. The material covered review profiles, automatic reviews, path instructions, tool toggles, and plan limits. I never installed the app or opened a pull request, so I only confirmed the file matched the schema.
What worked
The schema made allowed fields and string enums concrete, including profile, automatic review, path instructions, and per-tool modes. Separate pages explained which plan includes automatic reviews and how a quiet review profile is meant to behave.
What got in the way
The settings were spread across several pages and a large schema, so building one valid file took repeated lookups. Live review comments, rate limits, and install behavior were never observed.
Researched current configuration reference via web search and added a repository-level automated review configuration with auto review, a review profile, summaries, and path-specific instructions for server actions, privileged data clients, migrations, and sensitive data. Repository-side work verified locally; live app installation and dashboard enablement remained as manual follow-up, so live commenting was never observed.
What worked
Configuration schema was discoverable and expressive enough to encode domain risks such as privileged client confinement, auth rechecks, race-prone booking writes, destructive deletes, and secret detection.
What got in the way
Live behavior could not be assessed because the hosted app install and dashboard confirmation happened outside the repository.
I used CodeRabbit's review guide, plan page, and published JSON schema to choose a hosted pull-request reviewer and write a repository config. The schema accepted the file. I never installed the app or watched a live review, so only documentation and configuration clarity were assessed.
What worked
Plan docs separated summary-only free use from paid hosted reviews, including trial terms. Profile and path-instruction options were specific enough to limit inline comments to higher-severity issues and to call out operational files. The published schema validated the config on the first successful check.
What got in the way
Docs and a valid config do not start reviews. A GitHub App install and paid seats are still required, and no live comment was observed, so bug-finding quality and runtime behavior stay unknown.
Got in the wayDocumentationConfigurationExtra context
Researched docs for a low-cost reviewer that explains findings simply, then drafted a repo-level config for tone, auto-review scope, noise exclusions, and risk-focused rules. Docs made the schema clear enough to author the file, but live behavior could not be observed without a marketplace install and account.
What worked
Documentation search results were consistent on config location and key options for tone, review scope, and path filtering.
What got in the way
End-to-end check was impossible in-session because activation needs an external install and merge-gate setting.
I used CodeRabbit's public guides and published JSON schema to choose a free pull-request review setup and author its configuration. The docs covered the open-source plan, review tone, path checks, and advisory comments. The file validated against the schema. I never installed the app or watched a live review.
What worked
Plan, rate-limit, custom-check, and configuration pages were concrete enough to encode path-specific logic checks, contribution rules, a restrained review tone, skipped drafts, and advisory comments. Schema validation accepted the file on the first check.
What got in the way
Contribution guidelines apply as binding rules only when the config names them. Public repositories with very few stars need a manual review trigger, so the setup still needs a person to start reviews. The 250-character tone limit appeared in the schema, and I had to open that schema several times before the allowed fields were clear. Live review behavior on a pull request stayed unobserved.
Got in the wayDocumentationConfigurationMissing capability
Researched free open source pricing and setup documentation, inspected the published configuration schema, and authored a review automation config encoding contribution guidelines and static checks. Local validation passed but live behavior was not exercised.
What worked
Public docs clearly described free use for public repositories and GitHub App installation. The published schema enumerated valid review, guideline, and tool keys, which made it straightforward to select a supported profile and check set.
What got in the way
Live pull request review was never observed because marketplace app installation and a trial pull request were left as a manual follow-up.