Skip to main content
| CodeRabbit Verify tests how your application behaves, not only how its code reads. It starts your app in an isolated sandbox prepared from a coding environment, uses it the way a user would, mostly through a browser, and through its CLI or HTTP API when the change calls for that. It reports what it found: bugs with reproduction steps, blockers that kept it from testing, and the screenshots and session recordings behind each result.

Two ways to run Verify

Pull request runs don’t use your saved user journeys, and user journey runs don’t post on pull requests. See Test pull requests and User journeys.

How a run works

A run moves through these stages. On the Activity page, a waiting run shows Queued, then In progress with the stage name.
  1. Queued: the run waits for its turn, for example while another run on the same pull request finishes or while a sandbox becomes available.
  2. Inspecting and Preparing: Verify finds the repository’s environment and restores its prepared snapshot in a new sandbox, with the code to test checked out.
  3. Generating (pull request runs only): Verify reads the diff and explores the repository, and writes test scenarios for the behavior the change affects. If the change has no material effect, Verify can select no scenarios, and the run is Skipped.
  4. Testing: Verify starts your app with the environment’s startup script, confirms that it is ready, and works through each scenario or user journey. It takes screenshots and records the browser session as it goes.
  5. Aggregating and Publishing: Verify gives each scenario or journey one outcome, links each bug to the same bug found in earlier runs of the repository, and publishes the results.

Key concepts

Where Verify lives

  • In the CodeRabbit app: select Verify in the sidebar to open the list of repositories you’ve added. A repository’s page has two groups: Pull Requests, with PR runs and Settings, and Journey Testing, with User journeys, Run history, Bugs, Blockers, Schedules, and Settings.
  • Pull request testing settings: select Review > Verify in the sidebar, or Pull Requests > Settings on the repository’s page.
  • Activity: runs and outcomes across all your repositories, at app.coderabbit.ai/verify/activity.
  • On the pull request: one CodeRabbit Verify comment that Verify keeps up to date, with links to the pull request’s Verify tab in Change Stack.
Verify has no .coderabbit.yaml settings. You add and configure each repository in the app.

Availability

  • Verify is available on CodeRabbit Cloud for GitHub Cloud, GitLab.com, and Bitbucket Cloud repositories. See Access and billing.
  • Verify tests web apps and services. Native mobile apps aren’t supported.
  • Verify is in beta and free during the beta. The sidebar shows Free during beta next to Verify.
  • Verify uses the Coding Agent permissions. Admins and Members can use it by default; users with the Billing Admin role can’t. See Permissions.
  • If Verify doesn’t appear in your sidebar, it isn’t available for your organization yet. See Troubleshooting.

What’s next

Set up Verify

Prepare an environment, add a repository, and choose when Verify runs.

Test pull requests

Start a run with a comment or automatically, and read the Verify comment.

User journeys

Build a library of critical flows and test them on demand or on a schedule.