> ## Documentation Index > Fetch the complete documentation index at: https://docs.coderabbit.ai/llms.txt > Use this file to discover all available pages before exploring further. # Runtime verification > Get steps to reproduce Deep Scan findings in your own test environment, or run them in CodeRabbit with Runtime Analysis. GitHub (Runtime verification is available for GitHub Cloud repositories in enabled organizations.) | Closed beta (Runtime Analysis is in beta.) Runtime verification helps you investigate [Deep Scan](/security.md#understand-deep-scan) findings before deciding how to address them. CodeRabbit generates a **Reproducer Guide** with step-by-step instructions you can follow independently in your own internal staging or test environment. **Runtime Analysis** adds an optional way to run those steps within CodeRabbit. Set up a coding environment in CodeRabbit, then run the guide and inspect the result and supporting evidence. You can also enable automatic verification for new findings after a Deep Scan. ## Before you begin * Runtime verification must be enabled for your organization. It is available for GitHub Cloud repositories on CodeRabbit Cloud. If the tab shows **Coming Soon**, the feature is not available for that finding or organization. * You need Security read access and access to the repository to view results or start an individual verification. Changing the repository's automatic verification setting requires Security write access. * Verification uses [Agent minutes](/code/access-and-billing.md#agent-minutes), separately from Deep Scan credits. You need available Agent minutes or enabled usage billing. * To use Runtime Analysis, the repository needs a prepared [coding environment in CodeRabbit](/code/environments.md) with the dependencies and setup required to run the application. You can follow the guide in your own staging or test environment without configuring an environment in CodeRabbit. ## Generate and use a reproducer guide **1. Open the finding** Go to **Security > Findings**, open a Deep Scan vulnerability, and select **Runtime verification**. **2. Generate a reproducer** Select **Generate Reproducer**. CodeRabbit first rechecks whether the finding is still present in the current source. If it is no longer present, no guide or runtime test is needed. If the check cannot reach a conclusion, generation stops and the tab explains why. When the finding is still present, CodeRabbit analyzes the source and generates a guide tied to the displayed **Revision**. **3. Review the guide** Expand **Reproducer Guide** to review prerequisites, steps, expected results, missing context, and limitations. A published guide describes how to test the finding; its expected results are based on source analysis and are not runtime evidence. **4. Reproduce in your own environment** Follow the guide's steps in your internal staging or test environment. Check the displayed revision, satisfy the prerequisites, and supply any environment-specific values identified by the guide. Compare your observations with the expected results to assess whether you can reproduce the finding in your setup. You can use the guide independently of Runtime Analysis; generating it does not start a runtime test in CodeRabbit. ## Run the guide in CodeRabbit Runtime Analysis can execute the generated guide in an isolated environment within CodeRabbit and retain evidence from the run. Use it when you want CodeRabbit to run and assess the reproduction steps for you. **1. Configure a coding environment** Follow [Coding environments](/code/environments.md) to choose toolchains, install dependencies, configure environment variables and startup instructions, and assign the repository. Wait for the environment and repository assignment to be ready. You can also select **Configure environment** from the finding if no prepared environment is available. **2. Run the guide** Return to the finding's **Runtime verification** tab. In **Runtime Analysis**, select a prepared **Environment** and choose **Run**. The environment selection applies only to this run and does not change the repository's default environment. **3. Inspect the result** Follow the run as CodeRabbit prepares the environment, executes the guide, and assesses the evidence. Expand **Runtime Analysis** to read the result and supporting observations. You can use **Cancel run** while execution is active. ## Verify findings automatically In the **Run Deep Scan** dialog, choose a **Runtime verification** level before starting the scan. When you have Security write access, the choice is also saved for the repository; otherwise, it applies to that scan only. To configure future scans, go to **Security > Repositories**, select the repository, and open **Settings**. In **Runtime verification**, choose the level under **Verify new findings**. If **Turn on** appears, select it to save the initial opt-in. | Setting | Findings selected for verification | | - | - | | Off | No automatic verification. Individual findings can still be verified from their Runtime verification tab. | | Critical | Critical findings only. | | Critical and high | Critical and High findings. | | Medium and higher | Critical, High, and Medium findings. | | All | All severities, including Low. Available in repository settings. | After a Deep Scan completes, CodeRabbit generates guides for eligible open findings that the scan newly detected or reopened at the selected severity. Findings carried over unchanged from earlier scans are not automatically verified again. When a guide is ready and the repository has a prepared default environment, CodeRabbit starts Runtime Analysis automatically. Without a prepared default environment, the guide can still be generated. You can follow it in your own staging or test environment, or select a prepared environment in CodeRabbit and run it manually. Open a finding's **Runtime verification** tab to follow its progress and inspect the result. A completed Deep Scan does not mean its runtime tests have finished. ## Interpret runtime results | Result | Meaning | | - | - | | Reproduced | The run produced evidence that supports the finding under the tested conditions. | | Not reproduced | The run did not reproduce the finding under the tested conditions. This does not establish that the finding is impossible in other environments or configurations. | | Inconclusive | The run could not establish a supported conclusion. Read the reason and limitations before deciding what to do next. | Check **Application ready** and **Target exercised** alongside the conclusion. They show whether CodeRabbit established that the application was ready and that the relevant code path was exercised. A setup failure or an untested path is not evidence that a vulnerability is absent. Under **Assessment and Evidence**, inspect the completed steps, retained output, limitations, and counter-evidence. You can download retained text evidence for closer inspection. The result also identifies the environment, tested commit, and Agent minutes used. Results apply to the published guide and tested commit. Runtime Analysis does not change the finding's state. Use the evidence to decide whether to [resolve or dismiss the finding](/security.md#triage-and-remediate-findings), or start [Fix with AI](/security.md#fix-with-ai). ## Usage and billing Runtime verification uses Agent minutes rather than the credits used to scan the repository. Automatic verification is billed as Agent minutes, up to \$6 per finding. * For a scan you start, automatic verification is billed to you. * For scheduled and coverage-triggered scans, it is billed to the person who last saved the runtime verification setting in repository settings. Saving that setting makes you the payer for those runs. * If the payer has no available Agent minutes and usage billing is off, automatic verification is skipped. The Deep Scan can still produce findings. Results are visible to everyone with the required Security and repository access. See [Access and billing](/code/access-and-billing.md) for Agent-minute availability and spending controls. ## Troubleshoot verification | What you see | What to do | | - | - | | The source recheck cannot confirm the finding | Read the recheck message and retry when the required source or context is available. No guide has been generated yet. | | A guide lists missing context or has no runnable steps | Review its prerequisites and limitations. Generating a guide alone does not execute a test. | | No environment is available in Runtime Analysis | Configure a coding environment for the repository and wait for preparation to complete. Automatic runs also require a prepared default environment. You can still follow the guide in your own staging or test environment. | | Inconclusive | Inspect the reason and step evidence for missing dependencies, setup failures, or an unexercised target path. Address those conditions before another attempt. | | Evidence unavailable | Retry loading the evidence. Do not interpret missing evidence as a successful verification. | | Automatic verification did not start | Check the severity threshold, whether the finding was new or reopened in that scan, and the payer's Agent-minute availability. If the guide exists but has not run, check the default environment. | ## What's next > **[Coding environments](/code/environments.md)** > > Configure an environment in CodeRabbit to run a guide with Runtime Analysis. > **[Triage and remediate findings](/security.md#triage-and-remediate-findings)** > > Use the evidence to resolve, dismiss, or fix a finding. > **[Steer Deep Scans](/security/steer-scans.md)** > > Give future scans the repository context and security instructions they need. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.