CodeRabbit Security check

| CodeRabbit Security is a native GitHub check that fails while blocking Security review comments remain on a pull request. Choose a severity threshold, then require the check in GitHub to block merging. The check is off by default for every repository. It appears only after you enable it for a repository with PR Security review access. It is separate from the general CodeRabbit review check and from AI Deep Scan results.

​

Enable the check

You need Security write permission to change this setting. Your repository must have PR Security review access. The first evaluation includes existing visible Security comments on the pull request. Configure publication in these repository settings; there is no .coderabbit.yaml option for this check.

​

Severity thresholds

The threshold includes the selected severity and every higher severity. Threshold changes apply to newly published comments. Raising the threshold does not clear existing blockers, and lowering it does not turn comments previously below the threshold into blockers. Existing comments retain the decision made when they were first evaluated.

​

Require the check in GitHub

Publishing the check does not change your repository’s merge rules. To block merging, a GitHub repository administrator must configure a ruleset for the protected branch, or a branch protection rule, that requires CodeRabbit Security.

1. Let CodeRabbit Security complete a review with Success or Neutral so GitHub can list it as an available required check.
2. In your GitHub repository, open Settings > Rules > Rulesets. Create or edit a branch ruleset and select the branches you want to protect. If you use classic branch protection, edit the rule under Settings > Branches instead.
3. Enable Require status checks to pass before merging and add CodeRabbit Security. Select the CodeRabbit App as the expected source.
4. Save the rule. For a ruleset, set its enforcement status to Active.

See GitHub’s guide to creating a branch ruleset and its required status check rules for setup details. GitHub requires the check to pass on the latest commit; both Success and Neutral satisfy this requirement. A failed or unfinished required check blocks merging, subject to your GitHub bypass rules.

​

Understand the result

During a review, the check shows In progress. When evaluation finishes, it reports one of these results: Known blockers continue to fail the check even when other review evidence is unavailable. Incomplete analysis alone does not cause failure. A successful or neutral result does not guarantee that the code has no security issues.

​

What counts as a Security comment?

The check tracks visible inline comments published by CodeRabbit under Security & Privacy. Each comment counts once. When one comment contains multiple Security issues, its highest displayed Security severity applies. Replies, summary-only items, comments in other categories, and unpublished analysis do not count.

​

Read the report

Open CodeRabbit Security in the pull request’s Checks tab to see:

• The commit that was evaluated.
• Comment counts by severity: Found, Verified fixed, Remaining, and Blocking.
• Blocking comments with their file locations and direct links.
• The Block on threshold used for that review and a link to repository settings.

Counts carry across reviews of the same pull request. A dash (—) means the history needed for that count is unavailable; it does not mean zero. The report can list blockers admitted under an earlier threshold even after you change the setting.

​

Clear a blocking comment

Fix the issue, push a new commit, and let CodeRabbit review it. CodeRabbit uses its code-based comment resolution results to verify that the current code addresses the issue. Unrelated pushes do not clear blockers. If a published comment is deleted, a later review removes it from blocking only after confirming the deletion. Deleted comments are not counted as Verified fixed.

​

Disable or troubleshoot the check

Before turning off Publish CodeRabbit Security check, remove the check requirement from GitHub. Otherwise, GitHub can block merging while waiting for a check that will no longer be published. See GitHub’s required check troubleshooting guide. Disabling applies to subsequent reviews. A check already in progress finishes normally, and previously published checks remain visible. Disabling and re-enabling does not erase tracked blockers or history. If the check does not appear, confirm that publication is enabled for the correct repository, PR Security reviews are available, and a review has run since you enabled it. If your plan no longer provides PR Security reviews, new checks stop publishing even if the saved preference remains enabled.

​

What’s next