OpenAI's Codex Security Review Catches 74% of Real Bugs Semgrep Misses

OpenAI has expanded Codex Security Review into research preview, bringing context-aware security analysis directly into GitHub pull request workflows. Rather than scanning a diff in isolation, it reads the PR alongside the full repository and a project-specific threat model to surface findings that actually matter.

Why this differs from static analysis

Traditional tools like Semgrep or SonarQube generate noise: hundreds of alerts, most of them irrelevant to your specific application. Codex Security takes a different approach by reasoning about how data flows between services, where authentication boundaries sit, and which inputs reach sensitive operations. It uses language-model reasoning, test-time compute, tool use, and large context windows rather than fuzzing or signature-based scanning, which lets it trace multi-step attack paths instead of flagging suspicious function calls in isolation.

The analysis runs in three stages. During identification, it explores the repository and maps realistic attack paths. During validation, it attempts to reproduce each issue to confirm it is real. During remediation, it generates a concrete patch that teams can review and raise into a pull request.

How it differs from Code Review

Codex already includes a general-purpose Code Review feature for PRs. Security Review is a separate, deeper pass focused exclusively on security risks. It reduces noise by validating findings before surfacing them, incorporates your threat model and repository-level context, and ranks results with supporting evidence and patch suggestions. There may be some overlap between the two, but Security Review goes further on all three dimensions.

The full Security Report, accessible from the associated Codex task, includes severity ratings, attack paths, supporting evidence, validation status, and remediation guidance. What gets posted to the PR itself depends on your reporting threshold settings.

Setup

Getting started requires three things:

• Security Review research preview access for your workspace
• Codex cloud configured with a connected GitHub repository
• GitHub push or admin permissions on the repository

Once you have access, go to Codex settings and configure which PRs get reviewed. Three options are available:

• Follow personal — each contributor opts in individually
• Review all PRs — every PR in the repo gets reviewed
• Review team PRs — only PRs from members of your ChatGPT workspace

You can also control when the review runs: on PR open, on every push, or alongside Code Review. For manual triggering, drop @codex security review as a comment on any pull request.

Threat models: the highest-leverage setting

A threat model is a structured description of your application's assets, trust boundaries, and security assumptions. It tells Codex what actually matters for your specific system. Without one, findings drift toward generic priorities. You can point Codex at a threat model file already checked into the repo, or let it generate one per review if you do not provide one. The model is visible and editable, so teams can inspect and update the assumptions it works from.

Reporting thresholds and visibility

Automatic reviews surface only High and Critical findings by default. Manual reviews also include Medium severity. Both thresholds are configurable independently, and path-based overrides are available for specific parts of the codebase. One important note: findings posted to a PR inherit that PR's GitHub visibility. If the repo is public, so are the findings. The full report always stays private inside Codex.

Performance and limitations

Independent testing reports a 74% true positive rate across 31 findings on a 162,000-line, four-language corpus, compared to 20% for Semgrep and 28% for Snyk on the same dataset. Early beta data adds to the picture: within its first thirty days, the research preview scanned over 1.2 million commits across external repositories, surfacing 792 critical findings and 10,561 high-severity issues.

Real limitations apply:

• Codex Security finds source code vulnerabilities through context-aware threat modeling, but it does not replace dynamic application security testing, runtime monitoring, dependency scanners, or penetration testing.
• Analysis can be slow on larger repositories because it builds the threat model from scratch, and longer repository histories extend that initial setup time.
• The tool is still in research preview with no enterprise audit trail or compliance reporting. Snyk and SonarQube remain the compliance-grade standard.
• Codex proposes patches for human review. Those proposals can be turned into pull requests, but nothing modifies your code automatically.

Availability and cost

Security Review is available to ChatGPT Enterprise, Business, Edu, and Pro customers; Plus is excluded. During the introductory period it does not consume ChatGPT credits, though usage limits may apply. Access comes through your existing ChatGPT plan with no standalone subscription required.

The strongest fit is engineering teams with active GitHub repos and an existing security-review habit who want fewer false positives and faster fix suggestions. For teams hoping to replace a dedicated security function entirely, the docs are upfront that it will not.