---
title: "GitHub Cuts Secret Scanning False Positives by 75% With AI Verification"
subtitle: "GitHub and Microsoft cut secret scanning false positives by 75.76% using an LLM that reads how code actually uses a detected value, not just what it looks like."
url: "https://alphasignal.ai/news/github-cuts-secret-scanning-false-positives-by-75-with-ai-verification"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "GitHub"
company_url: "https://alphasignal.ai/company/github"
published: "2026-06-20T19:20:01.000Z"
updated: "2026-06-21T04:41:58.634Z"
type: "News"
topics:
  - "Security"
  - "Development"
subtopics:
  - "Prompt Injection"
  - "Code Review"
upvotes: 148
read_time: "6 min"
source_url: "https://github.blog/security/making-secret-scanning-more-trustworthy-reducing-false-positives-at-scale/"
comments: 0
---

# GitHub Cuts Secret Scanning False Positives by 75% With AI Verification

> GitHub and Microsoft cut secret scanning false positives by 75.76% using an LLM that reads how code actually uses a detected value, not just what it looks like.

Secret scanning is one of those features that sounds great until you're drowning in alerts about test passwords and placeholder UUIDs. At GitHub's scale, even small inefficiencies create real friction, and too many false positives make alerts harder to trust. When alerts feel noisy, developers spend more time triaging and less time fixing real issues. GitHub just published a detailed account of how they tackled this problem, and the results are hard to ignore.

## The scale of the problem

More than 39 million secrets were leaked across GitHub in 2024 alone, and every minute GitHub blocks several secrets with push protection. But blocking is only half the battle. The other half is making sure the alerts that do fire are worth acting on. Alert fatigue from false positives kills adoption -- and a scanner that cries wolf too often is almost as dangerous as no scanner at all.

Generic secret detection can generate more false positive alerts compared to the existing secret scanning feature, which detects partner patterns and has a very low false positive rate. To mitigate the excess noise, alerts are grouped in a separate list, and security managers and maintainers need to triage each alert to verify its accuracy. That manual triage burden is exactly what this new work aims to shrink.

## How the pipeline works today

GitHub's secret scanning pipeline combines pattern-based detection and AI-powered generic detection. The new verification layer was introduced to lower noisy, low-value alerts while preserving coverage. Think of it as three stages:

- **Pattern-based detection** -- catches known secret formats like partner API tokens using regex-style rules
- **AI-powered generic detection** -- uses a two-model architecture (GPT-3.5-Turbo for initial scan, GPT-4 as confirmer) to catch unstructured secrets like passwords that don't match any known provider pattern
- **LLM-based contextual verification** -- the new layer that evaluates candidates before surfacing them as alerts

The new verification step is the focus of this research. The reported enhancement inserts a context-aware reasoning layer that evaluates candidate findings against contextual signals before surfacing alerts. The post emphasizes verification rather than replacing existing detectors, which suggests an add-on filtering stage applying LLM-enabled heuristics at scale.

## Better context, not more context

The key insight here is deceptively simple. When you want an LLM to judge whether a detected value is a real secret, your first instinct might be to feed it as much surrounding code as possible. GitHub's research found the opposite is true.

![Comparison table showing 'More context' approach vs 'Better context' approach for secret verification](https://github.blog/wp-content/uploads/2026/06/blog-post-2.png?resize=1024%2C576)

Rather than passing entire files or repositories to the model, the system extracts a small set of high-signal information about how the value is actually used. Specifically, it looks for cases where a value is:

- Assigned to a variable and later passed into an API request
- Used in an authentication header
- Passed into a database client or cloud SDK call

Pattern matching can tell you a value *looks* like a secret. It cannot tell you whether the value is actually *being used* as one. A random UUID or a hardcoded placeholder string can look identical to a real token at the pattern level. The usage context is what separates them. And critically, most false positives can be resolved with focused, file-level context -- what matters is not how much code the model sees, but whether it has the right signals.

## Where Agentic Secret Finder comes in

GitHub collaborated with Microsoft Security and AI's Agents Offense team to bring more contextual reasoning into GitHub's secret scanning verification. The work adapts the verification approach from Agentic Secret Finder -- a broader detection and verification system developed to understand potential secrets in context rather than relying solely on pattern matches. Agentic Secret Finder is Microsoft's internal system for reasoning about whether something is a real credential, and GitHub essentially imported that reasoning capability into their verification step as a post-detection filter.

## 75.76% -- and they were aiming for 65%

![Bar chart showing false positive reduction: target 65% vs actual 75.76%](https://github.blog/wp-content/uploads/2026/06/blog-post-3.png?resize=1024%2C576)

The team evaluated the approach on hundreds of customer-confirmed false positive alerts. The target was a 65% reduction. The result came in at 75.76%, exceeding the goal while maintaining strong detection performance. In practice, this means a much higher proportion of surfaced alerts represent real exposures that need action.

A few important caveats worth knowing:

- The evaluation was on customer-confirmed false positives, not a random sample of all alerts -- so it measures precision improvement on known noise, not overall recall
- Generic secret detection may still miss instances of credentials checked into a repository, and the LLM will improve over time.
- The team is still evaluating on larger datasets and live traffic -- this is research-stage, not a finalized production benchmark

## What this means in practice

This improvement flows directly into [GitHub Secret Scanning](https://docs.github.com/en/code-security/concepts/secret-security/about-secret-scanning), which is available today. You don't need a Copilot subscription to use this. If you have Secret Protection ($19/seat), you get AI-powered generic password detection included. The contextual verification improvement is part of the same pipeline.

The practical impact for teams:

- **Less triage overhead** -- fewer alerts that turn out to be test data or placeholder values
- **Higher trust in alerts** -- when something fires, it's more likely to be real
- **Faster remediation** -- less time spent investigating noise means more time rotating actual leaked credentials
- **No coverage loss** -- the verification layer sits downstream of detection, so it doesn't affect what gets caught upstream

## The bigger picture

This work is part of a broader shift in how security tooling is being built. Hybrid approaches using regex and LLMs for secret classification in source code leverage LLMs' contextual understanding to reduce false positives -- and GitHub's implementation is one of the first production-scale deployments of this idea on a platform processing billions of pushes. Tools like TruffleHog and Gitleaks use regexes and entropy but often generate false positives and lack contextual understanding for obfuscated or non-standard secrets. The pattern-only approach has a ceiling, and GitHub is now clearly operating above it.

The assumption that needs updating across the field: more context fed to a model does not mean better reasoning. Focused, semantically meaningful context -- specifically, execution paths and usage signals -- outperforms brute-force code dumps. That principle applies well beyond secret scanning. You can [read the full technical writeup on the GitHub blog](https://github.blog/security/making-secret-scanning-more-trustworthy-reducing-false-positives-at-scale/), and [run a free secret risk assessment for your organization](https://github.com/get_started?with=risk-assessment&ref_product=code-scanning&ref_type=engagement&ref_style=button) to see where you stand today.

## Sources

- [X post by @github](https://x.com/github/status/2068413578374197367)
- [Making secret scanning more trustworthy: Reducing false positives at scale - GitHub Blog](https://github.blog/security/making-secret-scanning-more-trustworthy-reducing-false-positives-at-scale/)
- [GitHub improves secret scanning verification with LLM reasoning - Let's Data Science](https://letsdatascience.com/news/github-improves-secret-scanning-verification-with-llm-reason-f7cade8e)
- [Responsible detection of generic secrets with Copilot secret scanning - GitHub Docs](https://docs.github.com/code-security/secret-scanning/about-the-detection-of-generic-secrets-with-secret-scanning)
- [GitHub found 39M secret leaks in 2024 - GitHub Blog](https://github.blog/security/application-security/next-evolution-github-advanced-security/)
- [Secret Scanning Tools 2026 - GitGuardian Blog](https://blog.gitguardian.com/secret-scanning-tools/)