How to Review Your Vibe Coded App Before Deployment

In August, AI research lab METR published a security disclosure reporting an API key theft which occurred through a vibe-coded agent dashboard. An authentication flaw exposed the agent, which an attacker prompted to reveal its provider key. Over three weeks, the attacker consumed credits with an approximate commercial value of $600,000, although the provider supplied them for free.

AI code generation lets people with limited software development experience build and deploy applications. Those applications still need security checks before deployment.

A successful login demonstrates access for an authorized user. It leaves other questions unanswered. Does the backend reject a request without a session? Can the agent read the key that pays for its model calls? Can its worker modify the host?

Before you deploy your AI generated code, you need to test these boundaries directly. Check the rejected requests as well as the successful ones, trace credential access through the application, and remove permissions the workload does not need. The code, tests, and deployment configuration all belong in that review.

How METR’s Dashboard Exposed Its API Key

According to METR's disclosure, the compromised dashboard was running on a researcher’s personal EC2 instance that was intended to be protected by Google authentication. A fail-open vulnerability disabled authentication and exposed the app for several days.

The attackers also added an SSH key for persistent access. METR subsequently made security reviews for public deployments strict. The report does not provide the vulnerable code or establish which generation step introduced the flaw.

Fail-open means a security check encounters a problem and access remains available. Imagine a dashboard that skips authentication when a required configuration value is absent. The page loads and tasks still run, so a normal demonstration need not expose the defect.

I’ll use an illustrative dashboard throughout the checks below. It lets a user submit an agent task, inspect its output, and cancel it. This is an example of the way you can review your code. METR’s disclosure does not establish its internal architecture. Start with the backend request that creates a task.

Test The Requests that Should Fail

Authentication establishes who makes a request. Authorization determines whether that identity may perform the requested operation. Your backend must enforce both wherever protected resources require them. OWASP recommends denial by default and permission checks on every request.

Validating sessions requires strict ordering: verify identity before modifying state or dispatching work. AI code generators often push payloads into a task queue before completing authentication middleware, or validate tokens asynchronously inside the worker.