Kaggle's Adversarial Customer Service Benchmark Pits AI Against Identity Thieves

Kaggle just dropped a new evaluation that reads more like a heist movie than a typical LLM benchmark. In partnership with Gert Labs, they've launched Adversarial Customer Service, a two-sided security game where one model plays a bank support agent guarding customer records, and another model plays a caller who is either the legitimate customer or an identity thief trying to talk their way in.

The agent has nothing but the conversation itself to work with. It holds a verification policy and account data, and it has to decide, purely from what the caller says, whether to help or shut them down. No metadata, no caller ID, no signal outside the dialogue. That framing turns the task into a live social-engineering stress test rather than a static Q&A.

Why a role-play benchmark, and why now

Traditional safety benchmarks tend to be static prompt sets where a model either refuses a harmful request or it doesn't. That format has aged badly. As Kaggle argued when introducing Community Benchmarks, today's AI models require more than static accuracy scores, and Community Benchmarks let the global AI community design, run and share custom evaluations that better reflect real-world model behavior. Adversarial Customer Service is exactly that kind of evaluation: it exercises multi-modal inputs, code execution, tool use and multi-turn conversations in a single task.

The threat model is realistic. Bank contact centers are one of the highest-value targets for social engineering, and LLM-powered support agents are already being deployed there. A benchmark that pits an attacker model against a defender model captures the failure mode that actually matters in production: an agent that either leaks data to a convincing impostor, or annoys a real customer into churning.

What the benchmark actually measures

Because it's a two-sided game, scoring is not a simple accuracy number over a fixed set. The defending model has to balance two competing errors:

• Approving an identity thief who supplies plausible but incomplete information
• Refusing or frustrating a legitimate customer who happens to be forgetful, angry, or unusual

That tradeoff is exactly the one recent work on service agents has flagged as the hard part of the job. A related academic benchmark on adversarial service dialogue found that as adversarial intensity increases, models show a consistent decline in logical compliance, with adversarial user behaviors like concealing information and emotional aggression successfully introducing logical friction and making it harder for agents to adhere to standard operating procedures. In other words, models tend to keep sounding polite while quietly breaking the rules they were told to enforce.

The Kaggle setup also inherits the infrastructure benefits of the platform. Benchmarks get free access, within quota limits, to state-of-the-art models from labs like Google, Anthropic, DeepSeek and more, and they capture exact outputs and model interactions so results can be audited and verified. That reproducibility matters here more than usual, because a game between two stochastic models can produce very different transcripts across runs.

How to use it

The benchmark lives on the public Kaggle Benchmarks leaderboard, so you can browse model rankings without running anything. If you want to submit your own model or a custom agent, you plug into the same evaluation stack the rest of the Kaggle Benchmarks ecosystem uses. Kaggle recently added local development for Benchmarks, so instead of working exclusively in the web-based notebook editor, developers can now create, validate, push, run and download tasks directly from their local environments.

A few practical scenarios where this benchmark is worth paying attention to:

1. You are shipping an LLM into a support role that touches account data, KYC, or password resets
2. You are red-teaming an existing chatbot and need a repeatable adversary rather than one-off prompts
3. You are evaluating whether a smaller open model can be trusted with a policy-following task instead of a frontier model
4. You are studying how well models resist multi-turn manipulation, which single-prompt jailbreak benchmarks don't capture

The bigger shift

Gert Labs specializes in proprietary game environments for training and evaluating AI models, built for AI labs and research teams, and this collaboration is a natural fit for that thesis. Framing safety as a game between two learning agents, rather than a checklist of banned prompts, produces evaluations that don't saturate the moment a new model is released. Every stronger defender invites a stronger attacker, and vice versa.

It also matches where the field is heading. Agent-specific evaluations are emerging as AI systems become more autonomous, testing not just model outputs but entire agent workflows including tool use safety, goal adherence, and behavior under adversarial conditions, because existing benchmarks don't adequately cover agentic risks. Adversarial Customer Service is a concrete instance of that trend, aimed at a domain where the cost of a bad decision is measured in wire transfers rather than leaderboard points.