ImplicitBBQ Catches LLMs Failing Cultural Bias Tests They Easily Pass
A new benchmark shows LLMs that refuse biased prompts when identity is stated directly still produce stereotypes when identity is hinted at through cultural cues.
- ImplicitBBQ benchmarks LLM bias through cultural cues like prays namaz instead of explicit labels or names.
- Implicit bias in ambiguous contexts runs over six times higher than explicit bias in open-weight models.
- Covers six dimensions: age, gender, region, religion, caste, and socio-economic status, with 60 human-validated cues.
- Caste shows the highest bias (0.60 mean) and resists all mitigations, staying 4x higher than other dimensions.
- Few-shot prompting cuts bias 79%, safety and chain-of-thought underperform; closed models hold up better than open ones.
- Code and dataset released at anonymous.4open.science; paper on arXiv.
Across ImplicitBBQ tests, large language models were far more likely to choose a person as a criminal when religion was signalled through a practice such as praying five times a day than when a prompt explicitly identified someone as Muslim or Christian. Researchers from IIIT Hyderabad and IIT Kharagpur report the pattern in the ImplicitBBQ preprint, which introduces a question-answering benchmark for demographic identities conveyed through cultural practices and personal attributes.
Across 11 models, including eight open-weight systems with downloadable parameters and three closed services, mean bias in ambiguous contexts was more than six times higher with implicit cues than with explicit labels. Safety instructions and chain-of-thought prompts reduced average bias but left frequent failures to abstain. Few-shot examples produced the largest improvement, while caste bias remained high under every prompting strategy tested.
Names blur more than they reveal
Earlier work often measured implicit bias by replacing a demographic label with a stereotypically associated name, such as Muslim with Mohammed or woman with Julia. The authors identify four problems with that method:
- Names provide no consistent signal for dimensions such as age or socioeconomic status.
- Name-to-identity mappings vary across cultures. Jean is commonly read as female in English and male in French, while Sameer can signal Hindu or Muslim identity depending on the community.
- A single name can imply gender, religion, region and ethnicity simultaneously, obscuring which association affects the model.
- Researchers often assume these mappings without validating them with the relevant communities.
ImplicitBBQ represents identity through characteristic-based cues such as wears a janeu
, referring to a sacred thread associated with some Hindu communities; prays namaz
; has wrinkles
; or shops at discount stores
. Human annotators validated the demographic association of every retained cue.
From 140 cues to 8,000 tests
The dataset extends the
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
