Skin Cancer AI Treats Men and Women Unequally and Age Bias Is Worse
A new study maps how sex and age imbalances in training data degrade skin cancer AI, and finds the two biases need completely different fixes
Takeaways
- Researchers used linear programming to build controlled demographic training splits and tested three learning strategies on skin lesion classification.
- Sex bias in AI models is primarily a data imbalance problem; adversarial training only helps in narrow female-only dataset scenarios.
- Age bias is fundamentally different: younger patients consistently get better classification regardless of training data distribution.
- Neither reinforcing multi-task learning nor adversarial training reliably closes the performance gap in male-majority or mixed-sex datasets.
- Cross-dataset validation showed bias patterns shift significantly under domain shift, meaning single-dataset fairness results may not transfer to clinical deployment.
- Code and datasets are publicly available on GitHub for reproducibility and future research.
Skin cancer AI is getting better at reading lesions, but a growing body of evidence suggests it is not getting better equally for everyone. A new paper published in Machine Learning for Biomedical Imaging takes one of the most systematic looks yet at how demographic imbalances in training data translate into real performance gaps, and what it actually takes to close them. The short answer: sex bias and age bias are fundamentally different problems, and treating them the same way will not work.
Why this matters now
Recent advances in deep learning could transform routine screening for skin cancers, but along with the anticipated benefits, potential dangers arise from unforeseen and inherent biases. The ISIC dataset and other benchmarks are heavily skewed toward fair skin, often exceeding 70 percent representation. Less discussed, but just as consequential, are the imbalances along sex and age lines that quietly shape what a model learns to recognize.
Men and women have different distributions of melanoma subtypes, which can affect the final diagnosis. Prior work showed large differences in performance between male and female subjects even when training data was relatively balanced, suggesting these factors influence how difficult a skin lesion is to diagnose. The new paper builds directly on that foundation, extending the analysis to age and adding two bias-mitigation strategies to the comparison.
The controlled experiment
The core methodological contribution is a linear programming (LP) approach for building training datasets with precisely controlled demographic compositions. LP is an optimization technique that finds the best allocation of resources given a set of constraints. Here, it maximizes the number of usable lesion images from the ISIC archive while enforcing exact ratios of male-to-female patients, malignant-to-benign lesions, and age distributions simultaneously.
The LP method gives more control over patient subset assignment, adjusting the proportions of selected dataset attributes while keeping others constant. This matters because naive splits often create hidden imbalances: fix the sex ratio and you may accidentally skew the age distribution or the malignancy rate. The LP formulation prevents that.