Skip to main content
Back

University of Tennessee's Llama-3 Reads Cancer Reports With 0.976 F1

A LoRA-tuned Llama-3-8B extracts TNM cancer staging and biomarkers from pathology reports with 0.976 Macro F1, beating rule-based and single-task baselines.

Paper, Full Access
  • Llama-3-8B fine-tuned with LoRA extracts TNM staging and biomarkers at 0.976 Macro F1 across 10,677 pathology reports.
  • The causal LM head is replaced with seven parallel classification heads, eliminating generative hallucination risk.
  • Only 13.7M trainable parameters (0.34% of Llama-3-8B) with rank r=16 LoRA on Q/K/V/O matrices.
  • Multi-task learning lifts HER2 Macro F1 from 0.309 (single-task) to 0.967 by sharing features across correlated variables.
  • Beats prefix tuning by 2.5x in training time and crushes rule-based baselines on T Stage (0.993 vs 0.447 F1).
  • Trained on a single NVIDIA L40S with 4-bit QLoRA; dataset is private under HIPAA.

Cancer registries still rely heavily on people reading pathology reports by hand and typing structured fields into a database. A new study out of the University of Tennessee turns that job into a classification problem, fine-tuning Llama-3-8B with LoRA to pull tumor staging and biomarker status straight out of free-text pathology narratives, hitting a Macro F1 of 0.976 across seven clinical variables.

The interesting twist is that the authors do not use the model generatively. They strip off the causal language modeling head entirely and treat Llama-3 as a feature extractor feeding parallel classification heads, one per variable. That decision turns out to matter more than the choice of backbone.

Why free-text pathology breaks everything

Pathology reports are the source of truth for breast cancer staging, but the fields registries care about (Tumor-Node-Metastasis stage, histologic grade, and the ER/PR/HER2 biomarkers) are buried in unstructured narratives and scanned PDFs. Cancer registries often rely on manual data abstraction, a process that is notoriously resource-intensive and prone to human error.

The obvious alternatives all have known failure modes:

  • Rule-based regex pipelines are precise on standardized templates but fall apart on negations, OCR artifacts, and non-standard phrasing.
  • BERT-style clinical encoders like BioBERT and ClinicalBERT struggle with the long-range reasoning needed to tie tumor size in the Gross Description to the histology in the Final Diagnosis.
  • Zero-shot LLM prompting introduces hallucinations and schema drift. Generative decoding optimizes for the most probable next token rather than the most factually accurate category, which makes it a poor fit for rigid clinical databases.