Melbourne's QMTL Shrinks Multi-Task AI Heads by 12x With Quantum Circuits
A hybrid architecture replaces classical prediction heads with a compact quantum circuit, cutting head parameter growth from quadratic to linear.
Takeaways
- QMTL replaces classical multi-task heads with a two-stage variational quantum circuit on top of a classical backbone.
- Head parameters scale as O(T) versus O(T²) for classical heads under capacity-matched design, per the paper.
- Delivers up to 12× fewer head parameters than classical and 4–5× fewer than HQNN baselines.
- Matches or beats classical heads on GLUE, CheXpert, and Extended MUStARD benchmarks.
- Consistently outperforms hybrid HQNN baselines even when HQNN is given the same qubit budget.
- Runs on IBM Quantum hardware (ibm_boston, ibm_fez) with graceful degradation under real device noise.
A team from the University of Melbourne and CSIRO has proposed a hybrid quantum-classical architecture for multi-task learning that changes how the prediction heads are built. Rather than stacking one linear layer per task on top of a shared backbone, they hand the job to a single variational quantum circuit split into a shared encoder and small task-specific subcircuits. The head's trainable parameter count then grows linearly with the number of tasks rather than quadratically, while matching or beating classical baselines on GLUE, CheXpert, and multimodal sarcasm detection.
Why the heads keep bloating
In the standard hard-parameter-sharing setup, a large encoder (BERT, DenseNet, and similar) feeds a shared representation into one linear head per task. Joint training across related tasks improves generalization and data efficiency through those shared representations, but the task-specific parameters can grow rapidly as more tasks are added. Designing heads that preserve task specialization while staying parameter-efficient has remained an open problem.
The authors formalize the scaling issue. If the shared representation dimension grows as tasks are added (a reasonable capacity-matching assumption), classical heads end up at O(T²) parameters. Every new task both adds another weight matrix and forces the shared representation wider, which multiplies the per-task cost.
A two-stage quantum head
The proposed QMTL architecture replaces the stack of linear layers with a single variational quantum circuit that runs in two stages inside one register of qubits: