Skip to main content
Back

Harbin's KCS Framework Boosts Multi-Hop Reasoning by Varying Evidence, not Wording

A research team proposes Knowledge Composition Sampling, a framework that diversifies multi-hop question generation by varying which sentences a model reasons over.

Pro

  • Researchers introduce KCS, a framework that diversifies multi-hop question generation by sampling different sentence subsets rather than paraphrasing.
  • Sentence selection is modeled as autoregressive prediction trained with a probabilistic contrastive loss over context sentences.
  • Nucleus sampling at the sentence level balances accuracy against diversity, producing multiple varied evidence chains per answer.
  • KCS improves knowledge composition selection accuracy by 3.9 percent over the strongest BERT classification baseline.
  • Using KCS for data augmentation improved downstream QA metrics for Llama 3.1 and Qwen 2.5 on HotpotQA and 2WikiMultihopQA.
  • Code is available at github.com/yangfanww/kcs; the paper is accepted at EMNLP.

A team from Harbin Institute of Technology and MemTensor has introduced KCS, or Knowledge Composition Sampling, a framework aimed at one of the quieter bottlenecks in multi-hop question answering: there is not enough training data, and what exists tends to teach models shortcuts rather than reasoning. The approach reframes data augmentation as a knowledge-selection problem instead of a paraphrasing one.

Where multi-hop QA keeps tripping

Multi-hop question answering asks a model to stitch together facts from several documents to answer a single query. Even with high-quality datasets, sparsity remains a serious issue, and models trained on thin data tend to latch onto spurious patterns. When only a handful of questions exist per document set, models memorize surface cues rather than learn to reason across sentences.

Earlier work tried to fix this by generating more questions, but the authors argue those efforts miss the point. Existing methods concentrate on simple question generation and rarely pull in external evidence, such as relevant sentences from documents, that would push complexity and diversity upward. Rewording the same query ten different ways teaches a model nothing new about how facts connect.

Vary the evidence, not the wording

KCS shifts diversification to the knowledge level. Given a long context and an answer, selecting a different set of slightly varied supporting sentences (a "knowledge composition") can lead a question-generation model to produce a distinctly different multi-hop question, even when the answer stays fixed. Instead of paraphrasing, the system picks different subsets of supporting sentences and lets each subset seed a new question.