Skip to main content
Back

ETH Zurich's PoolDPP Makes AI Pick Smarter, More Diverse Output Batches

A NeurIPS paper formalizes diverse set generation from language models as a determinantal point process, with a finite-pool sampler and a KL bound that decays as 1/N.

  • PoolDPP models batched LM generation as a determinantal point process over sets of strings.
  • Finite-pool sampler with importance weighting; KL to target K-DPP decays as O(1/N).
  • Theorems are formalized in Lean, released at github.com/rycolab/pool-dpp.
  • Extends quality-diversity frontier on FineWeb open-ended generation vs K-means reranking.
  • On Ambrosia text-to-SQL, PoolDPP-MAP hits 0.392 precision at 0.238 recall (K=3).
  • On GSM8K, raises diversity substantially while staying within 0.01 of best pass@K.

Researchers at ETH Zurich, Cornell, McGill, and Mila introduced PoolDPP, a generate-then-select method that treats a batch of language-model outputs as one sampled set. The approach uses a determinantal point process, or DPP, to reward candidates with high model likelihood and penalize sets whose embeddings overlap. For applications that request K alternatives, PoolDPP explicitly optimizes coverage across the batch.

The NeurIPS 2026 paper reports a stronger quality-diversity frontier for open-ended generation, higher recall on ambiguous text-to-SQL, and substantially higher embedding diversity on math reasoning with pass@K within one percentage point of the best ancestral-sampling result. Code and a Lean formalization of the main theorem are available in the project repository.

Independent decoding clusters

Independent sampling gives each draw no information about the rest of its batch. Raising token-level temperature spreads probability across more continuations, yet several samples can still land in the same high-probability region and express the same idea with different wording.

Diverse beam search and related decoding heuristics encourage branches to separate during generation. According to the authors, those methods lack a global similarity model for completed strings, which can produce abrupt topic changes or incoherent outputs. PoolDPP instead scores the completed batch using both sequence likelihoods and embedding geometry.

Determinants score the whole batch

A fixed-size DPP assigns probability to each subset S according to P(S) ∝ det(L[S]), where L is a positive semidefinite similarity kernel. PoolDPP constructs that kernel from a quality weight w and an embedding φ:

L[i,j] = sqrt(w_i w_j) · <φ_i, φ_j>

The determinant equals the squared volume spanned by the quality-scaled embedding vectors. High-quality candidates increase that volume, near-collinear candidates shrink it, and duplicate vectors reduce it to zero. The resulting probability therefore accounts for every pairwise relationship in the selected batch.

Quality and diversity frontier for PoolDPP and comparison methods on FineWeb
PoolDPP extends the reported quality-diversity frontier on FineWeb generation.

The target quality weight for a complete sequence