Skip to main content
Back

Salesforce AI Maps How DeepSeek-R1 and o1 Redefined Reasoning

A Salesforce-led survey maps the entire landscape of LLM reasoning research, from prompt tricks to reinforcement learning and agentic workflows.

Pro

Paper, Full Access
  • Salesforce-led survey organizes LLM reasoning research along two axes: regimes and architectures.
  • Splits methods into inference scaling versus learning-to-reason, and standalone LLMs versus single and multi-agent systems.
  • Provides matched-notation walkthrough of PPO, REINFORCE, RLOO, GRPO, and DPO with state-action definitions.
  • Treats verifiers as central bottleneck, comparing outcome, process, and generative reward models.
  • Identifies shift from inference scaling to RL-based learning to reason, exemplified by DeepSeek-R1.
  • Flags evaluation, data scarcity, and reward hacking as the main unresolved challenges.

The past two years turned reasoning from a niche capability into the defining feature separating frontier models from ordinary chatbots. A new survey led by Salesforce AI Research, with collaborators from NTU, NUS, HKUST-GZ, and A*STAR, tries to impose order on the resulting chaos, organizing hundreds of papers along two clean axes and tracing where the field is actually heading.

The authors argue the literature has fragmented because researchers keep mixing up two different questions: when reasoning happens (at inference time versus during training) and where it happens (inside a single LLM, a tool-using agent, or a group of agents talking to each other). Their taxonomy pulls those apart and shows that most recent breakthroughs, DeepSeek-R1, OpenAI o1, Deep Research, and Manus, sit at specific intersections of that grid.

Two axes that organize everything

The core framing splits methods along orthogonal dimensions. Regimes define the stage at which reasoning is achieved, either at inference time or through dedicated training. Architectures determine the components involved, distinguishing standalone LLMs from agentic compound systems that incorporate external tools and multi-agent collaborations.

Within each cell, the authors apply a second lens: input versus output. On the input side, techniques modify prompts, demonstrations, or messages between agents. On the output side, techniques evaluate, rank, or refine multiple sampled candidates. This unified view explains why inference-time tricks and training methods often reuse the same machinery. A search tree that guides decoding can also generate training trajectories.

Why yet another reasoning survey

The pace of publication made this necessary. Research on reasoning regimes and agent architectures has accelerated sharply since Chain-of-Thought landed in 2022, and the authors note that prior surveys typically covered only prompting or only inference scaling, rarely addressing training regimes and agentic systems together.

The problem the field faces is concrete. Reasoning requires multi-step processing beyond token-by-token generation, human-annotated reasoning traces are expensive to collect, and it is unclear which combination of techniques generalizes. The survey's answer is to give practitioners a map of what has been tried, what worked, and where the boundaries are.