vLLM Matches DeepSeek's DeepInfer Within 6% on DeepSeek-V4.1-Flash
Open-source vLLM reproduces DeepSeek's own DeepInfer engine within 6% on decode throughput, hitting roughly $39/GPU/hour at the official API prices.
20 stories
Open-source vLLM reproduces DeepSeek's own DeepInfer engine within 6% on decode throughput, hitting roughly $39/GPU/hour at the official API prices.
A new framework called Rep2Text shows that a single hidden vector inside an LLM leaks about half the words of its input, raising privacy and interpretability questions.
A new preprint uses psycholinguistics to measure why frontier model outputs feel harder to read, pinning the blame on information-density pressure.
A new open-source annotator fills the gap between behavior labeling and bounding-box tracking, producing training data for interaction detection models.
A single adaptive drafter adjusts speculative decoding to batch size and hardware, posting up to 2.99x throughput on vLLM and 3x single-request speedups.
Humans& releases Ampersand v0.1, a 744B collaborator model trained with multi-month simulations and a new off-policy RL algorithm called CORO.
Vercel's AI SDK ships a first-party Bash tool that gives agents a shell, with pluggable sandboxes ranging from in-memory to full VMs.
Google's new 740M multimodal embedding model runs locally across text, images, and audio, powering a Mac meeting companion and mobile search demos.
OpenAI folds Orchestra Bio into its Rosalind life sciences stack, pushing AI agents past research into the operational grind of drug programs.
OpenAI added a 12-hour time-to-live option to API prompt caching, letting persistent agents keep shared instructions, tools, and context cached across the full workday.
Google and Meta ship a native PyTorch backend for TPUs with true eager execution, torch.compile, vLLM and SGLang support, and near-linear scaling to full pods.
Databricks unveils MemCurrent, a persistent memory system that triples agent pass rates while cutting tokens 31% and latency 26% in production.
Zed benchmarked 25 MCP tool search methods across 150 queries and 536 tools, then shipped weighted BM25 with normalization rules, running fully local.
Rough concept first, then the words. 10x to 31x faster than strong diffusion models, on 4x to 5x fewer training tokens
Anthropic let an internal Claude model run a 64-hour autonomous protein design campaign, producing 17 lab-validated binders against scorpion neurotoxins from four regions.
Vals AI's new ClaimBench pits 15 models on 1,960 financial claims paired with 10-K excerpts, and small decision models win on cost.
Mistral's new speech stack combines state-of-the-art transcription, expressive TTS, and a LiveKit-based agent SDK with sub-second turn latency.
Bria's new 8B image model ties Qwen-Image-2.1 for the open-weights lead on both text-to-image and editing benchmarks at $32 per 1k images.
A comparison of eleven reward functions reveals how to train LLM agents that respect token, API, and step budgets without an external controller.
Perplexity's agent now runs the model, planner, and tool execution entirely on Apple silicon, keeping private files and credits off the cloud entirely.