--- title: "Together AI Says Open-Source Kimi K3 Beats Claude Fable 5.1 on Legal Tasks" subtitle: "Together AI highlights that Moonshot's open-weight Kimi K3 outscores Anthropic's newer Claude Fable 5.1 by 60% on the hard slice of Harvey's autonomous legal agent benchmark." url: "https://alphasignal.ai/news/together-ai-says-open-source-kimi-k3-beats-claude-fable-5-1-on-legal-tasks" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "Together AI" company_url: "https://alphasignal.ai/company/togethercompute" published: "2026-09-10T03:22:37.000Z" updated: "2026-09-11T06:48:33.808Z" type: "News" topics: - "Reasoning" - "Benchmarks" subtopics: - "Test Time Compute" - "Math Reasoning" upvotes: 128 read_time: "4 min" source_url: "https://x.com/togethercompute/status/2097888447415386613" comments: 0 --- # Together AI Says Open-Source Kimi K3 Beats Claude Fable 5.1 on Legal Tasks > Together AI highlights that Moonshot's open-weight Kimi K3 outscores Anthropic's newer Claude Fable 5.1 by 60% on the hard slice of Harvey's autonomous legal agent benchmark. Inference provider [Together AI](https://www.together.ai/blog/kimi-k3-vs-claude-fable-5-on-deepswe-cost-and-coding) says Moonshot AI’s open-weight Kimi K3 scored roughly 60% higher than Anthropic’s Claude Fable 5.1 on the hard subset of Harvey LAB-AA, an autonomous legal-work benchmark. The result matters for teams choosing legal-agent models because Harvey recently deployed Fable 5.1 for customers, while K3 can be downloaded, self-hosted, and post-trained under Moonshot’s license. The reported 60% figure describes a relative lift, not a 60-percentage-point gap. The figures presented here omit the two absolute hard-split scores, task count, confidence intervals, and complete evaluation settings, which limits independent verification and prevents direct comparison with the overall LAB-AA leaderboard. ## Finished legal work, scored strictly Legal AI company Harvey developed the Legal Agent Benchmark around more than 1,200 law-firm tasks spanning 24 practice areas. Tasks include reviewing matter files and producing memos, contract redlines, and presentations. Each submission is graded against an expert rubric, and the all-pass metric awards success only when every criterion passes. Artificial Analysis’s LAB-AA implementation places an agent in a sandbox where it must inspect matter documents, use tools, and produce a finished deliverable. Together’s hard split concentrates on longer, multi-step matters that require planning, file handling, drafting, and revision without human intervention. ## One headline, two score sets Public overall LAB-AA results already place K3 ahead of several proprietary models: Overall Artificial Analysis Harvey LAB-AA scores cited by Together | Model | All-pass rate | | --- | --- | | Kimi K3 | 26.7% | | Claude Fable 5 | 14.2% | | Grok 4.5 | 13.3% | Claude Fable 5.1 is absent from that overall table. Together’s newer 60% claim covers only the hard subset, so developers should keep the two result sets separate when comparing models. Harvey’s early Fable 5.1 testing found its largest gains over Fable 5 in litigation and dispute resolution, with strong results in data privacy, cybersecurity, and capital markets. Harvey also reported that the model retained quality at its lowest reasoning setting while using roughly half as many output tokens. Those internal findings use a different evaluation scope from Together’s hard-split comparison. ## Cost strengthens Together’s case Together sells K3 inference, giving it a commercial interest in comparisons that favor the model. Its earlier [DeepSWE analysis](https://www.together.ai/blog/kimi-k3-vs-claude-fable-5-on-deepswe-cost-and-coding) compared K3 with Claude Fable 5 on software-engineering tasks. Pass@k measures whether at least one of k attempts solves a task. DeepSWE results reported by Together | Metric | Kimi K3 | Claude Fable 5 | | --- | --- | --- | | Pass@1 | 68.5% | 69.9% | | Pass@2 | 82.0% | 80.2% | | Pass@4 | 89.4% | 88.5% | | Cost per rollout | $4.65 | $13.41 | Fable 5 led on the first attempt, while K3 led when the harness allowed two or four attempts. Together calculated that K3 completed 2.8 times as many tasks per dollar. The LAB-AA claim extends its economic argument from coding agents to document-heavy legal workflows. ## Open weights at 1.56 TB Moonshot describes K3 as a 2.8-trillion-parameter model with vision support and a one-million-token context window. Its Kimi Delta Attention and Attention Residuals architectures are designed to support long-context processing. The [model weights](https://huggingface.co/moonshotai/Kimi-K3) are available on Hugging Face. The release is a 1.56 TB download, before accounting for runtime memory, key-value caches, redundancy, and serving overhead. Self-hosting therefore requires substantial distributed infrastructure or aggressive quantization. The custom Kimi K3 License governs use and redistribution, and teams should review its terms before deployment. ## A production checklist for legal agents - **Keep a human approval gate.** K3’s leading 26.7% all-pass score means 73.3% of evaluated tasks failed at least one rubric criterion. - **Run task-specific evaluations.** K3 ranks first on Artificial Analysis’s Harvey LAB-AA implementation, fourth on Vals AI’s implementation, and third on Vals AI’s legal-research benchmark. Harnesses and task distributions can change the ranking. - **Freeze the evaluation stack.** Record the model revision, tokenizer, quantization, system prompt, tools, reasoning level, token limits, attempt count, document set, and scoring rubric. - **Treat the harness as part of the system.** Harvey’s harness raised K3’s APEX score from 58.8% to 67.5%, an 8.7-percentage-point gain. - **Validate post-training on held-out work.** Harvey reports that [Tenet](https://www.harvey.ai/blog/post-training-update-harvey-tenet), its post-trained model, completed nearly twice as many held-out LAB tasks and 20% more LAB Contracts tasks than base K3. - **Budget for operational controls.** Production legal systems need privilege boundaries, access controls, source citations, retention policies, audit logs, and review workflows alongside model inference. ## What would settle the comparison Together’s claim supports including K3 in a controlled evaluation against Fable 5.1. Production selection requires matched prompts, tools, documents, token budgets, attempt counts, and review rules, followed by analysis of latency, total serving cost, licensing, and security. Publishing the absolute hard-split scores, sample size, confidence intervals, and harness configuration would make the reported 60% lead independently testable. ## Sources - [X post by @togethercompute](https://x.com/togethercompute/status/2097888447415386613) - [Kimi K3 vs Claude Fable 5 on DeepSWE: Cost and Coding](https://www.together.ai/blog/kimi-k3-vs-claude-fable-5-on-deepswe-cost-and-coding) - [Is Kimi K3 good at legal work? | OpenAgreements](https://openagreements.org/models/kimi-k3) - [Fable 5.1, Now Live in Harvey](https://www.harvey.ai/blog/fable-5-1-in-harvey) - [Harvey Tenet Research Preview](https://www.harvey.ai/blog/post-training-update-harvey-tenet) - [Kimi K3 Tech Blog: Open Frontier Intelligence](https://www.kimi.ai/blog/kimi-k3)