---
title: "RadixArk's Miles v0.1 Cuts Trillion-Parameter RL Sync Time by 86%"
subtitle: "RadixArk's open-source Miles framework runs asynchronous RL on a 744B-parameter model across 64 GB300 GPUs with a 263-second median step."
url: "https://alphasignal.ai/news/radixark-s-miles-v0-1-cuts-trillion-parameter-rl-sync-time-by-86"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-09-08T07:35:47.000Z"
updated: "2026-09-15T00:05:42.726Z"
type: "Paper"
topics:
  - "Post Training"
  - "Open Source"
subtopics:
  - "Rlhf"
  - "Lora"
upvotes: 536
source_url: "https://arxiv.org/abs/2609.08368"
comments: 0
truncated: true
---

# RadixArk's Miles v0.1 Cuts Trillion-Parameter RL Sync Time by 86%

> RadixArk's open-source Miles framework runs asynchronous RL on a 744B-parameter model across 64 GB300 GPUs with a 263-second median step.

RadixArk has released [Miles v0.1](https://github.com/radixark/miles), an open-source stack for reinforcement-learning post-training of large language models. It connects SGLang rollouts to Megatron-LM or PyTorch FSDP2 training, with exact token capture, mixture-of-experts routing replay, low-precision consistency checks, asynchronous execution, and three weight-synchronization methods.

Frontier post-training runs combine trillion-parameter mixtures of experts, long tool-using trajectories, and separate software stacks for inference and training. Small differences in kernels, precision, tokenization, or expert routing can distort policy updates without producing an explicit error. Miles extends THUDM’s lightweight slime framework, which has been used to train models including GLM-4.6.

## One policy, two numerical worlds

Reinforcement-learning policy updates compare the probability recorded during rollout with the probability recomputed during training. When the two engines assign different log-probabilities to the same sampled token, the resulting importance ratio reflects implementation differences alongside the intended policy change. Persistent mismatches can destabilize training.

Miles assigns each part of that workflow to a specialized component:

| Layer                | Implementation                | Role                                                                                                     |
| -------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------- |
| Rollout              | SGLang                        | Generates trajectories at high throughput and returns tokens, log-probabilities, and expert assignments. |
| Training             | Megatron-LM                   | Scales full-parameter training across large GPU fleets.                                                  |
| Alternative training | PyTorch FSDP2                 | Trains Hugging Face model implementations directly.                                                      |
| Session handling     | Miles session server          | Preserves tokenization across multi-turn tool use.                                                       |
| Weight updates       | Broadcast, P2P, or disk-delta | Moves updated parameters from training ranks to rollout engines.                                         |

The three-stage loop covers rollout, training, and weight publication. It can run synchronously in lockstep or asynchronously, allowing rollout engines to continue generating while the trainer consumes completed trajectory groups. Runtime metrics expose step timing, prefix-cache performance, and rollout-to-trainer log-probability divergence.

## Preserving the sampled token stream

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Miles v0.1: Production-Level Post-Training](https://arxiv.org/html/2609.08368)
- [radixark/miles GitHub repository](https://github.com/radixark/miles)
- [Introducing Miles - LMSYS Org](https://www.lmsys.org/blog/2025-11-19-miles/)
- [ROCm Support for Miles](https://www.lmsys.org/blog/2026-03-17-rocm-miles-rl-amd/)
- [THUDM/slime GitHub repository](https://github.com/THUDM/slime)
