Together AI Lets Developers Run Custom AI Training Without Managing GPUs
Together AI launched a post-training service that lets you write your own RL or SFT loop in Python while they manage GPU orchestration, rollout engines, and weight sync.
- Together AI launched Post-Training, a service for custom RL and SFT loops on open models.
- You write data, rewards, and loss in Python; Together runs generators, trainers, and weight sync.
- Supports LoRA and full-weight training on Meta, Qwen, Moonshot, and Z.ai models.
- Multiple LoRA experiments share one GPU reservation, cutting cost of parameter sweeps.
- Trainer matches generator precision and replays MoE expert choices to prevent RL drift.
- Recipes available in the post-training cookbook, with Tinker-compatible API.
Together AI has launched Together Post-Training, a managed service for running custom supervised fine-tuning and reinforcement-learning recipes on its GPU infrastructure. Developers write the training loop in Python and control the data, prompts, environments, rewards, verifiers, judges, optimization methods, adaptation format, and deployment target.
Managed fine-tuning APIs usually provide a fixed training recipe. Custom reinforcement learning requires rollout engines, distributed trainers, weight synchronization, and model-specific GPU configurations. Together Post-Training gives developers control over the recipe while the service provisions and operates those components.
| Developers control | Together operates |
|---|---|
| Training data and prompts | On-demand GPU capacity |
| Environments, tools, rewards, and verifiers | Model-specific parallelism and rollout engines |
| GRPO, PPO, and custom loss functions | Trainer and generator consistency |
| LoRA or full-weight training | Weight synchronization and job scheduling |
| Evaluation and deployment | Synchronous and asynchronous execution |
The five-call RL loop
The reinforcement-learning API reduces each step to five operations that developers can arrange and extend in ordinary Python:
sample()generates model responses, commonly called rollouts.your_reward()scores those responses with developer-defined code.forward_backward()calculates the loss and gradients.optim_step()applies the weight update.weights_sync()sends updated weights to the rollout generator.
GRPO and PPO are policy-optimization methods that use reward signals to improve generated responses. Supervised fine-tuning uses the same training primitives but omits rollout sampling and generator weight synchronization because it learns directly from labeled examples.
Reward functions, verifiers, and tool environments remain plain Python running in the developer’s sandbox. Existing evaluation harnesses and agent environments can therefore call the service without being rewritten around a proprietary reward format.
Multi-turn agent training can reuse cached state from earlier turns on the GPUs that processed them. Each subsequent turn processes the new tokens, reducing repeated computation across long conversations.
One reservation can run several experiments
Together allocates a model-specific GPU reservation with a configuration already tested for that model. Developers can open separate sessions for several experiments on the reservation, vary learning rates, reward functions, or data mixtures, and send metrics to tools such as Weights & Biases.
LoRA keeps the base model frozen and trains smaller adapter weights, which makes parallel experimentation cheaper than updating every model parameter. Multiple adapters can share the reservation and later load onto one inference endpoint for A/B testing. The winning adapter can be merged into the base weights, quantized for lower-cost inference, deployed on Together, or downloaded for another runtime.
When generator and trainer drift
Reinforcement-learning systems often use one process to generate responses and another to calculate weight updates. Differences in numeric precision, model versions, or mixture-of-experts routing can cause the trainer to optimize probabilities that do not match the generator’s behavior, reducing training stability and serving quality.
Together says its trainer matches the generator’s output precision and replays the generator’s expert-routing choices on supported mixture-of-experts models. Its losses also account for rollouts produced by older model versions during asynchronous training, and consistency checks catch problematic steps before their updates reach the weights.
Performance features include precompiled GPU execution and prefix caching, which avoids processing unchanged context repeatedly. Asynchronous mode overlaps rollout generation with training for higher throughput, while synchronous mode samples from the latest synchronized weights.
Catalog, compatibility, and billing
| Models | Open-weight models from Meta, Qwen, Moonshot, and Z.ai |
|---|---|
| Training formats | LoRA across the launch catalog and full-weight training on selected models, including Qwen3.5-35B-A3B |
| Recipes | Reference implementations in the GitHub cookbook |
| API compatibility | A Tinker-compatible interface intended to reduce migration work for existing Tinker clients |
| Deployment | Together endpoints or downloadable weights for external runtimes |
| Billing | Per GPU-minute while the model resource is ready |
Queueing, startup, and shutdown are free, according to Together, and idle resources shut down after one hour. Shared reservations and multi-LoRA training can improve utilization because several experiments draw from the same provisioned capacity.
Developers still own the experiment
Together Post-Training fits teams that need custom rewards, agent environments, policy-optimization methods, or model-specific verifiers but do not want to operate distributed training infrastructure. Developers remain responsible for data quality, reward design, environment security, evaluation, and model behavior. Adoption will also depend on model coverage, full-weight availability, API compatibility, session concurrency, and the GPU-minute cost of the required configuration.
Comments
