Skip to main content
Back

Nebius Turns Token Factory Into Runnable AI Recipes for Developers

Nebius published an open cookbook of notebooks and reference apps for its Token Factory inference service, covering agents, RAG, fine-tuning, and distillation.

Repo, Full Access
  • Nebius published an open cookbook of notebooks for its Token Factory inference platform.
  • Covers agents in CrewAI, LangChain, Agno, Google ADK, Pydantic AI, LlamaIndex, and AWS Strands.
  • RAG examples pair Qwen3 embeddings with Milvus, Qdrant, or Weaviate vector stores.
  • Includes LoRA fine-tuning, full post-training, and a worked distillation example for grammar checking.
  • Backed by an OpenAI-compatible API supporting DeepSeek, GPT-OSS, Llama, Nemotron, and Qwen.
  • MIT-licensed and available on GitHub with 260+ commits.

Nebius has expanded its Token Factory companion project into a broad collection of production-oriented examples. The MIT-licensed cookbook repository contains Jupyter notebooks and runnable applications covering API access, agents, retrieval-augmented generation, fine-tuning, distillation, structured output, and media generation.

Its OpenAI-compatible interface lets developers reuse familiar SDK request formats while directing traffic to hosted open-weight models. Switching providers can require only a new base URL, API key, and model name, although provider-specific features and response formats may still require code changes.

One repo, six layers

Most examples are notebooks designed for inspection and modification. Each directory addresses a distinct part of the LLM application stack:

Area What it covers
API clients A native API notebook, plus integrations for LiteLLM, ai-suite, and LlamaIndex.
Agents Examples using CrewAI, Agno, LangChain, Google ADK, LlamaIndex, Pydantic AI, and AWS Strands.
RAG PDF chat, Qdrant with Qwen3 embeddings, Milvus with GPT-OSS, and a LangGraph support agent backed by Weaviate.
Model adaptation LoRA adapters, full-model fine-tuning, and a grammar-model distillation walkthrough. LoRA updates a small set of adapter weights, reducing the compute and storage required for customization.
Inference controls Tool calling, batch inference, and guided JSON generation for outputs that must follow a defined schema.
Image and video An image LoRA example and a Higgsfield image-to-video workflow.

Using the hosted endpoints requires a Token Factory API key. Examples connected to services such as Qdrant, Weaviate, Notion, Calendly, Slack, or web-search providers can require additional accounts and credentials.

The serving layer underneath

Token Factory provides managed inference for teams that want to run open-weight models without operating their own GPU serving stack. Its catalog includes DeepSeek, GPT-OSS, Llama, NVIDIA Nemotron, and Qwen models, along with an option for customers to host their own models.

Nebius advertises more than 60 models, sub-second latency, autoscaling throughput, and a 99.9% uptime service-level agreement. The commercial model combines usage-based token pricing with LoRA and full-model fine-tuning, distillation pipelines, staging-to-production promotion, and token-level observability.

The cookbook turns those platform capabilities into concrete request flows, dependency choices, and deployment patterns. That makes it more useful for technical evaluation than a feature list alone.

Agents and RAG lead

The agents and RAG directories contain the repository’s most complete application patterns. One support-agent example combines LangGraph orchestration, Weaviate retrieval, web search, Notion and Calendly tools, and a Slack interface. The result resembles an internal operations architecture rather than an isolated chat demo.

The retrieval examples also expose the underlying components. RAG systems retrieve relevant documents before generating an answer, giving the model task-specific context at request time. The cookbook pairs embedding models such as Qwen3-Embedding with Milvus, Qdrant, and Weaviate, allowing developers to inspect document ingestion, vector search, orchestration, and response generation separately.

Production gaps remain

The collection offers broad framework coverage, but its depth varies by directory. Teams assessing it for production should account for several limitations:

  • No shared evaluation harness: the repository lacks a common suite for measuring answer quality, regressions, latency, or cost across examples.
  • Limited cost data: notebooks do not provide per-workflow token estimates or comparative infrastructure costs.
  • Uneven coverage: the agents directory contains dozens of examples, while areas such as tool calling receive much less attention.
  • Fragmented post-training guidance: some fine-tuning material depends on linked guides instead of fully self-contained notebooks.
  • No repository-wide operations standard: production teams still need consistent policies for authentication, secret rotation, retries, rate limits, monitoring, and failure recovery.

At the time of review, GitHub showed 1 open issue and 260 commits. Those figures describe an actively changing example collection, so teams should verify dependencies, API behavior, and unresolved issues before adopting a notebook as maintained application code.

Five practical entry points

  1. Validate the API surface. Run the native client notebook to confirm authentication, model names, streaming behavior, and error handling.
  2. Build a document assistant. Use the PDF and vector-database examples to prototype retrieval over private material without operating a model server.
  3. Compare agent frameworks. Point CrewAI, LangChain, and Pydantic AI examples at the same backend to compare abstractions and tool execution.
  4. Adapt a model. Follow the LoRA or distillation workflows to test whether a smaller customized model can handle a narrow task.
  5. Enforce structured output. Use guided JSON generation when downstream code requires responses that conform to a schema.

The portability payoff

In its launch announcement, Nebius cites Prosus reporting cost reductions of up to 26 times compared with proprietary models and identifies Hugging Face as an infrastructure customer. The cookbook does not include a benchmark that independently reproduces those claims.

For engineering teams, the repository’s measurable value is the shorter path from endpoint credentials to a working prototype. Its standard API shape and range of integrations support meaningful provider and framework comparisons, while production adoption still requires independent evaluation of quality, latency, cost, security, and operational reliability.