Skip to main content
Back

Toronto Students Burned TALOS-V2 Into a $50 FPGA Chip to Run AI

Two students burned Karpathy's microGPT into FPGA fabric, generating 50,000+ tokens per second with zero Python, zero CUDA, and zero DRAM in the inference path.

Repo, Full Access
  • Two U Toronto students implemented Karpathy's microGPT fully in SystemVerilog on a Cyclone V FPGA, hitting 50,000+ tokens/sec.
  • Uses Q4.12 fixed-point math with weights baked into on-chip ROM at synthesis, so DRAM disappears from the inference path.
  • A shared 16-lane streamed MatVec tile is time-multiplexed across Q/K/V, MLP, and the LM head.
  • Ships DE1-SoC top-level, RTL sampler, ModelSim testbench, Python weight export, and JTAG inference scripts under Apache 2.0.
  • A single MacBook P-core in C runs the same model ~71x faster, exposing runtime overhead as the real bottleneck.
  • Best read as a pedagogical reference for hardware transformer inference, not a production accelerator.

A pair of University of Toronto students published something that reads like a hardware koan: a transformer with no software stack. TALOS-V2 is a SystemVerilog implementation of Andrej Karpathy's microGPT that runs entirely on FPGA fabric, generating tokens at rates that make the exercise more interesting than the throughput number alone suggests.

Built by Luthira Abeykoon, TALOS-V2 is a custom SystemVerilog RTL implementation of microGPT running on a DE1-SoC board with a Cyclone V FPGA. It stores Q4.12 fixed-point weights in ROM, includes a hardware token sampler, and exposes a JTAG/MMIO control interface. The author's pitch: no GPU, no PyTorch, no CPU inference loop, just a transformer burned into silicon.

What is actually on the chip

The target model is deliberately tiny. Karpathy's microGPT is a 200-line, dependency-free Python file that trains and runs a GPT from scratch. It has 4,192 parameters, character-level tokenization with a 27-token vocabulary, a single transformer layer, and a scalar autograd built from primitives. Trained on 32,000 names in about a minute on a MacBook, it generates plausible new ones.

TALOS-V2 takes that model and turns every step (embeddings, attention, RMSNorm, MLP, and sampling) into explicit RTL. The repository ships:

  • A SystemVerilog inference core with a one-token-at-a-time FSM
  • A shared streamed matvec tile that gets time-multiplexed across Q/K/V, MLP, and the LM head