| Hey, | Welcome to today's edition of AlphaSignal.
Whether you are a researcher, engineer, developer, or data scientist, our summaries ensure you're always up-to-date with the latest breakthroughs in AI.
Let's get into it!
Lior |
|
|
|
IN TODAY'S SIGNAL | Top Project: Andrej Karpathy releases llm.c
Trending Repos: AutoCodeRover anthropic-cookbook Open-Sora-Plan ipex-llm
Top Lecture: 3Blue1Brown Episode 2
PyTorch Tip: Custom collate function in DataLoader
Read Time: 5 min 10 sec |
|
|
|
Enjoying this newsletter? Please forward it to a friend or colleague. It will help us keep this content free. |
|
|
|
TOP PROJECT | Language Models | Karpathy is Back with a Pure C Implementation of GPT-2 in <1000 Lines | ⇧ 12,098 ⇆ 1831 |  | What's New | Andrej Karpathy, previously on OpenAI's founding team and Director of AI at Tesla, released his second educational project on Language Models (LLMs).
This project focuses on training a GPT-2 model with 124 million parameters on a CPU using only C/CUDA, avoiding PyTorch.
The codebase contains around 1,000 lines of code in a single file, allowing for the training of GPT-2 on a CPU with 32-bit precision.
This is a phenomenal resource to understand how language models are trained under the hook.
Karpathy selected GPT-2 because its model weights are publicly available. The project uses C for its simplicity and direct hardware interaction.
First, the repo allows to download and tokenize a small dataset on which the model is trained. In principle, the model could be trained directly on this dataset.
However, the current CPU/fp32 implementation is still inefficient which makes it not practical to train these models from scratch yet. Instead, the GPT-2 weights released by OpenAI are initialized and fine-tuned on the tokenized dataset.
Karapthy is currently working on: - direct CUDA implementation, which will be significantly faster and probably come close to PyTorch.
- speed up the CPU version with SIMD instructions, AVX2 on x86 / NEON on ARM (e.g. Apple Silicon).
- more modern architectures, e.g. Llama2, Gemma, etc.
| Why This Matters | Karpathy's work contributes significantly to the open-source community and the field of AI. This second educational project goes one step further in democratizing AI by showing how a model can be trained and optimized using one single file of code. | Community Feedback | Andrey Karpathy: "Writing the llm.c training code would in my opinion be a very interesting, impressive, self-contained and very meta challenge for LLM agents." Dave Deriso: "I hope more developers rediscover the elegant efficiency of C, especially now that llm copilots help reduce the memory-intensive barriers of recalling syntax and the many built-in functions."
| | Get the code |
|
|
|
 | Transcribe and Understand 4 Billion Voices By Combining AI and LLMs | Using Speechmatics' Speech API, you can transcribe, translate, summarize, interpret, analyze, and understand the voices of over half the world’s population. All with a single API call that works in real-time.
For businesses, this means new markets, new opportunities, and a whole new customer base.
For users, this means access to accurate and inclusive technology.
This is foundational Speech Technology for the AI era. | | Start building (for free) | partner with us |
|
|
|
TOP OF GITHUB | Agents | | AutoCodeRover is autonomous software engineer that resolves GitHub issues in less than ten minutes each, outperforming developers who take an average of 2.77 days. It uses LLMs and debugging tools to identify patch locations efficiently, fixing about 22% of 300 tested real-world issues at a minimal LLM cost (~$0.5).
The tool employs program structure-aware code search and improves repair rates with test suites via statistical fault localization, significantly advancing AI software engineering practices. | ⇧ 1793 ☆ 711 | | Language Models | | The Anthropic Cookbook helps you integrate Claude into projects with Python snippets, requiring an Anthropic API key. It guides on enhancing Claude with external tools, data retrieval for accuracy, citing sources, employing Haiku as a sub-agent, handling embeddings with Voyage AI, image processing, Stable Diffusion for image generation, PDF parsing, automating evaluations, enabling JSON mode, and creating content moderation filters. | ☆ 1931 | | Vision | | This project aims to reproduce Sora (Open AI T2V model), allowing you to generate high-quality, text-controlled video content, specifically landscapes. You can train models to enhance resolution and duration, engage in text2video experiments, and refine model conditions.
It supports training on AI chips, including Huawei Ascend 910, with plans for future expansion to domestic hardware. | ☆ 7506 | | GPU/CPU | | ipex-llm is a PyTorch library for running LLM on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) with very low latency.
The implementation currently supports 50+ models and is based on previous work like vLLM, llama.cpp, qlora, etc. It can be used for low bit inference, or fine-tuning, and offers seamless integration other frameworks like LangChain, Llama-Index or Hugging Face transformers. | ☆ 5761 | | |
| |
|
|
Convert any text into real-time human-like voices | With Deepgram, developers can generate real-time, natural-sounding conversations at scale for various use cases, from AI agents to media workflows. Insanely fast: less than 250 ms latency High-quality voices: natural-sounding tones, rhythms, and emotions Scalable: Cost-effective for high-throughput application
Get started ↗️ |
|
|
|
TOP LECTURE | Language Models | 3Blue1Brown Episode 2: Attention in Transformers, Visually Explained | ⇧ 395,192 views |  | What's New | The latest @3blue1brown video on YouTube delves into the attention mechanism of transformers. It explains how the model represents tokens as vectors and how these vectors gain meaning from context.
This episode focuses on the technical aspects of the attention mechanism in the transformer architecture. | | Watch |
|
|
|
PYTORCH TIP | Optimization | Custom Collate Function in DataLoader | When training large language models (LLMs) on text data, you often have sequences of varying lengths in each batch. To efficiently handle this, you can use PyTorch's DataLoader with a custom collate function.
This function dynamically pads the sequences in each batch to the length of the longest sequence, rather than padding to a fixed maximum length. This reduces wasted computation from unnecessary padding.
To do so, implement a custom collate_fn that pads the text sequences using pad_sequence from torch.nn.utils.rnn. Then, pass this custom_collate_fn to the DataLoader to enable dynamic batching of variable length sequences. | from torch.nn.utils.rnn import pad_sequence from torch.utils.data import DataLoader
# Define a custom collate function def custom_collate_fn(batch): texts, labels = zip(*batch) texts = [torch.tensor(t) for t in texts] texts_padded = pad_sequence(texts, batch_first=True, padding_value=0) labels = torch.tensor(labels) return texts_padded, labels
# DataLoader with custom collate_fn data_loader = DataLoader(your_dataset, batch_size=2, collate_fn=custom_collate_fn)
|
|
|
|
Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics. |
|
|
|
Want to promote your company, product, job, or event to 175,000+ AI researchers and engineers? | | Sponsor |
|
|
|
|