| Hey, | Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.
We identify and summarize the top 1% news, papers, models, and repos in the AI industry. |
|
|
|
IN TODAY'S SIGNAL
| Read time: 5 min 15 sec | 🎖️ Top News
📌 The AI Conference
The AI Conference is just 2 weeks away, with speakers from Stability AI, OpenAI, Meta, The Pentagon and many more.
⚡️ Trending Signals
🛠️ Trending Repos
Kotaemon: build and customize RAG-based document QA systems
courses: craft optimal prompts for Claude
LitServe: Enterprise-scale, Lightning-fast serving engine for AI models.
🧠 Tutorial |
|
|
|
If you're enjoying AlphaSignal please forward this email to a colleague.
It helps us keep this content free. |
|
|
|
TOP NEWS
| Language Model | Nous Research introduced DisTrO an Architecture-agnostic distributed optimizers reducing inter-GPU communication by 1000x-10,000x. |
⇧ 2,916 Likes |
 |
What's New | DisTrO (Distributed Training Over-the-Internet) Slashes Inter-GPU Communication for Large-Scale Model Training
Key features of DisTrO include
1000x to 10,000x reduction in inter-GPU communication requirements for training large neural networks.
Compatibility with consumer-grade internet connections
Resilience to degraded or dropped nodes during training
Ability to accommodate new nodes joining mid-process
Potential for asymmetric data streaming in client-server setups
DisTrO enables efficient training over slow internet connections using heterogeneous hardware, potentially revolutionizing distributed machine learning. Architectural Details
DisTrO-AdamW matched the convergence rate of standard AdamW with All-Reduce in pre-training a 1.2B parameter language model.
It achieved this while cutting inter-GPU communication from 74.4 GB to 86.8 MB per step, an 857x reduction. The model was trained on 105B tokens over 25,000 steps, demonstrating DisTrO's effectiveness at scale.
Being architecture-agnostic and network-agnostic, making it adaptable to various model types and network topologies.
DisTrO supports distributed data parallel training with minimal overhead, integrating seamlessly into existing training pipelines. Impact on Training Performance
DisTrO's performance raises questions about scaling laws in model training.
The researchers hypothesize that bandwidth reduction may increase with model size, suggesting DisTrO could be even more effective for larger models.
Could lead to new paradigms in GPU design, favoring larger VRAM and narrower interconnects. While the initial tests focused on language models, the researchers suggest DisTrO could be applied to other types of large foundational models, such as diffusion models.
The team plans to release the code and complete methodology on GitHub, enabling further exploration and integration with existing distributed training frameworks. | |
|
Read the paper
|
|
|
|
| The AI Conference is just 2 weeks away and almost SOLD OUT! | With speakers from Stability AI, OpenAI, Meta, The Pentagon and many more. Discount codes expire this week!
2 Days | 75+ Speakers | 3 Tracks
Top AI Companies Exhibiting in Expo
AI Impact in Healthcare, Enterprise, Defense + More
Live AI Startup Competition with VC Judges
Speaker Office Hours - Meaningful Networking
AI After Dark Networking Mixer + Expo Booth Crawl Don't miss your chance to be at the forefront of AI innovation.
|
|
Register now to secure your spot!
| partner with us |
|
|
|
TRENDING SIGNALS
| RAG |
|
⇧430Likes | |
Anthropic |
|
⇧ 982 Likes | |
Google Gemini |
|
⇧32 Likes | |
Coding Assistant |
|
⇧ 2,900 Likes | |
Gen AI |
|
⇧ 4,200 Likes | | |
|
|
|
|
|
TRENDING REPOS
| RAG | |
☆ 1,010 Stars |
Helps you build and customize RAG-based document QA systems. Offers a clean UI for end-users and a framework for developers. Supports local LLMs and API providers like OpenAI. Includes hybrid retrieval, multi-modal QA, and complex reasoning methods. Configurable settings and extensible architecture allow fine-tuning of the RAG pipeline. |
| Prompt Engineering |
|
| ☆ 1,990 Stars |
A collection of Prompt Engineering courses released from Anthropic for building with Anthropic API. You'll learn prompt structure, troubleshooting, and advanced techniques across 9 chapters. It covers Claude's capabilities, common pitfalls, and use cases. The course uses Claude 3 Haiku, Anthropic's fastest model. |
| LLM ServIng |
|
☆ 1,211 Stars |
LitServe helps you serve AI models fast. It's built on FastAPI but 2x faster. You can deploy any model: LLMs, vision, audio, multi-modal. It handles batching, GPU scaling, and streaming. Works with PyTorch, JAX, TensorFlow. Self-host or use managed cloud. |
| |
|
|
|
|
|
TUTORIAL
| LLM Finetuning | Notebooks: Phi 3.5 Finetuning 2x faster + Llamafied for more accuracy |
Microsoft released Phi-3.5 mini last week with 128K context, and is distilled from GPT4 and trained on 3.4 trillion tokens. Unsloth library's maintainers uploaded a 4bit bitsandbytes quantized model and made some modifications to Phi-3.5 mini, with which you can eaziliy finetune it in Google Colab.
Here's how to use Unsloth to finetune Microsoft's new Phi-3.5 mini LLM:
Update Unsloth to the latest version. Access the Unsloth Colab notebook for Phi-3.5 mini.
Load the 'Llamified' Phi-3.5 mini model from Hugging Face.
Use 4-bit quantization for reduced memory usage.
Apply LoRA for faster finetuning.
Train the model on your dataset.
Monitor training loss to gauge performance.
Save your finetuned model.
This process gives you |
|
Colab notebook to finetune phi-3.5
|
|
|
|
Want to promote your company, product, service, or event to 190,000+ AI researchers and engineers?
|
|
Sponsor
|
|
|
|
|