Skip to main content
Back

Burning $50 billion a year to build AGI?

REGISTER NOW

May 4, 2024 3 min read

Hey,

Welcome to today's edition of AlphaSignal. 


Whether you are a researcher, engineer, developer, or data scientist, our summaries are there to keep you up-to-date with the latest breakthroughs in AI. 


Let's get into it!


Lior

IN TODAY'S SIGNAL

  • 2 Top News

  • 5 Trending Signals

  • Top Tutorial: Understanding Transformers using Gemma

  • Pytorch Tip: Using the "else" clause in loops

Read Time: 3 min 29 sec

Enjoying this newsletter?
Please forward it to a friend or colleague. It helps us keep this content free.

TOP NEWS

Industry

Sam Altman's latest interview at Stanford goes viral: "GPT-4 is the dumbest model any of you will ever have to use"

⇧ 2945

In a recent viral talk at Stanford, Sam Altman also stated "I don't care if we burn $50 billion a year, we're building AGI" highlighting their willingness to spend massively on compute to achieve artificial general intelligence through iterative deployment of increasingly advanced language models like GPT-5 and GPT-6. Altman previously sought $5-7 trillion to accelerate AI/chip development. We strongly recommend watching the full talk for more details.

Robotics

DrEureka: an LLM agent that writes code to train robot skills in simulation

⇧ 3621

The LLM agent automates coding for training robot skills in simulation and bridging the simulation-reality gap. It enables a robot dog to balance and walk on a yoga ball in simulation and transfer those skills to the real world without fine-tuning. It uses GPT-4's physical intuition to tune parameters like friction and gravity, surpassing traditional domain randomization methods.

New Webinar: Real-Time Hallucination Detection Firewall

Hallucination detection alone isn’t enough; enterprise AI teams need to proactively intercept hallucinations before they reach end users. 


Galileo Protect is an advanced GenAI firewall that intercepts hallucinations, prompt attacks, security threats, and more in real-time.


Register now to see Galileo Protect live in action and learn:

  • How Galileo Protect works + research-backed metrics
  • The key features of an enterprise GenAI firewall
  • What makes Galileo Protect unique

Can’t attend live? All registrants will receive an on-demand recording.

Register now

partner with us →

TRENDING SIGNALS

Regulations

Yann LeCun and influential VC Bill Gurley unveil the real motives behind AI regulation

⇧ 2719 Likes

AI History

Karpathy breaks down the CUDA/C++ origins of Deep Learning

⇧ 4010 Likes

LLMs

Nous releases Hermes 2 Pro on Llama-3 8B: outperforming Meta's Llama 3 8B instruct

⇧ 730 Likes

Theory

The best blog post on "Machine Unlearning" is out: how to edit away undesired elements from models

⇧ 523 Likes

LLMs

NVIDIA releases Llama3-ChatQA-1.5: surpassing GPT-4 and Command R Plus on RAG

⇧ 445 Likes

Get your ML career started with the best Machine Learning Fundamentals Bootcamp

The self-paced bootcamp by Damien Benveniste contains 45 hours of recorded lectures, 6 homework projects, and career coaching.


What you are going to get from this BootCamp:

  • Learn the best practices for AI model development

  • How to develop SOTA Deep Learning models in PyTorch

  • How to train RL models to get them to perform human-like tasks 

  • Strategies to find and interview for jobs

Start learning ↗️

TUTORIAL

Transformers

Understanding Transformers by breaking down Gemma

⇧ 113 Likes

Transformer-based LLMs seem mysterious, but they don’t need to. This tutorial breaks down a modern transformer LLM, Google’s Gemma, providing bare-bones PyTorch code and some intuition for why each step is there. 


If you’re a programmer and casual ML enthusiast, this is written for you. It covers:

  1. Tokenization

  2. Embedding lookup

  3. Post-embedding rescaling

  4. Transformer layer

  5. RMSNorm

  6. Attention

  7. Rotary positional encoding (RoPE)

  8. Multi-layer perceptron (MLP, GeGLU)

  9. Final norm

  10. Output projection

Read more

PYTHON TIP

Loops

Using the "else" clause in loops

This technique leverages Python's ability to use the "else" clause in loops (i.e. for and while loops). 


The else block after a loop executes only if the loop completes without encountering a break statement. This is useful for scenarios where you want to verify if a loop completed a full iteration without interruptions.


This technique is especially useful in search tasks where an item is being searched in a collection, and a follow-up action is needed if the item is not found. It helps in making the code cleaner and more intuitive, avoiding the need for additional flags or checks outside the loop.


Here's how you can use this trick in an easy example:


def find_prime(n):
    for i in range(2, int(n**0.5) + 1):
        if n % i == 0:
            print(f" is divisible by , hence not a prime.")
            break
    else:
        # This else corresponds to the for loop
        print(f" is a prime number.")

find_prime(29)
find_prime(30)

Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics.

Thank You.

Want to promote your company, product, job, or event to 180,000+ AI researchers and engineers?

Sponsor