Skip to main content
Back

Two new Gemini models with huge price drop from Google

READ MORE

September 25, 2024 6 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 6 min 8 sec

🎖️ Top News


📌 Ori

⚡️ Trending Signals

📌 AssemblyAI 


⚙️ Top Lectures


🧠 Deep Dive

  • Running inference of Llama3 70B on 4GB GPU and Llama3.1 405B on 8GB GPU with AirLLM library.

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

LLM

Google releases 2 new production-ready and updated Gemini models with substantially reduced pricing

⇧ 5,822 Likes

What's New

Google has unveiled Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002, updated versions of their Gemini 1.5 series LLMs. These releases build upon the models introduced at Google I/O in May 2024, offering enhanced capabilities across text, code, and multimodal tasks.


Key highlights

  • 7% increase in MMLU-Pro scores

  • 20% improvement in MATH and HiddenMath benchmarks

  • 2-7% enhancement in vision and Python code generation

  • 64% reduction in input token pricing for Gemini-1.5-Pro

  • 2x faster output and 3x lower latency for Gemini-1.5-Flash

Performance Improvements and Capabilities

  • The new models demonstrate significant advancements in math-related tasks, with a 20% boost in MATH and HiddenMath benchmarks.

  • Gemini-1.5-Pro-002 retains its 2 million token context window, enabling the processing of extensive documents and codebases.

  • Gemini-1.5-Flash-002 focuses on speed optimization, delivering twice the output speed and three times lower latency compared to its predecessor.

Pricing and Accessibility

  • Google has reduced pricing for Gemini-1.5-Pro by 64% for input tokens, 52% for output tokens, and 64% for incremental cached tokens on prompts under 128K tokens, effective October 1, 2024.

  • Rate limits have increased to 2,000 RPM for 1.5 Flash and 1,000 RPM for 1.5 Pro.

  • Developers can access these models through Google AI Studio and the Gemini API, with Vertex AI availability for larger organizations.

Model Output and Efficiency

  • Both models now produce more concise responses, with output lengths 5-20% shorter than previous versions.

  • The updates enable new applications in video understanding, document processing, and code analysis while potentially lowering implementation costs.

Multimodal Capabilities

  • The upgraded models show improvements in vision-related tasks, with performance gains ranging from 2% to 7%.

  • This enhancement extends to video understanding capabilities, allowing for more effective processing and analysis of visual content alongside text and code.

Integration and Practical Applications

  • The Gemini 1.5 series models are designed for general performance across a wide range of tasks.

  • They can synthesize information from 1000-page PDFs, answer questions about repositories containing over 10,000 lines of code, and create useful content from hour-long videos.

Read more

Accelerate your AI training and inference with $1,000 in matched GPU credit.

Build, train, and deploy AI models faster. Ori's platform supports open source fine-tuning, foundational model training, and inference tasks.

  • On-demand NVIDIA GPUs: Accelerate training and inference. Multiple options available.

  • Reserved instances: Lock in resources for critical workloads. 35% savings, 1-month minimum.

  • Serverless Kubernetes: Deploy models globally without cold starts. We handle cluster management.

Key features

  • Pay-per-use pricing

  • Multi-region support

  • Automated load balancing

Ori now offers $1000 credit for new users. Spend $1000 on any public cloud, get $1000 back. See terms.

Get started

partner with us

TRENDING SIGNALS

OpenAI

OpenAI rolls out Advanced Voice Mode to ChatGPT Plus users, enhancing AI interaction capabilities

⇧ 29,400 Likes

AI Agent

Tweet goes viral about a GitHub repo hosting many Generative AI Agent tutorials, from beginner-friendly to advanced architectures

⇧ 1,300 Likes

Text to Audio

Researchers unveil PDF2Audio, an open-source alternative to NotebookLM for podcast creation from documents

⇧ 2,100 Likes

Open source LLM

NASA and IBM release Prithvi WxC, an open-source 2.3B-parameter foundation model for weather forecasting

⇧ 1,400 Likes

Intel

Intel launches Xeon 6 and Gaudi 3, doubling AI workload performance at improved cost vs Nvidia H100

⇧ 801 Likes

Build Smarter Multilingual Apps with Automatic Language Detection

AssemblyAI just released enhanced language detection and speech-to-text accuracy for multilingual apps. Real-time captions and LLM features now extract key insights from audio data. 200,000+ developers already use AssemblyAI. Try it now with $50 in free API credits, no credit card required.

Start Building Today

TOP LECTURES

Finetuning

Advanced strategies to boost flux fine-tunes with artificial data

⇧ 438 Likes

This tutorial teaches you three techniques to improve Flux fine-tunes using synthetic data: generating images from a single input, reusing high-quality model outputs, and combining LoRAs. You'll learn to expand limited datasets, refine model performance, and diversify output styles. These methods help create more accurate and versatile image generation models.

PDF Parsing

Learn to extract Tables from PDF & load it as a Pandas dataframe

⇧ 1,300 Likes

Learn to use pdfplumber, a Python library for extracting data from PDFs. You'll learn to extract tables, text, and form data with precise positioning. You can also the explore the library for visual debugging, page manipulation, and customization options. You'll see how to integrate extracted data with pandas for analysis. This tool helps you automate PDF data extraction efficiently.

Machine Learning

Scaling up linear programming: PDLP solver for massive optimization problems

⇧ 354 Likes

Learn about PDLP, a new first-order method solver for large-scale linear programming. Understand its advantages over traditional methods, core algorithm based on restarted primal-dual hybrid gradient, and key improvements. Explore real-world applications in network traffic engineering, container shipping, and traveling salesman problems. Discover how PDLP scales to billions of constraints, outperforming commercial solvers.

DEEP DIVE

Llama

Running inference of Llama3 70B on 4GB GPU and Llama3.1 405B on 8GB GPU with AirLLM library

⇧ 117 Likes

AirLLM helps you run large LLMs on GPUs with limited VRAM.


The core of AirLLM is layer-wise inference. It loads only one layer at a time into GPU memory, freeing up space after each layer's computation. This approach stores model layers on disk, loading them as needed.


To optimize performance, AirLLM uses flash attention. This technique enhances CUDA memory access for faster processing. 


The library also offers compression options

  • 4-bit quantization

  • 8-bit quantization


These further reduce memory usage, AirLLM implements prefetching to overlap model loading and computation, improving speed. It supports various model types, including Llama, ChatGLM, and QWen.


By combining these techniques, AirLLM lets you run Llama3 70B on a 4GB GPU and Llama3.1 405B on an 8GB GPU.

Read more

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise