How Visko streams real-time world models on Google Cloud TPUs

TL;DR

Visko, an AI startup, successfully deployed its Live World Model on Google Cloud TPUs, achieving real-time video and audio streaming at 1.07x playback speed within 30 days.

✦ Why It Matters

Consider leveraging Cloud TPUs for your own real-time AI applications to enhance performance.

Key Takeaways

Utilize Cloud TPUs for low-latency generative AI applications.
Implement Ulysses context parallelism for single-stream video processing.
Optimize critical components by selectively porting to native JAX.
Leverage open-source tools like MaxDiffusion for performance enhancements.

How It Works

Visko's pipeline chains three generative models—video diffusion, super-resolution, and audio generation—on a single TPU slice. By using Ulysses context parallelism, the team splits the token sequence across TPU chips, allowing for efficient processing of video chunks.

Custom Pallas splash attention accelerates the attention computation, enabling real-time performance.

How Visko streams real-time world models on Google Cloud TPUs

Accelerating the search for antimicrobial peptides with Google Antigravity

Bringing the borderless Lakehouse into the agentic era with innovations for Gemini

What the Gemini agent means for startups

Teaching students to think and learn through questions

Accelerating the search for antimicrobial peptides with Google Antigravity

Bringing the borderless Lakehouse into the agentic era with innovations for Gemini

What the Gemini agent means for startups

Teaching students to think and learn through questions