Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 17·5 min read
Marionette: Predicting World States, Rendering Geometry, Painting Appearance
TL;DR
Marionette is a novel world model for interactive games that predicts 3D world states and synthesizes photorealistic visuals. It improves controllability and consistency in character interactions over long durations.
✦ Why It Matters
Implement Marionette's structured state modeling to enhance character interactions in your game projects today.
Key Takeaways
- Utilize a two-stage autoregressive model for world state prediction.
- Implement zero-parameter rendering for accurate geometry computation.
- Apply terrain colliders to reduce character ground penetration.
- Explore control-conditioned models for photorealistic visual synthesis.
How It Works
Marionette operates by first predicting a comprehensive 3D world state using a two-stage autoregressive model. This model captures essential elements like articulated skeletons and their movements.
A zero-parameter graphics bridge then translates this state into visual outputs, ensuring accurate geometry and occlusion without additional computational overhead. Finally, a video-diffusion model synthesizes high-quality images based on the structured controls, allowing for realistic character interactions.