NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 25·8 min read
EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation
TL;DR
Generating multi-shot videos while keeping characters consistent across scenes is difficult because storing full frames wastes memory and mixes character data with irrelevant background information. EM-Vid introduces an entity-centric memory bank—a compact storage of character-specific patches indexed by entity—with sparse token conditioning to reduce computation.
✦ Why It Matters
Engineers can generate consistent multi-shot videos more efficiently without retraining models, enabling practical video production workflows.
Key Takeaways
- Implement entity-centric memory for better consistency in video generation.
- Use sparse token conditioning to reduce computational costs.
- Adopt structured multi-shot scripts for organized video prompts.
- Incorporate noise-injection for fine-grained appearance control.
How It Works
EM-Vid utilizes an entity-indexed bank of latent patches to store information about recurring entities separately from transient scene context. This separation allows for efficient memory management and reduces the risk of irrelevant information leakage.
Sparse token conditioning focuses self-attention on only the relevant tokens associated with entities, which streamlines the computational process. The structured multi-shot script format organizes prompts effectively, while the budgeted memory update strategy ensures that the memory remains compact and relevant over time.