Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·21h ago
AI Research
arXiv CS.AI·May 27·5 min read
Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2
TL;DR
Large AI models for video generation consume significant memory and compute, making deployment difficult. Tail-Aware HiFloat4 applies W4A4 quantization (reducing weights and activations to 4-bit precision) to Wan2.2 video models using a custom numerical format, with special handling for sensitive layers and activation distribution.
✦ Why It Matters
Engineers can deploy video generation models with 8x smaller memory footprint while maintaining quality using this quantization approach.
Key Takeaways
- Adopt Tail-Aware HiFloat4 for efficient model quantization.
- Utilize W4A4 configuration for main linear layers.
- Preserve high precision in critical model components.
- Implement activation-tail-aware calibration for better performance.
How It Works
Tail-Aware HiFloat4 quantizes the main linear layers of the Wan2.2 model using a W4A4 configuration, which balances low-bit representation with performance. It introduces a calibration module that focuses on the 'tail' of activation distributions, allowing for more accurate channel masking and reducing the impact of outliers during the quantization process.