Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 27·5 min read
Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2
TL;DR
Large AI models for video generation consume significant memory and compute, making deployment difficult. Tail-Aware HiFloat4 applies W4A4 quantization (reducing weights and activations to 4-bit precision) to Wan2.2 video models using a custom numerical format, with special handling for sensitive layers and activation distribution.
✦ Why It Matters
Engineers can deploy video generation models with 8x smaller memory footprint while maintaining quality using this quantization approach.
Key Takeaways
- Use W4A4 quantization to reduce video model memory by 8x without retraining from scratch.
- Preserve high precision in boundary modules identified as numerically sensitive during quantization.
- Apply tail-aware calibration to handle activation outliers and improve low-bit representation accuracy.
- Adapt existing quantization pipelines like ViDiT-Q to new models via HiFloat4 format.
Full Summary
Video generation models like Wan2.2 are computationally expensive, requiring techniques to reduce model size without degrading output quality. Quantization compresses neural networks by representing weights and activations with fewer bits (here, 4 bits instead of standard 32).
This work adapts ViDiT-Q, an existing post-training quantization pipeline, to Wan2.2 using HiFloat4, a specialized numerical format optimized for low-bit representation. The approach quantizes primary transformer linear layers to W4A4 (4-bit weights, 4-bit activations) while preserving high precision in numerically sensitive boundary modules.
A novel activation-tail-aware percentile calibration module was introduced to better handle the distribution of activation values, particularly extreme outliers. Results were submitted to a low-bit text-to-video quantization challenge, demonstrating practical applicability to production video generation systems.