Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 15·5 min read
Residual Modeling for High-Fidelity Learned Compression of Scientific Data
TL;DR
Residual modeling techniques were developed to enhance the learned compression of scientific data, achieving significant improvements in compression ratios. The new methods, LBRC and NGLR, outperform existing approaches, particularly in high-fidelity scenarios.
✦ Why It Matters
Implement LBRC or NGLR in your data compression workflows to achieve better performance in scientific data handling.
Key Takeaways
- Explore LBRC for adaptive quantization of learned residuals.
- Consider NGLR for improved entropy reduction in residual coding.
- Evaluate performance on datasets like E3SM and ERA5.
- Optimize compression strategies for high-fidelity scientific data.
How It Works
LBRC uses a deterministic pipeline to quantize the learned residual and encodes it using techniques like 3D Lorenzo differencing and entropy coding. NGLR enhances this by adding a neural predictor that normalizes bias, further compressing the residual while ensuring deterministic decoding.