Sequential Functional Structured Tucker Compression for Large Language Model Attentions

TL;DR

A new framework called FTC enables efficient compression of large language model (LLM) attention mechanisms without fine-tuning, achieving significant performance improvements. It outperforms existing methods, particularly under aggressive compression, while maintaining low perplexity on benchmark tasks.

Key Takeaways

• Explore FTC for efficient LLM compression without fine-tuning.
• Utilize FTC to maintain performance under aggressive compression.
• Test FTC on various model sizes for optimal results.
• Apply findings to downstream tasks for improved outcomes.