Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 25·8 min read
Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning
TL;DR
Prior work (NEFTune) added uniform noise to embeddings during language model fine-tuning but lacked theoretical justification for why uniform noise outperformed Gaussian noise. Researchers developed SymNoise, a new fine-tuning method using symmetric noise in embeddings to better control local curvature.
✦ Why It Matters
Engineers can adopt SymNoise to achieve measurably better instruction-following performance in fine-tuned language models with minimal implementation overhead.
Key Takeaways
- Replace NEFTune with SymNoise for 6.7% performance gains on instruction-following benchmarks.
- Symmetric noise outperforms uniform and Gaussian noise by better controlling model curvature during training.
- Validate improvements across multiple datasets (Evol-Instruct, ShareGPT, OpenPlatypus) before production deployment.
- Investigate local curvature as a key mechanism for understanding why noise-based fine-tuning works.
Full Summary
Recent instruction fine-tuning methods inject noise into embeddings—the numerical representations of text—to improve language model performance. NEFTune (a prior technique) used uniform noise and empirically outperformed Gaussian noise, but the underlying reasons remained unexplained.
This work provides theoretical and empirical analysis showing these noise types perform comparably, then introduces SymNoise, which applies symmetric noise (noise distributed equally around zero) to embeddings. The key innovation is stricter regulation of local curvature—the rate at which the model's learned function changes—during training.
Testing on LLaMA-2-7B with Alpaca dataset, SymNoise scored 69.04% on AlpacaEval (a benchmark measuring instruction-following quality) versus 64.69% for NEFTune. Results held across stronger datasets (Evol-Instruct, ShareGPT, OpenPlatypus) and multiple model architectures, suggesting the approach generalizes well.