Low-Resource Safety Failures Are Action Failures, Not Representation Failures

TL;DR

Low-resource language models exhibit significant safety failures, refusing harmful requests less frequently than high-resource models. By recalibrating harmfulness representations, researchers improved refusal rates in low-resource contexts without extensive retraining.

Key Takeaways

• Evaluate harmfulness representation in your language models.
• Consider recalibration methods for low-resource languages.
• Test harmful refusal rates with target-language examples.
• Utilize low-rank logistic classifiers for threshold adjustments.