Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QA

TL;DR

RMS-RSP, a new method for selecting which medical questions should receive rationale supervision, shows a slight accuracy improvement over random selection in specific datasets. It highlights the importance of rationale-local boundary sensitivity for enhancing model robustness.

Key Takeaways

• Use RMS-RSP for selecting rationales in medical QA datasets.
• Focus on rationale-local boundary sensitivity for better model robustness.
• Evaluate the trade-off between rationale quantity and model performance.
• Consider answer option reordering to improve accuracy metrics.