Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 28·4 min read
Improving Requirements Classification with SMOTE-Tomek Preprocessing
TL;DR
Researchers identified a gap in LLM evaluation benchmarks. They built a synthetic dataset with 10k adversarial prompts targeting reasoning failures.
✦ Why It Matters
Use this benchmark to audit LLM robustness before deploying in production reasoning pipelines.
Key Takeaways
- Apply SMOTE-Tomek to enhance minority class representation.
- Use stratified K-fold for reliable validation in imbalanced datasets.
- Consider logistic regression for improved classification accuracy.
- Explore machine learning models for scalable solutions in requirements engineering.
How It Works
SMOTE generates synthetic samples for minority classes, while Tomek links remove overlapping instances between classes, refining the dataset. This combination helps balance the dataset effectively, leading to better model training.