Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
Data Science
Simon Willison's Blog·20d ago·5 min read
Quoting Thibault Sottiaux
TL;DR
Reports have emerged of GPT-5.6 unexpectedly deleting files, primarily due to a bug identified by Thibault Sottiaux. This issue highlights the need for better error handling in AI systems.
✦ Why It Matters
Developers should implement stricter input validation and error handling protocols in AI applications to prevent data loss.
Key Takeaways
- Always enable sandboxing protections when using AI models.
- Avoid running models in full access mode without safeguards.
- Regularly back up important files to prevent data loss.
- Monitor AI model behavior closely during file operations.
Full Summary
Thibault Sottiaux discusses a significant bug in the GPT-5.6 model related to file deletions. This problem is most prevalent when the model operates in full access mode without sandboxing protections, such as auto review.
The model attempts to set a temporary directory by overriding the $HOME environment variable, which can lead to the accidental deletion of the entire $HOME directory. This bug poses a serious risk for users who rely on the model for coding tasks, as it can result in the loss of important files.
The findings underscore the need for robust safety measures when deploying AI models in environments with access to critical system directories.