NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection
TL;DR
Existing benchmarks for code authorship detection do not reflect the mixed nature of AI- and human-generated code in industry. HybridCodeAuthorship is a new dataset that includes Python code files with interleaved human- and AI-authored lines.
✦ Why It Matters
Engineers can leverage the HybridCodeAuthorship dataset to improve AI code detection algorithms in real-world applications.
Key Takeaways
- Utilize HybridCodeAuthorship for training AI detection models.
- Evaluate detection algorithms on mixed authorship datasets.
- Focus on improving F1 scores for better accuracy.
- Consider real-world codebases when developing detection tools.
Full Summary
As AI code assistants become more prevalent, distinguishing between human and AI-generated code is essential for risk management and productivity analysis. Existing benchmarks typically focus on either fully human or fully AI-generated code, which does not represent real-world scenarios.
HybridCodeAuthorship was created to address this gap by providing a dataset of Python code files that contain both human- and AI-authored lines. The dataset was constructed using CodeSearchNet, which aggregates open-source code from GitHub.
Two state-of-the-art algorithms, including the AIGCode Detector, were evaluated on this dataset. Results indicated that the AIGCode Detector achieved an F1 score of 0.48 for chunk-level detection and 0.56 for line-level detection, highlighting the dataset's complexity.
This benchmark can help researchers and engineers develop better tools for code authorship detection in mixed-code environments.