Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 28·5 min read
6.5% of the Neuro-Symbolic Literature Can Be Reproduced from Its Published Artifacts, a Six-Stage Audit Framework and First Instantiation
TL;DR
A six-stage framework was developed to audit the reproducibility of neuro-symbolic AI literature, revealing that only 6.5% of studies could be reproduced from available artifacts. This highlights significant gaps in reproducibility and calls for better archiving practices in research.
✦ Why It Matters
Ensure your research includes complete and accessible artifact bundles to enhance reproducibility.
Key Takeaways
- Implement a six-stage audit framework for reproducibility assessments.
- Ensure all research includes accessible and verifiable code artifacts.
- Archive all research artifacts in a versioned manner.
- Prioritize reproducibility in research submissions to enhance credibility.
How It Works
The six-stage framework systematically retrieves, screens, and evaluates research records, focusing on the availability of public code artifacts. Each stage narrows down the studies based on specific criteria, ultimately assessing the reproducibility of the findings.