NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·May 25·8 min read
CoSPlay: Cooperative Self-Play at Test-Time with Self-Generated Code and Unit Test
TL;DR
LLMs struggle to generate correct code without ground-truth unit tests (reference tests), which are expensive to create and limit scalability. CoSPlay, a training-free framework, jointly improves code and self-generated unit tests through cooperative iteration, using execution feedback to prune weak solutions.
✦ Why It Matters
Engineers can now generate reliable code without expensive ground-truth test data, scaling inference-time computation instead of training.
Key Takeaways
- CoSPlay eliminates dependency on ground-truth unit tests, reducing data collection overhead for code generation systems.
- Use bidirectional pass-count signals from code-test execution matrices to iteratively refine both solutions and test quality.
- Apply output-consensus clustering to select final code when multiple candidates achieve equal test pass rates.
- Framework generalizes across model backbones and scales with increased inference token budgets without retraining.
Full Summary
Current methods for improving LLM code generation rely on ground-truth unit tests (reference test cases written by humans) for training or validation, creating a bottleneck since these are expensive to produce at scale. CoSPlay addresses this by operating entirely at test-time without ground-truth data, using a cooperative self-play mechanism where code solutions and self-generated unit tests iteratively improve each other.
The framework first generates diverse solution approaches and identifies failure modes to create discriminative unit test ideas. It then builds a Code-UT execution matrix tracking which tests pass which code solutions, using bidirectional signals to prune weak code and replace unreliable tests.
When multiple codes tie, it selects the final solution from the largest output-consensus cluster, since correct implementations agree on outputs while incorrect ones diverge. Experiments on four benchmarks show CoSPlay on Qwen2.5-7B-Instruct improved best-of-N accuracy from 22.1% to 33.2% and unit test accuracy from 14.6% to 78.3%, matching the RLVR model CURE-7B which requires expensive ground-truth training data.