NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
TL;DR
A gap in the evaluation of machine creativity has hindered advancements in evolutionary systems like AlphaEvolve. CreativeBench, a benchmark for assessing machine creativity in code generation, was developed, featuring two subsets: CreativeBench-Combo and CreativeBench-Explore.
✦ Why It Matters
Engineers can leverage CreativeBench to better evaluate and enhance the creativity of AI models in practical applications.
Key Takeaways
- Use CreativeBench to benchmark your AI's creative outputs.
- Apply EvoRePE to enhance creativity during inference.
- Focus on quality and novelty metrics for evaluation.
- Explore the effects of model scaling on creativity.
How It Works
CreativeBench consists of two subsets: CreativeBench-Combo for combinatorial creativity and CreativeBench-Explore for exploratory creativity. It employs an automated pipeline that generates challenges through reverse engineering and self-play, allowing for continuous evaluation and adaptation.
The benchmark uses a unified metric that combines the quality of generated code with its novelty, effectively distinguishing between creative outputs and those that are simply incorrect or repetitive.