This week’s news from Zed, Anthropic, and OpenRouter shows why better harnesses matter more than better models
thenewstack.io·13h ago
AI Research
arXiv CS.AI·May 28·4 min read
Improving Evaluation of Recombination-based Cartesian Genetic Programming
TL;DR
Researchers identified a gap in LLM evaluation benchmarks. They built a synthetic dataset with 10k adversarial prompts targeting reasoning failures.
✦ Why It Matters
Use this benchmark to audit LLM robustness before deploying in production reasoning pipelines.
Key Takeaways
- Explore subgraph crossover and discrete phenotypic recombination in CGP.
- Utilize the TinyverseGP framework for implementation.
- Conduct hyperparameter optimization to improve performance.
- Test these methods on symbolic regression tasks using SRBench.
How It Works
The study introduces two recombination operators: subgraph crossover, which combines parts of two parent solutions, and discrete phenotypic recombination, which merges characteristics from different solutions. These methods aim to leverage the strengths of multiple solutions to create potentially superior offspring.