NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·May 25·8 min read
Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimension
TL;DR
Bayesian optimization—a technique for tuning complex systems by testing them strategically—fails in high dimensions because it requires too many evaluations. DSEBO (Dynamic Shared Embedding Bayesian Optimization) automatically adjusts the search space dimensionality during optimization, starting low and expanding when progress stalls.
✦ Why It Matters
Engineers can optimize expensive systems without manually specifying dimensionality, reducing trial-and-error and computational cost.
Key Takeaways
- DSEBO automatically detects when to expand search space dimensionality during optimization.
- Reusing solutions across dimension changes accelerates convergence versus restarting.
- Regret bounds prove DSEBO balances approximation and optimization errors theoretically.
- Practical gains: reduced regret and runtime on high-dimensional real-world tasks.
Full Summary
Bayesian optimization is a method for finding optimal settings in expensive black-box functions (systems where you can only observe outputs, not internal mechanics). It struggles with high-dimensional problems due to the curse of dimensionality—the search space grows exponentially, requiring exponentially more evaluations.
Random embedding reduces this by optimizing within a low-dimensional subspace that captures the task's effective dimension (the minimal number of important variables). The challenge: effective dimension is unknown beforehand, forcing practitioners to guess or use trial-and-error.
DSEBO addresses this by starting optimization in a low-dimensional subspace, monitoring convergence, and dynamically switching to higher dimensions when solutions plateau. The method reuses previously evaluated solutions when expanding dimensions, improving initialization.
Theoretical analysis provides regret bounds showing DSEBO balances approximation error (from using a subspace) and optimization error (from incomplete search). Experiments on synthetic functions and real-world tasks demonstrate significant improvements in both regret and wall-clock time compared to state-of-the-art methods.