Scientific reasoning does not reliably translate into scientific forecasting in frontier AI

AI systems are increasingly employed to aid in predicting future scientific developments, yet their reliability in this area is questionable. Researchers introduced CUSP, a temporally grounded evaluation suite designed to assess event-level scientific forecasting across eight disciplines.

They tested six advanced AI models and found a notable disparity in forecasting accuracy, with models often identifying plausible mechanisms for future advances but failing to assess feasibility effectively. For instance, these models predicted scientific breakthroughs later than they were publicly recognized.

While providing additional scientific knowledge improved performance, it did not resolve the fundamental forecasting limitations. These results indicate that while AI systems demonstrate strong retrospective scientific reasoning, their predictive capabilities remain limited.

Consequently, scientific forecasting should be regarded as a crucial aspect of evaluating AI's role in research prioritization and decision-making.