CAE-bench is SWE-bench for engineering simulation. It measures whether a model can do computer-aided engineering — FEA, CFD, multibody dynamics, durability and optimization — in real CAE software. Tasks come from real engineering practice, not synthesized from papers. The current release covers CFD, combustion, battery and electronics-packaging thermal tasks.
Overall engineering-simulation score
Bars show the overall score. Black whiskers show the 95% interval. 77 tasks across combustion, battery, cfd and packaging; one scored trial per task. All models were run through the same agent scaffold (terminus-2). A model's score moves with how it is driven — the agent scaffold and the API route both.
The public task set ships as a Harbor dataset — each task is a container, a prompt, a reference solution and a verifier, runnable against any model with a completions endpoint.
Behind the public set is a much larger private task inventory built for post-training: engineering-simulation environments with a programmatic, non-hackable reward, in families parameterised into as many operating points as an RL run needs. If you are building post-training environments, get in touch.
The headline is the equal-weighted mean of these four numbers, so a domain where the ordering differs from the headline is worth seeing directly.
Two reference points bracket each run: a reference solution scores 1.0 by construction, and an empty submission has to score below 0.5.