margin-benchCan a language model run a plant?
Each model gets a simulated plant and a job: commission a control loop, make a measurement read right. It runs experiments, reads the trends, changes settings and checks the result, as an engineer would. Then an acceptance test decides, by simulation only. Every step, every thought the model shares and every second of plant time can be replayed here.
How it works
- A plant, not a quiz. Each environment is a simulated plant with hidden parameters drawn from a seed: a tank with an unknown delay and noise, a scale whose interference the model has to find. The model only learns what it measures.
- Tools, as an engineer has them. The model drives valves, sets controller parameters, builds a signal chain, looks at trends, spectra and scope traces, puts weights on the platform, and decides when it is done.
- Graded by simulation, never by another model. After the model hands over, an acceptance test runs on the plant and checks written requirements. The score is the weakest requirement's ratio; 1.000 means every one was met.
- Each environment proves itself. A scripted reference engineer passes every seed, working only through the same tools, and scripted known-bad approaches fail every seed. The test suite checks both.
- Limits from design rules. How hard an acceptance test is comes from a family of textbook designs on the true plant, never from any model's score.
- Everything is published. Every tool call, the data it returned, the model's messages and the reasoning it shares, wall time, tokens and cost.
Code, environments and rules are in the repository. Any model can be run on any environment with one command, through OpenRouter or a local OpenAI-compatible server.