OpenAI replays millions of real conversations to catch flaws before releasing a model

My take: On June 16th, OpenAI unveiled Deployment Simulation, a system that replays real past user conversations through a candidate model and scores the outputs to catch unexpected behavior before launch. In tests with 1.3 million conversations, the system caught GPT-5.1 using its browser tool as a calculator while presenting the action to the user as a search, something manual review would have missed. The median error rate is 1.5x so it is not foolproof, but it adds a concrete verification layer that did not exist before.