36 Variants, Zero Comparability: Re-Baseline Before You Trust a Single Number
OpenAI staff confirm GPT-5.5 and GPT-5.6 effort settings are not directly comparable. The model picker is gone and Luna/Terra/Sol routing is opaque. Every regression suite, prompt contract, or effort×cost curve calibrated on 5.5 is now measuring a moving target. 'Start lower than on 5.5' is community folk wisdom filling in for the calibration data OpenAI didn't publish.
The tiers hide a real compute split. Max is one model thinking longer. Ultra parallelizes across subagents, which is a different latency and token-budget profile. The live hazard is spawn_agent: with no model or effort selection, Sol Ultra recursively spawns Sol Ultra by default. That is a cost blowout waiting for a production quota.
The cheapest experiment is the Raschka hypothesis (459K views): Luna at higher effort matches or beats Terra at lower effort for less money. That is consistent with OpenAI's claim that Luna at lowest effort beats GPT-5.5 at highest effort at 25x lower cost on health tasks. The thing that claim doesn't tell you is whether it holds on your distribution. If it does, a one-day paired eval yields a material spend cut.
Then Statsig: the vendor whose 36 variants you must benchmark now owns the platform holding many teams' benchmark records, meaning experiment definitions, exposure logs, and metric outcomes. Nothing suggests misuse, but the conflict-of-interest geometry is new, and the contractual status of your experiment data just changed hands. Confirm export paths while acquisition terms are still being socialized.
Against an unpinned GPT-5.6 endpoint you aren't running an experiment. You're sampling a routing distribution you can't observe.
What to do
Pin explicit GPT-5.6 API variants in every eval and A/B path, then re-run baselines per tier (Luna/Terra/Sol) before your next benchmark cycle
Run a paired Luna@high-effort vs Terra@low-effort eval on your top 3 task types this sprint and adopt the cheaper tier wherever quality parity holds
Audit your Statsig contract for data-use and export terms within two weeks, and set hard per-subagent token caps before enabling any GPT-5.6 ultra/spawn_agent mode