OpenAI reportedly found a way to halve inference costs

This one’s paywalled, so I can’t see the details. But if it’s true that OpenAI found a way to more than halve the cost of inference, it could be a huge deal. Inference is the cost that scales with every query, and so far the race has mostly been about buying more chips to keep up. Bringing it down in software would shift the math for the whole industry.