OpenAI's Jalapeño Chip Beats Nvidia's Blackwell in Inference Benchmarks

My take: OpenAI unveiled benchmark results for Jalapeño, its first in-house inference chip, at the Hot Chips conference this week: between 1.5x and 1.9x more energy-efficient than Nvidia's Blackwell at peak throughput, and up to 4.1x faster on interactive workloads. The chip was co-developed with Broadcom and Celestica.

One important caveat: these benchmarks were published by OpenAI about its own product. When the company making the chip is also the one running the tests, there is no way to know for certain whether the results are fully free of bias. Independent evaluations would be needed before treating these numbers as definitive.

Even so, the broader message matters for any professional or organization planning their AI infrastructure: Google, Amazon, and now OpenAI have their own silicon. Nvidia's dominance in inference is no longer uncontested, and more competition will eventually translate into better options and pricing.

How is your organization thinking about its AI infrastructure strategy for the next two or three years?