There has been significant curiosity about what exactly Mira Murati’s Thinking Machines Lab is working on, given its $2 billion in initial investment and the impressive lineup of former OpenAI scientists who have joined its ranks. On Wednesday, the lab shared a blog post offering the public an initial peek at one of its initiatives: building AI systems capable of generating consistent answers.
Their research update, titled “Defeating Nondeterminism in LLM Inference,” seeks to explain why AI responses often appear random. For instance, if you ask ChatGPT the same prompt several times, you’ll often receive very different replies. While this unpredictability is widely accepted as a characteristic of current AI models—seen as fundamentally non-deterministic—Thinking Machines Lab believes this issue can be fixed.
The article, written by Thinking Machines Lab’s Horace He, explains that the unpredictability in AI outputs originates from the way GPU kernels—tiny programs running on Nvidia’s chips—are combined during inference (the phase after you submit a prompt in ChatGPT). He claims that by tightly managing this orchestration, models could become far more predictable in their outputs.
Besides offering more consistent results for researchers and businesses, He points out that reproducible AI responses might enhance reinforcement learning (RL) training. RL rewards AI for correct outputs, but if results keep varying, the data can become inconsistent. Making model outputs steadier could, according to He, streamline the RL process. Thinking Machines Lab has previously shared with its investors that it plans to use RL to tailor models for enterprise clients, as reported by The Information.
In July, Murati—who was formerly OpenAI’s chief technology officer—revealed that the first product from Thinking Machines Lab would debut within a few months and would be “valuable to researchers and startups building specialized models.” However, it’s not yet known exactly what this product will be, or if it will incorporate ideas from their work on reproducible outputs.
The lab has also announced plans to regularly share blog posts, code, and other research materials to “benefit the public, and improve our own research culture.” This article, which launches their new blog series “Connectionism,” is part of that initiative. While OpenAI also started with a pledge to openness, it has become more secretive as it grew. Time will tell if Murati’s team continues to honor this commitment to transparency.
This research blog gives an uncommon window into the inner workings of one of Silicon Valley’s most enigmatic AI companies. Although it doesn’t expose the full direction of their technology, it does suggest that Thinking Machines Lab is addressing some of the biggest unresolved questions in AI. Ultimately, the challenge will be whether the lab can resolve these issues and commercialize its research to support its $12 billion valuation.
