Earlier this year, OpenAI cofounder and former chief scientist Ilya Sutskever gained attention after leaving to start his own AI lab, Safe Superintelligence Inc. While he has stayed out of the spotlight since his departure, he made a rare public appearance at the Conference on Neural Information Processing Systems (NeurIPS) in Vancouver on Friday. Speaking on stage, Sutskever declared, "Pre-training as we know it will unquestionably end. " This refers to the initial phase of AI model development, where a large language model learns from vast amounts of unlabeled data, typically sourced from the internet, books, and other materials. "We’ve achieved peak data, and there’ll be no more. " Sutskever remarked that while current data can still advance AI development, the industry is exhausting new data sources. This limitation will eventually require a shift in how models are trained. He likened the situation to fossil fuels, noting that, like oil, the internet holds a finite amount of human-generated content. He predicted that next-generation models will be "agentic in real ways. " Although Sutskever did not define this term during his talk, it generally refers to autonomous AI systems capable of performing tasks, making decisions, and interacting with software independently. In addition to being "agentic, " Sutskever suggested that future AI systems will have reasoning capabilities. Unlike today's AI, which primarily relies on pattern recognition, future systems will be able to process information step-by-step in a manner more akin to human thought. He noted, "The more a system reasons, the more unpredictable it becomes, " comparing highly reasoning systems to advanced AI in chess, which can be unpredictable even to top human players. "They will understand things from limited data, " he added. "They will not get confused. " During his talk, Sutskever compared AI scaling to evolutionary biology, highlighting research that shows differences in brain-to-body mass ratios across species.

He noted that while most mammals share a scaling pattern, hominids (human ancestors) display a distinct slope on logarithmic scales. He suggested AI might similarly discover new scaling approaches beyond current pre-training methods. After his talk, an audience member asked how to create incentive mechanisms for developing AI that grants it freedoms similar to those humans enjoy. Sutskever reflected that while these are important questions, he lacks confidence to answer them comprehensively, as it might require a "top-down government structure. " When the audience member proposed cryptocurrency, the room chuckled. "I don't feel like I am the right person to comment on cryptocurrency, " Sutskever said. "But there is a chance what you are describing will happen. In some sense, it’s not a bad end result if you have AIs that want to coexist with us and have rights. Maybe that will be fine. . . I think things are so incredibly unpredictable. I hesitate to comment but I encourage the speculation. " If you work at OpenAI and would like to discuss this further, please contact me securely on Signal @kylie. 01 or via email at kylie@theverge. com.