Meta CEO Mark Zuckerberg announced significant advancements in generative AI training during an earnings call on Wednesday. He revealed that the development of the Llama 4 model is currently underway, utilizing a cluster of GPUs that surpasses anything else previously reported, specifically boasting more than 100, 000 Nvidia H100 chips. The initial launch of Llama 4 is anticipated early next year, with smaller models likely available first. This increase in AI training scale is believed to be crucial for building more advanced AI models. While Meta seems to have a lead, other major players in the AI field are also pursuing large compute clusters. In March, it was noted that Meta and Nvidia had previously worked with around 25, 000 H100s for Llama 3, whereas Elon Musk claimed his xAI venture had established a similar setup with 100, 000 H100s. Zuckerberg hinted at Llama 4's potential for new modalities, stronger reasoning, and quicker performance, though he didn’t elaborate on specific capabilities. Unlike proprietary models from OpenAI and Google, Meta’s Llama models can be downloaded for free, which has appealed to startups and researchers wanting more control over their AI systems. However, the open-source label comes with restrictions on commercial use, and Meta does not disclose training details. Managing such an extensive array of chips for Llama 4 poses engineering and energy challenges.

It’s estimated that a cluster of this size would require 150 megawatts of power, while the largest U. S. supercomputer, El Capitan, uses just 30 megawatts. Meta plans to spend up to $40 billion this year on infrastructure, reflecting a 42% increase from 2023. Despite rising operating costs of about 9%, Meta's overall sales from advertising have surged over 22%, leading to higher margins and profits even as significant funds are allocated to the Llama projects. In contrast, OpenAI, the leading developer of cutting-edge AI, continues to incur challenges, currently training the larger GPT-5 model while managing its operation as a nonprofit endeavor. Zuckerberg defended Meta's open-source strategy, asserting its cost-effectiveness and trustworthiness compared to proprietary systems. He believes Llama 4 will enhance a variety of features across Meta’s platforms, including the popular Meta AI chatbot, which attracts over 500 million users monthly. The company foresees monetization opportunities through advertising in this feature, suggesting a path to subsidize Llama for broader access.