Google's AI research lab, DeepMind, has released new findings on AI model training that claim to significantly boost both speed and energy efficiency. According to their research, their JEST training method is up to 13 times more efficient and 10 times more power-efficient than other techniques, resulting in much higher performance. This comes at a critical time when AI data centers' environmental impact is a growing concern. Traditional AI training methods focus on individual data points, whereas JEST takes a different approach by training based on entire batches. It involves creating a smaller model that evaluates data quality from high-quality sources and ranks batches accordingly. This evaluation is then compared to a larger, lower-quality dataset, and the JEST model determines the most suitable batches for training. The findings of the smaller model are then used to train a larger model. The research paper, available for more detailed information, explains the study's processes and future prospects. DeepMind emphasizes the importance of steering the data selection process towards curated datasets of smaller size, which is crucial for the success of the JEST method. In fact, the researchers claim that their approach surpasses state-of-the-art models, requiring significantly fewer iterations and less computational resources. Graphs comparing JEST to other methods, including SigLIP, demonstrate the method's superior speed and FLOPS efficiency. It is important to note that the effectiveness of this system relies heavily on the quality of the training data. Without a human-curated dataset of the highest quality, the bootstrapping technique used by JEST loses its efficacy. As a result, implementing JEST may be more challenging for amateur AI developers, as expert-level research skills are likely required for curating the initial high-grade training data. The timeliness of the JEST research is evident as the tech industry and governments worldwide are starting to address the enormous power demands of artificial intelligence.

The energy consumed by AI workloads in 2023 equaled approximately 4. 3 GW, nearly matching Cyprus's annual power consumption. Furthermore, the energy usage appears to be growing rapidly, with a single ChatGPT request consuming ten times more power than a Google search. According to Arm's CEO, AI may consume one-quarter of the entire U. S. power grid by 2030. The adoption of JEST methods by major players in the AI field remains uncertain. Training GPT-4o reportedly cost $100 million, and future larger models may approach the billion-dollar mark. Consequently, companies are likely searching for ways to reduce costs. Some believe that JEST methods could maintain high training productivity while minimizing power consumption, benefiting both the financial aspect of AI and the environment. However, it is more probable that companies will strive to maximize power utilization and exploit JEST methods to achieve hyper-fast training output. The battle between cost savings and scalability remains to be seen.