MLCommons, an AI benchmarking group, has released a new set of tests and results to evaluate the speed of high-end hardware in running AI applications and responding to users. These benchmarks measure the responsiveness of AI chips and systems in generating responses from data-packed AI models. This allows for an assessment of how quickly applications like ChatGPT can deliver responses to user queries. One of the new benchmarks, called Llama 2, focuses on measuring the speed of question-and-answer scenarios for large language models. It was developed by Meta Platforms and contains 70 billion parameters. Additionally, MLCommons included a second text-to-image generator, called MLPerf, based on Stability AI's Stable Diffusion XL model. In terms of raw performance, servers powered by Nvidia's H100 chips from companies like Google, Supermicro, and Nvidia itself excelled in both new benchmarks.

Some server builders also submitted designs based on Nvidia's less powerful L40S chip. Notably, server builder Krai submitted a design for the image generation benchmark using a Qualcomm AI chip that consumes significantly less power than Nvidia's cutting-edge processors. Intel also submitted a design based on its Gaudi2 accelerator chips, which yielded solid results according to the company. While raw performance is important, energy efficiency is another critical aspect to consider when deploying AI applications. Advanced AI chips require substantial energy, so finding a balance between performance and energy consumption is a significant challenge for AI companies. MLCommons addresses this through a separate benchmark category dedicated to measuring power consumption. (Reporting by Max A. Cherney in San Francisco; Editing by Jamie Freed)