Intel has officially launched its Gaudi 3 accelerator targeted at AI workloads today. While the new processors are slower than Nvidia's widely used H100 and H200 GPUs for AI and HPC applications, Intel is focusing on Gaudi 3's competitive pricing and lower total cost of ownership (TCO) to drive its success. The Gaudi 3 processor features two chiplets, which house 64 tensor processor cores (TPCs—utilizing a 256x256 MAC structure with FP32 accumulators), eight matrix multiplication engines (MMEs—256-bit wide vector processors), and a 96MB on-die SRAM cache with an impressive 19. 2 TB/s bandwidth. Additionally, Gaudi 3 comes equipped with 24 200 GbE networking interfaces and 14 media engines capable of processing H. 265, H. 264, JPEG, and VP9, thus supporting vision-related tasks. The unit is paired with 128GB of HBM2E memory across eight memory stacks, delivering a substantial bandwidth of 3. 67 TB/s. Compared to its predecessor, Gaudi 2, which includes 24 TPCs, two MMEs, and 96GB of HBM2E memory, the Gaudi 3 signifies a notable upgrade. However, it appears that Intel has simplified the architecture, as the Gaudi 3 processor now only supports FP8 matrix operations and BFloat16 matrix and vector operations, eliminating support for FP32, TF32, and FP16. In terms of performance, Intel claims the Gaudi 3 can achieve up to 1856 BF16/FP8 matrix TFLOPS and 28. 7 BF16 vector TFLOPS, all while operating at around 600W TDP. On paper, when compared to Nvidia's H100, Gaudi 3 shows slightly lower BF16 matrix performance (1, 856 vs. 1, 979 TFLOPS), half the FP8 matrix performance (1, 856 vs.

3, 958 TFLOPS), and significantly reduced BF16 vector performance (28. 7 vs. 1, 979 TFLOPS). However, beyond the technical specifications, the actual real-world performance of Gaudi 3 will be crucial. It must contend with AMD's Instinct MI300 series and Nvidia's H100 and B100/B200 processors. This competition rests heavily on software performance and other variables, which are yet to be fully assessed. Currently, Intel has shared slides suggesting that Gaudi 3 may provide a substantial price-to-performance advantage compared to Nvidia's H100. Earlier this year, Intel announced that a complete accelerator kit featuring eight Gaudi 3 processors on a single baseboard would be priced at $125, 000, making the cost per processor approximately $15, 625. In comparison, an Nvidia H100 card is being sold for around $30, 678, illustrating Intel's plan for a significant pricing edge over its competitor. Nevertheless, with Nvidia's Blackwell-based B100/B200 GPUs potentially offering considerable performance benefits, it remains uncertain whether Intel can sustain its competitive advantage.