Intel’s enterprise product segment has recently made new strides. In the CPU sector, the company has reclaimed its leadership position with the Xeon 6900P (Granite Rapids) product line after a long period away from the top spot. Meanwhile, in the field of artificial intelligence, Intel Gaudi 3 is not aiming to compete for the number one position but is instead focused on attracting more customers with a significantly lower price point compared to its rivals.
First of all, Intel Gaudi 3 features a chiplet design—specifically, it includes multiple AI dies (whereas Gaudi 2 only possesses a single AI die along with HBM memory dies). It consists of two AI dies placed side-by-side, surrounded by eight HBM2e dies with a total capacity of 128 GB. Despite the large memory capacity, HBM2e is still an older memory standard, making this AI accelerator somewhat “disadvantaged” compared to competitors using HBM3 from AMD and NVIDIA.
Technically, the two Gaudi 3 AI dies collectively feature 64 TPC, 8 MME, 96 MB SRAM, and 24 Ethernet interfaces, providing a total bandwidth of up to 1200 GB/s per direction. Gaudi 3 is also equipped with 16 PCIe 5.0 lanes, allowing it to interact with Xeon CPUs as a server. Additionally, Intel has integrated 14 decoders to accelerate the decoding process of image formats such as H.265, H.264, JPEG, and VP9. Compared to the previous version, Gaudi 2 only had 24 TPC, 2 MME, 48 MB SRAM, and 96 GB HBM2e, showing that Gaudi 3 possesses vastly superior power.
However, one drawback of the Intel Gaudi 3 is that it does not support TF32, FP32, and FP16 operators like Gaudi 2, but is only compatible with FP8 and BF16. This indicates that Gaudi 3 is primarily an AI accelerator. It will not be available for supercomputing applications that require FP32 processing or higher. The H100 maintains support for FP32, FP64, TF32, and FP16.
In terms of physical form factors, Intel Gaudi 3 is provided in three versions: PCIe (HL-338), OAM (HL-325L), and a baseboard containing 8 OAM chips (HLB-325). At the Computex event this past June, Intel announced that a rack server containing 8 Gaudi 3 chips costs $125,000, meaning each chip costs an average of approximately $15,625. Currently, an H100 card costs over $30,000, showing that the price of Gaudi 3 is only half that of NVIDIA’s equivalent product.
Regarding performance, Intel Gaudi 3 has three versions with the same 600 W TDP configuration, so there may be no difference in power between them (whereas the H100 SXM and PCIe versions do have a difference). According to Intel, when performing BF16/FP8 matrix calculations, Gaudi 3 reaches a speed of 1835 TFlops, while for BF16 vector, it is 28.7 TFlops. For the H100 (SXM version), these figures are 1979 TFlops for BF16 (matrix), 3958 TFlops for FP8 (matrix), and 1979 TFlops for BF16 (vector). However, when comparing LLaMA 3 model processing capabilities, Intel states that Gaudi 3 is 1.09 times more powerful than H100 at 8 billion parameters and 1.19 times more powerful at 70 billion parameters. This information shows that beyond hardware, software architecture also plays a crucial role in affecting actual computing performance.
Currently, Gaudi supports many popular artificial intelligence models globally, combined with various libraries and programming languages. Customers can experience the power of Gaudi 3 through the company’s Tiber Developer cloud service or on the IBM Cloud platform. For those wanting to experience it directly, server systems integrated with Intel Gaudi 3 will be provided by brands such as Dell, HP, and Supermicro, expected to be available in the fourth quarter of 2024.
According to Tinhte.vn
