Intel's enterprise business has seen some new progress. In CPUs, the company has taken back the lead with its Xeon 6900P (Granite Rapids) lineup after a long stretch away from the top spot. In AI, Intel Gaudi 3 isn't aiming to win the number one position. Instead, it aims to attract more customers with a price that is significantly lower than its rivals'.
To start, Intel Gaudi 3 is a chiplet design, meaning it is built from multiple AI dies (whereas Gaudi 2 had a single AI die alongside its HBM memory dies). It consists of two AI dies placed side by side and surrounded by eight HBM2e dies with a total capacity of 128 GB. Despite the large memory capacity, HBM2e is still an older memory standard, so this AI accelerator is at somewhat of a disadvantage compared with rivals from AMD and NVIDIA that use HBM3.
On the technical side, the two Gaudi 3 AI dies together have 64 TPCs, 8 MMEs, 96 MB of SRAM and 24 Ethernet ports, delivering a total bandwidth of up to 1200 GB/s in each direction. Gaudi 3 also gets 16 lanes of PCIe 5.0, which let it talk to Xeon CPUs acting as the host. In addition, Intel has integrated 14 decoders to speed up decoding of image and video formats such as H.265, H.264, JPEG and VP9. By comparison, the previous generation, Gaudi 2, had only 24 TPCs, 2 MMEs, 48 MB of SRAM and 96 GB of HBM2e, which shows that Gaudi 3 is far more powerful.
Even so, Intel Gaudi 3 has one drawback: unlike Gaudi 2, it does not support TF32, FP32 and FP16 operations, and is compatible only with FP8 and BF16. This shows that Gaudi 3 is primarily an AI accelerator. It won't be usable for supercomputing applications that need FP32 or higher precision. The H100 still supports FP32, FP64, TF32 and FP16.
In terms of physical form, Intel Gaudi 3 comes in three formats: PCIe (HL-338), OAM (HL-325L) and a baseboard holding 8 OAM chips (HLB-325). At Computex last June, Intel announced that a server rack holding 8 Gaudi 3 chips costs 125,000 USD, which works out to an average of about 15,625 USD per chip. An H100 card currently costs more than 30,000 USD, so Gaudi 3 is priced at only about half of the comparable NVIDIA product.
On performance, Intel Gaudi 3 comes in three versions with the same 600 W TDP configuration, so there may be no difference in power between them (whereas the SXM and PCIe versions of the H100 do differ). According to Intel, for BF16/FP8 matrix calculations, Gaudi 3 reaches 1835 TFlops, and for BF16 vector it reaches 28.7 TFlops. The corresponding figures for the H100 (SXM version) are 1979 TFlops for BF16 (matrix), 3958 TFlops for FP8 (matrix) and 1979 TFlops for BF16 (vector). However, when comparing performance on the LLaMA 3 model, Intel says Gaudi 3 outperforms the H100 by 1.09 times at 8 billion parameters and 1.19 times at 70 billion parameters. This suggests that beyond the hardware, the software stack also plays an important role in real-world compute performance.
Gaudi currently supports many of the world's popular AI models, along with a range of libraries and programming languages. Customers can try out Gaudi 3 through the company's Tiber Developer cloud service or on the IBM Cloud platform. For hands-on use, server systems with Intel Gaudi 3 will be offered by brands such as Dell, HP and Supermicro, and are expected to arrive in the fourth quarter of 2024.
Source: Tinhte.vn
