TrendForce’s latest AI Server Industry Report reveals that strong demand for advanced AI servers from leading CSPs and branded customers is expected to continue in 2024. At the same time, the gradual production expansion of TSMC, SK hynix, Samsung, and Micron has significantly eased shortages in Q2 2024. As a result, the lead time for NVIDIA’s flagship H100 solution has been reduced from 40-50 weeks to less than 16 weeks.
TrendForce estimates that AI server shipments in the second quarter will increase nearly 20% quarter-on-quarter and has revised the annual shipment forecast to 1.67 million units, representing a growth of 41.5% year-on-year.
TrendForce notes that this year, major CSPs continue to focus their budgets on purchasing AI servers, which is holding back the growth momentum of general servers. Compared with the high growth rate of AI servers, the annual growth rate of general server shipments is only 1.9%. The share of AI servers in total server shipments is expected to reach 12.2%, an increase of about 3.4 percentage points from 2023.
In terms of market value, AI servers contribute significantly to revenue growth, more than general servers. The market value of AI servers is expected to exceed $187 billion in 2024, with a growth rate of 69%, accounting for 65% of the total server market value.
North American CSPs (e.g. AWS, Meta) are continuously developing their proprietary ASICs, and Chinese companies such as Alibaba, Baidu, and Huawei are actively developing their own ASIC AI solutions. This is expected to increase the share of ASIC servers in the total AI server market to 26% in 2024, while GPU-based consumer AI servers will account for about 71%.
In terms of AI chip suppliers for AI servers, NVIDIA holds the largest market share, approaching 90% for GPU-based AI servers, while AMD’s market share is only about 8%. However, including all AI chips used in AI servers (GPU, ASIC, FPGA), NVIDIA’s market share this year is about 64%.
TrendForce observes that demand for advanced AI servers is expected to remain strong through 2025, especially with NVIDIA’s next-generation Blackwell (including GB200, B100/B200) expected to replace the Hopper platform as the mainstream platform in the market. This will also drive demand for CoWoS and HBM. For NVIDIA’s B100, the die size will be twice that of the H100, consuming more CoWoS. Leading supplier TSMC’s CoWoS production capacity is expected to reach 550,000-600,000 units by the end of 2025, with a growth rate close to 80%.
By 2024, the consumer H100 will be equipped with 80GB of HMB3. By 2025, leading chips such as NVIDIA’s Blackwell Ultra or AMD’s MI350 are expected to be equipped with 288GB of HBM3e, tripling the unit’s utilization. The overall HBM supply is expected to double by 2025 due to continued strong demand in the AI server market.
TrendForce analysts will be presenting keynotes at the FMS24 – Future of Memory and Storage Forum from August 6th to 8th. Presentations will cover topics such as HBM, memory (DRAM/NAND Flash), servers, AI servers, storage, and technology and capacity developments. We will also have booth #956 available for analyst meetings. If you are interested in these topics, we invite you to contact us. Please feel free to contact us. calendar an appointment or visit our stand!
