On February 27th local time, Canadian AI chip startup Tenstorrent announced a multi-level cooperation agreement with Japan's Advanced Semiconductor Technology Center (LSTC) to collaborate in designing advanced artificial intelligence (AI) chips.

It is worth mentioning that Tenstorrent will collaborate with Japanese semiconductor company Rapidus to develop the most advanced logic semiconductor technology, with the goal of achieving the world's best cycle time reduction services. Tensorrent will also utilize its Ascalon RISC-V CPU core technology to jointly develop RISC-V architecture CPU chips for LSTC's new edge AI accelerator.

In recent years, with the outbreak of large-scale generative AI applications such as ChatGPT and Sora, the demand for AI chips in markets such as cloud computing and AI servers has significantly increased, and the industry's attention to AI chips continues to rise.

Amidst the booming AI market, in addition to companies collaborating to strengthen research and development, recent industry news has also shown that AI chip production capacity is scarce, the important memory technology HBM required by AI is sold out, and the demand for high-end AI servers is increasing

AI chip production capacity is scarce

The demand for AI chips has skyrocketed, and their production capacity has also attracted industry attention. In early February, according to media reports, Nvidia and Intel reached an intention to cooperate in OEM production, continuously producing 5000 wafers per month. If all of it is used to produce H100 chips, under ideal conditions, a maximum of 300000 chips can be obtained.

In late February, Intel launched its first system level foundry for the AI era to the industry, Intel Foundry, and expanded its roadmap to establish and consolidate process technology leadership in the coming years.

Regarding this, TSMC founder Zhang Zhongmou, the leader in wafer foundry, stated at the opening ceremony of JASM at Kumamoto Factory in Japan that there will definitely be more demand in the semiconductor industry in the future. Recently, AI professionals have told him that what he needs is not only tens of thousands, hundreds of thousands, and tens of millions of wafers, but also 3, 5, and even 10 wafer factories.

However, Zhang Zhongmou believes that the demand brought by AI to the semiconductor industry has to some extent taken an intermediate value, that is, finding the answer from thousands of chip production capacity to 10 wafer factories.

In response to the shortage of AI chips, Liu Yangwei, Chairman of Foxconn's parent company, Hon Hai Precision, stated that Hon Hai's AI server business has been quite good this year. However, the overall AI server industry is still facing a major shortage of AI chips. Even if the supply of AI chips slows down in the second half of the year, it still cannot catch up with demand. We must wait until the production capacity of upstream new factories is opened to solve the shortage of materials in the industry chain.

HBM sold out

With the explosive popularity of AI, the market has a strong demand for high bandwidth memory (HBM), and storage giants are targeting HBM and actively expanding their production layout. Among them, Samsung plans to increase the maximum production of HBM to 150000 to 170000 pieces per month by the fourth quarter of this year. The company has invested 10.5 billion Korean won to acquire Samsung Display's factory and equipment in Tian'an City, South Korea, to expand HBM production capacity, and also plans to invest 700 billion to 1 trillion Korean won to build a new packaging line.

SK Hynix and Micron Technology have announced that HBM orders are about to be full. SK Hynix Vice President Kim Ki tae stated that the company's HBM has been sold out this year and preparations have begun for 2025; Sanjay Mehrotra, CEO of Micron Technology, revealed that Micron's HBM production capacity for 2024 is expected to be fully sold out.

The demand for high-end AI servers will exceed 60%

According to the latest estimate from TrendForce Consulting, based on the demand for high-end AI servers (including those equipped with NVIDIA, AMD, or other high-end ASIC chips) by major global cloud service providers (CSPs) in 2024, it is estimated that the four major CSP providers in the United States, including Microsoft, Google, AWS, and Meta, will account for 20.2%, 16.6%, 16%, and 10.8% of global demand respectively, totaling over 60%, ranking at the forefront of the world. Among them, AI server models equipped with Nvidia GPUs account for the majority.

TrendForce Consulting pointed out that in recent times, Nvidia's overall revenue source has been focused on data center business, mainly due to its GPU servers accounting for 60-70% of the overall AI market. However, it is still necessary to pay attention to three major situations in the future, which may limit Nvidia's development.

TrendForce Consulting believes that, firstly, due to changes in the international situation, China will be more committed to the localization of AI chips. However, Nvidia's H20 and other Chinese specific solutions may not be as cost-effective as existing H100 or H800 solutions, and Chinese customers may adopt them more conservatively than before, further affecting Nvidia's market share.

Secondly, considering both scale and cost, large American CSP companies, in addition to Google and AWS, are also expanding their adoption of self-developed ASICs by Microsoft, Meta, and others year by year.

Thirdly, due to industry competition from AMD, AMD adopts a high cost performance strategy, benchmarking against NVIDIA's peers. AMD offers only 60-70% of the price, and even representative or large-scale customers can enter the market with a lower price strategy. It is expected that Microsoft will be the most actively adopting AMD's high-end GPU MI300 solution in 2024.