The Battle for AI Computing Power: Storage Chip Giants, NVIDIA, and AMD

The Battle for AI Computing Power: Storage Chip Giants, NVIDIA, and AMD

In the race for AI computing power, storage chip giants NVIDIA and AMD have turned to SK Hynix to provide them with samples of the next-generation HBM3E chip, which is yet to be mass-produced. NVIDIA has requested SK Hynix to supply HBM3E as quickly as possible and is even willing to pay a premium for it. Currently, SK Hynix holds a 50% market share in the global HBM market for 2022, with Samsung following closely behind with a 40% share. Micron, a US storage chip giant, ranks third with a 10% share.

Since the second quarter, the demand for high-end servers driven by generative AI has increased rapidly. Consequently, the demand for high-density DDR5 and HBM products is also growing quickly. Jaejune Kim, the Executive Vice President of Samsung's memory business, discussed the memory demand in the server sector during a Samsung earnings call. Due to the strong demand for generative AI, investments in the data center sector have been focused on AI servers with limited capital expenditure, resulting in relatively limited demand for general-purpose servers and storage.

According to Kim Woohyun, SK Hynix's graphic DRAM sales (including HBM products) previously accounted for a single-digit percentage of the total DRAM sales. However, since reaching 10% of DRAM sales in the fourth quarter of last year, this proportion has rapidly increased to over 20% in the second quarter. SK Hynix has experienced significant growth in sales of HBM and high-density DDR5 modules for AI servers. The company expects HBM sales to more than double this year compared to last year and further increase next year.

Despite the increasing demand for AI computing power and storage chips, there are still several misconceptions surrounding GPUs in the field of generative AI. One common misunderstanding was the significant time consumed by data replication in each time step, accounting for about 70% of the time. This data replication was necessary to complete the various stages of the data flow process.

However, recent advancements in GPU technology have addressed this issue. By utilizing techniques such as data parallelism and memory sharing, GPUs have significantly reduced the time spent on data replication. This optimization has resulted in faster AI computations and improved overall efficiency.

Another misconception is that GPUs are only suitable for training AI models and not for inferencing. While it is true that GPUs excel in training due to their parallel processing capabilities, they are also highly efficient in inferencing tasks. GPUs can handle large-scale parallel computations required for inferencing, allowing for real-time predictions and analysis.

Additionally, there is a misconception that GPUs are only beneficial for deep learning models and not for other AI algorithms. While deep learning models heavily rely on the parallel processing power of GPUs, other AI algorithms also benefit from GPU acceleration. Tasks such as image recognition, natural language processing, and recommendation systems can all be significantly accelerated using GPUs, leading to faster and more accurate results.

To capitalize on the advantages of GPUs in generative AI, here are three actionable pieces of advice:

1. Invest in GPUs for AI infrastructure: Whether it's for training or inferencing, GPUs offer immense computational power and efficiency. Incorporating GPUs into AI infrastructure can greatly enhance performance and accelerate AI-driven tasks.

2. Stay updated with GPU advancements: GPU technology is evolving rapidly, with new features and optimizations being introduced regularly. Keeping up with these advancements ensures that AI systems can leverage the latest capabilities for improved performance and efficiency.

3. Consider GPU acceleration for various AI algorithms: While deep learning models are often associated with GPUs, other AI algorithms can also benefit from GPU acceleration. Evaluating the suitability of GPU acceleration for different AI tasks can help determine the optimal use of computing resources.

In conclusion, the demand for AI computing power and storage chips is on the rise, with storage chip giants like NVIDIA and AMD seeking the next-generation HBM3E chip from SK Hynix. As the market expands, SK Hynix's market share in the HBM sector has grown significantly, fueled by the increasing adoption of AI servers. While misconceptions about GPUs in generative AI persist, advancements in GPU technology have debunked many of these myths. By leveraging GPUs and staying informed about their advancements, organizations can unlock the full potential of AI computing power.