The Future Development Trend of Computing Chips: Insights from the Rise of Large Models like ChatGPT and More

The Future Development Trend of Computing Chips: Insights from the Rise of Large Models like ChatGPT and More

With the emergence of large models like ChatGPT, it is evident that the development of computing chips is heading towards a promising future. These models require significant computational power and efficient data transmission capabilities to achieve their full potential. In this article, we will explore the common points between the rise of these large models and the development trends of computing chips, as well as discuss some unique insights and actionable advice for designers and developers in this field.

One crucial aspect to consider when designing computing chips for large models is the efficient transmission of data between nodes. In a network with N nodes, the total number of connections required is N*(N-1)/2. Statistics show that currently, the east-west network traffic accounts for over 85% of the total traffic in large data centers. Furthermore, in AI training clusters with over 1000 nodes, the east-west traffic estimation surpasses 90%. This emphasizes the need for high-performance network optimization techniques such as congestion control, multipath load balancing ECMP, out-of-order delivery, scalability, fast fault recovery, and Incast optimization. By optimizing these functionalities, designers can achieve better high-performance network capabilities, enabling more efficient interaction of east-west traffic among cluster nodes.

Another significant development trend in computing chips is the evolution of memory architecture, specifically the collaborative development of CXL (Compute Express Link) and RDMA (Remote Direct Memory Access). Traditionally, swapping memory pages to solid-state drives (SSDs) results in significant performance losses. However, this drawback has created opportunities for new DRAM-based capacity tiers. Referred to as "remote memory," this DRAM can exist in another server or memory device. CXL, with its three-year history, has garnered extensive industry support surpassing previous coherent interconnect standards like CCIX, OpenCAPI, and HyperTransport. Interestingly, even though Intel was the original developer of the specification, AMD has also supported and implemented CXL. For server designers, adding CXL support only requires the latest EPYC or Xeon processors, along with careful consideration of PCIe channel allocation.

The CXL specification defines three device types and three protocols required for three different use cases. The CXL.mem protocol is essential for cache-coherent memory access. All three device types require the CXL.io protocol, but Type 3 devices use it only for configuration and control purposes. The key distinction lies in the fixed-length messages used by CXL.mem (and CXL.cache), while CXL.io employs variable-length data packets similar to PCIe. In versions 1.1 and 2.0, CXL.mem utilizes 68-byte flow control units (flits) to handle 64-byte cache lines. In CXL 3.0, a 256-byte flit introduced in PCIe 6.0 accommodates forward error correction (FEC) while optimizing latency by splitting the error check (CRC) into two 128-byte blocks.

Now, let's delve into some unique insights and actionable advice for designers and developers in the field of computing chips.

1. Prioritize High-bandwidth Networking Integration: As evidenced by the success of large models like ChatGPT, integrating high-performance networking capabilities into computing chips is crucial. This integration enhances the efficiency of east-west traffic interaction among cluster nodes and enables the design of even larger clusters.

2. Embrace CXL for Memory Architecture: The collaborative development of CXL and RDMA presents an exciting opportunity for server designers. By adding CXL support, designers can leverage the benefits of remote memory and incorporate DRAM-based capacity tiers into their designs. This not only enhances performance but also opens up new possibilities for memory-intensive applications.

3. Stay Updated on Specifications and Standards: The computing chip industry is continually evolving, with new specifications and standards being introduced. It is essential for designers and developers to stay updated on the latest developments, such as the CXL specification, to ensure optimal chip design and compatibility with emerging technologies.

In conclusion, the rise of large models like ChatGPT provides valuable insights into the future development trends of computing chips. Efficient data transmission capabilities, high-bandwidth networking integration, and advancements in memory architecture are key areas of focus. By incorporating these insights into chip design and development, we can pave the way for more powerful and efficient computing systems.