The Future of AI Hardware: Google's Next-Generation Chip and the Rise of Chinese Automakers in the Self-Driving Market

The Future of AI Hardware: Google's Next-Generation Chip and the Rise of Chinese Automakers in the Self-Driving Market

In the rapidly evolving field of artificial intelligence (AI), hardware advancements play a crucial role in pushing the boundaries of what machines can achieve. Recently, Google unveiled its latest AI chip, which aims to address the challenges faced in improving AI hardware performance. This new chip, known as the Tensor Processing Unit (TPU), is specifically designed for dense matrix multiplication. Through the use of High Bandwidth Memory (HBM), the memory bandwidth of these matrix math engines is increased by tenfold.

But Google's innovation doesn't stop there. They have also created specialized hardware accelerators for scatter/gather operations in sparse matrices, aptly named Sparsecore. These accelerators are embedded in the TPUv4i, TPUv4, and potentially the TPUv5e engines. By employing liquid cooling, system power efficiency is maximized, leading to improved cost-effectiveness. Additionally, the use of mixed precision and dedicated number representations enhances the actual throughput of the devices, referred to as "effective throughput" by Vahdat.

One notable feature of Google's AI hardware is the synchronous, high-bandwidth interconnect for parameter allocation. This interconnect, which is essentially an optical switch, allows for near-instantaneous reconfiguration of the network when jobs change on the system. Furthermore, it enhances the fault tolerance of the machines. This is a significant development for systems with tens of thousands of computing engines and workloads that take months to run. HPC centers worldwide have recognized the importance of this advancement.

Meanwhile, in the self-driving market, Chinese automakers are striving to replicate the success of Mobileye, a company that has exited the Chinese market. Car manufacturers are eager to have in-house algorithms and are increasingly turning to Nvidia, a dominant force in both marketing and technology, and surpassing low-computational power chip manufacturers like Mobileye.

The capabilities of self-driving systems vary among vehicle models that utilize Nvidia's Orin chip, which is already in mass production. Surprisingly, cheaper and lower-computational power chips, such as the EQ black box version, have not exited the market. Instead, they have taken the form of domain controllers, propelling hardware shipment volumes for companies like Zongmu Technology to the top three. In China, at least three major automakers are developing their own "customized low-computational power chips" to better adapt to their proprietary algorithms, including Transformer, and sensors. These chips are expected to be released and mass-produced by 2025. In other words, Chinese electric vehicle manufacturers, driven by sales volume and vertical integration ambitions, are perfectly replicating Tesla's journey from breaking up with Mobileye to collaborating with Nvidia and eventually customizing FSD chips for their algorithms between 2016 and 2020.

As for domestic self-driving chip companies, there is currently only one known mass production project for passenger vehicles, but the CPU of J5 has some defects, and overall shipment volumes are not significant. However, when it comes to the toolchain and compiler, one line of domestic self-driving SoCs, excluding Huawei, is considered the best. In terms of CPU, GPU, or IP within an SoC, they can be purchased if desired. The real barriers to chip development lie in the maturity of IP and the chip development platform. Once the stage of excessive hype and unclear understanding passes, it is highly likely that most in-house teams of automakers who have already developed algorithms, underlying software, and middleware using the Orin platform and its powerful toolchain will end up developing their own chips.

The notion of "software and hardware decoupling" only exists in the transitional phase where there is no clear definition of self-driving functionality, no experience in algorithm development, and blind pursuit of computational power. Most domestic chip companies in the field clearly follow the path of "chip defining AI" rather than "AI defining chip," with a strong presence of traditional chip companies' rustiness and rigidity.

In conclusion, the advancements in AI hardware by Google and the rise of Chinese automakers in the self-driving market are shaping the future of AI technology. As hardware capabilities continue to improve, it opens up new possibilities for AI applications and pushes the boundaries of what machines can achieve. To stay ahead in this rapidly evolving field, here are three actionable pieces of advice:

1. Invest in specialized hardware: As demonstrated by Google's TPU and Sparsecore, creating dedicated hardware accelerators for specific AI tasks can significantly enhance performance and efficiency.

2. Embrace liquid cooling: Liquid cooling technology has proven to be highly effective in maximizing system power efficiency, which ultimately leads to cost savings. Consider incorporating liquid cooling into AI hardware designs for optimal performance.

3. Foster collaboration between automakers and chip manufacturers: The success of Chinese automakers in replicating Tesla's journey highlights the importance of collaboration between automakers and chip manufacturers. By working closely together, they can develop customized chips that perfectly integrate with proprietary algorithms and sensors, ultimately driving advancements in the self-driving market.

By incorporating these strategies, companies can leverage the latest AI hardware advancements and position themselves at the forefront of the AI revolution.