How to build an AI factory with NVIDIA ERAs

TL;DR

NVIDIA Enterprise Reference Architectures provide validated end to end blueprints for deploying scalable AI infrastructure. They are tested in NVIDIA labs, use a scalable unit of four nodes, and optimize token efficiency, cost per million tokens, and power usage. The approach supports fast time to first token and applies across industries like manufacturing and healthcare.

Transcript

I am super excited about this one. This is a brand new live series from NVIDIA where we are trying to pull back the curtain on how AI factories are actually being built. I'm your host, Kaushik Shirhatti. And what we will be doing is we are going to bring in folks for every episode where we're bringing in folks who are actually in the trenches makin... Read More

Key Insights

• ERAs provide validated, end to end blueprints that reduce deployment guesswork for enterprise customers.
• A scalable unit in ERA design equals four nodes, enabling predictable growth from four to eight to more nodes as needed.
• ERAs are tested in NVIDIA labs and with OEM partners to ensure real world reliability before market release.
• Co design across hardware and software is essential to achieve near linear efficiency in token generation and power usage.
• Networking and storage decisions are integrated with compute to minimize bottlenecks and maximize throughput.
• The ERA framework supports multiple industry use cases, including manufacturing, healthcare, and financial services, showing broad applicability.
• ERAs evolve with hardware roadmaps, from Hopper to Blackwell and beyond, to stay current with GPU, DPUs, and networking advances.
• The enterprise focus includes governance, security, and sovereignty considerations, making deployments suitable for enterprise data centers.
• More videos from this channel:
  • How to create digital twins with Omniverse for AI?
  • How Is NVIDIA Building the Future of AI Computing?
  • How Does NVIDIA RTX Spark Reinvent the PC?
• More from this channel: NVIDIA

Questions & Answers

Q: What are NVIDIA Enterprise Reference Architectures and why do they matter?

NVIDIA Enterprise Reference Architectures are validated, end to end blueprints for AI infrastructure designed to accelerate deployment for enterprises while delivering predictable scale and performance. They are built in NVIDIA labs, tested with NVIDIA certified systems, and reviewed by a design review board with OEM partners to ensure a reliable, repeatable deployment pattern. This matters because it reduces guesswork, shortens time to first token, and provides a scalable foundation that can grow from four nodes to larger configurations as needs evolve.

Q: How is the scalable unit defined in ERAs and why is it important?

The scalable unit in ERAs is four nodes. This standard unit allows customers to validate performance at a smaller scale and then grow in a predictable, linear fashion to larger deployments. By defining a fixed growth increment, it helps plan capacity, networking, storage, and power requirements, while ensuring consistent performance characteristics across all scales. This approach minimizes surprises during expansion and supports budgeting and procurement processes.

Q: What does end to end validation involve in ERA testing?

End to end validation in ERA testing involves building the cluster in NVIDIA labs, validating against NVIDIA certified systems, and running representative workloads to confirm performance and reliability. The process includes testing East West networking for storage performance, North South networking, and design reviews with OEM partners to ensure the final solution behaves as expected in real deployments. This comprehensive validation reduces deployment risk for enterprises.

Q: How do ERAs address the needs of agentic AI and non sleeping agents?

ERAs address agentic AI needs by focusing on token efficiency and scalable, predictable performance, ensuring architectures can sustain continuous operation without degradation under heavy inference and reasoning workloads. They aim for optimal cost per million tokens and tokens per watt, achieved through co design across compute, networking, and storage. This creates a stable foundation for agents that retrieve information and call tools continuously.

Q: What are the main configurations covered by ERA and where can customers see them?

The main ERA configurations include hardware, software, and networking patterns designed to work together as a coherent AI factory. NVIDIA describes these patterns and provides validated designs that can be deployed with OEM partners such as Cisco, Dell, HPE, Lenovo, and others. Customers see these configurations through OEM offerings and NVIDIA technical documentation, including enterprise reference architecture overviews and whitepapers that explain the design principles.

Q: How do ERAs stay aligned with NVIDIA hardware roadmaps?

ERAs are treated as dynamic systems that evolve with NVIDIA hardware roadmaps. As NVIDIA introduces new GPUs and DPUs, such as moving from Hopper to Blackwell and beyond, ERAs incorporate these advances to maintain compatibility and performance. The architectures are designed to adapt to new networking switches and storage technologies as well, ensuring deployments stay current and capable of leveraging new efficiency gains.

Q: What industries have adopted ERA based AI factories and what benefits were observed?

ERAs have seen adoption across manufacturing, healthcare, retail, life sciences, financial services, and government. In manufacturing, for example, deployments enabled faster time to first token and cleaner deployments with reduced configuration drift. Across sectors, enterprises have benefited from validated patterns, reduced integration risk, and the ability to scale AI workloads while maintaining governance and security standards.

Q: Where can buyers go to acquire ERA based AI factory solutions?

Enterprises can work with NVIDIA OEM system partners or channel partners to acquire ERA based AI factory solutions. The process involves selecting an ERA aligned configuration, leveraging NVIDIA verified workloads, and using the OEMs to supply compute, networking, and storage with NVIDIA guidance. This pathway provides a vetted, repeatable deployment that aligns with enterprise needs for security, governance, and scalability.

Summary & Key Takeaways

• This summary outlines how NVIDIA Enterprise Reference Architectures (ERAs) deliver validated, repeatable blueprints for AI infrastructure, focusing on end to end validation, scalable units, and cross industry applicability. It emphasizes the practical deployment benefits such as faster time to first token and predictable performance.

• ERAs are designed to co operate across hardware and software layers, including certified systems, networking and storage, to ensure deployable and predictable AI factories for enterprises.

• The emphasis is on evolving architectures to keep pace with agentic AI demands, including non stop inference, efficient token generation, and governance considerations within enterprise environments.