NVIDIA is developing a “top-secret” dedicated inference computing platform designed to address the massive speed and cost demands of the next generation of AI agents.
The new system, which is expected to be officially unveiled at NVIDIA’s GTC conference in San Jose later this month, represents a major strategic shift from training-focused GPUs to specialized inference hardware.
The Strategy: Integrating Groq Technology
In a move to block competitors like Cerebras and SambaNova, NVIDIA has reportedly struck a $20 billion licensing deal with the chip startup Groq.
- Hybrid Architecture: The new platform will reportedly combine NVIDIA’s Blackwell/Rubin GPUs with Groq’s Language Processing Units (LPUs).
- Deterministic Performance: Unlike traditional GPUs that excel at parallel training, Groq’s LPU technology is designed for ultra-low latency, making it ideal for the “decoding” stage of LLMs (where words are generated one by one).
- Talent Acquisition: As part of the deal, NVIDIA has hired Groq’s founding CEO, Jonathan Ross (a key designer of Google’s original TPU), and its President, Sunny Madra.
OpenAI as the Anchor Customer
The platform is being built in close collaboration with OpenAI, which has reportedly been unsatisfied with current hardware speeds for complex tasks like software development.
- Codex Integration: OpenAI plans to use the new chip to power its Codex programming tools to compete with Anthropic’s Claude Code.
- Dedicated Capacity: This deal follows NVIDIA’s recent $30 billion investment into OpenAI, reinforcing a “cycle” where OpenAI uses NVIDIA’s capital to buy its high-end specialized chips.
Projected Impact on “Agentic AI”
NVIDIA’s pivot to a dedicated inference platform addresses the growing “Inference Gap” in the industry:
| Metric | New Inference Platform (Projected) | Current H100/Blackwell GPU |
| Primary Task | Real-time Reasoning / Agents | Model Training / Batch Processing |
| Token Cost | Up to 10x Reduction | High (Costly for long-running agents) |
| Latency | Near-Instant (LPU-based) | Variable (Parallel processing overhead) |
| Architecture | Deterministic/Modular | Standard Parallel GPU |
The Rubin Platform (January 2026 Announcement)
This new “secret” chip is likely a core component of the NVIDIA Rubin platform, which was teased at CES 2026. The Rubin architecture introduces:
- NVIDIA Vera CPU: A new high-performance processor.
- Inference Context Memory Storage: A new class of AI-native storage to help agents “remember” long-running tasks without re-processing entire datasets.
