The core job of AI infrastructure has shifted from supporting large-model inference to supporting the scaled operation of massive numbers of agents and the continuous production of high-quality tokens.

According to IDC, China’s enterprise AI agent market will be worth about 19 billion yuan in 2025, with a compound annual growth rate expected to exceed 110% from 2025 to 2028. Gartner’s view is more direct: by 2026, 40% of enterprise applications will integrate task-oriented AI agents.

In the large-model inference phase, AI infrastructure only needed to support a single input and a single output.

In the agent phase, infrastructure must support task decomposition, tool calling, multi-round collaboration and continuous operation, placing an entirely different set of demands on computing power.

Behind this are two capability gaps: whether massive numbers of agents can run at scale and with stability, and whether multiple models can work together to make agents smarter.

At the 2026 Open Compute Conference, Inspur Information presented new product solutions for both gaps.

New Requirements for Agent Infrastructure

In the agent era, expectations for AI infrastructure have changed as well.

In the past, enterprise AI deployments mostly meant connecting one or two models to handle relatively standalone tasks: one call, one response, and the job was done.

Agents work differently.

An agent application first breaks down a task, then calls tools step by step and collaborates back and forth. Behind it may be an entire group of sub-agents running at the same time.

Once deployed inside an enterprise, the number of agents can easily reach the thousands or tens of thousands.

How to keep such a large population of agents running stably and collaboratively has become an unavoidable new problem.

The pressure is also rising on individual models as the number of agents grows.

Some models are strong at logical reasoning, while others are better at writing text. These uneven capabilities cannot be fixed simply by adding more parameters.

Real-world tasks, meanwhile, are becoming more complex, making it hard to expect one model to do everything.

So another problem that must be solved is how to let multiple models divide work, collaborate and compensate for one another’s weaknesses.

For both of these ideas to work in practice, underlying computing power infrastructure has to lead the way.

As these new demands arrive, the first shift in infrastructure is that CPUs are becoming more important and taking on a larger role.

Under the old question-and-answer model, large-model inference involved one input and one output, and relied more heavily on GPUs.

Agents are different. They need to break down tasks, call tools, collaborate across multiple rounds and summarize results. These integer operations and logic-heavy processes run on CPUs.

Agents also do not clock out after a single run. Many must stay online year-round, greatly extending runtime.

As a result, the AI infrastructure industry’s computing power mix is changing, moving away from a GPU-centered model toward coordinated systems built from multiple types of computing power.

Once CPUs become more important, the next challenge is power density.

Zhao Shuai, deputy general manager of Inspur Information, said domestic AI cabinet power will reach 300 kW within this year, while some cabinets globally have already entered the megawatt range.

If CPU-side density remains at just a few kilowatts per rack, it cannot match the new power infrastructure of data centers.

As rack power keeps rising, cooling becomes a problem. Traditional air cooling, capped at about 40 kW to 50 kW per rack, can no longer keep up, making liquid cooling a necessity.

Running 40,000-Plus Agents in a Single Rack

To enable agents to run at scale, Inspur Information released what it calls the industry’s first CPU-native liquid-cooled full-rack server.

A single rack can support up to 384 CPU processors based on the open OCM, or Open Compute Module, architecture. It is compatible with x86 and ARM and can support the coordinated operation of more than 40,000 agents.

That scale is 40 times larger than Inspur Information’s “Qi Qian Xia” solution released in April this year.

Qi Qian Xia used a single 2U server to deploy 1,000 OpenClaw agents. This time, Inspur Information has filled an entire rack with agents.

The rack also uses the OCM architecture, which can support processors from different generations and architectures, eliminating the need to redesign the entire system for each new chip generation and sharply shortening the R&D cycle.

Packing 384 CPUs into one rack makes cooling an unavoidable hurdle.

Inspur Information proposed a new cooling concept: native liquid cooling.

The idea overturns the traditional logic of hybrid air-liquid cooling. In older cold-plate liquid-cooled server designs, cooling plates were attached to compute components, while memory, network cards and hard drives still required separate work to move heat out through fans.

This time, the approach is completely different: compute and cooling are designed together. The system breaks past the limits of traditional liquid cooling, which covered only CPUs, and brings all heat-generating components, including memory, network cards, optical modules and SSDs, into the liquid-cooling system, rebuilding the entire computing power system.

Inspur Information’s specific approach is to make the computing unit into an ultra-thin 2U form factor, fit 16 CPUs into one node, and lay memory, network cards and optical modules, which previously depended on fans and cables for cooling and connectivity, directly onto the motherboard. A single cold plate handles heat dissipation across the board, even eliminating the server tray.

This frees up the space once occupied by fans, cooling pipes and cables for compute and I/O resources. The full rack also adopts a cable-free design, supports hot maintenance, ensures zero service interruption and improves rack-level operations and maintenance efficiency by more than 100%.

Multiple Large Models Collaborate on One Task

To make agents smarter, Inspur Information also launched a multi-model fusion API on its YuanNao QiZhi EPAI platform and released the enterprise edition of the YuanNao SD200 supernode AI server.

Multi-model fusion sends the same task to multiple candidate models at once, lets each generate an answer independently, and then has a review-and-fusion model compare areas of consensus, disagreement, omissions and distinctive viewpoints before assembling a unified output.

Not every task has to go through this full process. For short tasks such as simple Q&A, tool calling and format conversion, the system routes them directly to a lighter single model.

Only complex, long-chain tasks are scheduled across multiple candidate models, avoiding overkill for simple jobs.

This mechanism achieved a score of 53.9% in the DRACO test, outperforming every single model in the same candidate pool.

The capability is now available externally through the multi-model fusion API. It can be plugged directly into applications like a regular model service, or configured into agents and development frameworks while preserving existing dialogue, reasoning and tool-calling workflows.

But that creates another problem: having multiple large models participate in one task inevitably raises the bar for underlying computing power. The system must fit multiple trillion-parameter models at once without sacrificing output speed.

That is the part the YuanNao SD200 supernode is designed to handle.

When the YuanNao SD200 supernode was released last year, it could already deploy four trillion-parameter large models simultaneously and generate tokens in 8.9 milliseconds, making it the first domestic product to break the 10-millisecond threshold.

This year, that figure has been pushed down further to 4.77 milliseconds, making it the first domestic solution to enter the sub-5-millisecond range. First-token latency has also been reduced by 35%.

Behind these gains are software-hardware co-optimizations such as multi-token prediction, W4A8 precision and JIT just-in-time compilation.

Multi-token prediction allows the decoding stage to generate multiple candidate tokens at once and then verify them, reducing the number of step-by-step generation rounds;

W4A8 lowers the compute precision of MoE modules in trillion-parameter models from BF16 to INT8, reducing pressure on memory access bandwidth;

JIT dynamically generates dedicated GPU kernels at runtime based on tensor shapes, bringing computing power closer to the characteristics of the hardware.

The YuanNao SD200 supernode has already been adapted for mainstream open-source models including Kimi K2.6, DeepSeek V4, GLM 5.2 and MiniMax M3.

Still, the threshold for this architecture remains high for many small and midsize enterprises.

For that reason, Inspur Information also launched the enterprise edition of the YuanNao SD200 supernode, which can be understood as a smaller version of the YuanNao SD200.

It reduces the scale-up compute domain from 64 cards to 16 cards and cuts first-token latency for trillion-parameter models by more than 40%, giving enterprises a lower-cost option for migration and adaptation.

As a result, enterprises that previously could only deploy 100-billion-parameter models for auxiliary use can now put trillion-parameter models into real production environments.

Agent Infrastructure Competition Is Changing

Today, the division of labor among CPUs, GPUs and software platforms is becoming more tightly integrated.

The software platform handles model access, task orchestration, resource scheduling, permission governance and result fusion. CPUs carry agent instances, tool calls, sandbox execution and interactions with business systems. GPUs handle model inference and token generation.

Only coordination across all three can support the stable operation of massive numbers of agents and the efficient execution of complex tasks.

If any link in this chain falls behind, the entire agent application will struggle to run smoothly.

This is also changing the focus of infrastructure competition in the agent era.

In the past, the contest was over who could support a single model better. Now it is about who can deliver stronger system-level coordination.

Strength at a single point is no longer enough. What matters is whether the entire chain runs smoothly and works in concert.