As large model parameter counts race toward the trillion scale, and models such as GPT-4o and Llama 4 keep pushing performance ceilings higher, the AI industry is also running into bottlenecks it has never faced before. The low efficiency of the Transformer architecture, staggering computing power consumption, and weak grounding in the physical world are becoming increasingly apparent. The path to artificial general intelligence, or AGI, urgently needs a breakthrough. ZTE recently released a paper, “Insights into the Computing Paradigm of Next-Generation AI Large Models,” that examines the core constraints facing AI today while outlining more promising frontier directions, offering an important reference point for the industry.

The State and Bottlenecks of LLMs: Hidden Risks Behind the Scaling Rush

In 2020, OpenAI described the scaling laws for large models: the final performance of large language models, or LLMs, depends on the stacked expansion of compute, parameter count, and training data. GPT-3, with 175 billion parameters, delivered performance far beyond contemporary models across tasks such as natural language understanding and knowledge-based question answering. In recent years, large models represented by DeepSeek-V3, GPT-4o, Llama 4, Qwen3, and Grok 4 have continued to validate this law.

Building an advanced foundation model requires stacking hundreds of thousands of accelerator cards, collecting hundreds of terabytes of text data, and training nearly a trillion internal parameters on an autoregressive Transformer architecture through pre-training and post-training. The sunk cost of the full training process is extremely high. X.AI’s Grok 4, for example, reportedly took half a year to complete pre-training on a 200,000-card distributed cluster built across two 150 MW data centers. As a result, LLM pre-training exploration and practice are largely carried out in industry, while academia is mostly limited to theoretical research and smaller-scale experiments with fewer than 7 billion parameters. Even though the current architecture still faces bottlenecks in algorithms, hardware, engineering, and cost, the vision of AGI and the effectiveness of scaling laws continue to drive heavier industry investment. In the short term, the trend toward larger models is hard to reverse.

Architectural Bottlenecks in LLMs Are Becoming More Prominent

The Transformer architecture has low computational efficiency and heavy memory-access requirements. In particular, the arithmetic intensity of a decode-only autoregressive structure is only 2, meaning it can perform just two calculations for every byte of data read. Convolutional neural networks, or CNNs, can reach arithmetic intensities in the hundreds, and their high data reuse rates can fully feed the matrix multiply-accumulate units in GPUs and domain-specific architectures, or DSAs. Transformers, by contrast, suffer from higher data-movement overhead, which leads to lower model FLOPs utilization, or MFU. Current hardware also struggles to parallelize special nonlinear operators in Transformer architectures, including Softmax, LayerNorm, and Swish. In short, LLM architectures depend heavily on advanced process nodes and high-bandwidth memory, or HBM, and carry high engineering costs. These are key bottlenecks limiting large-scale deployment and further performance gains.

In the future, as foundation models keep adding parameters, reasoning models produce much longer chain-of-thought contexts, and new high-performance computing applications such as AI for Science in biopharma become more common, the bottlenecks of the Transformer architecture will become even more pronounced. That trend increasingly conflicts with the slowdown of Moore’s Law. A technology path that relies on advanced process nodes to improve computing power and energy efficiency will run into problems such as the “power wall” and the “memory wall.” The von Neumann architecture, which separates computation and storage, will face severe challenges as large models demand ever greater scale and computing power.

The Road to AGI Remains Contested

In practice, current LLMs still suffer to varying degrees from hallucinations and poor interpretability. These problems have been masked as scaling laws continue to improve model capabilities. But the core of the autoregressive Transformer architecture is “next token prediction,” leading some AI scientists, including Yann LeCun, to argue that from the perspective of sparse coding and equivariant mapping principles, existing LLMs struggle to truly understand the physical world. The fundamental flaws of today’s neural networks, including LLMs, are:

Neurons do not have intrinsic learning, memory, or decision-making capabilities; intelligence appears only at the macro level of the network.

Current AI progress depends heavily on “brute-force scaling.”

They lack embodiment and hierarchy. As a result, there is still considerable debate in academia over routes such as mapping the physical world and building world models.

From an industry perspective, the scaling-law path still needs further exploration, because an exponential inflection point may still appear after a plateau. The core of this route is engineering optimization under a commercial flywheel. At the same time, the industry must explore new computing paradigms and algorithms beyond autoregressive modes, and even beyond Transformers. The future path to AGI will likely involve developing cognitive large models and embodied large models that can “perceive,” “think physically,” and “act.” These models will need to align directly with interpretable components and form what is described as autonomous consciousness through feedback from practice. High-energy-efficiency edge hardware and high-efficiency algorithms will therefore become critical engineering foundations for exploring embodied large models.

Engineering Improvements and Optimization of the Autoregressive LLM Model

In response to the issues described above, academia and industry have carried out a series of algorithm, system, and hardware improvements and optimizations around autoregressive LLMs.

Algorithms: Targeted Breakthroughs in Efficiency Bottlenecks

Attention Mechanism Optimization Becomes a Key Lever

Use cases such as document understanding, code analysis, and retrieval-augmented generation, or RAG, require models to support long-context input, while reasoning models represented by DeepSeek-R1 require support for long chain-of-thought output. Longer sequences cause the computational complexity of self-attention to rise at O(N2). As a result, improvements to attention mechanisms such as grouped-query attention, or GQA, and multi-head latent attention, or MLA, as well as operator optimizations represented by FlashAttention, have been widely adopted. Linear-attention mechanisms such as Linear Attention, RWKV, and Mamba are also emerging. In addition, rotary position embedding, or RoPE, interpolation schemes have been further optimized, and some attention mechanisms, such as native sparse attention, or NSA, and mixture-of-blocks attention, or MoBA, as well as long-context inference frameworks for multi-card scenarios, including Ring Attention and Tree Attention, are being used to reduce computation.

Low-Precision Quantization Moves Into Deployment

A typical operation in the decode-only architecture is general matrix-vector multiplication, or GEMV. This operation involves frequent data movement and low computational efficiency, consuming both computing power and bandwidth.

Using hardware-native low-precision data types such as FP8, FP4, and MXFP for model quantization can effectively reduce memory bandwidth requirements while equivalently increasing chip computing power utilization. Existing research shows that 4-bit quantization has a relatively optimal scaling rate and has already been used in inference scenarios. However, the errors introduced by quantization inevitably reduce model capability, while quantization and dequantization operations in nonlinear layers add extra overhead. Quantization can therefore only ease the compute and bandwidth bottlenecks.

Exploring Parameter Reuse Through Recurrence

Recurrent Transformer architectures, such as Universal Transformer and mixture-of-experts Universal Transformer, or MoEUT, implement deep recurrence through cross-layer parameter sharing. By introducing the recursive expressive capability of recurrent neural networks and using parameter sharing to allow weights to support multiple rounds of computation, these architectures can effectively improve arithmetic intensity and boost system performance when memory bandwidth is constrained. However, current experiments on this architecture are still small in scale, and its expressive power and stability after scaling remain unclear.

Cluster System Improvements: Coordinated Gains in Operating Efficiency

Traditional CNNs such as ResNet and YOLO have network parameters and computation only at the MB and GOPS, or billions of operations per second, level. On today’s hundred-TOPS-class computing units, at roughly 2 TOPS/W, they can usually run on a single card or a single machine. Modern LLMs, however, have enormous parameter counts and computation requirements, inevitably requiring multi-card and multi-machine cluster systems. These systems accelerate training and inference through parallel computing paradigms such as tensor parallelism, data parallelism, pipeline parallelism, and expert parallelism.

The MoE-based distributed computing paradigm can reduce training intensity for models with extremely large parameter counts. By activating only the top-K experts during each forward pass, the amount of computation is reduced versus a dense model by a factor equal to top-K divided by the total number of experts, lowering computing power demand. DeepSeek V3, for example, reduced the computation of its feed-forward neural network, or FFN, to 1/32 of the original level.

P/D-separated deployment can take advantage of the differences between prefill and decode in compute and bandwidth requirements. The prefill stage is compute-intensive and optimizes for time to first token, or TTFT; the decode stage is memory-access-intensive and optimizes for time per output token, or TPOT. Deploying the two separately prevents mutual interference and improves hardware utilization.

Cloud AI systems can work together with edge devices to solve large-model deployment problems when edge-side computing power resources are limited. Smaller-parameter models deployed on devices can support local real-time inference. Complex task decomposition and deep-reasoning tasks can be handled by larger-parameter models in the cloud. The analysis results are then fed back to the device, creating a “fast and slow thinking” system through edge-cloud AI collaboration.

Hardware Engineering: Technical Innovation to Break Constraints

LLM clusters borrow from traditional high-performance computing, or HPC, cluster engineering practices to optimize the current computing paradigm, with the following engineering innovations:

Microarchitecture DSA-ization: general-purpose graphics processing units, or GPGPUs, have introduced more specialized architectural designs commonly used in DSA fields. NVIDIA GPU Tensor Cores, for example, introduced asynchronous data movement and mixed-precision training, drawing on experience from dataflow computing paradigms.

Interconnect optimization: clusters are divided into scale-up and scale-out domains, with interconnect technologies matched to the computing paradigm. Scale-up serves as a high-bandwidth domain, using bus-like technologies such as NVLink to provide supernode connections with ultra-low latency of 200 ns, high parallelism across thousands of nodes, and native memory semantics, helping escape the scaling constraints of Amdahl’s Law. Scale-out, meanwhile, borrows remote direct memory access, or RDMA, technologies to support general expansion, reuses HPC collective communication primitives such as NCCL, and builds a parallel computing software model.

Optical-electrical hybrid clusters: as domestic computing power capabilities remain constrained, a software-hardware architecture based on silicon photonics and wafer-scale expansion, described as “small electrical computing, large optical interconnect,” is expected to become a key technology for building clusters with 10,000 cards or more, and even more than 100,000 cards.

New computing paradigms: in the process of solving bandwidth problems, new computing paradigms such as compute-in-memory, which break through the “memory wall” and “power wall” limits of the von Neumann architecture, have also attracted significant attention.

Compute-network-storage simulation platforms: optimizing the deployment of ultra-large-scale clusters with more than 10,000 cards requires simulation platforms to optimize computing power deployment and workflows across compute, network, and storage systems. Building simulation architectures with high accuracy and high timeliness is an urgent research challenge.

At present, two forward-looking hardware engineering technologies are especially important:

Reconstructing advanced computing architectures around optical I/O technology is a key technology for optimizing the LLM computing paradigm. It can support architectural innovations such as scale-up supernode connections with ultra-low latency at the hundred-nanosecond level, memory pooling, and remote memory extension.

Building a new memory system beyond HBM requires large-capacity, high-bandwidth memory based on 3D dynamic random-access memory, or DRAM, and capacitorless DRAM, combined with LLM memory-access traits such as “many reads and few writes” and “more sequential than random” access. Architecture designs may include heterogeneous media such as high-bandwidth flash, hierarchical caching, compressed computing, and compute-in-memory.

Evolution and Outlook for Next-Generation AI Large Model Computing Paradigms

The route of achieving AGI by continuously scaling extremely large parameter models through scaling laws is constrained by computing power, bandwidth, energy consumption, and training data. Achieving AGI may require fundamental changes, such as combining algorithmic models based on first principles of physics with hardware engineering of computational substrates.

Development Trend: Toward the Co-Evolution of Physical Principles and Hardware

Industry is exploring next-generation AI large model paradigms that do not center on next-token prediction. Models based on first principles such as energy and dynamics, because they can effectively represent various distributions and evolve naturally in physical systems, are expected to become core architectures for next-generation AI large models. The Boltzmann machine proposed by Hinton, for example, was inspired by the Ising model and Boltzmann distribution in statistical physics. It introduced stochastic, recurrent neural networks capable of learning latent data distributions and solving complex combinatorial optimization problems. Later restricted Boltzmann machines and deep belief networks accelerated the development of AI technology and helped drive the broad use of generative models in image generation, natural language processing, and reinforcement learning.

However, when these models based on energy and dynamics run on existing von Neumann computers, they still face major challenges in energy consumption and computational efficiency. This is because deterministic computing architectures based on Boolean logic face two key problems when handling generative models based on statistics and probability. First, the physical properties of complementary metal-oxide-semiconductor, or CMOS, devices limit their hardware implementation capability for simulating stochastic processes. Second, the efficiency of existing computing paradigms drops sharply when facing nondeterministic demands such as semantic ambiguity in natural language processing and real-time decision-making in dynamic environments. This bottleneck has created demand for new computing paradigms oriented toward statistics and probability. Through algorithm-hardware co-design, these paradigms could break the traditional process that separates memory and compute units. That could sharply improve energy efficiency and computing performance, opening a new path to overcoming today’s AI computing power bottlenecks.

Two Directions for Model Development

First, the model may still be a Transformer, but no longer an autoregressive next-token prediction model. From the goals of higher abstract spaces, stronger expressive power, and long-term learning capability, researchers are designing next-generation model structures. Representative work includes:

Diffusion LLM architectures, represented by models such as LLaDA and Mercury, use diffusion methods to turn the serialized generation process of autoregressive models into a parallel generation process from coarse to fine granularity. Under the same computing resources and model scale, this architecture can increase inference throughput by more than 10 times, cut computing energy consumption to one-tenth of the original architecture, and improve metrics such as reverse reasoning capability and context attention length.

Joint embedding predictive architectures, represented by the joint embedding predictive architecture, or JEPA, and large concept models, or LCMs, encode language, images, video, and other data into high-level latent spaces, learn abstract representations at the world-model level, and replace probability-based prediction with energy-based models in representation space, thereby improving model expression and planning capability.

Second, based on the first principles of physics, researchers can design model architectures and dataflows from the characteristics of computational substrates and according to the dynamic properties and energy-change trends of physical processes. Representative work includes:

Liquid neural models, or LFMs, represented by liquid structured state-space models, or LSSMs, are based on liquid time-constant network, or LTCN, models. This is a new continuous-time recurrent neural network inspired by small biological neural-dynamics models. It can be trained through backpropagation and has shown strong bounded and stable dynamic properties, excellent expressive capability, and high memory efficiency in time-series forecasting tasks.

Energy-based models, or EBMs, represented by Hopfield networks, restricted Boltzmann machines, or RBMs, and deep belief networks, or DBNs, provide a unified framework for probability-density estimation and representation learning. Their theoretical foundations can all be traced back to spin-glass models in statistical physics. EBMs represent the probability distribution they aim to learn by defining an energy function, so they can also serve as generative models that learn data distributions and generate new samples similar to training data. Compared with models that explicitly define probability distributions, EBMs are more flexible and can model more complex dependencies. In recent years, the theory of energy-based models has continued to develop, but it still faces many challenges. The computation of the partition function and sampling efficiency remain the main bottlenecks limiting applications. In addition, the design of energy functions lacks systematic guidance and often depends on experience and heuristics. Theoretical properties such as expressive power and generalization also need deeper study.

Three New Computing Paradigms

In future AI computing, energy consumption will become a more fundamental constraint than computing power. The root cause of inefficiency in current AI computing is that neural networks are implemented by using traditional von Neumann computing architectures to “simulate” neural-network computation through binary operations. In essence, this approach uses high-precision logic computing to handle AI tasks that only require low precision. Large amounts of energy are spent on data movement and error correction, leading to inefficient resource use. To further improve computing performance while reducing energy consumption, researchers have explored several new computing paradigms. The main idea is to adopt non-von Neumann computing structures and compute-in-memory. Current important and active research includes the following routes:

Computing Architectures Inspired by Physical Principles

Physical neural networks, or PNNs, are a technical path for building AI using the first principles of physics. Existing routes include optical computing, quantum computing, and electromagnetic computing.

Optical computing uses photons as information carriers for computation and transmission. It offers ultra-high speed, ultra-high bandwidth, low latency, and high parallelism. By using physical properties such as optical interference, diffraction, and intensity or phase modulation, optical computing can execute specific computing tasks directly in the analog domain, showing disruptive potential especially in AI computing. A Tsinghua University research team, for example, launched the Taichi series of optical computing systems, using spatial symmetry and reciprocity to build an optical neural network, or ONN, that integrates training and inference. But optical computing still faces severe challenges in integration density, device performance, system complexity, precision, and software ecosystems. Its maturity remains low.

Quantum computing is a new computing mode that follows the laws of quantum mechanics to control quantum information units for computation. Existing quantum algorithms and quantum neural network frameworks must operate under constraints of limited qubits and relatively high computational error rates. Examples include using quantum-weighted tensor hybrid networks, or QWTHN, for large-model fine-tuning; converting FFN training into a quadratic unconstrained binary optimization, or QUBO, problem and solving it with a quantum Ising machine; and using qubits to build reservoirs and implement reservoir computing. However, because its technical routes have not converged, qubit counts remain limited, and operating environments are demanding, quantum computing is still far from broad application.

Electromagnetic computing directly uses the properties of electromagnetic waves, including microwaves, millimeter waves, and terahertz waves, for information processing, rather than relying on traditional electronic switching states. Its core advantages include ultra-high-speed operation, high parallelism, and low transmission loss. Main implementation forms include microwave and millimeter-wave analog computing, programmable electromagnetic processing, and electromagnetic in-memory computing. By directly mapping mathematical operations through physical laws, electromagnetic computing shows application potential in specific domains such as linear transformations and real-time processing, though it remains at the laboratory stage.

Analog Computing Architectures Based on Material Properties

Researchers are exploring a range of neuromorphic devices that use intrinsic physical phenomena in materials to simulate the complex behavior of biological systems. Through specific connection methods, they build systems in which units are mutually coupled and can use the system’s own evolutionary properties to replace traditional computing processes. By using the intrinsic characteristics of materials and promoting joint design across algorithms, software, and hardware, this route could fundamentally change the traditional separation between AI algorithm software and hardware, enabling coordinated software-hardware optimization. Existing technical routes include probabilistic computing, attractor networks, and thermodynamic computing.

Probabilistic computing systems rely on probabilistic bit units, or p-bits, with true stochastic characteristics. They are an intermediate computing paradigm between quantum computing and digital logic, and can better use the latent properties of nature and probability than traditional computers. They have major advantages in application scenarios such as combinatorial optimization, factorization, key generation, and Markov chain Monte Carlo, or MCMC, sampling. Probabilistic computing systems can also train stochastic neural networks and deep generative models, such as deep Boltzmann machines.

An attractor is a set of values toward which a dynamical system tends under different initial conditions, and it can implement memory functions in dynamical systems. In 2024, Sun and others used the bidirectional resistive-switching characteristics of resistive random-access memory, or RRAM, devices to build hysteretic neurons, and then constructed a recurrent neural network that emerges from bipolar memristor circuits. Compared with traditional Hopfield networks, it offers advantages such as higher hardware efficiency and larger memory capacity.

Thermodynamic computing is based on thermodynamic principles. It uses nature’s inherent computing capability to develop new design principles for information-processing networks, with applications in future computing systems. Normal Computing builds state spaces with precise representation, expressive nonlinear functions, and scalable hardware units, enabling efficient sampling from complex distributions and addressing computational bottlenecks in physical simulation and machine-learning tasks.

Bio-Inspired Computing Architectures

Bio-inspired computing reconstructs computing architectures by simulating information-processing mechanisms in natural systems, breaking through the bottlenecks of the traditional von Neumann architecture. Current mainstream research directions include brain-inspired computing and DNA computing.

Brain-inspired computing broadly refers to a class of new information-processing architectures inspired by the brain. Built on large-scale parallel computing platforms, these architectures are expected to break the von Neumann bottleneck of separated storage and computation and offer high-energy-efficiency solutions for general intelligence problems.

DNA computing is a new computing paradigm that uses the biochemical properties of molecules for information storage and processing, offering advantages such as high storage density and low power consumption. In the future, DNA computing will empower data processing in the AI era through hybrid silicon-based and biological computing.

Bio-inspired computing architectures are moving from specialized accelerators toward general computing paradigms. In the short term, brain-inspired computing chips are likely to break out first in edge intelligence. Over the medium and long term, a converged architecture of “silicon-based systems plus biological swarm collaboration” will emerge, ultimately enabling intelligent computing systems with biological-level energy efficiency.

ZTE’s Exploration and Practice for Next-Generation AI Large Model Computing Paradigms

ZTE has pursued innovation at the microarchitecture level, such as using 8T SRAM digital compute-in-memory technology to build high-energy-efficiency AI accelerators. It is also exploring an XPU-PIM heterogeneous architecture that uses compression and quantization to accelerate edge-side large models, delivering order-of-magnitude improvements in energy efficiency and throughput compared with conventional GPUs.

In new AI algorithms and hardware implementation, ZTE has explored new technical routes starting from the first principles of physics. For example, drawing on the efficient parameter-sharing characteristics of recurrent Transformer architectures, ZTE studied their ability to replace multi-layer Transformer architectures. Using a single Transformer layer from GPT-2 small as a model “base block,” it can reduce the parameter count by more than 50% while maintaining the model’s expressive capability. As the base-block structure improves, both the number of base-block layers and the number of recurrence steps can be further reduced.

At the same time, the sparse Boltzmann machine, or DBM, architecture is particularly suited to using nonvolatile memory for low-power edge tasks because of its sparsity and its inference objective based on energy minimization. At a scale of thousands of neurons, training a single batch on a GPU takes more than 10 hours. A fast FPGA-based DBM computing unit, using a probabilistic computing paradigm and instantiating thousands of neurons and sparse connections between them, shortens single-batch training time to 5 minutes, achieving an acceleration of more than two orders of magnitude. In the future, using nonvolatile memory devices such as RRAM and MRAM could further reduce computational overhead and improve inference speed, meeting broader application needs for DBMs in edge inference scenarios.

In addition, ZTE has carried out a series of forward-looking studies in supporting engineering technologies such as optical interconnects and new memory, as well as architecture technologies including data-pooling systems with separated compute and storage, memory-semantic interconnect systems, and large-scale simulation platforms. One example is customized memory design based on large-model memory-access characteristics to implement high-bandwidth UCIe memory and improve large-model inference performance;

Conclusion

From GPT-3 to today’s trillion-parameter models, the scaling of large models has driven rapid progress in AI, but it has also exposed deeper problems in architectural efficiency, computing power consumption, and adaptation to the physical world. ZTE’s paper presents the industry’s current position clearly: on one hand, autoregressive Transformer architectures continue to extract value through algorithm optimization, cluster upgrades, and hardware innovation; on the other, exploration beyond traditional paradigms is now fully underway. The integration of first-principles physics models with new computational substrates is becoming a critical breakthrough point for AGI.

As the AI industry shifts from a “scale race” to an “efficiency revolution,” software-hardware co-design and cross-disciplinary technology integration will become core competitive strengths. ZTE’s exploration not only helps the company secure technical ground, but also contributes to the industry’s search for a more efficient and sustainable direction. As these frontier technologies mature, AI may finally reduce its dependence on brute-force scaling and move steadily toward general intelligence that can understand the physical world and possess autonomous consciousness.