AgiBot recently unveiled WITA-Omni, which it describes as the industry's first end-to-end embodied omni-modal interaction model. The most substantive term in that description is “omni-modal.” Robot interaction has traditionally revolved around cameras and microphones: understand an instruction, generate a response, then hand it off to an execution system to perform an action. These three tasks happen in separate stages. WITA-Omni adds touch and proprioceptive state to its inputs, and robot movements and facial expressions to its outputs.

These two additional channels are not there simply to inflate the model's specifications. AgiBot says the reason lies in the differences between operating environments. Phones, smart speakers and in-car systems face relatively limited interference. Robots must interact in noisier environments, with a wider range of people, across greater distances and while constantly changing position. In such settings, people do not infer intent through sound alone. The pressure of a handshake and whether an outstretched hand is an invitation or a gesture of direction both carry information that purely audiovisual models cannot capture.

AgiBot says the change gives robots, “for the first time, an interaction brain born for the physical world.” The “first” is the company's own characterization, but the substance of the change is clear: the list of interaction inputs and outputs has been rewritten, expanding what the model processes from sound and images to physical contact and joint states.

Putting Listening, Thinking, Speaking and Moving Into a Single Pipeline

AgiBot demonstrated the system at its AI Day. Xiong Yan, the company's co-founder, co-president, president of central R&D and head of its interaction business, held a 10-minute real-time exchange with a WITA-Omni-equipped Expedition A3-Ultra. AgiBot described the demonstration as an “extreme stress test” and said the system remained stable throughout.

The demonstration was enabled by a streaming, full-duplex architecture. Conventional systems operate sequentially: they wait for a person to finish speaking before interpreting the sentence, finish interpreting it before generating a response, and complete that response before driving movements and facial expressions. Each stage must wait for the previous one to finish. WITA-Omni instead receives and generates data continuously, producing speech, facial expressions and movements in parallel. Even while listening, the robot can respond by nodding or maintaining eye contact. AgiBot says response latency can be as low as about 400 milliseconds and averages roughly 600 milliseconds, nearly three times faster than the industry average.

More noteworthy is how the model decides whether someone has finished speaking. People pause, but a pause does not necessarily mean they are done. A sequential architecture cannot make that distinction; it must be assessed in real time within the stream. The system needs to determine whether someone has paused temporarily or finished speaking before deciding whether to respond, interject or yield the floor. That is also what allows a conversation to be interrupted and resumed at any time.

Training takes place in three stages. First, omni-modal pretraining jointly models vision, hearing, language, facial expressions and movement to establish a foundation for cross-modal understanding. Next, embodied behavior alignment uses proprioceptive robot data to ensure that speech, expressions and movements match the robot's actual capabilities. Finally, interaction-strategy reinforcement teaches the model when to respond, take a turn, yield the floor or initiate a conversation.

Knowing Whether to Speak Is Harder Than Knowing What to Say

WITA-Omni's omni-modal rejection system eliminates the need for a wake word. It combines conversational orientation, video, audio, voiceprints, lip movements, posture, semantics and context to decide whether a remark warrants a response. Conversations among multiple people and ambient announcements are especially prone to false activations, which the rejection system is designed to suppress.

This may look like a simple user-experience improvement, but it is effectively a gatekeeper for whether service robots can operate in public spaces. A robot that constantly responds by mistake cannot function in a shopping mall, hospital or hotel. One that is too reluctant to answer is equally useless. Striking the right balance is among the hardest parts of tuning such products: go too far and the robot becomes mute; fall short and it never stops talking.

Audio capture is another part of the system. WITA-Omni uses an in-house audiovisual perception front end that draws on the robot's proprioceptive state to cancel its own echoes and noise, continuously tracks a moving conversation partner, and combines voiceprints, lip movements and visual positioning to isolate the target speaker while suppressing ambient noise. A robot's motors and fans operate right beside its microphones. Without this layer, everything the model subsequently tries to understand would be built on noise.

Graduated Emotions and Seamless Switching Across Languages and Dialects

Changes on the expression side are easier to see than those on the input side. While listening, the model responds with nuanced facial expressions and natural head movements. When speaking, it combines parameterized facial expressions, expressive speech and coordinated body movements to convey dozens of emotions, including happiness, surprise, excitement, sadness, satisfaction, sobbing, seriousness and confidence. Each emotion also has degrees of intensity: happiness, for example, can range from a slight smile to hearty laughter.

Language support is another major addition. The model can communicate freely in more than 100 languages and dialects without manual switching. Combined with the full-duplex architecture, it can provide streaming simultaneous interpretation across both languages and dialects, with two-way translation. For service robots intended for overseas markets, this may prove more practical than emotional expression.

Intelligent FaceUI is the third feature: the robot can draw. When explaining a structurally complex product or teaching a child about science, the model decides whether difficult concepts would be clearer if visualized in real time on the robot's facial display instead of being laboriously explained through speech alone.

Always-on autonomy takes interaction from responding only when summoned to initiating contact. The model continuously senses its surroundings, the user's state and any changes, then independently reasons, makes decisions and acts. It can greet a VIP and offer a handshake, respond happily to a thumbs-up, or immediately wave goodbye when it sees someone waving.

A Fast System in the Foreground, a Slower One in the Background

Conversation demands fast responses, while complex tasks require deeper thought. Those requirements inherently conflict. WITA-Omni addresses this by separating them: a foreground Interaction Engine handles streaming comprehension and immediate responses, while a Background Agent handles reasoning, planning, tool use and long-horizon tasks.

The two systems share the same context and timeline. Users can add information, make corrections or cancel a request at any point, and the background task adjusts in real time. In effect, the design decouples response latency from task depth. A person does not have to wait for the background system to finish thinking before speaking, and the background system does not have to abandon long-horizon tasks to deliver an instant reply.

The Model Runs Inside the Robot, Not in the Cloud

AgiBot says deploying a large-parameter model inside a mass-produced robot addresses three issues: privacy, offline interaction and real-time computing. With cloud processing, data must first leave the robot and then return after computation. Privacy and loss of connectivity are unavoidable concerns, while round-trip latency is difficult to reduce. Drawing on the Expedition A3-Ultra's computing power and extensive inference optimizations, WITA-Omni is, according to AgiBot, the industry's first large-parameter model deployed directly on a mass-produced robot.

For real-world deployment, this may matter more than any individual benchmark. Once robots enter factories, shopping malls and hotels, network conditions cannot be controlled. They must be able to operate offline before they can truly go to work. The tradeoff is that keeping the model local fixes the computing power budget inside the robot, which also explains why AgiBot highlights inference optimization as a distinct capability.

Benchmarks, Architecture and Robots Already at Work

AgiBot says WITA-Omni has taken first place on four benchmarks related to embodied interaction: DailyOmni for omni-modal understanding, ActivityNet-TimeLens for temporal localization in video, MMAU for audio understanding and EmbSpatialBench for embodied spatial understanding.

In practical deployments, AgiBot robots equipped with the model are already providing informational presentations and theme-park tours in some entertainment and visitor-guide settings, performing interaction tasks in environments open to the public.

Within AgiBot's “Three Intelligences in One” product and technology architecture, interaction intelligence is positioned as the component responsible for real-world deployment. Put simply, a robot entering an actual workplace needs more than a body capable of performing tasks. It also needs a system that can understand the physical world and communicate naturally with people.

The sequence of this strategy is worth considering. In recent years, most of the industry's attention has focused on whether robots can perform useful work. Interaction has been treated as an add-on, something to address after task capabilities mature. By developing interaction as a standalone end-to-end model and giving it a dedicated launch, AgiBot is effectively acknowledging that the two must advance in parallel. When a robot stands in front of a person, the first question is whether it can communicate; only then comes the question of whether it can work.