Mark Sagar on Humanlike AI, Multimodal Learning and Transparent Avatars

For Mark Sagar, making AI feel more natural means giving people clearer ways to interact with it—and clearer signals about what it is doing. The chief scientific officer and co-founder of Soul Machines says language models can contribute to machine cognition, but human interaction also depends on vision, emotion, social cues and learning through experience. Digital faces, in his view, have a practical role when they help convey attention and intent rather than simply making software look human.

Sagar discussed the company’s work and this broader approach in an interview with Techopedia. He is a two-time Academy Award winner whose work includes digital characters designed to respond to verbal and nonverbal cues. His claims describe a research direction, not evidence that current systems reproduce human thought. The distinction matters: a model that can generate fluent language still lacks many of the perceptual and social abilities that shape how people learn and coordinate.

Building an interface people can read

Soul Machines creates digital people that users can interact with in a person-like way. Sagar describes the company’s long-term aim as building an intuitive, cooperative interface for AI. That idea extends beyond adding a spoken voice to software. People communicate through speech, but they also use gaze, facial expression and other signals to show what they are attending to and how they feel.

Voice assistants offer a useful comparison. They let people speak naturally, yet the system itself may give little indication of whether it is listening, following a request or having trouble. Sagar likens that arrangement to the disembodied computer HAL in 2001: A Space Odyssey: a voice responds, but users cannot see where its attention is directed. The result can feel one-sided, because the person supplies signals while the machine reveals few of its own.

A visible interface could change that exchange. If a digital person looks toward an object, reacts to a question or signals uncertainty, a user gets additional context for interpreting its response. Sagar sees such cues as a route to more fluid cooperation between people and machines, with interaction that adapts as a conversation unfolds. He compares the ideal to musicians trading riffs: each participant responds to what the others contribute, helping the group carry out a task.

That aspiration doesn’t establish that a humanlike interface will make an AI system more capable. It does suggest a design question for teams building conversational products: what does the interface tell the user about the system’s attention, assumptions and uncertainty? A face that merely decorates a text generator may add little. A face whose signals correspond to the system’s actual state could make an interaction easier to follow.

Language models are one part of cognition

Sagar regards generative AI and large language models as significant components of cognition, not complete accounts of it. LLMs learn relationships among words and symbols, which lets them represent and connect a large range of human knowledge. Those associations can support powerful language tasks, but they don’t by themselves provide visual perception, emotional processing or the full range of abilities involved in interacting with the physical world.

His account of human learning starts with experience. Babies encounter objects and events, gradually learn qualities such as hot, cold or heavy, and receive feedback from caregivers and other people. Language labels are part of that process: a child learns to associate words such as “red” and “green” with what they see. The learning is multimodal, linking words with perception, action and social response.

That contrast helps explain why a fluent response isn’t the same as a humanlike understanding of a situation. A language model operates through learned symbolic relationships; a person’s knowledge also develops through embodied encounters and ongoing feedback. Sagar’s argument is that future systems will need to combine these different kinds of processing if they are to support richer interaction. The interview doesn’t specify a technical recipe or a performance measure for reaching that goal.

The broader implication is architectural. Language models may remain central while working alongside other components that handle perception, emotional cues or interaction. Sagar expects users to encounter these capabilities as a unified interface, even if many systems work behind the scenes. The single conversational surface could conceal a collection of distinct models and tools.

Emotion can make interaction more informative

For Sagar, adding human features to technology is a way to make interaction more intuitive. Voice and language already draw on familiar habits; faces and emotional signals could add information people routinely use when talking to one another. But those cues matter only if they help users understand what is happening.

Consider a customer service exchange in which an agent appears confused by a question. A human interlocutor might notice the expression and offer more time or a clearer explanation. In Sagar’s account, facial signals can reveal what someone is attending to and how they are responding—information that shapes decisions and helps people adjust their behavior. An avatar that communicates a comparable state could give the user a useful prompt to clarify or change course.

This approach may also help address the uncanny valley, the discomfort people can feel when a digital figure looks almost human but behaves in ways that don’t fit. Sagar’s answer is that emotional responsiveness can help, because emotion is part of human communication and decision-making. The interview doesn’t claim that expressive animation alone resolves the problem. The important design issue is whether expressions are understandable and tied to meaningful system behavior.

There is a reciprocal element, too. A computer may try to infer what the user wants, but the user also needs some view of what the computer is considering. Sagar describes this as a two-way process: people form a theory of mind about others by asking what they intend or believe, and a system should communicate enough of its own state for users to form a grounded picture of its behavior.

He points to Baxter, a production-line robot fitted with eyes on a small screen. The robot looked toward the direction it was about to move, giving nearby workers a cue about the arm’s intended path. That visible signal could help people anticipate the robot’s action. It’s a compact example of transparency through behavior: users don’t need a complete account of a system’s internal computation to benefit from a clear indication of what it will do next.

Realistic expression without deception

Transparency also sets a boundary for digital people. Sagar says avatars can use realistic human expression without being designed to trick viewers into believing they are watching a real person. He raises deepfakes as a problem of visual ambiguity: a synthetic video can look like ordinary footage, leaving viewers uncertain about what they are seeing.

His proposed standard has two parts. First, an avatar should not aim to fool its audience. Second, its behavior should communicate something meaningful about the system’s assumptions or direction. A display of emotion that functions as a gimmick could make users read more into the interaction than the system can support. A signal grounded in what the system has detected or is about to do could instead help people understand the exchange.

That standard puts pressure on the link between an AI’s state and its presentation. If a digital person signals confidence when its underlying system is uncertain, the interface could mislead users even without a realistic video. If it signals confusion or pauses in a way that reflects a genuine limitation, users may know when to clarify a request. The interview sets out this principle but doesn’t explain how Soul Machines measures whether an expression accurately reflects a model’s internal state.

Baby X and the direction of research

Sagar says much of the company’s research focuses on multimodal interaction: handling the different signals and timing patterns that arise when people communicate. Conversation doesn’t always proceed in neat turns. People pause, overlap, gesture and respond to one another in different ways. Making a digital person feel fluid, he says, means dealing with those details rather than treating interaction as text passed back and forth.

The company’s most advanced work, according to Sagar, is Baby X, a digital infant designed to be taught and engaged through emotional and social interaction. He describes it as a way to investigate the basics of learning: how a system might be taught through experience and interaction before building on those abilities in more mature forms of learning. The interview gives no benchmark results or implementation details for Baby X, so the proposal is best understood as a research direction rather than a demonstrated path to humanlike learning.

Asked whether ChatGPT-like systems will merge with digital people, Sagar expects convergence in the interface. Users could interact with one autonomous, person-like system while separate components operate behind the scenes. He evokes fictional characters such as C-3PO and Data as examples of the familiar idea: a distinct embodied system that can communicate socially and act on its own. Those characters are analogies for an interface, not evidence that today’s AI has their abilities.

The practical takeaway is that humanlike AI depends on more than natural-sounding language or a convincing face. Sagar’s vision joins language with perception, social context and visible signals of intent. For developers, the test is whether those added cues help users understand and coordinate with a system—and whether the system’s presentation stays faithful to what it can actually do.

The interview transcript was edited for brevity and clarity.