Following in the footsteps of Neo NPC, the advanced NPC experiment revealed at GDC 2024, Ubisoft has unveiled its first playable generative AI research project which explores how AI can be used to deepen the player experience through real-time voice commands and enhanced gameplay. Dubbed Teammates, the playable experiment showcases the capabilities of an in-game AI, named Jaspar, as well as AI-enhanced NPCs who accompany and support the player dynamically, generating real-time responses and actions based on player voice commands.
The team behind the project aims to test the potential and limits of player interactions through natural speech, explore how generative AI can be used to enhance immersion, and test unique interactions that would not be possible without the technology. Director of Gameplay Gen AI Xavier Manzanares, Narrative Director Virginie Mosser, and Data & AI Director Rémi Labory gave us their insights on how the project has different goals to other AI projects, and how the team’s research resulted in Teammates.
Ubisoft Teammates: An Experimental AI Gaming Project
First and foremost, Teammates is an experimental research project, albeit a playable one. Built around the mechanics of a first-person shooter, players take on the role of a member of the resistance in a dystopian future, tasked with moving through an enemy base to locate five missing members of their team. In the short playable level, players must recover the final memories of those who were sent in previously to discover their fates, while fending off squads of enemies along their path. Besides the AI voice assistant – Jaspar – the player is accompanied by two NPC squad members – Pablo and Sofia – also featuring the new advanced capabilities.
Jaspar: The AI Voice Assistant
Jaspar is an AI assistant designed to extend beyond the basic functions of one. It can highlight enemies or objects, provide story and lore details, alter game settings on the fly, or even pause the game, all through simple and natural vocal instructions. But more than this, he is a dynamic and responsive character within the story. The underlying systems can process the information available within the Teammates experiment’s environment, and contextual cues, interpreting and adapting to the player’s vocal input and actions.
How Voice Commands Shape Gameplay
The same tech underpins the player’s two NPC teammates, Sofia and Pablo. Just like Jaspar, they can be commanded directly or chatted with, the only exception being that Sofia and Pablo physically inhabit the world.
An early scenario introduces players to the core mechanic: command through voice. In the introduction part of the experience, players encounter two enemies on patrol but have not yet been provided with a weapon. With only two armed teammates at their side, it is necessary to direct Sofia and Pablo to use environmental cover, then give specific instructions on when, how, and who to attack. The player’s voice has a strong impact, defining how Teammates reacts and the outcome of the battle.
Director of Gameplay GenAI Xavier Manzanares recalls how Jaspar developed based on the team’s early findings: “Our early experiments showed players were quickly connecting with the AI-driven NPCs and the voice assistant concept. Jaspar was helping players when they got lost or weren’t sure what to do, he could access menus and settings, tell players more about the world and the story. We really started to like Jaspar and saw how a system like this could be interesting for many different kinds of games.”
Why AI is Core to the design of the experience?
The goal the team hopes to achieve has players right at its heart. Narrative Director Virginie Mosser emphasizes the collaborative approach she has to working with the technology, both with players and with the AI NPCs themselves. “We hope players will feel like they’re shaping the story themselves, not just following it. When I talk to [my AI squad mate] Sofia, she reacts to what I tell her and it changes my experience. For me, this is a real breakthrough and allows players to experience the story in their own way.”
“This technology opens doors to new, personalized experiences,” states Labory, expanding on the ambition behind the project. “Player input shapes character reactions in real time, something traditional development can’t achieve. We’re also delivering a full pipeline, with the experience taking players from onboarding to debrief, which is a first.”
The team is aware of the criticisms around AI in games. The goal is not to replace creatives, but rather to find ways to enhance it by combining the strengths of the technology and the human creativity and ingenuity that are crucial to making games.
“At first, I had the same concerns as many others,” says Mosser. “But I discovered that it’s the exact opposite of removing the human from the process. I still write the story and character personalities, but instead of fixed lines, we create these kinds of fences that let NPCs improvise within the world but stay within the boundaries of the lore and motivations we have given them. They can improvise, but we still set the rules and direct the story and characters.”
What’s Next for Ubisoft’s AI Experiments?
The team is dedicated to research and experimentation, pushing the boundaries and exploring what’s possible. Their hope is to expand the ways players interact with games, to develop human-made works using the strengths of AI, and to discover how the tools could create deeper, more interesting experiences with storytelling and character interactions that wouldn’t be possible without the underpinning tech.
Teammates has already been shared with a few hundred players in a closed playtest, with their valuable feedback shaping the future of the team’s research. They aim to continue to test and develop the technology, building the tools and gathering vital reactions and suggestions from real players and creative teams within Ubisoft to refine and expand the systems they have built, all with a philosophy of human creativity, ingenuity, and curiosity at the heart of their work. They also plan to share more from the experience itself with an explainer video, so keep your eyes peeled to learn more about their work.
