Sony is working on AI-powered versions of PlayStation characters that can respond to voice input from the player during gameplay, a new leaked video revealed.
The video, which was shared with The Verge by an anonymous tipster before it got taken down on YouTube, shows an AI-powered version of the Horizon series protagonist Aloy that can respond to queries using synthesized voice and facial movements both in a demo setting and during Horizon Forbidden West gameplay, as highlighted by SIE director of software engineering Sharwin Raghoebardajal who demonstrated the technology in the removed video. The technology, powered by OpenAI’s whisper for speech-to-text and both GPT-4 and Llama 3 for conversations, is augmented by Sony's internal Emotional Voice Synthesis system and by Mockingbird technology that handles audio-to-face animation.
While this AI-powered version of Aloy was developed to show the technology internally, there's a very good chance that we will eventually see it implemented even on consoles, as parts of the technology can run directly on PlayStation 5 with little overhead. This glimpse of what the technology is capable of achieving doesn't exactly provide hints at how it could be implemented to enhance gaming experiences, but there's little doubt that it will impact future games considerably, considering the direction the industry is taking and how the PlayStation 6, in particular, is said will go all-out on AI.
Sony isn't the only major console manufacturer investing heavily in AI. Last week, Microsoft unveiled its Muse AI model capable of generating gameplay. This should help developers, especially in the early stages of development, although the contrary is more likely, given the current state of the industry.
