Dr. Shengcai Liao

Advancing Multimodal Personalized Digital Humans for Natural Human-AI Interaction

With the rapid advancement of generative AI and large language models (LLMs), digital humans are emerging as the next frontier in human-computer interaction. Unlike conventional chatbots, future digital humans will integrate multimodal perception (vision, speech, and language), natural embodiment through digital avatars, and personalized behavioral modeling to enable lifelike, engaging, and seamless communication. Our CVLab research aims to push the boundaries of digital human technologies by addressing three key scientific challenges: high-fidelity identity representation for realistic and customized avatars, personalized multimodal interaction generation capturing individual speech intonation, facial expressions, and gestures, and dynamic memory representation to support context-aware, individualized interactions over time.

To achieve this, we will develop novel multimodal generative models that unify text, speech, and vision, enabling realistic and emotionally expressive digital humans capable of real-time interaction. By leveraging advances in diffusion models, neural rendering, and memory-augmented LLMs, our research will create digital avatars with human-like personalities and knowledge bases, supporting long-term personalized engagement. These innovations will unlock transformative applications across companion robots, virtual assistants, industrial training, and digital entertainment, while contributing to the UAE’s Industry 4.0 strategy and digital economy growth.

Ultimately, our goal is to establish a unified framework for multimodal personalized digital humans, bridging AI perception, generation, and memory technologies. This will empower digital humans to evolve beyond static avatars toward autonomous, emotionally intelligent, and context-aware agents, enabling unprecedented levels of naturalness, efficiency, and user satisfaction in human-AI interaction.