Department of Information and Computing Sciences
Utrecht University - Netherlands
Talk Title:
Learning to Gesture: Deep Models for Non-Verbal Behavior in Digital Humans
Abstract:
With recent advancements in computer graphics, 3D digital humans have achieved an impressive level of visual realism. They are increasingly being integrated into diverse applications, including video games, customer service and finance chatbots, educational and healthcare simulations, remote communication, and social extended reality (XR). However, their ability to interact and move naturally within social contexts remains limited. As humans, we are highly attuned to non-verbal behaviors in emotional and social interactions. For digital humans to engage with us more naturally, they must be equipped with non-verbal communication skills such as facial expressions, gestures, and gaze. As they are deployed in more interactive environments, the demand to generate their behavior automatically and in real time is growing. Yet, capturing and synthesizing the nuanced, individual nature of non-verbal behaviors remains a significant challenge, hindered by limitations in data availability, the complexity of algorithms, and evaluation methodologies. In this talk, I will explore how AI and deep learning techniques—particularly those leveraging motion capture technology—can be used to model and generate the non-verbal behaviors of digital humans. I will present the state-of-the-art, highlight our recent research, and offer a critical examination of current evaluation practices in non-verbal behavior synthesis.
Bio:
I am an associate professor at the Human-Centered Computing Group, Department of Information and Computing Sciences at Utrecht University in the Netherlands and director of the Motion Capture and Virtual Reality Lab. I obtained my PhD degree in Computer Science from University of Geneva. I was a scientific collaborator at EPFL, Switzerland. Following that, I have been at the Institute for Media Innovation, Nanyang Technological University, Singapore as a research fellow.
My research is on believable virtual humans and social robots. I work on computational models of social and emotional behaviors and expressive character animation combining methods from computer graphics, artificial intelligence and human-computer interaction. My research is interdisciplinary with applications in games, virtual reality and robotics:
Expressive character animation (e.g. face, gaze, gestures..)
Social interaction and groups of virtual characters
Machine learning and deep learning models for character animation
Autonomous behavior for virtual humans and social robots
Modelling emotion and memory for virtual humans and social robots.
