“Large Language Models for Virtual Human Gesture Selection” by Parisa Ghanad Torshizi, Laura Hensel, Ari Shapiro, and Stacy Marsella. In 24rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025, (Detroit, MI, USA), 2025.
Co-speech gestures convey a wide variety of meanings and play an important role in face-to-face human interactions. These gestures have been shown to significantly influence the addressee’s engagement, recall, comprehension, and attitudes toward the speaker. Similarly, they have been shown to impact human and embodied virtual agent interaction. The process of selecting and animating meaningful gestures has thus become a key focus in designing embodied virtual agents. However, the automation of this gesture selection process poses a significant challenge. Prior gesture generation techniques have attempted to address this challenge in varied ways from fully automated, data-driven techniques -- which often struggle to produce contextually meaningful gestures -- to more manual approaches of crafting gesture expertise, which are time-consuming and lack generalizability. In this paper, we leverage the semantic capabilities of Large Language Models to realize a gesture selection approach that suggests meaningful, appropriate co-speech gestures. We first illustrate the information on gestures encoded into GPT4. Then we perform a study to specifically evaluate alternative prompting approaches for their ability to select meaningful, contextually relevant gestures and to align them appropriately to the co-speech utterance. Finally, we detail and demonstrate how this approach has been implemented within a virtual agent system, automating the selection and subsequent animation of the selected gestures for human-agent interactions.
BibTeX entry:
@inproceedings{TorshiziAAMAS2025,
author = {Ghanad Torshizi, Parisa and Laura Hensel and Ari Shapiro and
Stacy Marsella},
title = {Large Language Models for Virtual Human Gesture Selection},
booktitle = {24rd International Conference on Autonomous Agents and
Multiagent Systems, AAMAS 2025},
publisher = {AAMAS},
address = {Detroit, MI, USA},
year = {2025},
url =
{http://www.stacymarsella.org/publications/pdf/Parisa_AAMAS2025_Watermarked.pdf}
}
(This webpage was created with bibtex2web.)
Back to Stacy Marsella.