Speech Breathing in Virtual Humans: An Interactive Model and Empirical Study


Human speech production requires the dynamic regulation of air through the vocal system. While virtual character systems commonly are capable of speech output, they rarely take breathing during speaking - speech breathing - into account. We believe that integrating dynamic speech breathing systems in virtual characters can significantly contribute to augmenting their realism. Here, we present a novel control architecture aimed at generating speech breathing in virtual characters. This architecture is informed by behavioral, linguistic and anatomical knowledge of human speech breathing. Based on textual input and controlled by a set of low-and high-level parameters, the system produces dynamic signals in real-time that control the virtual character's anatomy (thorax, abdomen, head, nostrils, and mouth) and sound production (speech and breathing). In addition, we perform a study to determine the effects of including breathing-motivated speech movements, such as head tilts and chest expansions during dialogue on a virtual character, as well as breathing sounds. This study includes speech that is generated both from a text-to-speech engine as well as from recorded voice.

Publication DOI: https://doi.org/10.1109/VHCIE.2019.8714737
Divisions: College of Engineering & Physical Sciences > School of Informatics and Digital Engineering > Computer Science
College of Engineering & Physical Sciences
Additional Information: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Event Title: 2019 IEEE Virtual Humans and Crowds for Immersive Environments (VHCIE)
Event Type: Other
Event Location: Osaka, Japan
Event Dates: 2019-03-24 - 2019-03-24
Uncontrolled Keywords: 500 [Human-centered computing],Computing methodologies - Virtual reality,Human-centered computing - Empirical studies in HCI 500 [Computing methodologies],Procedural animation - [500],Virtual reality - [500],Human-Computer Interaction,Media Technology,Artificial Intelligence
ISBN: 9781728132198, 9781728132198
Full Text Link:
Related URLs: https://ieeexpl ... cument/8714737/ (Publisher URL)
http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Conference contribution
Published Date: 2019-05-14
Accepted Date: 2019-01-01
Authors: Bernardet, Ulysses (ORCID Profile 0000-0003-4659-3035)
Kanq, Sin-hwa
Feng, Andrew
Dipaola, Steve
Shapiro, Ari



Version: Accepted Version

| Preview

Export / Share Citation


Additional statistics for this record