Schematic diagram of the relationship between intrapersonal and interpersonal synchronization, whether unimodal or multimodal. A) and B) represent intrapersonal synchronization between modalities of a single speaker. A) The solid blue arrows highlight the unimodal relationship between gestures generated by a single individual (i.e., head-to-head, head-to-wrist). B) The dashed blue arrows highlight the multimodal relationship between voice and gesture produced by a single individual (i.e., head-to-voice; wrist-to-voice). C) and D) represent interpersonal synchronization between the modalities of speaker A and speaker B. C) The solid red arrows highlight the unimodal relationships between the movements of speaker A and speaker B, and between their voices (i.e., head-to-head, head-to-wrist, voice-to-voice). D) The dotted red arrows highlight the multimodal relationships between speaker A’s voice and speaker B’s gesture and vice versa (i.e., head versus voice; wrist versus voice). Credit: Fauviaux et al., 2024, PLOS ONECC-BY 4.0 (creativecommons.org/licenses/by/4.0/)
The dynamics of turn-taking in social interactions are important for the synchronization of speech and gestures, allowing conversations to proceed effectively, according to a study published September 25, 2024 in the open-access journal PLOS ONE by Tifenn Fauviaux of the University of Montpellier, France, and colleagues.
Conversations involve ongoing exchanges of verbal and nonverbal information. Previous research has demonstrated that gestures and speech synchronize at the individual level. But few studies have examined how this phenomenon may play out between individuals.
To fill this knowledge gap, Fauviaux and his colleagues used an online dataset consisting of 14 sessions of two people engaged in unstructured face-to-face conversations in which they were free to talk about specific topics. Each of these sessions included between one and four discussions, and the conversations lasted from seven to 15 minutes.
The researchers analyzed audio and motion data and measured the synchronization of speech and gestures at different time scales. They focused specifically on speech properties via the speech amplitude envelope and motion properties via head and wrist gestures.
The results corroborate previous research on the coordination of speech and gestures at the individual level, revealing synchronization at all time scales of conversation. In other words, there was greater-than-chance synchronization between a given participant’s wrist and head movements, and similar synchronization between these movements and vocal properties.
Delving deeper into the literature, the researchers also found that gestures and speech synchronize between individuals. In other words, there was coordination between the voices and bodies of the two speakers. Taken together, the findings suggest that this type of synchronization of verbal and nonverbal information likely depends on the dynamics of turn-taking in conversations.
According to the authors, this study enriches our understanding of behavioral dynamics during social interactions, both at the intrapersonal and interpersonal levels, and strengthens our knowledge of the importance of synchronization between speech and gestures. Future research based on this study could shed light on prosocial behaviors and psychiatric disorders characterized by social deficits.
The authors add: “How do my speech and behaviors influence or respond to the speech and behaviors of the person with whom I am conversing? This study addresses this question by examining the multimodal dynamics between speech and movement, both at the individual and dyadic levels. Our results confirm the intrapersonal coordination between speech and gestures at all temporal scales.
“It also suggests that multimodal and interpersonal synchronization may be influenced by the speech channel, particularly the dynamics of turn-taking.”
More information:
From unimodal to multimodal dynamics of verbal and nonverbal signals during unstructured conversation, PLoS ONE (2024). DOI: 10.1371/journal.pone.0309831
Provided by the Public Library of Science
Quote:How Synchronization Promotes Social Interaction: Taking Turns During Conversations Can Help Coordinate Signals (2024, September 25) retrieved September 25, 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without written permission. The content is provided for informational purposes only.
