Comparison of gesture-vocal synchrony and gestures of two children aged 3 to 26 months with and without autism spectrum disorderABSTRACT
The objective was to compare gesture-vocal synchrony in language functioning between mothers and babies and the gestural typology of babies from 3 to 26 months of age, one of them with autism spectrum disorder (case R), diagnosed at age 3, and the other without diagnosis (case B). It was select moments in which there was greater mother-baby interaction, from a bank of mother-baby interactions footage, from 3 to 26 months. It was analyzed using the Eudico Linguistic Annotator software ( ELAN), considering multimodal categories of sign language and speech of mothers and babies for descriptive statistical analysis. The results showed differences in the frequencies and types of gestures between the babies and also in the synchrony between them and their mothers. In the case of B., the gesture-vocal synchrony and variety of gestures is inserted in a context of conjunction between him and his mother in the first year of life, which gave rise to the second enunciative mechanism in the second year of life. In case R, the conjunction relationships were not established, as the difficulties in reading discomfort and annoyance gestures, very present in the baby since its first months of life, hindered the engagement and shared attention of the mother and her baby, disfavoring the inscription of gesture as language. R. showed language acquisition delay at 26 months. Although the gestural typology of B and R are similar, the frequencies, the quality and, above all, the gesture-vocal synchrony differ between dyads.
Keywords:
Autism; Speech; Multimodality; Language; Child Development
RESUMO
O objetivo foi comparar a sincronia gesto-vocal no funcionamento de linguagem entre mães e bebês e a tipologia gestual de bebês dos 3 aos 26 meses, sendo um deles com transtorno do espectro do autismo (caso R), diagnosticado aos três anos, e outro sem diagnóstico (caso B). A partir de um banco de filmagens das interações da mãe e do bebê realizadas dos 3 aos 26 meses, foram selecionados momentos em que houve maior interação entre mãe e bebê. Foram analisados por meio do software Eudico Linguistic Annotator (ELAN), considerando categorias multimodais de linguagem gestual e de fala das mães e dos bebês para análise estatística descritiva. Os resultados evidenciaram diferenças nas frequências e tipos gestuais entre os bebês e também na sincronia entre eles e suas mães. No caso de B. a sincronia gesto-vocal e variedade de gestos está em um contexto de conjunção entre ele e sua mãe no primeiro ano de vida, o que fez emergir o segundo mecanismo enunciativo no segundo ano de vida. No caso R, as relações de conjunção não se estabeleceram, pois as dificuldades na leitura de gestos de desconforto e incômodo pela mãe, muito presentes no bebê desde seus primeiros meses de vida, dificultaram o engajamento e a atenção compartilhada da mãe e seu bebê, desfavorecendo a inscrição do gesto como linguagem. R. apresentou atraso na aquisição da linguagem aos 26 meses. Apesar da tipologia gestual de B e R serem similares, as frequências, a qualidade e, sobretudo, a sincronia gesto-vocal diferem entre as díades.
Descritores:
Autismo; Fala; Multimodalidade; Linguagem; Desenvolvimento Infantil
INTRODUCTION
Gesture-speech synchrony has been studied in understanding the emergence of childhood fluency(1,2), highlighting, based on classic studies on human gestures, that there is continuity and integration between gesture and speech in the functioning of children's language. In this sense, synchrony occurs when the modalities of language use (speech and gesture) work together in linguistic production, creating points of salience in the utterance. This is an approach that offers contributions to thinking about atypical language development(3).
In this study, this multimodal approach to linguistic interactions is incorporated into the enunciative perspective of language acquisition proposed by Silva(4), in order to reflect on the initial conjunction and disjunction relations between mother and baby, permeated by processes of homology and interpretance in the relationship between the baby's gestural and vocal manifestations and the mother's verbal ones, present in the early protoconversations(5).
It is known that the transition from the baby's discursive dependence on the mother, present in conjunctive relations, to the recognition of the effects of their manifestations on the other, present in disjunctive relations, propels babies towards the constitution of the second enunciative mechanism, which is the shift from shown reference to spoken reference(4). For this transition to occur, the baby must be able to occupy their place of enunciation, and the mother must sustain this place for them(6).
In this sense, a study(7) highlighted both difficulties in the exercise of parental functions, in the supposition of the subject, and in the establishment of demand(7), as well as the risk of evolution towards an ASD (Autism Spectrum Disorder) condition(7). Oliveira et al.(8) observed a higher correlation between patterns of psychic suffering in the first six months of life. In the second, third, and fourth semesters of the babies' lives, psychic suffering and language acquisition delays coincide, but there are also cases of language acquisition delay without psychic risk. These studies showed that the presence of psychic suffering, whether or not it is moving towards an autistic structure, is significantly correlated with the presence of language acquisition delay, analyzed from an enunciative perspective.
The evaluation method in both studies(7,8) used the Enunciative Signs of Language Acquisition(9,10), which analyze the conjunctive and disjunctive relations of the first enunciative mechanism as well as the emergence of the second enunciative mechanism, but without a more detailed discrimination of gestural aspects. Therefore, in this research, the aim was not only to investigate the enunciative effects of psychic suffering but also to add a multimodal perspective to the study of the language acquisition process, especially regarding the first and second enunciative mechanisms(4).
This is based on the belief that investigating the multimodal characteristics of gesture-speech synchrony—such as speech, gesture, directed gaze, facial expressions, head movements, and other indicators—can shed light on signs for the early detection of risk and for timely intervention that seeks to establish mother-baby enunciative conjunction relations, which form the foundation for the emergence of subsequent enunciative mechanisms in the language acquisition process.
Considering the group of children with language acquisition delay, although there are already studies on gesture-speech synchrony in children with Down syndrome(3), no studies have been found addressing this synchrony in children with ASD within the national context, integrated with the enunciative perspective. Thus, the objectives of this study are to compare gesture-speech synchrony in language functioning between mothers and babies, as well as the gestural typology of babies aged 3 to 26 months, one of whom has autism spectrum disorder (Case R), diagnosed at the age of 3, and the other without this diagnosis (Case B).
METHOD
This is a qualitative, longitudinal, observational, analytical, and comparative case study;involving two children, one with ASD (R) and one without (B), followed from 3 months to 26 months of age. The study was approved by the university's research ethics committee under CAAE number 28586914.0.0000.5346, opinion number 1.929.266. The children's legal guardians signed the informed consent form, authorizing their participation in the study.
The video data of the interactions between the babies and their mothers analyzed in this study come from a larger database in which the babies were filmed across six age ranges for 15 minutes each. In addition, the babies and their mothers were assessed using the Enunciative Signs of Language Acquisition (SEAL)(9,10), the Clinical Indicators of Risk/Reference* for Child Development (IRDI)(11), PREAUT Signs(12), M-CHAT(13), and Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III)(14). The following collection and assessment procedures were applied for each age range:
-
Age Range 1 – 3 months and 1 day to 4 months and 29 days: In a seated position in an infant car seat (9 minutes). In this position, the mother was invited to sing (3 minutes), talk (3 minutes), and offer an object to the baby—a silent rubber dog (3 minutes). In prone (3 minutes) and supine (3 minutes) positions, the mother could optionally offer a toy or talk to the baby. Assessed with IRDI Range I, PREAUT Signs, SEAL.
-
Age Range 2 – 5 months and 1 day to 6 months and 29 days: In a seated position in an infant car seat (9 minutes). The mother was invited to sing (3 minutes), talk (3 minutes), and offer the rubber dog to the baby (3 minutes). In prone (3 minutes) and supine (3 minutes) positions, the mother could offer a toy or engage in conversation. Assessed with SEAL.
-
Age Range 3 – 8 months and 1 day to 9 months and 29 days: Seated without support if possible (9 minutes). The mother was instructed to sing to the baby (3 minutes), talk (3 minutes), and offer the rubber dog (3 minutes). In prone (3 minutes) and supine (3 minutes) positions, the mother was asked to stimulate the baby to roll over or crawl, and attempts by the baby to stand with or without support were observed. Assessed with IRDI Phase II, PREAUT Signs, SEAL.
-
Age Range 4 – 11 months and 1 day to 12 months and 29 days: In this stage, the baby was free to explore a box of themed toys (transportation, play food, dolls, animals) with the mother on an EVA foam mat. The baby could sit or walk as long as they stayed on the mat for filming purposes. Ten minutes were spent with the mother and five minutes with the therapist entering the scene to check some enunciative signs. Assessed with IRDI Phase III, SEAL.
-
Age Range 5 – 17 months and 1 day to 18 months and 29 days: Same procedure as Range 4. Assessed with IRDI Phase IV, SEAL, Bayley-III, M-CHAT.
-
Age Range 6 – 22 months and 1 day to 26 months: Same procedure as Ranges 4 and 5. Assessed with SEAL, Bayley-III, M-CHAT.
From the video database, which included moments when the mothers sang, talked, and played with the children, excerpts were selected in which face-to-face interactions were more prominent, standardizing both the duration and conditions between the two cases, as shown in Table 1.
It is important to highlight that once risk signs were identified in R. during the initial assessments, efforts were made to provide timely intervention; however, the child remained in therapy for only two months, from 10 to 12 months of age. This had an impact during Phase 5 but was not sustained over time, as by 26 months—his last recording—he exhibited clear signs of ASD.
Based on this selection, the data were analyzed using Eudico Linguistic Annotator (ELAN) software, which enabled the transcription of excerpts from key interaction moments for detailed visualization, time coding, and descriptions according to the established tracks, as described in Table 2.
The study also aimed to identify the emerging gestural typology in each case at each age range, as described in Table 3.
From the ELAN annotations, descriptive statistics were generated and are presented in the Results section.
RESULTS
Table 4 presents the results of the general categories of the analyses.
These results show that there are no significant distinctions between the mothers. However, among the children, clearer distinctions emerge in eye contact, child vocalization, and gesture-speech synchrony, with Case R consistently showing lower values. These differences become more visually evident in Figures 1 and 2.
Thus, while B progressively expands vocalizations towards speech, gesture-speech synchrony, and eye contact with the mother—reducing silence—R maintains lower levels of vocalization and gesture-speech synchrony, decreases eye contact with his mother, and increases periods of silence. In this way, it can be said that while dyad B consolidates language functioning through dialogue, dyad R does not evolve as expected over the course of the follow-up.
Regarding gestural development, Table 5 presents the general expressions, and Figure 3 shows the gestural typology of each child, based on the initial gesture classification used in the study.
In R's case, there were general facial expressions (Table 5), displaying gestures of complaint, discomfort, surprise, or nervousness that differed from B’s from the very first phase. In age range 2, complaints and discomfort dominated his manifestations. In age ranges 3 and 4, in addition to gestures of complaint, discomfort, and unease, gestures of happiness (Table 5), pantomime, deictic gestures, and emblems (Figure 3) were added. This shows that he may present a gesture typology similar to B in the expected variety of gestures but differs in the constant presence of discomfort, unease, and complaint. Another difference lies in gesture quality: R’s pantomime was always the same—reproducing a “vroom-vroom” for the toy car—demonstrating his hyperfocus on the car, while B’s pantomimes varied, covering different animals, vehicles, and feeding scenes.
Multimodal transcription examples from age phases 1 and 6 for both cases are presented in Chart 1.
As shown in Chart 1, R demonstrates more discomfort and less progress in shared attention and engagement with his mother compared to B. While B shows pleasure in physical contact, R avoids contact with his mother in age range 1. In age range 6, B dramatizes a dinosaur attack on a doll with his mother using pantomime, while R barely engages in his mother's play proposals.
It is important to highlight that during R’s moments of discomfort, it was common for the mother to interpret his reactions as anger rather than as signs of pain. Over the two years of follow-up, the mother was unable to perceive that R experienced physical pain linked to sensory issues, including tactile, auditory, visual, and vestibular hypersensitivity, among other difficulties.
DISCUSSION
From the very first phase, the difference between R and B is clear: in the multimodal transcriptions, R exhibits less varied gestures, with lower frequency and shorter duration. Notably, there is less synchrony between speech and gesture, as well as reduced eye contact, as shown in Figure 2. In addition, R produces fewer vocalizations than B and has more observable moments of silence (Figure 1). There are many instances of discomfort and unease in R, while B is consistently smiling and enjoying physical, visual, and bodily contact with his mother and with objects from the first phase onward. This is also evident in the multimodal transcription presented in Chart 1.
These findings highlight that R presents a sensory profile of hypersensitivity to visual, auditory, and tactile stimuli, which hinders his engagement in pleasurable interactions with his mother. This aspect is noted as a sensory-motor characteristic of infants who later develop ASD(15). Since R’s mother assumes a subject and a speaker in her child(6), she seeks to interpret his signals, showing that she invests in the dialogue, much like B's mother. However, the multimodal transcriptions show that R's mother struggles to validate her child’s feelings, often assigning directive interpretations to his expressions. This may be related both to her difficulty in perceiving the origin of her child’s discomfort and to R’s limited time-window for gesture-speech synchronization, meaning his abilities to synchronize and express himself did not allow sufficient space for his mother’s interpretation. As a result, she follows her own discourse, missing some of R’s manifestations, particularly in age ranges 4 and 5.
Furthermore, R's repertoire is restricted; although he performs several pantomimes (age range 5), they are always the same, characterized by repetitive behaviors consistent with an ASD diagnosis. It can be stated that R’s mother initiated the processes of homology and interpretance(6) necessary for establishing the conjunctive relations that form the basis of the first enunciative mechanism of language acquisition(4). However, R’s difficulties with engagement, shared attention(1,2), and the presence of significant discomfort hindered the maintenance of initial protoconversations in this dyad.
In contrast, in B’s case, the gesture-speech synchrony allows the mother to form broader interpretive hypotheses, consolidating from the early stages the conjunctive relations(4) in which B participates with vocalizations, smiles, and gestures, and the mother responds through her speech interpreting the child’s expressions(5). B demonstrates the ability to integrate gesture and speech in a way that allows the mother to recognize him as a speaker(7). This recognition enables the mother to gradually reduce physical contact in the later phases and to synchronize more seamlessly with her child. Overall, her interpretations align with B’s manifestations. B’s productions are not left adrift, and thus the transition from shown reference to spoken reference emerges clearly by the fourth age range, revealing his capacity for coreference(4). In addition, B’s gestural repertoire includes a wider variety of gestures (metaphorical, emblematic, and varied pantomimes), which supports the idea that his gestures are integrated into speech, as seen in studies on the relationship between gesture and fluency in child language acquisition(1,2).
Therefore, although R’s mother invests in the relationship and remains more talkative and uses more gestures to engage her baby than B’s mother, she is not successful in establishing conjunctive relations(4) with R because his sensory profile seems to trigger avoidance of eye contact and maternal touch, which disrupt the synchrony between mother and child. Interestingly, maternal touch may be perceived as painful by R, stripping it of the deictic quality that could help build shared reference between mother and child. In contrast, the pain felt by R may strip the gesture of its representative character and deprive it of linguistic representation. In B’s case, this process is not hindered, as he seems to experience pleasure and registers maternal touch as a deictic indication, where words can be inscribed within this representation(1,2).
In this regard, an essential condition to be analyzed in maternal behaviors in response to infant manifestations—crucial for establishing conjunctive relations(4)—is the mother's ability to offer an interpretation and validate the child’s feelings, as this has the effect of recognizing the child as both subject and speaker(6). On the child's side, it is necessary that they can register the mother’s gestures and touch as forms of representation within language functioning. When the child experiences pain and discomfort, as was often the case with R, it becomes difficult to extract the sensorimotor invariants necessary to create stable representations of self and the world, as proposed by Bullinger(15). On the contrary, pain throws R into an avoidance circuit of interaction with the other, which hinders the construction of representations necessary for the emergence of semiotic function.
Without shared attention, one of the early prerequisites for communication(2), it is difficult to establish the conjunctive relations proposed by Silva(4). Shared attention, as a capacity for coordinated attention with another person, is essential for social interactions because it helps understand the world, enables shared meaning-making, allows the interpretation of others, and fosters reciprocal affective synchrony. Gestural expressions also involve intentions, behavioral dispositions, and the experiential dimension of affects. The mothers of both children, in the two cases studied, interpreted their children’s manifestations based on their own experiences. In B’s case, the mother found meaning in the child’s gestural reactions because the context allowed her to interpret his expressions accurately. In R’s case, however, the mother, based on her own knowledge, struggled to achieve affective synchrony due to the absence of sensory experiences similar to those of her child(16).
It is known that gestures include hand movements, eye gaze, facial expressions, and head movements(1,2). More gestures were observed in the mothers than in the babies. Facial expressions convey people’s emotional states and help the interlocutor interpret paths to either promote or avoid interpersonal interaction. Universal emotions such as disgust, fear, joy, surprise, and sadness reveal facial characteristics based on physiologically rooted emotions, which are not learned but form a basis for composing more complex gestures(16). When these expressions are not understood, as occurred with R, it becomes difficult to sustain a place of enunciation(6), especially for a child whose production is anchored in gesturality.
Another fundamental aspect to consider, from the multimodal perspective of language acquisition(1,2), is that gesture is a modality of language and therefore belongs to the verbal domain. Gesture not only accompanies speech but also integrates with it and is fundamental for the infant to occupy their place of enunciation(6). It is clear that the more gestures B produces, the more he also progresses from shown to spoken reference. Thus, gesture can be viewed as a semiotic system that integrates the semiotic domain and is fundamental as the first form of meaning-making within the I-YOU language functioning(4). Gesture-speech synchrony lays the foundation for the baby to synchronize with the mother in language functioning, allowing the infant to occupy and maintain a place of enunciation(7).
Upon completing the analysis in this study, it becomes clear that there is a mismatch in the language functioning between R and his mother, and a progressively strengthened alignment between B and his mother. The data also highlight the importance of timely interventions so that mothers of children at risk of evolving towards an ASD diagnosis can understand how to compensate for their children's sensory difficulties, thereby enabling communication. This understanding is essential for sustaining a place of enunciation for children experiencing psychic suffering and for minimizing the symptoms that emerge as signs of crystallized suffering around the second year of life, including language disorder. Speech-language pathologists who know how to observe gesture-speech synchrony between mothers and babies can use this knowledge to build individualized therapeutic plans through timely interventions.
Although this case study is limited in its generalizability to the broader ASD population, it offers insights for future research that incorporates gestures as an important aspect of childhood speech fluency and therefore also fundamental in speech-language research.
CONCLUSION
Although the gestural typology of B and R is similar, the frequency, quality, and, above all, the gesture-speech synchrony differ in the language functioning of the dyads.
In B's case, without ASD, gesture-speech synchrony, eye contact, and vocalizations converge to sustain protoconversations with his mother. In R's case, with ASD, the mismatch within the dyad arises from the presence of pain and R’s difficulty in remaining attentive, engaged, and fully present in the relationship. His sensory restrictions do not create the bodily conditions necessary for him to progressively occupy his place of enunciation towards language acquisition and instead fuel the disconnection between him and his mother.
-
*
The original name of the script is risk but it was recently replaced by reference.
-
Study conducted at Universidade Federal de Santa Maria – UFSM - Santa Maria (RS), Brasil, Universidade Federal da Paraíba – UFPB - João Pessoa (PB), Brasil and Universidade Federal do Rio Grande do Sul – UFRGS - Porto Alegre (RS), Brasil.
-
Financial support:
There was a CAPES scholarship social demand for Gabriela Luisa Gantier Fernández's master's degree.
-
Data availability:
Research data is only available upon request.
References
-
1 Cavalcante MCB, Barros ATMC, Silva PMS, Nobrega PVA. Sincronia gesto fala na emergência da fluência infantil. Estudos Linguísticos. 2016;45(2):411-26. http://doi.org/10.21165/el.v45i2.984
» http://doi.org/10.21165/el.v45i2.984 -
2 Cavalcante MCB. Contribuições dos estudos gestuais para as pesquisas em aquisição da linguagem. Linguagem e Ensino. 2018;21:5-35. http://doi.org/10.15210/rle.v21i0.15199
» http://doi.org/10.15210/rle.v21i0.15199 -
3 Lima I, Delgado IC, Cavalcante MCB. Desenvolvimento da linguagem na síndrome de Down: análise da literatura. Distúrb Comun. 2017;29(i2):354-64. http://doi.org/10.23925/2176-2724.2017v29i2p354-364
» http://doi.org/10.23925/2176-2724.2017v29i2p354-364 - 4 Silva CLC. A criança na linguagem: enunciação e aquisição. Campinas: Pontes; 2009.
-
5 Kruel CS, Rechia IC, Oliveira LD, Souza APR. Categorias enunciativas na descrição do funcionamento de linguagem de mães e bebês de um a quatro meses. CoDAS. 2016;28(3):244-51. http://doi.org/10.1590/2317-1782/20162015190 PMid:27462732.
» http://doi.org/10.1590/2317-1782/20162015190 - 6 Souza APR. Clínica fonoaudiológica de linguagem com crianças pequenas e seus familiares. Santa Maria: Editora UFSM; 2022.
-
7 Bolzan RS, Moraes AB, Souza APR. Análise da relação entre eixos estruturantes na constituição do psiquismo e emergência de um lugar de enunciação de bebês com e sem atraso na aquisição da linguagem. CoDAS. 2023;35(1):e20210296. http://doi.org/10.1590/2317-1782/20212021296pt PMid:36259822.
» http://doi.org/10.1590/2317-1782/20212021296pt -
8 Oliveira LD, Moraes AB, Nunes SF, Costa I, Souza APR. Relação entre sinais enunciativos de aquisição da linguagem e a avaliação de linguagem pela escala Bayley III aos 24 meses. CoDAS. 2023;35(3):e20210221. http://doi.org/10.1590/2317-1782/20232021221pt PMid:37194767.
» http://doi.org/10.1590/2317-1782/20232021221pt -
9 Crestani AH, Moraes AB, Souza AM, Souza APR. Construct validation of enunciative sings of language acquisition for the first year of life. CoDAS. 2020;32(3):e20180279. http://doi.org/10.1590/2317-1782/20202018279 PMid:32578837.
» http://doi.org/10.1590/2317-1782/20202018279 -
10 Fattore IM, Moraes AB, Crestani AH, Souza AM, Souza APR. Validação de conteúdo e construto de sinais enunciativos de aquisição da linguagem no segundo ano de vida. CoDAS. 2022;34(2):e20200252. http://doi.org/10.1590/2317-1782/20202020252 PMid:34932657.
» http://doi.org/10.1590/2317-1782/20202020252 -
11 Kupfer MC, Jerusalinsky AN, Bernardino LMF, Wanderley D, Rocha PSB, Molina SE, et al. Valor preditivo de indicadores clínicos de risco para o desenvolvimento infantil: um estudo a partir da teoria psicanalítica. Rev Latinoam Psicopatol Fundam. 2010;13(1):31-52. http://doi.org/10.1590/S1415-47142010000100003
» http://doi.org/10.1590/S1415-47142010000100003 -
12 Olliac B, Crespin G, Laznik MC, Cherif Idrissi El Ganouni O, Sarradet JL, Bauby C, et al. Infant and dyadic assessment in early community-based screening for autism spectrum disorder with the PREAUT grid. PLoS One. 2017;12(12):e0188831. http://doi.org/10.1371/journal.pone.0188831 PMid:29216234.
» http://doi.org/10.1371/journal.pone.0188831 - 13 Muratori F. O diagnóstico precoce do autismo: guia prático para pediatras. Salvador: Núcleo Interdisciplinar de Intervenção Precoce da Bahia, 2014.
- 14 Bayley N. Bayley Scales of Infant And Toddler Development - Third Edition. Administration Manual. San Antonio, TX: The Psychological Corporation, 2006.
-
15 Bullinger A. Approche sensoriomotrice des troubles envahissants du dévelopment. Contraste [Internet]. 2006 [citado em 2024 Out 3];2(5):125-39. Disponível em: https://www.cairn.info/revue-contraste-2006-2-page-125.htm
» https://www.cairn.info/revue-contraste-2006-2-page-125.htm -
16 Lopez H, Agulla L, Zabaletta V, Vivaz L, Lòpez M. Rostro, gesto y emociones: procesamiento diferencial de las expresiones faciales emocionales en población infanto- juvenil según el sexo. Rev Argent Cienc Comport. 2017;9(3):31-43. http://doi.org/10.32348/1852.4206.v9.n3.18521
» http://doi.org/10.32348/1852.4206.v9.n3.18521
Edited by
-
Editor:
Stela Maris Aguiar Lemos.
Research data is only available upon request.
