The synchronization of gesture and prosody in French children’s multimodal pathway into negation

Résumé En

The expression of negation is a privileged locus to study multimodal combinations. There is a cross-modal continuity in the expression of negation speech acts, which are first mainly expressed by gestures, then by speech (Bates, Camaioni and Volterra, 1976; Bates et al., 1979). From the end of their first year on, children can express negation with headshakes then palms-ups and index waves. Prosody and gestures are also combined to express refusals, protests, epistemic negations or powerlessness, sometimes before the emergence of the first verbal negation markers. It is therefore crucial to analyze gestures and prosody with an integrative approach before and after the emergence of speech.The goal of this study was to analyze the synchronization of gesture and prosody in children’s expression of negation. We analyzed the longitudinal data of four monolingual children recorded monthly for one hour between the ages of 1;0 and 4;0 in spontaneous interaction with their parents (Paris Corpus, Morgenstern and Parisse, 2012). We studied the children’s productions within the MLU range of 1 to 4. We focused on the multimodal productions containing the word “non” (no) in isolation. Three types of analyses were conducted. First, we coded prosodic properties (direction of the intonation contour, accent range, register, duration, intensity), using PRAAT. Second, we coded nonverbal behavior (hand gestures, joint attention expressed through eye gaze and checking behavior, body movement and facial expressions), using ELAN. Third, we compared the prosodic and gestural analyses to look for directional and temporal synchronization patterns using AlphaPose, and comparing the outputs with our PRAAT extractions.At the prosodic level, results showed that the first vocal productions of “non” emerged around an MLU of 1.1 and were exaggerated at the prosodic level. Between an MLU of 1.1 and 1.8 (phase 1), “non” was mainly realized with rising intonation contours and increased syllabic duration. Between 1.8 and 2.8, (phase 2) it was mainly produced with rise-fall intonation contours and finally, between 3.3. and 4 (phase 3) with flat or falling intonation contours and reduced syllabic duration. Such an evolution seems to reflect a better control in the expression of negation as of an MLU of 2.8. At the non-vocal level, body movements were most often produced in coordination with verbal production and their direction was mostly synchronized with the direction of intonation contours (rising contours with rising gestures) during the first phase. The more the children expressed protests against adults, the more they exaggerated both their prosody (higher accent range, register, intensity and duration) and their body movements. During phase 2, they used mostly upper-body gestures and movements (head, chest) with a majority of forward and backward or oscillating movements in close parallel with their prosodic contours. As their mastery of speech developed, they gradually stopped exaggerating their prosody and resorted less to non-verbal behavior.Gestures, body movements and prosody provide powerful resources that the child integrates to make her multimodal entry into language. If children use each modality (vocal and visual) more and more skillfully thanks to adults’ scaffolding in everyday life interactions, both modalities actually develop together. This study therefore gives us insights on how children become experts in face-to-face social interaction, which is multimodal in nature.ReferencesBolinger, D. (1983). Intonation and gesture. American Speech, Vol. 58, 2, pp 156-174.Balog, H. and Brentari, D. (2008). The relationship between early gestures and intonation. First Language, 28, pp. 141-163.Bates, E., Camaioni, L. and Volterra, V. (1976). Sensorimotor performative. In: Bates, E. (Ed.). Language and Context: the Acquisition of Pragmatics. New York: Academic Press.Bates, E. (1979). Emergence of symbols in language and action: similarities and differences. Paper and Reports on Child Language Development, Stanford, n.17, p.106-118.Cruttenden, A. (1997). Intonation (2nd ed.). Cambridge: Cambridge University Press.Jusczyk, P. W. Dividing and conquering linguistic input (1998). In: Gruber, M. C.; Olson, K.; Wysocki, T. (Ed.). CLS 34. The Panels. v. 2, Chicago, p. 293-310.Mehler, J., Bertoncini, J. and Barriere (1978). M. Infant recognition of mother's voice. Perception, Bristol, n.7, p.491-497.Morgenstern, A & Parisse, C. (2012). The Paris Corpus. French Language Studies, 22(1), 7-12, Cambridge University Press.