2018 Sat Poster 6458

Saturday, November 3, 2018 | Poster Session II, Metcalf Small | 3:15pm

Using gestures in L2 vocabulary teaching: Human or robot tutors?
Ö. Demir-Lira, J. Kanero, C. Oranç, S. Koşkulu, I. Franko, O. Uluşahin, Z. Adıgüzel, T. Göksun

Gestures facilitate language learning (Goldin-Meadow & Wagner, 2005). However, not all studies observed the facilitatory effects of gestures, and specific conditions under which gestures aid language learning remain under debate (Congdon, 2016). We examined whether and how gestures facilitate second-language (L2) vocabulary learning in children by testing whether (1) type of facilitatory tool (gesture, on-screen highlighter) and (2) gesture type (deictic, iconic) affect learning outcomes. In addition to examining the characteristics of gestures, we also tested whether the gesturer makes an impact by comparing human versus robot tutors. We tested robots because the ability to gesture has been suggested as a strength of robots over other technological tools and robots are becoming increasingly common in educational settings (Kanero et al., 2018).

Seventy-seven Turkish-speaking 5-year-olds participated: Forty-three for the human tutor (M=66.96 months) and 34 for the robot tutor experiments (M=69.59 months). Children learned four pairs of English measurement words (e.g., small and big) with images of objects representing these words (e.g., small ball and big ball) presented on a laptop screen. All children experienced two learning conditions: one of the gesture conditions (Deictic or Iconic) and the on-screen Highlighter condition. In the Deictic condition, the tutor pointed to the corresponding object on the screen (e.g., big ball). In the Iconic condition, the tutor produced an iconic gesture representing the measurement word (see Figure 1 for an example). In the Highlighter condition, no gesture was performed and a red rectangle flashed around the object to draw children’s attention. During test, children were asked to point to the picture corresponding to the target measurement word.

A Generalized Mixed Effects Model using tutor type (human, robot), gesture type (iconic, deictic), sex (female, male) as between-subjects and facilitatory tool (gesture, on-screen highlighter) as a within-subject variable on the percentage of correct responses revealed a main effect of facilitatory tool, p=.03 – the highlighter condition led to better performance than the gesture conditions. We also found a marginally significant effect of tutor, p=.10, where children performed better with the robot compared to the human tutor. The main effects of gesture type, sex, and the interaction terms were not significant, ps>.20.

To our surprise, gestures did not result in significantly better learning, and gesture type had no significant effect on learning outcomes. The results suggest that gestures may not be as effective, or even distractive, for learners with no prior knowledge in a domain and when the task requires learners to pay attention to visuals in the learning environment. The difference between the gesture and on-screen highlighter conditions also suggests that the role of gesture in word learning might be different than simply guiding attention (Novack et al., 2016). Interestingly, children appeared to learn L2 words better with a robot than with a human. We will discuss possible explanations for the robot tutor advantage such as its novelty and the less lesson-like, friendlier atmosphere the robot might have created.

References

Congdon, E. L. (2016). Learning mathematics through action and gesture: Children’s prior knowledge matters (Doctoral dissertation, The University of Chicago).

Goldin-Meadow, S., & Wagner, S. M. (2005). How our hands help us learn. Trends in Cognitive Sciences, 9(5), 234-241.

Kanero, J., Geçkin V., Oranç, C., Mamus, E., Küntay, A. C., & Göksun, T. (2018). Social robots for early language learning. Child Development Perspectives.

Novack, M. A., Wakefield, E. M., Congdon, E. L., Franconeri, S., & Goldin-Meadow, S. (2016). There is More to Gesture Than Meets the Eye: Visual Attention to Gesture’s Referents Cannot Account for Its Facilitative Eff ects During Math Instruction. In Proceedings of the 38th Annual Meeting of the Cognitive Science Society.