RESNA Annual Conference - 2020

Exploring The Role Of Social Robots In Intergenerational Gameplay

Aishwarya K AravamuthanBegin Superscript1End Superscript, John E Muñoz Begin Superscript1End Superscript, Katrin FischerBegin Superscript1End Superscript, Shruti ChandraBegin Superscript1End Superscript, Jennifer BogerBegin Superscript1,2End Superscript, Kerstin DautenhahnBegin Superscript1End Superscript

Begin Superscript1End SuperscriptUniversity of Waterloo, Waterloo, ON, Canada; Begin Superscript2 End SuperscriptResearch Institute for Aging, Waterloo, ON, Canada

INTRODUCTION

The number of older adults is increasing; it is predicted that older adults will represent 23% to 25% of the total Canadian population by 2036 (1). Intergenerational interaction, specifically the interaction between a grandparent and their grandchild, has mutual as well as individual benefits in both the generations (2). Grandparent-grandchild communication helps in the emotional development of the grandchild, and the grandparents obtain satisfaction and companionship from the interaction (3).

Many studies suggest that collaborative gameplay fosters meaningful intergenerational interactions since games are conventionally attractive for both generations (4). Digital games that promote collaborative gameplay have produced enjoyable experiences among intergenerational members; however, these games involve the use of a PC or smartphone (5), which sometimes limits the interaction of the older generations. It has been shown that under specific conditions, a social robot can increase the interaction time between two players in multiplayer gameplay (6). For example, the study by S.Joshi et al. suggests that social robots have a positive impact on the intergenerational interaction in non-familial settings (7), demonstrating how social robots can be effectively used as a vehicle to foster connectivity and the creation of meaningful experiences among older adults and children. However, little is known on how to create interactive experiences that can leverage social robots empowered with games in fostering collaboration in intergenerational scenarios.

This research focuses on using an iterative design approach involving grandparents and their grandchildren to explore how a robot could mediate interaction in intergenerational gameplay. In this paper, we present the preliminary results of the first of a three-stage project that aims at exploring the role of a humanoid social robot in connecting dyads of grandparents and grandchildren. This first stage involves exploratory research to inform the design of robot-mediated gameplay for fostering intergenerational interactions.

METHODS

Objective:

The primary question that guided the research described in this paper is: what could the role of a social robot in an intergenerational multiplayer game be? This was explored through two sub-questions: i) How do dyads perceive their collaboration in a game? ii) What do the dyads of grandparent-grandchild envision the role of a social robot could be in a game?

Participants:

Participants were recruited as grandparent-grandchild dyads who could come to the research laboratory to participate in our study. The study was reviewed and approved by the University of Waterloo Research Ethics Committee. Our inclusion/exclusion criteria were:

Grandparent:

  1. age does not matter,
  2. must have a grandchild (6-10 years) who can come to the study location,
  3. must be able to travel to the study location,
  4. should be able to understand and speak English,
  5. normal to corrected vision.

Grandchild:

  1. 6-10 years old,
  2. should be able to understand and speak English.

Robot Used:

Figure 1 shows a social robot called Pepper.
It is a humanoid robot that has a tablet on its chest.
Figure 1: Pepper robot

We use a commercially available humanoid social robot, Pepper (Figure 1), for this study (8). Pepper is a good fit for this research as Human-Robot Interaction (HRI) studies have used it in different social scenarios, such as guiding people in a museum, and shopping malls, and it has been proposed for care for older adults (9-11). Pepper has a wide range of capabilities like speech, emotion recognition, which can make the interaction more engaging (12).

Pepper has 20 DOF and has tactile sensors in its head and hands. It has two cameras of 640*480 resolution each. The cameras are positioned in the head and mouth of the robot. Pepper also has four microphones at the top of its head and 3D sensors behind the eyes. A screen on Pepper's chest allows for audio-video display.

Game:

Several categories of games have been proposed as activities to mediate human-robot interactions for learning and entertainment. One of the most popular categories is puzzle games (13). We chose a puzzle game called Tangram, in which seven pieces of different geometric shapes are arranged together to form a desired shape (14). We chose the Tangram game as it can offer a collaborative activity, is suitable to be played by both children and adults, and it also provides a potential role for Pepper as a mediator in the gameplay. To make the interaction natural and comfortable for the participants, we used physical Tangram pieces. We modified the classic Tangram gameplay by dividing the pieces among the players to allow two players (instead of one), to arrange the shape collaboratively.

Study Setup:

Figure 2 shows the icon representation of the study setup.  
The left side shows the setup of sub-task 1 and the right side shows the setup of the sub-task 2.
Both the sub-tasks have a mediator, grandparent, and grandchild on three corners of the table.
Sub-task 1 in the absence of the robot and sub-task 2 in the presence of the robot.
Figure 2. Study setup

To study the role of Pepper in mediating the interaction, we decided to first observe dyads while playing the Tangram game naturally, without the presence of Pepper. In the second part of the session, Pepper was introduced to the dyads, and it was acting as an observer while players played another Tangram puzzle. Therefore, the study involved two sub-tasks (Figure 2):

  1. Sub-task 1 - Participants play Tangram with a human mediator: Dyads played Tangram in the presence of a human mediator. The human mediator observed the participants and provided help when asked. Afterward, participants provided feedback on their experience.
  2. Sub-task 2 - Participants play Tangram with a human mediator and the robot Pepper present: During this sub-task, the robot was present but did not interact in the game. The purpose of this sub-task was to introduce the robot and collect the participants' initial perception of the robot. The human mediator introduced Pepper to participants and collected feedback about their first impression as well as their thoughts regarding expected behavior from the robot.

The difficulty level of the Tangram challenge increased from sub-task 1 to sub-task 2.  

Study Protocol:

After obtaining written informed consent from the grandparent and the child's legal guardian for their participation, we also asked verbal assent from the child before the start of the session. The session started with participants completing a demographic form containing the dyad's demographic information as well as questions regarding how often they play games together, and what games they play. The moderator gave a basic introduction to the game, after which participants were asked to solve Tangram puzzles together in two sub-tasks. At the end of the session, participants were asked follow-up questions and completed a collaboration questionnaire. The whole session lasted no more than 90 minutes for each dyad.

Questionnaires:

Perceived Collaboration: To measure the collaboration between the dyads, we used a self-assessment collaboration questionnaire derived from the Coalition Effectiveness Inventory (CEI) (15). To make the scale more accessible to children, we collected the rating from the dyads using a pictorial scale of five smiley faces used by Miriam Donath based on a standard scale Smileyometer (16). We asked the grandchild to rate first so that the decision of their grandparent did not influence the grandchild. Particularly we asked: i) team working: how well did the two of you work together while playing Tangram? ii) communication: how well did you communicate today? and iii) problem-solving: how well did you solve problems together?

Post Session Questions: To understand how the dyads envision the role of Pepper in the Tangram gameplay, we asked the dyads the following questions: i) Let's imagine that Pepper learned how to play Tangram from observing you today. Would you like Pepper to be here the next time you play Tangram? ii) Why/why not? iii) What would Pepper be doing?

RESULTS

We conducted this study with four dyads of grandparents and grandchildren (grandparents aged between 52-74 years, and grandchildren aged between 7-9 years).

Perceived Collaboration:

Figure 3 shows the response of the dyads for the team working and communication questions.
The left graph shows the results for team working and the right for the communication question.
The x-axis shows the dyads (dyad1, dyad 2, dyad 3, dyad 4)
The y-axis shows the ratings (Awful, Not very good, okay, Really good, Fantastic)

For team working question:
Dyad 1:
Grandchild: Not very good
Grandparent: okay
Dyad 2:
Grandchild: okay
Grandparent: okay
Dyad 3:
Grandchild: Fantastic
Grandparent: Fantastic
Dyad 4:
Grandchild: Fantastic
Grandparent: Fantastic

For the communication question:
Dyad 1:
Grandchild: Really good
Grandparent: okay
Dyad 2:
Grandchild: not very good
Grandparent: okay
Dyad 3:
Grandchild: Really good
Grandparent: Really good
Dyad 4:
Grandchild: Really good
Grandparent: Really good
Figure 3: Questionnaire response

Figure 3 shows the dyads' response to the questions on team working and communication from the Collaboration questionnaire. Their response suggests that the dyads perceived good team working and communication between them. For the problem-solving, 2 dyads rated 'okay', one dyad rated 'Fantastic' and the other dyad rated 'Really good'.

Post Session Questions:

All four dyads said they would like to have Pepper present for the next time they play Tangram together. When we asked participants, 'What Pepper would be doing the next time they play Tangram?' most participants responded that they would like the robot to help them in the game. For example, one of the grandchildren responded,

'He will help us figure it out like just step-by-step, not like to tell us the exact answers but like just slowly work through.' One of the grandparents responded to the same question with 'We would appreciate some help when we are stuck.'

DISCUSSION AND CONCLUSION:

In some Tangram HRI studies, robots had been used as an active turn-taking player (14,17). In other studies, the robot helped the participants in the Tangram game, e.g., giving clues, revealing the correct and wrong pieces, picking up the pieces (14,18-20). In order to make the human-robot interaction more engaging, a few researchers have used storytelling robots with children (21,22). Unlike many HRI studies that focus on the interaction between a human and a robot, our research focuses on the collaboration between the intergenerational team mediated by a robot; namely, the intent is to use a social robot enrich the interactive experience of the humans with their interaction with each other (not the robot) as the focal point.

One of the results of our preliminary study in fostering collaboration for intergenerational dyads show that the dyads rated a high level of perceived collaboration for the Tangram game. This suggests that the Tangram game, along with our study setup, suited well in the collaborative intergenerational scenario. The high perceived collaboration can also be influenced by their previous experience of playing games together. Thus, in the next stage of our research, we will have the same game with a similar study setup.

Regarding the envisioned role of the Pepper, the responses of the participants suggest that having the robot provide help during gameplay would align with what they would expect, and what they feel would be a valuable contribution.  Considering this result, we include Pepper providing help in the gameplay as one of its characteristics. Therefore, we propose the role of a robot as a mediator where it would combine interactive storytelling along with the Tangram game. In other words, the robot would display tangram puzzles while unfolding a story and would also provide help while interacting during the gameplay.

The next step is to test how the proposed role of the robot works for the intergenerational scenario. For this, we plan to have a Wizard-of-Oz setting, in which a researcher (wizard) will control the robot. During the gameplay, when participants ask for help, the robot will either display a clue on its tablet or will give a verbal hint. This stage aims to understand the following aspects: 1- when and how the robot should interact with the dyads while they play the Tangram game and; 2- to see the suitability of the story narrating in this context. We believe this stage will give us insights for proposing a fully/semi-autonomous social robot in intergenerational gameplay.  

To conclude, in this paper, we have presented the results of the first stage of a multi-stage study on a robot-mediated intergenerational Tangram game. To the best of our knowledge, using a social robot to foster interaction among grandparent and grandchild is an underexplored area of research. We believe that Pepper robot mediating the Tangram game might provide a meaningful and entertaining experience for the intergenerational interaction.

REFERENCES:

  1. Seniors [Internet]. [cited 2020 Jan 31]. Available from: https://www150.statcan.gc.ca/n1/pub/11-402-x/2011000/chap/seniors-aines/seniors-aines-eng.html
  2. Costa L, Veloso A. Being (Grand) Players: Review of Digital Games and their Potential to Enhance Intergenerational Interactions. J Intergener Relatsh [Internet]. 2016;14(1):43–59. Available from: http://dx.doi.org/10.1080/15350770.2016.1138273
  3. Tomlin AM. Grandparents' influences on grandchildren. Handbook on grandparenthood. 1998:159-70..
  4. Boger J, Mercer K. Technology for fostering intergenerational connectivity: scoping review protocol. Systematic reviews. 2017 Dec;6(1):250..
  5. Zhang F, Kaufman D. A review of intergenerational play for facilitating interactions and learning. Gerontechnology. 2016;14(3):127–38.
  6. Zaga C, Lohse M, Truong KP, Evers V. The Effect of a Robot's Social Character on Children's Task Engagement: Peer Versus Tutor. In: Tapus A, André E, Martin J-C, Ferland F, Ammi M, editors. Social Robotics. Cham: Springer International Publishing; 2015. p. 704–13.
  7. Joshi S, Šabanović S. Robots for Inter-Generational Interactions: Implications for Nonfamilial Community Settings. ACM/IEEE Int Conf Human-Robot Interact. 2019;2019-March:478–86.
  8. Pepper | Softbank [Internet]. [cited 2019 May 17]. Available from: https://www.softbankrobotics.com/us/pepper
  9. Allegra D, Alessandro F, Santoro C, Stanco F. Experiences in using the pepper robotic platform for museum assistance applications. Proc - Int Conf Image Process ICIP. 2018;1033–7.
  10. Saad E, Neerincx MA, Hindriks K V. Welcoming Robot Behaviors for Drawing Attention. ACM/IEEE Int Conf Human-Robot Interact. 2019;2019-March:636–7.
  11. Martinez-Martin E, del Pobil AP. Personal robot assistants for elderly care: An overview. In: Intelligent Systems Reference Library. Springer Science and Business Media Deutschland GmbH; 2018. p. 77–91.
  12. Tanaka F, Isshiki K, Takahashi F, Uekusa M, Sei R, Hayashi K. Pepper learns together with children: Development of an educational application. IEEE-RAS Int Conf Humanoid Robot. 2015;2015-Decem:270–5.
  13. Leyzberg D, Spaulding S, Scassellati B. Personalizing robot tutors to individuals' learning differences. ACM/IEEE Int Conf Human-Robot Interact. 2014;423–30.
  14. Bernardo B, Alves-Oliveira P, Santos MG, Melo FS, Paiva A. An interactive tangram game for children with Autism. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 2016;10011 LNAI:500–4.
  15. FD Butterfoss. Coalition effectiveness inventory (CEI) self assessment tool. South Carolina: Center for Pediatric Research. Cent Heal DHEC. 1994;
  16. Read J, Macfarlane S, Casey C. Endurability , Engagement and Expectations : Measuring Children ' s Fun. Interact Des Child. 2002;2:1–23.
  17. Correia F, Guerra C, Mascarenhas S, Melo FS, Paiva A. Exploring the impact of fault justification in human-robot trust: Socially Interactive Agents Track. Proc Int Jt Conf Auton Agents Multiagent Syst AAMAS. 2018;1(September):507–13.
  18. Hirth J, Schmitz N, Berns K. Playing Tangram with the Humanoid Robot ROMAN. Proc Ger Conf Robot Ger Conf Robot (ROBOTIK), 7th, located conjunction with Robot Exhib Autom 2012, May 21-22, Munich, Ger [Internet]. 2012;18–23. Available from: http://www.dfki.de/web/forschung/publikationen/renameFileForDownload?filename=Hirth2012.pdf&file_id=uploads_1592
  19. Bonani M, Oliveira R, Correia F, Rodrigues A, Guerreiro T, Paiva A. What my eyes can't see,arobot can show me: Exploring the collaboration between blindpeople and robots. ASSETS 2018 - Proc 20th Int ACM SIGACCESS Conf Comput Access. 2018;15–27.
  20. Zamani MA, Mohammadi HB, Kerzel M, Magg S, Wermter S. Learning Spatial Representation for Safe Human-Robot Collaboration in Joint Manual Tasks.
  21. Park HW, Grover I, Spaulding S, Gomez L, Breazeal C. A Model-free Affective Reinforcement Learning Approach to Personalization of an Autonomous Social Robot Companion for Early Literacy Education. Proc 33th Conf Artif Intell (AAAI 2019). 2019;
  22. Shamsuddin S, Yussof H, Miskam A, Aminullah M, Hamid C, Malik NA, et al. Humanoid Robot NAO as HRI Mediator to Teach Emotions using Game-centered Approach for Children with Autism. 8th HRI Int Conf Hum Robot Interact [Internet]. 2013; Available from: https://pdfs.semanticscholar.org/af9b/fe93cdd96533275bb183645c680fba361759.pdf

ACKNOWLEDGEMENTS:

This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program. We thank the grandparent-grandchild dyads who participated in this research for sharing their time and knowledge.