Using assistive robots to promote inclusive education – First user trials

Pedro Encarnação1, Kim Adams2,3, Albert Cook2, Margarida Nunes da Ponte4, Anabela Caiado4, Teresa Leite5, Clarisse Nunes5, Joana Pereira4, Gonçalo Piedade1, Margarida Ribeiro1, Alexandra Martins5, and Mônica Silva5

1CLSBE, Catholic University of Portugal, Lisbon, Portugal

2Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada

3Glenrose Rehabilitation Hospital, Edmonton, AB, Canada

4Calouste Gulbenkian Rehabilitation Center for Cerebral Palsy, Lisbon, Portugal

5School of Education, Lisbon Polytechnic Institute, Portugal


This paper describes the first user trials results of an integrated augmented manipulation and communication robotic assistive technology to enable children with disabilities to actively participate, along with their typically developing peers, in academic activities requiring the manipulation of educational items. Five children with different degrees of physical and communication impairments used the system in their pre-school or first grade classes to perform academic activities designed together with their teachers. Goal attainment scales were defined for each child to assess their performance in the areas of communication, participation, autonomy, and academic achievement. Teachers were interviewed to evaluate their perceptions of the use of the assistive technology and its impact on the student and in the classroom. In general, goal attainment scaling results show that the outcomes were the expected or better for all academic goals. Progress was more moderate for communication, participation, and autonomy goals. Teachers opinions about the system and its impact were positive, though they pointed the need for technical support to prepare and conduct the adapted activities.


Under the inclusive education approach, children with special education needs are, to the maximum possible extent, placed in regular schools. In regular classrooms, students with disabilities should be involved in academic activities along with their typically developing peers, taking into consideration their individual needs. Many academic activities require the manipulation of objects while describing the activities or objects. This poses a challenge for children with physical and speech impairments.

Figure shows a child facing a computer screen with a communication grid. The grid includes vocabulary cells as well as robot command cells (e.g. forward, backward, left, right). On the same table where the screen is, it is possible to see a small car-like Lego robot with a gripper and a pen attached, along with educational items to be manipulated by the robot. Figure 1: Participant using the physical IAMCAT.
In order to support children participation in academic activities, an integrated augmentative manipulation and communication assistive technology (IAMCAT) was developed (Encarnação, et al., 2014a). Manipulation of items is achieved through a Lego® Mindstorms® car-like robot with a gripper and a pen attached. Robot control cells were included in communication boards of the augmentative and alternative communication (AAC) software by Sensory Software The Grid 2. Children can thus use their preferred computer access method (e.g. switches or an eye tracking system) to use the assistive technology both for communicating and controlling the robot (Figure 1). Both a physical and a virtual version of the IAMCAT were developed. With the virtual version, children control a virtual robot to manipulate virtual objects on a computer screen.

Children with disabilities trialed the system (physical or virtual) to perform academic activities in their regular classes. The activities were prepared with the participants' teachers and were designed to be performed by all students in class. This paper reports the results of the first user trials.


The experimental objectives were:

  1. Evaluate academic achievement when using the IAMCAT compared to baseline performance before intervention;
  2. Assess teachers’ perceptions of the use of the IAMCAT and its impact on the student and in the classroom (e.g., student’s engagement with activities, distractive and social inclusion factors);
  3. Compare virtual and physical robotic systems in relation to 1 and 2.



Five children with disabilities, enrolled in regular classes, were recruited in the great Lisbon area (Portugal) for the 2013-2014 academic year. Table 1 contains the profiles of this sample. Informed consents were obtained from parents. Each child either used the physical or the virtual robot as dictated by chance.

The participants’ teachers, four regular and five special education teachers were also involved in the study.

Training sessions

Before using the robot as an augmentative manipulation tool in class, participants went through a variable number of training sessions following the protocol reported in (Adams & Encarnação, 2011). Results of these training sessions can be found in (Encarnação, et al., 2014b). In general, after the training sessions, participants were able to a) drive to any workspace location, b) pick and place objects, c) use a pen to draw lines, and d) use the Grid system and switch between communication and robot control symbols.

Classroom sessions

Participants’ regular teachers and special education teachers were involved in the classroom sessions. A portfolio of IAMCAT-adapted Portuguese Language, Mathematics, and Science and Social Studies activities was presented to the teachers for them to better understand the capabilities of the IAMCAT. Then, the regular teacher, with the support of the special education teacher and of the research team, prepared academic activities in the above three curricular areas to be performed by all students in class. These were all activities framed in the context of the particular class, addressing the curriculum content as in their regular class planning. Examples of activities proposed by the teachers based on the story “Popville” by Anouck Boisrobert and Louis Rigaud were:

  1. Portuguese language activities: answering interpretation questions using the IAMCAT AAC capabilities or controlling the robot to draw a line towards the correct answer among several options; matching words and pictures using the robot to move the labels “village” and “town” to the corresponding pictures;
  2. Mathematics activities: drive the robot from the village to the city, through a road with junctions where the child has to choose the correct way; count the number of landmarks on the way from the village to the city and say the number using the AAC device;
  3. Science and social studies: matching means of transportation (subway and plane, horse and tractor, bicycle and boat) with the location (city, village, village & city) using the robot to move the pictures of the means of transportation to the corresponding pictures of the locations.

All necessary physical materials or the virtual scenarios were prepared by the research team prior to the classroom sessions. Class activities for participants #1, #4 and #5 were conducted by their regular teachers, while for participant #2 classes were given by the special education teacher. In the case of participant #3, at the request of her regular teacher, activities were conducted by one of the researchers. The special education teacher or one of the researchers provided technical support for the robot, and academic and robot control support to the study participant. The activities were proposed to the entire class: each participant had the opportunity to perform the activities using the IAMCAT and his/her peers did the activities with pencils on paper or cutting and gluing, as required by the particular activity. Three classroom sessions were organized for each child, one dedicated to each curricular area. Participant #1 did four classroom sessions since his teacher wanted to include a free drawing activity with the robot.

Goal attainment scaling (GAS), a criterion-referenced, individualized objective measure (Schlosser, 2004), was used to evaluate objective 1 and system use. GAS allows the identification of multiple and individualized goals for each child. Individualized goals in the following categories were developed by the participants' teachers or therapists: a) communication, b) participation, c) autonomy, d) Portuguese Oral Language, e) Portuguese Written Language, f) Mathematics, and g) Science and Social Sciences. Since no worsening in performance was expected, scales were developed such that all participants were at level -2 in all goals prior to the intervention (Schlosser, 2004). Example goals are shown in Table 2.

Data collection

Classroom sessions were videotaped with two cameras, filming the participant from the front and from the back, thus having a view of the participants face and also a view of his/her interactions with the system and of the activity.

To evaluate objective 1 teachers were asked to score participants’ performance according to the goals defined.  To evaluate objective 2, participants’ teachers were interviewed and a content analysis of the interviews was performed (Roberts, 1997) using the Atlas.ti® 6.2 software.


Table 3 presents the outcome scores referring to the seven goals defined. Both the regular and the special education teachers rated each participant’s performance based on their perceptions of the sessions.  In some cases, one of the teachers declined to rate some goals, because (s)he did not have sufficient awareness of the participant’s performance during the session. Table 3 shows either the lowest score between the regular and the special education teacher or the only score.  Inter-rater reliability, computed without considering the cases where one of teachers did not score the participant, varied from 0% (the two teachers did not agree in any of the goals) to 85.7%. Using the lowest score between the regular and special education teachers corresponds to a conservative choice, biased towards the ineffectiveness of the intervention. Assuming that all goals are equally important, T-scores were computed following (Schlosser, 2004). Before the intervention, by construction of the scales, all participants scored -2 in all goals and thus had a T-score of 18.4. If a participant achieves the expected outcome in all goals, the T-value would be 50. The higher the T-score, the better the participants’ performance with respect to the defined goals. An aggregated T-score across all participants of 46.8 was obtained.

Positive perceptions regarding robot use revealed by content analysis of the teachers’ interviews include: it is an enabling technology, can be used in the classroom and can have a positive impact in the entire class, children love the robot, it promotes inclusion, it facilitates the learning experience, it promotes playfulness. Negative perceptions include: it’s expensive, it requires technical knowledge to use it, it requires a lot of time to prepare the activities and materials, it may distract the students from the activities.


In general, participants achieved at least the expected level (score 0) in all academic goals. This shows that the academic activities, performed with the support of the IAMCAT, were an effective learning experience for the participants. Progression in the communication, participation, and autonomy goals was more moderate. This may be related to the fact that success in these goals is highly dependent on the participant’s proficiency using the system, and participants only took part in three or four classroom sessions.

Rating was done by the participants’ teachers that were involved in the definition of the scales and also in the intervention. Additionally, rating was based on their perceptions of the sessions, not on observable data. All these factors may influence the results (Schlosser, 2004). Scores given may reflect more the previous knowledge of the participant and the teacher’s expectations of the child’s performance than the participant’s actual performance. That may help to explain the low inter-rater reliability results. However, it is significant that teachers considered that all children had some progress in all goals with the only exception of the Portuguese written language goal for a particular participant.

Teachers were in general positive regarding the use of the IAMCAT to support children with neuromotor disabilities in performing academic activities. Most positive aspects pointed out were that the system enables children to actually participate in academic activities and that it promotes children’s inclusion in the classroom. Negative aspects were related to the need of time and technical support to adapt and implement the academic activities.

The low number of participants and their non-uniformity, which is characteristic of studies involving children with disabilities (Ottenbacher, Tickle-Degnen, & Hasselkus, 2002), prevent a robust analysis of objective 3 (comparing the use of the physical and the virtual versions of the IAMCAT). Nevertheless, it is possible to state that the results obtained were not very different for the two systems, though slightly better for the participants that used the physical IAMCAT.

Study limitations

The low number of participants is one of the study limitations. A second group of four participants is using the IAMCAT in the academic year of 2014-2015.

Table 1: Participants profiles
  P#1 P#2 P#3 P#4 P#5
Age (years) 6 6 6 5 5
Diagnosis Cerebral Palsy (bilateral spastic, tetraparesis) Cerebral Palsy (bilateral spastic, tetraparesis) Head Injury (hemiplegia and aphasia) Cerebral Palsy (bilateral spastic, tetraparesis) Global developmental delay
Grade 1st grade 1st grade Pre-school Pre-school Pre-school
Computer access Direct access through eye tracking Direct access through trackball Direct access through trackball Direct access through trackball Direct access through trackball
Robot Physical Virtual Virtual Virtual Physical
Table 2: Goal attainment examples
Scores -2 -1 0 1 2
Communication Uses the IAMCAT to answer activity related questions only when encouraged (without encouragement, answers through other communication modalities) Uses the IAMCAT to answer activity related questions requiring sporadic encouragement Uses the IAMCAT to answer activity related questions without incentive Uses the IAMCAT to answer activity related questions without incentive, and to comment the activity when encouraged Uses the IAMCAT to answer activity related questions and to comment the activity, without incentive
Science and social studies Associates two means of transportation to the urban context in which they are used Associates four means of transportation to the urban context in which they are used Associates the means of transportation to the urban context in which they are used Associates the means of transportation to the urban context in which they are used and is able to tell another mean of transportation not involved in the task Associates the means of transportation to the urban context in which they are used and relates them to the ones used by his/her family

Goal attainment scores presented reflect only the teachers’ perspectives. Since they were involved in the definition of the scales and also in the intervention, the results may be biased. A video analysis by an independent observer is ongoing aiming at scoring each participant’s performance with respect to the defined goals.


Table 3 – Goal attainment scale ratings
  P#1 P#2 P#3 P#4 P#5
Communication 0 0 -1 -1 -1
Participation 0 -1 -1 -1 1
Autonomy 0 -1 -1 -1 0
Portuguese oral language 0 0 0 0 1
Portuguese written language 0 0 0 0 -2
Mathematics 1 0 0 0 0
Science and Social Studies 1 0 0 0 0
T-score 54.52 45.48 43.22 43.22 47.74

The integrated augmentative manipulation and communication assistive technology is an effective tool to support the participation of children with neuromotor disabilities in academic activities. However, it is necessary to consider the limited time and support teachers have to prepare academic activities in their daily routines. To overcome this difficulty, a set of activities can be made available for teachers covering different curriculum topics.

The study did not identify major differences between the use of the physical or the virtual version of the IAMCAT in classrooms. The virtual version can be less expensive, easier to use by non technical persons, and easier to disseminate. However, modifying the virtual activities requires technical skills to program the virtual scenarios. Customizations are easier to implement with the physical robot (e.g., by changing or adding educational items to be manipulated).


Adams, K., & Encarnação, P. (2011). A Training Protocol for Controlling Lego Robots via Speech Generating Devices. In G. Gelderblom, M. Soede, L. Adriaens, & K. Miesenberger (Ed.), Everyday Technology for Independence and Care - AAATE 2011, Assistive Technology Research Series (pp. 517-525). Maastricht: IOS Press.

Encarnação, P., Adams, K., Cook, A., Nunes da Ponte, M., Caiado, A., Leite, T., et al. (2014a). Cultural issues in implementing an integrated augmentative communication and manipulation assistive technology for academic activities. Proceedings of the ISAAC 2014 Conference. Lisbon.

Encarnação, P., Adams, K., Cook, A., Nunes da Ponte, M., Caiado, A., Leite, T., et al. (2014b). Training on the use of an integrated augmentative manipulation and communication assistive technology for academic activities. Proceedings of the RAatE 2014 - Recent Advances in Assistive Technology & Engineering. Birmingham.

Ottenbacher, K., Tickle-Degnen, L., & Hasselkus, B. (2002). Therapists Awake! The Challenge of Evidence-Based Occupational Therapy. American Journal of Occupational Therapy , 56, 247-249.

Roberts, C. (Ed.). (1997). Text Analysis for the Social Sciences: Methods for Drawing Inferences from Texts and Transcripts. Mahwah, NJ: Lawrence Erlbaum.

Schlosser, R. (2004). Goal attainment scaling as a clinical measurement technique in communication disorders: a critical review. Journal of Communication Disorders , 37, 217–239.


This work was supported in part by Fundação para a Ciência e a Tecnologia under project PTDC/MHC-CED/4199/2012 and by Fundação Portugal Telecom.

Audio Version PDF Version