RESNA Annual Conference - 2021

A Robot-Assisted System For At-Home Telerehabilitation

Uri Feldman

Wentworth Institute of Technology, Department of Biomedical Engineering, Boston, MA


As population ages and people live longer, conditions such as stroke, Parkinson's, dementia, as well as major joint replacement surgeries, and injuries caused by accidents such as falls, become more prevalent. Telerehabilitation can play an essential role in promoting wellness and rehabilitation in the treatment of such conditions. Telerehabilitation is a modality of telehealth in which rehabilitation services are delivered by a therapist who is not physically present with the patient. The addition of a robot to assist with the therapy enhances the level of intervention which can be provided at a distance. Factors which make telerehabilitation valuable include the access and convenience of care delivery, improved adherence to therapeutic regimen, and lower costs, all of which result in improved outcomes. [1]  The Covid-19 pandemic has accelerated the adoption of telerehabilitation further, so that patients receive rehabilitation services while remaining in their place of living. [2]  As per RESNA's "Welcome Home" theme, the system presented here is a viable and potentially transformative telerehabilitation solution which guides patients with their physical therapy and tracks their progress while at-home.

Physical therapy (PT) promotes an individual's health and well-being through movement. More specifically, PT typically is intended to improve function by strengthening and increasing range of motion of parts of the body in need of rehabilitation. The general principle of PT is that more movement results in better outcomes. [3] One challenge in the practice of PT is that rehabilitative measures are not applied consistently. Although published studies suggest that therapy improves functional outcomes, the amount, intensity, timing, frequency, and duration have not been established. [4] For example, metrics such as the Medicare OASIS and the FIM (functional independent measure), require the therapist to perform subjective assessments of perceived level of functioning based on specific activities of daily life (ADL) and these can vary from therapist to therapist and even from session to session. [5] Objective measures such as the Berg Balance Scale, Tinetti's Balance and Gait Evaluation, 6-Minute Walk Test (6MWT), Functional Reach Test and the Timed Up and Go (TUG), are frequently used but these are task-specific and may not generalize consistently. [5-6]

Patient engagement and motivation present another challenge in the delivery of PT. Physical therapy treatments are generally performed over long periods of time and frequently result in patient boredom, lack of engagement, and worst of all, quitting. [7] This is particularly the case with children, who may not understand the benefits of the therapy or with individuals with conditions which affect their memory and may not be able to initiate the therapy or remember the steps on their own. Therapists often use methods such as motivational interviewing, person first language, and patient specific goal development to motivate patients during therapy sessions, but these methods are effective only while the therapist is present in person with the patient.

The objective of this project is to develop a robot-assisted system which engages the patient to conduct their therapy and which tracks their rehabilitative progress. The rehabilitative outcomes considered in this project focus on expanding upper body range of motion (ROM), strengthening core/trunk musculature, as well as developing and improving balance, proprioception and kinesthetic awareness for trunk weight shifting and righting reactions. The system is designed not only for those in need of rehabilitation but also for building strength and endurance in healthy individuals who can benefit from improved balance to prevent falls, such as the elderly. [8] It should be noted that this robotic system is not designed to be a replacement for therapists, but rather, to serve as a tool for therapists to deliver therapy at home so that patients can achieve their rehabilitative outcomes.


Telerehabilitation has been implemented effectively in several forms. There is a category of tabletop computer systems in which therapy is delivered through interactive games, where patient movements are tracked and monitored using motion sensors such as Kinect or telepresence cameras. While such systems have proven to be effective, they require a specialized sensor and are not always able to track movements under some conditions. [9]  Telerehabilitation systems which deliver therapy by having patients mimic a humanoid robot with movable limbs are also being developed. Examples of such types of systems include Lil'Flo and the NAOTherapist. [10-12]  In these telepresence systems, therapists can communicate with the patients remotely using video and voice to provide additional guidance and motivation. The system proposed here builds on these approaches. The system does not require patients to mimic motions of moving limbs on a humanoid robot, but rather, patients perform therapy by actively participating in games presented to them on a digital display. This results in a system which can be implemented more simply and without the need for additional moving parts or sensors.


System Architecture

The system developed in this paper is shown in Figure 1 and is made up of the following components:

  1. A robotic-base transports the system to the place where the patient is located to deliver the prescribed physical therapy treatment. The robot is guided via remote-control. An iRobot Create2 platform was used for the first prototype because of its simplicity and low cost (under $500). In a typical therapy scenario, the robotic base transports the system to the patient, where it positions itself, adjusts the height and angle of the display, and initiates the therapy.
  2. Tele-rehabilitation software  uses Restore Skills software, a commercially available software as a service system (SaaS) with around 10,000 montly users. [13] The therapy is delivered as games which engage the users to move relevant body parts. The system includes a wide variety of games which span many types of motion-based activities. The therapy regimen is customized for each patient by the therapist. In general, the activities are clear and engaging, so that is easy and enjoyable for patients to follow consistently while doing therapy.
  3. Tablet computer runs the therapy software. The tablet is connected via the internet to a secure online cloud server, where the data is collected and analyzed. The built-in camera tracks patient movements, including supination, pronation, flexion, extension, adduction, and abduction of trunk or limbs. The computer vision processing follows up to two circular shaped objects attached to the patient's body parts being rehabilitated at a rate of around 20 frames-per-second. The objects tracked by the camera can be any circular object of at least one inch in diameter and colored red, green, or blue, such as a small ball, a circular sticker, etc.
  4. A therapist console is an online graphical dashboard which allows the therapist to design an appropriate therapy regimen as well as to review the progress of each session the patient performs. The therapy is personalized for each individual patient and their specific rehabilitation needs. Metrics such as range of motion, number of repetitions, and endurance are monitored so that the therapy can be modified based on how the patient is progressing.
Picture illustrating the architecture of the robotic system. The robotic system is on the left-hand side of the picture. The system is built as a white tower mounted to the top of a wheeled robotic base which sits on the ground. A computer tablet is attached to the top of the tower with an orange frame assembly. The tablet camera is pointed towards a man seated about four arm lengths away from the tower. The man is facing the camera while holding a red ball with his left hand. The height of the robotic tower is level with the eyes of the seated man. On the right-hand side of the picture there is an inset photograph of a pair of hands typing on the keyboard of a laptop computer. The laptop screen displays faintly a rendering of the therapy software console.
Figure 1. The robotic tele-rehabilitation system consists of a tablet computer mounted atop a robotic base. A therapist console, as shown on the right, is used to design the therapy session and monitor progress.

The robotic tele-rehabilitation system consists of a tablet computer mounted atop a robotic base. A therapist console, as shown on the right, is used to design the therapy session and monitor progress.


The working hypothesis in this project is that as a patient performs their therapeutic plan repeatedly over time their range of motion improves, and their endurance increases. While a full evaluation remains to be done, a pilot study was performed. In this study, two subjects conducted 3-minute therapy sessions over the course of several days. Both subjects performed the therapy sitting down. One utilized the right arm the other subject utilized the left arm. Rehabilitative metrics used focused on upper extremity movements and ROM. The Restore game Ski Saga was selected because it guides patients in performing motions which exercise and develop actions including sustained upper extremity movement with functional ROM, crossing the midline (horizontal adduction and horizontal abduction), and lateral reach. The data collected from these sessions is summarized by metrics including total number of movements, ROM coverage/position and frequency, number of times extremity crosses midline to opposite side and number of times player completed movements correctly and incorrectly. These metrics are shown in the output data screens generated by the system, examples of which are shown in Figure 2.


Figure displays two examples of the data screens produced by the teletherapy system. Data screen for Subject 1 is on the left. A similar screen is shown on the right for Subject 2. Each of the data screens follows the same template. The differences between the two screens are the actual maps and data elements. The screen on the left, corresponding to the data for Subject 1, who performed therapy on their Left Upper Extremity. The two salient features in this screen are the two maps shown within. The map on the left corresponds to the controller path, which represents the trace of where the left arm was within the reachable therapeutic workspace. In this case, Subject 1 was able to reach a large part of the left side of the map, corresponding to their left side of the body. However, the right-hand side of the map is mostly empty, implying that the Subject was not able to reach their right-hand side easily. To the right, there is a heat map, which corresponds to the number of times Subject 1 traversed various regions of the reachable therapeutic workspace. Blocks inside the heat map are shaded a darker blue corresponding to regions of the workspace which were traversed more often. Regions shaded lighter were not traversed as frequently. Regions which were not traversed at all are white. The screen to the right corresponds to the data for Subject 2. The map on the controller map graph is spread out across most of the workspace, except for the upper right quadrant, corresponding to the right-hand side of the body. The corresponding heat map has a few dark blue blocks towards the center of the map plus a few scattered ones to the right and one on the upper left.
Figure 2: Therapeutic maps for two subjects (Subject 1 data on the left, Subject 2 data on the right)

Therapeutic maps shown in Figure 2 illustrate patient performance and other relevant metrics. The maps indicate coverage of reachable workspace as well as endurance in performing specific motions repeatedly. In these maps, positions of body parts being rehabilitated are recorded as horizontal and vertical coordinates (x and y) as a function of time. The coordinates plotted correspond to the right, center (midline), and left sides of the body. The traces shown in the Controller Path maps span the functional ROM of one upper extremity across the therapeutic workspace. The Heat Maps indicate the number of times the patient's upper extremity traversed each specific position in the therapeutic workspace. The blocks become darker blue in proportion to the number of times those positions were traversed. Thus, the darkest blue blocks were traversed the most, lighter blue blocks were traversed less, and non-shaded areas were not traversed at all.


While the metrics being used in the project need to be evaluated more fully, preliminary indicators are promising. For example, from the controller path map in Figure 2, for Subject 1 on the left, the Left arm (LUEx) spanned a large part of the workspace on the left side of the body, but it did not reach upper right quadrant in the diagram. This could be interpreted as the subject having difficulty with crossing the midline due to some limitation in ROM, which the subject had. In such a case, for example, a therapeutic goal could be to help Subject 1 build endurance and to increase the range of motion towards that upper right quadrant of the therapeutic workspace.

More generally, following this line of analysis, improvements in range of motion would appear as gradual outward expansions of the traces in the controller path map as the sessions progress with time. Similarly, increased endurance would be indicated by a greater number of areas of the heatmap shaded darker. Additional metrics are being investigated as potentially applicable to identifying the desired patterns and trends in the data. These may include scatter data statistics, as well as other trend analysis or pattern analysis methods.


While a working prototype was built and tested, additional work remains to be done on aspects of the project.  On the hardware aspect, the functionality of the robotic system is being developed. Functions such as navigation, positioning, adjustment of height of screen, mobility and tele-presence are being implemented. The Create2 robotic base used in the prototype is limited in its programming and interfacing capabilities and does not demonstrate the full potential of the system. A more powerful and flexible robotic base is being considered for the second-generation system. Later generations of the system could be capable of autonomous navigation. More work remains to be done on the data analytics aspects of the system as well. Additional therapeutic metrics are being identified to help in tracking rehabilitative outcomes in patients. In related work, acceleration-based metrics have been proposed by Lang and are being explored for use here. [14] Future generations of this type of telerehabilitation system could be made self-adapting by incorporating artificial intelligence methods to monitor progress and automatically adjust the therapy as the patient's performance evolves.


Robot assisted telerehabilitation focuses on the use of robotics and data analytics to help patients reach their rehabilitative outcomes. A prototype of a system which engages and guides patients in performing their physical therapy with minimal or no human assistance was developed. Unlike other sensor based or humanoid robot based systems, the simplicity in the design of this system enable physiotherapists, clinicians, and families to provide therapy services at the patient's place of living. The impact of this type of robot-assisted telerehabilitation system is far reaching and addresses the following issues: the growing shortage of therapists and therapy services, inefficient access to therapy, poor engagement by patients, inconsistency among clinicians and need for objective assessment and tracking of rehabilitative progress. While further work remains to be done, the results of this project can have direct and sustained impact on society and the wellbeing of people.


  1. Fu,M. J., M. Y. Harley, T. Hisel, R. Busch, R. Wilson, J. Chae and J. S. Knutson, "Ability of people with post-stroke hemiplegia to self-administer FES-assisted hand therapy video games at home: An exploratory case series," J Rehabil Assist Technol Eng, vol. 6, pp. 2055668319854000, 2019.
  2. Feldman,U., C. N. Larsen, E. Parmonova and D. Theobald, "Innovations in medical robotics: Surgery, logistics, disinfection, and telepresence," in Patient-Centered Healthcare Technology: The Way to Better Health, L. Goldschmidt and R. M. Relova, Eds. IET Books, Forthcoming, pp. Chapter 7.
  3. Damiano,D. L., "Rehabilitative Therapies in Cerebral Palsy: The Good, the Not As Good, and the Possible," J. Child Neurol., vol. 24, (9), pp. 1200-1204, 2009.
  4. Zariffa,J., M. Myers, M. Coahran and R. H. Wang, "Smallest real differences for robotic measures of upper extremity function after stroke: Implications for tracking recovery," J Rehabil Assist Technol Eng, vol. 5, pp. 2055668318788036, 2018.
  5. O'Connor,M. and J. K. Davitt, "The Outcome and Assessment Information Set (OASIS): a review of validity and reliability," Home Health Care Serv. Q., vol. 31, (4), pp. 267-301, 2012.
  6. Fruth,S. J., Fundamentals of the Physical Therapy Examination : Patient Interview and Tests & Measures. 2014.
  7. Dinesen,B., G. Nielsen, J. J. Andreasen and H. Spindler, "Integration of Rehabilitation Activities Into Everyday Life Through Telerehabilitation: Qualitative Study of Cardiac Patients and Their Partners," J Med Internet Res, vol. 21, (4), pp. e13281, 2019.
  8. Shumway-Cook,A., S. Brauer and M. Woollacott, "Predicting the Probability for Falls in Community-Dwelling Older Adults Using the Timed Up & Go Test," Phys. Ther., vol. 80, (9), 2000.
  9. Mousavi Hondori,H. and M. Khademi, "A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation," Journal of Medical Engineering, vol. 2014, pp. 846514, 2014.
  10. Sobrepera,M. J., V. G. Lee and M. J. Johnson, "The Design of Lil'Flo, a Socially Assistive Robot for Upper Extremity Motor Assessment and Rehabilitation in the Community Via Telepresence," medRxiv, pp. 2020.04.07.20047696, 2020.
  11. Pulido,J. C., C. Suarez-Mejias, J. C. Gonzalez, A. Duenas Ruiz, P. Ferrand Ferri, M. E. Martinez Sahuquillo, Ruiz De Vargas, Carmen Echevarria, P. Infante-Cossio, C. L. Parra Calderon and F. Fernandez, "A Socially Assistive Robotic Platform for Upper-Limb Rehabilitation: A Longitudinal Study With Pediatric Patients," IEEE Robotics & Automation Magazine, vol. 26, (2), pp. 24-39, 2019.
  12. Assad-Uz-Zaman,M., M. Rasedul Islam, S. Miah and M. H. Rahman, "NAO robot for cooperative rehabilitation training," J Rehabil Assist Technol Eng, vol. 6, pp. 2055668319862151, 2019.
  13. Restore Skills Software. Available:, 2021.
  14. Doman,C. A., K. J. Waddell, R. R. Bailey, J. L. Moore and C. E. Lang, "Changes in Upper-Extremity Functional Capacity and Daily Performance During Outpatient Occupational Therapy for People With Stroke," American Journal of Occupational Therapy, vol. 70, (3), pp. p1-p11, 2016.