INTRODUCTION

Cerebrovascular accident or stroke affects approximately 800,000 people in the United States each year and due to its chronic effects, it is one of the leading causes of serious disability [1]. Longitudinal studies show that 30 – 66% of individuals with hemiplegia fail to regain arm function six months post stroke while only 5 to 20% regain full recovery [2]. Hand and arm movement is one of the most important abilities to regain after a stroke, but it is also one of the most challenging. Post-stroke motor function recovery requires a high dose of recurrent and targeted upper extremity training [3]. The current healthcare model in the United States provides short inpatient rehabilitation stays with the focus on mobility leaving the hand and arm rehabilitation focus to come later during outpatient therapy. However, socioeconomic factors such as cost, geographic location, or transportation create challenges for many individuals preventing them from receiving the amount of rehabilitation they need to recover their hand and arm function. In order to make stroke care more accessible, we are developing a solution that is easy to use and affordable. The beta prototype that we have developed, R3THA, builds on the original innovation, the Home based Virtual Rehabilitation System, created at the New Jersey Institute of Technology [4].

R3THA provides individuals with stroke and clinicians with an engaging, versatile, and customizable upper extremity telerehabilitation system that is easy to use, portable, and affordable. Therapy for the hand and arm is delivered through engaging exergames. R3THA's structure is made up of a user station, a HIPAA compliant cloud data platform, and a web portal. The user station includes: a library of exergames that target specific functional hand and arm movements, an infrared camera used to capture finger and arm movements, and an optional arm support to relieve shoulder strain.

The following developments were made to the system: increasing the complexity of the exergames and algorithms, strengthening the kinematic assessments, and improving the user experience for the therapists to access patient data and system adjustments. We enhanced the adaptive game algorithms to increase the long-term playability of the system, improve player motivation and adherence, and have games adapt to players' performance for dynamic challenge levels. Our kinematic assessments measure range of motion, accuracy of hand open/close, wrist extension/flexion, and forearm supination/pronation to give therapists a baseline of ability from their patient. An arm support was designed which allows for standardization of these kinematic assessments. A HIPAA compliant therapist portal was optimized for therapists to log in and access patients' information, rehabilitation plan, and generated progress reports. The therapist can register and log in to the web portal and create an online patient profile. Additionally, the therapist can create a rehabilitation plan based on rehabilitation goals and patients' impairment levels. Once the patient plays the games, the portal can display a patient's progress report. Upon completion of these enhancements, it was imperative to evaluate the usability of the system.

This study aimed to have physical therapists, occupation therapists, and individuals with stroke evaluate the usability of the enhanced system. To determine how the system meets the needs of therapists and individuals with stroke, we performed a usability study with the primary outcome measure, a System Usability Survey (SUS). SUS is a highly reliable questionnaire and has become popular for end-of-test subjective assessments of usability. This survey score demonstrates the overall perceived usability of the system.

METHODS

All procedures performed in studies involving human participants were in accordance with the ethical standards of the Institutional Review Board of the New Jersey Institute of Technology.

Participants

We enrolled 8 licensed therapists and 8 individuals with chronic stroke (> 6 months post stroke). The inclusion criteria for therapists were the following: a) between the ages of 25 and 70 and b) licensed physical or occupational therapist. For individuals with stroke, the inclusion criteria were a) between the ages of 20 and 80, b) enough shoulder, elbow, wrist, and hand movement to actively interact with the video exergames. The exclusion criteria included the following: a) severe arm weakness, b) severe increase in tone, c) difficulty following instructions or paying attention to computer video exergames for at least 10 minutes, and d) visual problems that make it difficult for them to interact with an entire computer screen.

Procedure

The study consisted of four separate sessions for the therapists and three sessions for the individuals with stroke. During Session 1, a team member trained the therapists on how to use the system. They learned how to use the web portal, including signing in, creating/adding a patient, choosing a rehabilitation plan, and selecting exergames to prescribe. The rehabilitation plan includes selecting the following focus areas: range of motion, coordination, and motor control. These parameters correspond to features in the exergames, which focused on hand, wrist, and shoulder in this study.

In addition to this, they were trained on how to use the system launcher, which allows video conferencing with their patient. They were trained on how to calibrate the system and perform remote kinematic assessments. Three exergames focusing on hand, wrist, and shoulder movement were demonstrated so that the therapists could play the exergames first to understand the rehabilitation goals of each game. Finally, they reviewed the progress reports and gave feedback on what they felt was important to display in real time.  

For Session 2, the therapist and individual with stroke were seated in the same room. This allowed therapists to help set up the physical system, including a laptop, infrared motion capture camera, arm support, and video conferencing camera for the individual with stroke (Figure 1). The therapists controlled the individual with stroke's system launcher via a remote desktop application. They then guided the individual with stroke through the calibration, the remote kinematic assessments, and finally, three exergames.  During Sessions 3 and 4, the therapist and individual with stroke were seated in separate rooms to simulate the remote nature of the system. They communicated via video conferencing software which allowed them to see each other and the therapists to see the individual with stroke's hand and arm during the session. They repeated the steps in Session 2. Upon completion of Session 4, both participants filled out a survey. 

Analysis

The individual analysis for each participant's SUS score was performed, and then the median and mean values for the two groups were evaluated. The ten statements on the survey alternate between positive and negative, therefore each item's contribution (between 0 – 4) must be calculated. Statements can be seen in Table 1. For items 1, 3, 5, 7, and 9, the score contribution is the scale position minus 1. For items 2, 4, 6, 8, and 10, the contribution is 5 minus the scale position. Each item's score contributions are then summed and multiplied by 2.5 to achieve the final score, which ranges from 0 – 100 [5]. A higher score is associated with greater usability. We performed 3 separate SUS calculations based on data from: 1) all the participants, 2) only therapists, and 3) only individuals with stroke.

Table 1: System Usability Survey score contribution of individual items

SUS Analysis Item	Total Score	Therapist   Score	Stroke Score
I think that I would like to use this system frequently	4	3	3
I found the system unnecessarily complex	4	4	4
I thought the system was easy to use	3	4	2
I think that I would need the support of a technical person to be able to use this system	3	3	3
I found the various functions in this system were well  integrated	3	3	3
I thought there was too much inconsistency in this system	3	4	3
I would imagine that most people would learn to use this system very quickly	3	3	2
I found the system very cumbersome to use	3	4	3
I felt very confident using the system	3	3	3
I needed to learn a lot of things before I could get going with this system	3	4	3

RESULTS

This survey score demonstrates the overall perceived usability of the system. It is widely accepted that a score of 68 or more means that the device is considered acceptable to use [6]. For our usability study, the beta prototype was rated high. The survey score from all participants was 81.8, from only therapists it was 83, and the score for only individuals with stroke was 80, which all corresponds to a system with good usability. We performed further analysis to see the score contribution of individual items for the 3 groups (Table 1).

DISCUSSION

Overall, the perceived usability of R3THA as scored by therapists and individuals with chronic stroke was high, as the scores were all above 80. The therapists provided additional feedback that the web portal fit their needs, and they felt that it could be incorporated into their treatment regimen.

According to Bangor et al., scores in the 70s and 80s do not always translate to acceptance in the field. Therefore, future studies will be performed in the outpatient clinic to assess the usefulness in a less controlled environment [7]. As the scores were in the 80s, there is still room for improvement on the prototype. To further investigate this, individual analysis items that fell below a score of 3, were deemed as areas for improvement for the next prototype iteration. Two individuals with stroke scored the following prompts a 2: "I thought the system was easy to use", and "I would imagine that most people would learn to use this system very quickly". It is important to note that the improvements made to the web portal and launcher were designed with a focus on the therapist's point of view. These stroke subjects were more severely impaired, and it was difficult for them to interact with the physical system, especially the optional arm support. Some games require the use of the arm support while others do not. Investigations will be done to evaluate the ergonomics of the system because the positioning of the arm support on the table and the individual's height play a large role in comfort. One way to improve this will be to create a manual with guidance on when to use the arm support and where to place it based on height.

Two other similar remote hand and arm rehabilitation systems for individuals post stroke used the SUS as an evaluation [8,9]. In longitudinal studies, these systems were placed in the homes of individuals with stroke, and the individuals played rehabilitation games for 6 weeks. The MERLIN system scored a 77, and the SCRIPT Program scored a 69.  R3THA scored higher on the SUS compared to both systems. We anticipate that the trend will continue when we perform longitudinal studies with R3THA.

CONCLUSIONS

R3THA has a high acceptability rate as measured by the SUS for all three data sets, showing promise as an affordable and accessible rehabilitation tool for the hand and arm. More research and development work will be performed to ensure usefulness and ease of use in the future for clinicians as well as individuals with disabilities.