RESNA Annual Conference - 2019

HoVRS: Home-based Virtual Rehabilitation System

Qinyin Qiu, PhD, Amanda Cronce, Jigna Patel, PT, Gerard Fluet, DPT, PhD., Ashley Mont,
Alma Merians, PT, PhD, Sergei Adamovich, PhD.

Abstract

After stroke, sustained hand rehabilitation training is required for continuous improvement and maintenance of distal function. In this paper, we present a system designed and implemented in our lab:  the Home-based Virtual Rehabilitation System (HoVRS). Eleven subjects with chronic stroke were recruited to test the feasibility of the system and refine its design and the training protocol to prepare for a future efficacy study. HoVRS was placed at subjects’ homes, and subjects were asked to use the system at least 15 minutes every weekday for 3 months (12 weeks) with limited technical support and remote clinical monitoring.  All subjects completed the study without any adverse events.  Subjects on average spent 12 hours using the system. Nine out of the eleven subjects improved on the Box and Blocks Test (BBT), and ten improved on the Upper Extremity Fugl-Meyer Assessment (FM) and the Action Research Arm Test (ARAT). The outcomes of this pilot study warrant further investigation of the system’s ability to promote recovery of hand function in subacute and chronic stroke.

INTRODUCTION

Stroke is a leading cause of serious long-term disability in the United States.  The incidence of new or recurrent stroke in the US is 795,000 per year and the prevalence of chronic stroke is approximately 7 million (Go et al., 2014). Projections show that by 2030, an additional 3.4 million people or 3.88% of U.S. adults 18 and older will have had a stroke, a 20.5% increase from 2012 (American Stroke Association 2018). At six months post-stroke, about 65% of affected persons continue to have hand deficits that profoundly affect their ability to perform their usual activities and their independence (Dobkin, 2005; Lang, et al. 2006). This leaves a potential market segment of approximately 640,000 persons that may need long-term arm and hand rehabilitation. Restoration of hand function is of utmost importance since it is the loss of hand function that profoundly decreases quality of life by limiting the ability to perform feeding, dressing, and grooming, and further may limit the use of assistive as well as telecommunications technology (Brown et al. 1987, Grimby et al. 1998, Andren et al. 2004, Kwakkel et al. 2008).
Therapy in an inpatient rehabilitation center only lasts about 2-3 weeks. As outpatients, stroke survivors are typically only seen two to three times a week for short time periods. This volume of intervention falls far short of the hundreds of hours needed to re-establish normal hand function.  Recently published results of innovative lab-based interventions appear to have a similar problem (Lang et al., 2015, Winstein et al., 2016). It is therefore imperative to develop an intervention that can be delivered at home over a sufficient period of time to elicit improvements.
Innovative telerehabilitation systems have been developed using information and communication technologies to provide rehabilitation services at a distance. Many studies have developed video-game driven systems from commercially available gaming consoles such as Wii and Microsoft Kinect (Metcalf et. al, 2013), however, these systems do not address hand rehabilitation. Other groups, including members of our own team, have examined the use of custom-made telerehabilitation systems (Adamovich et. al, 2005, Turolla et. al., 2013) but they are not commercially available.  An ideal home-based telerehabilitation system has to be low cost, easy to setup, able to motivate the user for everyday use, generate progress reports for the user for self-tracking, and provide daily monitoring to remote clinicians. Exciting new technologies have now made this approach possible and hold promise for long-term benefit. These technological advances - for the first time - allow for virtual reality simulations interfaced with discrete finger and hand tracking that are affordable and easy to use.
Our product, the Home Virtual Rehabilitation System (HoVRS), provides intense upper extremity rehabilitation at home. It will allow patients to access hand/arm rehabilitation without the cost and transportation challenges associated with outpatient rehabilitation. HoVRS will consist of five elements:  1) an infrared camera specifically designed to capture finger and arm movements – a substantial improvement over rehabilitation activities provided by commercial game consoles like Kinect or Wii, 2) multiple engaging games that train the hand and arm using commercial gaming mechanics designed to optimize players’ motivation to perform these activities for long periods of time, 3) an optional exoskeleton designed to assist the patient’s arm as it moves against gravity (use of this support can be weaned and eliminated as patients get stronger),  4) monitoring and archiving software that will allow clinicians to design custom rehabilitation interventions, track a patient’s progress, and modify a patient’s rehabilitation program, in-person or remotely, and 5) a secure wireless data connector to collect detailed information on patient movement in real time. The secure communication channel will allow for remote monitoring by clinicians, remote technical support, and remote patient and clinician interaction face to face, while the patient uses HoVRS.
This study describes the experiences of the first eleven persons with stroke that participated in pilot testing of HoVRS in their homes.

system

NJIT HoVRS has two sub-systems to deliver home-based training: 1) a patient-based platform to provide the training and 2) a server based online data logging and reporting system. In the patient’s home, a cross platform virtual reality training application runs video games (developed in the Unity 3D game engine using the language C#) on their home computer. The Leap Motion Controller (LMC), an infrared tracking device, is used to capture motion of the hand and arm movement without requiring wearable sensors that may be difficult to put on independently or could potentially restrain movement. This allows the user to interact and control the games with their hand and arm.

A.  Hardware

The Leap Motion Controller (LMC) (Figure 1) is an $80 computer hardware sensor that captures detailed hand movement as well as hand gestures. The system consists of three infrared LEDs and two cameras. A data validation study showed the LMC to be accurate and reliable as long as the target is within its visual area (±250 mm of the LCM center) (Smeragliuolo et al., 2016). The device’s USB controller reads the sensor data into its own local memory and performs any necessary resolution adjustments. This data is then streamed via USB to the Leap Motion image API. From there, we programmed the system to feed tracking data into virtual reality activities by calling the Leap Motion API. If the subject is not strong enough to hold their hand over the Leap Motion Controller, the Armon™ Edero, a spring-based passive arm support was provided to the subject (Figure 1). Subjects were weaned from this support as they got stronger.

B.  Software

Subjects download games from the HoVRS website at the instruction of their therapist. After an initial configuration session, subjects initiate the system for subsequent sessions by choosing a simulation from a Graphic User Interface with a single mouse-click. Games initial levels are carried forward from their previous training sessions, eliminating the need for calibration at the start of each session.
Currently, twelve games have been developed.  Games can be grouped to 3 categories: Arm, Wrist and Hand.  Each simulation trains a specific movement pattern, such as wrist pronation or finger fractionation. A virtual therapist provides feedback, instruction and encouragement based on patient performance in all simulations. When game-play involves the control of a virtual object instead of a hand, there is a window that shows a virtual representation of the player’s hand as it appears above the LEAP camera.  There is a clinician configuration window that can be used to set up game conditions for variables such as work space size, activity speed, accuracy demands, etc.  

Arm games

Maze: Participant controls the movement of the virtual character with horizontal plane arm movements. They score by collecting spheres that they intercept along the maze path. The participant falls off the maze edge if they deviate from the defined path. As   the game progresses to higher levels; the platforms and bridges that the character must negotiate become narrower and turns become sharper and more frequent.  Later require the character jumping to jump over obstacles, which is accomplished by the player opening and closing their hand.
Whack-A-Mole: The player moves the hammer with horizontal plane arm movements. They swing the hammer using wrist pronation and supination movements. Players score points by whacking moles before they disappear.
Soccer Goalie: The player controls the position of the virtual goalie’s hands using horizontal plane movements to intercept approaching soccer balls traveling toward them. Goals score for all missed shots.

Wrist games

filler
Figure 1. HoVRS architecture design

Fruit Catching: To catch the fruit, the player controls the location of a collection basket through horizontal arm movement. To increase the score, the player uses forearm pronation/supination to drop the collected fruit into a second basket. The frequency of falling fruit decreases when the player misses 25% of the falling fruit.
Wrist Flying: Players score points by intercepting floating spheres with a plane. They control the vertical position of the plane by increasing/decreasing the pitch of their hand at the wrist.

filler
Figure 2: Daily game play duration

 3-D Wrist Flying: Players score points by intercepting floating spheres with a plane. The player controls yaw, pitch and roll of the plane with forearm lateral deviation, flexion/extension and pronation/supination.
Brick Breaking: There are two modes for the brick breaking game.  In the first mode, the paddle is controlled by the angle between the fingers and the palm as the players bend their fingers.  In the second mode, the paddle used to direct the ball follows the palm position of the player in the horizontal plane. The ball is directed by bouncing off the paddle and hitting the bricks above to clear the screen.  The difficulty of the game increases as the paddle gets smaller and multiple balls are introduced.

Finger games

Bowling: The player reaches forward and extends their fingers to apply force to the ball to knock over the pins.
Cars Game: The player control the car speed by opening and closing their hand (closing the hand slows the car down, opening speeds it up).  Speed control allows the subject to complete the course as fast as possible, which increases score, without crashing by going over speed bumps too quickly, which decreases score.  Later levels involve hand and wrist rotation to switch lanes and maneuver past introduced obstacles. 
Hand Flying:. Players score points by intercepting floating spheres with a plane. They control the vertical position of the plane by opening and closing their hand.
Piano: The player plays a song by pressing a specified key with a specified virtual finger. The amount of finger fractionation required to press a key increase as their ability to individually flex fingers improves. However, the control algorithm scales back down when they are unable to hit any key in at least 6 seconds.
Fruit Picking: The player must reach and use a pinch movement with their thumb and forefinger to pick apples and oranges from trees and sort them into the correct basket to increase the score.

methods

 Participants

filler
Figure 3. Average improvement in BBT, FM and ARAT.

Inclusion criteria were a) 40-80 years old, b) unilateral right or left sided stroke, c) score of 22 or greater on the Montreal Cognitive Assessment (Nasreddine et al., 2005), d) no hemispatial neglect or severe proprioceptive loss, e) Fugl-Meyer (FM) of  36-58/66, f) no receptive aphasia, g) intact cutaneous sensation (e.g. ability to detect <4.17N stimulation using Semmes-Weinstein nylon filaments. Exclusion criteria were a) orthopedic pathology limiting the ability to perform upper extremity movements without pain b) other central nervous system pathology.

Procedure

 HoVRS was placed in subjects’ homes for 3 months (12 weeks). Every weekday, subjects were encouraged to play at least 3 rehabilitation activities for 15 minutes, one from each of the three game categories. We did not schedule subjects to use the system at specific times or interact with them during every session in order to examine how often and how long subjects would use the system on their own.

Data acquisition

Table 1
Subject Age Gender Time Since stroke (years) Hemiplegic side
S1 63 Male 0.5 Right
S2 81 Female 1 Left
S3 67 Male 2.4 Right
S4 45 Female 16 Right
S5 55 Male 17 Right
S6 82 Male 7 Right
S7 56 Male 3 Right
S8 57 Male 1.5 Left
S9 66 Male 2.5 Left
S10 62 Male 5 Left
S11 47 Male 1 Left

Three clinical tests were conducted before and after the 3-month intervention: Box and Blocks Test (BBT), Action Research Arm Test (ARAT) and Fugl-Meyer Assessment (FM). In addition to the clinical tests, the number of hours that participants used the system was recorded and the number of required support sessions (in person and remote) were logged.

preliminary results

System usage time and support

Currently, eleven adults with chronic stroke were recruited via support groups and clinician referrals (Table 1). Average age was 62 ± 12 years old. Average time since stroke was 5 years. Two Female and six subjects were right side impaired. During the 12-week study, they each performed over 600 sessions using HoVRS in their homes. They required an average of 7 in-person support sessions and 5 remote support sessions. This group encountered 6 technical issues making it impossible for them to perform a session; 4 of these issues were resolved remotely and 2 required an in-person visit. 70% of these subjects used the system more than published reports using other systems.
Subjects on average spent 12 hours using the system. Average exercise time per day is 18.7 minutes, including 5 minutes in Cars game, 7.4 minutes in Maze Game and 6.3 minutes in PitchFly game (Figure 2).  Every day repetitions of hand open and close required during Cars game is 39, and wrist flexion/extension required during PitchFly game is 14.

Clinical results.

 After 3 months nine out of eleven subjects improved in BBT, ten out of eleven subjects improved in FM, and ARAT. In average, the whole group improved in BBT from 13.36 (SE = 1.14)  to 15.82 ( SE = 1.28), FM from 43.91 ( SE = 0.98) to 48.455 (SE = 0.89), and ARAT from 26.91 (SE= 1.21) to 31.46 (SE = 1.31) (Figure 3).

Discussion

  HoVRS is an affordable upper extremity rehabilitation system that provides whole arm activities including: 1) finger flexion/extension, 2) wrist flexion/extension as well as, 3) finger individuation. In addition, HoVRS trains 4) forearm rotation, 5) lateral wrist movement, 6) forward reaching, and 7) lateral reaching in combinations that mimic the way the arm and hand are used in the real world. We believe that the positive results on the FM and ARAT scores, which test more complex arm abilities, reflect the versatility of our system. This versatility also allows us to assist patients with arm problems that affect their ability to reach and forearm/wrist problems that affect their ability to rotate their forearms. In addition to rehabilitation activities, HoVRS also supports remote monitoring and system adjustment, which will significantly reduce the number of home therapy visits.

References

  1. Go, A. S., Mozaffarian, D., Roger, V. L., Benjamin, E. J., Berry, J. D., Blaha, M. J.,Turner, M. B. (2014). Executive summary: heart disease and stroke statistics--2014 update: a report from the American Heart Association. Circulation, 129(3), 399-410. doi: 10.1161/01.cir.0000442015.53336.12
  2. Dobkin, B. H. (2005). Clinical practice. Rehabilitation after stroke. N Engl J Med, 352(16), 1677-1684. doi: 10.1056/NEJMcp043511 Brown, M., Gordon, W. A., & Ragnarsson, K. (1987). Unhandicapping the disabled: what is possible? Arch Phys Med Rehabil, 68(4), 206-209.
  3. Lang, C. E., Lohse, K. R., & Birkenmeier, R. L. (2015). Dose and timing in neurorehabilitation: prescribing motor therapy after stroke. Curr Opin Neurol, 28(6), 549-555. doi: 10.1097/wco.0000000000000256
  4. Winstein, C. (2015). ICARE Primary Results: A Phase III Stroke Rehabilitation Trial. Paper presented at the International Stroke Conference, Nashville, Tennessee.
  5. Grimby, G., Andren, E., Daving, Y., & Wright, B. (1998). Dependence and perceived difficulty in daily activities in community-living stroke survivors 2 years after stroke: a study of instrumental structures. Stroke, 29(9), 1843-1849.
  6. Andren, E., & Grimby, G. (2004). Dependence in daily activities and life satisfaction in adult subjects with cerebral palsy or spina bifida: a follow-up study. Disabil Rehabil, 26(9), 528-536.
  7. Kwakkel, G., Kollen, B. J., & Krebs, H. I. (2008). Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair, 22(2), 111-121.
  8. Metcalf CD, Robinson R, Malpass AJ, (2013), Markerless motion capture and measurement of hand kinematics: validation and application to home-based upper limb rehabilitation. IEEE Trans Biomed Eng 2013; 60:2184–2192.
  9. Adamovich SV, Merians A, Boian R, Tremaine M, Burdea G, Recce M and Poizner H, (2005), Virtual Reality Based System for Hand Rehabilitation Post-Stroke, Presence, 14, 161-174.
  10. Turolla A, Daud Albasini QA, Oboe R, Agostini M, Tonin P, Paolucci S, Sandrini G, Venneri A, Piron L, (2013), Haptic-based neurorehabilitation in poststroke patients: a feasibility prospective multicentre trial for robotics hand rehabilitation. Comput Math Methods Med.
  11. Jurkiewicz MT, Marzolini S, Oh P. Adherence to a home-based exercise program for individuals after stroke. Topics in stroke rehabilitation. 2011;18(3):277-84.
  12. Miller KK, Porter RE, DeBaun-Sprague E, Van Puymbroeck M, Schmid AA. Exercise after stroke: patient adherence and beliefs after discharge from rehabilitation. Topics in stroke rehabilitation. 2017;24(2):142-8.
  13. Rimmer JH, Wang E, Smith D. Barriers associated with exercise and community access for individuals with stroke. Journal of rehabilitation research and development. 2008;45(2):315.
  14. Simpson LA, Eng JJ, Tawashy AE. Exercise perceptions among people with stroke: barriers and facilitators to participation. International journal of therapy and rehabilitation. 2011;18(9):520.
  15. Mihelj M, Novak D, Milavec M, Ziherl J, Olenšek A, Munih M. Virtual rehabilitation environment using principles of intrinsic motivation and game design. Presence: Teleoperators and Virtual Environments. 2012;21(1):1-15.
  16. Fitzgerald D, Trakarnratanakul N, Smyth B, Caulfield B. Effects of a wobble board-based therapeutic exergaming system for balance training on dynamic postural stability and intrinsic motivation levels. journal of orthopaedic & sports physical therapy. 2010;40(1):11-9.
  17. Lang CE, MacDonald JR, Reisman DS, Boyd L, Kimberley TJ, Schindler-Ivens SM, Hornby TG, Ross SA, Scheets PL. Observation of amounts of movement practice provided during stroke rehabilitation. Archives of physical medicine and rehabilitation. 2009;90(10):1692-8.
  18. Rand D, Givon N, Weingarden H, Nota A, Zeilig G. Eliciting Upper Extremity Purposeful Movements Using Video Games A Comparison With Traditional Therapy for Stroke Rehabilitation. Neurorehabilitation and neural repair. 2014;28(8):733-9.
  19. Peters DM, McPherson AK, Fletcher B, McClenaghan BA, Fritz SL. Counting repetitions: an observational study of video game play in people with chronic poststroke hemiparesis. Journal of Neurologic Physical Therapy. 2013;37(3):105-11.
  20. Smeragliuolo AH, Hill NJ, Disla L, Putrino D, Validation of the Leap Motion Controller using markered motion capture technology. Journal of biomechanics. 2016 Jun 14;49(9):1742-1750.
  21. Nasreddine, ZS, Phillips, NA, Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al., "The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment," Journal of the American Geriatrics Society, vol. 53, pp. 695-699, 2005.
  22. Adamovich SV, Fluet GG, Mathai A, Qiu Q, Lewis J, Merians AS. Design of a complex virtual reality simulation to train finger motion for persons with hemiparesis: a proof of concept study. Journal of neuroengineering and rehabilitation. 2009;6(1):28.
  23. da Silva Cameirão M, Bermúdez i Badia S, Duarte E, Verschure PF. Virtual reality based rehabilitation speeds up functional recovery of the upper extremities after stroke: a randomized controlled pilot study in the acute phase of stroke using the rehabilitation gaming system. Restorative neurology and neuroscience. 2011;29(5):287-98.
  24. Shirzad N, Van der Loos HM, editors. Adaptation of task difficulty in rehabilitation exercises based on the user's motor performance and physiological responses. Rehabilitation Robotics (ICORR), 2013 IEEE International Conference on; 2013: IEEE.
  25. Ram A, Leake D. Learning, goals, and learning goals. Goal-driven learning. 1995:1-37.
  26. Bostan B. Player motivations: A psychological perspective. Computers in Entertainment (CIE). 2009;7(2):22.
  27. Pea RD. The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity. The journal of the learning sciences. 2004;13(3):423-51.
  28. Landers RN, Bauer KN, Callan RC, Armstrong MB. Psychological theory and the gamification of learning.  Gamification in education and business: Springer; 2015. p. 165-86.
  29. Fluet, G. G., Merians, A. S., Qiu, Q., Davidow, A., & Adamovich, S. V. (2014). Comparing integrated training of the hand and arm with isolated training of the same effectors in persons with stroke using haptically rendered virtual environments, a randomized clinical trial. J Neuroeng Rehabil, 11, 126. doi: 10.1186/1743-0003-11-126.
  30. Qiu Q, Cronce A, Fluet GG, Patel J, Merians AS, Adamovich SV. (2016a). Home Based Virtual Rehabilitation for upper extremity functionality recovery post stroke. International Conference on Disability Virtual Reality and Assistive Technology September 2016, Los Angeles CA
  31. Krueger R. A. (1994). Focus groups: A practical guide for applied research (2nd ed.). Thousand Oaks, CA :Sage.

Acknowledgment

The development and ongoing clinical evaluation of HoVRS is supported by the Rehabilitation Engineering Research Center on Wearable Robotics funded by NIDILRR Grant #90RE5021.

Audio Version PDF Version