Wheelchair Propulsion Monitoring Before And After An Intervention: An Exploratory Study

Pin-Wei B. Chen¹, Kerri Morgan¹

¹Program in Occupational Therapy, Washington University in St. Louis

INTRODUCTION

Upper limb injuries are common among manual wheelchair users. After decades of wheelchair propulsion (WP) research, clinical practice guidelines (CPGs) were established to enable clinicians to teach wheelchair users how to propel their wheelchairs with optimal biomechanical efficiency. [1–5] These guidelines were designed to prevent upper limb injuries. WP patterns can be grouped into four categories: arc, single-loop over, double-loop over, and semicircular. The CPGs suggest using the semicircular propulsion pattern due to its biomechanical efficiency. [5] Researchers have examined WP usage in the laboratory setting; however, there is a lack of evidence regarding how users propel their wheelchairs in the real world. Tracking daily, real-world WP is technologically challenging without in-person monitoring, which may impose the Hawthorne effect, resulting in changes in propulsion behaviors. Prior research using wearable sensors (e.g., smart watches) and machine learning (ML) to detect propulsion techniques has shown promising results in a controlled laboratory setting. [6–8] Our previous pilot study using a support vector machine (SVM) and inertial measurement units (IMU) showed encouraging results. [9] To further build on this method, for this study, we collected data on a WP intervention wherein wheelchair users’ propulsion patterns before and after training based on the CPGs were measured with IMU sensors. Specifically, we thought that, due to variation among participants, a supervised, participant-specific ML model is needed in order to build a sufficient ML model for predictions. The novelty of this study is that, unlike previous research, we measure naturalistic changes before and after an intervention rather than performance of certain propulsion patterns. We aimed to explore three hypotheses: (1) ML models built from indoor propulsion patterns can predict outdoor propulsion patterns, (2) ML models built from indoor and outdoor propulsion patterns can detect indoor and outdoor propulsion patterns, and (3) a generalized ML model can be built by grouping all participants’ data to predict a new participant’s propulsion.

Table 1. Participant demographic information.
Participant	1	3	6	8
Height	5'6	5'10	6'1	5'10'
Weight	193.7 lbs.	196 lbs.	149.8 lbs.	146.6 lbs.
Age	50	47	58	34
Sex	Male	Male	Male	Male
Race/ethnicity	White	Asian	White	White
Upper limb pain	No	Yes	Yes	Yes
Level of injury	T8	T12	T9	T4
Injury	Complete	Incomplete	Incomplete	Incomplete
Years since injury	15 years	2.5 years	5 years	9 years
ASIA score	A	A	C	C
Average max speed	2.78 m/s	2.36 m/s	2.15 m/s	1.98 m/s
Average regular speed	1.18 m/s	1.22 m/s	1.07 m/s	0.86 m/s

METHODS

Design

The study was part of a randomized control trial with two groups: Education Group (EG) and Training Group (TG). All data tested in the current study were collected from the TG because there were no changes in propulsion pattern in the EG, and, therefore, there were no differences in movement patterns to be detected and developed in ML classifications. Both groups received three assessment sessions that were conducted before, immediately after, and three weeks after the intervention. The assessment consisted of various indoor and outdoor WP assessments, as well as collection of demographic information. Both groups received basic education regarding CPG-recommended biomechanically efficient WP techniques using video [10] and instruction materials [11]. The TG received additional six one-hour trainings.

Participants

Data were collected from four manual wheelchair users enrolled in the TG who had experienced spinal cord injuries. Table 1 shows the participants’ demographic information.

Equipment

Two ActiGraph GT9x (ActiGraph, Pensacola, FL) sensors were used to record acceleration and rotation at 100 Hz. A GoPro Hero was used to capture outdoor propulsion assessments and training sessions. The Wheelmill System (WMS), a motorized wheelchair treadmill designed in our lab, was used during the assessments. [12]

Procedure

Participants were recruited and screened over the phone and in-person to ensure that they met the inclusion and exclusion criteria. Only participants who displayed propulsion patterns that did not follow CPGs were enrolled in this study. For the first assessment, participants performed three trials of over-the-ground (OTG) propulsion on a 10-meter line at regular speed and then push wheelchair for three minutes on the WMS with the same regular speed measured during the OTG trials. Participants also performed outdoor propulsion in the parking lot outside of the lab for two to three minutes across approximately 200 meters. The parking lot consists of 5°–10° slopes, a flat surface with small potholes, and two thresholds. Participants were told to propel their wheelchairs at their regular speed. Throughout the assessment session, participants wore two sensors on their left arm: one at the lateral epicondyle and one at the wrist. We expected participants to propel using arc or single-loop over patterns during the first assessment.

After the first assessment, participants were scheduled for an hour of education. We used dynamic balance randomization with a random walk of up to one participant in group difference per randomization condition. The condition considered are level of injury, years since injury and gender. Half of the participants were allocated to the TG, who visited the lab six times for propulsion training with a skilled trainer. The other half of participants were allocated to the EG and received no propulsion training.

The second assessment took place approximately three weeks after the first assessment. The third assessment occurred three weeks after the second assessment. The measurements performed in the second and third assessments were the same as those in the first assessment. We expected that participants in the TG would learn from their training and push in a semicircular pattern, in accordance with the CPGs.

Data Processing and Analysis

Table 2. Statistical measures of each participant-specific SVM model evaluated with outdoor propulsion patterns
Participant	1	3	6	8
Sensitivity	0.556	0.000	0.943	0.976
Specificity	0.882	0.982	0.575	0.500
Precision	0.313	0.000	0.670	0.990
Recall	0.556	0.000	0.943	0.976
F1	0.400	0.000	0.783	0.983

All data analysis was performed with R version 3.3.0. The Caret package of R was used in building the ML algorithms. [13] Research Electronic Data Capture (REDCap) was used to store data for this investigation. Acceleration and rotation data were collected during the OTG and WMS sessions, as well as the outdoor propulsion sessions. Approximately six minutes of outdoor propulsion data, 30 seconds of indoor propulsion data, and nine minutes of WMS data were collected per participant. Observable errors were removed. The sequence of processing steps was as follows: (1) data organization and filtering, (2) data preprocessing, (3) data segmentation, (4) feature selection, (5) ML training, (6) ML evaluation, and (7) statistical analysis. [14] Steps 2–6 were repeated to retrain the ML by re-segmenting data, reselecting features, and rebuilding the ML models with different tuning. An SVM with linear kernel was used in this current report.

Feature variables, which were used in the ML algorithm, were created with each epoch (i.e., one-second data window) as one data point. Feature variables were generated from each axis of data (i.e., x, y, and z), as well as the first-order derivative and the magnitude of the three axes. The time-domain features (e.g., standard deviation, average, medium crossing) and frequency-domain features (e.g., dominant frequency) were calculated. ActiLife Feature Extraction tool was used to create one-second epochs for acceleration data. The feature of rotation data was created by an in-house algorithm to generate the above features. Once feature variables were generated, data were merged into a w by d’ feature space matrix in which w was the number of window size extracted and d’ was the number of features. Collinearity was removed.

All features were labeled by a trained staff member by reviewing the video recordings with BORIS v.7.4.6. [14] Data that were not clear to the observer were removed from the analysis. Data that did not contain WP movements were also removed from the analysis. Three sets of evaluations were performed: (1) participant-specific models were built using the indoor propulsion data with linear SVM and evaluated with outdoor propulsion data, (2) participant-specific models were built from all indoor and outdoor propulsions with linear SVM and evaluated with ten-fold cross-validation (CV), and (3) generalized models were built and evaluated with the leave-one-participant-out method. [9] The statistical measures of algorithm performance were calculated. The F1 measure was calculated with the following formula [17]:

F1 score = \frac{2 \times True Positive}{All True + All Positive}

ML model built with indoor propulsion pattern data, then evaluated using outdoor propulsion data.

F1 score equals to two times true positive divided by all true plus all positive. Because the F1 score is the harmonic mean of precision and recall, it is a comparative score that has been used throughout ML literature.

**Table 3. Statistical measures of each participant-specific SVM model with ten-folds CV**
Participant	1	3	6	8
Sensitivity	0.938	0.930	0.966	0.928
Specificity	0.958	0.939	0.973	0.891
Precision	0.936	0.928	0.972	0.933
Recall	0.938	0.930	0.966	0.928
F1	0.937	0.929	0.969	0.930
ML model built with all data—indoor, outdoor, and WMS—then evaluated with ten-fold CV.

RESULTS

**Table 4. Statistical measures of generalized SVM model evaluated with leave-one-participant-out method**
General Model
Sensitivity	0.327
Specificity	0.461
Precision	0.370
Recall	0.327
F1	0.347
ML model built with all data—indoor, outdoor and WMS.

No statistical significance test was compared. Table 2 shows statistics of predicting outdoor propulsion patterns with ML models built on indoor propulsion data. Across all participants, the sensitivity and specificity vary greatly. The F1 score indicated that most of these models predicted poorly. Table 3 shows statistics of the ten-fold CV of four participant-specific ML models built and evaluated using the data collected indoors, outdoors, and on the WMS and tested against data collected indoors, outdoors, and on the WMS. Sensitivity and specificity were high. All participant-specific models performed very well, with the lowest F1 score at 0.929. Table 4 shows statistics of the generalized model evaluated with four-fold CV of the leave-one-participant-out method. This F1 score was very low compared to participant-specific models.

DISCUSSION

Chronic overuse of the upper limbs during WP is one of the factors associated with upper limb pain and injury. Most existing research conducted in a laboratory setting. Our hope is to establish further evidence to push forward advancements in detecting WP patterns in real-world settings so that wheelchair usage can be passively monitored and to further improve our evidence in understanding chronic overuse injuries.

The results of this study show that, with the linear SVM method, environmental and participant variations significantly affect the model’s accuracy. Our previous pilot study shows that it is feasible to build ML models from WP performed on the WMS and to predict OTG propulsion with high accuracy. [9] In this study, we hoped to see that models built from data recorded indoors could be used to predict outdoor propulsion regardless of environmental effects since the propulsion trainings are mostly conducted indoors, but results show that, with the current method, this is not possible. However, if propulsion data are collected from indoors, outdoors, and on the WMS, it is feasible to predict both indoor and outdoor propulsion with high accuracy. This is demonstrated by the results of the participant-specific ML models evaluated with ten-fold CV.

Prior ML research in motion analysis has shown that it is feasible to generalize an ML model by training models across multiple participants. [8] This does not apply to our results, wherein we found that the generalized model evaluated with the leave-one-participant-out method did not perform well. This provides further evidence that participant-specific modeling may be the best way to accurately detect WP patterns in real-world scenarios. Intuitively, the explanation for this is that participant-specific ML models do not suffer from participant variability in predictions.

Limitations

This research was an exploratory study that looked at only a few participants. It is possible that a generalized ML model can be built if enough data are collected. It is also possible that different ML methods can yield better results and that we were limited by our ML techniques. This study also only tested ML built from a nearby area around the lab. The real-world environment is much more diverse, and it is possible that our current ML model will not be able to handle the changes of variation in movement patterns in other settings. However, we believe that this study provides a step forward in understanding what factors are important.

CONCLUSIONS

Results suggest that it is important to include behavioral changes due to environmental variables when tuning a proper ML algorithm. Participant variations may reduce model predictabilities. All of these factors need to be considered in the next step of building better ML models. In the future, we will test different ML algorithms to further our development of post-rehabilitation WP monitoring system

REFERENCES

Impink BG, Boninger ML, Walker H, et al. Ultrasonographic median nerve changes after a wheelchair sporting event. Arch Phys Med Rehabil. 2009; 90:1489–1494.
Boninger ML, Koontz AM, Sisto SA, et al. Pushrim biomechanics and injury prevention in spinal cord injury: recommendations based on CULP-SCI investigations. J Rehabil Research Dev. 2004; 42:9. DOI:10.1682/jrrd.2004.08.0103.
Shimada SD, Robertson RN, Bonninger ML, Cooper RA. Kinematic characterization of wheelchair propulsion. J Rehabil Res Dev. 1998; 35.
Boninger ML, Souza AL, Cooper RA, et al. Propulsion patterns and pushrim biomechanics in manual wheelchair propulsion. Arch Phys Med Rehabil. 2002; 83:718–723. DOI: 10.1053/apmr.2002.32455.
Paralyzed Veterans of America Consortium for Spinal Cord Medicine. Preservation of upper limb function following spinal cord injury: a clinical practice guideline for health-care professionals. J Spinal Cord Med. 2005; 28.
French B, Smailagic A, Siewiorek D, et al. Classifying wheelchair propulsion patterns with a wrist mounted accelerometer. In: Proceedings of the ICST 3rd International Conference on Body Area Networks (13–15). Tempe, Arizona: ICST; March 2008.
Garcia-Masso X, Serra-Ano P, Gonzalez LM, et al. Identifying physical activity type in manual wheelchair users with spinal cord injury by means of accelerometers. Spinal Cord. 2015; 53:772–777.
Hiremath SV, Ding D, Farringdon J, et al. Physical activity classification utilizing SenseWear activity monitor in manual wheelchair users with spinal cord injury. Spinal Cord. 2013; 51:705–709.
Chen PW, Morgan K. Toward community-based wheelchair evaluation with machine learning methods. J Rehabil Assist Technol Eng. 2018; 5:1–9.
Rice, I. (2011) Propulsion Powerpoint. [online] Herl.pitt.edu. Available at: http://www.herl.pitt.edu/propulsion/ [Accessed 03 Jan. 2018].
Rice LA, Rice IM. Evidenced based education interventions to preserve upper limb function among full time manual wheelchair users. Medical Research Archives. 2017;5(3).
Klaesner J, Morgan KA, Gray DB. The development of an instrumented wheelchair propulsion testing and training device. Assist Technol. 2014; 26:24–32.
Kuhn, M. Caret package. J Stat Softw. 2008; 28:5.
Friard O, Gamba M. BORIS: a free, versatile open‐source event‐logging software for video/audio coding and live observations. Methods Ecol Evol. 2016; 7(11):1325–1330.
Lara OD, Labrador MA. A survey on human activity recognition using wearable sensors. IEEE Communications Surveys and Tutorials, 2013; 15: 1192–1209.
Chinchor N. MUC-4 evaluation metrics. In: Proceedings of the 4th Conference on Message Understanding (22–29). McLean, VA: Association for Computational Linguistics; 1992.

Audio Version PDF Version

RESNA Annual Conference - 2019