RESNA Annual Conference - 2019

Interrater Reliability Of The Wheelchair Components Questionnaire Between Multicultural Wheelchair Professionals

Ethan D. McMichael1, Kris Riseling2, Karen Rispin 1,3

1LeTourneau University, 2Queen’s University, 3Assistive Technology Catalyst Project

INTRODUCTION

Maintaining good wheelchair condition is key to user mobility and safety. Component wear and failure can directly lead to serious injury for wheelchair users.[1] Allowing wheelchairs to deteriorate without repair significantly increases the user’s likelihood for injury.[1-3] For example, poor seating with damaged support can cause pressure sores.[2] Therefore, ensuring that each component on the chair is fully functional is vital.[1]

WCQ questions are composed of a visual analogue scale response with school grades as anchors and a line for explanatory comments.
Figure. 1. Example question from the WCQ
Wheelchair durability is especially crucial in low-resource environments (LRE) where the demands of daily life challenge the maintenance condition of wheelchair components.[5] Wheelchairs tend to wear more quickly when operating on rough ground and when users push them beyond their design capabilities.[5] Unfortunately LREs also often have a shortage of wheelchair services.[6] The few clinics that are available have limited resources and personnel and are often unable to meet local need for wheelchair maintenance.[6] A brief outcome measure that focuses specifically on wheelchair condition would enable evidence-based response to these challenges.[8]

The Wheelchair Components Questionnaire for Condition (WCQ) allows wheelchair professionals to quickly evaluate wheelchair condition.[7,8] The questionnaire has eight questions, each made up two parts: a visual analog scale and a line for explanatory comments. (See Figure 1.) The scale provides a quantitative rating for the wheelchair component’s condition expressed as a single value.[7,8] Comments provide qualitative data that explain the reason behind the quantitative scores.[7] Each question focuses on a single section of the wheelchair and is condition-specific; a poorly designed or undesired component in good condition should still receive a high score. The entire questionnaire takes less that 15 minutes to complete yet results provide an accurate snapshot of wheelchair condition without long-term tracking.[7,8]

Several previous studies support test-retest and initial interrater reliability of the WCQ.[4,8] However, the initial interrater reliability study only included two evaluators, both from North America. The goal of this study is to investigate the interrater reliability of the WCQ between a larger group of raters from several cultural backgrounds. An intraclass correlation coefficient greater than 0.75 would support good interrater reliability of the WCQ between multicultural users.[9]

METHODS

Interrater reliability testing compares raters’ results evaluating the same set of wheelchairs. Participants evaluating the wheelchairs were wheelchair professionals with training and experience in wheelchair provision. In addition, because the WCQ is intended for use in international LREs, this study sought raters with diverse cultural backgrounds. The WCQ is currently only in English so participants were also fluent in English. Wheelchairs were to have undergone noticeable wear in an LRE.

The study protocol was approved by the authors’ university and by partner organizations at the study site. The wheelchair users and their guardians gave assent and consent for the study.

IBM’s Statistical Package for the Social Sciences (SPSS) was used to calculate the interclass correlation coefficient between raters using two-way random effects.

RESULTS

Seven wheelchair professionals participated in this study, each with at least seven years of wheelchair experience. Three countries were represented: Canada, Kenya and the United States. See Table 1. for participant details. All raters used the WCQ to evaluate the same set of 12 wheelchairs belonging to children at a residential school for children with disabilities. The types of wheelchairs this study evaluated were designed for use in LREs and included Wheelchair for Kids, Motigo, Motivation active folding, Association of the Physically Disabled of Kenya Tumaini wheelchair and the Whirlwind Rough Rider.

Table 1. Participating wheelchair professionals’ details

Nationality
Qualifications Wheelchair Experience (years)
American Masters in Occupational Therapy, Assistive Technology Professional 7
American Doctor of Physical Therapy 8
Canadian Masters in Physical Therapy 7
Kenyan Certificate in Physical Therapy 8
Kenyan Orthopedic Technologist 8
Kenyan Orthopedic Technologist 8
Kenyan Orthopedic Technologist 8

Analysis indicated an intraclass correlation of 0.867 between raters with a 95% confidence interval from a lower bond of 0.707 to an upper bound of 0.955.

DISCUSSION

Results support the WCQ’s interrater reliability between multicultural users. The correlation coefficient was well above 0.75 indicating good reliability. Even the lower bound of the 95% confidence interval was 0.707 predicting that future studies, even with the most variance from this study, would still have an intraclass correlation above 0.7.

The WCQ could mitigate the problem of limited resources in clinical settings by highlighting typical maintenance problems. Results would provide a snapshot of a wheelchair’s condition and highlight which chairs are in need of the most maintenance. In addition to informing practical clinical decisions, results from larger studies utilizing the WCQ could assist manufacturers with design decisions.[7] By illuminating which wheelchair components commonly fail in a given location, the WCQ can point to needed design changes or special focus on maintenance.

Each rater had at least seven years of wheelchair experience prior to this study. Raters’ common acquaintance with the specific wheelchairs may have impacted their ratings and affected interrater reliability. The process of manually recording visual analog data and comments for each questionnaire in a spreadsheet is time-consuming. Future studies should verify reliability of the WCQ using electronic forms for data collection. In order to broaden the WCQ’s availability, translations to other languages would beneficial.

CONCLUSION

This study supports the reliability of the WCQ for use in LREs by wheelchair professionals of various cultural backgrounds.

REFERENCES

[1] Worobey, L., Oyster, M., Nemunaitis, G., Cooper, R. & Boninger M.L. (2012). Increases in Wheelchair Breakdowns, Repairs, and Adverse Consequences for People with Traumatic Spinal Cord Injury. American Journal of Physical Medicine & Rehabilitation, 91(6), 463-469.

[2] Hansen, R, Tresse, S. & Gunnarsson, R.K. (2004). Fewer accidents and better maintenance with active wheelchair check-ups: a randomized controlled clinical trial. Clinical Rehabilitation, 18, 631-639.

[3] Toro M.L., Garcia, Y., Ojeda, A.M., Dausey, D.J. & Pearlman, J. (2012). Quantitative Exploratory Evaluation of the Frequency, Causes and Consequences of Rehabilitation Wheelchair Breakdowns Delivered at a Paediatric Clinic in Mexico. Disability, CBR, and Inclusive Development, 23(3), 48-64.

[4] Rispin, K., DiFrancesco, J., Raymond, L.A., Riseling, K. & Wee, J. (2017). Preliminary inter-rater reliability of the wheelchair components questionnaire for condition. Disability and Rehabilitation: Assistive Technology, 13(6), 552-557.

[5] Pearlman, J., Cooper, R.A., Krizack, M. Lindsley, A., Wu, Y., Reisinger, K.D., Armstrong, W., Casanova, H., Chhabra, H.S. & Noon, J. (2008). Lower-Limb Prosthesis and Wheelchairs in Low-Income Countries. IEEE Engineering in Medicine and Biology Magazine, 27(2), 12-22.

[6] World Health Organization (2011). Joint Position Paper on the Provision of Mobility Devices in Less-Resources Settings. WHO Press.

[7] Rispin, K., Riseling, K. & Wee, J. (2017). A longitudinal study assessing the maintenance condition of cadres of four types of wheelchairs provided in low-resource areas. Disability and Rehabilitation: Assistive Technology, 13(2), 146-156.

[8] Rispin, K., Dittmer, M., McLean, J. & Wee, J. (2017). Preliminary reliability and internal consistency of the Wheelchair Components Questionnaire for Condition. Disability and Rehabilitation: Assistive Technology, 12(8), 852-856.

[9] Koo, T.K. & Li, M.Y. (2016). A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine, 15(2), 155-163.

ACKNOWLEDGEMENTS

Thanks to our partners at the study site and the wheelchair professionals who acted as raters.