Multi-Functional Wrist Orthtoic With Universal Gesture Recognition System

Shruthi Suresh1, Sudhanshu Manda1, Camila Marrero2, Lucas Jacob3, Bradley S. Duerstock1,4

1Weldon School of Biomedical Engineering, 2School of Engineering Education, 3Purdue Polytechnic Institute, 4School of Industrial Engineering, Purdue University

Abstract

Individuals with cervical spinal cord injuries (SCI) often require wrist supports or orthotics to stabilize their wrists and allow them to perform activities of daily living (ADL). Current tools to stabilize flaccid wrists after a SCI, such as wrist splints, primarily only provide support without too much additional functionality. We developed and designed a 3D printed wrist orthotic that can detect four different gestures through a Dynamic time warping based machine learning model. The model had a high accuracy in detecting the four different gestures and allowed the system to be customized to different individuals. The wrist orthotic can be adapted to allow a variety of ADL to be performed by tetraplegics.

introduction

Every year, there are approximately 17,000 new cases of Spinal Cord Injuries (SCI) in the United States alone. 53.9% of SCIs occur in the cervical region (C1-C7) and almost 44% (approximately 4000) of these individuals have injuries in the mid-cervical region of the spine (C3-C6) (NSCISC, 2015). Individuals with mid-cervical SCI usually have difficulties performing activities of daily life (ADL) that require fine motor control such as writing, opening doors with keys, due to weakness or paralysis in the flexor muscles. Wrist orthotics are often used in rehabilitation of individuals with SCI to promote the correct positioning of the wrist(Nas, Yazmalar, Şah, Aydın, & Öneş, 2015). Even though it primarily provides only support, with the addition of pockets in the palm cuff accessories such as dining utensils, pencils, pens or toothbrushes, tetraplegics can perform certain ADLs independently.  In individuals with lower cervical SCI, tenodesis splints are often used to allow pinching of the forefinger and thumb to pick up small objects (House, Gwathmey, & Lundsgaard, 1976). Neuroprostheses have also been used to assist individuals with SCI during rehabilitation and performing certain ADL with ease (Rüpp, Rohm, Schneiders, Kreilinger, & Müller-Putz, 2015). However, current tools to stabilize flaccid wrists after a SCI, such as wrist splints, primarily only provide support without too much additional functionality. Moreover, the devices are bulky, heavy and burdensome to wear (Romilly, Anglin, Gosine, Hershler, & Raschke, 1994). They also require a specialist to perform the fitting of the orthotic making them quite cumbersome to acquire.

In a previous study, we explored the use of electromyography (EMG) and an inertial measurement unit (IMU) to develop a 3D printed multi-functional wrist orthotic (MFWO) which would allow users to control different actuators using a set of gestures (Suresh, Madrinan Chiquito, Manda, Jacob, & Duerstock, 2016). The use of the IMU combined with an activation system allowed for a high accuracy in gesture detection. The implementation of a machine learning algorithm would allow for an improved accuracy as well as a more customized system.

In this paper, we improve the accuracy of the MFWO technology using a dynamic time warping (DTW) based machine learning algorithm. DTW is a well know technique which compares two time dependent sequences and identifies the similarities in them. (Müller, 2007). DTW has been commonly used in detection and recognition of gestures (Akl & Valaee, 2010; Pisharady & Saerbeck, 2013)

Method

Wearable Orthotic

Figure 1. The Figure shows a black color 3D printed wrist orthotic worn on the hand of an individual. The wrist orthotic has an Inertial Measurement Unit and wires connect it to the Raspberry Pi. The wrist orthotic fits the shape of the individual’s hand
Figure 1: Prototype of the wrist orthotic
The wearable wrist orthotic consists of a 3D printed plastic body made of ABS plastic and wrist bands made of leather. The orthotic, shown in Figure 1, is designed to be comfortable and provide adequate support to the wrist.

The orthotic holds an IMU (Sparkfun Electronics ®) with 9 degrees of freedom which is connected to a Raspberry Pi. The Raspberry Pi is connected to a monitor which allows the user to interact with the system.

Gestures

 

Figure 2. The Figure shows a subject in a wheelchair performing the Out-Hold gesture with the hand suspended away from the body. The wrist orthotic is attached to the Raspberry Pi
Figure 2: Subject performing the Out-Hold gesture with the wrist orthotic
As shown in Figure 2, four different gestures were performed by each of the subjects- In-Out, Out-In, In-Hold and Out-Hold. These gestures were chosen for the comfort and ease of execution.

Subject Testing

Five subjects were recruited to assess the accuracy of the DTW based machine learning algorithm. Of these subjects, two were individuals with cervical SCI and three were individuals without SCI.

The subjects were requested to follow prompts on a screen to perform each of the gestures four times. They were requested to repeat each of the gestures in a consistent manner. This was used as the training data for the machine learning model. After the system was trained on the four different gestures, the subjects were instructed to perform different gestures at random with each gesture being repeated five times.

REsults

Accuracy of detection of gestures

With 64 training samples and 100 testing samples, the machine learning algorithm detected the four different gestures with an almost 100% accuracy. The table below shows the accuracy of the machine learning model in detecting the four gestures

Table 1: Accuracy of gesture detection

Gestures
Accuracy
In-Out 99.87%
Out-In 100%
In-Hold 100%
Out-Hold 100%

discussion

In this study, we tested the accuracy of a DTW based machine learning model in the detection of four simple gestures. We identified a 100% accuracy of gesture detection when the model was trained with a small group of tetraplegics. The In-Out gesture was the only gesture which had an accuracy of less than 100%. This may be attributed to the individual’s body obstructing the hand from fully moving towards the body which causes a reduced range of motion. The use of an IMU with nine degrees of freedom as compared to our previous study (Suresh et al., 2016) wherein an IMU with six degrees of freedom was used, could also have allowed an improvement in the accuracy of gesture recognition.

Moreover, the current gestures are limited and simplistic in nature which may also account for the high level of accuracy detected by the model. In future work, we would like to include gestures from a tested lexicon (Jiang, Duerstock, & Wachs, 2014) which is more diverse and can be used to control a variety of actuators and mechatronics attached to the wrist orthotic. The implementation of the machine learning algorithm can also allow the wrist orthotic to be customized to every individual user.

False positives can be avoided using an activation system such as a pushbutton or an EMG system, which was validated in our previous study. The activation system will allow a timed period during which the user can perform the gesture.

The ability of the system to detect gestures with a high level of accuracy would promote the use of wrist braces to assist tetraplegics with activities of daily living. This would allow them to become more independent, find gainful employment and actively participate in their communities.

future work

The machine learning model will be combined with a modified wrist orthotic to allow for control of different mechatronic features using various gestures. Moreover, the system will be tested on different gestures and a larger population of individuals to ensure the accuracy of the MFWO system.

References

Akl, A., & Valaee, S. (2010). Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings (pp. 2270–2273). IEEE. https://doi.org/10.1109/ICASSP.2010.5495895

Center, N. S. C. I. S. (2015). 2015 Annual Report Complete Public Version, 106.

House, J. H., Gwathmey, F. W., & Lundsgaard, D. K. (1976). Restoration of strong grasp and lateral pinch in tetraplegia due to cervical spinal cord injury. The Journal of Hand Surgery, 1(2), 152–159. https://doi.org/10.1016/S0363-5023(76)80009-3

Jiang, H., Duerstock, B. S., & Wachs, J. P. (2014). An analytic approach to decipher usable gestures for quadriplegic users. 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 3912–3917. https://doi.org/10.1109/SMC.2014.6974542

Müller, M. (2007). Dynamic Time Warping. In Information Retrieval for Music and Motion (pp. 69–84). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-74048-3_4

Nas, K., Yazmalar, L., Şah, V., Aydın, A., & Öneş, K. (2015). Rehabilitation of spinal cord injuries. World Journal of Orthopedics, 6(1), 8–16. https://doi.org/10.5312/wjo.v6.i1.8

Pisharady, P. K., & Saerbeck, M. (2013). Robust gesture detection and recognition using dynamic time warping and multi-class probability estimates. In 2013 IEEE Symposium on Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP) (pp. 30–36). IEEE. https://doi.org/10.1109/CIMSIVP.2013.6583844

Romilly, D. P., Anglin, C., Gosine, R. G., Hershler, C., & Raschke, S. U. (1994). A Functional Task Analysis and Motion Simulation for the Development of a Powered Upper-Limb Orthosis. IEEE Transactions on Rehabilitation Engineering, 2(3), 119–129. https://doi.org/10.1109/86.331561

Rupp, R., Rohm, M., Schneiders, M., Kreilinger, A., & Müller-Putz, G. R. (2015). Functional rehabilitation of the paralyzed upper extremity after spinal cord injury by noninvasive hybrid neuroprostheses. Proceedings of the IEEE, 103(6), 954–968. https://doi.org/10.1109/JPROC.2015.2395253

Suresh, S., Madrinan Chiquito, D. F., Manda, S., Jacob, L., & Duerstock, B. S. (2016). Motor-Activated Multi-Functional Wrist Orthotic To Assist Individuals With Cervical Spinal Cord Injuries With Activities Of Daily Living. In RESNA/NCART 2016.

Acknowledgements

We are grateful to the Discovery Park and Center for Paralysis Research at Purdue University for their support.