Analysis Of Assistive Robotic Manipulator (Arm) Performance Based On A Task Taxonomy

Bethany Langdon1,2, Hongwu Wang3,4, PhD, Dan Ding3,4, PhD

1Dept. of Aerospace and Mechanical Engineering, University of Arizona, Tucson, AZ 85719

2Dept. of Biomedical Engineering, University of Arizona, Tucson, AZ 85719

3Dept. of Rehabilitation Science and Technology, University of Pittsburgh, Pittsburgh, PA 15260

4Human Engineering Research Laboratories, Dept. of Veterans Affairs, Pittsburgh, PA 15206

INTRODUCTION

In recent years, the development of assistive robots that help people with disabilities complete daily life functions has greatly increased. For example, assistive robotic manipulators (ARMs) can help people with upper-limb disabilities complete daily tasks independently. Previous studies have shown some of the benefits and abilities of ARMs through user testing and interviewing. An interview with iARM (Exact Dynamics, the Netherlands) users showed that the ARM was used mainly for “drinking, eating, and picking up objects” (Wakita et al., 2013, p. 1284). A study on the JACO ARM (Kinova Robotics, Canada) tested users’ ability to complete simple tasks like picking up a bottle, pushing buttons, grabbing a tissue, pouring water, and grabbing a straw (Maheu, Frappier, Archambault, & Routhier 2011). Another study with the RAPUDA ARM (National Institute of Advanced Science and Technology, Japan) evaluated the ARM’s ability to scratch a face (Wakita, Yoon, & Yamanobe 2012). While these studies have proved both the effectiveness of ARMs for certain tasks and the desire for ARMs, there is currently no systematic method for understanding and analyzing the capabilities of ARMs for a variety of daily tasks. In this paper, we propose to use a task taxonomy to analyze an ARM’s performance, which would help predict the ability of using an ARM to complete a task, categorize the difficulties an ARM encounters when completing tasks, and develop training strategies for users to effectively operate an ARM. 

METHOD

Task List

 A task list was composed based on the International Classification of Functioning (ICF) and user requests from previous studies. ICF examines a broad range of daily life functions and the required body functions, and thus it is often used as the basis of evaluation protocols for assistive technology (WHO, 2001). Daily life functions are divided into three general categories: body functions and structure, activities and participation, and severity and aspects of the environment (Matsumoto, Nishida, Motomura, & Okawa, 2011). As shown in previous studies, the activities and participation category can be used to compose a list of tasks an ARM can perform (Matsumoto et al., 2011; Tanaka, Yoshikawa, Oyama, Wakita, & Matsumoto 2013; Wakita et al, 2013, Wakita, Tanaka, & Matsumoto 2014; Wakita et al., 2012).

Also in previous studies, people with upper-limb disabilities requested that certain items be handled by the ARM (Choi, Deyle, & Kemp 2009; Maheu et al., 2011; Wakita et al., 2012). Furthermore, tasks can be organized into categories of eating/drinking, picking up/manipulating objects, personal hygiene, personal mobility, and leisure/work (Groothuis, Stramigioli, & Carloni 2013). Based on these previous studies and ICF, eleven common household objects of a range of sizes, weights, and materials from various categories were chosen for evaluation.

Task Taxonomy

Analysis of ARM performance is based on evaluation of the ARM’s capability of executing different hand/arm actions. Five different hand/arm actions based on ICF “hand/arm use” categories are used to describe how the ARM interacts with an object in grasp: picking up, carrying in hands, manipulating, putting down, and releasing (WHO, 2001). Grasping an object with the ARM is defined as enclosing an object or holding it in the fingers of the ARM (Bullock & Dollar, 2011). These hand/arm actions are further categorized by the required steps associated with each action (Table 1).

The difficulty of picking up or putting down an object is dependent on the type of surface: stable or unstable. A surface is any place where an object can be rested on and remain stationary. A stable surface is defined as a flat, horizontal surface where the object is less prone to being dropped on the ground while the user attempts to lift it, such as a table or a shelf. An unstable surface is a surface where the object is more prone being dropped on the ground while the user attempts to lift it, such as a slanted surface or a hook on the wall. Each step for the action and the type of surface are used to help evaluate the ARMs ability to pick up or put down an object (Table 2).

Evaluation Protocol

The JACO robotic arm, manufactured by Kinova Robotics, was used for testing. The JACO ARM has three different modes when in the 3-axis setting: translation, wrist, and fingers (“JACO Rehab Edition: User Guide,” n.d.). Translation mode allows for movement in the X, Y, or Z axes. Wrist mode includes lateral orientation, vertical orientation, and wrist rotation. The finger mode allows for the opening and closing of two or three fingers.

Two able-bodied adult test pilots tested the tasks. Both testers were trained using the JACO training module from Kinova (“JACO Rehab Edition: Minimum training,” n.d.) and a task board (Chung, Wang, Kelleher, & Cooper, 2013).

Table 1: Definitions of each hand/arm action step

Task

Picking Up

Carrying in Hands

Manipulating

Putting Down

Releasing

Surface Type

Average Success Rate

Take coin out of bowl

A, B, C

 

 

 

 

Stable

62.5

Take pill out of bottle (Use cereal)

A, B, C

 

 

 

 

Stable

37.5

Put coin into slot

 

Wrist, Translational

Pushing

 

 

 

43.8

Put key into lock

 

Wrist, Translational

Pushing

 

 

 

12.5

Pull hand towel off of rack

A, B, C

 

 

 

 

Unstable

87.5

Pull tissue out of box

A, B, C

 

 

 

 

Stable

100

Put dropped pill back into pill bottle (Use cereal)

A, B, C

Translational

 

 

A, B

Stable

81.3

Throw tissue away

A, B, C

Translational

 

 

A, B

Stable

100

Hold TV remote

A, B, C

Translational,  No Movement

 

A, B

 

Stable

93.8

Hold book

A, B, C

Wrist, Translational, No Movement

 

A, B

 

Stable

6.3

Put toothpaste onto toothbrush

A, B, C

Wrist, Translational

 

A, B

 

Stable

50

Pour cereal from container into bowl

B, C

Wrist, Translational

 

A, B

 

Stable

93.8

Stir food

A

 

Translational

 

 

Stable

56.3

Wipe whiteboard

 

Wrist, Translational

Translational

 

 

 

100

Turn a key

 

 

Wrist

 

 

 

0

Pull out a key

 

 

Pulling

 

 

 

31.3

Hang up a hand towel

 

 

 

A, B

 

Unstable

100

Turn a cup on a table a certain amount

 

 

Wrist, Translational

 

 

 

50

The JACO training module helped the testers become familiar with the modes of the ARM. Training involved lifting a water bottle to drink, pouring cereal from a cup to a bowl, eating cereal, and avoiding collision with objects on a table. The task board training proved the tester’s proficiency using the ARM and also the capability of the ARM to push different sized buttons, turn a knob, turn a door handle, flip a light switch, and flip a toggle switch.

Table 2: Tasks tested with the JACO ARM, the associated hand/arm action steps and surface type, and the average success rate of both testers

Hand/Arm Action Step

Definition

Picking Up A

Grasping an object in order to pick it up.

Picking Up B

Lifting a grasped object from a surface.

Picking Up C

Holding onto an object once it is lifted from a surface.

Carrying in Hands Wrist

Holding onto an object in the air while the ARM is in wrist mode.

Carrying in Hands Translational

Holding onto an object in the air while the ARM is in translation mode.

Carrying in Hands No Movement

Holding onto an object in the air without any movement.

Manipulating Wrist

Holding onto an object that has a contact point with a surface while moving the ARM in wrist mode.

Manipulating Translational

Holding onto an object that has a contact point with a surface while moving the ARM in translation mode.

Manipulating Pushing

Holding onto an object that has a contact point with a surface and pushing it away from oneself into a desired location.

Manipulating Pulling

Holding onto an object that has a contact point with a surface and pulling it towards oneself into a desired location.

Putting Down A

Putting an object down onto a surface.

Putting Down B

Removing the ARM from the object after it has been put down onto a surface.

Releasing A

Aiming above a desired location to release an object from grasp.

Releasing B

Releasing an object from the grasp of the ARM.

Both testers were given up to three hours and eight trials to complete each step of a task. For all steps, if the tester successfully completed the step, a point was given. If the trial was unsuccessful, zero points were given. Distinguishing when a trial was unsuccessful differed for each step. The tester was allowed to continually attempt steps of Picking Up A or B (Table 1). A trial involving these steps ended if the tester felt the step was not possible and wished to start a new trial. A trial involving Picking Up C, Carrying in Hands Wrist, Translational, or No Movement, or Manipulating Wrist, Translational, Pushing, or Pulling ended when the tester dropped the object. A trial involving Manipulating Wrist or Translational ended when the tester lost connection of the object with a surface. If the tester lost grip with an object while attempting the steps of Manipulating Pushing or Pulling, a trial ended. Lastly, trials ended when the tester attempted the steps of Putting Down A or B, or Releasing A or B, which are the final steps a task. If steps preceding the failed step were completed during the trial, those steps were given one point. Once a step was unsuccessful in all eight trials, assistance was then given to ensure the next step of the task was attempted. An average percentage success rate was calculated for each task and each step by taking the total number of points given and dividing by eight.

RESULTS

Looking first at the JACO ARM’s ability to complete each task, the ARM is capable of pulling a tissue of the box, throwing a tissue away, wiping a whiteboard, and hanging up a hand towel (Table 2). All of these tasks have a 100% average success rate. Other highly successful tasks include holding a TV remote and pouring cereal from a container into a bowl (93.8% average success rate). The most difficult tasks are putting a key into a lock (12.5% average success rate), holding a book (6.3% average success rate), and turning a key (0% average success rate).

Table 3: Success rate of both testers for each hand/arm action step

Tested Hand/Arm

Action Step

Success Rate Tester 1

Success Rate Tester 2

Picking Up A Stable

86.3

91.3

Picking Up B Stable

81.3

88.8

Picking Up C Stable

86.3

85.0

Picking Up A Unstable

100.0

100.0

Picking Up B Unstable

87.5

87.5

Picking Up C Unstable

100.0

100.0

Carrying in Hands Wrist

97.5

92.5

Carrying in Hands Translational

98.6

88.9

Carrying in Hands No Movement

93.8

100.0

Manipulating Wrist

41.7

58.3

Manipulating Translational

75.0

91.7

Manipulating Pushing

18.8

43.8

Manipulating Pulling

37.5

75.0

Putting Down A Stable

100.0

96.9

Putting Down B Stable

100.0

100.0

Putting Down A Unstable

87.5

100.0

Putting Down B Unstable

100.0

100.0

Releasing A

93.8

100.0

Releasing B

100.0

100.0

The ARM is capable of picking up, carrying, manipulating, putting down, and releasing objects. Successful completion of each action depends on the step and surface type (Table 3). Picking up objects from an unstable surface as compared to a stable surface has a higher average success rate (95.8% vs. 86.5% success). The average success rate for picking up an object was 91.2%. Carrying objects in the JACO hand had an average success rate of 95.2%. Objects were dropped the most when using translational movement (93.8% average success rate). The average success rate of manipulating objects was 55.2%. The most difficult manipulation is pushing an object (31.3% average success rate). Translational movement is the most successful at 83.3%. Putting down objects was slightly more successful on stable surfaces as compared to unstable surfaces with a 99.2% success rate for stable surfaces and 96.9% for unstable surfaces. Releasing objects had a 98.4% success rate. The most difficult step of releasing was aiming properly (Releasing A).

DISCUSSION

The task taxonomy helped us better understand the capabilities of the ARM. It can be used to predict how the ARM will perform with other tasks. The task taxonomy also showed that the ability of an ARM to complete tasks is largely dependent on three aspects: the user, the environment, and the limitations of ARM itself. Throughout this performance evaluation, we observed that each of the three aspects could be improved in order to make it easier or possible for an ARM to complete a task.

The user aspect is affected by both training and strategy. The effect of strategy was seen in the “Pull out key” task. Grasping the key flat between the two fingers of the JACO was never successful, but grasping the key by squeezing the sides was successful. The effect of strategy was also seen in the “Hold book” task. Both testers began the task by lifting up the book by its cover. Tester 1 then stood the book up on its side and rearranged the position of the ARM’s fingers to get a better grasp, resulting in one successful trial. Tester 2, on the other hand, continued to hold the book by its cover, resulting in only failed trials.

Environmental factors include the size, material, and initial position of objects. These factors most strongly affect the ability to grasp an object in order to pick it up, carry it in the hands, or manipulate it. For example, both testers were never successful at the “Turn key” task. For the “Pull key” task, tester 1 had a 37.5% success rate and tester 2 had a 25% success rate. For the “Put key into lock” task, tester 1 was never successful and tester 2 had a 25% success rate. The low success rates are due to the fact that keys are thin and smooth so the JACO’s fingers cannot provide a strong, firm grip.

 Another environmental factor is the initial position of an object, which affects the ARM’s ability to pick up an object. The effect of initial position was seen in the “Hold book,” “Pick pill out of the bottle,” and the “Take coin out of bowl” tasks. Grasping the book laying flat on the table was the most difficult step. The cereal could only be picked up out of the bottle if it was in an ideal initial position, such as standing upright near the top of the bottle. Lastly, picking up a coin from the bowl was easiest when it was resting on another coin.

Environmental factors can be improved through the assistance of adapters. Adapters provide a simple, convenient way to make a task easier. For instance, Tester 1 attempted the “Put key into lock” task eight times and was unsuccessful each time. A ninth try was attempted with the aid of an adapter on the key and this time the task was successful. The “Turn key” task for both testers was only successful when using the key adapter. Other adaptations can be made to improve the feasibility of tasks as well. These adaptations could include a larger handle for the spoon, an adapter on the toothpaste that allows for a better grip, and an adapter on a book to ensure it can be picked up despite its initial position.

ARM capability cannot be changed by the user but can be changed at the design level. The JACO is incapable of moving along several axes simultaneously. In translation mode, the JACO could complete simultaneous axis movement translationally, such as along X while moving along Z, but could not complete rotation at the same time (“JACO Rehab Edition: User Guide,” n.d.). In order to complete any rotational movement, the JACO must be put in wrist mode. The inability of the JACO to complete this movement caused difficulty of the “Turn a cup” task. Modes needed to be frequently switched in order to keep the cup in contact with the table. Adding the capability of simultaneous axis movement could improve an ARM’s ability to complete tasks.

Other restrictions to completing tasks based on ARM capability include the length of the ARM, the load limit for objects, the size of the hand and fingers, the grip strength of the fingers, and the ARM’s resistance to the environment changes such as temperature. Any improvements to each of these restrictions could improve the ARM’s ability.

FUTURE WORK

            Based on the results, the JACO is capable of performing simple tasks without assistance. Future work should use the task taxonomy proposed in this paper to analyze and evaluate more tasks including complex sequential tasks. For example, instead of just putting the toothpaste on the toothbrush, the whole process of tooth brushing including the step of opening the cap and actually brushing teeth should be included. Also it would be beneficial to recruit end-users of the ARM in the evaluation. The results of the study are limited due to the fact that only two able-bodied test pilots tested the tasks and that only a small sets of household tasks were evaluated.

 CONCLUSION

            Evaluation of an ARM requires the creation of a task taxonomy based on a task list and the hand/arm action steps necessary to complete each task. The JACO ARM is capable of picking up, putting down, manipulating, carrying, and releasing objects, but success is dependent on the step and surface type. Improvements to the user strategy through better training strategies, ARM capability through enhanced design, and the environment through adapters can make a task easier or make a challenging task possible.

REFERENCES

  1. Bullock, I. M., & Dollar, A. M. (2011). Classifying Human Manipulation Behavior. IEEE, 1-6.
  2. Choi, Y. S., Deyle, T., & Kemp, C. C. (2009). A list of household objects for robotic retrieval prioritized by people with ALS. IEEE, (version 092008), 510-517. doi:10.1109/ICORR.2009. 5209484
  3. Chung, C.-S., Wang, H., Kelleher, A., & Cooper, R. A. (2013). Development of a Standardized Performance Evaluation ADL Task Board for Assistive Robotic Manipulators. In Proceedings of the Rehabilitation Engineering and Assistive Technology Society of North America Conference. Seattle, WA.
  4. Groothuis, S. S., Stramigioli, S., & Carloni, R. (2013). Lending a helping hand: Toward novel assistive robotic arms. IEEE Robotics & Automation Magazine, 20(1), 20-29. doi:10.1109/MRA.2012.2225473
  5. International Classification of Functioning, Disability and Health (2001). World Health Organization.
  6. JACO Rehab Edition: Minimum training requirement guide (n.d.). Kinova, V1.0.3, 1-15.
  7. JACO Rehab Edition: User Guide (n.d.). Kinova, V1.2.7, 1-35.
  8. Maheu, V., Frappier, J., Archambault, P. S., & Routhier, F. (2011). Evaluation of the JACO robotic arm: Clinico-economic study for powered wheelchair users with upper-extremity disabilities. IEEE, 1-5. doi:10.1109/ ICORR.2011.5975397
  9. Matsumoto, Y., Nishida, Y., Motomura, Y., & Okawa, Y. (2011). A concept of needs-oriented design and evaluation of assistive robots based on ICF. IEEE, 1-6. doi:10.1109/ICORR.2011.5975437
  10. Tanaka, H., Yoshikawa, M., Oyama, E., Wakita, Y., & Matsumoto, Y. (2013). Development of assistive robots using international classification of functioning, disability, and health: Concept, applications, and issues. Journal of Robotics, 2013, 1-12. doi:10.1155/2013/608191
  11. Wakita, Y., Oyama, E., Yoon, W., Tanaka, H., Matsumoto, Y., Blom, A., & Stuyt, H (2013). User Evaluation of Service Robotic Arms Based on ICF through Interviews with People with Upper-limb Disability. IEEE, 1282-1287.
  12. Wakita, Y., Tanaka, H., & Matsumoto, Y. (2014). Analysis and Design of Service Robots based on ICF. IEEE, 1916-1920.
  13. Wakita, Y., Yoon, W., & Yamanobe, N. (2012). User evaluation to apply the robotic arm RAPUDA for an upper-limb disabilities patient's daily life.

ACKNOWLEDGEMENTS

Thank you to Brandon Petrouskie, Dean Palacios, and Joshua Chung for their help and support on this project. This work was supported by the Research Experience for Undergraduates Program on Quality of Life Technology (National Science Foundation # EEC 1358903), the Craig Neilsen Foundation (#278966) and the VA Center of Excellence on Wheelchairs and Related Rehabilitation Engineering (# B6789C).