BACKGROUND

This paper reports on progress toward developing a software tool for the automatic configuration of mice and other pointing devices. Auto-configuration, if successful, could improve productivity and comfort during computer use for people with physical impairments. We focus here on work done to adjust double-click settings completely automatically, during users’ typical computer use.

Within Windows, two settings govern the definition of double-clicks. Double-click time (DCT) defines the allowable time between two clicks in a double-click. Double-click distance (DCD) defines the allowable distance between two clicks in a double-click. By default, two clicks must occur within 500 ms and 2 pixels of each other to qualify as a double-click.

One approach to accommodating users who may have difficulty meeting the double-click criteria is to adjust DCT and DCD to allow more time and “wiggle- room” between clicks. To make it easier to make these and other mouse-related adjustments, we developed an application called Pointing Wizard. The user completes a short mouse task, and Pointing Wizard then suggests changes to the Windows mouse settings that may enhance performance. With user approval, the wizard activates the suggested changes.

The While use of Pointing Wizard significantly improves mouse performance for users with physical disabilities (Koester 2009; LoPresti 2008), it relies on separate, prescribed tasks, requiring the user to stop what they are doing to run the wizard. This dependence on prescribed tasks is a significant limitation, as it takes time and effort away from the user’s primary task.

We want to move away from prescribed tasks to using natural data as the basis for settings recommendations. “Natural data” refers to user performance data gathered during the user’s typical computer use in the real world, as they perform tasks within their own software applications. With natural data, the user doesn’t have to remember to launch a special program, or take the time to run it. This makes it much more likely that settings will actually be updated over time, and would be less intrusive and distracting to users.

As a first step toward a configuration agent using natural data, this study focused on recommending double- click settings based on natural data. Additional work is ongoing related to other mouse settings such as mouse gain.

RESEARCH QUESTION

Can we develop a software system that will make accurate and effective double-click adjustments, using only natural data, and without knowledge of the specific task being performed?

METHODS

This study collected natural data, developed a classification model to correctly identify a user’s intended double-clicks from the stream of natural data, then used the model-identified double-click data to make recommendations for double-click settings. The main metrics for success were the accuracy of the classification model as well as the accuracy of the resulting recommendations, relative to gold standard benchmarks.

Basic Protocol

20 individuals participated: 15 with upper extremity physical impairments and 5 without. Medical diagnoses included spinal cord injury, cerebral palsy, stroke, and multiple sclerosis.

Each subject completed a series of scripted computer tasks, using their preferred pointing device, during a single session. The session was divided into three parts. The tasks in Parts 1 and 3 were intended to be typical computer tasks that any user would do at one time or another, and included 8 tasks such as drilling into folders, making simple edits in Word, and navigating through pages in a website. Part 2 asked participants to run through the Pointing Wizard software. Task instructions were presented on paper, and an experimenter was present with the participant to verbally walk through the instructions and answer any questions. The session lasted approximately 1 1⁄2 hours.

Collect Natural Data

Modified CRUMBS software was used to record every mouse move event, button press, and button release during the session (Hurst 2010). This formed the stream of natural data for each participant. A data set of click intervals was created using all possible sequential pairs of clicks; each case in the data set represented one instance of two clicks in sequence, including the time and distance gaps between the clicks. The data set included 6531 click intervals, across all 20 subjects.

Establish a Gold Standard

Each click interval in the data set was placed in one of two categories: (1) as truly an intentional double-click (whether successful or not); and (2) as just two separate clicks. We reviewed Morae Recorder videos of user behavior during the tasks, and identified 1722 true double-clicks based on a specific rubric.

Derive a Double-click Classification Model

To find a classification model that matches the ‘gold standard’ classifications as closely as possible, the 6531 intervals were included in a logistic regression analysis, with dDistMax and dTime as the predictor variables and the gold standard classification as the dependent variable. dDistMax is the larger of the horizontal and vertical distances between clicks in each interval, and dTime is the time between clicks. The resulting equation models the log odds of an interval being an actual double-click, as a function of time and distance.

The logistic analysis also measured the accuracy of the classification model and classified each interval using the model equation. For each interval, the model equation yields a value for p, the probability that the interval is in fact a double-click attempt. The model classifications then depend on the specific cut-off value for p; when the model value exceeds the cut-off, the interval is classified as a double-click. By default, the cut-off value is 0.5, but we explored cut-offs of 0.6, 0.7, 0.8, and 0.9, all using the same model equation.

Recommend Double-Click Settings

The most accurate classification model was chosen and used to create recommendations for the user’s double- click time and distance. We compared recommendations using the model-identified double-click intervals to the ‘gold standard’ based on the actual double-clicks. The algorithm already developed for the Pointing Wizard was used to generate recommendations.

Data Analysis

The primary metrics used to assess the success of the classification model were the following [4]:

1. Sensitivity - proportion of actual double-clicks correctly identified as such
2. Specificity - proportion of actual single-clicks correctly identified as such
3. Accuracy – combined ability to identify double-clicks and single-clicks correctly.
4. Descriptive statistics for average DCT and DCD, comparing values for the model-identified intervals to those for actual double-click attempts.

Success of the double-click recommendations was measured by comparing the recommendations for the model-identified double-clicks to the ‘gold standard’ recommendations from the known double-click intervals.

RESULTS

Overall accuracy of the classification model was quite high, as shown in Table 1 for cut-off values of 0.5, 0.6, and 0.7. Cut-off values of both 0.5 and 0.6 yielded excellent classification. Cut-off values of 0.8 and 0.9 yielded too few true positives for some individuals, so those values were not considered further.

Table 1. Classification model success for different cut-off values, across all 20 subjects.

Cut-off	Sensitivity	Specificity	Accuracy
0.5	94.5	93.6	93.8
0.6	90.0	94.6	93.4
0.7	82.2	95.6	92.1

Despite the high accuracy, we observed that, for people without physical impairments, the model- identified double-clicks consistently overestimated double-click time. While the overestimate was modest, about 90 ms, this could lead to suggesting changes in settings for people who don’t need them. Therefore, we introduced an initial screening step, which used the classification model to identify people who are likely to need changes in double-click settings.

Screening Step

People needing settings changes can be identified automatically by the classification model with cut-off = 0.5, using the following screening rule:

If (median DCT > 350 ms) or (average DCD > 1 px), then the person likely needs changes.

Figure 1. Screening rule to identify who may need settings changes. Red box shows boundary defined by the screening rule. d

Data for individuals who meet these criteria can then be examined further to determine just what those changes should be. Conversely, those with both values below those thresholds can be screened out as not requiring any settings changes. The categorizations of this screening rule were compared to how Pointing Wizard categorized these individuals during the experimental session. As shown in Figure 1, the screening rule matched Pointing Wizard’s categorization exactly, with a sensitivity of 100% (12/12) and a specificity of 100% (8/8).

Classification for Those in the ‘Need Changes’ Group

Table 2 re-examines the classification model’s accuracy just for people in the ‘Need Changes’ category. Here, the cut-off of 0.5 was the best-performing model for identifying double-clicks.

Table 2. Classification model success for the 12 individuals in the ‘Need Changes’ category.

Cut-off	Sensitivity	Specificity	Accuracy
0.5	93.2	95.2	94.6
0.6	87.2	96.1	93.5
0.7	77.2	96.8	91.1

This model also estimated double-click time (DCT) and double-click distance (DCD) very accurately, as shown in Table 3. Actual Average is the average for all intervals identified via video as actual double-click attempts. This represents a ‘gold-standard’ for the true measurement. Model Average is the average of double- click times for all intervals that the model classified as double-clicks.

Table 3. Double-click metrics based on model-identified double-clicks. N = 12 individuals in the ‘Need Changes’ category. Max diff is for the individual with the largest difference

V ariable	Actual avg.	Model avg.	Difference avg.	|Difference| avg.	Max diff
DCT (ms)	502.4	485.1	17.2	36.7	100.9
DCD (px)	1.61	1.72	-0.11	0.31	-1.60

In summary, using the classification model with p=0.5 on natural data from 12 individuals who needed settings changes, the model:

1. identified single- and double-clicks with 95% accuracy (94.6, from Table 2)
2. estimated DCT within 7.3% on average (36.7/502.4)
3. estimated DCD within 19.2% on average (0.31/1.61).

Recommendations Using the Model

Figure 2. Recommendations for double-click time. d

Figures 2 and 3 show the resulting recommendations for each of the 12 subjects in the ‘Need Changes’ category. For double-click time, the recommendations from model-identified double-clicks are within 100 ms of those from the actual double-clicks. For double-click distance, the median deviation is 2 pixels. These results suggest that our automated model could make appropriate recommendations for both double-click time and double- click distance.

CONCLUSION

Figure 3. Recommendations for double-click distance. d

This study led to the following “recipe” for automatically adjusting double-click settings:

1. Collect a series of click intervals for the individual, during regular computer use. Each interval case includes dDistMax and dTime values.

2. Run this set of click intervals through the logistic model equation to calculate a probability, p, for each one being an intended double-click. Classify the model-identified double-clicks by using a cut-off value of p > 0.5.

3. For those model-identified double-clicks, calculate the median double-click time (model median DCT) and average double-click distance (model avg DCD).

4. If (model median DCT > 350) or (model avg DCD > 1.0), then this person likely needs changes. In that case, continue on to generate the double-click recommendations in Step 5. If that’s not the case, stop here and recommend default settings for this person.

5. Feed the model-identified double-clicks into the existing algorithms for double-click time and double-click distance. Recommend the resulting values.

For the data in this study, this recipe yields double-click recommendations that are accurate to within 100 ms for double-click time and 2 pixels for double-click distance. We have implemented this recipe into a prototype software system and look forward to field testing it with actual users.