Developed Outdoor recreation assessment process

precision study

Irene Hsu1, MS, Nathan E. Tolbert2, AAS, Seanna L. Hurley2, MS,

Peter W. Axelson2, MSME

1Washington University, St. Louis, MO and 2Beneficial Designs, Inc., Minden, NV


This project is developing a Developed Outdoor Recreation Assessment Process (DORAP) to be used to collect attributes for outdoor recreation features and evaluate whether they comply with accessibility standards. Access to outdoor recreation environments is essential to the full inclusion and integration of people with disabilities into independent living, family activities, and society. A preliminary repeatability and precision study was completed on seven subjects. Preliminary data suggest that a comprehensive data collection process that is reproducible in the natural environment is possible and can be used to determine compliance.


Standards for specific outdoor recreation features now exist within the Architectural Barriers Act (ABA) Accessibility Standards and apply to national parks and other outdoor areas developed by the Federal Government (US Access Board, 2013). Nevertheless, there is no standardized process for assessing the accessibility of recreation features, e.g., picnic tables, fire rings, etc., used for outdoor recreation and camping facilities. Access to all developed outdoor recreation areas is critical since most Americans use these facilities for activities such as walking, family gatherings, picnicking, nature viewing, photography, camping, bicycling, and boating (USDA & N.O.A.A., 2000). DORAP is being developed to collect attributes about outdoor recreation features and evaluate whether they comply with accessibility standards. An objective inventory of outdoor recreation facilities will improve usability for people with disabilities and allow land managers to identify and create a plan for removing barriers. The process must function equally well for those who have little knowledge of the "natural" environment versus the "built" environment and those who may have extensive knowledge in the outdoor recreation field, but little or no experience in the accessibility field. It also must overcome the various conditions inherent to the natural environment.


This study’s purpose was to determine: 1) what feature attributes needed to be evaluated in order to verify compliance and 2) if a valid assessment process can be created for evaluators with varying experience.


Development of the assessment process

The development of draft DORAP was a reiterative process between field-testing and revisions to improve comprehension and validity. Each feature in the ABA technical guidelines was analyzed and collection attributes were defined to verify compliance.

Draft DORAP was field-tested by: 1) two project outdoor accessibility experts with mobility impairments, 2) two project leaders and two outside accessibility professionals, and 3) seven subjects with varying data collection and outdoor accessibility experience.


Seven subjects (five male, two female) with varying experience with the outdoor environment and/or accessibility evaluations performed the preliminary precision testing. Three subjects were accessibility professionals with experience in assessments, two were landscape architects with no accessibility assessment experience, and two had no experience in land management or accessibility assessments.


Illustration of a wheelchair knee and toe space profile gauge in use to evaluate the clear space provided beneath a picnic table in the outdoor recreation environment for a wheelchair users knees and toes Figure 1 : Example of profile used to verify minimum knee and toe space clearance
The data collection equipment consisted of standard manual tools (Tape Measure, SmartToolTM, Force Gauge, Rolla-Wheel), a laptop with automated collection software, and a wheelchair toe-and-knee-clearance profile gauge (Figure 1).

A profile gauge was developed in order to objectively evaluate wheelchair access to tables. The profile is the exact size of the minimum clearance space required under objects and can be slid horizontally under an object to verify that the entire clear space is free of obstructions.

 Precision testing

A preliminary repeatability study was conducted by a study leader on eleven features. Three assessment trials were performed on three different days using the same collection tools.

An intermediate precision study was conducted with seven subjects with no prior training or instruction. Each subject performed one trial measuring each attribute for each of the eleven features in the presence of the study leader. Each subject was asked to complete the trial by reading the help text and observing the visual guide images. The project leader collected information on mechanics, comprehension, and any variance in the process observed between subjects.

Verifying compliance with the guidelines

Verification of compliance was determined by querying the collected value against the minimum or maximum requirement of the ABA guideline and assigned a compliant (1) or noncompliant (0) value.


For the preliminary repeatability study, the three trials were averaged for each feature and the standard deviation was calculated. For the intermediate precision study, the seven trials across subjects were averaged for each feature and the standard deviation was calculated. For the compliance query, the Boolean results (1, 0) were averaged across trials for each feature where 100% or 0% represents unanimous compliance or non-compliance, respectively.


Development of assessment process

Figure 2 : "Min Fire Surface Height" visual guidance image with help text for pivoting cooking surface

Illustration in elevation view depicting the measurement of the fire building surface in a pivoting cooking grill

Help Text: Rotate the grill in different directions. Determine and record the minimum height of the fire-building surface above the ground surface.

Review of the ABA Accessibility Standards identified twenty outdoor recreation features. Feature attributes, e.g., dimensions, slopes, index lists, and forces, were defined for accessibility evaluations, assessed for compliance by experts, and incorporated into the draft DORAP (See Table 1). For example: ABA Section 1011.5.1 states: "Fire building surfaces shall be 9 inches minimum above the ground," thus related help text and image were created (See Figure 2).

The Yes/No attributes are feature observation questions, such as “Is the clear space free of any obstacles?” Associated guidance text and visual guide images were created to facilitate evaluator comprehension and valid DORAP data collection across varying levels of experience from experts to novice.

Table 1 : Features with collection attributes

Feature Type
Number of Collection Attributes
Y/N Dims Slope List Force
Bear Proof Receptacle 10 3 4 1 1
Dumpster Type Receptacle* 7 2 4 1 0
Standard Receptacle 10 3 4 1 1
Bench 7 2 4 1 0
Cooking Surface Fixed 6 6 4 1 0
Cooking Surface Pivoting* 6 6 4 1 0
Fire Ring* 6 4 4 1 0
Fireplace or Woodstove 6 4 4 1 0
Other (e.g., bear box)* 9 6 6 1 1
Outdoor Rinsing Shower* 10 6 4 1 1
Parking Space 2 1 6 1 0
RV Parking or Pull Up Space* 2 1 6 1 0
Table* 11 16 6 3 0
Tent Pad* 12 2 6 1 0
Tent Platform 12 3 6 1 0
Trail Head Sign 8 5 4 0 0
Utility Hookup 10 3 4 2 1
Viewing Location* 8 5 4 1 0
Viewing Scope* 11 18 4 2 1
Water Hydrant* 8 6 6 1 1

Y/N= Yes/No; Dims=Dimensions

* Features included in repeatability and precision study


Seven subjects successfully used the profile gauge to determine if the minimum wheelchair clear space provided was compliant at the picnic table feature.

Precision of assessment process

These data suggest that the draft DORAP is repeatable for all data collection attributes, except for the clear space measurements. The full data set for the Cooking Surface (Grill) - Pivoting is shown in Table 2 and 4 (data on repeatability and precision, respectively). These data suggest that the draft DORAP is reproducible for all data collection attributes, except for the clear space measurements.

Table 2 : Preliminary repeatability study (n=1)
Cooking Surface Trial 1 Trial 2 Trial 3 Avg (SD) Compliant
Max Surface Height (in)* 35.00 36.00 36.50 35.83 0.764 0%
Min Surface Height (in)* 31.00 30.25 30.25 30.50 0.433 100%
Min Fire Surface Height (in)* 28.00 27.00 27.25 27.42 0.520 100%
Raised Edge Width (in)* 0.125 0.125 0.125 0.125 0.0000 100%

Clear Space

Min Length (in)* 31.00 33.00 33.50 32.50 1.323 100%
Min Width (in) 33.00 45.00 45.50 41.17 7.077 100%
Max Grade (%)* 4.4 3.0 3.0 3.5 0.81 100%
Max X-slope (%)* 4.0 3.0 2.8 3.3 1.15 100%
* Preliminary data demonstrates repeatability

Verifying compliance with the guidelines

The data from both precision studies returned is shown in Table 2 and Table 3 “Compliant” columns. These data illustrate that the query result of compliant or non-compliant for all attributes were either 100% or 0%, which indicates unanimous agreement by all subjects, except for the clear space grade and slope measurements.


Development of the assessment process

These preliminary results suggest that a viable DORAP data collection process can be achieved. One challenge was that the guidelines were created for application to new construction and renovations, not for evaluation of existing facilities. Multiple discussions were held with US Access Board members regarding intent and methods to verify compliance in order to develop a relevant process for existing facilities.

The development process identified three repeating components that apply to most features: dimensions (length, width, depth, height), clear ground space (slopes, length, width), and observations (Yes/No, Text Entry, and Index List) that confirm compliance. For the pivoting grill, the heights of the fire-building and cooking surfaces were measured, the clear space was identified and measured for size and slope, and observations were made about connectivity to access routes, drainage issues, etc. This feature was representative of all eleven features evaluated.

Table 3 : Intermediate precision study for cooking surface-pivoting (n=7)

Feature Dimensions
Subjects Avg SD Com-pliant
1 2 3 4 5 6 7
Max Surface Height (in)* 35.00 37.00 37.00 35.00 36.50 36.50 36.00 36.14 0.852 0%
Min Surface Height (in)* 31.00 30.75 30.50 30.50 29.50 30.25 30.00 30.36 0.497 100%
Min Fire Surface Height (in)* 27.50 27.00 28.00 27.00 26.50 27.25 26.50 27.11 0.537 100%
Raised Edge Width (in)* 0.187 0.125 0.187 0.125 0.125 0.125 0.125 0.143 0.0305 100%
Clear Space
Min Length (in) 32.00 96.00 33.00 20.00 37.00 33.50 36.00 41.07 24.860 100%
Min Width (in) 84.00 54.00 41.00 36.00 44.00 45.50 46.00 50.07 15.923 100%
Max Grade (%) 2.9 2.0 3.0 3.2 4.0 3.0 6.4 3.5% 1.41 86%
Max X-slope (%) 3.9 2.1 3.1 4.9 5.5 2.8 5.1 3.9% 1.30 86%
* Preliminary data demonstrates intermediate precision
During the preliminary evaluation and observations of the study trials, opportunities for variation were identified. For example, the measurement of the width of the clear space created a challenge because subjects would measure the width between obstructions, not necessarily the clear space width centered on the feature. Therefore, the clear space width process has since been revised to be measured from the center of the feature to the nearest obstruction.


Use of the profile gauge in the assessment process significantly reduced the time required to collect wheelchair space attributes. The profile gauge converts nine different width, height, and depth tape measure collections into three collections (one profile check, one index list selection, and one measurement of any non-compliant dimension that may exist). The profile gauge produces reproducible results in both the built and natural environments and is applicable to any wheelchair space requirement.

In order to maintain intermediate precision of the process, basic guidance for the collection of certain attributes were defined. For example, “when measuring height of an attribute from the ground surface, ensure the tip of the tape measure rests on the surface of the ground and is not pushed into the native soil.”

Precision of assessment process

Photo of a woman at a campsite squatting down to measure the grade of the clear ground space in front of a cooking grill with a SmartToolTM Figure 3 : Measurement of grade at a clear ground space on natural soil
Measurements of the dimensions of each feature and the observations made regarding compliance were preliminarily proven viable with intermediate precision, except for clear space measurements. The evaluation of clear ground space should improve with the development of a clear space gauge that will produce more repeatable results than the current evaluation process. Because natural environments offer many conditions for variability that do not exist in the built world, grade and cross slopes on the clear space at features were variable (Figure 3). The data demonstrate that measurement of the paved parking surface space was repeatable, in contrast to the other natural surfaces. An average of multiple measurements taken across the clear space surface should be reviewed for all natural-surface measurements. The extreme value for each collection should also be reported to identify potential barriers to be mitigated during routine maintenance.

Future research should include 1) defining the minimum number of measurements required, 2) establishing a tolerance for compliance of natural surfaces, and 3) conducting repeatability and reproducibility studies with greater subject numbers. These research results will be submitted to the US Access Board for consideration with respect to existing facilities.

Verifying compliance with the guidelines

During the query writing process, some attribute collections were revised because the ability to verify compliance of the value was not straightforward. For example, two collections for the minimum fire-building surface height at fire rings were required to verify the height of the fire-building surface inside the fire ring to the clear ground surface on the outside of the fire ring. Other requirements were not simple to apply in practice. For example, tent pads are required to provide 48 inches of clear space around the tent; however, there is no single size for a tent. Research on typical tent sizes resulted in tent pads classified mathematically as compliant for up to a 2-man, 4-man, 8-man, etc. tent.


Results of the research revealed that the draft DORAP can adequately provide an evaluation process for outdoor developed areas for features that exist in outdoor recreation areas, as identified in the ABA guidelines. These data suggest that the DORAP accessibility attribute evaluations are repeatable and reproducible, with the exception of clear space measurements on natural surfaces.


  1. US Access Board. (2013). Recreation facilities. In Architectural Barrier Act accessibility standards. Retrieved January 12, 2015 from
  2. USDA Forest Service & N.O.A.A. (2000). National survey on recreation and the environment summary report #1: Outdoor recreation participation in the United States. Retrieved August 28, 2006, from


This project is funded by the U.S. Department of Agriculture through the Small Business Innovation and Research program grant #2013-33610-21051.

Audio Version PDF Version