RESNA's Exam Development process is led by psychometric consultants and guided by the input of assistive technology subject matter experts. Decisions regarding exam content distribution and exam passing scores are made by committee and are based on data provided by practitioners in the field. The processes, outlined below, promote an exam that is fair, reliable, valid, and legally defensible.
Job Analysis Study
Development of the current ATP exam was initiated in 2008/2009 with a job analysis study which was facilitated by psychometric consultant, Knapp and Associates. A panel of seven subject matter experts identified the knowledge and skills typically demonstrated by assistive technology professionals with basic competence. After feedback was solicited from a wider group of subject matter experts, the knowledge and skills outline was then validated through a survey of practitioners. The resulting survey data yielded the test blueprint, which specifies how many exam questions are dedicated to each task in the Exam Outline.
Item (Test Question) Writing
RESNA then formed another panel of subject matter experts to write a bank of test questions to represent the content areas outlined on the test blueprint. The test items were then reviewed and revised as necessary.
After the test was assembled with new items in accordance with the blueprint, about 100 candidates piloted the new exam. Knapp and Associates then facilitated a cut-score study to determine the passing score for the exam. A panel of subject matter experts rated the difficulty level of each test item in respect to basic competency, and then the panelists' aggregated ratings data were analyzed to identify the range of statistically acceptable cut-off scores. After reviewing pilot testers' exam scores in relation to the identified cut-score range, RESNA's Professional Standards Board approved a passing cut-off score.
Later, as new questions were developed and pretested, psychometric consultant Prometric constructed new exam forms that were statistically equated to be equivalent in difficulty to the exam analyzed in the Cut-Score study.
Exam Update (updated ATP exam launches July 1, 2016)
in 2014, RESNA conducted a new job analysis study (again facilitated by Knapp and Associates) to ensure that the ATP exam continues to test the knowledge and skills that are typically demonstrated by ATPs on the job. In July of 2015, The Ohio State University's (OSU) Center on Education and Training for Employment (CETE) facilitated a Task Analysis of the new Exam Content Outline, in which a panel of subject matter experts provided detailed guidance to the volunteers who would assemble in the following months to write new exam items. Two groups of subject matter experts later convened to write and review new exam questions. The updated exam will launch July 1, 2016. Score reports will be delayed for candidates who test between July 1, 2016, and October 30, 2016. Following this first round of testing, RESNA's consulting psychometrician will review the exam performance data and then hold a workshop in which a group of subject matter experts will determine a fair cut-off passing score for the new exam. Candidates who test between July 1 and October 30, 2016 are expected to receive their score reports in late November or early December.