INDIAJob analysis survey
The starting point for the exam development process is a job analysis survey that is conducted every five years to identify the skills and knowledge areas currently being used in the field. In order to create appropriate items for the survey instrument, an advisory board is appointed by the sponsoring ASQ division.
This committee, like all of the subsequent exam development committees, is composed of ASQ members who work in the area to be tested and are already certified. In the case of brand new certification exams, ASQ volunteer member leaders who are subject matter experts in the field to be tested serve on the committee. These volunteers identify typical job responsibilities (what people do on the job) and the knowledge bases required (what people need to know in order to perform their job). The committee also reviews a wide variety of textbooks to ensure that these topics are covered in reference material that is readily available.
In the case of the survey advisory board, its work results in a questionnaire that asks respondents to rate each item in the survey in terms of criticality (how important is this task or knowledge) and frequency (how often is this task performed or knowledge used). The survey is piloted to ensure that the questionnaire itself is as clearly presented and as easy to complete as possible.
Member input
Once the survey instrument has been approved and all necessary revisions made, it is sent to a sample of certified ASQ members who either work in the area to be tested (as identified by job title) or supervise employees who perform the tasks identified in the survey. Except for the deliberate job title selection, the 2,000 plus certified ASQ members who receive the survey are randomly selected across geographic locations and industry types to ensure that no one industry or region skews the results of the survey.
The data from the survey are then analyzed and a set of recommended tasks and knowledge areas is presented in a summary report, which is submitted to the sponsoring division. The rule for recommendation is that a task or knowledge must score at or above the midpoint: in other words, earn a score of at least 2.5 on a scale of 1 to 5, for criticality or frequency. The sponsoring division approves the results, which are then submitted to the Certification Board for final approval.
After the approval process, a BOK committee is formed. This committee of subject matter experts, includes some members of the advisory committee and again has representatives from across a wide demographic spectrum.
The BOK committee’s primary task is to translate the job analysis results into meaningful categories that can be tested. Committee members use a variety of quality tools to facilitate this process, and the final result forms an outline of major topic areas, supported by appropriate subtopics.
The committee then determines how many questions will be asked in each area of the BOK, based on the importance of the topic as well as the depth of testable material in that topic. The committee again uses tools such as multivoting and prioritization matrices to help guide the decision-making process.