Our Test Development Process
The first thing you should know about our test development process is that it follows very strict professional principles and guidelines set forth by the human resources industry, specifically:
- The American Psychological Association’s Standards for Educational and Psychological Testing.
- The United States Equal Employment Opportunity Commission’s Uniform Guidelines on Employee Selection Procedures.
- The Society for Industrial and Organizational Psychology’s Principles for the Validation and Use of Personnel Selection Procedures.
You can rest assured that every test we develop undergoes a comprehensive and systematic process to ensure its validity. This process relies on two parts: content relevance and criterion relatedness.
Our test development process always starts with a thorough job analysis. This is when we gather information from subject-matter experts (SMEs). SMEs are the men and women who have been doing the job for a number of years — and doing it well — in departments and organizations located all over the country. The information we gather is very important to ensuring that the content of the test is relevant to the job. There are three steps to this process:
- Job Analysis Questionnaire. SMEs complete a job analysis questionnaire, which asks them to rate three things:
— How important specific tasks are to the job.
— How frequently those tasks are performed.
— How important the knowledge, skills, abilities, and personal characteristics (KSAPs) listed are to performing the job effectively.
- Data Analysis. Once all the data has been collected from the questionnaires, we analyze it to determine the most important KSAPs needed to do well in that particular job.
- Test Blueprint. Our analysis provides us with the information we need to create a blueprint for developing test items.
One Step Further
When developing promotional tests, which are rank-specific tests that help you assess the promotional potential of your employees, we take the content relevance part of the development process one step further.
After the tests are developed, we ask a group of SMEs (for promotional tests, they are supervisory and administrative-level employees) to review the test questions, and for each one, rate its clarity, importance and relevance to the job.
In this part of the validation process, which is completed for all our entry-level, multiple-choice tests, we take steps to ensure that test scores directly relate to a candidate’s job performance. There are four steps to the process:
- Gather Test Scores. Once the test is drafted, we ask men and women from departments and organizations around the country, who are currently doing the job, to take the test.
- Gather Performance Ratings. Next, we ask their direct supervisors to evaluate their job performance using a standard form provided by us.
- Test the Test. Then we evaluate the information, facts and figures gathered in steps 1 and 2 to identify patterns — specifically, to ensure that a candidate’s test score does, in fact, predict his or her job performance.
- Item Analysis. Finally, an item analysis is conducted to identify test items that are not working well. For example, test items that are statistically shown to be too easy or too difficult, or that fail to show a difference between those who do well on the test — and those who don’t.
Combined, the experts who make up our test development team have over 80 years of experience. They are recognized experts in their field, having received national awards for their work, been asked to speak at numerous conferences and workshops, served as expert witnesses and government advisors.
It takes the help of agencies like yours from all over the country to ensure we’re creating tests that are both effective and fair. Your expertise provides us with the essential data we need to continue developing assessment products of the highest standard. In short: we can’t do it without you.
There are three stages of the test development process where we need your help.
- Job Analysis. In this stage we ask you to complete a questionnaire that rates three things: how important specific tasks are to the job; how frequently those tasks are performed; and how important the knowledge, skills, abilities, and personal characteristics listed are to performing the job effectively.
- Subject-matter Expert (SME) Review. At this stage we ask you to review the test questions, and for each one, rate its clarity, importance and relevance to the job.
- Validation Studies. This stage has two parts: 1) we ask the men and women in your agency who are doing the job to take the test; and 2) we ask their direct supervisors to evaluate their job performance using a standard form provided by us.
What do you get for your participation besides the great feeling of knowing you’re helping public service agencies all across the country — even the world?
- You get discounts. For each stage in which your agency participates, you'll receive a discount of up to 30% off future test orders.
- You'll be better equipped to defend your jurisdiction's selection process if you help develop the test you're using.
- You'll hire better candidates. High-quality tests help you select high-quality candidates, saving your agency the costly expense of turnover while upholding the reputation of the hard-working men and women in your field.
- And it’s free! A test development and validation project of this scale done internally could cost your agency in excess of $100,000. Participation in our test development projects is free — plus, you'll earn discounts toward future test orders!
To learn more
and complete your participation form
, just click on a test title from the list of New Tests in Development.
Please note: discounts may not be utilized on the PSACS.
We’re always adding new tests to the development schedule — and every test development project comes with opportunities for great benefits and discounts for participating departments. You’ll find current information about where we need help here, so be sure to bookmark this page and check back often to see what’s new!
Current tests in development: