BCGInstitute: Serving the HR Community

Should we be validating obviously valid tests? As an example, should we validate a typing test for a data entry position? How would we proceed with such a validation of the test? Would we simply need to document that the test accurately measures words-per-minute and typing accuracy?

Would we just need to validate the use of the test results such a words-per-minute and accuracy measurements used as a minimum qualification for the position? If so, how would that be accomplished?

Hi Bippie:

I work with the testing software division at BCG and can give an answer from our perspective. The answer would be YES! There are a few aspects that I believe you should consider:

1. Does the content of the test match the content on the job? (format, frequency of letters to numbers, special characters used, on-screen or from a paper document, etc.)
2. Is the way it is keyed in the test similar to the way it is keyed on the job? (is it typed into a blank document, or does one tab from one text box to another?)
3. Is it a critical skill or ability? (is this a fundamental duty that a person must possess in order to be successful on the job, or does a person simply need to fill out a few forms each day and a hunt-and-peck typist would suffice?)
4. How frequently is this ability used?
5. Is better / stronger performance of this duty differentiating between good and better performers? (If someone can type 5 or 10 WPM faster than another, will they be able perform significantly better on the job than someone else? If they type eight hours a day, the answer may be yes, whereas if they type only fifteen minutes a day, the answer might be no.)
6. Is this ability learned in a brief orientation, or something that is normally trained on the job? (if not, it probably is a good ability to test for... if so, you should not test for it...)

Once these issues are established by a pool of subject matter experts, then you can move to the next step, which would be to create a defensible, job related cutoff score. What, in the opinion of your subject matter experts, is a minimum qualifying score for someone entering into this position on the first day of the job?

Even though it may be completely obvious to everyone that measuring typing skills for a typist position (for instance) is valid, the real value comes when someone complains that the process was not fair. Once your recruitment practice is questioned, it is very comforting to reach into a file drawer and grab your documentation that shows exactly how you validated the test, and why the cutoff score you used was a fair one.

We actually automate this process in the testing software that our company has available. We call it a "validation wizard" and it makes the process very easy to facilitate. Please let me know if we can provide you with a demo or sample.

Best regards,

Mike

Hi Bippie.

I would like to add my voice to Mike's. I completely agree with what he said. The key issue when a duty is "obvious" such as typing for a typist, is the level of quality required (how many error's are permitted? Is spell check employed or is accuracy absolutely necessary?) How fast must the typist be able to type? The answers to these questions are not obvious at all.

A job analysis will allow you to make sure that the correct emphasis is being placed on the skill being tested, identify whether it is critical or just highly desirable, and will help you get the kinds of information needed to set pass points. Also, it will help you determine whether the job is similar across all geographic locations, shifts, etc.

The process Miked described is sound and will help ensure that you would be protected in the face of possible litigation.

Jim

I also suggest that you might look at Question 38 in the Questions and Answers portion of the federal Uniform Guidelines on Employee Selection Procedures (http://www.uniformguidelines.com/questi ... swers.html) which states:

"38. Q. Can a user rely upon written or oral assertions of validity instead of evidence of validity?

A. No. If a user's selection procedures have an adverse impact, the user is expected to produce evidence of the validity of the procedures as they are used. Thus, the unsupported assertion by anyone, including representatives of the Federal government or State Employment Services, that a test battery or other selection procedure has been validated is not sufficient to satisfy the Guidelines."

This shows that one must have evidence of validity in order for a selection device to be considered "valid."

Jim Kuthy, Ph.D.
Principal Consultant
Biddle Consulting Group, Inc.

Well . . . . I am glad you have the time to invest in such a validation project. And I guess we need some back-up for our decisions - but in my minds-eye a simple content validation centered statement would be fine backup.

Smaller agencies and company's do not always have a validation program and yet do not get in trouble because of giving typing tests for positions that type. I would use my time on assessments that have a lot of ROI.

Peter Rogosin, President
Job Analysis Training Group/ Publicpersonnel.com

You are right that this example is one of the least-likely situations to be challenged, but it is still best to evaluate the content. I believe that making a "simple content validation centered statement" (as you suggest) would likely include a review of the six points suggested earlier (see my January 12 post) and possibly others, as well. It would probably be overkill to begin with a job analysis and work through the process just to validate the keyboarding test, though doing so and ending up with a valid selection process (including validation of the keyboarding test) would be the right thing to do. There are many benefits to performing a job analysis, if time allows.

You only need the automobile insurance if you get into an accident. In this case, if the selection process (and specifically the test) is challenged, having gone through some process (rather than none at all) and having documented the results will help to minimize an employer's exposure and ensure fairness for the applicants (and it seems that this is the point you were making, as well.) There is nothing wrong with either of these if an employer can make the time to do so.