How to Judge a Psychological Test
As long as the number of potential job applicants is greater than the number of job openings, selection of some nature will be required. Moreover, a poorly designed test or improper test use may keep you from hiring the best candidates. The aim of this article is to ensure that the assessment instruments used during the selection process help to decrease this error.
Most test developers and test users have good intentions, yet due to inadequate training, there is some considerable misuse of test data. This article has been written to warn about bad use of tests in general and about "nonsense" claims that some commercially oriented people may make about the tests they put on the market. Most of these comments apply to any test, so anyone publishing one of these tests should at least have a reasonable answer to these comments. Their answers should explain what they do to address these specific issues.
Test design problems can be classified into two groups. The first group has to do with the construction of the test itself and the theoretical basis that underlies the test. A test without a solid theoretical foundation cannot deliver good results. When a test has a solid theoretical foundation, the psychometric analysis of the test will confirm these foundations and will point out the problem areas in the use of this test. Let us go a bit deeper into these two issues:
There are numerous ways to misuse test date. In their book, Eyde et. al. (1993) indicate 86 specific elements and 7 broad factors that represent common problems in test use. This is a summary of the problems mentioned:
|INPUT PROCESS||TABULATION PROCESS||POSSIBLE SCORING PROBLEMS|
|Test taken via pen-and-paper||Results tabulated by a person||Possible data entry error and tabulation error|
|Test taken via pen-and-paper||Results tabulated by a computer||Possible data entry error|
|Test taken online||Results tabulated by a computer||Virtually no possible scoring problems|
Problems with Test-Takers
Lack of Self-Knowledge - Test-takers might not know enough about themselves to accurately answer the questions. Overestimating or underestimating one's own abilities is a common challenge.
Falsification - Psychological tests are often quite transparent, and it seems obvious to many observers that job applicants would not willingly report undesirable behaviors that would ruin their chances for employment (e.g. Goldberg et al. 1991). You should always carefully examine a test before you use it for your organization. Is there right and wrong answers? Will an educated candidate be able to know what you want to hear? For example, some tests ask: At work, are you:
Of course everyone will pick answer 4! Test designers should be able to prove to you that they can eliminate falsification. This is a major problem, because the number of "false positives" in test results (this happens when people that pass the test, and then do not perform as expected).
This article should raise some questions regarding responsible and accurate test use. There are good tests out there, but they must answer to these issues. The answers to these questions have to come from test developers. If you can't get those answers right away, al least get a commitment that these answers should be available by the time you end a pilot project.
Articles on Specific Metaprograms
|Buy from Amazon:
7 Steps to Emotional Intelligence
Patrick E. Merlevede, Denis Bridoux
Mastering Mentoring and Coaching with Emotional Intelligence
Patrick E. Merlevede, Denis Bridoux
Our Research page will teach you the research background of jobEQ, why thorough research is important, and how we use it to your advantage.
Many of the jobEQ articles discuss specific metaprogram patterns that the iWAM and our other tools measure. Click here to find out more about the patterns.
last modified: 2006/Aug/07 15:56 UTC