Test error & score confidence (2)

In this unit, you will learn how to determine whether differences in psychometric test scores are likely to be true differences or simply due to measurement error. Dr Tyler explains how to use the standard error of measurement and the standard error of difference to judge the certainty of observed score differences, either between two tests for one candidate or between two candidates on a single test. You will also gain practical advice on interpreting results and communicating confidence levels to decision makers.

After this talk, you will:

• Understand what the standard error of difference (SE diff) is and how it relates to the standard error of measurement (SEM).

• Know how to calculate the standard error of difference between two test scores for the same candidate or between two candidates on the same test.

• Be able to interpret whether observed differences between test scores are likely to be true differences or simply due to error.

• Recognise the importance of not relying solely on raw score differences when making decisions based on test results.

• Understand the critical values needed before concluding there is a true difference between scores.

• Recall and apply the rule of thumb regarding standard deviations to estimate confidence in true score differences.

• Realise the significant impact that misinterpreting test scores and ignoring error can have on people’s careers and well-being.

• Appreciate the importance of using other sources of evidence alongside psychometric test results in decision-making.

• Be aware of your responsibility to ensure accuracy and fairness when interpreting and reporting psychometric test results.

• Be prepared to move on to the next topic of psychometric test validity, having built a solid understanding of reliability and confidence in test scores.