Assessing psychometric test validity (2)

In this unit, you will learn about the concept of validity in psychometrics, including why reliability is a crucial foundation before considering validity. The session covers various types of validity—such as face, faith, content, construct, and criterion-related validity—explaining how they are assessed and why some are more meaningful than others. You’ll also hear about potential biases in validity studies and the importance of evaluating whether a test is truly fit for its intended purpose.

After this talk, you will:

• Understand the difference between concurrent and predictive criterion-related validity.

• Know the strengths and weaknesses of concurrent and predictive approaches to validity.

• Be aware that predictive validity requires following up with respondents over time, impacting resources and logistics.

• Recognise the role of face validity and why candidate buy-in is important, even if a test has low face validity.

• Know that construct and criterion-related validity are statistically assessed and should be checked in the test manual.

• Understand what consequential validity is and why it relates to the broader effects of testing.

• Know the importance of checking the context and sample of validity studies to judge their relevance for your needs.

• Be able to explain the significance of sample size, the size of the correlation, and statistical significance when interpreting validity evidence.

• Understand that strong correlations with large sample sizes are more reliable and less likely to have occurred by chance.

• Be able to use the “coefficient of determination” to estimate how much of performance a test score predicts.

• Know to look for information such as sample size, type of validity, and statistical significance when reviewing test manuals for evidence of validity.

• Understand that not all statistically significant correlations are practically useful, especially if they only predict a small amount of the variance.