Assessing psychometric test validity (1)

In this unit, you will learn about the concept of validity in psychometrics, including why reliability is a crucial foundation before considering validity. The session covers various types of validity—such as face, faith, content, construct, and criterion-related validity—explaining how they are assessed and why some are more meaningful than others. You’ll also hear about potential biases in validity studies and the importance of evaluating whether a test is truly fit for its intended purpose.

After this talk, you will:

• Understand that validity in psychometrics is about whether a test is fit for its intended purpose.

• Know that reliability must always be checked and assured before considering a test’s validity.

• Be aware of the different types of validity, including faith validity, face validity, content validity, construct validity, and criterion-related validity.

• Realise that faith validity is not a proper type of validity and should not be relied upon when selecting or using tests.

• Know that face validity refers to whether a test appears, on the surface, to measure what it claims to—but that this is not always connected to how well a test actually works.

• Understand that content validity considers whether the test covers the entire domain it claims to assess and if its items reflect real-world requirements.

• Recognise that construct validity is assessed by comparing the new test with established tests of the same construct, looking for strong correlations.

• Understand that criterion-related validity is about whether test scores can predict relevant outcomes or performance, either concurrently or in the future.

• Be aware that validity cannot be measured with a single “aspirational” figure, as is sometimes done with reliability.

• Know that biases in validity studies can affect the observed validity figures, either deflating or, in rare cases, inflating them.

• Appreciate the importance of moving beyond intuition or recommendations and instead basing decisions on proper evaluation of test reliability and the different types of validity.