Reliability & validity in personality assessment

In this unit, you will revisit the key ideas of reliability and validity in the context of personality assessments. You will learn about the unique challenges of measuring these properties in personality tests, including the need to review reliability and validity for each scale separately, the importance of addressing issues like the criterion problem and curvilinear relationships, and the risks involved with multiple correlations and chance findings. The session also touches on best practices, such as using multiple regression for validity and checking for proper corrections when many correlations are reported.

After this talk, you will:

• Understand how reliability and validity apply specifically to personality assessment.

• Know why reliability is generally lower for personality tests compared to ability tests.

• Recognise the importance of reporting reliability for each scale rather than using an average across scales.

• Be able to explain two ways of reporting validity: on a scale-by-scale basis and by using multiple regression.

• Understand the importance of forming a hypothesis before running statistical analyses.

• Be aware of the criterion problem and why the reliability and validity of the criterion itself also matters.

• Know what 360 data is and why it can improve the assessment of test validity.

• Recognise the problem of multiple correlations and the importance of applying a correction for shrinkage.

• Understand the difference between linear and curvilinear relationships, and how this affects personality test validity.

• Be able to spot issues such as chance effects and “fishing trips” in published personality test data.

• Appreciate the need to critically evaluate the quality and ethics of test publishers.