Psychometric test reliability (1)

In this unit, you will learn about the importance of psychometric test reliability, the basic concept of correlation and how it relates to reliability, and the different levels of measurement used in test manuals. You’ll also be introduced to the distinction between reliability and validity, understand which statistics are appropriate for each type of data, and consider what makes a test suitable to use in practice. The unit aims to give you the skills to assess tests confidently and to recognise the value of consistency in test scores.

After this talk, you will:

• Understand the importance of reliability and validity in psychometric testing.

• Be able to explain the difference between reliability (consistency) and validity (fitness for purpose).

• Recognise why it’s vital not to use tests just because someone else recommends them, but to assess their psychometric properties yourself.

• Appreciate that correlation measures the relationship between two variables and is key to understanding reliability.

• Know that correlation values range from -1 (perfect negative) to +1 (perfect positive), with 0 meaning no relationship.

• Understand that correlation does not imply causation.

• Be familiar with the different levels of measurement: nominal, ordinal, and interval, and know which correlation statistics apply to each.

• Be able to spot when the incorrect correlation statistic is used for a type of data and understand why that matters.

• Recognise the importance of reviewing the quality of data and statistics used in test manuals before choosing a psychometric test.

• Be able to define reliability as the consistency of a test or measurement.

• Understand that reliability is linked to the confidence we can have that a test score reflects a person's true score.

• Be aware that in reliability analysis, negative correlations are rare and usually a sign of a problem, whereas in validity studies, negative correlations can be acceptable.

• Be prepared to consider real test data and to evaluate reliability statistics in practice.

• Start thinking critically about what makes one test more reliable than another, and the factors (such as test length and number of items) that affect reliability.