Item response theory demonstration

In this talk, you will learn about the differences between classical test theory and item response theory, using the example of a football try-out to illustrate each approach. You will see how classical test theory relies on administering the same set of tasks to everyone and scoring the total, while item response theory adapts the difficulty based on performance, making it more efficient. The session also discusses why item response theory can be slower to implement due to the need for large item banks and more complex statistics.

After this talk, you will:

• Understand the basic difference between classical test theory and item response theory.

• Know how classical test theory assesses ability by asking the same questions or tasks to everyone and simply adding up correct responses.

• Recognise that classical test theory often involves many repeated trials, which can lead to predictability and potential sharing of answers.

• Understand that item response theory is more efficient and adapts the difficulty level based on a person’s previous answers or performance.

• See that item response theory helps to quickly find the right difficulty level for each individual, without needing to ask every possible question.

• Know that item response theory requires large banks of questions and more advanced statistics, making it more costly and time-consuming to set up than classical test theory.

• Appreciate why item response theory can be more effective but is slower to catch on due to these challenges.

• Be able to explain how both approaches might be used in a practical setting, like trying out a football player for a team.