Background
Musical memory is a topic that has inspired creative and innovative experimental designs, including intrinsically motivating tasks such as Hooked on Music (Burgoyne et al., 2013) and TuneTwins (Li et al, 2023). With creative and innovative designs, however, come challenges to measurement. Even traditional set-ups like the old–new paradigm can lead to suprisingly complex statistical models (e.g., Müllensiefen and Halpern, 2014).
Aims
Our presentation will explain how a family of item-response models known as Rasch models (Rasch, 1960) can establish reliable, interval-scale measurements even for complex experimental designs. Rasch models enjoy a fundamental measurement property known as specific objectivity, and are the only probabilistic models to have this property and also a guarantee that their parameters have a monotonic relationship with a participant’s score for an experiment. If researchers with minimal statistical experience can design a scientifically valid scoring rule for their experiment, then a statistician can design a statistically reliable Rasch model to match it.
Main Contribution
The presentation will focus on Rasch models underlying two paradigms we have used to study memory: Hooked on Music, which incorporates response time and ‘feeling of knowing’ in addition to musical recall, and TuneTwins, which needs to handle variable, player-driven trial structures. We will explain our design considerations and demonstrate several modelling options for each paradigm, including how they might be adapted for other experiment types. Although empirical results are not the primary focus, we will also show examples of the model fits on pilot data.
Discussion and Conclusion
Reliable measurement is essential to understanding musical ability, and the Rasch framework is a powerful but surprisingly simple means to ensure it.
Musical memory is a topic that has inspired creative and innovative experimental designs, including intrinsically motivating tasks such as Hooked on Music (Burgoyne et al., 2013) and TuneTwins (Li et al, 2023). With creative and innovative designs, however, come challenges to measurement. Even traditional set-ups like the old–new paradigm can lead to suprisingly complex statistical models (e.g., Müllensiefen and Halpern, 2014).
Aims
Our presentation will explain how a family of item-response models known as Rasch models (Rasch, 1960) can establish reliable, interval-scale measurements even for complex experimental designs. Rasch models enjoy a fundamental measurement property known as specific objectivity, and are the only probabilistic models to have this property and also a guarantee that their parameters have a monotonic relationship with a participant’s score for an experiment. If researchers with minimal statistical experience can design a scientifically valid scoring rule for their experiment, then a statistician can design a statistically reliable Rasch model to match it.
Main Contribution
The presentation will focus on Rasch models underlying two paradigms we have used to study memory: Hooked on Music, which incorporates response time and ‘feeling of knowing’ in addition to musical recall, and TuneTwins, which needs to handle variable, player-driven trial structures. We will explain our design considerations and demonstrate several modelling options for each paradigm, including how they might be adapted for other experiment types. Although empirical results are not the primary focus, we will also show examples of the model fits on pilot data.
Discussion and Conclusion
Reliable measurement is essential to understanding musical ability, and the Rasch framework is a powerful but surprisingly simple means to ensure it.