Squeezing More Information from Experiments: A Bayesian Approach

Mark Pitt and Jay Myung

Abstract

Computational models of cognition are intended to describe or explain aspects of cognitive functioning and behavior, such as how we make decisions, how we remember, and how we read. The precision afforded by a mathematical formalization can be harnessed to improve empirical tests of the models themselves, increasing the efficiency of data collection in experiments by requiring fewer trials and increasing the informativeness of the data collected, possibly leading to more decisive outcomes. In this talk, I will provide an overview of our research program into developing methods for optimizing the design of experiments. Working within a Bayesian inference framework, the problem is cast as a search through the space of all possible experimental designs for the given models to identify the one design whose posterior probability of satisfying some utility function (e.g., discriminating models, estimating parameters of a model) is highest. Examples of its application in a few content areas will be illustrated, and outstanding problems as well as future applications will be discussed.