How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria

• Published in: Behavior research methods Vol. 49; no. 2; pp. 513 - 537
• Main Authors: Lerche, Veronika, Voss, Andreas, Nagler, Markus
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.04.2017; Springer Nature B.V
• Subjects: Bayesian analysis; Behavioral Science and Psychology; Chi-square test; Cognitive ability; Cognitive Psychology; Comparative analysis; Computer Simulation; Contaminants; Decision Making; Diffusion; Diffusion models; Humans; Information processing; Mathematical models; Mental task performance; Models, Psychological; Optimization; Parameter estimation; Probability; Psychology; Reaction Time; Response time; Simulation; Mathematical models; Diffusion model; Fast-dm; Reaction time methods
• ISSN: 1554-3528; 1554-3528
• DOI: 10.3758/s13428-016-0740-2

Diffusion models (Ratcliff, 1978 ) make it possible to identify and separate different cognitive processes underlying responses in binary decision tasks (e.g., the speed of information accumulation vs. the degree of response conservatism). This becomes possible because of the high degree of information utilization involved. Not only mean response times or error rates are used for the parameter estimation, but also the response time distributions of both correct and error responses. In a series of simulation studies, the efficiency and robustness of parameter recovery were compared for models differing in complexity (i.e., in numbers of free parameters) and trial numbers (ranging from 24 to 5,000) using three different optimization criteria (maximum likelihood, Kolmogorov–Smirnov, and chi-square) that are all implemented in the latest version of fast-dm (Voss, Voss, & Lerche, 2015 ). The results revealed that maximum likelihood is superior for uncontaminated data, but in the presence of fast contaminants, Kolmogorov–Smirnov outperforms the other two methods. For most conditions, chi-square-based parameter estimations lead to less precise results than the other optimization criteria. The performance of the fast-dm methods was compared to the EZ approach (Wagenmakers, van der Maas, & Grasman, 2007 ) and to a Bayesian implementation (Wiecki, Sofer, & Frank, 2013 ). Recommendations for trial numbers are derived from the results for models of different complexities. Interestingly, under certain conditions even small numbers of trials ( N < 100) are sufficient for robust parameter estimation.