Data-analytic strategies for examining the effectiveness of daily interventions

• Published in: Journal of occupational and organizational psychology Vol. 88; no. 3; pp. 587 - 622
• Main Authors: Lischetzke, Tanja, Reis, Dorota, Arndt, Charlotte
• Format: Journal Article
• Language: English
• Published: Leicester Blackwell Publishing Ltd 01.09.2015; British Psychological Society
• Subjects: Behavior change; Behavior modification; Cognitive models; daily interventions; Data analysis; diary studies; ecological momentary interventions; effectiveness; Effectiveness studies; Employees; Evaluation; Everyday life; Intervention; multilevel models; Organizational behavior; Organizational effectiveness; Work experience; Work place; Workplaces
• ISSN: 0963-1798; 2044-8325
• DOI: 10.1111/joop.12104

Interest in the use of ecological momentary interventions – that is, interventions that are implemented in participants' everyday lives – to change experiences and behaviours has grown rapidly in recent years. In particular, EMIs in which the intervention is delivered on a daily basis (daily interventions) can be easily combined with daily diary studies to analyse intervention effects on dependent variables (DVs) that fluctuate over time. This article first provides a typology of research designs to classify daily intervention studies according to (1) the type of assessment of the DV (global assessment on a small number of fixed occasions, daily assessment, and repeated assessment within days) and (2) the type of control (within‐subjects vs. between‐subjects designs) used in the studies. We then demonstrate how multilevel models can be used to examine the effects of a daily intervention on the mean levels of the DV. We differentiate between general effectiveness, differential effectiveness, and conditional effectiveness and show how these effects are represented in the models. As an illustration, we apply some of the models to data from a daily intervention workplace study (N = 51 full‐time employees) that focused on the effects of savouring exercises on calm mood and vigour. Practitioner points In work and organizational psychology, a recent trend in intervention study designs is to implement an intervention on a daily basis and incorporate a daily diary assessment approach. To examine intervention effectiveness in pre‐/post‐designs, multilevel models are the method of choice because they circumvent the shortcomings of ANOVA approaches. In intervention studies with daily assessments, multilevel models can be applied to examine the intervention's general effectiveness and differential effectiveness (i.e., Who benefits more from the intervention?). Daily intervention studies with repeated assessments within days allow researchers to examine conditional effectiveness (i.e., On which days is the intervention more effective?).