Impact of Referral Source and Study Applicants' Preference for Randomly Assigned Service on Research Enrollment, Service Engagement, and Evaluative Outcomes

• Published in: The American journal of psychiatry Vol. 162; no. 4; pp. 781 - 787
• Main Authors: Macias, Cathaleene, Barreira, Paul, Hargreaves, William, Bickman, Leonard, Fisher, William, Aronson, Elliot
• Format: Journal Article
• Language: English
• Published: Washington, DC American Psychiatric Publishing 01.04.2005; American Psychiatric Association
• Subjects: Adult; Ambulatory care; Bias; Biological and medical sciences; Community Mental Health Services - methods; Design of experiments; Employment, Supported - statistics & numerical data; Female; Health Services Research - methods; Health Services Research - standards; Humans; Male; Medical sciences; Mental Disorders - psychology; Mental Disorders - rehabilitation; Mental Disorders - therapy; Mental health services; Methodology. Experimentation; Motivation; Outcome Assessment, Health Care; Patient Participation - psychology; Patient Satisfaction; Preferences; Proportional Hazards Models; Psychiatry; Psychology. Psychoanalysis. Psychiatry; Psychopathology. Psychiatry; Random sampling; Randomized Controlled Trials as Topic - methods; Randomized Controlled Trials as Topic - standards; Referral and Consultation; Research Design - statistics & numerical data; Research Subjects - psychology; Researcher-Subject Relations - psychology; Salaries and Fringe Benefits - statistics & numerical data; Survival Analysis; Techniques and methods; Participation; Clubhouse; Health service; Individual preference; Assertive community treatment; Motivation; Methodological bias; Therapeutic schedule; Human; Evaluation; Data analysis; Attitude; Controlled therapeutic trial; Experimental study; Research program; Treatment; Randomized design; Follow up study; Preference; Risk factor; Mental disorder; Community treatment; Application method; Typology; Comparative study
• ISSN: 0002-953X; 1535-7228
• DOI: 10.1176/appi.ajp.162.4.781

OBJECTIVE: The inability to blind research participants to their experimental conditions is the Achilles' heel of mental health services research. When one experimental condition receives more disappointed participants, or more satisfied participants, research findings can be biased in spite of random assignment. The authors explored the potential for research participants' preference for one experimental program over another to compromise the generalizability and validity of randomized controlled service evaluations as well as cross-study comparisons. METHOD: Three Cox regression analyses measured the impact of applicants' service assignment preference on research project enrollment, engagement in assigned services, and a service-related outcome, competitive employment. RESULTS: A stated service preference, referral by an agency with a low level of continuity in outpatient care, and willingness to switch from current services were significant positive predictors of research enrollment. Match to service assignment preference was a significant positive predictor of service engagement, and mismatch to assignment preference was a significant negative predictor of both service engagement and employment outcome. CONCLUSIONS: Referral source type and service assignment preference should be routinely measured and statistically controlled for in all studies of mental health service effectiveness to provide a sound empirical base for evidence-based practice.