Accounting for site-selection bias in before–after studies for continuous distributions: Characteristics and application using speed data

• Published in: Transportation research. Part A, Policy and practice Vol. 49; pp. 256 - 269
• Main Authors: Kuo, Pei-Fen, Lord, Dominique
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier Ltd 01.03.2013; Elsevier
• Subjects: ANCOVA; Applied sciences; Before–after study; Exact sciences and technology; Ground, air and sea transportation, marine construction; Normal distribution; Policy; Regression-to-the-mean; Road transportation and traffic; Site selection bias; Transportation planning, management and economics; Vehicle speed; USA, Florida; Policy; ANCOVA; Regression-to-the-mean; Before–after study; Vehicle speed; Site selection bias; Normal distribution; Script; Speed; Data analysis; Before and after study; Simulation; Characteristics; Site selection; Before-after study; Traffic management; Traffic safety; Step by step method; Road traffic; Computing method
• DOI: 10.1016/j.tra.2013.01.023
• ISSN: 0965-8564

► Before–after studies can be plagued by important methodological limitations. ► This study presents a method that can reduce selection effects. ► The study was accomplished using simulated and observed speed data. ► Entry criteria, between-subject and within-subject variances affect the bias. ► The proposed method provides a more precise estimate than the Naïve method. The before–after study is still the most popular method used by traffic engineers and transportation safety analysts for evaluating the effects of an intervention. Compared to the cross-sectional study, the before–after study has lower within-subject variability since it directly accounts for changes that have occurred at the study sites. However, although this kind of study may offer superior performance, it can still be plagued by important methodological limitations, which could significantly alter the study outcome. They include the regression-to-the-mean (RTM) and site-selection effects. The primary objective of this study consists of presenting a method that can reduce the selection effects when an entry criterion is used in before–after studies for continuous data (e.g. speed, reaction times, etc.), without relying on the use of a control group. The distribution of the data could follow a normal or lognormal distribution. The study objective was accomplished using simulated and observed speed data collected in Florida. The proposed method documented in this paper was compared to the Naïve, Control Group (CG) and the Analysis of Covariance (ANCOVA) methods. The simulation results show that the proposed method provides a more precise estimate than the Naïve method, as expected. In addition, the method performs better than the CG and the ANCOVA methods when similar control group data are not available. The results also show that higher entry criteria, lower between-subject variances, and higher within-subject variances cause higher selection biases. When traffic engineers and urban planners evaluate or compare different strategies, the proposed method can be applied to adjust naïve estimators of treatment effectiveness documented in previous studies without similar control group data.