Sample size determination for conducting a pilot study to assess reliability of a questionnaire

• Published in: Restorative dentistry & endodontics Vol. 49; no. 1; p. e3
• Main Authors: Bujang, Mohamad Adam, Omar, Evi Diana, Foo, Diana Hui Ping, Hon, Yoon Khee
• Format: Journal Article
• Language: English
• Published: Korea (South) The Korean Academy of Conservative Dentistry 01.02.2024; 대한치과보존학회
• Subjects: Statistical Review; 치의학; Pilot study; Questionnaire; Sample size; Reliability
• ISSN: 2234-7658; 2234-7666
• DOI: 10.5395/rde.2024.49.e3

This article is a narrative review that discusses the recommended sample size requirements to design a pilot study to assess the reliability of a questionnaire. A list of various sample size tables that are based on the kappa agreement test, intra-class correlation test and Cronbach's alpha test has been compiled together. For all calculations, type I error (alpha) was set at a maximum value of 0.05, and power was set at a minimum value of 80.0%. For the kappa agreement test, intra-class correlation test, and Cronbach's alpha test, the recommended minimum sample size requirement based on the ideal effect sizes shall be at least 15, 22, and 24 subjects respectively. By making allowances for a non-response rate of 20.0%, a minimum sample size of 30 respondents will be sufficient to assess the reliability of the questionnaire. The clear guideline of minimum sample size requirement for the pilot study to assess the reliability of a questionnaire is discussed and this will ease researchers in preparation for the pilot study. This study provides justification for a minimum requirement of a sample size of 30 respondents specifically to test the reliability of a questionnaire.