All Bachelors are Unmarried Men (p <  0.05)

• Published in: Quality & quantity Vol. 42; no. 1; pp. 53 - 73
• Main Author: Smedslund, Geir
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.02.2008; Springer; Springer Nature B.V
• Subjects: Conceptualization; Empirical Methods; History, theory and methodology; Hypotheses; Marriage; Methodology; Methodology of the Social Sciences; Migraine; Psychological Research; Psychology; Quantitative analysis; Questionnaires; Research Subjects; Researchers; Senses; Sensory perception; Single Persons; Social research; Social Sciences; Social theory; Sociology; Tests; conceptual analysis; nonempirical; significance testing; quantitative analysis; pseudoempirical; Quantitative Analysis; Test; Empirical Research
• ISSN: 0033-5177; 1573-7845
• DOI: 10.1007/s11135-006-9036-4

This paper adds to the list of criticisms against null hypothesis significance testing (NHST). I argue that when researchers do not analyze the conceptual relations among their variables, they may fail to distinguish between logical implications and empirical relations. It does not make sense to use significance testing on hypotheses involving conceptually related phenomena. The widespread lack of conceptual clarification also leads to very small effect sizes in psychology because it causes study participants to understand the stimulus material in different ways. Therefore, they answer in an inconsistent way. Researchers show an extremely low degree of ambition when they seek to show that psychological phenomena differ from chance, or when they try to disprove a hypothesis claiming that a psychological phenomenon does not exist. I see significance testing as a poor solution to the problem of tiny effect sizes in psychology. I recommend that psychological researchers be more explicit both about their main hypotheses and their auxiliary hypotheses. As examples, I analyse all quantitative articles in Issue 1, 2005 of the Journal of Health Psychology.