This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Cahan, S. Search for Related Content Social Bookmarking                         What's this?

Research news and Comment: Statistical Significance is not a "Kosher Certificate" for Observed Effects: A Critical Analysis of the Two-Step Approach to the Evaluation of Empirical Results

In order to prevent misleading conclusions based on spurious observed effects—especially seductively large ones—Robinson and Levin (1997) suggested a two step approach to the reporting and evaluation of empirical results. According to the two-step model, the evaluation of the magnitude and substantive significance of obtained effects should be conditional upon their statistical significance: Author should first indicate whether the observed effect is statistically improbable, and only if it is should they then indicate how large or important it is. The purpose of their paper is (a) to show that even through the underlying intention—to prevent unwarranted evaluation of spurious observed effects—is a laudable one, the two-step approach is inappropriate for this purpose, and (b) to reiterate the preferred approach, namely increased sample size and computation of confidence intervals.

Educational Researcher, Vol. 29, No. 1, 31-34 (2000)
DOI: 10.3102/0013189X029001031


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				V. Perselli Troubling the Angels Revisited Cultural Studies <=> Critical Methodologies, May 1, 2008; 8(2): 224 - 245. [Abstract] [PDF]

				K. J. Klauer and G. D. Phye Inductive Reasoning: A Training Approach Review of Educational Research, March 1, 2008; 78(1): 85 - 123. [Abstract] [Full Text] [PDF]

				D. A. Byrnes, G. Kiger, and Z. Shechtman Evaluating The Use Of Group Interviews To Select Students Into Teacher-Education Programs Journal of Teacher Education, March 1, 2003; 54(2): 163 - 172. [Abstract] [PDF]

				J. K. Roberts and R. K. Henson Correction for Bias in Estimating Effect Sizes Educational and Psychological Measurement, April 1, 2002; 62(2): 241 - 253. [Abstract] [PDF]

				J. R. Levin and D. H. Robinson Research news and Comment: Rejoinder: Statistical Hypothesis Testing, Effect-Size Estimation, and the Conclusion Coherence of Primary Research Studies Educational Researcher, January 1, 2000; 29(1): 34 - 36. [Abstract] [PDF]