Newborn and Infant Nursing Reviews
Volume 10, Issue 1 , Pages 55-59 , March 2010

Retrospective Statistical Power: Fallacies and Recommendations

  • Lihshing Leigh Wang, PhD

      Affiliations

    • Corresponding Author InformationAddress correspondence to Lihshing Leigh Wang, PhD, Educational Studies Program, School of Education, University of Cincinnati, Cincinnati, OH 45221.

References 

  1. Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York: Academic; 1988;
  2. Cohen J. A power primer. Psychol Bull. 1992;112:155–159
  3. Cohen J. The earth is round (p < .05). Am Psychol. 1994;49:997–1003
  4. Levin JR, Robinson DH. Rejoinder: statistical hypothesis testing, effect size estimation, and the conclusion coherence of primary research studies. Educ Res. 2000;29:34–36
  5. Onwuegbuzie AJ, Leech NL. Post hoc power: a concept whose time has come. Underst Stat. 2004;3:201–230
  6. Hubbard R, Lindsay RM, Why P. values are not a useful measure of evidence in statistical significance testing. Theory Psychol. 2008;18:69–88
  7. Kline RB. Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research. Washington, DC: American Psychological Association; 2004;
  8. Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med. 1994;121:200–206
  9. Robinson DH, Levin JR. Reflections on statistical and substantive significance, with a slice of replication. Educ Res. 1997;26:21–26
  10. In:  Rothstein HR,  Sutton AJ,  Borenstein M editor. Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. Chichester (UK): Wiley; 2005;
  11. Rotton J, Foos PW, Vanmeek L, Levitt M. Publication practices and the file drawer problem – A survey of published authors. J Soc Behav Pers. 1995;10:1–13
  12. Erturk SM. Retrospective power analysis: when?. Radiology. 2005;237:743
  13. Hoenig JM, Heisey DM. The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55:19–24
  14. O'Keefe KJ. Post hoc power, observed power, a priori power, retrospective power, prospective power, achieved power: Sorting out appropriate uses of statistical power analyses. Commun Methods Meas. 2007;1:291–299
  15. Smith SD. Statistical tools in the quest for truth: hypothesis testing, confidence intervals, and the power of clinical studies. Ophthalmology. 2008;115:423–424
  16. Fidler F, Cumming G. The new stats: attitudes for the 21st century. In:  Osborne JW editors. Best Practices in Quantitative Methods. Thousand Oaks (Calif): Sage; 2008;p. 1–12
  17. Schmidt FL, Hunter JE. Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In:  Harlow LA,  Mulaik SA,  Steiger JH editor. What If There Were No Significance Tests?. Mahwah (NJ): Lawrence Erlbaum; 1997;p. 37–64
  18. American Educational Research Association . Standards for reporting on empirical social science research in AERA publications. Educ Res. 2006;35:33–40
  19. American Psychological Association . Publication Manual of the American Psychological Association. 6th ed.. Washington, DC: American Psychological Association; 2009;
  20. Mulaik SA, Raju NS, Harshman RA. There is a time and a place for significance testing. In:  Harlow LL,  Mulaik SA,  Steiger JH editor. What If There Were No Significance Tests?. Mahwah (NJ): Lawrence Erlbaum; 1997;p. 65–115
  21. Wainer H, Robinson DH. Shaping up the practice of Null Hypothesis Significance Testing. Educ Res. 2003;32:22–30
  22. Yuan K-H, Maxwell S. On the post hoc power in testing mean differences. J Educ Behav Stat. 2005;30:141–167
  23. Thomas L. Retrospective power analysis. Conserv Biol. 1997;11:276–280
  24. Zumbo BD, Hubley AM. A note on misconceptions concerning prospective and retrospective power. The Statistician. 1998;47(Part 2):385–388
  25. Gillett R. Post hoc power analysis. J Appl Psychol. 1994;79:783–785
  26. Thomas L, Krebs CJ. A review of statistical power analysis software. Bull Ecol Soc Am. 1997;78:126–139
  27. Sedlmeier P, Gigerenzer G. Do studies of statistical power have an effect on the power of studies?. Psychol Bull. 1989;105:309–316
  28. Sun S, Pan W, Wang L. Rethinking Observed Power: Concept, Practice, and Implications. Manuscript submitted for publication in methodology: European Journal of Research Methods for the Behavioral and Social Sciences; 2009.
  29. Thompson B. Foundations of Behavioral Statistics: An Insight-Based Approach. New York: Guilford; 2006;
  30. Finch S, Cumming G, Thomason N. Reporting of statistical inference in the Journal of Applied Psychology: little evidence of reform. Educ Psychol Meas. 2001;61:181–210
  31. Gerard PD, Smith DR, Weerakkody G. Limits of retrospective power analysis. J Wildl Manag. 1998;62:801–807
  32. Oakes JM, Feldman HA. Statistical power for nonequivalent pretest-posttest designs. Eval Rev. 2001;25:3–28
  33. Levine M, Ensom MHH. Post hoc power analysis: an idea whose time has passed?. Pharmacotherapy. 2001;21:405–409
  34. Goodman SN. A comment on replication, P-values and evidence. Stat Med. 1992;11:857–859
  35. Matcham J, McDermott MP, Lang AE. GDNF in Pakinson's disease: the perils of post-hoc power. J Neurosci Methods. 2007;163:193–196
  36. Hogarty KY, Kromrey JD. RETR_PWR: an SAS macro for retrospective statistical power analysis. Behav Res Methods Instrum Comput. 2003;35:585–589
  37. Froman T, Shneyderman A. Replicability reconsidered: an excessive range of possibilities. Underst Stat. 2004;3:365–373
  38. Steiger JH, Fouladi RT. Noncentrality interval estimation and the evaluation of statistical models. In:  Harlow LL,  Mulaik SA,  Steiger JH editor. What If There Were No Significance Tests?. Mahwah (NJ): Erlbaum; 1997;p. 221–257
  39. Taylor DJ, Muller KE. Computing confidence bounds for power and sample size of the general linear univariate model. Am Stat. 1995;49:43–47
  40. Wilkinson L, Task Force on Statistical Inference . APA Board of Scientific Affairs. Statistical methods in psychology journals: guidelines and explanations. Am Psychol. 1999;54:594–604
  41. Colegrave N, Ruxton GD. Confidence intervals are a more useful complement to nonsignificant tests than are power calculations. Behav Ecol. 2003;14:446–447
  42. Overall JE. Classical statistical hypotheses within the context of Bayesian theory. Psychol Bull. 1969;71:285–292

PII: S1527-3369(09)00180-9

doi: 10.1053/j.nainr.2009.12.012

Newborn and Infant Nursing Reviews
Volume 10, Issue 1 , Pages 55-59 , March 2010

Article Tools

* Image Usage & Resolution