Posts Tagged p-value

Psychology Journal Bans Significance Testing

p-valuesThis is perhaps the first real crack in the wall for the almost-universal use of the null hypothesis significance testing procedure (NHSTP). The journal, Basic and Applied Social Psychology (BASP), has banned the use of NHSTP and related statistical procedures from their journal. They previously had stated that use of these statistical methods was no longer required but can be optional included. Now they have proceeded to a full ban.

The type of analysis being banned is often called a frequentist analysis, and we have been highly critical in the pages of SBM of overreliance on such methods. This is the iconic p-value where <0.05 is generally considered to be statistically significant.

The process of hypothesis testing and rigorous statistical methods for doing so were worked out in the 1920s. Ronald Fisher developed the statistical methods, while Jerzy Neyman and Egon Pearson developed the process of hypothesis testing. They certainly deserve a great deal of credit for their role in crafting modern scientific procedures and making them far more quantitative and rigorous.

However, the p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously. Further, the p-value is widely misunderstood. The precise definition is:

The p value is the probability to obtain an effect equal to or more extreme than the one observed presuming the null hypothesis of no effect is true.


Posted in: Basic Science, Clinical Trials, Science and Medicine

Leave a Comment (117) →

Beware The P-Value

Part of the mission of SBM is to continually prod discussion and examination of the relationship between science and medicine, with special attention on those beliefs and movements within medicine that we feel run counter to science and good medical practice. Chief among them is so-called complementary and alternative medicine (CAM) – although proponents are constantly tweaking the branding, for convenience I will simply refer to it as CAM.

Within academia I have found that CAM is promoted largely below the radar, with the deliberate absence of public debate and discussion. I have been told this directly, and that the reason is to avoid controversy. This stance assumes that CAM is a good thing and that any controversy would be unjustified, perhaps the result of bigotry rather than reason. It’s sad to see how successful this campaign has been, even among my fellow academics and scientists who should know better.

The reality is that CAM is fatally flawed in both philosophy and practice, and the claims of CAM proponents wither under direct light. I take some small solace in the observation that CAM is starting to be the victim of its own success – growing awareness of CAM is shedding some inevitable light on what it actually is. Further, because CAM proponents are constantly trying to bend and even break the rules of science, this forces a close examination of what those rules should actually be, how they work, and their strengths and weaknesses.


Posted in: Clinical Trials

Leave a Comment (179) →