Dr. Joseph Sirven: The Drug Test No One Talks About
What’s the most important test in health that no one talks about? KJZZ medical commentator, Dr. Joseph Sirven, answers.
“Doc, I want that urine test that will tell me if the drug works.”
“I’m not sure what you’re referring to.” I said.
“The test, you know, that if you check the pee then you know if the medication actually works. I want to know what my pee test is.”
It finally hit me what he was referring to. Gently, I reassured him that the "P" test isn’t a urine analysis. My patient actually has a valid point. He was talking about the P threshold statistic for when a drug is approved by the FDA.
So what is a P value?
In medical research, the starting assumption when comparing say a new drug versus a placebo — a sugar pill — is that there is no difference between the two. The P value is the threshold that says a new medication is indeed better than placebo.
Currently the threshold is set at a P value of .05 which means that there is a 1 in 20 chance that the outcome is simply random and not related to the benefits of the new drug, test or whatever you are testing. The problem is that the P value somehow became the Holy Bible of clinical research.
Recently, the American Statistical Association sought feedback on lowering the P value threshold from .05 to .005 so that there is an even stronger burden of proof for new medical discoveries. This is huge and could have a dramatic impact on what healthcare could look like in the future.
If this shift is approved, it’s estimated that literally one-third of results from previous medical studies would now be called into question. Basically our understanding of what we believe to be clinically true, especially for therapies, diets, etc. may no longer be.
Personally, I welcome this change.
Strengthening the P value threshold may make research outcomes more clinically relevant. It may also curb expensive medications with great marketing but minimal improvement on what already exists. It could even make us better health consumers so that when we hear a headline, we’ll know whether it’s actually clinically significant not just statistically.
To paraphrase Benjamin Disraeli, There are three types of lies: “Lies, damn lies, and statistics.”
Dr. Joseph Sirven is a neurologist at the Mayo Clinic.