RCT Plausibility Scale
After a few intro paragraphs, I want to present a scale of probability to estimate a value of a “prior” to plug into the formula for obtaining a Bayes Factor. The scale can help to estimate a value, but will still rely on an estimate, the non-quantitative element in Bayesian simulations. However, the checklist may at least provide some objective bases on which to hang a value, and that value would actually make a semi-quantitative statement of its own. Although that value would retain some subjective quality, it would at least be backed by known quantities and laws of nature.
Begging your patience again, I became aware of this problem in 1999 when asked to moderate an online (BioMednet.com) debate on “CAM” among 4 physicians. My role soon morphed into participant-debater when I could not get all to agree on what I thought was obvious common ground to proceed with the discussion – that 1) concepts that violate scientific laws do not have to be subjected to clinical trial (RCT) and that trial results had to be interpreted in light of previous knowledge; and 2) clinical trials could not constitute adequate evidence in the absence of plausibility because their results were too varied and inconsistent. The matter was p-recipitated by systematic reviews (SRs) showing efficacy of acupuncture in back pain. I was truly surprised when one of the participants (Dr. Edzard Ernst) assured me that indeed, RCTs were now the gold standard for efficacy. The debate went downhill from there.