There is a tradition in medical training called Journal Club. The first rule of Journal Club is you do not talk about Journal Club. In Journal Club, at least in the iterations in which I have participated, one article is selected by an attending, everyone reads it, then the strengths, weakness and applicability are discussed by the group. Usually a top notch, ground breaking article was the focus, one that had high potential clinical impact. But since they were good articles in good journals, there was not a lot to learn about in reguards to critical thinking. While the attending would put the article in context and maybe discuss some rudimentary statistics, there was little that was discussed about the quality of the study. The main take home from every study was to question the applicability of the results to populations that were not old, white males, since it seemed all the ground breaking studies back in the day were a VA Cooperative study of one sort or another.
As I remember it, there was not really a conceptual frame work with which to evaluate studies. Bayes theorem, and its application to clinical medicine was never explicitly discussed outside of testing, where you have to consider the prior plausibility of the patient having a disease before you can decide if the test results is a true positive or not. In Portland, Oregon, the chance that a Lyme serology is a false positive is much greater than a test done in Portland, Maine. Generally speaking in the information overload state that is the practice of medicine, clinical trials are generally taken at face value and tests are considered infallible. Which is a shame, as I wonder how much suboptimal medicine is inflicted on patients by not considering prior plausibility and how accurate a given test is in either ruling in or out a disease. There seems to be a whole industry built around treating patients with no risks for Lyme but have positive tests of doubtful provenance. We never discussed the prior plausibility and its effect on the outcomes of a studied treatment. (more…)