I’m beginning to appreciate Seth Roberts’ point in my comments section about the “flaws in conventional scientific tests.” Not that I withdraw my objections to Roberts’ diet theories. But Roberts is right to say that the limitations of his “research” are shared by an awful lot of “conventional” science.
Consider the latest findings on lower back pain, dutifully reported by Reuters. A paper in the American Journal of Public Health (subscription only) drew some pretty bold conclusions about the treatment of back pain:
We found that (1) participation in recreational physical activities reduces the likelihood of concurrent and subsequent low back pain, related disability, and psychological distress and (2) use of back exercises increases the likelihood of concurrent and subsequent low back pain and related disability.
Now, as someone who has suffered — sometimes excruciatingly — from lower back pain, I can tell you that this is surprising. A few years ago, when things had gotten so bad that I was virtually unable to go out for a recreational walk, I decided to research the matter. (I had tried everything from physiotherapy to chiropractic therapy to accupuncture.) Since I had been diagnosed with spondylolisthesis L5-S1, I searched the scholarship on the subject. What leaped out at me at the time was that one technique that appeared to have success at treating low back pain in all the studies I consulted was a series of extension exercises designed by Robin McKenzie. (Yes, I’m simplifying matters.) What struck me even more was that these techniques had been published in a thin little popular book called Treat Your Own Back. Believe me, it’s not every day that you see a popular self-help book cited approvingly in the scholarly literature.
But what absolutely floored me was that the technique worked for me. It took weeks, but for the first time in almost a quarter century, I was — and am — pain-free. Now I realize that this appears to be the same kind of personal testimonial that I have criticized in Seth Roberts and David Zemanek. The difference, I would suggest, is that the technique had already been the subject of extensive scientific testing. My own “self-experimentation” was simply confirmation of the fundamental principle.
So, does the latest study prove that I was simply the victim of the placebo effect? Are back exercises really counter-productive? Well, let’s look a little at the study. One would think that a study making the sweeping claim that “back exercises” cause pain might have paid careful attention to the precise design and character of the exercise performed. Instead, we get this little confession in the paper:
Furthermore, information on specific types of back exercises was not collected. Although little evidence indicating that some specific exercise regimens are more effective than others exists in the literature, certain exercise regimens may be more effective than others. Also, we relied on participants’ self-reports of their exercise and physical activities, and it was not feasible to validate responses with other strategies such as direct observation.
So, not only did the researchers ignore the specific nature of the exercise being performed (not all extension or flexion exercises are the same), they never even bothered to check how — or whether — these exercises were being performed at all. Shouldn’t it be obvious to the editors of the American Journal of Public Health that the study’s conclusions are empirically unsustainable?