Why is this interesting? - The Cochrane Edition
On health, empiricism, and systematic reviews
Noah here. I first became aware of Cochrane when my wife was pregnant with our first daughter. She put me in charge of figuring out what was safe for her to eat and what wasn’t, and I found a great book called Bumpology that set out to answer those questions (amongst many others) with empirical research. Most of that evidence came from systematic reviews, which look across existing studies and attempt to come up with a meta-answer to the question they were all asking (mentioned in WITI 6/18 - The Oldest Child Edition). The leading group in the field is the not-for-profit Cochrane, which operates with over 30,000 volunteers, very little paid staff, and effectively no headquarters.
Why is this interesting?
As I was doing some deep dives into my own health over the last few years, I found Cochrane Reviews to be a pretty incredible resource. The NIH has a massive free database of them available and they're a much better way to get answers to specific medical questions than any attempt to read and make sense of individual papers. (Those individual pieces of research have different-sized datasets and methodologies, which is why I’m always skeptical when I read a newspaper article about how eating apples upside-down has been proven to extend your life.) What I've been wondering is how Cochrane actually does their work, which turns out to be a harder question to answer than you’d think. Although the Cochrane site has a section all about how they work, it is more focused on the principles and goals than the process.
Stories about Cochrane are basically non-existent as well. The journal Science had a pretty good piece from 2003 on the organization’s founding:
A decade ago, a small group of epidemiologists and clinicians set out to change medical practice. Their radical notion: Physicians should base treatment decisions on the best available evidence on whether a potential therapy is likely to work. And that evidence, they argued, isn't likely to come from textbooks or a few large, controlled trials. Instead they reasoned, the best way to see through the mass of data on a specific intervention would be to cull all available studies, give failing marks to any that don't measure up, analyze the rest, and synthesize the results into a single "systematic review."
And here’s the New York Times from 2014 on what they do:
Cochrane divides human disease among 53 independent “review groups” of volunteer experts who oversee each analysis. They turn out more than 400 systematic reviews annually. In recent years, the reviewers have provided definitive guidance for adult influenza vaccination (fairly useful), exercise to treat depression (no clear evidence but suggestive), and when to cut the umbilical cord postpartum (wait awhile). They also examine less critical but still important topics such as whether an electric toothbrush is superior to manual brushing (it is) or if flossing prevents cavities (who knows?).
After those two standouts I found a small handful of other pieces, but the coverage is very thin. In the end what amazes me about Cochrane is that for all its reach and a massive mission (“to promote evidence-informed health decision-making”), they fly well-below the radar for the vast majority of us. Maybe that’s fine, as their work is more focused on medical professionals than the patients they treat, but in an age of WebMD symptom spirals and constant conversations about misinformation, it’s surprising a group that is consistently putting out reliable empirical data doesn’t get more attention. (NRB)
Chart of the Day:
I’m a few weeks late here, but just got around to watching HBO’s Chernobyl and it’s impressive. Definitely lived up to the hype for me. Also really enjoyed the podcast along with it. Here’s some detail about ratings for the show and a vow of the search traffic it produced. (NRB)
Quick Links:
I thought this was a good point from Shira Ovide, who covers Amazon for Bloomberg: “When Amazon says it's 4% of total retail sales or whatever, ask yourself (and Amazon): What's the numerator, and what's the denominator? Amazon knows its US sales (the numerator), and we don't. Does the denominator include ALL US retail sales? Even gasoline? Is that fair?” (NRB)
Alert: Boards of Canada did a two-hour mix with new material threaded in. Thanks to our friend and WITI reader BP for the link. (CJN)
Good deep dive on Japanese menswear with a quote from WITI contributor W. David Marx. (CJN)
Thanks for reading,
Noah (NRB) & Colin (CJN)