“From cold plunges to collagen to celery juice, the $1.8 trillion global consumer wellness market is no stranger to fads, which can sometimes surface with limited clinical research or credibility. Today, consumers are no longer simply trying out these wellness trends and hoping for the best, but rather asking, “What does the science say?” – McKinsey.com
As someone who works in this space this trend, outlined in a recent McKinsey post, has me worried. To be specific, I worry that “evidence based” is itself becoming another fad, much like collagen and celery juice.
What Science Says
Don’t get me wrong, I’m not suggesting that evidence or science are not important or valuable. However, as a society I’m not sure we have the right idea about what “evidence based”, or science writ-large, is about. Science is not a treasure trove of irrefutable facts. It is a philosophical outlook with a couple of important points. First, you are asked to doubt and test things, rather than accept them at face value. Second, when you do test things you must focus on what you observe. If you run an experiment expecting to see one thing and you see another, you must be open to the idea that your expectation was incorrect. It’s the opposite of my favorite line from Steven Colbert during his Colbert Report days: “I’m not a fan of facts. Facts can change, but my opinion never does.”
I think it is important to understand that it is this approach—skepticism and testing—that gives science its value. The data that gets produced by following this philosophy still has to be scrutinized.
One current influential voice in the wellness space who has handled this issue thoughtfully is Peter Attia. For the record, I don’t agree with everything Attia says, but in his book Outlive, he does a good job of pointing out how science and the evidence it produces can easily be misleading or wrong. A great example he gives is from a study done by Hatori, et al that looked at intermittent fasting in mice. The study showed that mice who fed only within an eight hour window (a popular intermittent fasting protocol in fitness) were healthier and lived longer than the mice without the window. Now, you might be tempted to view that study as evidence in favor of the eight-hour feeding window. The problem, Attia points out, is that mice only have a two year life span and will die after only forty-eight hours without food. That means, he says, that an eight hour fast in a mouse is like a multi-day fast in a human.
In other words, this particular study looks like evidence in favor of humans using the eight-hour window, but it actually isn’t. It is really only evidence that using an eating window, of some duration, might be useful for humans, because it seems to be useful in mice. To get actual evidence for humans though, you have to do research with humans.
Another important point that others have raised is the issue of replicability in science—the so-called “replicability crisis.” In a nutshell, the replicability crisis is the observation that many studies that purport to show an outcome can not be reproduced, i.e. the same experiment fails to yield the same results. The “crisis” is that much of the time, that fact never gets reported. Studies that are rerun and fail to work as before don’t get enough play to refute the original conclusion.
Part of the reason for this may be our incentive systems. Researchers want to publish their studies in prestigious journals, and journals want to present interesting findings. Research that shows a negative finding usually doesn’t fit the bill. Thus, one study that showed an interesting outcome, maybe randomly, can be published while others that would refute the finding quietly fade into the night, leaving the audience to believe that the one study is the truth. And I would add that media outlets often pick up the original finding and report it in their health and wellness sections, but are unlikely to report later on that the study was non-replicable.
What To Do, What To Do
As human beings, we all have a desire to feel safe and confident. That’s one reason that science is appealing. It feels like it provides answers that we can feel secure about. However, that desire can hamper our critical thinking. Just because something is “evidence based” does not mean it is right. All it really means is that we have some data, from somewhere. What the data says requires further consideration.
If you are the average person looking to be healthier, when you see something being sold as “evidence based” you’ll still have to do some critical thinking. You may want to evaluate the evidence for yourself. If you’d rather not suffer the prose of scientific literature, you may just want to question the person selling the service. What can they tell you about the “evidence?” Is there a lot of it? Where did it come from? Are there competing ideas? As they answer these questions, you can judge for yourself if they seem to be building a logical, thoughtful answer.