“Steaks for breakfast, steaks for lunch, steaks for brunch—grass fed, massaged beef, all day long.” Conor McGregor was reveling in his meat eating persona. As the Game Changers tells us, he was hyping up a stereotype about manly men eating meat because he was about to get in the ring with Nate Diaz, a vegan fighter. “I’m a lion in there,” McGregor announced to his plant-eating opponent at a pre-fight press conference. “Your little gazelle friends are gonna be starin’ through the cage looking at your carcass getting eatin’ alive.” Considering this is a documentary on the merits of veganism, we know where this is heading. Diaz goes on to a blood-spattered victory and the vegan community rejoices. For the Game Changers it’s a powerful display of their thesis and we the audience are ready to be convinced. What follows, unfortunately, is a butchering of scientific argumentation.
The McGregor/Diaz match makes for a great early segment in the film largely because McGregor is notorious for his trash talk. His bravado and belittling attitude toward Diaz is great primer for the audience. We want to see him lose, if only for that big mouth. But what happens next is a pivot towards a position that can’t be sustained by the evidence they provide, namely that veganism is not merely compatible with high performance but is in fact the reason for it. The switch happens with the introduction of a slew of other vegan athletes at the apex of their sports. After McGregor taps out, we meet strongman Patrik Baboumian, olympic cyclist Dotsie Bausch, and olympic weightlifter Kendrick Farris, a cast of characters united by their proclamation that their best performances happened after going vegan and that the switch is the reason for their success. Farris says that after going all plant he won several championships and found himself wishing that he “had [gone vegan] a long time ago.” Bausch, an impressive person who at thirty-nine became the oldest cyclist to medal in the olympics says “my diet was the most powerful aspect to me being able to perform and produce for the U.S. team at the Olympic Games.” Baboumian, an amazing strength athlete with several championships to his name says “when I stopped eating meat I got stronger and bigger,” and when people ask him how that happened without meat he answers, “have you seen an ox eating meat?”
These days, there’s a strong cultural urge—at least on one side of the political spectrum—to go along with this premise. It would be incredibly convenient if going vegan could solve our environmental and our public health issues while at the same time turning us all into high-octane athletes. Who wouldn’t want to save the planet, boost their health, and kick ass while doing it. It would indeed be a game changer. Whether or not these athletes or the film are right though, is frankly beside the point. What is more important for the viewer is what we can learn about gathering, evaluating, and presenting evidence.
Controls, Falsifiability, And Cherry Picking
Let’s start with the McGregor/Diaz match. First, let’s take a moment to summarize what this example is actually saying: if you are vegan you will beat Conor McGregor in a cage match. Put that way, it seems a bit of a silly hypothesis from the start. Why are we using McGregor as the test here? Is it because he likes meat so much? Aren’t there other athletes who like meat? Or could it be solely because he provides such excellent footage of vegan-bashing? In any event, that being vegan will cause you to defeat McGregor is what they are giving us, so let’s go with it.
In a scientific test of whether or not this hypothesis is accurate, the first thing we would need to know is if there are other factors that could help someone beat McGregor. We know that, in fact, there are. McGregor has an overall record of 22-4 in the UFC, meaning that three other people besides Diaz have beaten the man.1 One of them, Khabib Nurmagomedov is, according to an internet article on what he eats, not vegan. The other two don’t have any food articles on the internet and it would therefore be unscientific to say they are meat eaters too, but shall we take bets on it? Moreover, and frankly more importantly, on August 20, 2016, Diaz and McGregor faced each other a second time and McGregor won, a detail the Game Changers would have done well to mention. All of this tells us that to be scientific about the role of diet in the original match between Diaz and McGregor, we would have to control for other variables. In layman’s terms, we have to separate other possible explanations for the outcome from our analysis because if we don’t, we can’t suss out what is really making the difference. Was Diaz’s diet what lead to his win, or some other characteristic he shares with Nurmagomedov? If Diaz’s diet were in some way related to the win, why didn’t it work out in the rematch; what else is going on here? How about McGregor himself? How do we know he didn’t lose the match because of something he did? Maybe he stayed out late the night before. Maybe he had a nagging injury that we don’t know about. Maybe he had salad for dinner.
This error of analysis is persistent throughout the film. Consider our other group of athletes, who really drive the narrative that veganism is the key to their championship performances. Bausch, the thirty-nine year old cyclist who claims her plant based diet was the “most powerful aspect” to her olympic success also tells us that, prior to the games, she had been struggling with injuries and believes it was her diet that helped her recover and come back stronger than ever. This really muddies up the picture. As an Olympian, she was undoubtedly well cared for by a slew of physical therapists, sports medicine doctors, strength and conditioning coaches, and others and she most likely worked hard at implementing whatever treatments they suggested. Any one of them could have been a deciding factor in her recovery. Beyond that, Bausch may have truly believed, in perfectly good faith, that plants were giving her a new lease on her athletic life at the same time she was undergoing all of this work. Especially for an elite athlete looking at the end of her career, that would be a huge psychological boost that might have allowed her to go out and push harder and further than she otherwise would have believed she could. Whether or not this is in fact behind her performance is immaterial. The point is that we can’t know because we haven’t controlled for any of these variables.
The weightlifters also lack controls, but rather than continue to beat that horse let’s think about another important scientific principle that they illustrate. In science, falsifiability is the idea that if a hypothesis can’t be tested then it isn’t science. For instance, the statement “if I had kryptonite I’d be able to jump to the moon” is not falsifiable because nobody has any kryptonite to put it to the test and never will. There is some debate over this principle, especially in heavily theoretical sciences like quantum physics, which produce very useful tools for predicting and manipulating real-world scenarios but sometimes prove difficult or even impossible to test experimentally.2 Be that as it may, our weightlifters in the Game Changers are not quantum physics experiments and their cases do not require much philosophical debate. They each claim that they got bigger and stronger because they went vegan. The only way to test that claim would be to travel back in time and have them go through the same training regimen, at the same age, but on a meat diet. Since we don’t have time machines we can’t test that hypothesis, making it non-falsifiable and thus, at the very least, questionable.
On a final note, Baboumian’s quote that oxen don’t eat meat is an example of another scientific no-no: cherry picking. Oxen are strong, and they don’t eat meat, which sounds like great evidence for the virtues of plant eating. However, Grizzly bears are exceptionally strong creatures too and they do eat meat. And while wolves might not have the same absolute strength as either of them, would you want to have to face a wolf one-on-one in the wild? There are other good, animal related, examples of cherry picking later in the film. They suggest, for example, that since human intestines are longer than many carnivore’s, we must be plant eaters. What they don’t mention is that our intestines are also shorter than many herbivore’s. They further note that great apes have canine teeth but that they use them not for meat eating, but for biting each other, a reference meant to debunk the claim that human canines are evidence we’re meat eaters. Fair enough, canine teeth don’t prove that humans are carnivores. Then again, generally speaking, humans don’t bite each other the way apes do, so we’re left with little more than a shoulder shrug when it comes to fangs. The point is you can’t select the evidence you like while disregarding the evidence you don’t, no matter how entertaining it might be.
And speaking of entertainment…
The Erection Test
Gamer Changers seems to be at least partially aware that anecdotal stories alone aren’t quite enough to build the argument they’d like. They therefore conduct a couple of fun experiments within the film meant, it would seem, to amuse as much as to inform. One of the most attention grabbing involves three young athlete’s nocturnal erections. Put briefly, the film claims repeatedly that plant based diets are better for the flow of blood in the cardiovascular system. To boost the credibility of the claim three young, strapping, collegiate male athletes are recruited for a test that involves feeding them a plant based meal on one night, a meat based meal on another, and then measuring their erection strength while they sleep. Men, you see, get erections throughout the night—we shall not digress into the reasons why. (It’s less exciting than you think anyway.) Through the use of two rings attached to sensors, the athlete’s erection strength is measured following each different meal on their respective nights. You can guess what they find. The erections are stronger when they eat plants, which the film fits into the overall narrative that plant based meals improve health and athletic performance—although one doubts that this is the kind of performance these young men are focused on.
Putting aside whether or not the erection test should really be used as a proxy for cardiovascular health or athletic prowess, the way the experiment is done is cringeworthy on its own. It’s not entirely clear what the athletes know about the experiment, however, given the scenes that we see with them talking to the narrator, it’s not a stretch to imagine that they have some idea what this is all about. They’re in a movie about veganism and, presumably, haven’t been living under rocks. All things being equal, they would probably have some inkling that the plant based meal is supposed to be the superior one. To be fair, it’s possible they were kept in the dark and that they had no clue that eating vegetables is considered healthier, but the film does not specifically tell us, which is a detail any real scientific experiment would absolutely need to comment on for reasons we’ll cover in a moment. Worse than the question of how much they know about the expected results is the fact that they seem to know at meal time which diet they are being fed! One of the men comments that he was expecting the vegan meal to taste “nasty.” Another says “I didn’t think we were going to get a burrito, I thought we were going to get a salad.”
To cut to the chase, people can’t be tested like this. The gold standard for testing the effect of an intervention on people is a double blind, placebo controlled study. In this design, neither participants nor the researchers know whether a subject is receiving a real treatment or a placebo. This is the top standard because results of experiments on people are so easily distorted by bias. When researchers know what the participants are receiving, the chances are higher that they will consciously or unconsciously manipulate the data to show the result they want. With regard to subjects, the placebo effect is so powerful that sometimes receiving anything at all, including a sugar pill, will create the effect being hoped for. That’s why, particularly for medicines, a group of subjects with the real intervention has to be tested against a group getting a fake intervention, i.e. a placebo such as a sugar pill. Only then can the researchers be confident that the intervention is what is causing the result and not the expectation on the part of the subject. To reiterate, this is the main issue with cyclist Dotsie Bausch’s statements earlier in the film—she believed, or at least hoped, that switching to plants would give her a boost in the twilight of her career and, voila, it did.
This “experiment” in the film simply can’t be used as evidence of anything. The subjects and the researchers know too much. If you are in to model penises though, you can give it a watch.
It’s Not Veganism, It’s The Film
The problems in this movie are persistent and many fold, and one could go on. That is unfortunate because The Game Changers’ mission is laudable. It is pretty well established that our current level of meat consumption and the industry that supports it is out of balance. We’re very likely eating more animal products than is healthy. It’s also being produced at great environmental cost and the ethics are problematic to say the least. Being right on those counts though can’t save this film from its egregiously poor presentation of science and evidence. At best, what we could take away from the movie is that going vegan does not doom your performance—at least not in the short run—and that it would be worth looking into this whole plant eating thing further. How far could we take it and what would the benefits be to our health and performance? That would be nice to know. The Game Changers can’t tell you.