wrongness, and how to be wrong about it

This is me coming out to defend my people. I’ll explain how in a minute; just bear that in mind as I warm up.

From The Evils of Corn Syrup: How Food Writers Got It Wrong, by James McWilliams, Associate Professor of History at Texas State University, San Marcos:

Grist.com wasted no time in headlining the study as a “breakthrough work on high-fructose corn syrup and weight gain.” Laskawy declared that the debate over high fructose corn syrup “may be approaching a conclusive end.” At organicauthority.com, Scott Shaffer called the study “the nail in the coffin for the unhealthy school lunch programs that fill our kids with high-fructose corn syrup.”

Not until Marion Nestle, the noted NYU nutritionist, critically assessed the Princeton study did the feeding frenzy abate. “I’m skeptical,” she decreed. “I don’t think the study produces convincing evidence of a difference between the effects of HFCS and sucrose on the body weight of rats.” Her evaluation, which would prove to be supported by other experts, revealed that the authors failed to account for how they measured calorie intake, that the results they found were inconsistent, and that the observed differences between sucrose and HFCS were “statistically insignificant.” Exactly how the authors reached their conclusion, she added, “is beyond me.”

I am not what you’d call plugged into the network of expert opinion on nutrition (never does McWilliams name these “other experts”); and I respect Marion Nestle a lot, so I would probably have taken her word for it if Bocarsly et al. hadn’t been my colleagues back in the day. But they were. (In case you care: I took Bart Hoebel’s class, although he probably doesn’t remember me, and Elyse Powell was my student for two semesters.) So I read what McWilliams had to say, and what Nestle had to say, and what my people in Green Hall had to say.

McWilliams takes Nestle’s criticism as authoritative. Like I said, I would have been guilty of same, had things been different — but right off this is a problem, because you have a historian taking a nutritionist’s opinion as authoritative regarding an experiment by a neuroscientist. And when a nutritionist says the inference in a neuroscience experiment is “beyond her,” at least two things could be going on, only one of which is that the inference is wrong. Because:

And:

The third study used female rats (number not given) and observed them for 7 months. At the end of the study period, female rats fed HFCS plus chow for 12 hours a day weighed 323±9 grams. Female rats fed sucrose plus chow under the same conditions weighed 333±10 grams. This result is not statistically significant. (MN)

Look at the asterisk in the table. See what that means? In the female rats, the 24-hour HFCS are significantly different from chow-fed controls. It says nothing about the difference between groups 2 and 3.

Nestle is fixated on the length of exposure, which is reasonable — but the rationale for including the 12-hour exposure is mentioned pretty plainly in the paper (previous work indicates that it precipitates binge-eating behavior), which also explains why it would run counter to her intuitions (or be, in her words, “inconsistent.”) Luckily, she and James McWilliams don’t need me to say this, because Bart Hoebel already said it in the comments. His response is pretty spot-on, which is all I really have to say —

— and thus we pop one up in the stack, to James McWilliams and how food writers got it wrong.

But is there much to be said here either? I mean, you’ve got to take my word for it or do the work yourself — Marion Nestle said that Hoebel and his collaborators got it wrong, and, quoth I, she was wrong. James McWilliams bought her wrong analysis without even bothering to read the free lesson in scientific interpretation provided by Bart, which had been hanging out there, flagged in red by Nestle, for nearly half a year before McWilliams’ piece was published. That says all I care to know about James McWilliams’ opinions on science.

Also, the study is not nearly as complicated or confusing as Nestle makes it out to be. 90% of the damn thing is in Table 1. I do think there are a few problems of exposition. It’s easy to read the sentence from the abstract, “Rats with 12-h access to HFCS gained significantly more body weight than animals given equal access to 10% sucrose, even though they consumed the same number of total calories, but fewer calories from HFCS than sucrose,” and assume it applies to all the results, even though a careful read makes it pretty clear it only applies to Experiment 1. Likewise, the literature that takes the place of the obvious sucrose controls (some of it generated by the Hoebel lab itself) isn’t perspicuously cited early in the paper, which would have helped.

And, finally, what both Nestle and McWilliams miss in the discussion section is that this paper didn’t come out in isolation. Read the discussion section: Lots of other investigators have confirmed different physiological effects of fructose relative to other sugars. Assertions that “a sugar is a sugar” (cf. the Corn Refiners Association, with whom Nestle “agree[s]… on this one”) don’t just contradict Bocarsly et al., they contradict all that stuff they cited. And maybe it is all wrong, but at that point you’re talking about a lot more than a few rogue neuroscientists at Princeton.

It’s interesting to think about the broader context here. Nestle, I think, was just asleep at the switch; she read this one wrong. McWilliams, though, I can’t help but suspect after the following: “The irony in this mad dash is that a smoking gun already exists to condemn HFCS as the embodiment of culinary evil.” Or, lightly paraphrased, “You don’t need science to hate HFCS; you already have politics.” And generalized: “Need an argument to a conclusion? Why not use mine?” Why are we so comfortable stipulating ideologies — why, that is, should we assume that the value of the political justification is equivalent to the value of the scientific justification? Not everyone is out to make someone do something; some people just want to know the truth. It’s weird that this needs to be said. This is not an argument for the superiority of science over politics — if the Hoebelian scientific concerns are dealt with, the Pollanoid political concerns remain, and vice versa. No shit. But McWilliams has elided the fact that Hoebel and Pollan are doing different things, and neither feels, nor should feel, any obligation to induce the same ideology as the other. Coordinated condemnation is not, nor should it be, their goal.

If the point doesn’t come through by now, it never will, so I’ll leave it hang. Anyway, the moral overall is simple: As Aesop once said, “Don’t trust a historian in Texas to tell you that a nutritionist in New York is right about a neuroscientist in Jersey, unless he’s done his goddamn homework.”

(I don’t take the Bocarsly study as definitive of anything in particular, by the way. I think the findings are what the authors say they are, and I think they’re consistent with the hypothesis they advance. That’s all you generally ask of a single study. This study delivers that, and it’s enough.)

(And I really do respect Marion Nestle. But if she’s going to go after my people, she needs to do it better.)

One thought on “wrongness, and how to be wrong about it

  1. Matt, you jump too quickly to your former colleague’s defense, as their paper makes a claim that is clearly false, based on its own data. The paper prominently states: “3.3. Female rats with 7 months of HFCS access gain significantly more body weight compared with sucrose-fed controls”. (See article for full quote) This is NOT VALID. The chart you helpfully reproduced shows: Female rats with 7 months of 12-h HFCS + 12-h chow access having LESS body weight than those with 12-h sucrose + 12-h chow access.
    Surely you don’t think the appropriate comparison is between line 1 and 3 of the female section! Certainly, the appropriate comparison is between line 2 and 3 of the female section; as you imply, 12-hour exposure is only appropriately compared to 12-hour exposure. Ceteris paribus is a foundation of good science. And that comparison does not support the claim.
    Furthermore, when one looks at body FAT, and TG levels reported, they do not support the central conclusion I highlighted either.

    Do you dispute that the paper makes a clearly false claim?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s