The Fund supports several networks of state health policymakers to help identify, inspire, and inform policy leaders.
The Fund identifies and shares policy ideas and analysis on topics important to state health policymakers, particularly on issues related to state leadership, primary care, aging, and total costs of care.
Keep up with news and updates from the Milbank Memorial Fund. And read the latest blogs from our thought leaders, including Fund President Christopher F. Koller.
The Fund publishes The Milbank Quarterly, as well as reports, issues briefs, and case studies on topics important to health policy leaders.
The Milbank Memorial Fund is an endowed operating foundation that publishes The Milbank Quarterly, commissions projects, and convenes state health policy decision makers on issues they identify as important to population health.
December 5, 2019
Early View Perspective
Gerald M. Oppenheimer
Back to The Milbank Quarterly
In July 2017, the New York Times reported that the National Institute on Alcohol Abuse and Alcoholism (NIAAA) would be funding a $100 million randomized controlled trial (RCT) to examine an issue that had been the subject of scientific and public health controversy for four decades: whether moderate alcohol consumption was protective against cardiovascular disease. Central to the report was the revelation that the alcohol industry would be funding the study through the National Institutes of Health Foundation. The director of the NIAAA sought to reassure anyone alarmed by noting that “the money has no strings attached; [whoever] donates to the Foundation has no leverage whatsoever, no input into this study.”>1(pD1) Such private funding was not unprecedented; since 1996, the foundation had received $1 billion from corporations and other sources. In 2014, for example, the foundation announced the formation of the Accelerating Medicine Partnership, a 5‐year effort involving $230 million, half from pharmaceutical firms. For some organizations, such as the well‐regarded advocacy group Public Citizen, no firewall could preclude the corrupting impact of funding by corporations with a direct interest in the outcome.
In March 2018, a new investigative report in the Times revealed that there had been an “irregular relationship” between the NIAAA and the alcohol industry while securing funding for the study.2 This followed months of remonstrations by Michael Siegel of Boston University and other concerned researchers.3 Responding to these new disclosures, Public Citizen demanded a formal investigation:
The fact that the NIH accepted tens of millions of dollars from the alcohol beverage industry was bad enough… . But the revelation that senior NIH officials orchestrated an aggressive campaign to solicit such industry funding, if confirmed, clearly undermines public trust in the integrity of the study and is a stain on the NIH’s reputation.4
Within days, the National Institutes of Health (NIH) Director Francis Collins, announced the formation of an advisory group to examine whether the organization and funding of the barely begun RCT had violated the NIH’s rules and procedures.5 Three months later, after an exhaustive investigation, the NIH announced that it was ending the trial. Collins stated, “So many lines have been crossed that people were frankly shocked.”6 The NIH committee found that there had been ongoing interaction since 2013 between senior NIAAA personnel and the alcohol industry, whose discussions involved matters bearing on the scientific planning of the study that went “well beyond the norm.”7 Furthermore, the committee wrote, there had been ongoing discussions between the NIAAA and the eventual principal investigator of the study that “violated the well‐established principle” that program staff not give preferential treatment to individuals. In addition, two new peer reviewers recruited by the NIH concluded that on methodological grounds alone, the study should not have gone forward as designed.7(appF)
In December 2018, after a careful review that some people believed was long overdue, the NIH issued exacting guidelines for dealing with external sources of funding. Critically, the report, with obvious reference to the history of the beverage industry in seeking to limit the impact of regulatory measures that might negatively affect consumption, noted that “reviews of public‐private partnerships should integrate a contextual assessment of reputational risk to the agency.” Among the matters to be taken into account were the “mission of the partner [and] the magnitude of the gift.”
It would be easy to view this saga as yet another example of corporate efforts to buy science and of government officials becoming pawns in a process that put at risk the very science they were charged to protect. Researchers concerned about the public health impact of alcohol consumption were especially leery. Critics warned that “alcohol producers are engaged in a campaign to capture the hearts and minds of alcohol researchers.”8
But such an approach fails to appreciate the whole story. It does not recognize that scientists within the NIAAA had concluded that an RCT was a priority, however problematic its funding might be, given the strident debate over whether observational epidemiology had or could ever determine whether alcohol had a protective effect on coronary heart disease (CHD), a matter of importance to public health.
For good reason, those involved in studying controversies in the contemporary history of public health policy have paid substantial attention to unearthing and describing the ways in which corporate interests have sought to shape, distort, undercut, or hide evidence concerning risk. A recently published article in The Milbank Quarterly noted that Coca‐Cola sought to influence the Centers for Disease Control and Prevention (CDC)’s policy on obesity.9 The history of tobacco industry efforts to shape, undercut, or mask evidence about the risk of smoking hovers over almost all research into public health controversies involving products that could threaten population health. When there is a formal commitment to evidence‐based policy, industry has used it to demand further exacting evidence as a way of thwarting regulatory efforts. Exemplifying these concerns about the manipulation of uncertainty are two important studies, Doubt Is Their Product: How Industry Assault on Science Threatens Your Health by David Michaels, the former director of the Occupational Safety and Health Administration under President Bill Clinton, and Merchants of Doubt by Naomi Oreskes and Erik Conway.
Critically important as this perspective is, it has, however inadvertently, distorted our understanding of some controversies, especially when the nature of scientific evidence remains equivocal and when corporate interests have an obvious stake in the outcome of debate. In recent years, this has been true in
If we are to take seriously the challenge of making public health policy that is informed by the available scientific evidence, then it is important to capture the richness of controversies as they emerge, develop, and are resolved or remain a matter of controversy. There is an important literature in the history, philosophy, and sociology of science that provides analytic frameworks for understanding these debates, but such work depends on the careful presentation of case studies on which broad theoretical frames can draw. This history of the controversy over the public health implications of moderate alcohol consumption is important, not only in itself, but also because it contributes to the evidentiary base critical to a broad understanding of controversies involving evidence‐based public health policy.
A Surprising Finding
What became a four decades‐long controversy over the impact of moderate drinking on heart disease began in 1974 with a consensus‐challenging study by Arthur Klatsky and his colleagues.14 Published in the Annals of Internal Medicine, the study, which reported that abstainers from alcohol had the highest proportion of primary heart attacks, drew very careful and tentative conclusions, mindful that two important cohort investigations, the Framingham Heart Disease Study (1966) and the Chicago People’s Gas study (1963), had previously found no significant association between alcohol consumption and CHD.15,16
Could it be that alcohol, despite being often perceived as a threat to health, is protective when consumed in moderation?17 Coronary heart disease, responsible for more than 400,000 deaths in 1974, had been the leading cause of US mortality since the 1920s.18 For epidemiologists and policymakers intent on preventing an incurable disease, finding a new, potentially modifiable risk factor was of considerable public health interest and demanded further study.
In 1981, the British cardiovascular epidemiologist Michael Marmot and his colleagues, using data from the famous Whitehall longitudinal study of civil servants, reported that after 10 years of follow‐up, moderate drinkers had a lower total mortality rate than did non‐ and heavy drinkers. Marmot and colleagues considered and rejected the possibility that nondrinkers were “already sick” at the time of examination, that is, that they had already received a diagnosis that led to their decreased drinking and a higher than average mortality rate.19 They reasoned that if abstainers were less healthy, they should have had their highest death rates early in the study, declining thereafter, but the data showed that this was not the case.
A decade later, Marmot and Brunner, in reviewing the accumulating evidence of more than 25 studies, both prospective and case‐control, reaffirmed the probable validity of the often observed J‐shaped curve, largely the consequence of lower rates of death from coronary heart disease among moderate drinkers. They wrote,
Despite the flaws of each individual investigation, what stands out … is the high consistency of the findings in populations as diverse as British civil servants, men in different regions of the United Kingdom, men and women in Framingham, Yugoslavs, Puerto Ricans, Japanese Americans in Hawaii, … and others.20
The Consensus Challenged
What had been a surprising proposition in 1974 became, over the following 15 years, a growing consensus, with studies of the potential benefits of moderate drinking appearing in major medical journals. But that prevailing harmony was provocatively challenged in 1988 by British physician Gerald Shaper and his colleagues. In The Lancet, they argued that data from their British Regional Heart Study of middle‐aged men did not support the hypothesis of a J‐shaped curve.21 Based on their data, they argued that previous studies had committed two epidemiological errors, those of selection bias and uncontrolled confounding.
Shaper drew a portrait of abstainers that underscored their unsuitability as a control group, and he asserted that
a large proportion of abstainers … are ex‐drinkers … [who] are older than drinkers and they include a higher percentage of manual workers and unmarried men… . They also have a higher prevalence rate of high blood pressure, diabetes, gall bladder disease, and bronchitis, and the highest rates of regular medication.22
Shaper’s work supported later skeptics of the heart protection thesis, who sharpened his analysis of selection bias and confounding. Researchers who argued for the protective effect of alcohol, Shaper held, were guilty of reverse causation, turning the cause (poor health, predisposing lower alcohol consumption) into the effect.
While mindful of Shaper’s critique, studies over the next decade consistently supported the hypothesis that alcohol was protective against CHD and perhaps stroke. By 2000, an analysis of mortality and alcohol consumption using data from two National Health Interview Surveys, representative of the US population, reflected the now prevailing scientific consensus. The authors wrote that “in large populations, excess alcohol intake had been found to be associated with excess death, liver cirrhosis, some cancers, hemorrhagic stroke, cardiomyopathy, and hypertension. On the other hand, clear evidence exists for a protective effect of moderate alcohol intake against coronary artery disease” (italics added).23
In 2001, Kenneth Mukamal, who became a leading protagonist of the dominant hypothesis, and Eric Rimm published an article (subsequently reprinted by the NIAAA) that consisted of a systematic literature review and a meta‐analysis of 42 peer‐reviewed publications.24 They faulted Shaper’s hypothesis, citing studies that distinguished between long‐term abstainers and sick quitters in their design or analysis or excluded participants with heart disease, yet nonetheless observed that moderate drinkers had the lowest rates of CHD. Such outcomes persisted when a considerable number of clinical and social risk factors were controlled for. “In summary,” they wrote, “all of this evidence implicates alcohol consumption … as the primary factor in the lower rates of cardiovascular disease found among moderate drinkers.”
But did alcohol consumption play a causal role in reducing CHD?25 On methodological grounds, Mukamal and Rimm argued for the improbability of conducting a valid long‐term RCT. But they pointed to the success of short‐term clinical trials studying alcohol’s effect on cardiogenic outcomes like cholesterol levels. Rimm’s meta‐analysis of 42 such trials found that alcohol had a beneficial impact on risk factors for atherosclerosis, inflammation, and thrombosis. Citing this analysis, they concluded, “Taken together, the estimated changes induced by a consumption of 30 grams of alcohol appear to result in a 24.7 percent reduction in the risk of coronary heart disease,” similar to that seen in the observational studies.
Despite the broad consensus that the J‐shaped curve was stable and real, researchers continued to assess its strengths and limitations.26 In 2005, a commentary in The Lancet captured the mood of such equivocation regarding the protective effect of alcohol, finding a history of uncontrolled confounding in observational studies that researchers had prematurely discounted.27 Warning of “artifactual associations,” the commentary raised the specter of a “recent debacle.” Hormone replacement therapy for postmenopausal women, whose supposed cardiovascular protective effect had been supported by many observational studies, was unexpectedly discovered to promote heart disease, once put to a RCT. That dramatic reversal would come to haunt the alcohol debate over the next decade, a warning of the limitations of epidemiology without such a trial.
Systematic Skepticism of Alcohol’s Protective Effect
In 2006, 24 years after Shaper’s initial challenge, a new critique went far beyond expressions of methodological uncertainty, challenging the entire corpus of research supporting the consensus. The American and Canadian researchers Kaye Fillmore and Tim Stockwell, who emerged as vocal adversaries of the orthodox position, and their colleagues threw down their gauntlet, contradicting the majority of their peers.28 Whereas Shaper had raised a counterhypothesis based on data from his single cohort study, Fillmore and her coauthors used meta‐analyses to contradict a generation’s work.
Like Shaper, Fillmore and her colleagues underscored the methodological threat of misclassification bias by setting out to determine how such errors affected research outcomes. Categorizing studies by whether they contained former drinker and/or occasional drinker misclassifications, they ended up with 54 prospective investigations that examined the relationship between alcohol consumption and all‐cause mortality. A significant J‐shaped curve was found only in those analyses of articles containing misclassification errors. Their analysis of seven studies free of misclassification errors found no significant difference in mortality between long‐term abstainers and drinkers. They concluded, “The contrast between the least biased group of studies to the larger group of more biased studies is consistent with the well‐documented observation that as people age they both abstain and cut down to very occasional drinking for health reasons, disability, frailty and/or medication use.” Most important, they advised that researchers and policymakers “give pause” before accepting the protective alcohol thesis and take care not to misinform or harm the public. At Sweden’s Karolinska Institute, researcher Sven Andréasson made clear that
pretty well all media have informed the general public at regular intervals … about the positive health effects of one or two glasses of wine. This has doubtless affected alcohol consumption. It is difficult to blame the media for this but how should we view the role of the researchers? … In a field where the risk of causing damage is very real, there is now a strong need for more caution and more critical appraisal of the methods used.29
Point‐Counterpoint: The Hardening of Scientific Camps
In the following years, the debate continued in a point‐counterpoint fashion. It was as if the contending sides had come to inhabit two epistemic universes. New studies claiming that their findings again supported the protective impact of moderate alcohol consumption were greeted by methodological challenges that sought to undercut claims that these results were free of previously identified sources of bias. If attention shifted to the biological pathways that could explain an apparently protective effect, they were dismissed by critics as causally unrelated to CHD.30-32
The evidentiary melee inevitably led to policy questions: What were the implications for those concerned with morbidity and mortality at a population level and for those responsible for providing clinical services? There was no doubt that heavy drinking contributed to mortality globally. In addition, for some, the “protective effect” hypothesis might, however inadvertently, lead to increased alcohol consumption at a population level. But there the agreement ended, and two very different understandings took hold.
In 2016, the Canadian researcher Michael Roerecke cautiously framed the issue in a manner similar to that characterizing the advice by most researchers since Klatsky’s 1974 paper.33 “Advice about lifestyle,” Klatsky had written, “must be based on something less than certainty.” Health professionals had to combine common sense and science.
Given their reading of both the available evidence and its limits, Stockwell and his colleagues explicitly embraced a very different normative framing, the “precautionary principle” that had informed those concerned with how to respond to the threat of environmental degradation in the face of evidentiary uncertainty. Most famously framed in the Wingspread Declaration of 1998, it states that “when an activity raises threats of harm to human health … precautionary measures should be taken even if some cause and effect relationships are not yet established scientifically.”31 Those who upheld the hypothesis that moderate alcohol consumption reduced CHD thus bore the responsibility of demonstrating its validity. The role of science‐based criticism was mainly to cast evidence‐justified systematic doubt. But the consequence of this was that the debate devolved into an epidemiologic quandary awaiting an as‐yet‐to‐be‐agreed‐upon methodological solution.
Complicating matters was the argument that alcohol might also be carcinogenic. For those who believed that moderate drinking could be cardioprotective, what they took to be the relatively small risk of alcohol‐related cancers demonstrated a net benefit to the population’s well‐being. Conversely, a 2016 paper in Addiction charged that there was “strong evidence” that alcohol caused cancer in seven sites of the body and was responsible for 5.8% of all cancer deaths globally. If alcohol was not cardioprotective, then the burden of alcohol‐related cancers should be of paramount importance.34
The tone of mutual mistrust that had come to characterize the epistemological divide between supporters of the protective effects of moderate drinking and those who had emerged as the most persistent critics of what they viewed as bad science was revealed in an exchange of letters in 2016 in the Journal of Studies on Alcohol and Drugs in response to a report by Stockwell and his colleagues.35 Seventeen members of the International Scientific Forum on Alcohol, which describes itself as a volunteer organization of 46 practitioners and scientists, accused Stockwell and his colleagues of promulgating “misinformation in the name of scientific method,” and asserted that their failure to acknowledge evidence challenging their critique did “a major disservice to the scientific community.”36 In reply, Stockwell and his colleagues turned the charge of scientific irresponsibility back onto their opponents. They argued that given the “extensive list” of potential conflicts of interest involving the alcohol industry, “who [is] really guilty here of promulgating misinformation?” In the end, “only rigorous, independent and impartial science” that was free of commercial vested interests could definitively resolve the issues at hand.37
Given these fierce debates, how did the NIAAA, charged with charting federal research and policy analysis, respond?
The NIAAA and the Case for Moderate Drinking
Since its establishment in 1979 as an independent institute, the NIAAA has consistently concluded that the evidence supports the protective effects of moderate alcohol consumption. As a consequence, the institute developed an institutional commitment that shaped its response to the controversy over moderate drinking, underscoring the challenge of de‐implementation in the politics of scientific controversies.38,39 In 1992, in its official publication, Alcohol Alert, the NIAAA sought to summarize the prevailing science,40 stating that there was “a considerable body of evidence that lower levels of drinking decrease the risk of death from coronary heart disease.” As to whether research showing a protective effect was based on the mistaken classification of abstainers, the Alert claimed that studies investigating the challenge “did not support the conclusion.” Finally, on a matter that in the following years became critical to the debate on moderate drinking, the NIAAA minimized the risks of cancer. Although some cancers were linked to the “heaviest drinkers,” it said, the association between moderate drinking and female breast cancer was weak.
The policy and clinical implications of these findings were made clear by the NIAAA’s director in an accompanying statement.40 In general, if an individual was drinking moderately and was not at special risk, there was “no reason” to recommend anything different. Nonetheless, a cautionary note accompanied most discussions at the time: nondrinkers should not be advised to begin drinking despite the potential cardiovascular benefits.
Finally, in 2015, as challengers to the science underlying the claims that moderate drinking posed no risk and might indeed be protective became more vocal, the institute declared in “Alcohol Facts and Figures,” meant for widespread public access, that in most Western countries where CHD, cancer, stroke, and diabetes were the primary causes of death, large epidemiological studies “consistently” showed that alcohol reduced mortality.41 This position was echoed in a 2018 statement from the American Heart Association, another stakeholder, that even though alcohol consumption was a well‐established modifiable risk factor for breast cancer, it was “outweighed” by its cardioprotective effects. Moreover, for older women, cardiovascular disease posed a “greater risk of mortality than breast cancer itself.”42 That same year, the NIAAA estimated that in 2005, 26,000 deaths in the United States were averted “as a result of the benefits attributed to alcohol consumption.”41
Bringing Closure to the Debate?
Even before the alcohol debate had reached its recent fevered pitch, the NIAAA had concluded that an RCT was essential to resolve the questions that continued to roil the clinical and public policy arenas. This was not a surprise. The evidence‐based medicine movement that had sought to establish criteria for the evaluation of evidence had placed RCTs at the summit of the evidentiary pyramid. The efforts of the Cochrane Collaborative and the US Preventative Services Task Force underscore this methodological perspective.43,44 In June 2013, NIAAA staff prepared what it termed a “business” proposal for an RCT of moderate drinking. “ONLY VIA an RCT” could the implications of daily moderate drinking be determined in a manner that met the gold standard of clinical proof (caps in original).7 A failure to undertake such a trial would continue the already decades‐long controversy over bias and confounding in observational studies.
Since it was unlikely that the NIAAA would provide sufficient funding for such a costly study, industrial funding was proposed as an option. Were this path to be pursued, it was formally asserted that all aspects of the study’s design, the selection of the principal investigator, the analyses of the study’s findings, and eventual publications were to be solely under the NIAAA’s control. Aware of how such alcohol industry funding might compromise the perception of the study’s findings, the authors of this proposal shockingly suggested, in language that later haunted the institute, that “all attribution of funding would be to NIH grants, not to the source of where any of the monies in the grant pool may have originated.”
But even if fundable, could such a trial successfully produce definitive, credible evidence that would resolve the long controversy? Kenneth Mukamal, who in 2001 had argued that an RCT would be all but impossible to implement, changed his position.45 Selected in 2016 as the principal investigator of the NIAAA’s Moderate Alcohol and Cardiovascular Health (MACH) Trial, through a process that was later judged as biased in his favor, Mukamel had come to believe that the methodological barriers could be overcome. Specifically, controlled dietary trials and some short‐term trials involving alcohol use had recently been successfully conducted. “Definitive answers to controversies surrounding moderate alcohol intake may finally be forthcoming.”
On February 15, 2018, when it would have been impossible to ignore how divisive the debate over moderate alcohol had become and at a moment when an investigative report had already uncovered the problematic relationship between the NIAAA and the alcohol industry, the recruitment of subjects for the MACH 15 trial began. The worldwide study was to include 7,800 participants, 50 years of age or older and at an above‐average risk for cardiovascular disease. After an average of six years’ follow‐up, those who were advised to consume one drink a day (15g) were to be compared with abstainers on incidents of cardiovascular disease and diabetes.46 The decision not to include cancer as an end point was deemed by critics of the study as evidence of methodological bias. Consequently, because of the controversy sparked by the industry’s funding and the study’s design, the RCT was halted after four months.47 Commenting on the end of the trial, Boston University’s Michael Siegel noted the vital role of 14 of his academic colleagues and the media in forcing Francis Collins’s attention to the trial’s transgressions, writing, “It takes a village—it certainly did to bring down this scientifically fraudulent and unethical research.”3
There is no prospect in the foreseeable future for a new trial. For those who believe that only an RCT will provide conclusive evidence, there is no alternative but to address alcohol consumption without evidentiary certitude. Despite the faith of those like Mukamal, however, an RCT may not have provided definitive evidence about the hypothesized protective effect of moderate drinking. The history of RCTs is strewn with studies whose results remain controversial, whose findings of no difference, for example, have been attributed by protagonists to insufficient statistical power or that have raised the possibility of poor compliance by the study’s participants.48(pp164‐228),49(pp151‐189) Indeed, there is a strong possibility that these issues and perhaps others would have compromised an RCT of the alcohol hypothesis. Conversely, some researchers have suggested that instead of an RCT, a well‐designed observational study could provide outcomes comparable to those of randomized controlled trials, “so the RCT need not be considered in all cases the gold standard” (anonymous reviewer).
How to act in the face of incomplete data has been a recurring challenge for those committed to evidence‐based clinical practice and public health policy. But what constitutes sufficient evidence varies with the expert’s mission or specialty. When, for example, the US Preventive Services Task Force was asked to confront the question of whether to endorse a recommendation by the CDC that all adults routinely receive HIV screening, it could not do so, deciding after careful study that the evidence remained insufficient. More was required. At the CDC, proponents of routine screening based on their review of the same evidence concluded that they could not await the “mother of all clinical trials” before moving forward: “We don’t have the luxury of not acting.”50
We believe that respect for the science and the ethics of clinical practice and public health policy dictate that where uncertainty exists, it should not be masked.
Certainly, truly informed consent to a medical intervention rests on the assumption that a patient has been told, in words that she can comprehend, of both the risks and the benefits of the proposed treatment. In a parallel manner, for truly informed democratic decision making, an ethics of public health should require that the community be alerted of the extent to which evidentiary debate surrounds a course of action.
Such a posture would foreclose any public health messages designed to characterize moderate drinking as an incontrovertible threat to population health or as certainly cardioprotective. We have at least one published example of a physician recommending that nondrinking middle‐aged patients “have a drink … with dinner” as a way of reducing the risk of all‐cause mortality,51 but such proposals are rare.
More common are efforts not so different from those made when the possible benefits of moderate drinking were first given serious consideration: “If you drink moderately, there is no reason to stop. If you do not drink, there is no reason to start.” Such a position may seem unsatisfactory at either an individual or a population level. But at this moment, it is the best we can do.52 After 40 years of research, evidence‐based policy is currently trapped in an epistemological conundrum in which a clear statement of uncertainty is the most responsible policy.
Acknowledgments: The authors would like to thank epidemiologist Katherine Keyes of Columbia University for her invaluable advice during this project.
Address correspondence to: Gerald M. Oppenheimer, Mailman School of Public Health, Columbia University, 722 West 168th Street, 9th floor, New York, NY 10032 (email: email@example.com).
Read on Wiley Online Library
Published in 2019 DOI: 10.1111/1468-0009.12437
Get the Latest from the Milbank Memorial Fund
The Milbank Quarterly’s multidisciplinary approach and commitment to applying the best empirical research to practical policymaking offers in-depth assessments of the social, economic, historical, legal, and ethical dimensions of health and health care policy.