Tobacco Control: The Long War – When the Evidence Has to Be Created…?
Produced in Collaboration with the Centers for Disease Control and Prevention (CDC)
Improving Population Health:
The Uses of Systematic Reviews
Melissa Sweet and Ray Moynihan
Tobacco Control: The Long War—When the Evidence Has to Be Created
Summary: Fearless advocacy and a willingness to experiment with novel approaches drove a ground-breaking tobacco control program in California. It had a dramatic impact in changing community attitudes towards smoking and reducing tobacco consumption in the early 1990s, but has subsequently lost some of its impact. The program, and others that followed elsewhere, contributed to the development of an evidence base, which includes many systematic reviews, on effective tobacco control interventions.
Public health history was made in November 1988 when Californians voted for a citizen-initiated law called Proposition 99. It increased tobacco taxes and devoted 20 percent of the money raised to tobacco control, creating the world’s largest tobacco control program almost overnight. However, this victory marked only the beginning of a long and difficult battle on behalf of what has been described as one of the most intensive, aggressive public health interventions ever undertaken.53 The challenge for Californian policymakers, who were breaking new ground in the early 1990s, was how best to spend their windfall—about $100 million per year—while fighting ongoing battles with antagonistic political, professional, and industry interests.
Dileep Bal was one of the key tobacco warriors within the California Department of Health Services for fifteen years, until recently moving to his current position as district health officer at the Hawaii State Department of Health. While at the California Department of Health Services, Bal oversaw the distribution of funding to one of the first statewide anti-tobacco media campaigns, as well as to local health departments, community-based organizations, schools, and other agencies to promote tobacco prevention and cessation. Bal says about 30 percent of tobacco control funds was allocated to media campaigns, about 30 percent to local health departments, about 30 percent to competitive grants, and 10 percent to evaluation. The centerpiece was the innovative media campaign, which, rather than taking the traditional health promotion approach of warning that “smoking will kill you,” attacked the tobacco industry directly and aimed to change social norms around smoking. The goal was to overturn industry marketing portraying smoking as sexy and cool.
“For a government program, it was very, very atypical,” recalls Bal, whose job came under threat many times. “It was very unusual for government to take on what was referred to as a legally constituted industry. I elected to take on the tobacco industry head-on and I have the scars on my back to show for it.”
Bal laughs when asked about the role of scientific evidence in guiding policy decisions. “There was no science on how to do a community intervention on something of this global dimension,” he says. “Where there is no science, you have to go and be venturesome—you can’t use the paucity of science as an excuse to do nothing. We created the science, we did the interventions and then all the scientists came in behind us and analyzed what we did.”
Bal is frustrated by colleagues who want to wait for high-level evidence before acting. “Public policy must be propped up by science but must not be a handmaiden of the science,” he says. “Most scientists will say you need a randomized controlled trial level of proof to do a community intervention. That’s horse feathers. We tried twenty-five things—twelve worked and we renewed those. Empirical trial and error is the oldest scientific device and we used it to distinction.”
Another critical aspect of the program, says Bal, was having advocates outside the bureaucracy who were fearless in speaking out. These included Stanton Glantz, professor of medicine at the University of California, San Francisco, and his colleague John Pierce, a professor at the University of California, San Diego (UCSD). Both Glantz and Pierce documented the program’s initial dramatic success in dropping smoking rates between 1989 and 1993, before cutbacks in funding and political support stymied further progress.54 More recently, the program has been described as stagnant and facing an uncertain future.55
Despite the setbacks, there is no doubt the program has had a dramatic impact—and not only because in its first eight years, it is estimated to have prevented 58,900 deaths between 1989 and 1997.56 Its impact reached beyond California’s borders, providing other jurisdictions with confidence that smoking rates could be dropped and guidance about what interventions might work. California’s contribution to knowledge about tobacco control was particularly important after the unprecedented, multibillion dollar settlement reached between the tobacco industry and most state governments in 1998.
Not all states used these funds for tobacco control, but the settlement contributed to many new anti-tobacco programs. Tobacco control is one of the most common reasons that policymakers turn to the Community Guide,57 which lists more than a dozen systematic reviews in the areas of preventing the initiation of tobacco product use, increasing cessation, and reducing exposure to environmental tobacco smoke. The Community Guide also includes economic evaluations for many recommended interventions.
Systematic reviews have also helped policymakers to appreciate the unreliability of research and review articles supported by the tobacco industry.58 One such investigation found that review articles concluding secondhand smoke was not harmful were about ninety times more likely to be written by authors with financial ties to the tobacco industry rather than those without such ties. Another systematic review found that lower quality studies were much more likely to conclude that smoke-free regulations adversely affect the hospitality industry and were also much more likely to be industry-funded. This review would not have had such an impact on policymakers, according to one of its authors, Michelle Scollo, a researcher with the Cancer Council Victoria in Australia, if the researchers had not been proactive in presenting the findings to politicians and public forums using simple, straightforward language and a graphic PowerPoint presentation. Effective communication of systematic reviews requires “persuasive speech” to help policymakers connect the evidence with their constituency, adds Lisa Bero, a professor and a prominent member of the Cochrane Collaboration, based at the University of California, San Francisco. “It’s not so much about spin but about making the numbers come alive,” she says.
Many experts believe that reports issued by prominent authorities, such as the U.S. surgeon general, have been more influential in driving policy than systematic reviews, which were scarcely cited in an online poll by the journal Tobacco Control of the most important and influential PubMed papers in tobacco control.59 Bero says it is unfortunate many of the landmark reports on tobacco control have been consensus documents, rather than the product of the methodology and rigor of a systematic review. “If there’s a way we can introduce the systematic review methodology more into these reports, not only would they be more influential, they’d be stronger,” she says. There is also room to increase the role of evidence in tobacco control advocacy, she adds.
Bero also points out that systematic reviews played an important role in tobacco control by establishing evidence of the harm caused by secondhand smoke exposure. “Without that evidence of harm, there would have been no need to make policy at all,” she says.
A solid evidence base is only one element of effective advocacy, argues Simon Chapman, professor of public health at the University of Sydney, Australia, and editor of Tobacco Control. He cites a survey of Australian policymakers and advocates, suggesting that economic, ideological, and anecdotal arguments often outweigh scientific evidence in policy decisions about environmental tobacco smoke. Politicians and political advisors told the researchers doing the survey that evidence has limited power in governmental deliberations. One political advisor said “evidence” presented by any lobbyist is viewed skeptically because lobbyists tend to spin their findings to suit their positions. A politician suggested that anecdotes and “folklore” carry more weight in political conversations than scientific studies and other forms of independent evidence.60
In the United States, some of the leading players in tobacco control policy are skeptical about the value of systematic reviews in population health policymaking. Lawrence Green, adjunct professor in the Department of Epidemiology and Biostatistics at the University of California, San Francisco, who was closely involved in evaluating tobacco control programs for the WHO and the CDC, argues that many systematic reviews of population health interventions have been “sterile” and of little help to policymakers. “They’ve produced a lot of studies that are high in internal validity [to help maximize their chances of demonstrating effectiveness] but relatively low in external validity [their relevance, generalizability, and applicability to real-world situations are questionable],” he says. “This means they screen out a lot of studies that might have greatest relevance for policymakers and leave in highly controlled studies, which as a result of all the experimental controls have become somewhat artificial.”
Green says his experience evaluating the California tobacco control program suggests that its comprehensive approach was the reason for its success, rather than the effectiveness of any one intervention. “School-based programs may not have much effect by themselves, but when combined with community efforts, they take on a whole different meaning,” he says. “And when they’re supported further by the mass media, it takes on even more meaning. And the mass media wouldn’t be effective without what’s going on in the home. And then effective enforcement of what kids can buy across the counter is also important. Each of these interventions by themselves may have limited effect if not supported by the others but combined you get a synergistic effect.”
When Green was working for the CDC, helping to promote a manual called Best Practices in Comprehensive Tobacco Control, he says they “took some heat” from headquarters because they broke rank with the use of the term best practices by going beyond controlled trials for evidence. They pushed for the manual to include case studies from California and Massachusetts rather than limiting it to systematic reviews of highly controlled studies. Public health differs from evidence-based medicine because it is intervening across very complex systems and populations rather than across the relatively homogenous biological organisms called patients, he says. “In public health we can’t afford to be too precious about our definitions of evidence,” he says. “It was the comprehensiveness of the approach that we wanted to represent to the other forty-eight states as best practice. And it was the real-world, real-time experience of the two states that made the manual more credible to other states than if it had been limited to carefully controlled studies in more academic settings and circumstances.”
As previously discussed, systematic reviews can include many methodologies other than randomized controlled trials. Pierce, director of the Division of Population Sciences, Moores UCSD Cancer Center, believes randomized controlled trials are best suited to studies of plants and pharmaceuticals and do not provide a useful basis for population health decisions. Their use is particularly limited in communication studies where the intervention cannot be controlled. “In reality, people change their behavior because of social norms, and the rules for systematic reviews frequently leave out studies that address social norms,” he says. “Very few people have done good evaluative studies in which social norms were changed.”
The ideal tobacco control program doesn’t exist and population-based programs are political and unpredictable. “We have some things that work in some places, some things that don’t,” Pierce says. “A population program is more like a rodeo—if I’m on a bull, all I can do is tell you where I’ve been, I can’t tell you where I’m going. It’s not like having a map. Things change in front of you, you can’t control the events.”
• Changing social norms through legislative and regulatory changes as well as innovative media campaigns is an effective way of influencing individuals’ behavior.
• Achieving policy change requires courage, persistence, and a thick skin. A reliable evidence base also helps.
• A comprehensive range of interventions working at multiple levels is more likely to have an impact than any single intervention.
• Systematic reviews are more likely to have an impact if efforts are made to disseminate and “sell” their findings.
• Systematic reviews are more likely to have an impact if they address issues of external validity and relevance as well as issues of internal validity and bias.
• Credible evaluations of state programs and policies are needed to continue to add to the evidence base.
More Anti Smoking Playbooks