Health: New Study Casts Doubt on Claim
New Study Casts Doubt on Claim that Smoking Bans Substantially Reduce Heart Attack Admissions
New Study Concludes that Smoking Ban in Arizona Decreased Heart Attacks, Despite Increase in Heart Attacks in Most of the State
was seen in counties with previous bans.”
Professor
Department of Community Health Sciences
Boston University School of Public Health
801 Massachusetts Avenue, 3rd Floor
Boston, MA 02118
New Study Concludes that Bowling Green Smoking Ban Reduced Heart Disease Admissions by 47%; Unfortunately, Science is Weak and Conclusions Unjustified
The study concludes: “A reduction in admission rates for smoking-related diseases was achieved in Bowling Green compared to the control city. The largest reduction was for coronary heart disease, where rates were decreased significantly by 39% after 1 year and by 47% after 3 years following the implementation of the ordinance. … The findings of this study suggest that clean indoor air ordinances lead to a reduction in hospital admissions for coronary heart disease, thus reducing health care costs.”
The Bowling Green ordinance eliminated smoking in public places, including restaurants, but exempted free-standing bars. Bar areas of restaurants were also exempted, as long as they were isolated in an enclosed room.
The Rest of the Story
Unfortunately, the study conclusions are, in my view, unsupported by the data, and the science backing up the study is quite poor. Like the other studies which have claimed to have found a drastic reduction in heart attack admissions attributable to a smoking ban, this one is yet another example of shoddy science making its way into tobacco control research.
The chief flaw of the study is that it is unable to rule out the very likely possibility that the observed changes in heart disease admission in Bowling Green during the study period are due primarily to random variation, rather than to the smoking ban.
To see what I mean, let’s look at the actual data. Here are the annual standardized heart disease admission rates for Bowling Green during the study period. Note that the ordinance went into effect in March 2002:
1999: 35
2000: 24
2001: 24
2002: 36
2003: 22
2004: 26
The paper presents data for 2005, but since only the first six-month period of data are available, it is not valid to compare the 2005 data with the preceding year. The paper simply doubles the admission rates from the first six months, but that is invalid due to the well-established seasonal variation in these rates.
You can see several important things by examining these data.
First, there is tremendous natural (random) variation in the heart disease admission rates in Bowling Green. Because we are dealing with such small numbers of admissions, the percentage change in admissions from one year to the next is very high, even without any smoking ban. For example, from 1999 to 2000, there was a 31% decline in admissions. From 2001 to 2002, there was a 50% increase in admissions. Clearly, these changes were not due to the smoking ban. They reflect, at least in part, the underlying random variation in these data.
Given the fact that annual changes in heart attack admission rates of between 30% and 50% are common in Bowling Green, it is completely unjustified to conclude that the observed 39% decline in the first year following the smoking ban was attributable to the smoking ban.
More likely, it was simply random variation that led to an “abnormally” high heart disease admission rate in 2002. The rate was bound to fall in 2003 due simply to this pattern of random variation.
Does the study conclude that the absence of a smoking ban in Bowling Green caused the 31% decline in admissions in Bowling Green from 1999 to 2000? Of course not. That “drastic” decline was due to absolutely nothing. Just random variation in the data.
Does the study conclude that the whopping 50% increase in heart disease admissions between 2001 and 2002 was due to the implementation of the smoking ban? Of course not. That would be an unfounded conclusion given the high degree of random variation in these data.
The second very important thing to notice is that the rate of heart disease admissions in Bowling Green was exactly the same before the smoking ban as after the smoking ban. In 2001, the year prior to the smoking ban, the rate was 24. In 2003, the year after the ban, the rate was 22. In 2004, two years after the smoking ban, the rate was 26. Thus, the average rate in the first two years following the smoking ban was 24 – exactly the same as it was prior to the ban.
How can one possibly conclude that the Bowling Green smoking ban decreased heart disease admissions by 47% when the rate in 2000 and 2001 (prior to the ban) was 24, and the rate in 2004 (after the ban) was 26?
The conclusion of the study, it turns out, is based heavily upon the low admissions rate during the first six months of 2005. You can’t possibly draw any valid conclusion until the full 2005 data are in. And you certainly don’t want to just double the 2005 early-year data in order to present the appearance of a very low annual rate for the entire year.
The study findings are also largely dependent on whether you categorize the “abnormally” high rate of 36 observed in 2002 to pre-ban or post-ban. If you call the heart attacks in 2002 mostly post-ban observations, then the high rate in 2002 is attributed mainly to post-ban, and thus there is the appearance of an increase, not a decrease in heart disease admissions immediately following the ban.
On the other hand, if you call the heart attacks in 2002 mostly pre-ban observations, then the high rate in 2002 suddenly becomes attributed to pre-ban, and thus there is created an appearance of a decrease, not increase in heart disease admissions following the ban.
It is quite interesting, then, to note how the paper treated the 2002 data. Although the smoking ban went into effect early in 2002, the study treats the 2002 data as being pre-ban. In reporting the change in heart disease admissions rates during the first year of the ban, the study compares the rate of 36 in 2002 (which it considers pre-ban) with the rate of 22 in 2003 (post-ban). This yields an estimate of a 39% decline in admissions.
The problem is that most of that rate of 36 in 2002 is actually post-ban, since it went into effect in March of that year. If you want a true idea of the pre-ban rate, go back to 2001 which is unequivocally pre-ban. The rate in 2001 was 24. The rate in 2003, which is unequivocally post-ban, was 22. That represents a decrease of 8%, not 39%.
It seems odd that the paper uses the rate of 36 in 2002 as the pre-ban rate to compare both the one-year change and three-year change (for which the paper reports a drop of 47% – from 36 to 19). The reason I say it is odd is because this rate was actually a post-ban rate, for the most part.
The paper justifies its characterization of the 2002 data as pre-ban by arguing that a full six months of enforcement are necessary before health effects from reduction in secondhand smoke exposure or reduction in smoking prevalence or smoking intensity would be observed: “Due to the novelty of the ban, initial resistance by its opponents and legal wrangling over its enforcement, we believed that several months of consistent enforcement would be needed before citizens would actually change their behavior.”
It sounds to me like this is more of a convenient excuse for treating the high observed post-ban heart disease admission rate in 2002 as pre-ban than it is an objective way to conduct this analysis. I’m not suggesting that this was intentionally done to try to make it appear that there was a decline in heart disease rates; I’m just pointing out that this type of manipulation is highly subjective, and given the actual data in question, is unacceptable. It basically throws the entire analysis into question.
Given the question about what points exactly should be considered pre-ban versus post-ban, the most objective way to handle this would be to use the 2001 data as pre-ban and the 2003 data as post-ban. These categorizations are unequivocal. And if you do that, you find a drop from 24 to 22 — hardly evidence of any substantial effect of the smoking ban.
And if you go to the second year post-ban, you find an increase from 24 to 26. Again, this is hardly evidence of a decline in heart disease admissions due to the smoking ban.
I should also point out that the assumption that it takes several months of consistent enforcement before people change their behavior is unsupported. The data I have published from Boston’s smoking ban demonstrates that within the first few days of the ban, there was basically 100% compliance. The change in secondhand smoke exposure was almost immediate.
There are several other serious flaws in the study.
The study provides no evidence to support its assertion that there was a drastic drop in secondhand smoke exposure, cigarette consumption, and smoking prevalence in Bowling Green in response to the smoking ban. If you are going to conclude that the smoking ban is what was responsible for the observed changes in heart disease rates, you ought to document that there actually was a dramatic reduction in secondhand smoke exposure, cigarette consumption, and smoking prevalence. The paper does none of these things.
The study also makes another serious mistake. It uses a fancy statistical model (an ARIMA – or autoregressive integrated moving average model) to estimate the change in monthly heart disease admission rates in those months in which the smoking ban was in force. It finds that in Bowling Green, the rate was 1.7 lower per month when the ban was in force, and that estimate is statistically significant. In Kent, the comparison community, the rate was 1.1 lower per month when the ban was in force, and that estimate was not statistically significant.
The paper then concludes that since the 1.1 per month decline in admission rate in Kent was not statistically significant, these data do not show a parallel significant change in heart disease admission rates in Kent following the implementation of the smoking ban in Bowling Green.
However, this is not the appropriate way to conduct this analysis. The proper way to compare these estimates is to statistically determine whether the observed decline of 1.7 per month in Bowling Green is different from the observed decline of 1.1 per month in Kent. Since we know the standard errors of each of these estimates, we can determine whether these estimates are statistically different from each other.
Since the paper does not provide the standard errors, I cannot conduct that analysis. However, given the levels of statistical significance for these estimates in the paper, I was able to make a rough estimate of what the standard errors likely were. Based on my calculations, it is highly likely that the decline in heart disease admission rate of 1.7 per month in Bowling Green is not statistically different from the decline of 1.1 per month in Kent. In other words, it is likely that the actual analysis in the paper confirms that there was no significant decline in heart disease admission rates in Bowling Green due to the smoking ban.
I recognize that this may be a difficult point to understand, so let me give an example to illustrate it. Suppose that I want to determine whether the smoking ban in Massachusetts resulted in an increase in average temperatures in Massachusetts compared to temperatures in New Hampshire. We want to therefore compare the change in annual mean temperatures in Massachusetts with the change in annual mean temperatures in New Hampshire. We find that in Massachusetts, the average mean temperature increased by an average of 0.5 degrees per year, with a standard error of 0.1. Thus the increase was statistically significant. In New Hampshire, the average mean temperature also increased by an average of 0.5 degrees per year, but with a standard error of 0.3, so that the increase was not statistically significant.
By the reasoning provided in the paper, one would conclude that there was no parallel significant increase in temperatures in New Hampshire; thus, the change in Massachusetts must have been due to the smoking ban. However, it is readily apparent from these data that there was exactly the same observed increase in temperature in the two states. The correct way to do this analysis is to compare the two estimates of the average annual decline in temperatures and see if they are statistically different. In this case, the two estimates are 0.5 and 0.5, which are clearly not different. This shows how one can draw the wrong conclusion if one conducts the analysis in the wrong way.
The final point that deserves mention is that the smoking ban in Bowling Green was a partial smoking ban. It exempted free-standing bars and bars within restaurants. Thus, it is much less plausible that such a ban would have had a dramatic effect on smoking prevalence. Smokers likely chose to go out to restaurants or bars that continued to allow smoking. There is little evidence that partial smoking bans result in significant smoking cessation.
The rest of the story is that upon closer examination, the study which purports to demonstrate that a smoking ban in Bowling Green resulted in a massive decline in heart disease admissions demonstrated nothing of the sort, and possibly demonstrated that there was no significant decline in admissions attributable to the smoking ban. Like its predecessors (e.g., Helena and Pueblo), this is another example of shoddy science that apparently now passes as acceptable in tobacco control research.
New Study Casts Doubt on Claim that Smoking Bans Substantially Reduce Heart Attack Admissions
The authors, David W. Kuneman and Michael J. McFadden, analyzed data on hospital admissions for acute myocardial infarction from the HCUP project (Healthcare Cost and Utilization Project), which is “a family of health care databases and related software tools and products developed through a Federal-State-Industry partnership and sponsored by the Agency for Healthcare Research and Quality. HCUP is based on statewide data collected by individual data organizations across the United States and provided to AHRQ through the HCUP partnership. … HCUP data are used for research on hospital utilization, access, charges, quality and outcomes. … Researchers and policymakers use HCUP data to identify, track, analyze and compare hospital statistics at the national, regional and State levels.”
Specifically, the authors examined the total number of hospital admissions for acute myocardial infarction in the year prior to and after a smoking ban in each of four states which enacted some form of smoking ban in restaurants and/or bars during the period for which data from HCUP are available (1997-2003): California, New York, Florida, and Oregon.
For California, a complete ban on smoking in bars was implemented in January 1998. Heart attack admissions increased from 40,608 in the year preceding the bar smoking ban (1997) to 43,044 during the year following the smoking ban (1998), an increase of 6.0%.
For New York, a complete ban on smoking in bars and restaurants was implemented in July 2003. Heart attack admissions increased from 31,728 in the year preceding the bar smoking ban (2002) to 31,888 during the year in which the smoking ban was implemented (2003), an increase of 0.4%.
For Florida, a ban on smoking in restaurants (free-standing bars excluded) was implemented in July 2003. Heart attack admissions decreased from 40,077 in the year preceding the bar smoking ban (2002) to 39,783 during the year in which the smoking ban was implemented (2003), a decrease of 0.7%.
For Oregon, a ban on smoking in restaurants which allow children was implemented in July 2001. Heart attack admissions increased from 4,957 in the year preceding the ban (2000) to 5,125 in the year following the ban (2002), an increase of 0.4%, and there was almost no change in heart attack admissions during 2001 — the year in which the ban was implemented (4,927, 0.1% decrease from 2000).
The authors point out that none of these findings provides any suggestion that the statewide smoking bans had any immediate and substantial effect on heart attack admissions.
The authors point out that while the total number of heart attacks studied in Helena and Pueblo totaled 315, the total number of heart attacks in this study was over 315,000, or 1,000 times higher. They suggest that this larger sample size as well as the examination of state-wide data rather than just data in isolated cities makes the conclusions from this study more stable than from the existing studies on this topic.
The paper concludes: “Statistically this larger population base makes for a far more stable statistical environment and the data from this population would provide a far sounder scientific basis for decisions about smoking bans that will affect the lives and livelihoods of millions of people.”
The Rest of the Story
In addition to confirming Kuneman and McFadden’s findings, I extended their analysis by:
- examining trends going back in time as far as 1997, the earliest available online data (in order to have a more stable baseline period to establish secular patterns); and
- examining trends in heart attack admissions in all the other states in the online database without smoking bans that included data for the entire study period 1997-2003 (a total of 8 states – New Jersey, South Carolina, Utah, Washington, Arizona, Colorado, Hawaii, and Iowa; Massachusetts was not included because of the extensive local smoking bans) (in order to have a comparison group).
For New York, overall trends were similar to those in the comparison states and to the nation as a whole, except that New York did not experience the slight decline in heart attack admissions during 2003 that was observed elsewhere. In New York, admissions for heart attacks increased by 0.4% from 2002 to 2003, while heart attacks decreased by 3.1% in the comparison states and by 2.8% nationally during the same time period.
For Florida, heart attack admissions increased slightly faster than in the comparison states between 1997 and 2000, but the patterns were similar from 2000-2003. There was a slight decrease in heart attacks between 2002 and 2003 in Florida (0.7%), the comparison states (3.1%), and the nation as a whole (2.8%).
For Oregon, there was a 0.4% increase in heart attack admissions from 2000 to 2002, while admissions in the comparison states dropped by 0.7% during the same period, and admissions nationally increased by 4.3%.
Commentary and Conclusions
I think Kuneman and McFadden are to be congratulated for having made an important contribution to the analysis of this research question. I think that their analysis, along with my extension of that analysis, provides compelling evidence that brings into question the conclusion that smoking bans have an immediate and drastic effect on heart attack incidence.
In fact, these analyses demonstrate that on a state-wide level, there is no suggestion of any large-scale effect on heart attack admissions associated with the implementation of statewide bans on smoking in child-friendly restaurants, all restaurants, bars, or bars and restaurants.
If there were a true 27% or 40% decrease in heart attack admissions due to smoking bans that occurred almost immediately (within six months, as claimed), one would have expected to see a demonstrable decline in such admissions in states that implemented such bans.
An effect of such smoking bans can certainly not be ruled out, especially because the 2004 data for New York and Florida are not yet available (so only the first six months post-ban could be examined). However, it does seem quite clear that if there is an effect, it is not nearly as immediate or as dramatic as suggested in press releases. (see also Pueblo release and Bowling Green press release and Greeley news article)
My honest appraisal of what is going on here is that anti-smoking groups have been too quick to go to the media with definitive claims of a drastic and immediate effect of smoking bans on heart attacks when the scientific evidence is simply not sufficient to support such claims. What is happening, I believe, is that the anti-smoking agenda is driving the interpretation of the science. As I stated before, it is an agenda which, in this case, I wholeheartedly support (I have been lobbying for workplace smoking bans, especially those in bars and restaurants for 21 years). However, I don’t think the importance of the ultimate objective justifies the use of shoddy science to support that objective.
At this point, I must make 3 critical points:
First, I am not suggesting that there was anything wrong with the studies that were done in Helena and Pueblo or that the authors did anything wrong in stating their conclusions within the Helena paper. What I am suggesting, instead, is that drawing definitive, generalized conclusions based on these two small studies, and sending out press releases to the media with these definitive conclusions (before the Pueblo study has even been published) is irresponsible and undermines the scientific credibility of the tobacco control movement.
Within themselves, it may be that the Helena and Pueblo studies are quite solid (I have argued not with respect to the Pueblo study, but there is room for differing interpretations of the evidence); however, that doesn’t mean that the evidence is sufficient to support a general conclusion that smoking bans reduce heart attacks by 27-40%. The fact that population-wide studies with much larger sample sizes do not seem to bear out these findings is exactly the reason why one has to be careful in concluding an effect with a small and select sample (and especially, in the face of huge random variations in secular trends in a small geographical area).
Second, I am not suggesting that this is a reason not to support smoke-free restaurant and bar laws. In fact, one of the things that I think tobacco control groups have been doing wrong is using data such as this to support such ordinances. I think the reason for these laws is that secondhand smoke is a substantial workplace hazard for bar and restaurant workers. That’s it. Whether the laws end up reducing heart attacks (probably by virtue of smokers quitting or cutting down) or not is not relevant in my mind to the issue of whether we should protect workers from a substantial and preventable occupational hazard.
I think by harping on these data, anti-smoking groups have set themselves up for failure, and therefore done a disservice to the overall effort to protect workers from secondhand smoke. Because now that valid scientific doubt is being cast on this exaggerated claim, it may well hurt the effort to protect these workers.
This is what I meant when I suggested that the credibility of the movement is being threatened by the tactics being used. If the focus of the debate shifts to the validity of the heart attack reduction claim rather than the need to protect workers from a severe and preventable occupational hazard, then we may well lose the debate. I fear this is now going to happen now that the “cat is out of the bag.”
Third, and finally, I am not concluding here that smoking bans do not reduce heart attacks. I am not even concluding that smoking bans did not reduce heart attacks in Helena or Pueblo. What I am concluding is that the overall evidence does not support the conclusion that the observed declines in heart attack admissions in Helena or Pueblo (or Bowling Green or Greeley) are in fact: (1) real, rather than simply chance variations; (2) attributable to the smoking ban, rather than some other factor; and (3) widely generalizable to other communities.
It is possible, for example, that local smoking bans may have an effect that state-wide smoking bans do not have. Perhaps all the local media attention focuses public attention on the matter and results in publicity that ends up changing smoking behavior. And perhaps that doesn’t happen as effectively at a state level. But I think a lot more research is needed before we can conclude that the reason why we don’t observe a substantial reduction in heart attack admissions associated with smoking bans on a state-level is that the effect only holds at a local level.
Moreover, I would point out that in my analysis of trends in heart attacks in Massachusetts, where there was a huge proliferation of smoke-free bar and restaurant regulations between 2000 and 2003, I found that heart attack admissions increased in Massachusetts by 31.8% during this time period, compared to a 2.4% decline in the comparison states, and a 1.5% increase nationally.
In short, what I am concluding is that it is far too premature to conclude that smoking bans reduce heart attacks substantially and in a short period of time. And that as much as anti-smoking groups are doing important work by promoting smoke-free bar and restaurant laws, it simply is not responsible (nor wise strategically, I think) to be using shoddy science to support this cause. In the long run, it is our credibility (and ultimately therefore, the health of the public) that is going to lose out.