The Piedmont study is an example of junk science and as much as we in the tobacco control field would like to accept its conclusions, doing so is going to make us hypocrites, destroying our credibility.
Secondhand data on secondhand smoke
By Jerome Arnett, Jr.
HELVETIA, W.Va.–The fed eral government’s 30-year anti-smoking crusade has been so successful that there are now more ex-smokers than smokers in the United States. But about a quarter of the population continues to smoke cigarettes, and over the past decade a new health hazard has been fabricated and publicized.
Piedmont Study Methodology is Similar to Studies Showing Adverse Economic Effects of Smoking Bans; Junk Science Cuts Both Ways
By Michael Siegel
A careful examination of the methodology used in the Piedmont study which purported to show that the smoking ban in Italy reduced heart attack admissions by 11% reveals that it is essentially the same as that used by a number of studies, attacked by anti-smoking groups, which found that restaurant smoking bans have an adverse economic impact on businesses.
The Piedmont study compared age-standardized heart attack admission rates of Piedmont residents during the 5-month period February-June 2005, which immediately followed the smoking ban (implemented in January 2005), with heart attack admission rates during the same period (February-June) for the previous 4 years.
The Rest of the Story
Here is a brief review of studies using similar methodology to that used in the Piedmont study, which concluded that smoking bans caused an adverse economic impact on restaurants.
1. Laventhol & Horwath, 1990
Laventhol and Horwath conducted a study that was quite comparable to the Piedmont study in terms of the methodology used to assess the effect of a smoking ban (Laventhol & Horwath. Preliminary Analysis of the Impact of the Proposed Los Angeles Ban on Smoking in Restaurants. Los Angeles: Laventhol & Horwath, 1990).
This study compared restaurant sales in Beverly Hills during a 3-month period (2nd quarter, 1987) following the implementation of a restaurant smoking ban to the comparable period the year earlier (2nd quarter, 1986). As in the Piedmont study, the authors excluded data from the first month in which the ordinance was in effect, considering this to be a transition period.
The authors found a 6.7% decline in restaurant sales in Beverly Hills, which they attributed to the smoking ban. Unlike the Piedmont study, these authors did go to the trouble of including a comparison group in the study. They examined changes in restaurant sales in Los Angeles during the same period and found that there was a 10.3% increase.
2. Masotti & Creticos, 1991
Masotti and Creticos also examined the effects of a smoking ban using nearly the same methodological approach as in the Piedmont study (Masotti L, Creticos P. The Effects of a Ban on Smoking in Public Places in San Luis Obispo, California. Northwestern University and Creticos & Associates, Inc., 1991).
They compared sales tax receipts indicating the level of restaurant sales in San Luis Obispo (California) during a 5-month period (exactly the same amount of time) following the smoking ban’s implementation to restaurant sales during the comparable period during the previous year.
Compared to the 3rd and 4th quarters of 1989, there was a drop of 3% and 26%, respectively, in restaurant sales in the 3rd and 4th quarters of 1990 (the smoking ban went into effect during the 3rd quarter of 1990).
Unlike the Piedmont study, a comparison group was included (actually, two). The study finds that there was no corresponding decline in restaurant sales for the same time periods within the county of San Luis Obispo as a whole or in the state as a whole. In fact, there were slight increases in restaurant sales (2.6% and 4.2% for the 3rd quarter and 2.0% and 2.4% for the 4th quarter).
3. Lilley & DeFranco, 1999
Lilley and DeFranco studied the change in the number of bars and number of bar employees in California before and after the statewide bar smoking ban went into effect (Lilley W, DeFranco L. The Impact of Smoking Restrictions on the Bar and Tavern Industry in California. Washington: InContext Inc., October 26, 1999).
The ban was implemented on January 1, 1998. So Lilley and DeFranco compared the number of bars and bar employees on January 1, 1997 to the corresponding figures for January 1, 1999. They reported a 7.4% decline in the number of bars and a 12.7% decline in the number of bar employees, which they attributed to the smoking ban.
As a comparison, the report examined overall retail trade trends, finding that during the same period, retail trade employment increased by 4.2% and total employment was up by 5.7%.
Like the Piedmont study, this report stratified the data to look specifically at effects among the bars which the authors expected to suffer the most from a smoking ban: smaller bars. In fact, they found that the decrease in establishments and employment was most dramatic among small bars with 5-9 employees (17.9% drop in establishments; 16.4% drop in employment).
4. Lilley and DeFranco, 1996
Another study by Lilley and DeFranco examined trends in the number of restaurant jobs in New York City before and after its restaurant smoking ban (Lilley W, DeFranco L. Restaurant Jobs in New York City, 1993 Through First Quarter 1996, and the Restaurant Smoking Ban. Washington, DC: InContext Inc., 1996.).
Compared to January 1, 1993 (prior to the smoking ban), the number of restaurant jobs in New York City declined by 2,779 by the first quarter of 1996 (about one year after the smoking ban went into effect). This represented 4.0% of New York City’s restaurant job base. In contrast, the political jurisdictions surrounding New York City gained 1,937 restaurant jobs during the same period, representing 5.0% of the restaurant job base. The paper attributes these changes to the smoking ban.
Conclusion
The rest of the story is that the same methodology relied upon by anti-smoking groups to tout a dramatic effect of smoking bans on heart attack rates has been used to demonstrate a dramatic effect of smoking bans on restaurant sales. But while the latter studies have been condemned by anti-smoking groups and called junk science, the former studies are being used to support widespread public claims.
It appears that the anti-smoking movement judges the quality of science by the nature of its findings. Here, we have a number of studies that used essentially the same methods. The studies which produced results that hurt the anti-smoking agenda were attacked and trashed by the tobacco control movement; yet when studies with essentially the same methodology produced results that support our agenda, we tout those same studies around as proof that our agenda is supported by this science.
In other words, when the science produces favorable results, it is science; when it produces unfavorable results, it is junk.
I’m afraid that there is a double standard here. Our scientific judgment appears to be obscured in a mesh of hypocrisy.
One anti-smoking group’s website is devoted to debunking a series of what it claims are invalid studies which reported an adverse economic impact of smoking bans. Interestingly, one of the criteria that the TobaccoScam web site sets forward in determining what makes a “bad” study is the failure to use at least one full year’s worth of data. If anything, the need for a full year’s worth of data is even more important for a study of changes in heart attacks than restaurant sales, since one would expect that there could be drastic changes in restaurant sales that could occur very rapidly, while changes in heart attack rates would take considerable time to be realized.
Based on the criteria set forth on the TobaccoScam website, the Piedmont study would be classified as a “bad” study, in the category of “Cooking the Books.”
In other words, the methods used in the Piedmont study were characterized by anti-smoking groups as being an attempt to “cook the books” and artificially find an adverse economic impact of smoking bans. These methods, however, are apparently rock solid as long as the resulting finding is a favorable one. The criteria set forth on the TobaccoScam web site don’t seem to apply to tobacco control studies, only to those conducted by our “opponents.”
Also of interest is the fact that many of the above economic impact studies were paid for, or commissioned by the tobacco industry. What this means is that we are essentially using the same techniques that the tobacco companies used to try to demonstrate an adverse economic impact of smoking bans to demonstrate a positive immediate health benefit of these policies.
Are we so biased that our desire to produce results that support our agenda is clouding our scientific judgment enough that we are willing to condemn a study with unfavorable findings but tout around a study of essentially the same methodology with favorable findings?
It appears that the answer is yes. At least I will be forced to assume the answer is yes until I see someone or some group from within tobacco control which is willing to criticize the Piedmont study and others like it, as they did the tobacco industry-sponsored studies which used similar methodology to find an adverse economic impact of smoking bans.
Piedmont Italy Study Becomes the Latest in a String of Junk Science Papers on Effects of Smoking Bans on Heart Attack Admissions
By Michael Siegel
An article published online in the European Heart Journal concludes that a national smoking ban in Italy resulted in an 11% decline in heart attack admissions in the Piedmont region of northern Italy during the first 5 months following its implementation.
The study compared age-standardized heart attack admission rates of Piedmont residents during the 5-month period February-June 2005, which immediately followed the smoking ban (implemented in January 2005), with heart attack admission rates during the same period (February-June) for the previous 4 years. Rates were examined separately for men and women and for persons older than or younger than 60.
Among those older than 60, there was a small increase (5%) in the heart attack admission rate from 2004 to 2005. For those under 60, the heart attack admission rate decreased by 11%.
The actual rates, by year and sex for those under age 60 were:
MEN
2001: 1.21
2002: 1.25
2003: 1.31
2004: 1.35
2005: 1.24
WOMEN
2001: 0.22
2002: 0.22
2003: 0.19
2004: 0.24
2005: 0.19
As a comparison, heart attack admission rates during the 3 months prior to the smoking ban (October-December 2005) were compared to rates during the same period one year earlier. Among those younger than 60, the rate increased by 6%.
The paper concludes that the smoking ban resulted in the observed 11% decline in the heart attack admission rate in Piedmont.
The Rest of the Story
This is an example of what I would consider to be junk science.
To isolate a five-month period during one year following a smoking ban, compare it to the same five-month period during the previous year, observe a decrease in heart attack admissions, and conclude that the decline was due to the smoking ban is not solid science. It is more on line with what I would term pure speculation.
In fact, this is the precise type of methodology that we in tobacco control have attacked as being unreliable in concluding that smoking bans have resulted in a decline in restaurant sales. Tobacco industry commissioned or funded studies, using the same methodology and finding a decline in restaurant sales associated with smoking bans have been blasted by tobacco control groups as being junk science. There is no reason why we should not view studies using the same methodology in the same way, even though the only real difference is that their findings are supportive of, rather than in opposition to, our agenda.
The biggest problem with this study is that it is impossible to rule out the simplest of alternative hypotheses: that the observed decline in heart attack admissions is simply due to random variation in the data. In other words, it is very possible that the rate of heart attack admissions would have declined in the absence of a smoking ban. It is impossible to tell, and it is impossible to even make a reasonably solid judgment in the absence of any presentation of underlying secular trends over any significant length of time and the absence of any comparison group.
A single point does not indicate a trend, and that is really the fatal flaw of this paper. It is entirely possible that the admission rate simply blipped up a little in 2005 and that in 2006 it will go back down a little. Or that the blip up in 2005 is simply a reflection of an overall trend of decreasing heart attacks during this time period that is not specific to Italy. There’s just no way to know.
If you simply look at the data, you’ll see that concluding that there is a substantial decline in the heart attack rate due to the smoking ban is unfounded. In fact, if you graph out the data for women under 60, it actually appears that the observed decline in admissions in 2005 is due solely to the fact that there was a slight upward blip in 2004.
Based on the heart attack admissions rates among Piedmont women under 60 during the months of February-June from 2001-2003, the observed heart attack admission rate for 2005 should have been 0.16. Instead, it was 0.19. Does this mean that the smoking ban increased the rate of heart attacks among Piedmont women?
The truth is that the observed rate of heart attacks among Piedmont women, ages 60 and younger, during February-June 2005 is exactly the same as it was during the same period in 2003. This doesn’t bode well for a conclusion that there was a substantial decline in heart attacks due to the smoking ban.
The complete absence of a comparison group is another fatal flaw. One simply has no idea what the underlying secular trend in heart attacks was from 2004 to 2005 in the overall region. Thus, it is impossible to attribute any observed decline in the rate to the smoking ban, rather than to a secular trend that would have been observed anyway, even in the absence of the smoking ban.
But the most peculiar aspect of the paper is that the authors felt compelled to stratify their results by age. This is in contrast to any of the previous papers on this topic. So it is somewhat surprising to find this stratification in this paper.
Unfortunately, a closer analysis of the data suggests the reason why the data may have had to be stratified. If one examines the total number of heart attack admissions in Piedmont during the study period, one observes what appears to be an increase in heart attacks, with a 2% increase from February-June 2004 to 2005, the precise comparison period used to draw the study’s major conclusion.
Is it possible that in the original analysis of the data, the paper found this 2% increase, and that the idea of stratifying the data occurred only after the failure to find a decrease in heart attacks when the question was examined in the standard way that it has been looked at in prior research?
The bottom line, that cannot be altered with data manipulation, is that using the same standards of analysis that the authors of the Helena, Saskatoon, and Pueblo studies used, the Piedmont study has demonstrated that the implementation of the smoking ban was associated with a 2% increase in heart attacks. The number of heart attacks from February-June 2005 increased from 3581 to 3655.
The paper argues that the fact that the decline in heart attacks was found only among those under age 60 is evidence that the conclusions are valid. But if you do enough stratification, you’re bound to find some group in which heart attacks declined. To me, this severely weakens, rather than strengthens, the study’s conclusions. The fact that the data had to be manipulated far beyond what has been done in previous research in order to find what appears to be the desired effect greatly weakens the study’s conclusions.
While the paper tries to rationalize its decision to stratify on age based on the supposition that younger people would be more likely to be most affected by a smoking ban, this decision doesn’t jive with the supposed conclusions from Helena and Pueblo. In addition, one could make the argument that if secondhand smoke reductions are going to cause a reduction in heart attacks, this is going to occur among those people with the most severe and brittle coronary artery disease, among whom a slight trigger, such as secondhand smoke, could cause an acute coronary event. But this group is likely to be somewhat older or at least to include older as well as younger individuals.
Another curious quirk of the research, which also comes out of the blue (not done in any of the previous studies on this issue) is the exclusion of most of the data collected in the research. The basic comparison that is made consists only of data from February-June of 2005 versus the same 5 months during 2004. However, the study collected data on heart attacks during the remainder of 2005 and during all the months of the previous 4 years. Why weren’t all of these data used to establish the seasonal and secular trends and random variation in the data and then to examine the complete 2005 heart attack pattern in light of this? Why would one jump to a premature conclusion before even observing the pattern for the entire year?
Perhaps the most interesting aspect of the paper is its conclusion that a reduction in secondhand smoke exposure among nonsmokers caused by the smoking ban could cause an 11% reduction in heart attacks, while the effects of the ban on reducing active smoking could only cause a 0.7% reduction in heart attacks.
As I stated earlier, one would expect the effects of any reductions in active smoking due to smoking bans to have a more substantial effect on heart attacks than any reduction in secondhand smoke exposure (note that active smoking causes far more heart attacks than secondhand smoke). So what the paper has really shown is that one would only expect a very small reduction in heart attacks attributable to a smoking ordinance within a short period of time.
There is another reason why I think one would not expect to see a drastic effect of reduced secondhand smoke exposure on heart attacks within a several month period. In order to expect such an effect, one would have to postulate that secondhand smoke triggers acute cardiac events in persons with severe coronary artery disease, who are basically time bombs waiting to go off. In other words, the slightest insult to the system is capable of triggering a heart attack.
Well, if you eliminate secondhand smoke exposure, these people are still going to be susceptible to any other trigger. Eating a high-fat meal causes endothelial dysfunction, and might also trigger a cardiac event. So it is not clear that simply eliminating secondhand smoke exposure would prevent these individuals from suffering heart attacks.
It seems to me that we’ve set ourselves up for a giant failure. We’ve now led to the world to expect that we’re going to be able to demonstrate drastic and immediate reductions in heart attacks following smoking bans. But I don’t think such an effect is plausible. So when more carefully conducted studies, with longer follow-up periods, are finally conducted, they are most likely not going to find such effects. Then, instead of simply reasoning that one wouldn’t expect dramatic effects, the public is going to conclude that the whole thing was a big hoax. By making this the cornerstone for our arguments in support of smoking bans, when this thing is shown to be untrue, the whole building might come crumbling down.
Although I’ve been quite harshly critical of the conclusions of the Helena and Pueblo studies, the Piedmont study is by far the weakest of the three. There is no comparison group, it fails to analyze all the available data, it is forced to stratify the data in order to find an effect, and it truly uses only one data point following the implementation of the smoking ban.
The study actually finds an increase in heart attacks from 3581 to 3655, a 2% increase, that is associated with the implementation of the smoking ban. Thus, in some ways, this study actually disproves the conclusions from Helena and Pueblo. Yet the data are manipulated in a way that tries to make it appear that there was a dramatic decline in heart attacks. Even accepting the data manipulation, this conclusion is completely unfounded.
The Piedmont study is an example of junk science and as much as we in the tobacco control field would like to accept its conclusions, doing so is going to make us hypocrites, destroying our credibility.
Anti-smoking groups and researchers need to discredit this study’s conclusions in order to make it clear that we have some scientific integrity and that as much as we would like to see dramatic effects from our interventions, we will not disseminate information to the public to support our agenda unless it is based upon solid science.
Smoking Ban Snuffs Heart Attack Hospital Admissions
October 03, 2006
Peggy Peck
In the five months after a tough smoking ban was enacted, hospital admissions for acute myocardial infarction dropped by 11% in the Piedmont region in northern Italy.
