Dear Editor,

We thank Pande and Soumerai for their comments. However, we believe that they present an overly simplistic critique of the study's findings and methodology that is not grounded in the reality of public policy evaluations. First, and most importantly, we did not conclude, as they suggest, that the intervention was ineffective. We were similarly disappointed by reports from some press outlets that suggested otherwise.

The 15% of patrons who reported seeing and using calorie information purchased fewer calories. In 2009, almost 1.3 million New Yorkers reported eating fast food at least once a week , resulting in approximately 3 million visits weekly (1). This means that very substantial numbers of New Yorkers will use this information to purchase fewer calories each week. Furthermore, there were statistically significant declines in the total number of calories purchased in several major chains within the overall restaurant sample.

Since mandatory calorie labeling by restaurant chains has been adopted nationwide in the U.S. and will go into effect next year, and with this expansion the incentive to industry to change its practices grows, we concluded that further rounds of assessment are needed to measure the full impact. It should be noted that the calorie labeling regulation also serves the simple purpose of assuring a consumer's right to know. In that regard, calorie posting was considered useful by 84% of New Yorkers(2). Nevertheless, that information alone will not be sufficient to reverse the obesity epidemic is not a surprise. Rather, it is a challenge for future action.

Pande and Soumerai's initial point suggests that evaluation of calorie posting in New York City could be meaningfully pursued only with Cochrane-quality randomized trials. While this may represent a scientific ideal, the restaurant industry, which aggressively opposed the policy through the courts, would simply not have cooperated with randomization. Randomized trials have not been feasible for key public health policies such as those that required seatbelts, raised tobacco taxes or removed lead from paint. While using other jurisdictions as controls is a reasonable alternative suggestion, it was not feasible within the narrow timeframe of a few months between approval and implementation. New York City rapidly designed and funded this study to ensure a large and representative sample of affected fast food chains and their customers. We agree with Pande and Soumerai that the study has important limitations, which were clearly discussed in the publication. However, the presence of these limitations does not obviate our findings and, as with all publications, serves to inform the reader as to how the results should and should not be interpreted.

Unfortunately, no other jurisdiction, subsequent to the implementation of this measure, has been able to carry out even a similarly scaled evaluation. This reflects the typical difficulties confronting policy evaluation, where timely studies are often the only option, and while imperfect, still furnish critically useful information. We welcome other studies with different designs that complement our work, particularly if baseline measures of sufficiently representative samples can be taken before a calorie labeling policy goes into effect elsewhere.

In regard to Pande and Soumerai's concern about contamination of the baseline data, the information previously provided by Subway at baseline was restricted to a small sticker on the display case and provided calorie values for only a few of their products. Thus it was not equivalent to earlier implementation of comprehensive calorie posting. More importantly, the evaluation of a public policy, which was ultimately the goal of the study, is to measure what changes occur in society with its implementation. If one company voluntarily provided some information at baseline, that was, in fact, the societal condition prior to the policy and hence does not invalidate our findings.

It seems obvious, but perhaps merits note, that the food industry is well positioned to assess the impact of calorie labeling upon its sales through its cash register data. However, with the exception of the Starbucks chain, which supported a large study whose results showed a 6% decline in calories purchased without loss of revenue at their establishments,(3) no other company has chosen to share its sales data.

We welcome discussion and debate on how to assure the evaluation of public policy innovations, and believe there is an urgent need for funding pools designed to support the assessment of innovative public health policies within typical fast moving timeframes. As nations, we need a bigger commitment to the evaluation of large scale public health measures with the potential to change population health status.

Lynn Silver MD, MPH, FAAP

Mary T. Bassett, MD, MPH

(1) New York City Community Health 2009, Unpublished data.

(2) Quinnipiac University Polling Institute. New Yorkers big on sugar drink tax, calorie counts 2011. www.quinnipiac.edu/x1302.xml?ReleaseID=1425

(3) Bollinger B, Leslie P, Sorenson A. calorie posting in chain restaurants. The National Bureau of Economic Research, Working paper no. 15648. 2010. Nber.org/papers/w15648