Rapid Responses to:

EDITORIALS: Andrew H Briggs New methods of analysing cost effectiveness BMJ 2007; 335: 622-623 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Evaluating HTA reports by a checklist based on 14 items

Andrea Messori   (16 June 2008)

Evaluating HTA reports by a checklist based on 14 items 16 June 2008
Andrea Messori, Coordinator Laboratory of Pharmacoconomics, c/o Azienda Sanitaria Pistoia, Italy Send response to journal: Re: Evaluating HTA reports by a checklist based on 14 items	To expand the use of Health Technology Assessment by health-care professionals, simplified approaches to handle the various methods of the assessment are increasingly needed. Searching for the keywords "checklist" and "health technology assessment" on PubMed retrieves a total of 11 citations (search completed on 14 June 2008). Among these 11 citations, 9 do not present any original checklist, one mentions the INAHTA checklist as the best reference in this area (1) and, finally, the paper by Hailey (2) is the one where the INAHTA checklist was originally published. One point of controversy is that the INAHTA checklist is not intended as a scorecard to rate the quality of HTA reports. In fact, this checklist is mainly intended to be an aid to improving transparency, consistency and homogeneity of HTA reports; so, this checklist does not intentionally handle the issue of the methodologic quality of the reports. As a result, even after the publication of the INAHTA checklist, the issue of the methodologic quality of a HTA report is left to very technical assessments that in most cases evaluate whether or not the report under examination meets the recommendations or guidelines that have been prepared by agencies and scientific institutions. These guidelines are typically represented by very lengthy and very specialized documents, e.g. see the guidelines issued by the WHO (3) or those issued by the Canadian agency (4). This raises the need to develop a simplified checklist dealing with the basic methodologic points that a “good” HTA report should satisfy. As a very preliminary proposal in this area, we present the translation of a simplified checklist developed a few weeks ago by our Laboratory of Pharmacoeconomics and originally written in Italian (Table 1) . We hope that this Rapid Response can be useful to stimulate some debate on this issue. ____________________________ TABLE 1 - CHECKLIST OF ITEMS TO BE INCLUDED IN A “GOOD” HTA REPORT EVALUATING AN INNOVATIVE TECHNOLOGY A) ANALYSIS OF PUBLISHED STUDIES 1) The published literature is examined with particular reference to data on the effectiveness of the innovative product as well as data on the patient populations who could be treated with the innovative treatment. These data are also examined to identify some criteria that can indicate the “appropriate” use of the innovative product. 2) Guidelines and/or official recommendations concerning the use of the innovative product are examined both in the scientific and the technical literature. 3) The following items are examined to better evaluate the new product in comparative terms: a) identification of the best “comparator”; b) identification of the best end-point to measure the outcome after the new product or the comparator; c) estimation of the extent of the (incremental) improvement shown by this end-point when the innovative product is used in substitution for the comparator. 4) Analysis of the agreement between the indications resulting from the original studies (item 1) and those resulting from the guidelines (item 2). 5) Identification of the situations where the use of the innovative product might be inappropriate. B) ANALYSIS OF LOCAL DATA ON THE USE OF THE INNOVATIVE PRODUCT 6) When available, local (and/or epidemiologic) data on the use of the innovative product are retrospectively collected and then critically reviewed; this retropective review examines the agreement between the pattern of real uses and the uses suggested by clinical trials, meta- analyses and/or guidelines. If no retrospective data are available, the assessors make a decision on whether a new prospective collection of these data can be both feasible and helpful. 7) In this critical review of local data, an attempt is made to classify these "real" uses into appropriate and inappropriate ones. C) SEARCHING FOR INFORMATION ON THE PRICE OF THE INNOVATIVE PRODUCT 8) Information on the price of both the innovative product and its comparator is retrieved using local, national, and international data. 9) A separate analysis is carried out to examine the variability of these prices across different institutions, regions, and countries. D) ESTIMATING THE INCREMENTAL COST AND THE INCREMENTAL BENEFIT FOR THE COMPARISON OF THE INNOVATIVE PRODUCT VS. THE COMPARATOR (“value for money” or not enough “value for money”) 10) A synopsis is prepared that summarises the incremental cost and the incremental benefit in the comparison between the innovative product and the comparator; the approach that employs a reference case is explicitly suggested for this purpose. 11) The assessors examine the possibility to carry out a cost-benefit analysis wherein both costs and benefits are expressed in monetary units; if this analysis is appropriate, an overall economic balance is separately determined for the innovative product and the comparator. 12) The assessors examine the possibility to carry out a cost- effectiveness analysis wherein both costs are expressed in monetary units and benefits are expressed as a gain in life years or QALYs; if this analysis is appropriate, the cost per life year gained (or per QALY gained) is calculated and compared with the threshold currently adopted in that country or region. E) DESIGNING MEASURES TO IMPROVE THE COST/EFFECTIVENESS PROFILE OF THE INNOVATIVE PRODUCT 13) The assessors discuss which interventions (e.g. information, guidelines, prospective data collection, actions directed to prescribers, restrictions of use for the product) can minimise the inappropriate use of the innovative product. 14) The assessors discuss which interventions can improve the cost per QALY gained for the innovative product (es. new price negotiations, methods to run auctions, determination of the maximum cost-effectiveness price recognisable to the product, designing rules and methods for paybacks by the manufacturer). ____________________________ REFERENCES 1) Porzsolt F, Kajnar H, Awa A, Fässler M, Herzberger B. Validity of original studies in health technology assessment reports: significance of standardized assessment and reporting. Int J Technol Assess Health Care. 2005;21(3):410-3. 2) Hailey D.Toward transparency in health technology assessment: a checklist for HTA reports. Int J Technol Assess Health Care. 2003;19(1):1- 7. 3) Sorenson C, Kanavos P, Drummond M. Ensuring value for money in health care: the role of HTA in the European union, WHO Regional Office for Europe, January 2007, http://www.euro.who.int/document/E91271.pdf 4) Guidelines for the economic evaluation of health technologies: Canada [3rd Edition]. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2006. Competing interests: None declared