November 22, 2006 Bloomberg
Bayer Drug Shows Flaw in Studying Safety, Harvard's Avorn Says
Safety concerns about a Bayer AG heart surgery drug illustrate why the U.S. health-care system can't rely on pharmaceutical companies to provide research on risks, a Harvard University professor wrote in a medical journal.
The Bayer drug's links to heart attacks and strokes could have been evaluated through a large, randomized trial comparing the product with competing medicines, Harvard's Jerome Avorn wrote in an article in tomorrow's New England Journal of Medicine, without specifying how many patients would have to be involved. Such a study hasn't been conducted, Avorn wrote.
``It is naive to expect companies to voluntarily fund studies that could sink lucrative products,'' wrote Avorn, a professor of medicine. The Food and Drug Administration ``lacks the regulatory clout to require them'' and the U.S. should make a commitment to support studies of drug risks, he said.
Lawmakers have criticized the FDA for failing to protect consumers from unsafe drugs following Merck & Co.'s withdrawal of the Vioxx painkiller, which was linked to elevated risks of heart attacks and strokes. The FDA says it is trying to improve its ability to detect harmful side effects and wants to use increased fees paid by companies seeking new drug approvals to do so.
Bayer's Trasylol, used to prevent bleeding during heart surgery, has been linked to at least 51 deaths, according to the FDA. About 4.3 million patients received Trasylol from 1984 through last year, the agency said.
Private research -- including a study commissioned by Bayer -- has indicated that Trasylol, also known as aprotinin, may increase the risk of heart attacks, strokes and death. Key data from the studies haven't been published, leaving questions about the significance of the findings, Avorn said in an interview.
Bayer hired researchers to complete a study this year that involved reviewing hospital records of 67,000 patients. Bayer said it erred by failing to disclose the research to the FDA before the agency convened a meeting of outside advisers in September to assess the drug's risks. The FDA learned of the Bayer study after a researcher working for a company contractor alerted the agency.
The FDA called the meeting to examine a report published in the New England Journal of Medicine in January linking the drug to heart attacks and kidney failure. The committee of advisers said the findings weren't enough to add risk warnings.
The studies aren't necessarily conclusive because patients given the drug may have had more serious pre-existing conditions than those given other medications, Avorn said. Doctors haven't been able to see data needed to judge the significance of the research, he said.
The lead researcher on the study published in January, Dennis Mangano, initially said he would release the supporting data to the FDA only with several restrictions. The FDA rejected the conditions. Mangano, chief executive officer of the Ischemia Research and Education Foundation in San Bruno, California, said in a letter to the New England Journal of Medicine that he later dropped the conditions and didn't hear back from the agency.
If the FDA had analyzed that data and confirmed the drug's risks, ``the action of the committee, which concluded that no labeling change was necessary, might have been different,'' Jeffrey M. Drazen, the journal's editor in chief, said in an editorial to be published in tomorrow's edition.
In a separate article in the journal, the chairman of the FDA advisory panel criticized Bayer for failing to provide its study.
``The failure of Bayer to disclose all of its data on aprotinin seriously undermined the advisory committee process and hindered the safety review,'' wrote William R. Hiatt, a professor of medicine at the University of Colorado School of Medicine.
Bayer, based in Leverkusen, Germany, in a statement repeated its position that the company made a mistake by failing to provide its study to the FDA before the advisory panel meeting. Bayer said in October that it suspended two senior employees because they didn't inform supervisors that the study had been completed. The company has since provided the study to the FDA, which is reviewing the data.
``Bayer is committed to sharing all safety information with relevant regulatory agencies,'' the company said in an e-mailed statement. ``The company is determined to take all appropriate steps to assure that something like this does not happen again.''
A company investigation into what happened is continuing, said Fred Fielding, a Washington lawyer hired by Bayer to complete the inquiry, in an interview. By Justin Blum
November 23, 2006 New England Journal of Medicine
Judging the Safety of Aprotinin
To the Editor: I was troubled by the recent, near-unanimous decision of the Food and Drug Administration (FDA) Cardiovascular and Renal Drugs Advisory Committee publicly advocating the safety of aprotinin1 - despite considerable evidence to the contrary,2,3,4 including our recent study2 and the FDA's own initial 1993 warning regarding "kidney toxicity."5 We learned after the Advisory Committee's meeting of newly disclosed records of 67,000 patients, yielding data that were consistent with our warnings (renal failure, stroke, and heart failure)2 and that were inconsistent with the positions of the committee and Bayer.1,6
In defending the committee's decision, Dr. Hiatt makes allegations in this issue of the Journal regarding our "transparency" (FDA access to our data).7 In fact, there is no question that our independent, nonprofit research groups (the Ischemia Research and Education Foundation IREF and the Multicenter Study of Perioperative Ischemia McSPI reseasrch group) made every effort to have the FDA review in detail all of the source and derivative data from our study2 and all associated documentation and operating procedures.
In February 2006, after publication of our article,2 the FDA asked that we present our data to them. We agreed and sent a team of IREF scientists to the FDA. The FDA and Bayer then asked that we send them our database, because they each wanted to combine our data with Bayer's data and then perform a number of unspecified analyses. We agreed to the FDA proposal, making three requests: that patient privacy and legal trust be preserved, that the methods used to mix our patient data with the Bayer patient data be specified, and that in addition to their analyses, the FDA independently analyze our data, comment on each finding of our study published in the Journal,2 and make the results known to the public. In May (4 months before the Advisory Committee meeting), frustrated by the delay and concerned that our data would not be reviewed, I abandoned all three requests and informed the FDA that we would provide our database and any and all source data for its review. Hearing no response, after several weeks, I wrote to the FDA and again asked for a review of our data. However, the FDA refused, and we were told that a review was not necessary. As I publicly stated to the Advisory Committee on September 21, 2006, we stand by our offer to the FDA to provide for review any and all data regarding our publication without constraint or condition.
The FDA and its Advisory Committee should take a conservative, protective stance when independent evidence regarding drug safety presents itself. Instead, they appear to be protecting the drug rather than the patient.
Dennis T. Mangano, Ph.D., M.D.
1. FDA advisory committee strongly endorses use of Trasylol in CABG surgery patients. Medical News Today. September 25, 2006. (Accessed November 7, 2006, at http://www.medicalnewstoday.com/medicalnews.php?newsid=52568.)
2. Mangano DT, Tudor IC, Dietzel C. The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006;354:353-365. Abstract/Full Text
3. Hunter D. First, gather the data. N Engl J Med 2006;354:329-331. Full Text
4. Vlahakes GJ. The value of phase 4 clinical testing. N Engl J Med 2006;354:413-415. Full Text
5. Approval of aprotinin. Press release of the Food and Drug Administration, December 30, 1993. (Accessed November 7, 2006, at http://www.fda.gov/bbs/topics/NEWS/NEW00453.html.)
6. Harris G. FDA says Bayer failed to reveal drug risk study. New York Times. September 29, 2006.
7. Hiatt WR. Observational studies of drug safety -- aprotinin and the absence of transparency. N Engl J Med 2006;355:2171-2173. Full Text
New England Journal of Medicine November 23, 2006
Observational Studies of Drug Safety - Aprotinin and the Absence of Transparency
William R. Hiatt, M.D.
The full safety profile of a new drug is rarely known at the time of approval by the Food and Drug Administration (FDA). Most drug-development programs designed for treatments of symptomatic indications are underpowered to detect any increased risk of rare drug reactions or change in background event rates attributable to the drug. Large, post-marketing, randomized, controlled trials provide robust data on drug safety but may be subject to multiple sources of bias. Observational studies of a drug's effects in clinical practice can offer additional information on risks. The recent discussions of aprotinin (Trasylol, Bayer) by the Cardiovascular and Renal Drugs Advisory Committee of the FDA, which I chair, provide insight into the strengths and weaknesses of using observational data to assess drug safety and highlight the importance of using a transparent and open process when reviewing such data.
Post-marketing observational studies permit the evaluation of drug safety in a large number of patients in a real-world setting, where practice patterns, including off-label use, can be assessed. Such studies have limited ability to determine causation, but they can detect signals that may suggest a safety concern.
In an observational study, decisions to use specific drugs are made by local physicians, according to their perceptions of the risks and benefits for particular patients. Such treatment allocation results in an imbalance in demographic and risk factors between patients given the drug of interest and those given an alternative or no treatment. An imbalance in various clinical factors will directly influence safety outcomes when patients at higher risk receive the drug. Therefore, appropriate statistical methods must be used to adjust for the nonrandom assignment to treatment groups.1
One way to address imbalances between groups is to use a propensity score that incorporates confounders and other covariates into a model predicting the probability of assignment to a particular treatment. This score can be used for adjustment or for matching patients who have similar probabilities of receiving a treatment. Differences in outcomes between treated and untreated patients (or patients treated with a comparison drug) with equal propensity scores provide a less biased estimate of treatment effect.
Aprotinin was approved by the FDA in 1993 as a means of reducing perioperative blood loss and the need for blood transfusion in patients undergoing coronary-artery bypass grafting. Neither the clinical trial database that led to approval nor the numerous randomized, controlled trials conducted after approval identified an association between aprotinin and any short-term increase in the risk of death or nonfatal cardiovascular events or any serious renal toxic effects (except for a transient increase in the serum creatinine concentration). However, in early 2006, two observational studies were published that raised serious concerns about the drug's safety.2,3
One study, by Mangano et al.,2 evaluated 4374 patients undergoing coronary-artery bypass surgery. End points were prospectively defined, and data on a large number of clinical variables were collected for each patient. The decision to use aprotinin, aminocaproic acid, or tranexamic acid to inhibit fibrinolysis or to withhold antifibrinolytic therapy was made by the treating physician. Since group assignment was not random, it is not surprising that patients assigned to the group receiving aprotinin were at much higher risk for adverse cardiovascular outcomes than were the other patients in the study, who were treated with alternative therapies. When a propensity score was used in a logistic-regression model to adjust for these baseline differences, aprotinin was associated with a nonsignificant increase in the risk of death (odds ratio, 1.59; 95% confidence interval CI, 0.76 to 3.34) and with significant increases in the risks of renal events (odds ratio, 2.34; 95% CI, 1.27 to 4.31), cardiovascular events (odds ratio, 1.42; 95% CI, 1.09 to 1.86), and cerebrovascular events (odds ratio, 2.15; 95% CI, 1.14 to 4.06). The authors concluded that "the observed association between aprotinin and serious end-organ damage indicates that continued use is not prudent."
The second study, by Karkouti et al.,3 compared the risks associated with aprotinin with those associated with tranexamic acid in 898 patients undergoing high-risk cardiac surgery. The authors used a propensity score to match patients who had been given the different treatments - an approach that permitted the identification of a population of patients who were subsequently fully matched according to a large number of covariates. Calculations from the study data, performed by the Colorado Prevention Center, indicated no significant difference between the two treatments in the overall risk of myocardial infarction (odds ratio, 1.20; 95% CI, 0.52 to 2.75), stroke (odds ratio, 1.15; 95% CI, 0.56 to 2.40), or death (odds ratio, 0.91; 95% CI, 0.56 to 1.46). However, among patients who had abnormal renal function at baseline, those given aprotinin had a significantly increased risk of postoperative renal dysfunction (odds ratio, 1.69; 95% CI, 1.07 to 2.69). Because this study included fewer patients than the study by Mangano et al., it had less power to detect a safety problem.
These observational studies did not permit a definitive conclusion about cardiovascular or renal risk, but they did raise concern. The FDA appropriately issued a public health advisory about the potential risks posed by aprotinin and urged physicians to monitor their patients carefully for renal, cardiac, and cerebral toxic effects.4 The agency also convened a meeting of the Cardiovascular and Renal Drugs Advisory Committee on September 21, 2006, to evaluate the evidence on the cardiovascular and renal toxic effects of aprotinin. Our committee reviewed the published studies and the global safety and efficacy database submitted by Bayer, which included 45 randomized, controlled trials involving 2249 patients who received aprotinin and 2164 who received placebo.
The Bayer safety analysis was confined to the unadjudicated adverse events reported within 7 days after the administration of aprotinin, which included 520 deaths or serious cardiovascular or renal events. The results did not reveal any increased risk of fatal or nonfatal cardiovascular events. According to the Colorado Prevention Center's calculations based on Bayer's data, the odds ratio for death was 1.14 (95% CI, 0.80 to 1.62); for myocardial infarction, 1.17 (95% CI, 0.93 to 1.49); for stroke, 0.71 (95% CI, 0.42 to 1.18); and for renal failure, 1.15 (95% CI, 0.74 to 1.78). Two studies in which myocardial infarctions were adjudicated by an independent review committee showed a nonsignificant increase in the risk of myocardial infarction, with a point estimate of the odds ratio as high as 2.24 (95% CI, 0.56 to 9.00).
How can the results of these trials and the study by Karkouti et al. appear to be so different from the findings of Mangano et al.? Part of the problem may derive from the inability of observational studies to identify and measure all relevant covariates that may influence the outcome; similar studies using a few different covariates can sometimes come to opposite conclusions.
FDA advisory committee meetings usually include an independent review of data - either those provided by the sponsor or those used by the authors of relevant published studies. This independent and transparent process becomes even more critical in the evaluation of observational studies, whose conclusions can be highly influenced by the statistical methods used and the inability to identify all confounding factors. Unfortunately, Mangano did not give the FDA or the committee full access to his data, which would have allowed the agency to perform an independent review and analysis to validate his group's findings. Although there are many legitimate concerns with regard to the sharing of data (for instance, confidentiality and the need for informed consent), the lack of independent review greatly limited the committee's ability to draw conclusions from the study. Having reviewed all the data available, the committee decided that there was insufficient evidence to require an additional warning on aprotinin's labeling and agreed that the clinical data supported an acceptable safety and efficacy profile for aprotinin.
Days after the committee meeting, the FDA was made aware of additional observational data from the sponsor that had not been presented at the meeting. Bayer evidently had commissioned an observational study involving 67,000 patients who were given aprotinin.5 According to the initial FDA review of data from that study, aprotinin may be associated with "increased risk for death, kidney failure, congestive heart failure and stroke." The failure of Bayer to disclose all its data on aprotinin seriously undermined the advisory committee process and hindered the safety review.
Although observational studies cannot be definitive (and so should evoke measured responses), they can provide important new safety information that can direct further scientific and regulatory actions - if their findings can be confirmed. Since further analysis of the new data provided by Bayer is ongoing at the FDA, conclusions about the overall safety of aprotinin cannot be drawn at this time. Still, the aprotinin story demonstrates that full disclosure and a transparent process are essential in evaluating the findings of all studies germane to drug safety and the public health.
Dr. Hiatt is a professor of medicine at the University of Colorado School of Medicine and president of the Colorado Prevention Center - both in Denver - and is the current chair of the Cardiovascular and Renal Drugs Advisory Committee of the FDA. Opinions expressed in this article are those of the author and do not necessarily reflect the opinions of the FDA or the advisory committee.
1. Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757-763. Abstract/Full Text
2. Mangano DT, Tudor IC, Dietzel C. The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006;354:353-365. Abstract/Full Text
3. Karkouti K, Beattie WS, Dattilo KM, et al. A propensity score case-control comparison of aprotinin and tranexamic acid in high-transfusion-risk cardiac surgery. Transfusion 2006;46:327-338. CrossRefISIMedline
4. Food and Drug Administration. Public health advisory: aprotinin injection (marketed as Trasylol). Updated September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/cder/drug/advisory/aprotinin.htm.)
5. Idem. FDA statement regarding new Trasylol data. September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/bbs/topics/NEWS/2006/NEW01472.html.)