Commentary

My father once told me when I was a child: “if you believe all what you read, do not read!” Later on, I realized that this’s true, especially in the era of evidence based medicine. Recently, data fabrication has emerged as a nail in a coffin for clinical trials [1]. There is a natural tendency to assess the results of randomized clinical trials as either positive or negative according to whether the P value for the primary outcome measure is <0.05 or >0.05. However, such an interpretation is overly simplistic. The primary endpoint result is just a starting point in the comprehensive evaluation of the totality of the clinical evidence. If the trial endpoint came positive, some questions should be asked. First, what the P value exactly is, a P value of < 0.001 means strong evidence without any reasonable doubt, in contrast to a P value of just 0.04. For example, the PARADIGM–HF trial [2] the P value for endpoint was 0.001. This prompted the regulatory authorities to approve neprilysin inhibitor for use in heart failure with reduced ejection fraction. Second, what’s the absolute risk reduction (ARR) and number needed to treat (NNT)? For example in FOURIER trial [3] of a new PCSK-9 inhibitor to manage dyslipidemia, there was just 1.5% ARR corresponding to a NNT of about 70, making many editorialists to question the cost-effectiveness of the drug (costs about 14.000 USD/ year). Third, is the endpoint a clinical one or a surrogate one? The classic example here is the ACCORD trial [4] although the surrogate endpoint of HBA1C reduction reached statistical significance, the clinical endpoints showed no significant change and in fact mortality tended to increase in intensive blood glucose control arm! Fourth, what about data of subgroups? Clinical research has always taught us to be extremely skeptical when interpreting data of subgroups but we could learn much from them. Look here at the PLATO trial [5] in which the benefit of Ticagrelor was not so robust in the USA subgroup, a finding that could be explained by the high dose of Aspirin used by USA population. This lead to FDA recommendations to limit Aspirin dose to 100 mg when taken with Ticagrelor. Fifth, what is the sample size? A small trial lacks power, and positive treatment effects are susceptible for exaggeration. I remember here a small trial published in New England Journal testing effect of N-acetyl cysteine (NAC) on reducing Contrast-induced nephropathy (CIN) [6] this trial showed a 90% relative risk reduction of CIN with NAC. Almost nothing in world has such a massive treatment effect. A subsequent meta-analysis including much larger number of patients confirmed the neutral effect of NAC on reduction of CIN [7]. Now, we came to the other side of the coin, the trial endpoint is negative, what can be done? How can we salvage the trial? And what went wrong? Some important questions should also be asked. First, was the population you are testing appropriate? I refer here to the story of Ivabradine. In BEAUTIFUL [8] and SIGNIFY trials [9], done in population with ischemic heart disease it provided a little bit disappointing results. It was only when they shifted to patients with heart failure in the SHIFT trial [10] that the results came positive making the way for the drun into guidelines. Second, could we claim for non-inferiority? A good example here is the VALIANT trial [11] in which the confidence interval excluded a noninferiority margin of 1.13, allowing authors to claim for noninferiority of Valsartan compared to Captopril. Third, can alternative analysis method help? A good illustration here could be the difference between intention-to-treat (ITT) and as treated analyses. Please, look with me at STICH trial [12] that showed no difference between bypass surgery and medical therapy for patients with ischemic LV dysfunction when analyzed by ITT, but when data were analyzed by as-treated analytical method, results came back significantly favoring bypass surgery in those patients. Forth, are the endpoints accurately defined? A typical example here could be the CHAMPION PLATFORM trial [13] that showed odd ratio for Cangrelor versus Clopidogrel in the right direction but with a P value of 0.17. The authors realized that they had a major problem with the definition of Myocardial Infarction especially periprocedural Myocardial Infarctions. Therefore, CHAMPIONPHOENIX trial [14] asked essentially the same question but with clearer adjudication of Myocardial Infarctions and the results came significant with a P value of 0.005. These were some highlights on how to look at data of clinical trials. The space in this short commentary is not wide enough to get all points surrounding this issue. Have a nice time; this is Ahmed Bendary signing off and good bye.

References

1. Carlisle J (2017) Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia.
2. McMurray J, Packer M, Desai A, Gong J, Lefkowitz M, et al. (2014) Angiotensin–neprilysin inhibition versus enalapril in heart failure. N Engl J Med 371: 993-1004.
3. Sabatine M, Giugliano R, Keech A, Honarpour N, Wiviott S, et al. (2017) Evolocumab and clinical outcomes in patients with cardiovascular disease. N Engl J Med 376: 1713-1722.
4. Gerstein HC, Miller ME, Byington RP, Goff DC Jr, Bigger JT, et al. (2008) Action to Control Cardiovascular Risk in Diabetes Study Group: Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med 358: 2545-2559.
5. Wallentin L, Becker R, Budaj A, Cannon C, Emanuelsson H, et al. (2009) Ticagrelor versus clopidogrel in patients with acute coronary syndromes. N Engl J Med 361: 1045-1057.
6. Tepel M, Van Der Giet M, Schwarzfeld C, Laufer U, Liermann D, et al. (2000) Prevention of radiographic-contrast-agent–induced reductions in renal function by acetylcysteine. N Engl J Med 343: 180-184.
7. Sun Z, Foe Q, Cao L, Cheng L, Li Z (2013) Intravenous N-acetyl cysteine for prevention of contrast-induced nephropathy: a meta-analysis of randomized controlled trials. PLOS ONE 8: e55124.
8. Fox K, Ford I, Steg P, Tendera M, Robertson M, et al. (2008) Heart rate as a prognostic risk factor in patients with coronary artery disease and left-ventricular systolic dysfunction (BEAUTIFUL): a subgroup analysis of a randomised controlled trial. The Lancet 372: 817-821.
9. Fox K, Ford I, Steg P, Tardif J, Tendera M, et al. (2014) Ivabradine in stable coronary artery disease without clinical heart failure. N Engl J Med 371: 1091-1099.
10. Swedberg K, Komajda M, Böhm M, Borer J, Ford I, et al. (2010) Ivabradine and outcomes in chronic heart failure (SHIFT): a randomised placebo-controlled study. The Lancet 376: 875-885.
11. Pfeffer M, McMurray J, Velazquez E, Rouleau J, Køber L, et al. (2003) Valsartan, captopril, or both in myocardial infarction complicated by heart failure, left ventricular dysfunction, or both. N Engl J Med 349: 1893-1906.
12. Velazquez E, Lee K, Deja M, Jain A, Sopko G, et al. (2001) Coronary-artery bypass surgery in patients with left ventricular dysfunction. N Engl J Med 364: 1607-1616.
13. Bhatt D, Lincoff A, Gibson C, Stone G, McNulty S, et al. (2009) Intravenous platelet blockade with cangrelor during PCI. N Engl J Med 361: 2330-2341.
14. Bhatt D, Stone G, Mahaffey K, Gibson C, Steg P, et al. (2013) Effect of platelet inhibition with cangrelor during PCI on ischemic events. N Engl J Med 368: 1303-1013.