Examples from clinical trials – S. Wassmann



welcome back to the next to the next part so we now have something more interactive how to interpret technical trials and I have two panelists dr. Sava is an dr. Lewis so this is really about showing trials and discussing principles of how to interpret the results of child's okay I would like to start with this so three questions you have to ask yourself if you look at a clinical child why was it done who was studied and how was it done so three simple questions so if we start like this the why is the hypothesis of the child so we picked the improve-it trial and that is there was a big study in acs patients investigating the value of ezetimibe in lowering cholesterol and events in patients post acs so three questions two of them I would like you to remember number one does lowering LDL cholesterol with a non statin agent ezetimibe reduce events so question number one keep that in mind and number two is even lower even better which means bringing down cholesterol lower than the recommended target value of seventy does that reduce more events so is even lower even better two questions keep that in mind okay who has studied inclusion criteria so we see patients with acs and patients with lowish LDL cholesterol so patients had had to have LDL cholesterol let's say around 100 to be included how was it done that's the study design so that's very important when you look at interpreting a trial so how was it done so we had patients post SES ten days and you can see LDL cholesterol between 50 and 125 this was inclusion criteria and then 18,000 patients were randomized to receive either simvastatin 40 milligrams or simvastatin plus ezetimibe so that was the treatment arm and the goal was to achieve certain LDL cholesterol levels so there were follow-up visits and the duration was not set by year it was an event driven trial the plan was three years approximately took seven okay because the the event rate was lower than expected then there was a primary combined endpoint this is very important to keep in mind what the primary endpoint is and you see it's the CV death the my and then hospital mission for unstable angina coronary revascularization or stroke so if you like that's a maze end point plus hospitalization and revascularisation so a combined endpoint baseline characteristics always look at that you see typical characteristics here one fourth females one third diabetics you can see here different types of acute coronary syndromes so that's quite classical I would say and that is something important here that's the the mean LDL cholesterol at enrollment this was ninety five milligrams per deciliter so kind of low alright so this is what was achieved LDL cholesterol changes and if you look at the medium time average LDL it's approximately seventy in the similar arm and approximately fifty three in the ezetimibe arms so seventy was reaching the goal that was set by guidelines and the other one was approximately seventeen milligrams per deciliter lower this is something to be kept in mind when we interpret the trial as well so and now this is the primary endpoint in the intention-to-treat analysis again this combined endpoint and you see the kaplan-meier curves here and when you look at this there was a relative risk reduction of six percent over seven years statistically significant number needed to treat of fifty over the course of seven years so these are the results now what I would like you to tell me is what do you think does these results mean okay so remember the questions the hypotheses of the trial so have we achieved these hypotheses yes yes which one sorry now just to say it again okay so you say some somewhat achieved okay exactly so can we so question number one is even lower even better and the question number two can we achieve this with a non statin agent with the Zetas might well what is it the answer is yes right so you have it's of course on top of statin treatment but we had similar statin in both arms and on top there was ezetimibe we do have a significant reduction of events so yes so as etem i can reduce cardiovascular events question number one is even lower even better we have 70 milligrams per deciliter target and we have 53 so that is lower and it's significant right so if you look at it that way if you interpret it you can say yes the trial has achieved what it wanted so the research research questions were answered so the next thing is of course how do you interpret that in the clinical setting so you know do we think that in the clinical setting when we when we make treatment decisions does everybody need is that tamiya yes no this is a completely different question and I don't want to go into this I would like to show this this was very nicely done I think when you look at this regression curves showing or investigating the reduction in LDL cholesterol in the reduction in event rates that we have in the different lipid trials you can see here it's perfectly on this line so for the 17 milligrams per deciliter LDL cholesterol reduction there was an the number of events reduced were perfectly on this line that were thought that we could achieve okay so we have just a low reduction in LDL cholesterol but this translates perfectly into reduction in events and it perfectly to what we know from other charts okay should we give this drug to certain patients that's a different question I don't want to ask that question now okay all right so now we have primary and secondary endpoints so again the primary endpoint here is a combined endpoint statistically significant then there are other secondary endpoints also combined endpoints my point here is if you look at them somebody might go and say hey great secondary endpoints all positive so my questions are these secondary endpoints valid can we say those are significantly reduced what do you think yes okay what is what is what is the important fact it could be let's say primary endpoint non significant secondary endpoints significant we do have that sometimes right so let's say we have a combined endpoint not significant May's endpoint secondary endpoint significant it is consistent completely agree so the primary endpoints the powers made for the primary and the study was made for the for the primary endpoint is significant then we allowed to test further for secondary endpoints they are significantly reduced so that's valid okay but keep in mind when you have when this when the primary endpoint has not been reached well it's not sticked is statistically significantly different then it doesn't really matter what is beyond that that's how pathes is generating hmm okay all right so levels of endpoints are like this one the best endpoint is this that is the hardest and per we have to count dead bodies you know heart failure trials we can count dead bodies so all cause mortality then it's cause specific mortality for example fatal myocardial infarction cardiovascular death and then there's non-fatal clinical events non fatal mi non-fatal stroke hospitalization these are quite important to also for health economics obviously and then the surrogates LDL cholesterol hba1c whatever you want to take and we always got to be careful with that then there's quality of life I'm not sure if it should be that low I think quality of life is quite an important thing I think it should be higher but this is important this is just a surrogate and we know that not all drugs that lower LDL cholesterol actually lower events right let's take niacin for example and it's done the continua cassock and stuff like this okay when we look at the endpoints of the improvement trial all cause mortality not different cardiovascular death not significant but then we have two here mi stroke ischemic stroke when you look at this so here's a significant reduction oops sorry so are these important individual endpoints I think they are that's non-fatal non-fatal events okay subgroup analyses that's the rule of the four piece I like that one so the subgroup analysis needs to be pre specified adequately powered plausible and practically relevant you can test anything in subgroup analysis but does it make sense maybe not I think you heard something about that before okay what about improve it just look at this these were pre specified subgroups of the trial sex age diabetic status prior lipid-lowering therapy yes/no and the baseline LDL cholesterol so all of these I think quite they fulfill the 4p rule and now look at this you see the point estimates here and the confidence intervals and here is one with an asterisks which has a P for interaction of 0.02 3 so when you look at this or maybe I asked my question first so what you think of this subgroup analysis so what does it mean we have here we have this for the diabetics if we go back this looks not so bad either what you think of this one here you know that people might come maybe pharmaceutical industry might come and show you this and say hey we have special benefit in this group you know with prior lipid-lowering therapy what do you have to look for I put it here look for the p4 interaction okay so this has a significant p4 interaction so this is a subgroup where there is some difference the others no there is no difference what does it mean the results of the overall population can be can be attributed to the subgroup okay this is what it means there might be a difference here in the diabetics but so check out the p4 interaction that's important okay so now what they did they did a subgroup analysis in diabetic and non-diabetic cohort so when you look at this these are the diabetics they have higher risk you can read this with a couple of my risk estimate here and you see a difference relative risk reduction of 14% in this group and then you have the non diabetics where there's no difference so now again what does that mean oh should we only treat diabetics now nope why not very good I agree why yeah perfect perfect absolutely absolutely I agree okay the trial was done in the overall cohort right then we have results for this so we should we should treat all of them completely right but it does show that number one diabetics are at higher risk and maybe they benefit even more but it's completely right if you want to know you need to do a new trial and test this hypothesis so hypothesis-generating excellent good very good so we switch to another topic this is a anticoagulation for atrial fibrillation so this is the rocket child which tested rivaroxaban versus warfarin and patients with atrial fibrillation and increased risk this was the statistical methodology so you see something about sample size primary efficacy evaluation the Primus efficacy parameter was stroke and non-systemic embolism and you see here non-inferiority on treatment superiority on treatment testing and then intention-to-treat okay so these are the results kaplan-meier curves separate quite early non-inferiority met quite clearly and now when we look at the next this is now testing for superiority now you have the on treatment groups i do that so now you have the on treatment cohort and you see here significant p-value and you have the intention to treat not significant so you can imagine that that makes quite a difference especially for the pharmaceutical company that that sells the drug so what should we do intention-to-treat or on treatment analysis for testing superiority intention-to-treat on treatment intention-to-treat okay so I think Alexander talked about that okay so its intention to treat so intention to treat is this gold standard to test for superiority obviously the company like this here very much okay and you know that there were some difficulties many many events occurred after active treatment in the and the follow-up period after that but that is important so just to show you this that's the full study period that is covered in the intention-to-treat analysis versus the on treatment period which is always active treatment plus some extra time which is defined in the study protocol so if you have an event a stroke outside the on treatment period it's not counted in the on treatment analysis but it is counted in the intention-to-treat analysis so it gives you a bad idea that's like how can i say that the the analysis that asking that that is always assuming the worst case and this is why we're using intention-to-treat for this and you can see in the on treatment analysis we count this sorry in the entrepreneur's we do not count this event in the intention to retreat we do count this event and that does make a big difference so the the basic rule is if you test for non-inferiority not extended told you about this it's on treatment analysis when we check for superiority its intention to treat and safety is always the on treatment analysis I think he touched that as well okay these are the standards okay now the Aristotle trial I have a different point to make here this is also anticoagulation same thing testing in this case a pics of man versus warfarin in patients with atrial fib and these patients received five milligrams VI D apixaban or warfarin and there was a reduced dose like in all of these trials is which is a 2.5 and as you may remember the 2.5 milligram dose was only given to patients with certain characteristics for example impaired kidney function or the 2 or 3 rule which involved age body weight and kidney function so pharmacological criteria okay so statistics very clearly primary outcome was the same stroke systemic embolism non-inferiority after that tested for superiority bleeding okay for Sioux priority and then it was all cause mortality that was tested these are the findings so non-inferiority met superiority met so it takes about was superior to offerin in preventing strokes and systemic embolism these are the safety events so now I'm just going back to this now you've seen the results so a number of I think like Russell 5 6 percent of patients had the 2.5 dose so now my question for you is when you when we have the trial results here what do we know about this dose of 2.5 milligrams the reason I put this here is how can we interpret the results of what the patients treated with two point five milligrams the reason for this is that we see many patients treated with the lower dose who are not actually adequate for this dose okay this is because doctors don't want to have bleeding's and they go down to the lower dose so what do we know do we know anything about the two point five milligrams I'll just show you this subgroup analysis okay result for all page this is stroke this is major bleeding this is the all patient and this is the 2.5 milligram dose and this is the 5 milligram dose p4 interaction not significant which means both groups do identical right so can we treat all patients with 2.5 now no why not yeah of course exactly and that's very very important that's the point I would like to make so there were very clear criteria when to use the reduced dose that is Mir pharmacology so it's it's according to kidney function age and stuff which of course translate into translates into different plasma levels okay so basically in patients that are older have less weight and less kidney function plasma levels will be higher so it's a pharmacological definition and only in those patients we can say that the 2.5 milligram dose is as good as the 5 milligram dose in the other patients okay but if you treat a patient without these criteria with two point five milligrams you might have under dosing and you cannot say if you have these results and that's important and physicians don't understand this and they always not always but many I think it's like what 30 40 percent that I used where the lower dose is used although it should not be and we know that this is it can be hazardous and increased stroke rate yes but you had a question triple therapy oh that's a completely different story that's we do not have data on this yet we will have with your gustas trial I showed you the Augustus trial yesterday so the Augustus trial is exactly testing this it but it's only investigating the five milligram dose but then five milligrams of a pixel band versus warfarin and then another arm testing triple versus dual therapy I showed you that yesterday okay we have no data on two point five and I'm not personally I'm not very happy with two point five and triple therapy because it's half dose you know it's different with with the bigger trend and and the rivaroxaban okay but the point I would like to make is you need to be sure that dosing is according to to the to the criteria that we have and that's true for all of these drugs okay now this is the engaged trial this is it duck Seban in atrial fibrillation and it's again a big trial again patients with AF at increased risk and here we had three treatment arms it was warfarin it was high-dollar duck 760 QD and low dose of duck 730 QD and then again there were dose reduction criteria also in this trial primary efficacy endpoint stroke and systemic embolism so why am i showing you this because they used a different methodology so they said primary primary efficacy for non-authority was on treatment and they used something modified intention-to-treat okay in superiority testing was intention-to-treat analysis safety again on treatment so what is the mod hopes what is modified intention-to-treat does anybody know what that is I'm not sure of Alex on that touch that okay what is intention to treat the patient is randomized right and he's in the trial regardless of whether or not he takes it the study drug right so what is modified I think it's quite a smart idea we had this example of the guy who was randomized and it's run over by a bus right you remember that without having ever taken a drug and he's counted as an event okay this person who's randomized run over by a bus that's an SAE that's an event so okay so what does it mean modified intention-to-treat are all patients randomized who took at least one dose of study medication that's what they do just to show you it did not make much of a difference in this trial so when you look at this we have let's say in this group we have 7,000 34 patients and in the modified intention to treat 7,000 to so thirty patients were actually randomized and did not take a pill so they were not included in the modified intention-to-treat okay so why should we use this and I asked this to Vasil that's a good question I cannot answer this we thought that they wanted to be even more correct yeah okay I'm not sure if that will be the standard for other trials I think it's still intention-to-treat analysis which is the standard for superiority testing I would like to show you that again in this trial it did make a difference sorry yeah so just to show you here what they did was they used it they did the on treatment analysis this is for the primary endpoint and look at this it says p-value for superiority and significance that's in the on treatment analysis and when you look in the intention-to-treat analysis superiority was was not met okay so again it made a difference for interpreting the results okay so now this that it's not inferior that's what you have to conclude these are the statistics so honestly I do not know why they chose the modified intention-to-treat and put this in this analysis because in the statistical plan they said in the end superiority testing is made by intention-to-treat so honestly I can't I can't tell you why they did it but but the interpretation is see it's not inferior so that's what they said okay so right after that that was this meat analysis so with the results of the engaged trial that was this meter analysis published in the lancet Noack treatment versus warfarin treatment in patients with atrial fibrillation you see it here this is for stroke systemic embolism this is for major bleeding's this is for secondary endpoints so different drugs different doses different patients maybe so what's the value is that is it adequate does it add value is it correct what do you think look at this looks quite good significant sorry so why is it why is it valid to do this why is it not it's completely right you have to check the the heterogeneity and you can see here it is quite high but not significant that's for the primary efficacy end point when you look at bleeding look at this okay it looks completely different why is that what do you think now you talked about this okay all right because of definitions of bleeding right so that's why it's it has more heterogeneity okay but why can we use this why is it let's say technically okay to do this meter analysis I know I know Gian Luigi's answer to this what is it testing it's not testing drugs versus drug right it's testing a principle it's testing yeah yeah and it's it's testing the principle of Nowak versus warfarin treatment and I personally think it's a it's a good analysis I think it adds it does add some value but you always have to check out when you see this you have to check out the heterogeneity I think that's quite important yeah apostle basically to 80 bits of the big agenda started occupation so you can't really compare the biggest fan and pizza pan as right yeah a new principle has been tested yes absolutely exactly exactly okay great so I have something different now now this is on lipid treatment ever local map it's the Oscar program so you might be around that so the Oscar program was a very heterogenous trial program how as I can say so ever lock him up was was tested in all kinds of different patients populations and phase 2 and phase 3 studies now what they did was the following they they had all these phase 2 and phase 3 studies and those patients who were on a vilaça map and were free of side effects and took that pill were asked to take part in the Oslo program okay so in the end four thousand four hundred sixty-five patients of the previous phase to phase 3 studies were enrolled in the Oslo program and were then randomized so that was like a new study cord and they were randomized to receive a local map or standard of care okay and then there were like Ostler one Ostler to which different courts because they took their phase two trial patients and the other hand their phase three trial patients and then the data from both trials if you wish were combined in the analysis okay so now here's my question do you think that's okay what do you think about this study design is that adequate what do you think it's important isn't it I mean that was that was very very major information coming out further for the new drugs so it did make a difference so is it okay of course of course of course okay number one I have to say Ostler was a safety study okay it's not an outcome study it's a safety study but you're completely right of course it's selection by it's because you're only taking the patient who actually were on drug and tolerated the drug I personally think it's a very very smart idea to have to have very low dropout rates potentially in the trial that's coming so I think it's correct and from statistical point it's completely correct but you're right it has selected certain patient so but they wanted to do this because this was the intention of the trial Ostler was testing primarily safety secondarily LDL lowering and then they had an exploratory outcome a pre-specified outcome on events okay so primary safety secondary LDL and then an exploratory outcome which was cardiovascular events so just very briefly this was testing the the safety a little signal here was neurocognitive events secondary endpoint LDL cholesterol reduction about 60% relative reduction in these groups and then look at this here we have the cardiovascular outcomes and you see a wonderful separation of the curves 50 percent relative risk reduction so what is this again think of the pharmaceutical industry perspective you go to see doctors and say hey look at this look at the outcomes isn't that great so can you do it of course you can do it but what is the importation of this in this context sorry it's effective okay so can you go and say of a local ma produces cardiovascular events after this analysis that's it that's it absolutely that's the point to make this was an exploratory outcome and not the primary end point of the study so this is again hypothesis-generating so this is how we have to look at it it's very encouraging obviously but it's hypothesis-generating and that's obviously what was done it was tested in a major outcome trial and you know that's the fourier trial so for ei 27,000 patients randomized to have a look him up and placebo and there was a significant benefit for the overlock come up arms but that's exactly what needs to be done so it's hypothesis-generating tested in an outcome trial and that's what they did this one yeah there you go yes yeah yeah right right yeah but it's it is very important this is why we need randomized control trials it's exactly this okay you have different magnitude of effect in some cases you have neutral sometimes you even have a negative effect so that's why we actually need randomized control trials in the specific patient population powered for a certain primary endpoint that I couldn't agree more so that's a wonderful example okay so my last issue here so this has come something completely different now this is about a medical device respect was a trial that tested the efficacy of a PF o occluder so patients with a cryptogenic stroke on we would say embolic stroke of unknown source who had a persistent foramen ovale in the te study were enrolled here okay because you know that in these patients clots may cross through the P fo from the right side to the left side of the HM and then cause ischemic strokes we know this so basically the respect trial looked exactly at this so cryptogenic stroke and p fo and patients who had other explanations for stroke for example carotid disease or atrial fibrillation were excluded okay so and the end point was recurrence of non-fatal ischemic stroke or fatal stroke okay so this was basically what was tested in the trial so approximately 1000 patients randomization one two one two Medical Group quite heterogeneous therapy but one half got medical treatment the other one was randomized to the device so those of you who do interventional or may know about it this is an Amplatz P vocoder which is simply placed in the interatrial septum to occlude the the persistent foramen ovale okay so this group receives dual antiplatelet therapy and then there was a te study done after six months so very briefly safety was excellent almost you know nothing happened same rate for atrial fibrillation no strokes during the procedure bleeding was was was identical there were more vascular access complications because obviously if you randomized to medical treatment you don't do anything with the patient if you try to do a closer intervention you need vascular access which is done to the femoral vein so you have to poke in patients who you may cause some problems and obviously there were more complications here but other than that everything was identical so it was very safe it was very successful technical success meaning that occluder devices in the patient very high 99% procedure process success was a definition within hospital and then affective closure was evaluated six months after the study so they did a te study and they actually looked at residual shunting so ninety-three percent of patients had an effective closure of their of their defect high technical technical success and a very safe procedure now this is the outcome as for all trials intention-to-treat is there to test for superiority so when you look at this 50% relative risk reduction for the endpoint of stroke okay but look at the p-value 0.08 so not a statistically significantly different look at the number of endpoints 9 versus 16 okay so a low number of endpoints so I have to say that so now here's the sad story about this so to have a stroke because of P fo is very rare so it's something you have to say that so it's one reason but it's not very commonly happening obviously every fourth of us has a p fo so we're not all suffering strokes right so it's a rare event now once you suffered one stroke to have another one is even you know is it's also very odd thats when neurologists will always say let's go for medical treatment first and then way it's okay and if it hits you a second time that we can think about closure so that's why now what happened in the trial was that patients were randomized and while waiting for the procedure three patients in the device trial device group suffered stroke okay they were waiting to get the procedure and in that waiting time they suffered a stroke which is very very rare so it's very unfortunate but it happened okay so what what does that mean it means in the intention-to-treat analysis patients were randomized they're included in the analysis right so that's an event that's a stroke so basically it was counted as an event in the device group although they never had a device okay so when we look in the s tweeted cord so these are the patients who actually got a closure device you have a 70% risk reduction significantly reduced okay so different interpretation so let's save that we've been intention-to-treat to say hey this was a positive child but it was not because intention-to-treat analysis would not was not significantly different so then they put all this together you know you see the the relative risk correction 50% 70% number needed to treat went down over the years it makes sense so but the interpretation was quite clear technically of course you have to say neutral trial there's no benefit intention-to-treat okay now we know that even after longer follow-up of these patients there was a highly significant benefit and this was just published last year in the New England Journal together with two other trials showing that there is a benefit at least in certain patients with p fo closure okay so now I have this there were several meter analyses and I would like to show you this so technically there are different occluder systems okay so look at this one this is the result for the Amplatz so there were three different trials I have to say that testing this two of them use the Amplatz device and one use the so-called star flex device so when you look at this analysis just looking for the Amplatz device there was a significant benefit even in these trials when you look for the star flex design a device it was not just look at this thing this looks like I don't know what to say to this so actually implanted these that were terrible so this is one thing I also implanted these they are very easy to handle from an interventional perspective it's it's easy going straight forward so now look at this testing for heterogeneity there's no heterogeneity it's not stick tests you know it's not different so how can that be just look at these monsters here how can there not be heterogeneity explain that to me please why is that look look at these beasts you know they look completely completely different ugly nasty beasts here and very smooth easy going so why is that you're not comparing the two of what are you comparing you're comparing a principle the same thing okay you're comparing a principle the principle of closure device versus medical treatment and that's why there's no heterogeneity okay but medically it makes a big big difference of what you put into a patient okay and now we have this that was even before the long follow-up after the respect trial so this was something looking just comparing the intention-to-treat analysis of the trials of the three trials and looking for the s treated analysis and this is not DIF cystic lead if ur n't in the asteroid analysis there was a significant benefit so and now comes my last slide my question for you you know why should we use intention-to-treat in medical device rods or should we do as treat it or should we maybe do something modified intention-to-treat so what is your opinion this is it's clear what is you know the standard of care the standard is intention-to-treat okay but this is a perfect example why I think it's not the right way to go so what should we doing in the future what do you think there's no right or wrong answer here because I haven't I haven't I have a proposal for you but what is your opinion both both is always good so you just choose what you want right yeah I mean this I think it's a perfect example why intention-to-treat although it's the standard it doesn't really make sense that's my opinion on the other hand if you go for as treated it means the patient has a an occluded device in his chest and it did work out right so then he's counted in you have to keep in mind that when you do a procedure like this you have to poke a patient have to go in you have to cross the septum you're doing something you can harm a patient by doing this and this is safety of course and so I personally think it's it should be something like modified intention-to-treat because you're attempting the closure at this moment you're poking you're going in and maybe you fail because of a complication or you you can't put the device but that should be counted in because you actually you know doing something with the patient so that's my personal opinion it should be something like modified intention-to-treat because then you have the patients with the device but you're also covering the safety aspects of that but that is not standard this is just my personal opinion okay I think this is my last slide I told you we will do this ten times I think we did not I think it was only six times but here it is again okay join the working group we are great people membership is free and you can go through this link which you will find on the on the webpage of our working group ok thank you very much

Leave a Reply

(*) Required, Your email will not be published