Anyone with experience in RCT design? Advice would be appreciated...

[ZagDoc]

Would appreciate some advice from anyone who has had experience designing or running a RCT before... I can't give too much detail due to disclosures but the general situation is this:

We have a medical device which we are in the process of applying for FDA investigational device status for. IDE status is generally granted concurrent with a randomized controlled clinical trial to demonstrate safety/efficacy. The device is an implantable device used to treat a rare condition which carries a mortality of ~20% in it's most severe form. We've used it before in humans with a life-threatening form of the disease (n=1 cases) with great outcomes and no side effects. I'm in the middle of a research fellowship and the device development has landed right in my lap. Unfortunately, I don't have any formal post-doctorate training clinical trial development, etc.

Our current protocol has a primary endpoint of a radiographic measurement which is a direct measurement of severity of disease. Our goal is to run an evaluator-blinded study, with the control arm being "conventional treatment" and the therapeutic arm being "our device + conventional treatment." We've run some stats and would only need 5 patients per arm for 80% power to demonstrate improved radiographic outcomes (i.e. improvement in severity of disease, which would demonstrate efficacy of the device).

Several clinicians I have run the study design by are concerned with not having mortality as a primary endpoint of the study for a couple reasons: 1) that there is a relatively significant rate of mortality (20%) in this patient population, 2) we have not sufficiently demonstrated safety of the device in our prior work, 3) that we cannot draw any actual conclusions on device effect on survival without including mortality as a primary endpoint, and 4) if we assume equipoise, we cannot determine when we have crossed the threshold where we can no longer ethically deny patients the device unless we better characterize the rate of mortality of the control group.

There are some significant challenges with including mortality in a composite primary endpoint. First, mortality in the control arm is expected to be around 20%. Even if we presume a 15% benefit in mortality by the device, to detect a difference between 20% vs. 5%, we'd need almost 100 patients per arm to have 80% power. Given the rarity of the condition we are treating, this would make sufficient recruitment to power the study prohibitive.

One other issue with including mortality as a primary endpoint is that treatment switching is inevitable, particularly for the control arm. Given that we have humanitarian use designation for our device, we do not want to withhold the device from control arm patients who decompensate during the course of the study and are at imminent threat of death. But this would siphon the sickest patients from the control arm into the therapeutic arm and diminish the statistical significance of the effect on mortality and increase the #s needed to show statistical significance.

I had a couple thoughts on how to mitigate this. One is to stratify randomization into low risk and high risk groups (where mortality in the control arm is expected to approach 100%), with the expectation that the rate of mortality would be much higher in the high risk group. We could then power the study sufficiently to detect (1) statistically significant difference in airway patency in the low risk group and (2) statistically significant difference in mortality of the high risk group. But I do not know if this is unsound clinical study design or if, even if we stratify, whether N needed to power would be prohibitive.

Another thought would be to not include just mortality but rather define the endpoint as “treatment failure”, which would be a surrogate endpoint for mortality. Treatment failure would be defined as death, life threatening event (harbringer event), or a significant complication (of the device or of conventional treatment) that would result in death if not hospitalized. We could then allow switching at treatment failure and define rate of failure or time to failure as a composite endpoint, with the expectation that most high risk group patients will fail. This would decrease our N needed to power the study and make recruitment more feasible.

I also have not found anyone who has been able to tell me whether having a composite primary endpoint that includes both a direct measurement and a surrogate endpoint which is a rate or time to event is sound clinical trial design or not.

Any thoughts on these issues would be appreciated... I really want the trial design foundation to be solid for sake of being able to stand on our results. It's been hard to find anyone at my institution who has direct experience with RCT trial design who can vet the protocol.

If the mods feel this thread would be better served in another sub-forum, feel free to move.

---

Members don't see this ad.

 

[dl2dp2]

This forum is likely not the right place for this. However, being a clinical trialist, I can tell you that from what it sounds like you will probably need what is called response adaptive randomization. The best known/old example is the pediatric ECMO trial (Bartlett et al). The statistical expertise you will need is likely going to be substantial because you will definitely need clinical trial simulation analysis before initiating the protocol or even getting it through the IRB. There's Scott Berry who does Bayesian adaptive designs consulting and Richard Simon at NIH who is a leader in this field. You might want to get in touch with them.

 

[ZagDoc]

This forum is likely not the right place for this. However, being a clinical trialist, I can tell you that from what it sounds like you will probably need what is called response adaptive randomization. The best known/old example is the pediatric ECMO trial (Bartlett et al). The statistical expertise you will need is likely going to be substantial because you will definitely need clinical trial simulation analysis before initiating the protocol or even getting it through the IRB. There's Scott Berry who does Bayesian adaptive designs consulting and Richard Simon at NIH who is a leader in this field. You might want to get in touch with them.

Thanks for the lead!

 

[gutonc]

I am not a clinical trialist (but I play one on TV...I also sit on our IRB). While mortality would certainly be an awesome endpoint, it's unlikely to be realistic as the primary endpoint for the study you describe because I doubt you have the number of devices you'd need actually manufactured and ready to go. It would also probably result in a study with a 20 year enrollment period. No reason not to include it as an exploratory or secondary endpoint but your primary endpoint, since you've implanted exactly one of these devices so far and have a compassionate use exemption, should be safety. As you pointed out, you could include something more squishy like "event free survival" as a secondary endpoint with an event being defined as any of the outcomes you've described, including death.

I'm only really familiar with the details of drug studies, not devices, but you're basically describing what, in drug-world, would be a Phase I study with safety and dose as the primary endpoints, and various measures of efficacy as secondary or exploratory endpoints. Since you have a radiographic measurement, I would argue that you should use that as either a co-primary (with safety) or secondary endpoint and include EFS as a secondary endpoint and mortality as an exploratory endpoint.

Now that I've rambled on with all of my ideas, I should point out that what you really need to do is get together with a very good biostatistician with experience in designing device trials, and come up with appropriate endpoints on an empirical basis. If your institution doesn't have one available (and if you're at a major academic medical center, they do, you just need to find them) then you need pony up the cash for the consulting fees for one outside of your institution.