Lets talk about a painful subject. I am of the opinion and Im far from alone that the most reliable way to determine if a possible therapy has any usefulness is a randomized, double-blinded controlled clinical trial. I can be a bit more specific than that, even: lets make that a trial that is run with sufficient statistical power to have a good chance of providing a meaningful readout.

The worldwide coronavirus pandemic has featured some well-run trials that have truly advanced our knowledge of the disease and how to treat it. But it has featured far, far more garbage. That word was chosen deliberately. There have been too many observational trials, too many uncontrolled (or poorly controlled) ones, too many open-label ones, and above all, there have been way too many trials whose number of patients would be insufficient to tell us much of anything even if everything else had been run properly.

I am not revealing any hidden tricks of the trade here. Clinical trial design is a subject with a very large literature, and there are any number of people and organizations who can provide useful guidance on both its theoretical and practical aspects. Among these aspects are the calculations that should be made for how many patients a trial is likely to need to be well-powered enough for a clean read on its clinical endpoints. You can start to learn the basic outlines of the subject online. Now, thats not to say that its an easy subject to get ahold of. Youre going to have to estimate some of your key parameters as well as you can, among them what you think the effect size of your treatment might be, what the patient-to-patient variability might be like, the time course of treatment that might be needed, and more. Just picking the proper clinical endpoints is a subject all in itself (and its one that can have a huge effect on a trials design and on its chances for success). And at the other end of things, your inclusion criteria and patient enrollment process is a place for serious thought, too. Who should be evaluated (or definitely not evaluated) in your trial, and how long will it take you to round those people up? Where are you thinking about doing all this, anyway?

There are a wide variety of trial designs out there as well, and you can find yourself sorting through some that are clearly inappropriate to the problem at hand, some that would be great if you had about ten times as much money and time as you do, and several that at first glance look like they could all work out, but which have real-world differences that its crucial that you be aware of. You would be well advised to consult with experience practitioners before you start, to make sure youre on the right track.

Unfortunately, underpowered, badly-run, and badly designed trials have been with us for a long time. Here are some well-justified concerns from 2002, for starters, and various fields of clinical research undergo periodic bouts of soul-searching over the years about these issues. But the pandemic year has really made some of our problems more obvious. Not only do we have trouble with badly run trials, but mixing in with that is a bandwagon effect. Clinicians all over the world just piled onto some of the coronavirus ideas, and kept piling on for months and months and months.Think, for example, about the hydroxychloroquine situation. Now, I still get messages condemning me as an implacable, irrational foe of the One True Coronavirus Therapy. But its worth remembering that I started out as a Huh, I dont know how that would work, but lets look into it person, which I really think should be the default setting. And in that spirit, I was all for running trials and getting more hard data.

But what did we get? A search through clinicaltrials.gov for hydroxychloroquine|coronavirus gives you 113 trials. Whats more, thirty-six of those are still listed as recruiting patients. This is ridiculous, but its not amusing. There are some large, well-controlled data sets available that indicate that HCQ is very likely not a useful therapy, but as you can see, there are also dozens of other smaller ones that say Yes! No! Maybe! Sorta! Kinda! Kinda Not! Depends! Could Be! Who Knows? And that adds up not just to a lack of knowledge, it turns into an actual hindrance to knowledge as you try to sort through the data. The heap of fuzzy indeterminate results also fuels the extrascientific political and cultural arguments about the drug, since everyone can find some sort of support for whatever opinion they might have.

You have to think that there were other therapies that deserved a look in the clinic as compared to the forty-third, sixty-seventh, or ninety-eighth hydroxychloroquine study. Youll recall that for a while, HCQ ended up mixed into other clinical trials just because everyone wanted it or imagined that it was some sort of standard of care, and that did no one much good, either. Now, HCQ isnt the only offender, but its a big one, and I think it illustrates what we should try not to do next time.

How, then, should we try not to do that? (Update: some thoughts here on this problem from a distinguished team of authors with exactly the same concerns). Its not like the US (to pick a big example) has a National Clinical Trial Authority that passes judgment on these things. To be honest, the downsides of having such an agency might worry me even more. But letting everyone go into Headless Poultry Mode and pile up overlapping crap in the clinic isnt such a good way to go, either. You would hope for a little more coordination among major medical research centers, and youd also hope for some local university/research hospital review boards to be aware that greenlighting the East Porkford Covid-19 Treatment Study with 47 patients isnt really going to advance medical science very much. Especially when its covering the same ground as the trials kicking off in Mashed Potato Falls, Rancho Malario, and Kidneystone Pass. But Im being unfair to East Porkford some of these lackluster trials were conducted at larger institutions that should have known better. The way were set up, its down to the review boards and the sources of funding to police things better, and to keep their heads while all about them are losing theirs.

And its also down to the NIH and the CDC to lead the way more than they did during 2020. The RECOVERY trial in the UK has been an example of what can be accomplished in that line. The NIH has helped run some good trials, but weve had nothing that comprehensive in the US as compared to the UK effort, and I really wish we had. I fear that some day, eventually, were going to have a chance to do better, and I hope that we take it.

More:
The Problem with COVID-19 Clinical Trials | In the Pipeline - Science Magazine

Comments are closed.