Achieving Statistical Significance when Global Patient Populations are Limited

Achieving Statistical Significance when Global Patient Populations are LimitedImage | AdobeStock.com

Highlights of a recent panel debate moderated by Christian K Schneider, head of Biopharma Excellence, which explored emerging approaches to clinical research and data collection for therapies linked to rare diseases.

Biopharma Excellence recently brought together experts in biotech and the evolving regulatory environment to discuss the challenge of achieving statistical significance with novel and highly targeted therapies, when target populations can be impossibly small.

From a regulator’s perspective, Daniel O’Connor, whose remit at the UK’s MHRA is Innovation Acceleration within a regulatory context, noted that the challenge of achieving statistical significance when target populations are tiny is an age-old issue. Ideally a result would be both clinically meaningful and statistically significant, he said, while conceding that randomised trials aren’t always viable, especially with very rare conditions.

He urged drug developers to seek scientific advice before finding workarounds. “Don’t make trade-offs on your own. Have those discussions with the regulators and other decision-makers.”

Supplementing clinical trial research data

So, what about the potential for supplementary data sets – such as pharmacodynamic read-outs, histological evidence, and so on – which can help enable regulators to reach robust decisions? Options also include use of historical controls, where old data is compared with new data from new trials.

Nick Sireau, of AKU Society, talked about his early experiences of trying to get new drugs approved for Alkaptonuria (AKU). AKU Society is a specialist patient group he created because his two sons both suffer from the ultra-rare disease. It relies heavily on grant funding and donations, with access to funds often difficult. It doesn’t help that the authorities’ requirements for clinical evidence have been quite restrictive up to now.

Fifteen years ago, 40 patients were recruited to an AKU drug trial that lasted three years and focused on a single endpoint (hip rotation), Nick explained. “The trouble is, that AKU affects patients very differently,” he noted. “It can affect any of the joints. So, to just look at 40 patients with a single endpoint proved futile.” Although patients were reporting that they could walk further, that their pain had reduced, that they were feeling better since joining the trial, the study itself failed.

This blow led AKU Society to form a consortium of linked organisations across Europe, and a composite end point – a proposition which by then was well received in the EU. “EMA told us they wanted us to have as a primary endpoint the metabolic/homogentisic acid/culprit molecule, reducing that to nearly zero; but also clinical or secondary endpoints as a positive trend – even if we didn’t reach statistical significance. That changed everything.”

A progressive regulatory agenda

Daniel said he believed the industry was now witnessing an evolution in clinical research data and regulatory thinking, resulting in a much more proportionate approach with regard to looking at the disease condition based on how much is already known: “not just thinking about either a randomised study or a single-arm study.”

Oncology’s use of basket studies has helped shine a light on what’s possible, he added. “But there’s also a lot of work now looking at shared molecular entities in rare diseases – something the International Rare Diseases Research Consortium (IRDiRC) is moving on quite quickly now.” The toolbox to get to the clinical research data now needed is visibly expanding, he argued.

Likely next steps will need to come from a collaborative discussion and feasibility conversation – with patients; with researchers; with the regulators – about what’s viable with a clinical development program. “We need to think about timelines, too,” Daniel added. “Three years is a long time to run a study.”

Reimbursement: identifying & aligning endpoints

The discussion moved on to the topic of reimbursement, which in turn requires clarity about endpoints. Developers and regulators may be thinking about patient risk/benefit, while payers are more likely to be weighing up the cost/benefit.

The panel suggested that the industry as a whole must think more broadly in terms of endpoints that really matter, particularly when it comes to very rare diseases for which there have previously been no real treatment options – and to include the patient voice in all of this, as early as possible.

Nick said he sensed from EMA that accommodating the patient perspective was something it had now built into its structures, whereas on the whole the big pharma companies were less proactive about patient engagement. “Most drug companies will start working on something and then get in touch with us right at the last minute when things have started to go wrong,” he said. “There doesn’t seem to be any systematic way that pharma companies engage with patient groups, particularly in the rare disease space.”

Involving patients at an earlier stage, finding out what’s most important to them and using this to direct clinical research data, is a growing area of focus among those thinking about all of this more strategically. Where cures aren’t possible or may still be a way off, for instance, factors such of quality of life can be hugely significant to patients.

Daniel talked about the Innovative Medicines Initiative (IMI), SISAQOL-IMI, which is working on recommendations about how to analyse and interpret data on health-related quality of life (HRQOL). It is led by the European Organisation for Research & Treatment Of Cancer and, although it is specific to oncology, some of its findings are likely to support the broader application of patient-reported outcomes.

“That’s not only in clinical studies, but also in a real-world setting – to be able to truly monitor how patients are doing across their journey with a particular medicine after authorisation. It’s something the MHRA is really keen on,” he noted. “We’re working very hard to try to embed patient-reported outcomes as much as we possibly can in the clinical input that we give to those developing medicines.”

A wish-list for change

The panel ended with their wish-list for change if nothing was off the table. For Rachelle Jacques of Akari Therapeutics, multi-stakeholder approaches to overcoming practical barriers at a macro level is the way to go – rather than one company at a time; one patient advocacy organisation at a time; one regulatory body at a time – if the 95% of today’s remaining unmet needs are to be addressed sooner rather than later.

Speaking for patients, Nick said his wish would be finding a funding model that really works particularly for ultra-rare diseases, to fund studies which are otherwise just not commercially viable. “The US is very fortunate to have the Chan Zuckerberg Initiative, which is doing really good things in building patient groups with significant funding,” he noted.

About the panel

  • Christian K Schneider, head of Biopharma Excellence and Chief Medical Officer for biopharma at PharmaLex, is a former regulator at both the British MHRA and the Danish Medicines Agency.
  • Nick Sireau is CEO and Chair of Trustees for the AKU Society which is dedicated to improving the lives of Alkaptonuria (AKU) patients through patient support, community building, and medical research.
  • Rachelle Jacques is President and CEO of Akari Therapeutics, a late-stage biotechnology company developing advanced therapies for auto-inflammatory and orphan diseases.
  • Daniel O’Connor is Deputy Director with responsibility for the Innovation Accelerator and Regulatory Science at UK healthcare agency, the MHRA.