AI and Clinical Research: the Possibility of Decision Accuracy

Clinical research is the test of the medications to achieve real application. After leaving the limited space of cells and the ideal animal models, the effect and safety of the medications after entering the human body is a complex issue, which is affected by many factors such as individual biology, environment, and epidemiology. We expect that ideal drugs can not only be fully utilized by the patients who need them but also avoid the waste of R&D and clinical resources causing by the ineffective drugs. Therefore, the design of the clinical research is a very scientific and rigorous work.

The process of clinical research design needs to address an unresolved clinical problem, for example, the existing targeted drugs for certain tumors can not provide patients with long-term survival benefits, or certain hypertension drugs can not further reduce the systolic blood pressure of refractory hypertension. Of course, the needs of patients must also be considered. If a drug is effective but of no significant difference compared with the existing treatments, and it is very expensive, obviously it can not be accepted by patients and payers. The determination of inclusion and exclusion criteria is the most challenging part of clinical research design. Determining the objects and exclude the subjects that should not be studied is not only a test of the medical and statistical skills of clinical research designers but also dependent on extensive experience. Researchers often search a large number of previous research articles and data for clues and supports, and combined with their own experience, they finally develop a set of standards. Some of the standards are conventional, some are unnecessary, which will bring obstacles to the clinical research and follow-up analysis. Undoubtly, there are still some missing, which may be unusual, but to ensure that the results of the study are correct, researchers tend to set more standards. Therefore, how to reduce the subjective factors of clinical research design and increase the objectivity of clinical research design is a field worthy of exploration.

Liu Ruishan et al. published the algorithm tool trial Pathfinder on GitHub, and the related research was also published in Nature. The research team used the electronic health record database derived from Flatiron Health HER to simulate several completed clinical studies of non-small cell lung cancer with propensity score weight, and quantitatively studied the impact of each inclusion and exclusion criteria on the number of patients and test results, and then the impact of each criterion on the results was evaluated by Shapley value (the weighted average of the impact on HR after adding each criterion to the ruleset, if less than 0, indicating the inclusion of this criterion can improve the effectiveness of the test). The results showed that compared with the previous criteria, the data-driven criteria deleted 9 inclusion or exclusion criteria, the overall survival rate decreased by 0.05, and the number of eligible people increased from 1553 to 3209, an increase of about 107%. Similar results are obtained in the confirmatory analysis of other data sets and security data (1)

The intuitive impression of clinical research is that it is inefficient. According to the analysis of clinical trials from January 2000 to April 2019, only 10% of the trials ended successfully. The reasons for the failure include lack of efficacy or safety, design defects, lack of funds, inability to recruit enough patients, improper operation in the process of the trial, etc. With the establishment of electronic medical data systems (such as Flatiron Health EHR) and the popularity of wearable devices (such as Apple Watch), the proportion of AI used in clinical research will increase. Probably the current role of artificial intelligence in promoting clinical trials is still overestimated, but it is imperative to improve clinical trials. Natural language processing may help to obtain medical history data and information mining, but it needs to conquer the complexity of clinical texts (such as unstructured and professional knowledge), which makes it very difficult for machines to understand. AI can help patients to search for ongoing clinical trials. If patients search for ongoing clinical trials on Baidu and Google manually, it might take them a whole day and night to get a reliable piece of information. Clinicaltrial.gov has collected information of more than 300 thousand clinical trials from 209 countries, but it takes some time even for professional doctors to find the right information, let alone the patients. Crimteria2query and Dquest developed by Weng Chunhua of Columbia University can help patients filter out a certain amount of invalid information. Antidot and Deep6ai can recruit subjects with the help of AI tools, which is about 8 times more efficient than traditional recruitment (2). Although some pharmaceutical companies think that the design of clinical trials is not as complicated as they think, it may be the view of the era of traditional drug treatment. Human genome data promotes accurate personalized medicine, and artificial intelligence and data-driven patient selection will adapt to a variety of trial queues. At present, the association of genomic analysis (CGP) with clinical features in HER is also being promoted. Singal G et al. published an article in JAMA to correlate CGP data (such as driving genes, tumor mutation load TMB, etc.) with clinical features (OS, treatment time, ORR, etc.) recorded in her. Among the 4064 patients with non-small cell lung cancer, 817 had EGFR, ALK, or ROS1 mutations, the TMB of smokers (8.7 [IQR, 4.4-14.8]) was significantly higher than that of non-smokers (2.6 [IQR, 1.7-5.2]). Compared with patients with TMB less than 20, patients with TMB greater than 20 who received PD-1 or PD-L1 antibody therapy had longer OS (16.8 [95% CI, 11.6-24.9] vs. 8.5 [7.6-9.7]) and longer treatment time (7.8 [95% CI, 5.5-11.1] vs. 3.3 [95% CI, 2.8-3.7]) (3). This may also be an application scenario for data-driven clinical trial design and decision-making, and some start-ups in the United States (4) and China (5) also show strength and ambition.

At present, the theoretical discussion on the improvement of artificial intelligence for clinical trials is far greater than the verification. If artificial intelligence is judged as an unreasonable experimental design, it seems to be a paradox whether pharmaceutical companies and research institutions will spend time and effort to carry out it only to see if the artificial intelligence is right. Artificial intelligence, as an assistant decision-making tool, still has a long way to go, and the reason why it is called "intelligence" is that AI’s value should not be limited to "assistance", but to replace the clinical trial itself. If we can complete a clinical trial in front of the computer over a cup of coffee, which may cost tens of millions of dollars and recruit tens of thousands of patients, and need to sign an agreement with a CRO employing hundreds of people and conduct a multicenter, prospective, randomized controlled study for several years in the past, this is how to release the greatest scientific and commercial value of AI.

References：

1, Liu RS, et al. Nature. 2021;592:629-633.

2, Woo M, et al. Nature. 2019 Sep 25.

3, Singal, et al. JAMA. 2019;321:1391-1399.

4, Trial.ai: https://www.trials.ai.

5. Deep Intelligence Pharma (深度智耀): https://www.dip-ai.com