Machine-learning-based tools are being applied to the diagnosis and management of a range of disease states. In many cases, these tools can improve patient care and management, but there is also concern that this technology can become disconnected from the realities of frontline practice. To address concerns about the application of this technology in the diagnosis and prognosis of patients with multiple sclerosis (MS), Tom A. N. Fuchs, MD, PhD, and colleagues conducted a retrospective, multicenter case–control study of adult patients with relapsing–remitting MS. The research team, in collaboration with global partners through the MSBase research consortium, used machine learning to develop two complementary tools for individualized 5-year risk estimation. The first is DAAE-M (Disease duration, Age at disease onset, Age, Expanded Disability Status Scale, and disease-Modifying therapy), which is optimized for transparency, software-neutral use, and mitigation of indication bias. The second is ELIE (Empirical Landmark-based Individualized Estimation of MS progression risk), which is optimized for dynamic landmark-based modeling, complex treatment histories, and mitigation of immortal-time bias. The results of the study were published in the Journal of Neurology. Dr. Fuchs spoke with Physician’s Weekly regarding the findings.
Physician’s Weekly: Can you describe the machine-learning tools you have developed to estimate 5-year progression risk in people with MS?
Dr. Fuchs: We recently developed the DAAE-M score, a machine-learning-based tool for predicting disease progression over 5 years in people with multiple sclerosis. Integrating information about disease status and use of disease-modifying therapies, the DAAE-M score is accurate, makes consistent risk predictions across settings worldwide, and can be used to estimate the risk of transition to secondary progressive MS or high-disability with progression independent of relapse (Lorscheider criteria).
Notably, we used machine learning to build the tool for greater accuracy, but we also took extra steps to ensure the final predictive model was transparent, interpretable, and usable in clinical settings in under 30 seconds.
The DAAE-M score is freely available here: https://tomafuchs.com/daae-score/
And it will soon be available through the EPIC electronic medical records global platform.
Why do you think it was important to add these tools to the ongoing diagnosis, treatment, and management of MS?
The DAAE-M score is perhaps most useful for engaging patients in conversations. There is no one-size-fits-all treatment algorithm for people with multiple sclerosis – but knowledge about the future can contribute to dialogues between physicians and their patients, especially during critical windows of opportunity for rehabilitation or treatment changes. We view the DAAE-M score as a source of free information, one which can be accessed if desired or ignored if not. These tools offer information much like a weather report, where some may choose to pack an umbrella, and others may choose to go out in a t-shirt because the risk of rain is low. The report itself is another tool in the toolbelt, to be wielded when helpful.
What were the objectives of the study you developed?
The objectives of our study were to develop a machine-learning-based predictive model that was accurate, useful, and usable for physicians treating patients with multiple sclerosis. In this case, we targeted the outcomes of transition to secondary progressive multiple sclerosis and transition to high disability with progression independent of relapse. These transitions are relevant for our patients because after reaching secondary progressive multiple sclerosis or high-disability progression independent of relapse, people have more precipitous disability worsening and no longer respond as well to medical therapy or rehabilitation.
What were the most impactful findings of the study?
I think the most impactful finding in this study is a finding related to the barriers between science and clinical practice. Machine learning is being used and abused on a large scale in scientific environments, yet little attention is given to global validation or user experience. We feel that practicing physicians are being left out of too many conversations, and that much of medical science fails to meet the needs of daily practice. We tried to address that societal issue in this project and engaged practicing physicians at every stage to ensure we provided an experience that met their needs and expectations. This led us down an interesting path, where we had to behave not only as researchers but also as software developers – always aiming towards a particular user experience rather than allowing the technology itself to dictate the direction of our research.
How do you think these findings can be applied to practice?
We believe the DAAE-M score and similar tools could impact daily clinical practice by fostering more nuanced, data-driven, and risk-based dialogues between physicians and patients. That said, we actually feel that the future of tools like the DAAE-M score is even more interesting. As physicians become accustomed to interacting with more accurate and usable predictive tools, practice will evolve, requiring new tools to improve patient care. This interaction between use and need will engender future innovations that we hope practicing physicians feel empowered to contribute toward.
Why are machine-learning tools so important to progressive care of patients with MS?
In neurology, we are accustomed to using tools such as the CHA2DS2-VASc score to assess stroke risk. This sort of algorithm serves as a good example of how predictive tools can guide medical practice. In the current age, though, we can use machine learning to further refine tools like this.
What still needs to be studied? What requires further research?
We have identified important future directions for this line of research and broken them down into two pathways: depth and breadth. The depth pathway is one in which we continue to validate the DAAE-M score across new countries and clinical environments to ensure its predictions remain accurate, reliable, usable, and interpretable. One important feature of this work will be a “randomized-controlled trial” of sorts, in which we investigate how machine-learning-based tools, such as the DAAE-M score, affect clinical practice and patient outcomes. This is an important future direction for machine-learning research. The breadth pathway of research is equally important. In this pathway, we develop new predictive tools for new important outcomes. In our discussions with physicians, we have identified the following outcomes that we think will be important to predict: higher disability (score of 6 on the Expanded Disability Status Scale) and high cognitive dysfunction. We hope to cover more important outcomes with new predictive tools in the future, and we are excited to see how this line of work develops as we hear more from doctors.
Is there anything else that you would like to share?
We urge physicians to constantly push researchers to have clinical practice in mind as they conduct research. What are the barriers to clinical implementation? What are the tools you wish you had? What would stop you from using them? What do you need to trust the tool for daily use? These barriers and thoughts could be considered at the earliest stages of research design, but they require that practicing doctors have a seat at the table. Most researchers do not have the lived experience of a doctor and need your voice.