Large Multimodal Models (LMMs) present an innovative approach in healthcare leveraging the integration of heterogeneous data types to improve and optimize patient diagnosis and management. This approach shows particular promise for diseases in which combining data is crucial for accurate diagnosis, or when expert personnel are scarce, and the differential diagnosis is complex.
Neglected Tropical Diseases (NTDs), particularly skin NTDs such as Buruli ulcer, leprosy, yaws or scabies, represent an optimal scenario in which to apply this methodology, given the need for an early diagnosis, the complex differential diagnosis and the scarcity of dermatologists in rural areas.
We developed and tested two assistants for the diagnosis, management and follow-up of patients with suspected skin NTDs in Côte d’Ivoire: i) “Florence,” an expert in Skin NTDs; and ii) “Florence Pro,” an augmented version of Florence equipped with a guide from World Health Organization on skin NTDs tailored for front-line health workers and the ePilly Trop book on tropical infectious diseases.
The configuration of these two assistants has adhered to the CO-STAR framework augmented with one-shot learning. Afterward, these were deployed into a mobile App, MultiSpot, facilitating the combination of clinical data and images of skin lesions.
This system was tested at the Divo Regional Hospital in Côte d’Ivoire, where clinicians evaluated its effectiveness in diagnosing simulated cases of patients with skin conditions. Following testing, a performance survey was conducted, measuring system usability and accuracy metrics, reaching promising results.
Preliminary findings underscore the potential of this tool in diagnosing and managing patients with suspected skin NTDs, being particularly useful for health workers and individuals with limited expertise in dermatology.
Subsequent steps entail implementing a user journey in the assistants, and conducting this study on a larger scale, encompassing a broader spectrum of cases and diverse personnel.
Co-authors:
Elena Dacal*, Iago Veiras*, Oscar Darias, Jaime García-Villena, Alvaro López- Caro, Alejandro Angulo, Labiya Toure, Ange Théodore Yao Kouakou, Aboa Paul Koffi, Christian R. Johnson, Emma Saéz-López, Israel Cruz, Miguel Luengo- Oroz
*co-first authors
Affiliations
Spotlab, Madrid, Spain
Programme National de Lutte contre l’Ulcère de Buruli – Cote d’Ivoire
Fondation Raoul Follereau, Paris, France. University of Abomey-Calavi, Abomey-Calavi, Benin
Department of Microbiology, Paediatrics, Radiology and Public Health, Faculty of Medicine, University
of Zaragoza, Zaragoza, Spain. Spanish Network for Research on Respiratory Diseases (CIBERES),
Carlos III Health Institute, Madrid, Spain
National School of Public Health, CIBERINFEC, Instituto de Salud Carlos III, Spain