The World Health Organization (WHO) met with Kenya’s Ministry of Health on November 25, 2024 to discuss preliminary results from the first-ever real-world evaluation of the WHO Skin NTDs app, powered by two algorithms based on artificial intelligence (AI).
This study – carried out in collaboration with TDR, the Open University of Catalonia, the Catalan Institute of Health, the Kenya Medical Research Institute (KEMRI) and the University of Bristol – is the first of its kind to simultaneously evaluate two algorithms based on AI: one developed by Universal Doctor for WHO, focused on 12 skin-related neglected tropical diseases (cutaneous NTDs); the other developed by Belle.ai, personalized at the request of the WHO for 24 common skin conditions.
The study was conducted in five counties in Kenya, where 40 frontline health workers at the primary health care level of the Ministry of Health collected 605 images of skin lesions from an equal number of patients between June and October 2024.
“Skin diseases remain a significant public health problem in Kenya, accounting for up to 10 to 30% of outpatient consultations. Added to this challenge is the shortage of dermatologists in the country. The WHO Skin NTDs app, powered by AI, is designed to help health workers improve disease management,” said Dr Joyce Onsongo, NTD Lead at WHO Kenya.
Preliminary results showed an average sensitivity of approximately 80% for both algorithms compared to diagnoses provided by three board-certified dermatologists.
“Being able to carry out this study in a relatively short period of time in remote areas is already a great achievement for everyone who made this study possible. The results are very encouraging,” said Dr José Antonio Ruiz Postigo, a medical officer with the WHO Global Program for Neglected Tropical Diseases.
In addition to evaluating the performance of the AI algorithms, the study also collected comprehensive feedback from frontline healthcare workers who used the application in their specific healthcare settings. Results from the user version of the Mobile Application Rating Scale (uMARS) showed an average score of 3.82 out of 5, while the group discussions reflected a score of 8 out of 10.
“Health professionals felt empowered, their self-esteem increased significantly and they were trusted more by peers and patients to assess skin lesions,” said Professor Carme Carrion, lead researcher at the Open University of Catalonia.
“It was interesting to see that frontline healthcare professionals felt the app was their partner in diagnosing skin conditions and helping to destigmatize patients affected by skin NTDs,” said Dr Ruth Nyangacha , research scientist at KEMRI and the principal investigator of the site in Kenya.
The next step is to conduct a thorough analysis of the results to determine the path forward.
WHO plans to replicate this study in Brazil, Cameroon, Ivory Coast and India in the next biennium to assess the performance of AI algorithms in different epidemiological contexts.
————————————
The updated app is available for free on Android ( here) and iOS ( here) devices.