The quality of automated detection of sinusitis based on radiography results in practical healthcare (prospective diagnostic study)

Purpose. To study the diagnostic accuracy of software based on artificial intelligence technologies for the diagnosis of maxillary and frontal sinusitis (J01, J32) when used in practical healthcare. Materials and methods. Prospective diagnostic study (10.2023–12.2024) using the STARD 2015 methodology. The Index Test is four anonymized AI services for analyzing the X-ray imaging of the paranasal sinuses, integrated into Unified Radiological Information Service (ERIS) as part of the Moscow Experiment. The Reference Standard is an original methodology of clinical monitoring with a double expert review, an assessment of accuracy according to the criteria of “interpretation” and “localization”, calculation of sensitivity, specificity and AUC. After testing in the ERIS circuit, the AI services worked with real data and underwent monthly clinical monitoring. Results. From October 2023 to December 2024, AI services analyzed 126,547 radiographs of the paranasal sinuses, of which 53% were processed by AI service-4. The scope of the research varied due to the different duration of the services. Regular clinical monitoring covered 2,320 radiographs. The correctness of pathology detection, the accuracy of its classification and localization were evaluated. The highest diagnostic accuracy was found in AI services-1 and 2 (AUC 0.98 and 0.95; integral score – 95.1% and 83.3%). They demonstrated high specificity and precise localization of pathologies. AI services-3 and 4 demonstrated comparatively lower results (AUC 0.89; integral score – 76.6% and 79.4%), with lower specificity for AI service-3 (0.65). Conclusions. As part of the Moscow Experiment, AI services successfully analyzed 126,547 radiographs of the paranasal sinuses (2023–2024), showing an average level of accuracy (AUC 0.93) and quality (83.6%). The results obtained confirm the applicability of AI in clinical practice. Differences in accuracy between services require regular monitoring of safety and quality. The developed methodologies make it possible to effectively carry out such control. The achieved level of quality opens prospects for the transformation of the interaction of doctors of different specialties, which will be the topic of future research.