Comparison of artificial intelligence and rheumatologists in nailfold capillaroscopic evaluation

OBJECTIVE:  The aim of this study was to evaluate the diagnostic accuracy of a pretrained Vision Transformer model on nailfold capillaroscopy images, comparing its performance to that of expert rheumatologists.

METHODS:  We retrospectively analyzed 104 anonymized images (23 normal, 81 pathological) from publicly available datasets. The pretrained Vision Transformer model was applied without fine-tuning. Two rheumatologists independently assessed the same image set. Accuracy and interrater agreement were calculated.

RESULTS:  The AI model produced 0% clinical applicability, failing to generate meaningful classifications. In contrast, the consensus of two rheumatologists achieved the highest diagnostic performance, with 94.2% accuracy and a Cohen’s Kappa of 0.827. Individual rheumatologist evaluations yielded comparatively lower accuracy and agreement.

CONCLUSION:  We retrospectively analyzed X anonymized images (XX normal, XX pathological) from publicly available datasets. The pretrained Vision Transformer model was applied without fine-tuning. Two rheumatologists independently assessed the same image set. Accuracy and interrater agreement were calculated.

KEYWORDS:
Nailfold capillaroscopy; Artificial intelligence; Diagnostic imaging; Scleroderma, systemic; Deep learning