AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge

Coen De Vente, Koenraad A Vermeer, Nicolas Jaccard, He Wang, Hongyi Sun, Firas Khader, Daniel Truhn, Temirgali Aimyshev, Yerkebulan Zhanibekuly, Tien-Dung Le, Adrian Galdran, Miguel Angel Gonzalez Ballester, Gustavo Carneiro, R G Devika, P S Hrishikesh, Densen Puthussery, Hong Liu, Zekang Yang, Satoshi Kondo, Satoshi KasaiEdward Wang, Ashritha Durvasula, Jonathan Heras, Miguel Angel Zapata, Teresa Araujo, Guilherme Aresta, Hrvoje Bogunovic, Mustafa Arikan, Yeong Chan Lee, Hyun Bin Cho, Yoon Ho Choi, Abdul Qayyum, Imran Razzak, Bram Van Ginneken, Hans G Lemij, Clara I Sanchez

Research output: Contribution to journalArticleResearchpeer-review


The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.

Original languageEnglish
Pages (from-to)542-557
Number of pages16
JournalIEEE Transactions on Medical Imaging
Issue number1
Early online date15 Sept 2023
Publication statusPublished - Jan 2024


  • Algorithms
  • Artificial Intelligence
  • Diagnostic Techniques, Ophthalmological
  • Fundus Oculi
  • Glaucoma/diagnostic imaging
  • Humans


Dive into the research topics of 'AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge'. Together they form a unique fingerprint.

Cite this