Over the past years, the number of challenges has risen significantly (for example, from 15 MICCAI challenges in 2018 to 50 in 2024). These challenges play a crucial role in advancing our research field, as they not only provide valuable benchmarking datasets but also set new standards by showcasing state-of-the-art methods for specific problems. Given their importance, ensuring the highest quality of these challenges is essential.
The Special Interest Group (SIG) for Challenges has implemented several measures to enhance quality, including the introduction of reporting guidelines, metric recommendations, and educational webinars, since there was no quality control before its establishment. Despite these efforts, quality issues persist.
The SIG is now working towards establishing best practices, such as introducing "lighthouse challenges" that undergo a more thorough review process, including data evaluation.
Want to know which three challenges have achieved lighthouse status? Attend the SIG for Challenges Report session being held today (October 9) at 16:30 in Orion Tent (Main Hall)!