REVIEWER GUIDELINES

This document provides detailed guidelines to reviewers for MICCAI 2024. We believe it is important to summarize what makes a good MICCAI review and some of the expectations from you as a reviewer. We also include the rules that MICCAI 2024 adopts for paper anonymization as part of its double-blind peer review process. Please read these guidelines as part of the overall MICCAI 2024 review process document.

Please be aware that reviews of accepted papers will be made public (without disclosing the reviewers' identity), together with author responses and Area Chair meta-reviews.

Reviewers will be acknowledged in the conference proceedings. The top reviewers will be offered free registration to attend MICCAI 2024.

1. What Makes a Good Review

The role of a reviewer is to identify excellent papers that the MICCAI community must hear about, and tell the program committee which papers are of wide interest and could have a great impact on the field. A good review expresses an informed expert assessment about the paper and backs it up with details on strengths and weaknesses of the paper.

The components of the reviewing form are as follows:

A brief summary of the paper, which can be as short as a few sentences. This part tells the PC what the major contributions are, what the authors did, how they did it, and what the results were. It also helps authors to verify that the reviewer understood their approach and interpretation of the results.
The assessment of the reviewer about the major strengths of the paper. A reviewer should write about a novel formulation, demonstration of clinical feasibility, an original way to use data, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Provide details that justify your assessment. For instance, if a method is novel, explain what aspect is novel and why this is interesting.
The assessment of the reviewer about the major weaknesses of the paper. Provide a list of points that summarize your concerns about particular aspects of the paper. Provide details that justify your assessment. For instance, if a method is not novel, provide citations to prior work.
The assessment of the reviewer about the clarity of presentation, paper organization and other stylistic aspects of the paper. It is important to know whether the paper is very clear and a pleasure to read, or whether it is hard to understand. Provide details that justify your assessment. For instance, detail whether the paper is hard to read because of its technical level, or because of suboptimal organization.
Comment on the reproducibility of the paper. Where possible, we encourage authors to use open data or to make their data and code available for open access by other researchers. We understand that due to certain restrictions, some researchers are not able to release their proprietary dataset and code; therefore, a clear and detailed description of the algorithm, its parameters, and the dataset is highly valuable. Please provide comments about whether the paper provides sufficient details about the models/algorithms, datasets, and evaluation.
Detailed constructive comments should be provided to help the authors to improve their paper or to expand it into a journal version. Comments should be backed up by detailed arguments. Minor problems, such as grammatical errors, typos, and other problems that can be easily fixed by carefully editing the text of the paper, should also be listed.
Your recommendation whether to accept or reject the paper: Taking into account all points above, should this paper be presented at the conference? Is it an interesting contribution? Is it a significant advance for the field? Is the paper of sufficiently high clinical impact to outweigh a lower degree of methodological innovation? Please remember that a novel algorithm is only one of many ways to contribute. Other examples include (but are not limited to) a novel interventional system, an application of existing methods to a new problem, and new insights into existing methods. A paper would make a good contribution if you think that others in the community would want to know it. As a guide, note that MICCAI typically accepts around 30% of submissions.
A justification of your recommendation. What were the major factors in making your assessment? How did you weigh the strengths and weaknesses? Make sure that the reasons for your overall recommendation to accept or reject are clear to the program committee and the authors.
Ranking of this paper in your review stack: This information will be taken to calibrate the overall rating. Please try your best to avoid ties.
The expertise of the reviewer. If your expertise is limited to a particular aspect of the paper, this should be brought to the attention of the AC. The review is more likely to be taken seriously if the limitations of the reviewer's understanding are clearly acknowledged.

Please avoid:

simply summarizing the paper and adding a couple of questions about low-level details in the paper.
expressing an opinion without backing it up with specifics. For instance, if a method is novel, explain what aspect is novel and why this is interesting. If the method is not novel, explain why and provide a reference to prior work.
being rude. A good review is polite. Just like in a conversation, being rude is typically ineffective if one wants to be heard.
asking the authors to substantially expand their paper. The paper should be evaluated as submitted. The conference has no mechanism to ensure that any proposed changes would be carried out. Moreover, the authors are unlikely to have room to add any further derivations, plots, or text.

Your reviews will be published anonymously. However, please make sure they are written in a way that you would approve to appear under your name. Outstanding reviewers will be acknowledged and offered free registration to attend the MICCAI conference.

2. Specific Reviewing Notes

Historically, we have a very large number of papers in Medical Image Computing (MIC) but not so many dealing with Computer-Assisted Interventions (CAI). Additionally, we now have dedicated sessions for Clinical Translation and Health Equity. To ensure that we select an appropriate spectrum of papers in all categories and sessions, please keep the following points in mind while reviewing.

General review considerations: When reviewing all MICCAI papers, consider

whether the proposed methods are innovative or
whether the application is innovative.

In particular the following questions should be asked:

Is the topic of the paper clinically significant?
Do the authors clearly explain data collection, processing, and division methods?
Do the data appropriately represent the range and diversity of possible patients and disease manifestations?
Are the data labels (if applicable) of sufficient quality to support the claimed performance or analysis of the algorithms?
Do the authors report a sufficient number and type of performance measures to accurately represent strengths and weaknesses of the algorithms? Are performance measures reported with measures of uncertainty or confidence (e.g., error bars, standard deviations, etc.)?
Are the results and comparison with prior art placed in the context of a clinical application in terms of significance and contribution? Have the authors performed a proper statistical analysis of results (e.g. p-values)?
Does the work make a substantial contribution to the field or the society, or is it mostly incremental over previous work?
Do the authors discuss limitations and other implications of their methods and directions for future research?

Specific considerations should be given to the following categories of submissions:

CAI-based papers: We encourage submissions of papers relating to the implementation of, and training for, Computer-Assisted Intervention approaches. In particular, we wish to highlight the use of Medical Image Computing techniques that have become integral components of Computer-Assisted Intervention. We encourage technologies, such as point-of-care imaging, that are suitable to make healthcare more accessible. Specific considerations for your review of CAI papers should include but are not limited to:

Presentation of a device or technology that has potential clinical significance.
Demonstration of clinical feasibility, even on a single subject/animal/phantom.
Demonstration of robust system integration and validation.
Novel MIC approach to solving an unmet CAI need.
Proposal of a cost-effective (frugal technology) approach to implementing an otherwise expensive CAI solution.
Description of a system or device that is robustly validated against appropriate performance metrics.
Human factors evaluation of CAI systems.

Clinical Translation papers: This session will emphasize the move of MIC and CAI research from theory to practice by reflecting on the real-world challenges and potential impact for translating MIC and CAI methodology into clinical workflows and clinical evaluations. The philosophy of this dedicated session is to keep a high standard for methodology development while enabling a strong focus on the clinical application. Specific considerations for your review of Clinical Translation papers should include but are not limited to:

Barriers and challenges in translation, and how to overcome these
Robustness and reliability evaluation of algorithms
Insights into usability of MIC methods and CAI systems
User interaction, adoption and acceptance
Performance monitoring and clinical deployment

Health Equity papers: This session will focus on geographical health equity and global health challenges. Hence, we welcome submissions that contribute new methods and applications that are attuned to diverse healthcare settings, in terms of data, infrastructure, resources, and costs, especially to address challenges in limited-resource settings. Ultimately, this session will showcase how innovation in MIC and CAI can bridge healthcare gaps and offer affordable and high-quality care to under-served populations worldwide. Specific considerations for your review of Health Equity papers include but are not limited to:

Addressing inequalities in the context of MIC and CAI
Assessing fairness of MIC methods and CAI systems
Approaches for mitigation of bias in data collection, curation and annotation
MIC and CAI solutions for remote and low-resource settings
Biomedical image computing for neglected diseases

3. Formal Rules

Confidentiality: You have the responsibility to protect the confidentiality of the ideas represented in the papers you review. MICCAI submissions are by their very nature not published documents. The work is considered new or proprietary by the authors. Authors are allowed to submit a novel research manuscript that has been archived for future dissemination (e.g., on the arXiv or BioRxiv platforms). Sometimes the submitted material is still considered confidential by the authors' employers. Sending a paper to MICCAI for review does not constitute a public disclosure. Therefore, it is required that you strictly follow the following recommendations:

Do not show the paper to anyone else who is not directly involved in assessing the paper If you request the help of your colleagues and students, they will also be subject to the same confidentiality.
The use of LLMs (such as ChatGPT) is allowed as a general-purpose writing assistance tool. Reviewers should understand that they take full responsibility for the contents of their reviews, including content generated by LLMs that could be construed as scientific misconduct or plainly false (e.g., incorrect summaries of the paper content). You may use an LLM to polish the wording of your review (e.g. to correct grammar) once you have written it. But you may not show a paper or any part of a paper to an LLM. The PCs interpret showing a paper to an LLM as a deliberate violation of confidentiality. You must vouch for, and be responsible for, the accuracy of your review.
Do not show any results, videos/images or any of the supplementary material to non-reviewers.
Do not use ideas from a paper that you review to develop new ones of your own before its publication.
After the review process, destroy all copies of papers and supplementary material associated with the submission.

Conflict of Interest: The blind reviewing process will help hide the authorship of papers. If you believe that you recognize the work or the author and feel it could present a conflict of interest, decline the review to the Area Chair and inform the Program Chairs. You have a conflict of interest if any of the following is true:

you belong to the same institution or have been at the same institution in the past three years,
you co-authored together in the past three years,
you hold or have applied for a grant together also in the past three years,
you currently collaborate or plan to collaborate,
you have a business partnership,
you are relatives or have a close personal relationship.

4. Anonymization Rules

MICCAI 2024 follows a double-blinded reviewing process, according to which anonymity should be preserved for both sides, i.e. reviewers and submitting authors.

Anonymity should be kept in mind during the paper submission, review, and the rebuttal process.

Ensuring anonymity: Papers violating the guidelines for anonymity may be rejected without further consideration. At the same time, reviews that reveal the reviewer's identity are likely to have lower impact in the PC's decision process. Please keep the following in mind during the reviewing process:

Authors are asked to preserve their anonymity during the reviewing process, including not listing their names, affiliations, websites and omitting acknowledgments. All this information will be included in the camera-ready and published version
Please see the Submission Guidelines for additional details on how authors have been instructed to act in order to preserve their anonymity
Reviewers also should keep their identity invisible to the authors at all times.
Reviewers should not ask authors to cite their papers unless it is essential (e.g., the author is expanding on the reviewer's previous work or is using their dataset); this is unprofessional and also compromises the reviewer's anonymity.
If you accidentally discover the identity of the authors of a paper, make every effort to treat the paper fairly. It is NOT acceptable to accept or reject a paper based on the prior bias a reviewer might have about its authors.
Please report any potential breach of the anonymization rules in your assigned reviews.

ArXiv papers: with the increase in popularity of publishing technical reports and arXiv papers, sometimes the reviewer may accidentally uncover the authors of a paper.

Reviewers should not attempt to identify authors based on arXiv submissions or other publicly available technical reports. If the reviewer accidentally uncovers the authors' identity via arXiv, they should not allow this information to influence their review.
ArXiv papers are not considered prior work since they have not been peer-reviewed. Therefore, citations to these papers are not required and reviewers should not penalize a paper that fails to cite an arXiv submission.

Thank you, in advance, for your efforts and contributions toward yet another successful MICCAI Conference,
MICCAI 2024 Program Chairs