This document provides detailed guidelines to reviewers for MICCAI 2023. We believe it is important to summarize what makes a good MICCAI review and some of the expectations from you as a reviewer. We also include the rules that MICCAI 2023 adopts for paper anonymization as part of its double-blind peer review process. Please read these guidelines as part of the overall MICCAI 2023 review process document.
Please be aware that reviews of accepted papers will be made public (without disclosing the reviewers' identity), together with author responses and Area Chair meta-reviews.
1. What Makes a Good Review
The role of a reviewer is to identify excellent papers that the MICCAI community must hear about, and tell the program committee which papers are exciting and could have a great impact on the field. A good review expresses an opinion about the paper and backs it up with details on strengths and weaknesses of the paper.
The components of the reviewing form are as follows:
- A quick summary of the paper, which can be as short as a few sentences. This part tells the PC what the major contributions are, what the authors did, how they did it, and what the results were. It also helps authors to verify that the reviewer understood their approach and interpretation of the results.
- The opinion of the reviewer about the major strengths of the paper. A reviewer should write about a novel formulation, demonstration of clinical feasibility, an original way to use data, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Provide details that justify your opinions. For instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The opinion of the reviewer about the major weaknesses of the paper. Provide a list of points that summarize your concerns about particular aspects of the paper. Provide details that justify your opinions. For instance, if a method is not novel, provide citations to prior work.
- The opinion of the reviewer about the clarity of presentation, paper organization and other stylistic aspects of the paper. It is important to know whether the paper is very clear and a pleasure to read, or whether it is hard to understand.
- Comment on the reproducibility of the paper. Where possible, we encourage authors to use open data or to make their data and code available for open access by other researchers. We understand that due to certain restrictions, some researchers are not able to release their proprietary dataset and code; therefore, a clear and detailed description of the algorithm, its parameters, and the dataset is highly valuable. Please provide comments about whether the paper provides sufficient details about the models/algorithms, datasets, and evaluation. Please take the authors' answers to the reproducibility checklist into account.
- Detailed constructive comments should be provided to help the authors to improve their paper or to expand it into a journal version. Comments should be backed up by detailed arguments. Minor problems, such as grammatical errors, typos, and other problems that can be easily fixed by carefully editing the text of the paper, should also be listed.
- Your recommendation whether to accept or reject the paper: Taking into account all points above, should this paper be presented at the conference? Is it an interesting contribution? Is it a significant advance for the field? Is the paper of sufficiently high clinical impact to outweigh a lower degree of methodological innovation? Please remember that a novel algorithm is only one of many ways to contribute. Other examples include (but are not limited to) a novel interventional system, an application of existing methods to a new problem, and new insights into existing methods. A paper would make a good contribution if you think that others in the community would want to know it. As a guide, note that MICCAI typically accepts around 30% of submissions.
- A justification of your recommendation. What were the major factors in making your assessment? How did you weigh the strengths and weaknesses? Make sure that the reasons for your overall recommendation to accept or reject are clear to the program committee and the authors.
- Ranking of this paper in your review stack: This information will be taken to calibrate the overall rating. Please try your best to avoid ties.
- The expertise of the reviewer. If your expertise is limited to a particular aspect of the paper, this should be brought to the attention of the AC. The review is more likely to be taken seriously if the limitations of the reviewer's understanding are clearly acknowledged.
Please avoid:
- simply summarizing the paper and adding a couple of questions about low-level details in the paper.
- expressing an opinion without backing it up with specifics. For instance, if a method is novel, explain what aspect is novel and why this is interesting. If the method is not novel, explain why and provide a reference to prior work.
- being rude. A good review is polite. Just like in a conversation, being rude is typically ineffective if one wants to be heard.
- asking the authors to substantially expand their paper. The paper should be evaluated as submitted. The conference has no mechanism to ensure that any proposed changes would be carried out. Moreover, the authors are unlikely to have room to add any further derivations, plots, or text.
Before submitting a finished report, a wise referee asks, "Would I be embarrassed if this were to appear in print with my name on it?”
2. Specific Reviewing Notes
Historically, we have a very large number of papers in Medical Image Computing (MIC) but not so many dealing with Computer-Assisted Interventions (CAI). To ensure that we select an appropriate spectrum of both MIC and CAI papers in the mix, please keep the following points in mind while reviewing.
General review considerations: When reviewing all MICCAI papers, consider
- whether the proposed methods are innovative or
- whether the application is innovative.
In particular the following questions should be asked:
- Is the topic of paper clinically significant?
- Do the authors clearly explain data collection, processing, and division methods?
- Do the data appropriately represent the range of possible patients and disease manifestations?
- Are the data labels (if applicable) of sufficient quality to support the claimed performance of the algorithms?
- Do the authors report a sufficient number and type of performance measures to accurately represent strengths and weaknesses of the algorithms? Are performance measures reported with confidence intervals?
- Are the results and comparison with prior art placed in the context of a clinical application in terms of significance and impact? Have they performed a proper statistical significance analysis of results?
- Does the work make a significant contribution to the field or the society, or is it just incremental over previous work?
- Do the authors discuss limitations of their methods and directions for future research?
CAI-based papers: We particularly encourage submissions of papers relating to the implementation of, and training for, Computer-Assisted Intervention approaches. In particular, we wish to highlight the use of Medical Image Computing techniques that have become integral components of Computer-Assisted Intervention. We encourage technologies, such as point-of-care imaging, that are suitable to make healthcare more accessible. Areas considered significant in a CAI paper include:
- Presentation of a device or technology that has potential clinical significance.
- Demonstration of clinical feasibility, even on a single subject/animal/phantom.
- Demonstration of robust system integration and phantom validation.
- Novel MIC approach to solving an unmet CAI need.
- Proposal of a cost-effective (frugal technology) approach to implementing an otherwise expensive CAI solution.
- Description of a system or device that is robustly validated against appropriate performance metrics
- Psycho-physical/Human factors evaluations of CAI systems
All papers should discuss limitations of proposed systems and provide a clear description of how the data used for the study were acquired.
3. Formal Rules
Confidentiality: You have the responsibility to protect the confidentiality of the ideas represented in the papers you review. MICCAI submissions are by their very nature not published documents. The work is considered new or proprietary by the authors. Authors are allowed to submit a novel research manuscript that has been archived for future dissemination (e.g., on the arXiv or BioRxiv platforms). Sometimes the submitted material is still considered confidential by the authors' employers. Sending a paper to MICCAI for review does not constitute a public disclosure. Therefore, it is required that you strictly follow the following recommendations:
- Do not show the paper to anyone else including colleagues or students. These colleagues and students will also be subject to the same confidentiality.
- Do not show any results, videos/images or any of the supplementary material to non-reviewers.
- Do not use ideas from a paper that you review to develop new ones of your own before its publication.
- After the review process, destroy all copies of papers and supplementary material associated with the submission.
Conflict of Interest: The blind reviewing process will help hide the authorship of papers. If you recognize the work or the author and feel it could present a conflict of interest, decline the review to the Area Chair and inform the Program Chairs. You have a conflict of interest if any of the following is true:
- you belong to the same institution or have been at the same institution in the past five years,
- you co-authored together in the past five years,
- you hold or have applied for a grant together also in the past five years,
- you currently collaborate or plan to collaborate,
- you have a business partnership,
- you are relatives or have a close personal relationship.
4. Anonymization Rules
MICCAI 2023 follows a double-blinded reviewing process, according to which anonymity should be preserved for both sides, i.e. reviewers and submitting authors. Anonymity should be kept in mind, during the paper submission, review, and the rebuttal process.
Ensuring anonymity: Papers violating the guidelines for anonymity may be rejected without further consideration. At the same time, reviews that reveal the reviewer's identity are likely to have lower impact in the PC's decision process. Please keep the following in mind during the reviewing process:
- Authors are asked to take reasonable efforts to preserve their anonymity during the reviewing process, including not listing their names, affiliations, websites and omitting acknowledgments. All this information will be included in the camera-ready and published version.
- Please see the Author Guidelines for additional details on how authors have been instructed to act in order to preserve their anonymity
- Reviewers also should make all efforts to keep their identity invisible to the authors.
- Reviewers should not ask authors to cite their papers unless it is essential (e.g., the author is expanding on the reviewer's previous work or is using their dataset); this is unprofessional and also compromises the reviewer's anonymity.
- If you accidentally discover the identity of the authors of a paper, make every effort to treat the paper fairly. It is NOT acceptable to accept or reject a paper based on the prior bias a reviewer might have about its authors.
- Please report any potential breach of the anonymization rules in your assigned reviews.
ArXiv papers: with the increase in popularity of publishing technical reports and arXiv papers, sometimes the reviewer may know the authors of a paper.
- Reviewers should not attempt to identify authors based on arXiv submissions or other publicly available technical reports. If the reviewer accidentally uncovers the authors' identity via arXiv, they should not allow this information to influence their review.
- ArXiv papers are not considered prior work since they have not been peer-reviewed. Therefore, citations to these papers are not required and reviewers should not penalize a paper that fails to cite an arXiv submission.
Thank you, in advance, for your efforts and contributions toward yet another successful MICCAI Conference,
MICCAI 2023 Program Chairs
