List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Xiao Ma, Zetian Zhang, Zexuan Ji, Kun Huang, Na Su, Songtao Yuan, Qiang Chen
Abstract
Myopia is a manifestation of visual impairment caused by an excessively elongated eyeball. Image data is critical material for studying high myopia and pathological myopia. Measurements of spherical equivalent and axial length are the gold standards for identifying high myopia, but the available image data for matching them is scarce. In addition, the criteria for defining high myopia vary from study to study, and therefore the inclusion of samples in automated screening efforts requires an appropriate assessment of interpretability. In this work, we propose a model called adjustable robust transformer (ARTran) for high myopia screening of optical coherence tomography (OCT) data. Based on vision transformer, we propose anisotropic patch embedding (APE) to capture more discriminative features of high myopia. To make the model effective under variable screening conditions, we propose an adjustable class embedding (ACE) to replace the fixed class token, which changes the output to adapt to different conditions. Considering the confusion of the data at high myopia and low myopia threshold, we introduce the label noise learning strategy and propose a shifted subspace transition matrix (SST) to enhance the robustness of the model. Besides, combining the two structures proposed above, the model can provide evidence for uncertainty evaluation. The experimental results demonstrate the effectiveness and reliability of the proposed method. Code is available at: https://github.com/maxiao0234/ARTran.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_49
SharedIt: https://rdcu.be/dnwHu
Link to the code repository
https://github.com/maxiao0234/ARTran
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
The paper works on novel frameworks for high myopia screening in OCT images. The authors identify challenges (limited dataset, lack of generalized classification criteria, variations among clinical practices) in this task, and proposed several key components (APE, ACE, SST) in the ARTran workflow. Based on experiments and ablation studies, the authors show the proposed components could help solve task challenges.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
– clarity. The authors clearly show the symptoms, causes, and pathological developments of high myopia. They identify the challenges and discuss current research ideas/drawbacks. Their proposed method is clear and easy to understand.
– novelty. The authors apply the transformer architectures to trace long-range dependencies in the task. And they utilize Adjustable Class Embedding and Anisotropic Patch embedding to suit the task characteristics better. The formulation of shifted subspace transition matrix is detailed. – comprehensiveness. The authors carry out sufficient experiments with model variations and compare them with existing papers to draw the final conclusion.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
– problem formulation. The task has no definite forms, and the formulation must fully address the disease in aspects such as judgment, symptom identification, and information transformation. The paper simplifies the task as a binary classification problem, which only provides coarse information extraction. An ideal model should be able to tell the anatomical/geometric discoveries related to the disease. Binary classification may help in clinical settings, and the simplified formulation may never be adequate for clinical accuracy. – method explanation. In the method section, a large portion is devoted to ACE (Adjustable Class Encoding) and SST (Shifted Subspace Transition Matrix). But these components are more or less based on heuristics, without strict proof. In Fig 2, I could not figure out the meaning of the cones. Is there a better way to connect Class Posterior Range in Fig 2 with the text description in Section 2?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors work on a closed-source dataset.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The task formulation could be bolder. Binary classification can address some concerns in the clinical setting, but eye doctors may want more detailed geometric and semantic information related to diseases. A simplified formulation would actually constrain applicability and model generalization. As it has been mentioned in the paper, different clinicians would have different criteria, adjustable class may be a way to accompany this. However, machine models could extract more objective information and leave the judgment to the human operators.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Based on the clarity and extension of experiments, I believe the authors have done useful research and derived meaningful results. The proposed ARtran model and its components could be used to address the high myopia classification in the OCT task. There are minor weaknesses and suggestions, which indicate possible improvement directions.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposed a novel framework called adjustable robust transformer (ARTran) for high myopia diagnosis using optical coherence tomography (OCT) images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The model is suitable for variable screening encoding by using an adjustable class (ACE).
(2) The paper proposed shifted subspace transition (SST) to improve the model’s robustness.
(3) The experiment is sufficient and the results can demonstrate the effectiveness of the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) Please explain how to get equation 1.
(2) Equation 5 in the manuscript differs from Equation 4 in the supplementary material, as Equation 5 is missing a coefficient of 2 that is present in Equation 4.
(3) Figure 3 should be explained more clearly in the article.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code is released.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please see the weakness section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Based on the strength and weaknesses mentioned above.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This work aim to classify high myopia from macular OCT scans. The authors applied patch embedding to first extract features from the images, and an adaptive strategy was applied to classify the images with different threshold criteria.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Noisy labeling and inconsistent threshold definitions were considered.
Ablation and comparison studies. Uncertainty analysis. - Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The images included in the dataset were not completely independent of each other. Adjacent B-scans are partially overlapped in space. The shape of the retina in OCT images is relative. It is affected by anterior segment optics, imaging system. It is unclear whether the classification is highly dependent on the shape of the retina or some fine features in the retinal layers. 6 x 6 mm field of view seems to be too narrow to have a comprehensive assessment of myopia. Regions around the optic disc should also be included.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good reproducibility. The dataset is not public. The code is online.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Overall it is an interesting, well-written paper which targets a hot topic in ophthalmology. As mentioned in the main weaknesses session, there are some concerns. First, it is unclear whether the decision is mainly on the shape of the retina in the image, which is actually not reflecting the true ocular shape.
Second, the macular scan can could not provide comprehensive information and optic disc region should also be considered. Third, images are not totally independent from each other. Maybe sampling the scans more sparsely to avoid overlapping. Last, the pitch size in OCT image is anisotropic, so it is unfair to claim anisotropic patch embedding only based on the imbalanced pixel number. - Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The methodology is relative new but whether it is clinically interesting to classify myopia with a narrow OCT image is questionable.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
the concerns are fairly addressed.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes a novel deep learning framework for high myopia screening by applying modifications (APE, ACE, SST) to the ViT backbone and acheives the SOTA performance. I recommend a rebuttal so that authors may address reviewers’ concerns, especially the generalizability of the method. Ablation studies demonstrates the necessity of the design. There is a confusion point that the authors claim to adapt variant inclusion area in the introduction, but only one criteria is utilized for labeling according to dataset description. Considering reviewers’ opinion, I recommend a rebuttal. Authors should focus on the questions about dataset generation and method explanation.
Author Feedback
- Dataset generation (Meta-Reviews; Reviewer #4) – For the labeling criteria (Meta-Reviews), the variant inclusion area is determined when setting the hyperparameters of the benchmark of inclusion criteria and the adjustment coefficient. In Eq.1 of the manuscript, the variant inclusion area ranges from -8.5D to -3.5D. We have reported the experimental results in Fig 3(b), where the adjustment coefficients from -1.0 to 1.0 correspond the inclusion criteria of SE from -8.5D to-3.5D. – For the independent explanation of the dataset (R4), B-cans near the center contain a lot of myopia-related information which may be missed if sparsely sampling. Adjacent sampling can provide more evidence for decision making and uncertainty evaluation. Besides, we tried various sampling strategies (sparsely, fully, adjacently) in our early experiments, and chose this adjacent scheme because of the better performance. Experimental results will be supplemented in the ablations in the final version. – For the shape of OCT images (R4), we mitigate this problem from strategies of dataset construction and image augmentation, which increases the diversity of the training data. We considered the comprehensiveness of the dataset distribution, as shown in Fig 1 in the Appendix, we included data for each myopic interval. We considered the diversity of the shape within each myopic interval, as shown in Fig 1 and Fig 3 in the manuscript, with differences shapes of similar SE. We implemented data augmentation (Section 3.1) to increase shape diversity. We will supplement this description in Section 3 in the final version. – For the 6x6 field of view (R4), images with wider fields of view are more useful for decision making, but many existing meaningful myopia studies were focused on macular region. We mentioned in the introduction that screening for high myopia can help in the study of pathological myopia. Ohno-Matsui et al. proposed a grading system for atrophic pathological myopia in “International photographic classification and grading system for myopic maculopathy”. Ruiz-Medrano et al. proposed classification criteria for “ATN” of pathological myopia in “Myopic maculopathy: Current status and proposal for a new classification and grading system (ATN)”. Both studies are associated with myopic macular degeneration which primarily causes vision loss or even loss in people of working age. Moreover, our algorithm has good generalizability and can be used directly to wider OCT images and potentially obtain higher accuracy. We will supplement descriptions of the clinical interests in Introduction and Conclusion in the final version.
- Anisotropic patch embedding (R4). – For our APE, our “anisotropic” is proposed as a novel method to replace the vanilla “square” patch embedding of transformer, which is based on pixel level. Our motivation is to design a module that mitigates the information imbalance of OCT images (Section 2.1). The actual tissue size in our patch is also anisotropic (1.5x0.07mm). If this is still confusing to readers, we will change it to “non-square”.
- Method and formulation (Meta-Reviews; R2&R3) – For the ACE and SST, the references in Section 2.2 provide strict proofs and generalizations of the transfer matrix. However, the past methods are special cases of our approach (fixed as benchmark SST in Fig 2), which we extended to adjustable applications. The detailed proofs and the meaning of the cone (simplex) can be found in the Appendix. We will consider putting the symbols from the manuscript into Fig 2 for better description. – For the Eq. 1, this is our definition of the classification labels for this study when SE measurements and manually specified adjustment coefficients are included. In this study, we set -6D as the benchmark and plus or minus 2.5D as the adjustment range, so we get the adjustable labels in Eq.1. For the Eq.5, we made an error in writing the manuscript and the coefficients in Appendix and the code are correct.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Pros:
- the methodology is relatively new
- the topic is also interesting.
- The results demonstrate the effectiveness of the proposed methods. Cons:
- the generalizability of the method
- method explanation After Rebuttal:
- reviews are more consistant and positive;
- major issues are well explained
- no strong support is received
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed the concerns of R4, so that all reviewers voted for accepting the paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors proposed a novel approach for classifying high myopia from macular OCT scans. Reviewers‘s concerns on the details are well addressed.