List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Jiayu Chen, Lei Mou, Shaodong Ma, Huazhu Fu, Lijun Guo, Yalin Zheng, Jiong Zhang, Yitian Zhao
Abstract
The segmentation of corneal nerves in corneal confocal microscopy (CCM) is of great importance to the quantification of clinical parameters in the diagnosis of eye-related diseases and systematic diseases.
Existing works mainly use convolutional neural networks to improve the segmentation accuracy, while further improvement is needed to mitigate the nerve discontinuity and noise interference.
In this paper, we propose a novel corneal nerve segmentation network, named NerveFormer, to resolve the above-mentioned limitations. The proposed NerveFormer includes a Deformable and External Attention Module (DEAM), which exploits the Transformer-based Deformable Attention (TDA) and External Attention (TEA) mechanisms. TDA is introduced to explore the local internal nerve features in a single CCM, while TEA is proposed to model global external nerve features across different CCM
images. Specifically, to efficiently fuse the internal and external nerve features, TDA obtains the \textit{query} set required by TEA, thereby strengthening the characterization ability of TEA.
Therefore, the proposed model aggregates the learned features from both single-sample and cross-sample, allowing for better extraction of corneal nerve features across the whole dataset. Experimental results on two public CCM datasets show that our proposed method achieves state-of-the-art performance, especially in terms of segmentation continuity and noise discrimination.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_8
SharedIt: https://rdcu.be/cVRvs
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a method for segmenting corneal nerves in CCM images. The method consists of encoder (pre-trainied ResNet34), two attention modules, and a CNN-based decoder. Experiments show that method performs better than a number of other methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The problem is relevant, and the approach is sound.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The biggest weakness of the paper is a relatively poor presentation. In places the paper is unclear, which makes it less convincing. The introduction is very messy. First it introduces a number of studies on fibres - that’s fine, but many of those studies (also quite recent) are not put in relation to the proposed method, and are also not present in comparison. Later it mentions [11], which seems to be improvement of [10]. Then it goes on mentioning the weaknesses of [10]. For other methods mentioned in introduction is unclear why are there relevant – some are general segmentation methods (not especially developed for nerves), while some are specialized pipelines. Moreover, the figure 1 uses results of [10] and [2] for motivation, but the text says almost nothing about why those methods are relevant (apart from having weaknesses). In the figure 1, the yellow text should be placed on the left of each row – as it is now it seems that nerve discontinuity is only relevant for CS-Net. The similar comment applies for figure 2. In places, language is informal and unclear: “…Inspired by [21] we motivate our model too focus on…”, “…method presents better immunization against artifacts…”. The key contributions: DEAM with TDA and TEA should be motivated and explained more clearly. E.g. the first sentence explaining TDA is not explaining much – and it just keeps on. Figure 3 is a bit confusing. For nerve continuation the arrow points to a place where the proposed network does well, but in other places (top middle) the nerve is broken by the proposed method, while other methods do better. Also, if other methods are performing so poorly on Langerhans cells, are those method relevant for comparison. Acording to the table 1, the methods shown in figure 3 are not competing to the best place. Also, table 1 seem to contains only one method developed for CCM (another one for fiberous structures). Why it that?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The explanation gives an ide of implementation. Still, it would be difficult, if not impossible, to fully reproduce this work – there are many implementations choices not covered.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please address the weaknesses.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Whille not convincing, the works seems to improve the state of the art.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The paper presents a transformer based network for nerve fibre segmentation in CCM images. The proposed transformer based part contains an intra-image local spatial attention and inter-image attention.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The combination of internal local attention and external attention is novel. The proposed method outperformed other state-of-the-art methods on two public CCM datasets. Ablation study shows the effectiveness of the prosed two blocks.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The theoretical justification of the proposed method is weak. A lot of implementation detail is missing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Based on the description in the paper, it is difficult to reproduce the exact result due to missing details. No code is publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper presents a transformer based network for nerve fiber segmentation in CCM images. The transformer based part contains an intra-image local spatial attention and inter-image attention. The proposed method outperformed other state-of-the-art methods on two public CCM datasets. Ablation study shows the effectiveness of the prosed two blocks. The paper is generally well written, but could be improved by addressing the following comments. -“The TDA enables the proposed DEAM to learn more crucial information in a single CCM image”. How “crucial information” is learnt? Requires more intuitive explanation. -A_hqk and W_h in equation (1) was not explained. Explain how the sampling offset is determined. -Please explain Norm in equation (2). -What does “xN” mean in Fig. 2. N also represent the number of elements in page 5 line 4. Are they the same meaning? -Parameter setting, e.g. H in equation (2), N in Fig. 2. H is also the height of image. -Please perform statistical test when claiming one metho is better than the other in Table 1 and 2. -It would be interesting to see the visual result of Backbone+TEA and Backbone+TDA in figure 3. This helps in intuitively understanding the effects of each block. Pick one of CS-Net or TransUnet, or use two rows of examples instead of four, if requires more space.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall a sound method but justification of the proposed method is weak. A lot of missing implementation details, which is difficult to confirm if the comparison to other methods was fair (e.g. number of layers, number of parameters, if model was fully converged, etc.).
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
This paper presented a new transformer design for corneal nerve segmentation. The proposed method achieves state-of-the-art performance in two CCM dataset by learning features from single CCM image and common properties from multiple samples.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper points out a common issue in corneal nerve segmentation – the background artifacts (e.g., Langerhans cells).
- The proposed method combines the merits of two transformer designs (deformable DETR and external attention) and successfully applied it to alleviate the identified issue.
- The state-of-the-art performance is achieved on the two CCM datasets, and an ablation study is included to verify the effectiveness of each component.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The generalizability. I am not sure about the generalizability of the proposed method, as no description of the test setting is provided. Based on Ref.8, external attention relies on two learned memory units which are the representation of the training dataset. The proposed method seems to preserve one such unit.
However, such a dataset-level representation could be less representative if the pre-trained model is tested on a different dataset. Thus, it is important to know how the memory unit is used at the test time and how well the model can be generalized to different datasets.
-
Insufficient baselines. As the proposed method is a fusion of deformable DETR (Ref. 21) and external attention (Ref. 8), it may be a good idea to include these two methods in the baselines to verify the benefit of this fusion design.
-
The novelty of TDA is unclear. I cannot see any significant difference between TDA and the deformable attention module in Ref. 21. If this is a major contribution, I would recommend to highlight the difference and demonstrate the motivation and novelty.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The detailed network architecture is not included in the paper. But the authors commit to releasing the code in the reproducibility checklist.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper demonstrates superior performance on two training datasets. But I am not confident about its generalizability, and I believe the two source methods (deformable DETR and external attention), which the proposed approach inspired from, should be included for a more comprehensive evaluation. Plus, I wish the authors could illustrate the novelty of TDA if it is a major contribution.
I cannot recommend acceptance based on current manuscript. But if the authors can address all these three major concerns, I would be inclined to accept the paper.
Besides these three major concerns, here are two minor suggestions.
- Fig. 1 and Fig. 3 seem redundant to me. It may be better to merge them and provide more discussion and analysis in the saved space.
- For published papers (e.g. Ref. 21), it would be better to cite their published version instead of arXiv preprint, unless there is significant difference between these two versions.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The model generalizability;
- The imcomplete baselines;
- The novelty of TDA module.
- Number of papers in your stack
1
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The rebuttal partially addresses my concern about the generalizability. But the FDR increases by about 13.4% in this direct test. It indicates a fairly large increase of false positives. I highly recommend authors visualizing the results and check the location sof these false positives. If they are located somewhere like the Langerhans cell region, it would be a warning sign.
Plus, I still cannot see the novelty of TDA. It seems the authors simply adopt the deformable attention module from [21] with the same motivation proposed in [21], “enforce the entire framework to focus on a small set of important sampling points around the reference point.” If combining TDA and TEA to build DEAM is the main contribution, it may be better to highlight this point and provide the motivation for such a combination.
I am still on the fence about acceptance. But the merits slightly weigh over weakness, if the increasing false positive is not at the Langerhans cell region.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers commented on the novelty of several ideas proposed in the paper such as the new transformer design using a combination of internal local attention and external attention, learning features from single CCM image and common properties from multiple samples. The method showed superior performance on two public CCM datasets. Several weaknesses were also pointed out including a lot of implementation details missing, weak reproducibility, and unclear contribution/novelty of the TDA module.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
We thank the reviewers for taking time to review our paper and we appreciate their positive feedbacks to the effectiveness of our method (e.g., “ The problem is relevant, and the approach is sound.” by R1, “Ablation study shows the effectiveness of the proposed two blocks.” by R2 and R4.) and to our technical novelty (e.g., “The combination of internal local attention and external attention is novel.” by R2. Here we provide our point-to-point responses to address their concerns.
(2) A memory unit is essentially a learnable parameter that models the associations between different CCMs across the dataset during training. Once the best model is obtained, its parameters are fixed and we can directly segment new CCMs.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors submitted a strong rebuttal and clarified most reviewer questions. Reviewer 4 (previously recommending weak reject) has changed the rating to weak accept post rebuttal. The paper now has one accept and two weak accepts after rebuttal. The paper should be acceptable.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors propose a method for segmenting corneal nerves in CCM images. There were originally concerns about generalizability and novelty. The authors have partially addressed concerns and R3 has raised the score to ‘weakly accept’. I am glad to accept the paper for acceptance at MICCAI 2022.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a new transformer design for corneal nerve segmentation in CCM. images. The method consists of an encoder (pre-trainied ResNet34), two attention modules, and a CNN-based decoder. The combination of internal local attention and external attention is novel. Ablation study shows the effectiveness of the proposed method measured on two CCM dataset. The rebuttal answers most questions. Even if there are still points that are not clear, the paper deserves to be presented at MICCAI. My proposal is therefore “Acceptance”.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6