List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Dong Liang, Jun Liu, Kuanquan Wang, Gongning Luo, Wei Wang, Shuo Li
Abstract
The morphological changes in knee cartilage (especially femoral and tibial cartilages) are closely related to the progression of knee osteoarthritis, which is expressed by magnetic resonance (MR) images and assessed on the cartilage segmentation results. Thus, it is necessary to propose an effective automatic cartilage segmentation model for longitudinal research on osteoarthritis. In this research, to relieve the problem of inaccurate discontinuous segmentation caused by the limited receptive field in convolutional neural networks, we proposed a novel position-prior clustering-based self-attention module (PCAM). In PCAM, long-range dependency between each class center and feature point is captured by self-attention allowing contextual information re-allocated to strengthen the relative features and ensure the continuity of segmentation result. The clutsering-based method is used to estimate class centers, which fosters intra-class consistency and further improves the accuracy of segmentation results. The position-prior excludes the false positives from side-output and makes center estimation more precise. Sufficient experiments are conducted on OAI-ZIB dataset. The experimental results show that the segmentation performance of combination of segmentation network and PCAM obtains an evident improvement compared to original model, which proves the potential application of PCAM in medical segmentation tasks. The source code is publicly available from link: https://github.com/LeongDong/PCAMNet
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_19
SharedIt: https://rdcu.be/cVRyx
Link to the code repository
https://github.com/LeongDong/PCAMNet
Link to the dataset(s)
Reviews
Review #1
- Please describe the contribution of the paper
In this work, the authors propose a self-attention module based on a computational process on class centers, and they call it as the position-prior clustering-based self-attention module (PCAM). The PCAM is a plug-in module, which could be integrated into an up-sampling layer of the decoding half in a UNet/VNet-like network structure. From the experimental part, the proposed method could achieve the best overall results by comparing with other existing approaches.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
Based on some experimental results and some related work that this paper referred, I think the PCAM is a flexible plug-in module, and could be treated and utilized as a self-attention module to strengthen the relative features for the segmentation targets in certain layer. Although the authors only apply the PCAM for the knee cartilages segmentation problem, I think this module could be further used for other organ/tissue segmentation.
-
The reproducibility of the PCAM is not difficult.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The continuity of segmented results may negatively affect the diagnosis of knee cartilage diseases. For example, the cartilage defects in [2]. A higher continuity in the segmented outputs may fetch up or conceal the defect situations in cartilages, which may cause diagnostic errors. Thus, I don’t think the key contribution (i.e., higher continuity) that the proposed method claimed is completely matched to the medical scenario on knee cartilages.
-
The proposed module has a high similarity to the “Class Center” step in [10], which seriously reduces the novelty of this paper. And the author does not use lots of contents to do method-level comparisons with this highly relevant method.
-
Although the coarse-to-fine approaches ([3, 4]) and the ROI-fusion approach ([5]) did the knee cartilage segmentation, their goal is to increase the resolution of processed data while overcoming the limited GPU memory resources. Yet your goal is to propose a class-center based self-attention module and add it to a baseline model to increase some performance (e.g., the continuity). Adding your module will increase the memory cost of any baseline. Thus, comparing with these coarse-to-fine/ROI-fusion approaches may not directly demonstrate the superiority of your method.
-
3D visual comparisons could clearly show what your method improved in detail, yet I only saw some quantitative results.
-
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the PCAM is not difficult.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
I think the proposed method is a flexible module, and I think it should be applied for more organ extraction problems (including the “health” knee data).
-
The proposed method is an improved/modified self-attention idea, and the authors should focus on the comparisons to the self-attention or attention-based approaches. E.g., describing more about the improvement to [10] or its follow-up papers, or comparing with some different attention-based papers and their follow-ups (Attention U-Net: Learning Where to Look for the Pancreas).
-
Adding 3D visual comparisons for 3D segmentation.
-
I have a concern. The experimental data has a very large size in 3D, and the GPU has 11-GB memory, how did you implement your method on a 3D network with 4 down-sampling and 4 up-sampling layers, and also the batch size is 4?
-
I suggest that if you used the Erode operations, you should clearly say it, and avoid some vague implementation descriptions in the sub-section “Position-prior module”.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
If the authors emphasize that the proposed method is for a medical problem/scenario, then it must be shown that the method is highly matched to the problem (the methodology and experiment to support the method to match the problem).
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper proposes a position-prior clustering-based self-attention module (PCAM) for automatic knee cartilage segmentation in MR images: 1). lightweight PCAM that can be plugged in to networks 2). Application of clustering-based self-attention module on knee cartilage segmentation. 3). The proposed method outperforms State-of-The-Art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed position-prior and clustering-based attention module is technically novel.
- The paper is well-structured.
- Validation is thorough. Comparisons between the proposed method and previous methods are provided.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
N/A
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code is not provided and the reproducibility is questionable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Technical novel; Outperform the previous methods.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The paper present a self-attention module with clustering to improve the possible discontinuous segmentations of cartilage from knee MR images. Proposed module can be added to different upsampling layers of U-net type segmentation models.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
PCAM is a novel formulation that can be added to an available DL model to improve knee cartilage segmentations. The evaluation of the approach was done by comparing the approach with the current state-of-the art segmentation methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors included Ref. 9 in the paper for dilated convolutions. Adding an approach that uses dilated convolutions to the comparative study will improve the quality of the paper.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Please include a link to the code within the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The use of PCAM clearly improves the performance of the cartilage segmentations. PCAM includes 3 submodules, as a future study, is it possible to experiment the effect of individual submodules to the performance gain?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an add-on module that could be used to improve global connectivity of segmentations on different tissues. The approach is flexible to be incorporated into any encoder decoder -based segmentation models.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The majority of the reviewers listed the novelty of the position-prior clustering-based self-attention module (PCAM) which is used to improve the segmentation accuracy of the CNN. However, the reviewers also noted several weaknesses that need to be addressed. Therefore, I invite the authors for a rebuttal to answer these main points which are summarized below.
-Continuity of the segmentation outcome is not a good metric for knee cartilage segmentation (Rev 1). Depending on the knee disease (such as OA) there can be cartilage defects that would require the segmentation algorithm not to have the continuity information presented in the paper. This should be discussed.
-The authors should discuss the influence of different submodules which are part of the PCAM module (Rev3).
-Describe the improvements over [10]. What are the strengths of PCAM module over the prior self-attention models (Rev1)
-Implementation of the method in terms of memory consumption should be discussed (Rev 1)
-Looking at table 2 the results between nnU-Net and the nnU-Ne+PCAM module are very close. Statistical significance should be discussed (are the improvements significant).
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
Thank all reviewers for giving positive comments that the proposed method “outperforms State-of-The-Art methods”; “is technically novel”; “improves global connectivity of segmentation”; “lightweight”; “is flexible”; “achieves the best overall results”; “strengthens the relative features”. “The paper is well-structured”. “Validation is thorough”. Especially, R2 (strong accept) and R3 (accept) directly accept the paper.
R1: The proposed method would obtain discontinuous segmentation result where the knee cartilage is discontinuous rather than conceal the defective knee cartilage. Because adjacent points with different features are hardly assigned the same label, which ensures discontinuity of defective cartilage as shown in formulas (4) and (5).
R1: Compared with the method in [10], the main strengths of PCAM are: 1)More flexible. PCAM can be inserted into any up-sampling layer of the network flexibly and the segmentation result is obtained by only one-time inference. But the method in [10] is only applicable to the last layer of the network. It depends on the segmentation result, which is obtained by second-time inference. 2)More consistent with the real feature distribution. The similarity is calculated in PCAM to form a soft constraint for feature transference, which is more consistent to the feature distribution of knee cartilage. But in [10], the feature of the class center is estimated by the hard constraint of the predicted label map neglecting the similarity between feature point and class center. 3)Higher precision on class center estimation. The position-prior module in PCAM can effectively eliminate some false positives in predicted label map through the morphological operations to improve the precision of center estimation, as shown in Table 3. 4)More suitable for 3D knee cartilage segmentation. PCAM is designed for 3D knee cartilage segmentation but the method in [10] is only applicable to 2D task which is modified to 3D version in experiment for comparison.
R1: Compared with the coarse-to-fine method, our model improves segmentation accuracy with less computational burden in an end-to-end scheme. Although PCAM brings extra memory occupancy, it is endurable. In experiment, center/random cropping is adopted to reduce the size of input data from 384×384×160 to 256×256×32. For the largest feature map in Table 3, the memory occupancy of PCAM is 0.79GB compared with that of baseline model which is 9.71GB. Thus, models with batch size 4 are feasible in 11GB GPU.
R3: Thank you for future work’s suggestions. It is definitely possible to do those experiments. The three sub-modules have very positive influences: 1)The position-prior module reduces the false positives in predicted label map by morphological operations and improves the precision on estimating the feature of class center with the clustering-based module. 2)The clustering-based module reduces the computational burden and memory occupancy of affinity matrix by estimating class centers under the predicted label map. 3)The self-attention module improves the accuracy of segmentation by rectifying the predicted labels through the features transferring from class centers to feature point.
Meta reviewer: Compared with nnU-Net, the nnU-Net+PCAM obtains statistical significance on the accuracy of segmentation continuity which is evaluated by the 0-dimension Betti number (Persistent Homology-a Survey, 2005). We have done t-test between the methods with and without PCAM on segmentation continuity. We obtained p-value<0.005 on FC and p-value<0.01 on TC segmentation tasks, which indicates PCAM improves the segmentation continuity significantly.
All reviewers: Thanks for all reviewers’ suggestions on our “future study”. The experiment on dilated convolution and the performance gain of modules will be included in our future work. We had prepared 2D and 3D visual comparisons of segmentation results. All related data, graphs, and codes will be uploaded to Github later.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have provided a strong rebuttal and answered most of the questions raised by the reviewers. Camera-ready version should include the statistical test against nnUnet vs nnUnet+PCAM module, and the explanation that the method will handle discontinuous cartilage anatomy.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper received two accept recommendations and one reject. The reviewer who recommended reject also commented on some strengths of the paper such as flexibility and good performance. The rebuttal answered well questions raised by the meta reviewer and reviewers. Overall the paper proposes a flexible self-attention module with clustering and demonstrated that it can be plugged into different upsampling layer of U-net like models. When compared with state-of-the-art segmentation models, the proposed method shows clear improvement on the Knee Cartilage Segmentation task.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In this work, the authors propose a self-attention module based on a computational process on class centers, and they call it as the position-prior clustering-based self-attention module (PCAM).
Two of 3 reviewers are on (strong) accept. Third reviewer’s concerns are (1) bias towards healthy cartilage (2) similarity to [10] (3) memory consumption (4)lack of 3D visualisation
Rebuttal satisfactory address these concerns
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
11