List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Xin You, Ming Ding, Minghui Zhang, Yangqian Wu, Yi Yu, Yun Gu, Jie Yang
Abstract
Atrial fibrillation (AF) is one of the most common types of cardiac
arrhythmia, which is closely relevant to anatomical structures including the left atrium (LA) and the left atrial appendage (LAA). Thus, a thorough understanding of the LA and LAA is essential for the AF treatment. In this paper, we have modeled relative relations between the LA and LAA via deep segmentation networks for the first time, and introduce a new LA & LAA CT dataset. To deal with uncertain boundaries between the LA and LAA, we propose the semantic difference module (SDM) based on diffusion theory to refine features with enhanced boundary information. Besides, disconnections between the LA and LAA are frequently observed in the segmentation results due to uncertain boundaries of the LAA region and CT imaging noise. To address this issue, we devise another connectivity-refined network with the connectivity loss. The loss function exerts a distance regularization on coarse predictions from the first-stage network. Experiments demonstrate that our proposed model can achieve state-of-the-art segmentation performance compared with classic convolutional-neural-networks (CNNs) and recent Transformer-based models on this new dataset. Specifically, SDM can also outperform existing methods on refining uncertain boundaries. Codes are available at https://github.com/AlexYouXin/LA-LAA-segmentation.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_12
SharedIt: https://rdcu.be/dnwLm
Link to the code repository
https://github.com/AlexYouXin/LA-LAA-segmentation
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
Left atrium & appendage segmentation from Cardiac CT, which explicitly addresses (1) boundary ambiguities between the two structures and (2) enforce connectivity between them with smart models and dedicated training strategies and losses.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- very rigorous evaluation including (a) ablation studies for each of the components and subcomponents and (b) comparison to competing methods (and even in combination with)
- detailed mathematical & algorithmic description of the different components
- relevant technical & clinical problem
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I don’t really see any major weaknesses, overall very sound and well written. Certainly there are some smaller areas to optimize (see detailed comments)
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
anonymized sources to reproduce & details on the choice of hyperparameters in the supplementary material.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1) while the paper is very sound and well written overall, and the components make sense to me, it is still a bit difficult to get an actual intuition why the diffusion & SDM component actually improves performance in boundary definition between LA & LAA. I realize section 2.1 last paragraph is actually trying to do that, maybe this could be explain in a more layman language or with an extra figure
2) according section 2.1 paragraph 3 the diffusion process smoothes along edges, but not across, which sounds very much like a bilateral filter. it would be great to briefly explain in one sentence why (if?) a (simpler?) bilateral filter may not be enough
3) in Fig 3 LA & LAA seem to be confused.
4) Please explain why the network cannot be trained end-to-end with the connectivity loss. Would it be possible in future research to fuse the SDM & CRN?
5) the outlier suppression in Fig 1 in the supplementary material actually seems quite complicated. Why not just do a connected components analysis (take the largest, discard the rest) in the style of https://stackoverflow.com/questions/53576830/how-to-find-the-largest-connected-region-using-scipy-ndimage#53578754 using scipy?
6) The paper claims that a dedicated dataset has been created, will this be shared with the community after acceptance / publication?
7) And just a suggestion beyond the scope of this paper (maybe for a journal publication): if possible try to get a couple of annotations from multiple persons to compare automation performance with inter-user variability - particularly with ambiguous structures, this is basically the gold standard
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper addresses very relevant technical problems (esp when number of volumes is limited), which help to improve segmentation results and positively impact clinical workflows and acceptance of automation.
Overall high quality contributions, well organized and rigorously evaluated. Great work!
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper proposed a solution to segment uncertain boundaries by introducing a semantic difference module based on diffusion theories. In addition, the method incorporates a connectivity loss to promote connectivity between different labels.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Very interesting idea of diffusion inspired module
- Thorough evaluation with latest segmentation methods
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
I’m not sure if the proposed connectivity loss is working at all. By looking at the implementation, how does the gradient backpropagate through the torch.nonzero and the torch.argmax operator given both are non-differentiable?
-
It’s not clear to me how the diffusion process helps identify the ambiguous boundary when there is no edge feature
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Code is given which make the experiments easy to reproduce
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
What is the data stats, e.g. the spatial size, pixel spacing, vendor for the curated data?
-
Statistical analysis is required given the performance is so similar.
-
One relevant paper is missing in the introduction section: Jin, Cheng, et al. “Left atrial appendage segmentation using fully convolutional neural networks and modified three-dimensional conditional random fields.” IEEE journal of biomedical and health informatics 22.6 (2018): 1906-1916.
-
Since the boundary of LAA and LA is ambiguous, please list the annotation SOP used to ensure consistent annotation across annotators.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I feel the implementation is problematic thus render the conclusion meaningless
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
-
This paper is the first deep learning segmentation method to model the relative relationship between the left atrium (LA) and the left atrial appendage (LAA).
-
The paper introduces a semantic difference module (SDM) to improve the segmentation for uncertain boundaries between the LA and LAA. This SDM is based on diffusion theory.
-
A Connectivity-Refined Network (CRN) with connectivity loss is used in a second stage to improve the coarse predictions from the first stage.
-
The paper also introduces a new LA & LAA CT segmentation dataset.
-
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The LAA is a challenging anatomy for segmentation, given its proximity to the LA and lack of clear contrast between these two anatomical structures. As argued in the paper, identification of the LAA has important clinical applications so therefore progress on its accurate segmentation has value.
The semantic difference module based on diffusion theory appears to be novel. Leveraging diffusion theory, the paper introduces a novel way of processing features through learnable difference kernels to localize fuzzy boundaries.
A novel connectivity loss is introduced to train the second-stage network to improve the coarse predictions and help fill in undersegmented regions between the LAA and LA.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness of the paper is clarity. There are several points in the paper that are essential to its implementation, but not clearly described, rendering the paper hard to reproduce. First, it was unclear to this reviewer where the Deep Feature G in the SDM comes from and how it is constructed. It appears to be a feature map extracted in the bottleneck layer of the Stage 1 network. However, the paper nor the supplementary material describe this clearly. It was also unclear why this feature map is 2D, when the feature tensor at the bottleneck layer will have a 4D(?) shape (neglecting the batch dimension) therefore a description of its construction is missing. Relatedly, it’s unclear to this reviewer why using this map as a diffusivity term in Equation 2 helps the segmentation. As argued in the paper, the boundary between the LA and LAA is unclear, and it is unclear what the map is capturing, particularly looking at Figure 2c. It would be helpful to show a zoomed in figure highlighting the semantic guidance map and the location of the LA and LAA including the boundary to make it clearer how the guidance helps the segmentation. Finally, in the “Fuse” step in Figure 2c, what operation is performed (e.g. concatenation, addition)?
Some additional points lacking clarity:
-
Figures 1 and 3 state the LAA is green and the LA is red. However, since the LAA is a small anatomy compared to the LA, it would appear the figures are mislabelled. Please check for correctness.
-
The paper claims that the semantic difference module (SDM) refines features with enhanced boundary information; then why is it not used in the second stage of CRN?
-
The connectivity loss used to train the second-stage network is not implemented for the first-stage network; there is no clear intuition or ablation study behind this strategy. Relatedly, would it be possible to implement a single network that implemented both SDM and CRN? Such a design may have advantages in terms of computation or accuracy.
-
In Figure 2 could be improved for clarity. It is unclear what the different color arrows represent. The input to Figure 2b appears to be a joint segmentation map and the original image, but the segmentation map includes the image which does not appear to be correct.
-
In the learnable difference filters shown in Figure 2c, are the non-center values constrained to be positive? If not, it is possible these filters will not be difference filters. For example, if the filters are all negative the filter will be a blurring filter producing a negative output.
-
How are the patches cropped for the CRN? It’s unclear what size they are, the policy for cropping them, and how many are used.
-
- The paper claims UNeXt is a transformer-based model, but UNeXt does not feature Transformers. Instead, it is comprised of CNN and MLP components. For comparisons to Transformer-based models, comparisons to SwinUNetR or related methods may be preferred.
-
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The dataset introduced by the paper is not available to the research community to accelerate the research direction further. The code is available which can be implemented for other datasets.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
• In the title, please change “atrium” to “atrial” as the anatomy is known as the left atrial appendage.
• In Figure 1, the expert annotation for 3d (case 2) is unavailable; it would be preferrable to show this.
• The downward arrows at each stage at the encoder side are missing in Figure 2(a) and Figure 2(b).
• The batch size needs to be included while describing the implementation details.
• The channel calibration (CC) is missing in Figure 2, although it’s described as a key component in the model description and ablation study.
There are many small grammatical issues, please have the paper proofread carefully. Some of these include: • Abstract, please change “the diffusion theory” to “diffusion theory”
• Page 1, please change “cardiovascular diseases, which” to “cardiovascular disease and”
• Page 1, please change “Left atrial appendage closure” to “left atrial appendage closure”
• Page 2, please change “Computer Tomography” to “Computed Tomography”
• Page 2, please change “many works have” to “work has” as work is uncountable or consider changing to “the literature has”
• Page 2, please change “structure of LAA” to “structure of the LAA”
• Page 2, please change “between LA” to “between the LA”
• Page 2, please change “researches” to “researchers”
• Page 2, please change “emphasize on” to “emphasize”
• Page 2, please change “which is widely used in computer vision” to “and is widely used in computer vision”
• Page 3, please change “diffusion theories” to “diffusion theory”
• Below equations, like Eq 1 and 3, please continue the paragraph without indenting
• Page 3, please change “boundaries of LAA” to “boundaries of the LAA”
• Page 4, please change “neighborhood centering” to “neighborhood centered”
• Page 4, please change “are introduced to generate” to “is introduced to generate”
• Page 4, please change “fusing original” to “fusing the original”
• Page 4, please change “dimension” to “dimensions”
• Throughout the paper, please captialize the “D” in “Dice” as it is a person’s name
• Throughout the paper, please captialize the “H” in “Hausdorff” as it is a person’s name
• Page 5, please change “region of LAA” to “region of the LAA”
• Page 5, please change “neighbored with” to “neighboring with”
• Page 5, the symbol “S” is used to represent two things. First, above Eq. 5 it represents a surface, in and then below Eq. 5 it represents a scaling coefficient. Please use different symbols for clarity.
• Page 6, please change “Experiment Settings” to “Experimental Settings”
• Page 7, please change “Experiment Results” to “Experimental Results”
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, this paper has a lot going for it. There appears to be novelty with both the SDM and the CRN components, and the results show the method has advantages to other methods. The CRN in particular seems helpful to fill in missing regions between the surfaces. However, the paper has some detriments as well, primarily a lack of clarity and missing justifications for choices made in the implementation. The experimental results may be improved with comparisons to other approaches that have enhanced skip connections and additional ablation studies would be helpful.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The rebuttal has provided some answers to my questions, which is helpful. Overall I think this paper is stil lacking in clarity and requires some improvements to be at a publishable standard. However I’m ok to accept given the novelty of the paper as it has some interesting ideas.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Summary – The paper proposes a new semantic based approach for uncertainty based segmentation of the left atrial appendage on CT scans
Strengths – Addresses an important and challenging problem. The results are fairly impressive
Weaknesses – Lack of clarity with regard to methodology pointed out by all reviewers. Some reviewers do not understand the cost function and how it could actually be used for boundary detection. R3 calls out a number of typos in the paper.
Author Feedback
Q1-Dataset(R1&R2): The dataset contains 80 CT cases acquired by Siemens SOMATOM Force, and each volume consists of 256~528 slices of 512x512 pixels, with a voxel space of 0.45x0.49x0.49mm3. And we will disclose it after publication. For the annotation consistency, two clinicians individually annotated 40 CT cases, then one senior expert with over 20 years of experience corrected those annotations, especially the delineation of uncertain boundaries between the LA and LAA. Q2-The semantic guidance in SDM(R2&R3): 1) We detailedly explain why the semantic guidance helps to localize uncertain boundaries. As shown in Fig 2a, the deep feature G in SDM refers to the feature from the precedent decoder layer. And the ground truth in 2a corresponds to features in Fig 2c. Here the diffusion process will drive the original differential feature to focus on boundaries of anatomies as shown in 2c, including the RA, RV etc. However, predicted boundaries between the LA and LAA are not accurate enough only with the diffusion process. After introducing the semantic guidance, the refined boundary feature highlights boundaries between the LA and LAA, and suppresses the activation on other boundary regions. 2) Specifically, the semantic guidance has no response to patches without ambiguous boundaries, which will suppress activations on all boundary regions. 3) Due to the limited space, we do not visualize the semantic guidance map, and we will add that in the revised version. 4) The feature map is 4D while the visualization is 2D for simplicity. 5) Besides, qualitative results in 2c tell that K_o generates features with rich boundary information, indicating that not all values in the filters are negative. Q3-The differentiable connectivity loss(R2): We feel sorry to bring you such confusion for the reason that we uploaded invalid codes by mistake. And we have updated them at https://anonymous.4open.science/status/LA-LAA-segmentation-A158. Firstly, please refer to Appendix for the detection of boundary boxes. Then this loss indeed supports the gradient backpropagation, in which torch.argmax(TA) and torch.nonzero(TN) are replaced by other operations. Specifically, the index of the farthest point from the centroid can be calculated by TA, then it is reused as the tensor index, which is differentiable. TN is used to get all indexes of the LA and LAA regions. Here we introduce a tensor(3xHxWxL) to record indexes, then multiply it with the attention mask, which is also differentiable. Q4-Patch cropping for CRN(R3): We collected 50 predictions of the validation dataset(Stage 1) generated by various models in Table 1, in which there are 30 predicted masks with disconnections between the LA and LAA. Then we randomly split them as 35/15 for training and validation according to proportions. The patch size is 160x160x192. For the cropping strategy, as we mentioned in the last paragraph of Page 5, we increase the sampling ratio of the whole LAA region compared with that in Stage 1. Q5-An end-to-end network(R1&R3): 1) Implementing a single model that fuses SDM and CRN is interesting. However, disconnections between the LA and LAA mainly result from imaging noise, and only a small ratio of CT scans has the characteristic. Thus, an end-to-end network will learn biased features due to data imbalance. Also, the loss will bring a large computational cost for training. Hence, we train a two-staged model. By collecting predictions of validation datasets in Stage 1, we can balance the ratio. Then, CRN serves as the postprocessing network with a tiny weight, and the proposed loss provides the connectivity prior. 2) For cases with disconnected predictions between two anatomies, the semantic guidance cannot well guide the diffusion process. Thus, SDM makes no difference when employed in CRN. Q6-Unclarity(R2&R3): Thank you for your detailed corrections, including the fuse step in Fig 2c, UNeXt as a MLP-based model and the missing citation. We will carefully revise all typos.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
One reviewer responded, while the reviewer has initially suggested a weak reject, post-rebuttal the reviewer has now gone to a weak accept. This means that the paper has a strong accept and a weak accept and a weak reject suggesting that overall it passes the threshold for acceptance.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a novel method to model the relative relationship between the left atrium and the left atrial appendage. In the first-round of review, the reviewers anonymously provide positive reviews on this paper and raised some concerns. I think the rebuttal had addressed the concerns about the clarity of the method, and the use of the cost function. In my opinion, this method is interesting for the community to discuss during the MICCAI conference, and its technical merits weigh over weakness. For these reasons, the recommendation is toward acceptance.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This work proposes a novel diffusion-based segmentation method which provides uncertainty estimation. The reviewer’s comments about clarity of method have been addressed by the author’s feedback.