List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Rui Sun, Huayu Mai, Naisong Luo, Tianzhu Zhang, Zhiwei Xiong, Feng Wu
Abstract
Existing methods for unsupervised domain adaptive mitochondrial segmentation perform feature alignment via adversarial learning, and achieve promising performance. However, these methods neglect to the differences in structure of long-range sections. Besides, they fail to utilize the context information to merge the appropriate pixels to construct part-level discriminator. To mitigate these limitations, we propose a Structure-decoupled Adaptive Part Alignment Network (SAPAN) including a structure decoupler and a part miner for robust mitochondrial segmentation. The proposed SAPAN model enjoys several merits. First, the structure decoupler is responsible for modeling long-range section variation in structure, and decouple it from features in pursuit of domain invariance. Second, part miner aims at absorbing the suitable pixels to aggregate diverse parts in an adaptive manner to construct part-level discriminator. Extensive experimental results on four challenging benchmarks demonstrate that our method performs favorably against state-of-the-art UDA methods.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_50
SharedIt: https://rdcu.be/dnwDY
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
Existing methods for unsupervised domain adaptive mitochondrial segmentation perform feature alignment via adversarial learning but neglect the differences in structure of long-range sections. This paper proposes a Structure-decoupled Adaptive Part Alignment Network which consists of a structure decoupler and a part miner for robust mitochondrial segmentation. Extensive experimental results on four challenging benchmarks demonstrate that the method performs favorably against state-of-the-art UDA methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
This paper proposes a new model SAPAN which consists of a structure decoupler and a part miner for robust mitochondrial segmentation. The structure decoupler models the long-range section variation in distribution and morphology and decouples the variation from features. A part miner as discriminator to discriminate the detailed local differences between the source and target domain.
-
Extensive experimental results on four challenging benchmarks demonstrate that the proposed method performs favorably against state-of-the-art UDA methods. Extra ablation studies are included to to analyze each component of the proposed SAPAN.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The proposed method is limited to EM datasets.
-
Too many typos. The authors should carefully proofread the paper. For example, “these methods neglect to the differences” -> “these methods neglect the differences”, “To further improved” -> “To further improve”, “which are utilizing to facilitate” -> “which are utilized to facilitate”
-
Missing proper citations. Are the presented Structure Decoupler and Part Miner original proposed in this paper? If not, please include proper citations.
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Poor. Code is not provided.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please the authors include proper citations in the paper, and need more proof reading to correct typos. It would be great if the authors can also discuss about the limitations.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is effective and comprehensively evaluated on four challenging benchmarks.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
In this work, the authors proposed a novel framework for domain adaptive mitochondria segmentation in EM images. The proposed method is constructed by structure decoupler and part discriminator modules. Extensive experiments on several UDA mitochondria segmentation benchmarks have indicated the effectiveness of the proposed method by outperforming several SOTA methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) The proposed modules are novel. Specifically, the structure decoupler modules facilitate domain-invariant feature learning by preserving the morphology information. The part-based discriminator has alleviated the noise during the feature adaption process.
2) The proposed method has outperformed various SOTA UDA segmentation methods on various cross-domain mitochondria segmentation settings.
3) The overall paper is clearly presented and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) Some important ablation studies are not included. For the PM-based discriminator, there lacks an ablation study on whether it can work better than the traditional domain discriminator for the feature maps.
2) Some details about the datasets are missing. Specifically, the main paper and the supplementary material do not indicate how many images are used for training, validation, and testing of the experiments.
3) Some results in Table 1 are missing. For the Oracle and No Adapt methods, their results under all mAP and MCC are not reported.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
All the datasets used in this work are public, and the source code is not provided for paper review.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please address my concerns in the ‘weakness’ section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although there remains some weakness in this work, the overall paper has several strengths, such as the novel methods, good presentation, and outstanding performance. Overall, my initial rating of this work is ‘6: accept’.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The authors’ rebuttal has addressed my concerns, so my final rating is: ‘6: accept’.
Review #3
- Please describe the contribution of the paper
This paper proposes the Structure-decoupled Adaptive Part Alignment Network (SAPAN) for mitochondrial segmentation in electron microscopy images. The proposed approach addresses the unsupervised domain adaptive (UDA) problem by aligning segmentation results from source and target domains through adversarial training. SAPAN incorporates a structure decoupler to model long-range section variation in structure and a part miner to adaptively aggregate diverse parts for accurate discrimination. Experimental results demonstrate that SAPAN outperforms state-of-the-art UDA methods on several challenging benchmarks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper proposes the Structure-decoupled Adaptive Part Alignment Network (SAPAN) for mitochondrial segmentation in electron microscopy images. The proposed approach addresses the unsupervised domain adaptive (UDA) problem by aligning segmentation results from source and target domains through adversarial training. SAPAN incorporates a structure decoupler to model long-range section variation in structure and a part miner to adaptively aggregate diverse parts for accurate discrimination. Experimental results demonstrate that SAPAN outperforms state-of-the-art UDA methods on several challenging benchmarks.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
However, there are some limitations to consider. The authors claim that eS and eT condense the structure information for the whole volume of the corresponding domain, but it is not clear how the structure and the content are decoupled or how the decoupled features help with the task. Additionally, it is unclear how the structure features are supervised. Furthermore, the part-aware prototype approach used in this paper is a widely used technique in computer vision tasks, and thus, the novelty of the approach is limited.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
OK
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This paper proposes the Structure-decoupled Adaptive Part Alignment Network (SAPAN) for mitochondrial segmentation in electron microscopy images. The proposed approach addresses the unsupervised domain adaptive (UDA) problem by aligning segmentation results from source and target domains through adversarial training. SAPAN incorporates a structure decoupler to model long-range section variation in structure and a part miner to adaptively aggregate diverse parts for accurate discrimination. Experimental results demonstrate that SAPAN outperforms state-of-the-art UDA methods on several challenging benchmarks. However, there are some limitations to consider. The authors claim that eS and eT condense the structure information for the whole volume of the corresponding domain, but it is not clear how the structure and the content are decoupled or how the decoupled features help with the task. Additionally, it is unclear how the structure features are supervised. Furthermore, the part-aware prototype approach used in this paper is a widely used technique in computer vision tasks, and thus, the novelty of the approach is limited.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See comments
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
No more question
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a domain adaptive mitochondria segmentation method. Overall, the paper can be followed and experimental results show some improvement. Strength: 1) The paper is well organised and easy to follow. 2) The results show some improvements. Weakness: 1) The rational of the proposed method is not clear. For example, why the structure decoupler in 3.2 can decouple the structures? 2) The novelty of the proposed method is not clear. Structure decouple has been proposed in computer vision. 3) There is a lack of support that the feature is indeed decoupled.
Author Feedback
We thank the reviewers for their constructive comments. The major concerns are replied to below.
Clarification of Novelty. (1) Structure Decoupler (SD). As the first work to strive to model structure differences in long-range sections neglected by previous methods, leading to hard-to-align features from different domains, we believe that SD has its own unique merits. A natural way is to leverage optical flow that characterizes the motion information to model the structure, considering the similarity between time series (video) and space series (EM volume). However, introducing an additional pre-trained network to estimate optical flow inevitably increases the complexity of the model and is often prone to noise in EM. We draw inspiration from TDN[19], which models feature differences as an alternative to optical flow, to model long-range sections variation in morphology (structure), which can be seen as part of the style leading to domain discrepancy. Then we seek to whiten the obtained domain-specific structure style in an AdaIN modulation in pursuit of features with easy-to-align properties. (2) Part Miner (PM). Prototype-based methods have been explored as an effective way in CV tasks, but most methods tend to generate prototypes in a fixed way through grid sampling or clustering. While our filters are learned through the whole dataset during training, endowing prototypes with ability to adaptively aggregate pixels to accurately discriminate local differences between source/target domains. Furthermore, for the first time, we encapsulate these prototypes into PM to construct a part-level discriminator instead of traditional pixel-level one. Therefore, we do NOT simply apply the concept of prototypes.
More support of SAPAN. To take a close look at effect of SD and PM, we evaluate the modularity of feature distribution, which measures the separability between clusters. Lower modularity means less discrepancy between clusters. We view the foreground (background) features from source/target domains as two clusters and calculate f./b. modularity for each feature distribution under different settings (T means pixel-level traditional discriminator): ~~~~T |SD+T| PM |SD+PM (Ours) f.~0.412|0.327|0.343|0.228 b. 0.375|0.301|0.332|0.230 We can observe that though PM discriminator can better align the distributions of source/target domain than traditional one, it suffers a bottleneck because long-range section variation is hard to align by adversarial training. Equipped with SD, the gap between different domains is further alleviated, consistent with Fig.3.
R1: Limited to EM. Reply: We adopt EM as a representative application scenario, mainly because the available datasets support a relatively comprehensive investigation, in terms of different species and acquisition devices. R1: Typos. Reply: Thanks, we will fix these. R1: Citation. Reply: Thanks for suggestion. We will add more citations. See Clarification of Novelty. Reply: In this paper, we assume only one source domain exists. But in real scenarios, gathering information from multiple source domains is possible for future work. R2: Lack of ablation on PM. Reply: As claimed in Sec.4.4, we use pixel-level (traditional) discriminator as our baseline. Thus 1st and 3rd row in Tab.3 shows that PM discriminator is better than traditional one. R2: Missing details about datasets and result in Tab.1. Reply: VNC III contains 20 sections of 1024x1024, used as source training set. Lucchi-Train and Lucchi-Test both contain 165 sections of 1024x768, used as target training and testing set alternately. MitoEM-R and MitoEM-H both contain 500 sections of 4096x4096, 400/100 of which are used as the training/testing set. Oracle~~97.5|92.9|92.3|86.8|99.1|94.2|93.7|88.8 NoAdapt 74.1|57.6|58.6|40.5|78.5|61.4|62.0|44.7 We will supplement these details. R3: How to supervise eS/eT? Reply: The structure features are learned implicitly through data-driven without supervision. See Clarification of Novelty.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal has addressed reviewer‘s concern.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In the rebuttal, the authors addressed most of the concerns raised by the reviewers, especially the novelty and the effectiveness of the proposed method are clarified and supported. All the reviewers agreed that the paper is acceptable. Therefore, I recommend accepting this paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After reading reviews and authors rebuttal, I feel that the authors have positively addressed most of the concerns raised during the reviewing period. In particular, the novelty of the structure decoupler, as well as additional requested experiments have strengthened the current submission. As a negative point, after reading the authors response and the corresponding section for the part miner, it remains unclear to me which is the contribution on this part. More concretely, authors argue that for the creation of prototypes, this is the first work to leverage the prototypes to do a part-wise comparison, instead of a pixel-wise one. Nevertheless, I did not understand what authors meant. Furthermore, I am aware of several works (for example in the context of few-shot segmentation) that employ superpixels to generate the prototypes, resulting in part-image prototypes. Having said this, I believe that this work has some merits, and I recommend its acceptance, encouraging the authors to clarify this point, as well as the rational of the proposed method (unanswered in the rebuttal), in the camera ready version.