List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Zihang Xu, Haifan Gong, Xiang Wan, Haofeng Li
Abstract
Automatic tissue segmentation of fetal brain images is essential for the quantitative analysis of prenatal neurodevelopment. However, producing voxel-level annotations of fetal brain imaging is time-consuming and expensive. To reduce labeling costs, we propose a practical unsupervised domain adaptation (UDA) setting that adapts the segmentation labels of high-quality fetal brain atlases to unlabeled fetal brain MRI data from another domain. To address the task, we propose a new UDA framework based on Appearance and Structure Consistency, named ASC. We adapt the segmentation model to the appearances of different domains by constraining the consistency before and after a frequency-based image transformation, which is to swap the appearance between brain MRI data and atlases. Consider that even in the same domain, the fetal brain images of different gestational ages could have significant variations in the anatomical structures. To make the model adapt to the structural variations in the target domain, we further encourage prediction consistency under different structural perturbations. Extensive experiments on FeTA 2021 benchmark demonstrate the effectiveness of our ASC in comparison to registration-based, semi-supervised learning-based, and existing UDA-based methods.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_31
SharedIt: https://rdcu.be/dnwLM
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
This paper propose a novel unsupervised domain adaptation in fetal brain tissue segmentation; They design a apperance and structure consistency framework to address a practical UDA task adapting publicly available brain atlases to unlabeled fetal brain MR images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper proposes a new application to transfer the segmentation knowledge from the publicly available atlases to unlabeled fetal brain MRIs from new centers.
- For the significant variances in the shape of abnormal fetal brain tissue, this work design a new consistency to overcome this difficulty. This work is not only limited in domain style alignment.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The fourier transform applied in domain adaptation is not novel. The dual segmentaion consistency is common used in semi-supervised segmentaion. For the variance of abnormal fetal brain tissue, this framework incorporates the structure perturbation. However, the structure perturbation technique is the existed method. (Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: Cutmix: Regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 6023–6032 (2019)). How to generate images with structure perturbation is not clearly explained. The proposed framework is an integration of different methods.
- Due to the fetal development, the larger gestation week, the more complex the brain tissue. In experiment, the author only distinguish the results of normal and abnormal subjects. However, the results of different gestation weeks are ignored.
- For the training process, it is not stated whether the source-target data needs to be paired in the same gestation weeks.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code will be public. The experiments performed on public datasets.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1.It is necessary to clarify the difference with the FDA[24].
- Due to the fetal development, the larger gestation week, the more complex the brain tissue. However, this work ignored the variance of appearance from gestation ages.
- The gestational age of dataset should be added. 4.In the experiments, only ues dice to evaluate the segmentaiton performance. The statistically significant improvements should be stated in experiment results.
- How to generate images with structure perturbation is not clearly explained.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the proposed method.The proposed framework is an integration of different methods. As a image segmentation method, only the dsc as metric is to evaluate the performance.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
In rebuttal, the author clarify the difference between ASC and FDA methods, and providing more quantitative results on different GA groups. The content is convincing to me.
Review #3
- Please describe the contribution of the paper
This paper presents a UDA method that leverages the consistency of appearance and structure in multiple domains for fetal brain MRI segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The experiments are sufficient, and the results look sound.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some designs, i.e., the amplitude swapping in FFT and structure perturbation, are confusing.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of this paper is good because 1) the code and data will be available, and 2) the implementation details are clear.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
One of my main concerns is why the authors only swap the low-frequency regions in FFT, not the full amplitude map. Such design will only enable the appearance of the swapped images changed in low-frequency regions, not boundaries. This can be observed in the cortex boundary in X_sft in Fig.3.
The structure perturbation is confusing to me. Following the reference, the authors use CutMix as the perturbation strategy. But how it performed is not detailed in this paper. Please clarify.
It would be better to give some descriptions of the architecture of the teacher/student network.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall this paper is good, I recommend this paper be accepted after some minor issues are addressed.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Overall, the author’s rebuttal partially addressed my concerns. The difference regarding the brightness change between X_sft and Xs is too minor to be observed visually. Some quantitative analysis may be helpful.
Review #4
- Please describe the contribution of the paper
This paper proposes an unsupervised domain adaptation framework for fatal brain tissue segmentation consisting of appearance and structure consistency. Experiments on FeTA2021 dataset show the effectiveness of proposed method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The fetal brain segmentation in UDA setting is an important and practical problem.
-
The experiments are sufficient and comprehensive, including the comparison with SoTA methods and ablation studies.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
From my point of view, it is not obvious that swapping the low-level spectrum of two domains would obtain the domain-aligned images. As this is the essential assumption of this paper, I would suggest the authors add more explanations to this point.
-
Some expressions need be more accurate and rigour. For example, “traditional UDA employing GAN to synthetic style-transfer images hardly capture such domain shift. Thus, we align the low-level statistics based on Fourier transformation to narrow the distribution of the two domains”. It is weird to me, as the domain shift is hard to be captured by GAN but can be aligned by a simple Fourier transform?
-
Again, the apperance is aligned by frequency-based appearance transformation. But from the visualization in Fig.3, it seems X_sft does not look like X_t. Maybe this transformation is just a data augmentation strategy instead of a data alignment strategy?
-
There are many typos if I understand correctly. “the target domain image Xt and its aligned image X_sft” might be X_tfs? “Given the inputs of Xsft and Xsft of teacher and student models”?
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors would provide the code, and this paper utilizes the public dataset.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- More explanations to the basic motivation could be added.
- The expressions should be more accurate. Please refer to the weaknesses for more details.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The fundamental assumption is not convincing enough. The written could be improved.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
appearance transfer with including appearance and structure consistency constraints. They use the method to segment fetal brain MRI from the FeTa challenge only supervised by labels from independent fetal brain atlases (normal and pathological). The reviewers agree that the application of domain transfer for fetal brain MRI segmentation is important. The experiments are convincing and comprehensive. However, they expressed concern about the methodological contribution of this work. Some parts of the methodology are not well described (e.g., structure perturbation), and the motivation of only using the low-frequency for appearance transfer is unclear. In the rebuttal, the authors are expected to clarify their methodological contributions and motivate and justify the appearance transfer. Additionally, the authors should improve the description of their methods and add information about the data including the gestational age.
Author Feedback
Thank you for the insightful comments. Our responses are as below: R2: The difference between ASC and FDA. 1) FDA adopts appearance alignment from the source domain to the target domain, which encourages the model to extract semantic concepts that are robust to domain shifts. However, this idea cannot be directly applied to our target domain, the fetal brain MRI data without annotations. We are the first to introduce domain appearance consistency on the target data to overcome this challenge. 2) We find that appearance alignment may affect certain semantic information. To address this, we employ a form of dual consistency that directs the model to focus on invariant information between the two domain appearances. 3) The ASC addresses the challenge of variance in tissue structure in pathological subjects by employing a teacher-student model to achieve structural consistency.
R3&R4: 1) Why swap the low-level spectrum, 2) Fourier transform better than GAN? 3) X_sft does not look like X_t? 1) According to Fig. 3 and FDA[1], swapping the low-level spectrum between images can exchange their style/color/brightness while unchanging semantic content. As discussed in [1], swapping the higher spectrum introduces unwanted artifacts, so we align appearance by only swapping the low-level spectrum. 2) In the FDA paper[1], FDA has outperformed the GAN-based method CyCADA[2] on 2D semantic segmentation datasets. 3) Appearance modeled by low-level spectrum can mean brightness. In Fig 3, X_s is bright and X_t is dark. After swapping appearance, the brain part in X_sft becomes darker than X_s and has close brightness with X_t. Similarly, the brain parts of X_tfs and X_s have close brightness to indicate a similar appearance. [1] Fda: Fourier domain adaptation for semantic segmentation, CVPR 2020. [2] CyCADA: Cycle-Consistent Adversarial Domain Adaptation, ICML 2018.
R2&R3: The operation of structure perturbation. First, we use a 3D cuboid mask consisting of a single box that randomly covers 25-50% of the image area at a random position, to blend two input images, which are sampled from the same batch. Then we blend the teacher predictions for the input images to produce a pseudo label for the student prediction of the blended image. Such an operation changes the original structural information, reduces the overfitting risk, and increases the robustness of the model to adapt to different structural variations.
R2: The influence of gestation ages. Whether pair source-target data in the same gestation weeks? 1)To study the impact of gestational age, we follow FETA2021 to divide the test set into 2 age groups (20-28 weeks & 29-35 weeks). The differences in evaluation metrics are as below. The results show that higher gestational age performs better, and is similar to the analysis in [3]. It may be due to that the high-age fetus has a clearer brain structure. Our ASC obtains the best Dice (76.7 & 82.4) in both low- and high-age groups. 2)As shown in [3], gestational age plays a small role in the success of segmentation, so we do not pair the source-target data in the same gestation weeks. Methods 20-28 29-35 All Supervised (Dt) 80.7 84.9 82.0 W/o Adaptation 72.8 80.3 75.3 SCALE 64.8 76.9 68.7 FDA 74.3 81.3 76.7 OLVA 73.2 80.6 75.6 DSA 73.5 80.9 75.9 CUTMIX 74.0 80.8 76.2 ASE-NET 73.8 81.3 76.2 ASC (ours) 76.7 82.4 78.5 [3] Fetal Brain Tissue Annotation and Segmentation Challenge, MICCAI 2021.
R3: The architecture of the teacher/student network. Following the top-ranked method in the FeTA2021 competition, we use SegResNet as the backbone for the teacher/student model.
R4: Two typos describing our methods. The correct writing is “the target domain image X_t and its aligned image X_tfs” and “Given the inputs of X_t and X_tfs of teacher and student models”. We will correct them in the revision.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors addressed the main concerns raised by the reviewers, especially regarding the difference between ASC and FDA. I recommend acceptance.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors may not fully address the concerns raised by R#4.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors proposed a unsupervised domain adaptation method for fetal brain tissue segmentation. Reviewers raised some concerns on the method details and motivation of some modules. The provided rebuttal seems answered these questions.