List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yanwu Xu, Shaoan Xie, Maxwell Reynolds, Matthew Ragoza, Mingming Gong, Kayhan Batmanghelich
Abstract
An organ segmentation method that can generalize to unseen contrasts and scanner settings can significantly reduce the need for retraining of deep learning models. Domain Generalization (DG) aims to achieve this goal. However, most DG methods for segmentation require training data from multiple domains during training. We propose a novel adversarial domain generalization method for organ segmentation trained on data from a \emph{single} domain. We synthesize the new domains via learning an adversarial domain synthesizer (ADS) and presume that the synthetic domains cover a large enough area of plausible distributions so that unseen domains can be interpolated from synthetic domains. We propose a mutual information regularizer to enforce the semantic consistency between images from the synthetic domains, which can be estimated by patch-level contrastive learning. We evaluate our method for various organ segmentation for unseen modalities, scanning protocols, and scanner sites.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_64
SharedIt: https://rdcu.be/cVRXC
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes synthesizing new (unseen) domains in an adversarial manner for domain generalization in medical image segmentation. An adversarial domain synthesizer (ADS) is developed to generate images with the hardest perturbations. Mutual information is calculated to preserve the semantic information in the synthesis image. The performance has been evaluated on various organ segmentation tasks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is nicely organized and easy to follow. The authors have done a good survey and analyzed the limitations of existing methods. The proposed adversarial synthesizing method with the guidance of mutual information regularization is relatively new in DG. The algorithm is clearly presented. Both quantitative and visualization results are reported to verify the effectiveness of the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The mixup ratio is not clearly introduced. Are there any strategies for collecting patch pairs from the source and the synthesized images? Experiments are only conducted for CT->MRI. How about the results of MRI->CT?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of this paper is relatively high because most implementation details, such as batch size, learning rate, and parameters of the optimizer, are provided. The authors also describe the details of data prepossessing and evaluation metrics.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The descriptions of ADS network architecture are some implementation detail, which is better written in the experiment section to increase the model’s flexibility.
- There is a gap between the image X and its bathes (X_p and X_n). More explanations are needed to clarify equation 2.
- The paper argues that the proposed method outperforms the meta-learning-based methods on computation cost. It would be better to give some theoretical or experimental evidence.
- As the proposed method includes an adversarial network, the authors should consider analyzing the convergence of the algorithm.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
As mentioned in the strength section, this paper is satisfactory in the aspects of novelty, algorithm design, and experiments.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The authors propose a single domain generalization method that can be trained on a single source domain while being generalizable to various novel testing domains. The proposed method is based on an adversarial domain synthesizer that plays a contradictory role as a segmentor, making the segmentor sufficiently generalizable after model convergence.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The concept of the adversarial domain synthesizer is intriguing and novel.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The authors make such a bold assumption that the real unseen domains are subsets of the collections of synthetic domains. However, the experiments are not sufficient to support such an assumption. Many key questions have not been fully addressed: 1) Are there any theoretical guarantees that support the above assumption? 2) Is there bias between the synthetic domains and the real unseen domains? For example, do the synthetic images show low noise level? 3) What is the segmentation performance on source domain where the segmentor was trained? How does the segmentation model perform after domain generalization compared with being trained on real target domains?
- I don’t think using an adversarial framework can avoid overfitting to a regular pattern, as claimed by the authors. It is almost practically impossible for a GAN model to converge to its global optima. In real application scenarios, it is very likely that the adversarial synthesizer only covers a subset of all possible “styles” in medical imaging.
- The experiments section is not well written. Specifically, Secs 4.3 and 4.4 are very short and Fig. 3 is very confusing. It is unclear for the general audience which subfigure corresponds to which experimental settings (CT/MR or multi-scanner).
- The paper lacks some insights about model design. For example, why the authors use two different $T$ networks but not one with different $z$ samples?
- The authors should talk about the limitations of their method. I believe the domain generalizability of the proposed method is not unlimited.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good. The authors listed the data source, model architecture, and promised publishing their code after acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The idea of using adversarial domain synthesizer for better generalizability is interesting. However, the authors should narrow down their focus and application scenarios (e.g., which imaging modalities, healthy or pathological data, etc) and show experimentally and (hopefully) theoretically that the method holds the water.
- Given that the objective is domain generalizability, showing cases that the method fails is particularly critical.
- The authors should present some examples of the synthetic images and explore how samples $z$ affect the “style” of the synthetic images.
- Figure 2 is inconsistent with the text. Fig. 2 left (blue boxes) shows negative examples are selected from $X$, while Fig. 2 middle shows negatives are selected from $\hat{X}_n$. Additionally, $f_q$ in the text is inconsistent with $f_n$.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the method itself shows some merit, the paper overall is not clearly written. The experiments being insufficient to support the hypothesis of the paper mutes the excitement.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
The paper proposes a strong assumption for single domain generalization that the distribution of unseen domains belong to the distribution of synthetic domains. Under this assumption, they design a new adversarial augmentation method for organ segmentation. The experimental results achieve the state-of-the-art and show the effectiveness under different settings. In general, the organization and overall motivation are clear.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The method is novel. This paper designs an adversarial domain synthesizer with a mutual information regularization, which can reduce the negative effects of adversarial learning and achieve high generalization performance. 2.An interesting use of contrastive loss. Using the patch-level contrastive loss as a surrogate for mutual information estimator, which can reduce computational cost and achieve similar results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.The reproducible findings are not clear from the current submission. The detailed information of how to use the random parameter z during training is not mentioned. 2.The motivation of using patch-level contrastive loss is a little bit weak, the contribution would be more clearly if the paper can compare several different mutual information estimators.
- The assumption is too strong and needs more explanation between the assumption and proposed method.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- clear for the experiment settings and network architecture.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I think overall the paper is good and interesting, but the detailed motivation of loss function and network architecture design need to be reported for providing decent insights.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The organization of the paper is good. Some technical novelty.
- Number of papers in your stack
1
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes an adversarial augmentation-based single domain generalization method for organ segmentation. The overall reviewers’ opinion about the proposed method is positive, and the reviewers were in agreement with the novelty of adversarial domain synthesizer. However, concerns were raised regarding lack of convergence analysis for the algorithm, experiments are not sufficient to support the assumption made, clarify on the limitations of the method. The authors should carefully address these concerns in their rebuttal.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Author Feedback
We thank the reviewers for their insightful feedback. To address concerns raised by all reviewers, we give a point-by-point response to the comments below. R#1 Convergence analysis Our method stems from adversarial training to learn models robust to adversarial attacks. It is different from the generative adversarial network (GAN). In the training of GANs, the positive data is sampled from the real data distribution, and the negative data is sampled from the fake data distribution. The generator will finally achieve an equilibrium with the discriminator via the min-max game when the real and fake distributions match. However, in adversarial training, the classifier is only trained on the perturbed data transformed by the transformer, which has no such equilibrium between the classifier and the transformer. What we usually do is find a proper constraint on the perturbation. Thus, convergence analysis is very trevious in adversarial training. R#1 The mixup ratio The mixup ratio is randomly sampled from the uniformed distribution U [0,1] for each training example. R#1 collecting patch pairs from the source and the synthesized images? We follow the same sampled strategy as CUT-GAN, which samples 256 random patches from feature maps of multiple layers. R#3 bold assumption, theoretical guarantee, bias between synthetic and real unseen domains, synthesizer covers a subset of real styles We have to emphasize that single domain generalization is an ill-posed problem, which is unsolvable without additional assumptions. It is also well known that the assumptions in any machine learning methods can never be perfectly met in real data. It is inevitable that the synthesized and unseen domains may have some gap, and that is why we proposed an advanced data augmentation method to reduce the gap as much as possible. The effectiveness of our method has been verified in the experiments. Proving theoretical guarantees is important but it is out of the scope of our paper. We believe our encouraging empirical results would attract more people to work on the theoretical aspects of single domain generalization. R#3 Adversarial framework cannot avoid overfitting to a regular pattern, GAN is impossible to converge to its global optima Please refer to R#1 Convergence analysis. R#3 Experiments is not detailed; Fig. 3 misses the explanation. The training settings of the experiments are detailed in Section 4, and we will release the code reference if we get to the final camera-ready step. We will add the detailed description of Fig.3, which shows two-segmented samples from target dataset of each experimental setting by rows.2 R#3 Two different T networks rather than two different z with one T. We agree that our model can also use two different z with one T, but the two choices would not differ in a noticeable gap in the aspects of training speed and computational cost. R#3 Limitation of our method. Our model is only applicable to image data and best fits medical images because the modules are specifically designed for medical images. R#4 Design of random variable z. We show the structure of z in Figure 1. (b), where z is first projected to multiplier and bias codes via a linear layer, and then the multiplier and bias codes will be multiplied and then added to the normalized feature maps as done in StyleGAN. R#4 Motivation for using patch-level contrastive loss is a little bit weak. Need comparison of other methods The motivation is that estimating the accurate mutual information between two images needs the explicit likelihood of image pixel-level distribution, which would lead to tremendous computational cost. Thus, inspired by the findings in contrastive learning, we use the patch-level contrastive loss as a surrogate estimator of mutual information. R#4 The assumption is too strong and needs more explanation of the assumption. Please refer to R#1 Convergence analysis.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The author addressed most concerns from the reviewers in the rebuttal. Recommend to accept and ask the authors to reflect the rebuttal points in the paper if finally accepted.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a new method for single domain generalization, aka. data augmentation strategy.
The idea of using an adversarial manner to learn augmentation parameter is new. The AC votes for accepting this paper. - After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper is well-written and the proposed method is well-motivated. The rebuttal addresses the major concerns raised by R3.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5