Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Ziyuan Zhao, Fangcheng Zhou, Zeng Zeng, Cuntai Guan, S. Kevin Zhou

Abstract

Domain shift and label scarcity heavily limit deep learning applications to various medical image analysis tasks. Unsupervised domain adaptation (UDA) techniques have recently achieved promising cross-modality medical image segmentation by transferring knowledge from a label-rich source domain to an unlabeled target domain. However, it is also difficult to collect annotations from the source domain in many clinical applications, rendering most prior works suboptimal with the label-scarce source domain, particularly for few-shot scenarios, where only a few source labels are accessible. To achieve efficient few-shot cross-modality segmentation, we propose a novel transformation-consistent meta-hallucination framework, meta hallucinator, with the goal of learning to diversify data distributions and generate useful examples for enhancing cross-modality performance. In our framework, hallucination and segmentation models are jointly trained with the gradient-based meta-learning strategy to synthesize examples that lead to good segmentation performance on the target domain. To further facilitate data hallucination and cross-domain knowledge transfer, we develop a self-ensembling model with a hallucination-consistent property. Our meta-hallucinator can seamlessly collaborate with the meta-segmenter for learning to hallucinate with mutual benefits from a combined view of meta-learning and self-ensembling learning. Extensive studies on MM-WHS 2017 dataset for cross-modality cardiac segmentation demonstrate that our method performs favorably against various approaches by a lot in the few-shot UDA scenario.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_13

SharedIt: https://rdcu.be/cVRyr

Link to the code repository

https://github.com/jacobzhaoziyuan/Meta-Hallucinator

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a method for few-shot cross-modality domain adaptation, with the goal of training a model on a label-scare source domain and then adapting the model to an unlabeled target domain. The method leverages meta-learning, mean-teacher based semi-supervised learning, and image-and-spatial transformation. The effectiveness of method is validated on a popular cross-modality domain adaptation dataset for cardiac segmentation, showing improved performance over prior works on this challenging scenario.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • This paper tackles a challenging domain adaptation problem, for which the source domain only has a few labeled data and other data are unlabeled. A reasonable method is proposed and obtains improved and promising performance.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • The main weakness is that the methodology lacks clarity. The method consists of multiple components and needs to be trained in separate steps with different types of data. Without clearly knowing how each component works and how the data split, it is difficult to evaluate the method.

    • It is not known how the labeled source data, unlabeled source data, and unlabeled test data are sampled and split respectively in the meta-training and meta-testing stages. How to simulate the structural variances and distribution shifts? What types of data are inputted into the teacher model, student model, and the hallucinator respectively?

    • The effect of the hallucinator is confusing. In the method section, it is presented that the hallucinator is to produce “more meaningful target-like samples” x^{s\to t}. However, implementation details present that “Since limited labels are provided in the source domain, we transform target images to source-like images for training and testing”. The two descriptions are conflicting with each other.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
    • The training hyperparameters are clearly described, but some methodology details are missing.
  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    • The inputs to each network shown in Fig. 1 are unclear and the training process also lacks clarity. The authors could consider adding an algorithm box to better explain the pipeline.

    • Using only mean teacher (MT) significantly improves the performance from 14.0% to 49.9%. MT is a semi-supervised learning technique, but the result seems to show that MT benefits the cross-modality adaptation to a large extent. Could the authors provide more explanations on this result?

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The methodology lack clarity, thereby it is difficult to determine how each module of the proposed method works to achieve adaptation under the challenging scenario.

  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposed a novel method of learning cardiac image segmentation from few-shot cross-modality dataset. The core idea is using meta-learning to train a transformation-consistent meta-hallucination framework for unsupervised domain adaptation with a few labels.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper is well written.
    2. The idea of meta-hallucination framework is novel. Unsupervised domain adaptation is a useful technique for segmentation problems where labels are sacrce. Combining meta-learning and semi-supervised augmentation might be a good attempt.
    3. The experiment results are outstanding and convincing. The performance seems to be much better than other few-shot methods, and comparable with the fully supervised method.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Since the main contribution of this work is unsupervised domain adaptation, it is expected to see experiment results on more datasets of different types and modalities. Currently, there is only results on heart images provided.
    2. The ablation study discusses only meta-hal and meta-seg. More detailed analysis could help to understand the proposed method better, e.g., how the choice of support images will affect the performance, what the hallucinator learned through meta-learning.
  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Authors agreed to release the code.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    What is the difference between generators in GAN and hallucinator in this paper?

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Novel method for an important medical imaging problem.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    2

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    To achieve efficient few-shot crossmodality segmentation, this work proposes a novel transformation-consistent meta-hallucination framework, meta-hallucinator, with the goal of learning to diversify data distributions and generate useful examples for enhancing cross-modality performance.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The manuscript is well-written and easy to understand.
    2. The idea to achieve efficient few-shot cross-modality segmentation with limited labeled source data is interesting.
    3. The experiment is sufficient to prove the effectiveness of the proposed method.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The author should consider adding another experiment of few-shot learning (FSL) method, e.g. SSL-ALPNet[1], in target domain to show the upper bound of FSL;
    • Some minor errors:
      • The ‘leverage’ in the second line of page 3 should be ‘leverages’.
    • The proposed method adpots CycleGAN to achieve unpaired image translation for image adaptation, which is compution intensive.
      [1] Ouyang, Cheng, et al. “Self-supervision with superpixels: Training few-shot medical image segmentation without annotation.” European Conference on Computer Vision. Springer, Cham, 2020.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
    • The dataset is public available.
    • The authors promise to release the codes.
  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
    • More badly case analysis
    • More experiments on different types of distributions shift.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    • the idea is simple and straightforward.
    • the sufficient experiment to prove the effectiveness of proposed method
  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    Despite this work has received mixed scores, most comments are mainly minor. Indeed, R2 and R3 highlight the novelty of the proposed approach, the satisfactory results obtained by the proposed method and the clarity of the paper. Nevertheless, I strongly recommend the authors to address several concerns raised by the reviewers. These include: unclear points raised by R1 (splits for data and sampling strategies), and reasoning behind the significant improvement of MT model (R1). While R1 and R3 stress the need of adding more experiments in additional data distributional shifts, I consider that the current experiments are enough for a conference version, and these could be performed in a journal extension.

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    1



Author Feedback

We sincerely thank the reviewers and the meta-reviewer for their constructive comments. We are encouraged that reviewers find that few-shot UDA is a challenging task (R1) and our idea is reasonable (R1), novel (R2, R3), and interesting (R3). We are glad they find our approach to be a good attempt (R2), evaluated with sufficient experiments (R3), achieving outstanding and convincing results (R2). Below are our response to the main concerns. Q1: Lack of clarity on methodology. (R1) We will add more descriptions of the methodology in the final version. Moreover, an algorithm box or data flowchart will be added in our further work. Q2: How to simulate the structural variances and distribution shifts? (R1) For meta-learning, different pairs can be formed to simulate the transformations between source and target domains, thereby improving model generalization against structural variances and distribution shifts. Q3: What types of data are inputted into the teacher model, student model, and the hallucinator respectively? (R1) Labeled data are inputted into the student, while both labeled and unlabelled data are inputted into the teacher. Pairs of source and target domains are inputted into the hallucinator. Q4. The effect of the hallucinator is confusing. (R1) The hallucinator is to learn the image-and-spatial transformations between different domains for addressing domain shift and label scarcity. Image transformation and spatial transform can be performed separately. By transferring target images to the source domain, we can better observe the segmentation performance in the source domain. Q5: The explanation of the results of MT models. (R1) For SSL and Aug methods, we first implemented image-level adaptation (CycleGAN) to close the domain gap. Therefore, MT improves the performance from 40 % to 49.9% instead. We will add more implementation details of other methods in the final version. Q6: What is the difference between generators in GAN and hallucinator in the paper? (R2) In GAN, the generator focus on style translation, while our hallucinator also considers the spatial transformations and involves the spatial transformer network. Q7: Further analysis with more datasets, more methods, and more case analysis (R2, R3) In our future work, we will use more datasets with different types and modalities to further evaluate our model. And more methods, including few-shot learning and ablation studies, and case analysis will be further investigated. We will also revise the paper based on other valuable comments. Thanks again for the time and effort spent in helping improve the quality of the work.



back to top