List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Jun Gao, Qicheng Lao, Qingbo Kang, Paul Liu, Le Zhang, Kang Li
Abstract
Breast and thyroid lesions share many similarities in the feature representations of ultrasound images. However, there is a huge lesion scale gap between the two diseases, making it difficult to transfer knowledge between them through current methods for unsupervised domain adaptation. To address this problem, we propose alesion scale matching approach where we employ a framework of latent space search for bounding box size to re-scale the source domain images, and then the MonteCarlo Expectation Maximization algorithm is used for optimization to match the lesion scales between the two disease domains. Extensive experimental results demonstrate the feasibility of cross-disease knowledge transfer, and our proposed method substantially improves the performance of unsupervised cross-disease domain adaptation models, with the Accuracy, Recall, Precision, and F1-score improved by 8.29%, 6.41%,11.25%, and 9.14% on average in the three sets of ablation experiments.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_63
SharedIt: https://rdcu.be/cVRXB
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
(1) Demonstrate the feasibility of cross-disease knowledge transfer, i.e., to transfer knowledge from thyroid lesions to breast lesions in an unsupervised cross-disease domain adaptation manner. (2) To address the lesion scale gap problem by proposing a lesion scale matching approach, i.e., search for bounding box size in latent space for rescaling, together with the Monte Carlo Expectation Maximization algorithm. (3) Demonstrate the method with improved performance on one private thyroid US dataset and one public breast US dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) Identified the problem of knowledge transfer between two different diseases. 2) proposed a lesion scale matching approach by searching for bounding box size in latent space together with the Monte Carlo Expectation Maximization
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) This method is proposed based on existing unsupervised domain adaptation By adding the lesion scale matching the performance can be improved. However, the authors did not discuss the scenario when the knowledge of target data is not available. 2) The authors need to provide a detailed “algorithm” in a table to show the whole process. 3) The authors haven’t compared with existing “batch normalization” based method.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The source codes are not provided. One of the dataset is private dataset. There is no detailed “Algorithm” presented in a table, hence reproducibility may not be that straightforward.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1) Please give some explanation on E1. (2). 2) Section 2.2 regarding how to determine the source and target size, this is totally based on prior knowledge for target domain? 3) Can the authors include an algorithm to show the whole process of the method to help understand. 4) Can the author compare the proposed method with existing batch normalization-based domain adaptation method? For example: • Revisiting Batch Normalization For Practical Domain Adaptation by Y. Li et al. • Domain-Specific Batch Normalization for Unsupervised Domain Adaptation by W.-G. Chang et al. CVPR’19
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The better results of using lesion scale matching. The authors may need to give a detailed “Algorithm” for reproducibility of the work No comparison with existing domain adaptation methods on batch normalizaiton.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
In the rebuttal, the authors have mostly addressed two major concerns from me. Regarding the question: when the priori knowledge of the range of tumor size is not available is acceptable. The authors proposed to extract such knowledge from the pseudo label based on existing tumor segmentation of target data, which is acceptable. Further, the authors have provided results on two batch-normalization methods and showed advantages of the proposed method.
Review #2
- Please describe the contribution of the paper
This paper proposes an unsupervised approach to transfer knowledge across diseases i.e. from thyroid nodule to breast nodule classification in ultrasound imaging. It does not require labelled data in the target domain. To address the lesion scale gap across the two domains, they propose a lesion scale matching solution, which entails a framework of latent space search for bounding box size and a MC expectation maximization algorithm.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper addresses a critical important problem that may be of help in other similar scenarios for cross-disease transfer of knowledge;
- The paper propose an original approach, with clear methodology and experimental results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Perhaps the choice of ResNet50 needs to be motivated
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It appears that authors meet the requirements.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Perhaps the choice of ResNet50 needs to be motivated
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Crucial problem
- Original methodology
- Good experimental design
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
This paper propose a lesion scale matching approach to rescale the source domain image and Monte Carlo Expectation Maximization algorithm is employed to match the lesion scale. It shows noticeable improvement in average in 3 sets of ablation studies.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Due to the data limitation in medical domain, this work holds a valid and practical motivation. Extensive experiment studies have been conducted. Especially how the scale disparity affects the estimation accuracy in target domain which address the key argument in this work.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
However, I have to point out, the scale mismatching issue is a typical problem in the computer vision community. Namely, the classification /regression are based on samples from the bounding box. Namely, we will not use a city view street image for car classification but the car image patch crop instead.
It is also a standard pipeline in most of the data augmentation pipeline to introduce the randomness in scale which is the latent variable z in this work.
Though it shows nice theoretical insight about the random scale (LS) here from prior perspective and employs monte carlo EM.
It at least missing a strong baseline based on the conventional data augmentation pipeline to random scale z instead using a fixed z in the ablation study. - Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
code and dataset will be avaialbe according to the submission.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Due to the similarty of the standard data augmentaiton pipeline, the impact seems to be limited.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please check weakness.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviewers think that the paper has substaintial merits. In particular, the paper addresses an important problem in cross-disease knowledge transfer. The proposed approach is original, and the methodology/experiments are clear. However, reviewers also pointed out significant weaklesses related aaa. Specifically, authors did not discuss some important scenarios where target data are not available. Authors also did not compare to some popular batch normalization baseline. The third reviewer wanted to see comparisons to a strong baseline based on conventional data augmetnation and some ablation study. After considering all reviewers’ comments, the area chair would like to invite authors to submit a rebuttal addressing the reviewers’ concerns.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
10
Author Feedback
We thank the AC and reviewers for recognizing the substantial merits in our work, including the importance of the research, the originality of the proposed approach and the clarity of the methodology/experiments. In this rebuttal, we clarify the raised concerns, i.e., discussion of the scenarios without target knowledge, and the comparisons to the two suggested baselines where we find that our method (79.41%) still outperforms the batch normalization baseline (~70%) and the conventional data augmentation baseline (76.77%).
➤ Reviewer #1
Discussion of the scenario when the target knowledge is unavailable. We thank the reviewer for the constructive question. A priori knowledge of the range of tumor size for each disease is usually not difficult to obtain by literature. However, when such knowledge is indeed unavailable, we can instead use the pseudo labels of the target data generated by existing tumor segmentation methods and extract the pseudo knowledge on the tumor size as a replacement for the ground truth.
The authors need to provide a detailed “algorithm” in a table. We will summarize our algorithm in a table and release our source code on GitHub upon acceptance of this submission.
Missing “batch normalization” based method. We thank the reviewer for suggesting the BN-based methods. We compare our method (79.41%) to the two suggested BN-based methods, i.e., DSBN and AdaBN (both give lower accuracy ~70%) in our setting, suggesting that the BN-based methods are indeed orthogonal to our method, and do not function through the lesion scale gap problem which is crucial for cross-disease domain adaptation.
➤ Reviewer #2
- Perhaps the choice of ResNet50 needs to be motivated. We thank the reviewer for the detailed comment. Since our work is established on the current UDA methods where most work typically adopt ResNet50 as the backbone for fair comparisons among different methods, therefore, we also follow the routine choice of using ResNet50.
➤ Reviewer #3
- Missing a strong baseline based on the conventional data augmentation pipeline. We thank the reviewer for bringing up a very good point, and we are happy to clarify this rather subtle but absolutely crucial matter, as the latent variable z and its range is key to our entire reasoning. We argue that z 1) is not equivalent to obtaining scale invariance via data augmentation and 2) CANNOT be chosen completely randomly. Data augmentation with scaling (in the CV community) ensures network output that is invariant with respect to scaling, but in our cross-disease domain adaptation case (Medical), we do NOT WANT scale invariance. This is because lesion size is a criterion in classifying malignant vs benign, with malignant lesions being larger. Breast lesions are typically twice as large as thyroid lesions (table 1), and if scale zoom is too small for example, a malignant lesion in the target breast domain is shrunk and may potentially be erroneously classified as benign. Thus, correcting for scale mismatch is fundamental to domain adaptation and is not merely an artifact of the data acquisition process that we wish to achieve invariance with respect to, i.e. camera distance in CV. Combining eq. (3) and statistical a priori anatomical information, we obtain a range of [336,414] for z, and this theoretical range is experimentally verified with highest accuracy at z = 392 (see fig. 3). As indicated in figure 3 and table 3, choosing sizes smaller or larger than this range only DECREASES the accuracy. Thus scales out of our desired range not only is theoretically unjustified but also empirically performs worse. Although we believe the results in figure 3 and table 3 sufficiently demonstrate that random scaling will only lead to worse results, for completeness, we include an additional new ablation suggested by the reviewer. Compared to the completely randomized scale data augmentation, our method improves the accuracy from 76.77% to 79.41%.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Reviewers think that the paper has substaintial merits. In particular, the paper addresses an important problem in cross-disease knowledge transfer. The proposed approach is original, and the methodology/experiments are clear. However, reviewers also pointed out significant weaklesses. Specifically, authors did not discuss some important scenarios where target data are not available. Authors also did not compare to some popular batch normalization baseline. The third reviewer wanted to see comparisons to a strong baseline based on conventional data augmetnation and some ablation study.
In the rebuttal, the authors sufficiently addressed two major concerns from reviewers. The authors have provided additional results on two batch-normalization methods and showed advantages of the proposed method.
The AC voted to accept this paper with good novelty and minor weaknesses.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper investigates an important and interesting problems, cross-disease transfer of knowledge. The authors builds their framework on domain adaptation. The proposed framework is not very new, while the problem is interesting and important. The key weakness (lack of comparison with strong baselines) was addressed in rebuttal. Therefore, the AC votes for accepting this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have clearly rebutted/clarified the points raised by the reviewers. I would like to recommend an accept for this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2