List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Jieun Lee, Kwanseok Oh, Dinggang Shen, Heung-Il Suk
Abstract
An increase in signal-to-noise ratio (SNR) and susceptibility-induced contrast at higher field strengths, e.g., 7T, is crucial for medical image analysis by providing better insights for the pathophysiology, diagnosis, and treatment of several disease entities. However, it is difficult to obtain 7T images in real clinical practices due to the high cost and low accessibility. In this paper, we propose a novel knowledge keeper network (KKN) to guide brain tissue segmentation by taking advantage of 7T representations without explicitly using 7T images. By extracting features of a 3T input image substantially and then transforming them to 7T features via knowledge distillation (KD), our method achieves deriving 7T-like representations from a given 3T image and exploits them for tissue segmentation. On two independent datasets, we evaluated our method’s validity in qualitative and quantitative manners on 7T-like image synthesis and 7T-guided tissue segmentation by comparing with the comparative methods in the literature.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_32
SharedIt: https://rdcu.be/cVRyM
Link to the code repository
https://github.com/2jieun2/knowledge_keeper
Link to the dataset(s)
https://www.nitrc.org/projects/ibsr
Reviews
Review #1
- Please describe the contribution of the paper
The proposed method fuses features from 7T-like images and 3T images to segment brain tissues. As 7T images are not available simply compared to 3T, the proposed method use 7T features without directly using 7T images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) Clear presentation and well written manuscript
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) Novelty is limited. Knowledge transformation and synthesizing 7T from 3T images have been used a lot in the literature.
2) I expected to see comparison results with methods that “synthesizing 7T-like image first and then conducting segmentation” to justify the proposed method.
3) In Table 2, the authors only showed the effectiveness of plugging 7T-like features in the segmentation frameworks, e.g., the 3D UNet framework. However, their proposed method has used extra data to train the teacher network compared to 3D UNet and DSC improvement is not significant (around 1%).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
based on the checklist, yes.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I suggest appending the comparison results with methods that “synthesizing 7T-like image first and then conducting segmentation” to justify the proposed method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Lack of proper comparison results.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This work proposed a knowledge keeper network to guide brain tissue segmentation by taking advantage of 7T representations without explicitly using 7T images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work presents a knowledge keeper network to extract 7T-like representations from 3T image, without paired-7T images during training.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Figure 1(c): what is the difference between Input 7T images, Reconstructed 7T images and Target 7T images? More information are required to understand how the teacher network works.
- Section 2: “Through two-stage KD methods, KKN learns to infer 7T-like representations from a 3T image using a paired 3T-7T dataset {X,Y}”, why paired 3T-7T dataset is needed during training (Figure 2(b)). This conflicts with authors’ statement.
- Figure 3: By comparing the result of the proposed method and ground truth, I do not think the result has 7T representations. Only the intensity distribution is similar with 7T images but it does not have tissue details that 7T images have.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No source code was provided.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Provide more information to verify the effectiveness of the teacher network.
- Please further specify how obtain 7T representations from 3T images. (Figure 2 (b))
- Recovery tissue details of 3T images are more important than simply simulating intensity distribution.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The simulation of 7T representations for 3T images is not clear.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The author proposed a novel knowledge keeper network (KKN), trained via KD and KT, to guide and train a brain tissue segmentation model using 7Tlike representations in a 7T-free domain.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method is novel and effective. They propose a novel knowledge keeper network (KKN) to extract 7T-like representations from a 3T image that can further guide tissue segmentation to take advantage of the contrast information of 7T without 7T images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The author claims that existing methods lack applicability and generalizability on datasets not including 7T images because they require pairs of a 3T image and its corresponding 7T image for training. But in this manuscript, the author still needs paired data to train the teacher model. How to explain that? Besides, I don’t know what the practical application is. In my opinion, the task is to segment brain tissues, why don’t you segment those directly by developing a better approach? Although the performance will be better if you translate a 3T image to a 7T image, you need 7T images to train the KNN, which is almost not available. So I think the practical application is limited.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good. The author has opened the code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The author claims that existing methods lack applicability and generalizability on datasets not including 7T images because they require pairs of a 3T image and its corresponding 7T image for training. But in this manuscript, the author still needs paired data to train the teacher model. How to explain that? Besides, I don’t know what the practical application is. In my opinion, the task is to segment brain tissues, why don’t you segment those directly by developing a better approach? Although the performance will be better if you translate a 3T image to a 7T image, you need 7T images to train the KNN, which is almost not available. So I think the practical application is limited.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I am confused about the practical application.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper introduces a knowledge keeper network for the 7T-free brain tissue segmentation. Specifically, an autoencoder is trained first to learn the 7T-image representations; then a GAN-based network is used to construct the 7T-guidance knowledge keeper network in a teacher-student training manner; finally, a segmentation network is performed using 3T and 7T-like features. The authors validated their methods on two independent (paired and unpared) datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper is well-written and easy to follow.
- The results are strong where the proposed method outperforms other state-of-the-art methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- This paired dataset is in-house, which will affect the reproducibility of this work.
- This work lacks some feature-level analysis for the effectiveness of the knowledge keeper network.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the paper is slightly limited due to the private paired dataset.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- It would be better to directly visualize the difference between the features from 3T and 7T images, and 7T-like features to demonstrate the effectiveness of the proposed knowledge keeper network.
- Ablation study to validate the regularizations for the autoencoder should be performed.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, this paper is good. The methodology is novel and sound; the results seem promising.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper receive relatively positive comments. I agree that the studied problem is interesting and the proposed framework is new and reasonable. While the reviewers also propose some valuable suggestions, such as clarififacation of the practical application of this framework, comparison with methods that synthesize 7T-like image first and then conducting segmentation, more analysis about the obtained features. The authors need to address these comments in final version, which would largely improve the quality of this paper.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Author Feedback
We sincerely thank the meta-reviewer and reviewers for their valuable comments and suggestions. Based on suggestions, we will reinforce our work and make a more comprehensive analysis in the future extension version. Here we would like to respond to comments raised by reviewers (R1~R4: Reviewer #1~#4, MR: Meta-Reviewer).
- (R1, MR) Comparison with methods that synthesize 7T-like image first and then conducting segmentation
- We agree that the comparison with the method of Fig. 1(a) can justify our proposed method. We conducted an additional experiment for training the 3D U-Net, which is one of the baseline models for segmentation, by using 7T-like images instead of 7T-like representations on the IBSR dataset (i.e., in a 7T-free domain). We observed lower DSC scores (CSF: 70.46, GM: 87.79, WM: 83.74) and the unclear edge of tissue details compared with our proposed method. It demonstrated the limitation of the existing method that it is prone to unexpected deformation or distortion of the inherent features in 3T informative for segmentation while mapping to 7T at the image level.
- (R2, R3) The difference in training data and tasks between a 7T domain and a 7T-free domain
- While we use 3T-7T pairs to extract 7T information for training the teacher network and knowledge keeper network (KKN) in a 7T domain, only 3T image is used for the downstream task (tissue segmentation) in a 7T-free domain. Since our goal is to take advantage of the contrast information of 7T in a 7T-free domain, the KKN is trained to transform a 3T image to 7T-like representations without taking the 7T image as the input. In order to transfer information of 7T indirectly to the KKN, we need a well-trained teacher network, which can distill the knowledge, i.e., 7T representations, at the feature level. Such 7T representations learned on 3T-7T pairs can be transferred directly to an independent segmentation model even if there are no 7T images. By training the model for 7T-guided segmentation, we can obtain better tissue-discriminative information than the baseline.
- (R4, MR) More analysis of the obtained features
- In our framework, the teacher network extracts features from the 7T input image, and the KKN extracts features from the 3T input image substantially in a feature extractor and then transforms them to 7T-like features in guide blocks. So we can define the 7T features, 3T features, and 7T-like features as the feature maps generated from the teacher network, a feature extractor, and guide blocks of the KKN, respectively. As a simple way to visualize features, we obtained the averaged feature map at each level of the network or modules. We observed that 7T-like features had similar contrast to 7T features and spatial structure of 3T features. Also, 7T-like features showed more clear tissue details than 3T features on the IBSR dataset. Note that we refer to 7T features extracted from the 7T image of the paired 3T-7T dataset for qualitative assessment since there are no 7T images on the IBSR dataset. However, the perceptual difference was less evident in the higher-level features due to their small size. So we calculated the L2-norm between two averaged feature maps for each level’s quantitative assessment. We observed that the distance between 7T-like and 7T (D1) was significantly closer than the distance between 3T and 7T (D2) for all five levels as below (The distance, D1 or D2, is reported with mean±sd across folds). Feature level | D1 | D2 1 | 0.0183±0.0105 | 0.9258±0.0608 2 | 0.0165±0.0147 | 0.5262±0.0186 3 | 0.0083±0.0055 | 0.3620±0.0082 4 | 0.0076±0.0053 | 0.5854±0.0183 5 | 0.0200±0.0092 | 0.7496±0.0296
- (R4) Ablation study to validate the regularizations for the autoencoder (teacher network)
- We agree that the ablation study for the teacher network can explain why the regularizations are adopted for training. We observed that our proposed method achieved better performance of PSNR (+0.25) and SSIM (+0.01) when applying two regularizations.