List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Wan Liu, Qi Lu, Zhizheng Zhuo, Yaou Liu, Chuyang Ye
Abstract
Deep learning based methods have achieved state-of-the-art performance for automated white matter (WM) tract segmentation. In these methods, the segmentation model needs to be trained with a large number of manually annotated scans, which can be accumulated throughout time. When novel WM tracts, i.e., tracts not included in the existing annotated WM tracts, are to be segmented, additional annotations of these novel WM tracts need to be collected. Since tract annotation is time-consuming and costly, it is desirable to make only a few annotations of novel WM tracts for training the segmentation model, and previous work has addressed this problem by transferring the knowledge learned for segmenting existing WM tracts to the segmentation of novel WM tracts. However, accurate segmentation of novel WM tracts can still be challenging in the one-shot setting, where only one scan is annotated for the novel WM tracts. In this work, we explore the problem of one-shot segmentation of novel WM tracts. Since in the one-shot setting the annotated training data is extremely scarce, based on the existing knowledge transfer framework, we propose to further perform extensive data augmentation for the single annotated scan, where synthetic annotated training data is produced. We have designed several different strategies that mask out regions in the single annotated scan for data augmentation. To avoid learning from potentially conflicting information in the synthetic training data produced by different data augmentation strategies, we choose to perform each strategy separately for network training and obtain multiple segmentation models. Then, the segmentation results given by these models are ensembled for the final segmentation of novel WM tracts. Our method was evaluated on public and in-house datasets. The experimental results show that our method improves the accuracy of one-shot segmentation of novel WM tracts.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_13
SharedIt: https://rdcu.be/cVD4U
Link to the code repository
https://github.com/liuwan0208/One-Shot-Extensive-Data-Augmentation
Link to the dataset(s)
https://db.humanconnectome.org
Reviews
Review #1
- Please describe the contribution of the paper
In this paper, the authors intend to implement one-shot segmentation of white-matter tracts by a novel data augmentation method. This work is extended from [10] (namely IFT) by borrowing its pretraining and fine-tuning framework, and the data augmentation is implemented by its proposed random cutout and tract cutout strategies. Then the augmented training images are used in constructing WM tract segmentation model (using TractSeg as the backbone). They demonstrate their segmentation performance using the downsampled HCP dataset (CQ) and the in-house dataset (IH).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The motivation for one-shot WM tract segmentation is sound, and their implementation in the random cutout and tract cutout is easy and clear to understand
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The data augmentation method is quite straight-forward, and it is not convincing that implementing cutout as the augmentation is the most suitable solution to this topic. Lack of literature on data augmentation for segmentation, and also no comparison between the proposed method with the SOTA augmentation method. Descriptions in Section 3 are not clear and should be organized in a more comfortable way.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper has some descriptions of the methodology with provided equations, which should help the readers to understand the main idea of the work and reproduce its algorithm. Though it is still suggested for the authors to provide the source code online, which can be used to fully validate the proposed method and understand its mechanism.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
It is suggested that the authors provide an overall pipeline of the framework, which can help the reader to make a clearer view of the major works in this paper. Descriptions in Section 3 are too complicated and unclear to understand how they manage to demonstrate the validity of the proposed method, there are many Abbreviations jumping everywhere in that section, and I strongly suggest that they write Section 3 and organize all the terms there in a more comfortable way. The authors seem to try augmenting the training data for WM tract segmentation, by implementing cutout to the one-shot data. I believe that the authors should further claim why they think that cutout is the most appropriate to this task, as there are a lot of researches in data augmentation for segmentation, many of whom are even for one-shot task. The authors have discussed none of the augmentation literature, which is also surprising to me, and they also haven’t compared their method with the alternative augmentation methods available. Besides, it is also curious to know what is the upper bound of the WM tract segmentation performance, when using normal number of training samples instead of one-shot manner to construct the segmentation model. In this way, it is clearer to know if this one-shot segmentation has reached its limits, or there remains some spaces for further improvements.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work introduced in this work is complete, the experiments seem to be nice, but their lacking of discussion in the SOTA augmentation methods hinder their contribution, and the writing in Section 3 also need improving.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The authors propose a method for segmenting novel white matter pathways with only a single annotated image by extensive data augmentation. Data augmentation is based on different types of masking out image regions. While the compared state of the art performs indeed well for images with few-shot annotations using a transfer learning framework, which relies on a fine tuning strategy, the segmentation based on one-shot annotations is challenging. In their experiments, the authors train segmentation models for each augmentation strategy separately and then ensemble the results of these models for the final segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors address the ongoing challenge of reducing the annotation time for novel white matter tracts, their presented method could lead to further improvements in this field.
Using their approach, the authors find a solution to extend the performance of an already existing and established method (TractSeg) and enable it to perform well in a one-shot scenario.
The authors use an in house and public available datasets and on both datasets the performance compared to the baseline is improved.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The discussion is extremely brief. Limitations and possible ways for further improvements are not discussed.
The authors use existing methods and combine them. No methodlogical novelty.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors use open available data of the HPC. The authors use TractSeg as backbone, for which code is available. No code is provided in for the framework their paper is based on, nor in their paper. However, mathematical definitions are provided and based on the checklist the reproducibility seems to be given.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
For the training process, the authors used the original HCP data. To mimic a more challenging and realistic scenario, they decided to downscale the original HCP data and use this downscaled data to further train the model with respect to the new wm tract. This simulates that the data for further training was obtained differently than the already annotated data from the first training. However, it would be interesting to show the performance of the network even if the further training was also based on the initial data (in this case, the original HCP data).
It would also be good to directly compare the one-shot scenario with data augmentation to the few-shot scenario without data augmentation, both in the context of the IFT. This would display on whether few-shots may eventually become some kind of obsolete.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well structured and the method is evaluated properly. The authors address a the weakness of an already existing approach and solve this weakness by introducing their approach of extensive data augmentation. I do not recommend oral presentation since the manuscript does not include any methodological novelty. It should be accepted though, since the approach to enable successful one-shot training is of interest to the community.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
still strong accept. useful paper
Review #3
- Please describe the contribution of the paper
This is an interesting paper about using transfer learning to predict white matter tracts in dMRI data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Given the fact that in the field the studies of analyzing white matter are limited by the number of available tract segmentations, the proposed method can be useful to generate potentially more based on existing data without requiring tons of manual seminations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
One minor comment that I have the private dataset used for testing has very similar quality with the HCP data. Wonder if the authors have any comments about applying to it to low quality, clinical style dataset.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
good
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This is an interesting paper about using transfer learning to predict white matter tracts in dMRI data. Given the fact that in the field the studies of analyzing white matter are limited by the number of available tract segmentations, the proposed method can be useful to generate potentially more based on existing data without requiring tons of manual seminations.
One minor comment that I have the private dataset used for testing has very similar quality with the HCP data. Wonder if the authors have any comments about applying to it to low quality, clinical style dataset.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Given the fact that in the field the studies of analyzing white matter are limited by the number of available tract segmentations, the proposed method can be useful to generate potentially more based on existing data without requiring tons of manual seminations.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
I have no further concern about the paper.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a deep learning method for the segmentation of white matter fiber bundles in the one-shot scenario in terms of the availability of training data. While the reviewers have overall positive opinions about this work, some moderate concerns were raised. Both reviewer 1 and reviewer raised concerns about the novelty of the proposed method in the network architecture and data augmentation. Please also clarify data augmentation related questions from reviewer 1, and suitability of the proposed method for clinical image data from reviewer 3.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
Q1: Reviewer 1 (R1) and Reviewer 2 (R2) raise concerns about the novelty of the proposed method. Response: We would like to clarify that the focus of this work is to develop a novel approach to one-shot segmentation of novel white matter (WM) tracts, instead of developing a new architecture or data augmentation approach for generic WM tract segmentation. To address the problem, unlike previous works that explore knowledge transfer for improved segmentation, we propose to better exploit the single annotated data at hand. This is achieved by performing extensive data augmentation. Since different data augmentation strategies could lead to conflicting learning targets if they are used together, we further propose to train multiple models with these strategies separately and ensemble the results. The ideas described above are the novelty of the proposed work, and to achieve our goal we can simply exploit some existing tools, which makes our method easy to implement without sacrificing its effectiveness. In addition, although Cutout is an established data augmentation approach, we have still developed a variant of Cutout that pays attention to the tracts, and the results have demonstrated the benefit of this proposed variant.
Q2: R1 wonders why Cutout is suitable for the topic and suggests that there should be literature review on and comparison with other data augmentation methods. Response: Existing data augmentation methods can be roughly grouped into the following categories. First, traditional data augmentation (TDA) can be achieved with basic image transformation, such as spatial rotation, translation, and scaling, as well as intensity perturbation. Second, generative models can be trained to obtain synthetic training samples. Third, annotated images can be mixed for data augmentation. Lastly, image masking can be used to generate additional training data (like Cutout). However, TDA alone cannot generate very diverse augmented data, which is insufficient for the challenging one-shot setting. This is also demonstrated in the experiments by the baseline results (CFT and IFT), where basic image transformation was applied online by default in the TractSeg framework. The training of generative models usually requires a large amount of annotated training data. Although some few-shot or one-shot methods are also developed, they at least require a large amount of unannotated training data, which is not guaranteed in our problem. Thus, the second category is not suitable for our problem either. Image mixing requires at least two annotated images, and thus it cannot be used in the one-shot setting. Cutout and its variant can be applied even when only one annotated image is available. Therefore, we believe that they are most suitable for our one-shot problem. We will clarify this and review existing data augmentation methods.
As mentioned in Sect. 2.3, by default TDA is applied in TractSeg. Therefore, in the experiments we have actually compared our results with those achieved with TDA alone. We will clarify this. Since the other data augmentation methods are not suitable for the task, they were not considered.
Q3: Reviewer 3 wonders how the proposed method can be applied to low-quality clinical datasets. Response: We would like to clarify that for the CQ datasets the pretrained model was trained on the original high-quality HCP data, but the fine-tuning and evaluation were performed with the generated low-quality dMRI scans that resemble clinical datasets. The results indicate that the proposed method can be applied to low-quality clinical datasets. However, it is still possible to further improve the performance on low-quality clinical datasets, for example, with even more diverse data augmentation, and this can be investigated in future work. We will add this discussion, which also addresses R2’s comment on the very brief discussion.
Code will be shared after the work is published. Other writing or minor issues will be addressed.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I agree with the consensus from the reviewers and meta-reviewers about the acceptance of this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Interesting idea and valuable contribution.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
upper
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I agree with the reviewers that, even though novelty in this work is not ground breaking, it satisfies the expectations for presentation at MICCAI, addresses an interesting problem, and provides proper evaluation.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1