List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Qikui Zhu, Yanqing Wang, Lei Yin, Jiancheng Yang, Fei Liao, Shuo Li
Abstract
Deep learning-based methods have obtained promising results in various organ segmentation tasks, due to their effectiveness in learning feature representation. However, accurate segmentation of lesions
can still be challenging due to 1) the lesions provide less information than normal organs; 2) the available number of labeled lesions is more limited than normal organs; 3) the morphology, shape, and size of lesions are more diverse than normal organs. To increase the number of lesion samples and further boost the performance of various lesion segmentation, in this paper, we propose a simple but effective lesion-aware data augmentation method called Self-adaptive Data Augmentation (SelfMix). Compared with existing data augmentation methods, such as Mixup, CutMix, and CarveMix, our proposed SelfMix have three-fold advances: 1) Solving the challenges that the generated tumor images are facing the problem of distortion by absorbing both tumor and non-tumor information; 2) SelfMix is tumor-aware, which can adaptively adjust the fusing weights of each lesion voxels based on the geometry and size information from the tumor itself; 3) SelfMix is the first one that notices non-tumor information. To evaluate the proposed data augmentation method, experiments were performed on two public lesion segmentation datasets. The results show that our method improves the lesion segmentation accuracy compared with other data augmentation approaches.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_65
SharedIt: https://rdcu.be/cVRwS
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
SelfMix tried to solve the challenges that the generated tumor images are facing the problem of distortion, by adaptively adjusting the fusing weights of each lesion voxels based on the geometry and size information from the tumor itself
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1)It is the first one that notices non-tumor information among data augmentation methods for lesion segmentation. It may improve the accuracy of lesion segmentation when the training data set is small. 2) Good evaluation. The author compared this method to trandiational methods and recent related papers.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) The authors did not explain why this method is distortion free. This method computed voxelwise weights with distance map and generate new images by combining two images with the weights. As organs in different images may have different distribution, distortion may still happen.
2) Symbols were not used in a good way in equations. In Eq.1 and 3, the same symbol appeared in the both sides of the equations
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good. More details of the “random selected non-tumor regions” should be given.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1) Better writing is needed, especially for the equations. 2) More details are needed for the “fusion” step. 3) It should be explained more clearly why it is distortion free.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It should be explained more clearly why it is distortion free.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The authors proposed ‘SelfMix’ for image segmentation which extends existing ‘CutMix’ and ‘CarveMix’. It is tumor-aware and considers background information. It allows more realistic images for training the segmentation model and improves the results overs baselines.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed ‘SelfMix’ extends existing ‘CutMix’ and ‘CarveMix’, which is tumor-aware and consider background information.
- More realistic images for training.
- Improved numbers on baselines.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
Clarification on the selection of non-tumor regions. For liver lesion, I guess the non-tumor regions should only be liver tissue. Otherwise, the synthetic image would not realistic. However, I didn’t see a clarification on this.
-
Demonstration of sample synthetic images. Unfortunately, there is no more synthetic images to give the reader more ideas how good the proposed SelfMix is compared to cutMix and CarveMix. Fig 1 is not clear enough. Maybe find another case for demonstration.
-
Results It would improve the quality of this work if the SelfMix is done in 3D and training the segmentation models in 3D. How cutMix is implemented in Table 2? Are the tumor allocated on random positions?
-
Presentaions. Fig. 1 is nice but it can be improved. I could not see the difference between CutMix and CarveMix. Too many typos.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
details look good to me.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
Clarification on the selection of non-tumor regions. For liver lesion and kidney, I guess the non-tumor regions should be the liver or kidney tissues. Otherwise, if the synthetic images use completely background pathes, they would not realistic. However, I didn’t see a clarification on the selection of non-tumor regions.
-
Demonstration of sample synthetic images. Unfortunately, there is no more synthetic images to give the reader more ideas how good the proposed SelfMix is compared to cutMix and CarveMix. Good and bad samples are all helpful.
-
Results It would improve the quality of this work if the SelfMix is done in 3D and training the segmentation models in 3D. In Table 3, the number of using 100% data for UAD drops compared to the one achieved by 75%, why? How cutMix is implemented in Table 2? Are the tumor allocated on random positions?
-
Presentaions. Fig. 1 is nice but it can be improved. I could not see the difference between CutMix and CarveMix. Image content is a bit different. Please crop the image carefully.
Too many typos. x. in Sec. 2.1, Fig 2 -> Fig. 2; multi-steps -> multiple steps ; date –> data
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Nice technical contirbution on an effective method for data augmentation in image segmentation. But the presentation, clarification and results can be greatly improved.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
I would keep the initial rating ‘weak reject’ as the rebuttal does not fully address my concerns. Especially,
(1) 3D or 2D? The authors said they used a V-Net but in Section 3.1, they said ‘each slice is resized to 512× 512 and then randomly cropped to 256 × 256 for training. I am confused if it is 3D or 2D. I could not find any 3D keywords in the main text either. (2) ‘The images generated by CutMix and CarveMix are the same in Fig. 1’, then why put two same images into Fig. 1? We expect to see the difference between the two methods, isn’t it?
Review #3
- Please describe the contribution of the paper
An effective lesion generation method is urgently needed to boost the performance of lesion segmentation. This paper proposes a novel data augmentation framework through better utilizing the lesion and non-tumor region information.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Motivation is strong. Method is easy to understand. Experiments on two public lesion segmentation datasets show that the designed method improves the lesion segmentation accuracy compared with other data augmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The abstract pays several sentences to describe their motivation, which is of course vital for a paper. However, the talk about method is short, lack of details. And they present their advances similar with the sentences in contribution part(end of introduction). Maybe description part should be reconsidered to be brief and bullet point about method itself should be given.
- Some descriptions are not true. i.e. “Compared with normal organizes, the lesion has a very small amount and meanwhile only occupies a small region in the whole image, which leads to less information can be proved to CNNs” Actually, some head-neck organs are very small while lesions like glioma maybe very large. I understand why the authors talk like that, but accurate description is important in a research paper.
- From Fig 2, it seems that when fusing tumor information with non-tumor region, there is a mix both inside or outside tumor lesion. However, the mixed tumor is still labeled with the fused tumor contour. How to keep the label accurate?
- Section 2.2 “Relationship with Mixup, CutMix and CarveMix” is not method part. Normally it should be talked about this in discussion.
- Some typos. i.e., in Fig 3, TAD should be TDA, is that right?
- There is a constrain in augmentation of medical images, how to make sure the augmentation is clinical available. i.e. how do you make sure to locate a simulated tumor in probable position? Do you manually choose non tumor region?
- The performance of Vnet without data augmentation should be presented.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The description is clear. Reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
see the main weaknesses part
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Method
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
No more concerns.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviewers agree there is merit in the proposed data augmentation method for lesion segmentation. They also raise several concerns, such as how to ensure no distortion, need clarification on the selection of non-tumor regions, how to make sure the augmentation is clinically available, typos and better writing is needed, etc. Furthermore, importantly, the performance on the public dataset, e.g., LiTS, seems to be quite low. The leading result on the LiTS website is already 0.825, while the proposed SelfMix only achieves 0.621 in this paper. A strong baseline framework is important to check whether the proposed lesion augmentation method can really improve the segmentation accuracy.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Author Feedback
Thanks to the meta-reviewer and reviewers for recognizing the contributions, motivations, and improvements in our work (“all reviewers agree the work is meaningful contributions and improvements”)
- Contribution (“SelfMix solved the challenges…”-R1, “Nice technical contribution…”-R2, “This paper proposes a novel…framework”-R3).
- Motivation (“It is the first one that notices non-tumor information …”-R1, “Motivation is strong…”-R3).
- Improvement (“Good evaluation…”-R1, “…improves the results overs baselines”-R2, “Improved numbers on baseline…”-R2, “the designed method improves…accuracy” -R3). -The questions that affect the comments are clarified: Q1: About distortion free (“why this method is distortion free”-R1) A1:Our method solves the distortion challenge from three aspects:1) Different from GAN-based methods, the tumor generated by our model is constructed by the true tumor information, which avoids the distortion in the content of generated tumor. 2) Both tumor and non-tumor information are considered while fusing two image voxels, which further avoids the distortion that could exist in the fusing region. 3) The fusing weights are adaptively computed using the structure and geometry information of the lesion itself, which further avoids the distortion from the distributions of the tumor itself. Q2: About non-tumor regions (“selection of non-tumor regions.” – R1, R2, R3) A2: The non-tumor regions are selected from the normal tissue where tumors can grow. Specifically, in our paper, the non-tumor regions were randomly selected from the normal liver or kidney tissues. This setting makes the tumor generated by our method clinically appropriate and reasonable. Q3: About results (“It would improve the quality of this work if the SelfMix is done in 3D and training in 3D. using 100% data for TAD drops compared to the one achieved by 75%, why? How CutMix is implemented?”-R2) A3: 1)I agree. Accurately, the fusing weight was computed in 3D, which contains the whole structure and geometry information of the lesion. SelfMix was also operated in a 3D manner using the 3D fusing weight. The segmentation method (Vnet) was also a 3D network. 2)We have re-checked the segmentation results. Since no post-processing was used, some samples exist too many noises, which affects the final performance. 3) CutMix directly replaces the selected non-tumor region with the tumor patch to generate synthetic images. And the selected non-tumor region position was set to the same in compared methods for fairly comparison. Q4: About presentations (“no more synthetic images given. In fig.2, not see the difference between CutMix and CarveMix”-R2) A4: 1) Thanks! Fig.1 has summarized and shows the shortcomings of generated tumors produced by start-of-the-art methods. And we will further prove that by providing more synthetic images in the revised version. 2) The images generated by CutMix and CarveMix are the same in Fig. 1, due to CarveMix having the same operation (directly replaces ) as CutMix when CarveMix does not scale the size of the tumor. Q5: About the label (“How to keep the label accurate?”-R3) A5: The question has been considered and solved by SelfMix. To guarantee the consistency and accuracy between generated tumor and label, SelfMix ensures the tumor information occupies a higher weight (larger than 0.50 while fusing computing) than the non-tumor during fusing, which makes tumor features occupy the main component inside the mixed region. The above settings make the label of the fusing tumor always consistent with the tumor feature and accurately mark the region with tumor features. Q6: About details and writing (“more details are needed. symbols in equations”-R1, “some typos”-R2, R3, “the description part should be reconsidered… descriptions, Section 2.2, Vnet without data augmentation…” -R3) A6: Thanks! All the constructive suggestions will be adopted in the final version.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed most reviewers’ comments well. However, my question about the low Dice on LiTS is not mentioned. A strong baseline is important to evaluate whether the proposed data augmentation method is really useful, especially to be considered to guide practice. In addition, as pointed out by Reviewer #2, whether the used segmentation approach is 2D or 3D is unclear. This is important for tumor segmentation in 3D medical imaging. However, in overall consideration, the paper has merit, although there are some weaknesses.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This article proposes a new framework for data augmentation by considering information from both tumor and non-tumor regions. The rebuttal addressed the important issues, such as distortion free. However, explanation about the result on LiTS website is not clear. Overall, the paper has merit. Even if there are some weaknesses, I think the article deserves to be presented at MICCAI. My proposal is therefore ‘acceptance”.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Data augmentation is of course a major and important topic for medical imaging. The authors here propose an interesting approach to data augmentation that is of interest to the community. Many concerns were addressed in the rebuttal, but authors did not address the meta-reviewer’s question as to why their approach achieves such poor results on LiTS, given that even methods from the 2017 MICCAI challenge post much better results. This puts into question the baseline used. And without having confidence in the baseline, one cannot be confident in the improvements garnered by the author’s method (would the authors’ approach improve results on a stronger baseline where this less room for improvement?). This coupled with major clarity issues in the writing and presentation make me recommend reject.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR