List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Joohyung Lee, Jieun Oh, Inkyu Shin, You-sung Kim, Dae Kyung Sohn, Tae-sung Kim, In So Kweon
Abstract
Volumetric images from Magnetic Resonance Imaging (MRI) provide invaluable information in preoperative staging of rectal cancer. Above all, accurate preoperative discrimination between T2 and T3 stages is arguably both the most challenging and clinically significant task for rectal cancer treatment, as chemo-radiotherapy is usually recommended for patients with T3 (or greater) stage cancer. In this study, we present a volumetric convolutional neural network to accurately discriminate T2 from T3 stage rectal cancer with rectal MR volumes. Specifically, we propose 1) a custom ResNet-based volume encoder that models the inter-slice relationship with late fusion (i.e., 3D convolution at the last layer), 2) a bilinear computation that aggregates the resulting features from the encoder to create a volume-wise feature, and 3) a joint minimization of triplet loss and focal loss. With MR volumes of pathologically confirmed T2/T3 rectal cancer, we perform extensive experiments to compare various designs within the framework of residual learning. As a result, our network achieves an AUC of 0.831, which is higher than the reported accuracy of the professional radiologist groups. We believe this method can be extended to other volume analysis tasks.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_75
SharedIt: https://rdcu.be/cVRu0
Link to the code repository
https://github.com/JoohyungLee0106/rectal_MR_volume_classification
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
In this paper, the authors proposed a volumetric convolutional neural network to discriminate T2 from T3 stage rectal cancer with rectal MR volume. A variety ways for combining 2D slice-level features into 3D volume-level features were compared. The authors selected the best performing model through extensive experiments in an in-house dataset of 567 patients.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is nicely organised and written. Very good introduction, nice background on the disease and current methods of generating 3D features for classification.
- Good experimental design and the results are clearly presented.
- The finding in this paper is a good start point for other work in this direction.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The performance of the proposed method is only marginally better than the baseline 2D model. Specifically, a variety of network structures are tested but only the f-rMC5 model outperforms the baseline f-R2D model.
- A few details are missing. See costructive comments for justification.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good. Sufficient details are provided to reproduce the work. Dataset will not be public but it’s understandable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
In this work, the authors investigated the performance of various types of approaches for aggregating 2D features into 3D features for colorectal cancer staging. Specifically, network structures, loss functions and aggregation functions are tested. The authors adopted the approach where they first determine one factor accroding to the best performing model and gradually add more factors into consideration one at a time while keeping perviously defined factors fixed. It’s understandable they chose this approach but it would be better if they could do more rounds of selctions or try all possible combinations.
There are a few minor problems:
- How long is the interval between MR imaging and surgical resection for the patients in this study?
- Variations in scanner types should be taken into account.
- One big challenge for moving from 2D to 3D is the computation requirement. It would be helpful to include an analysis on number of parameters of different network structures.
- Attention weighting is implemented as softmax in this paper. It’s better to call it softmax weighting/pooling to avoid confusion.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Despite having a few problems, the paper has proved its point through extensive experiements. Even though the performance is not outstanding, the findings in the paper are useful for future work in this direction.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Most of my concerns are addressed and I believe the paper is of interest to the community.
Review #3
- Please describe the contribution of the paper
Using a volumetric convolutional neural network, the authors developed an automatic CAD system to differentiate between T2- and T3-stage rectal cancer. The network contains a CNN feature extractor that maps medical volume to frame-wise features. As well as, a depth aggregation function summarizes the frame-wise features into a volume-wise feature.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The manuscript is well-written and organized.
- The author creates a good combination between center loss, triplet loss, and depth aggregation function to enhance the results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-The introduction should contain more details about the open research problems. The introduction should rewrite with a general overview of the study, such as what are the rectal cancer stages? What challenges face the physician to distinguish between T2- and T3-stage rectal cancer? How did you solve these challenges? -why did the authors distinguish between T2- and T3-stage rectal cancer, not other stages?
- Is the manner of rectal cancer treatment depend on the cancer stages? Which one?
- you mentioned in the manuscript, “Our goal is to solve a clinically challenging problem: to distinguish T2-stage rectal cancer from T3-stage rectal cancer with MRI.” Then please compare your automated accuracy and the clinical accuracy for this problem. -Please add a graph to show the value of the loss function during training. This will reflect the behavior of the new loss function.
- Is the segmentation for rectal cancer good preprocessing step for this classification?
-
You didn’t define the parameter p in equation 1.
-
what are these appreciations of Acc(T2) and Acc(T3) mean? Why is there accuracy for T2 and T3 separately?
- would you please add the ROC cures?
- I think the conclusion should be more simple.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
NA
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- This paper is ready with a bit of modification to publish in the top journal.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work contains an excellent engineering contribution
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
1, a hybrid convolution model is introduced to extract rectal tumor features. 2, a bilinear scheme is employed to conduct pooling for every layer. 3, the classification performance is carried out over 3D MRI rectal volumes.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method idea of this paper is acceptable. And it yields encouraging results which exceeds performances of radiologists.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1, except the machine parameters, the MRI database of rectal cancer need more details about tumors, patients, and data, such as tumor size, tumor volume, patient ages, patient gender, slice thickness, voxel size and so on. 2, Tome notations of x+,x-,f+ and f- are required to specified in for Eq(3) 3, The output of Layer4 should be some 2D data. How to conduct 3D convolution over 2D data on Layer5 is an import issue which needs more details in section 2.2. 4, The experiment section mentioned the loss many times. But all loss values of different models are missing in Table 1 and Table 2. 5, I disagree the assumption of video and VMI’s similarity since video is considered as 2.5 dimensional data while volumetric data is one kind of 3D data. 6, 3D volume contains more information than 2D slices. The results over 2D are better than 3D data in Table 1 which needs some analysis. 7, This comparisons in Table 3 occur among different datasets. 8, Table 1, Table 2 and Table 3 do not give their parameters adopted in experiments.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
1, The localization of rectum region needs more details since it contains both tumor tissue and other benign tissues. this paper doses not specify either the whole rectum region or ROI of tumor is utilized in the experiment. 2, In Fig 1, the input of this CNN Network and H-W pool need to be specified. From 4th layer to 5th layer, the procedure is ambiguous. The structure of every layer should be plotted in Fig 1 or specified in Section 2.2. 3, Which feature is omitted in the GLCM features since PyRadiomics has 24 GLCM features while this paper chooses 23 features? 4, Some important training parameters are also absent, such as epochs and batch size.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
To distinguish the T2 stage from T3 for rectal cancer is a big challenge especially over MRI dataset. This paper employed a reasonable CNN model for this issue. The experimental results are fantastic. However, some aspects of dataset, CNN architecture and experimental analysis are required to be added and revised. Moreover, some evaluation figures such as ROC, Accuracy and loss should be necessary in the experiment section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea of this paper is fantastic and the novelties of the method also acceptable. But some details of the dataset, network architecture and experimental outcome are required more efforts.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The idea of this method is novel. The structure of the paper is very good and its experiment results are encouraging. According to these one-to-one responses, authors have answered all the questions I concerned. All of them are clear, reasonable and accurate. I think this verion is acceptable. However, the concept of frame is confusing. It should be the concept of video. In medical imaging, I suggest to replace it with “slice” which is more accurate.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Major strength is the development and systematic evaluation of CNN approach to go from 2D to 3D data. Weaknesses include some missing details on the dataset and computational complexity.
Rebuttal must address:
- summary of scanner/CT variations present
- computational complexity/# of parameters
- clinical relevance for T2 vs T3
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Author Feedback
We thank all reviewers (R2, R3, R4) and meta-reviewers (MR1) for their helpful comments.
[MR1.1, R2.6.2, R4.3.1] Sec. 2.1 of the uploaded manuscript contains scanner information. In the final version of our paper we will add additional subject demographic characteristics (age, sex, tumor volume, and N stage) and device information (manufacturer and scanner names, magnetic field strength, and voxel size) details stratified by T2 and T3 subsets. To address scanner-induced variability, we apply N4ITK and CLAHE during preprocessing. [MR1.2] We will include parameter counts in Tab 1 in our final manuscript. [MR1.3, R2.6.3, R3.3.1, R3.3.2-3] Proper differentiation between T2 and T3 stages is critical as it is the sole determinant in deciding to use chemo-radiotherapy. The decision to use such treatment methods impacts a patient’s clinical outcome and is also both physically burdensome and expensive. Despite this importance, the ability for clinicians to correctly differentiate the two stages using MRI varies widely amongst both radiologists and surgeons. We will add more on these details to our final version.
[R2.3.1] Most studies on 3D medical image analysis utilize either 2D convs or 3D convs. In contrast, we propose and compare the performance of different mixtures of 2D conv and 3D convs. We found that fusing the third axis information at the last layer (f-rMC5) outperforms both full 2D and full 3D model. For a fuller comparison between the full 2D CNN (f-R2D) and f-rMC5, we performed the f-R2D based experiment. These results reported in AUROC are as follows: i) +triplet loss(Tab 2): f-R2D: 0.814, f-rMC5: 0.834 ii)+triplet loss+bilinear(Tab 3): f-R2D: 0.804, f-rMC5: 0.831 [R2.6.1] Surgical resection is normally performed between 1-3 weeks after MR imaging. [R2.6.4, R3.3.10] We will adopt the suggestions. [R3.3.4, R4.3.7] In Sec 3.4, we compared the performance of human radiologists from previous works and our proposed method. We acknowledge the limitation of this comparison due to the differences in datasets (though we can earn some idea about how competitive our method is) and plan to compare the discrimination performance of radiologists and our proposed method in a future journal extension. [R3.3.5, R4.3.4] Although we recorded these values every epoch, we did not include plots in our manuscript due to the space limitation. We will add the graph if there is sufficient space for the final version of the manuscript. [R3.3.6] The shape of the rectum varies widely patient-by-patient and MRI slice-by-slice. We used segmentation as part of our prediction pipeline to reduce the complexity introduced by this variance. [R3.3.7] p is the softmax probability for each class. We will add this definition to our final version. [R3.3.8] Acc(T2) is the proportion of the correctly classified T2 over the ground truth. We will edit Acc(T2) and Acc(T3) to Recall(T2) and Recall(T3) to avoid confusion, respectively. [R3.3.9] To evaluate performance, we use AUC(AUROC) instead of plots due to the space limitation. [R4.3.2] We will clarify the notations in Eq3. [R4.3.3] We will further clarify our model architectures at Sec 2.2 of our final manuscript. [R4.3.4] As addressed in Sec 2.3, we use focal loss(eq.1) for Tab 1. For a fixed encoder, we experimented with additional loss functions in Tab2. [R4.3.6] We address this issue in our Introduction. The 2D CNN seems like a suboptimal solution for volume data as it doesn’t encode information along the z-axis; however, other studies have shown that the performance of 3D segmentation methods deteriorates with anisotropic volume. [R4.3.8] All experiments for Tab 1, Tab 2 and Tab 3 share the same parameters. For the additional parameters used in Tab 2, e.g., triplet margin, we followed their reference paper.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Fairly comprehensive review that addresses all the concerns raised from the original submission. Paper is acceptable for MICCAI.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper is well-written. The importance of accurate discrimination between T2 and T3 stage rectal cancers has been clearly described. The motivation of addressing the controversy between 2D and 3D analysis is also interesting. The results can be useful for researchers working on similar problems. This paper can be further improved by comparing with the SOTA results.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper has merits and novel contributions to solve a challenging problem with encouraging results, which will be of wide interest to the MICCAI readership. The response provided by the authors to the first-round comments are very detailed and convincing. As indicated in the rebuttal, the significance of the proper differentiation between T2 and T3 stages is important to be added to the revised version.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1