Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Lisha Yao, Yingda Xia, Haochen Zhang, Jiawen Yao, Dakai Jin, Bingjiang Qiu, Yuan Zhang, Yanting Liang, Suyun Li, Xian-Sheng Hua, Le Lu, Xin Chen, Zaiyi Liu, Ling Zhang

Abstract

We propose DeepCRC, a topology-aware deep learning-based approach for automated colorectum and colorectal cancer (CRC) segmentation in routine abdominal CT scans. Compared with MRI and CT Colonography, regular CT has a broader application but is more challenging. Standard segmentation algorithms often induce discontinued colon prediction, leading to inaccurate or completely failed CRC segmentation. To tackle this issue, we establish a new 1D colorectal coordinate system that encodes the position information along the colorectal elongated topology. In addition to the regular segmentation task, we propose an auxiliary regression task that directly predicts the colorectal coordinate for each voxel. This task integrates the global topological information into the network embedding and thus improves the continuity of the colorectum and the accuracy of the tumor segmentation. To enhance the model’s architectural ability of modeling global context, we add self-attention layers to the model backbone, and found it complementary to the proposed algorithm. We validate our approach on a cross-validation of 107 cases and outperform nnUNet by an absolute margin of 1.3% in colorectum segmentation and 8.3% in CRC segmentation. Notably, we achieve comparable tumor segmentation performance with the human inter-observer (DSC: 0.646 vs. 0.639), indicating that our method has similar reproducibility as a human observer.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_54

SharedIt: https://rdcu.be/cVRuF

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper presents DeepCRS, a method to segment colorectal cancer (CRS). DeepCRS guides the tumor segmentation using a surrogate task to locate the colorectum, allowing the model to better differentiate the target organ from the rest. The results show that the proposed approach obtains superior performance than nnUNet.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • Novel approach to segment CRS: the surrogate task proposed helps the model to better identify the organ and improves the results. This solution tackles a problem inherent to the task (organ with a long shape that is hard to distinguish) in a simple and effective way.

    • Complete experimental validation: the model starts from nnUNet and performs two major modifications: adding self-attention layers at each stage and adding the surrogate regression task to guide segmentation. Both modifications are evaluated independently and together to show their effect in colorectum and tumor segmentation.

    • Results reach human performance: the method obtains similar metrics to the human annotators.

    • Complete related work: the paper correctly places itself with respect to the literature.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The relevance of some results in unclear: in table 2, the last row shows the results of nnUNet and Swin UNETR in the colon task of the Medical Segmentation Decathlon, but there are no results for DeepCRC. Hence, the reason to include results on that dataset is confusing. In addition, the analysis of those results is confusing because it is combined with the conclusion on the in-house dataset (last paragraph of Quantitative results).
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Neither code nor the dataset will be made publically available. The description of the experiments is complete and might be sufficient to reproduce the experiments in some extent.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    MSD is used both to refer to the Medical Segmentation Decathlon and the Mean Surface Distance, making the analysis confusing.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a simple and novel approach to solve the task at hand. The experimentation validates all the design choices and the results prove the improvement obtained by the surrogate task intorduced.

  • Number of papers in your stack

    6

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The authors propose an extension of a UNet like method to improve segmentation of the colo-rectum and colorectal tumours.

    • Addition of a colorectal coordinate transform to capture positioning with the addition of a regression loss
    • Addition of attention to the network This accounts for the relatively unusual shape characteristics of the colo-rectum compared to other organs.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Generalisable extension of a Unet like architecture for the colon. Nice adaption of a standard segmentation approach to specific characteristics of a certain problem. In this case the length of the colon.
    • Appears to be a novel deep learning approach to the problem
    • Good comparison to the literature in table 1 that provides a clear reference and differences in the datasets
    • Evaluation - Good comparison to nnUNet and the impact of both proposed extensions as well as a range of performance metrics including detection rate and DSC in Table 2. This provides clear justification of the impact of the additions.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Would have been more valuable to also apply to public colorectal datasets with prepartion e.g. the medical image decathlon dataset so that true comparisons can be made to the state-of-the-art. This would demonstrate the generalisability of the method and a wider comparison to published results.
    • The authors conduct 5-fold validation on the 107 case dataset. While not overfitting during training, there is a risk that algorithm decisions and hyperparameters were tuned to this dataset. It would strengthen the paper to see an additional independent dataset or testing of generalisability on the medical image decathlon dataset
    • No code and models are planned to be released making this approach difficult to verify independently, especially given that a custom in-house dataset was used.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    No training, evaluation code or trained models provided make this algorithm potentially difficult to reproduce. Especially given the fairly complex adaptations to the training loss and network to incorporate both the predicted coordinate map and attention.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
    • This approach appears to have a lot of potential and thorough validation is performed on a small in-house dataset with cross validation
    • To show more generalisability and further independent testing it would be valuable to show performance on Decathlon-Task08 which is reference for comparison. Even if the dataset does involve preparation — which should make the task easier. Alternatively, releasing the code would make this easier to verify independently.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Paper has potential but for MICCAI standards it needs evaluation on a wider or alternative dataset, or release of the code to allow reproducibilty and independent testing on other datasets.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    2

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper studies a segmentation problem of the colorectum (colon and rectum) and CRC in routine abdominal CT scans. The task is interesting and valuable. The idea of adopting topology information to enhance the segmentation performance is rational and novel.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The motivation of the paper is solid and clear;
    2. The task of the paper is interesting and important;
    3. The solution of the algorithm is novel;
    4. The experimental results to some extent validate the effectiveness of the algorithm.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    To make the contribution of the paper clearer and make it to be valuable to a broader range of readers, more discussion about the existing literature on how topology information is exploited is important.

    More state-of-the-art segmentation algorithms suitable for colon rectum and colorectal cancer Segmentation should be compared.

    The expression of the paper is clear in most of the circumstance. However, there are still typos and grammar mistakes in the paper. It should be checked carefully. Here is an example: “We validate our proposed method on a in-house dataset, including 107 CT scans with manual colorectum and CRC annotations.”

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The dataset is an in-house dataset. This could be a problem of code reproducibility.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    Please check part 5 and part 7 for detailed information.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    1. The motivation of the paper is solid and clear;
    2. The task of the paper is interesting and important;
    3. The solution of the algorithm is novel;
    4. The experimental results to some extent validate the effectiveness of the algorithm.
  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    The author propose a new deep learning based method for colorectum segmentation and CRC segmentation, and evaluated its performance on a privately collected dataset out performing nnUNet.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The authors are the first to provide the exploration of automated segmentation of the colorectum and CRC in routine abdominal CT scans.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The author only evaluates their model on a in-house dataset. Authors are encouraged to evaluate on a second public dataset, such as the one used in [2, 6, 17] in Table 1. Because the variance shown Table 2 looks large (5.8% and 27.5%), I cannot tell if the improvement are significant and consistent.
    • The use of self-attention alone is not new in medical image analysis. Papers such as “Attention U-Net: Learning Where to Look for the Pancreas” already proposed this idea.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The dataset is private and no mention of whether will be made public. The authors do not mentioned N/A for the code part, which I assume will not be made public available.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    Please see weakness.

    • The use of centerline of the colorectum as another loss seems to be the main methodology contribution of this paper. However, I do not see clearly/intuitively why predicting the centerline coordinate would help. It would be good if the authors could elaborate on their motivation a bit more.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Although the author claimed they are the first to provide the exploration of automated segmentation of the colorectum and CRC in routine abdominal CT scans, experiments are not sufficient to claim better result and methodology contribution is limited.

  • Number of papers in your stack

    7

  • What is the ranking of this paper in your review stack?

    3

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    Several reviewers agree that this work shows an interesting contribution specific to the problem of segmenting colorectal cancer structures from CT. While the idea of self attention is not new, combined with the regression task employed to perform multi-task learning, a benefit in segmentation performance is shown. All this in an area where segmentation is a hard task. Reviewer concerns are about the generalizability and reproducibility of this approach due to a lack of published code, no providing of the in house comparison dataset to the general public and no experiments on a related publicly available dataset. However, from the point of view of this meta reviewer, the benefits and potential future uses of the methodological approach slightly outweigh these drawbacks, and the ideas should be discussed at MICCAI. In case of final acceptance, please thoroughly study the reviewer comments to create a revised version of the final paper, however, as per the MICCAI reviewing guidelines, no new experiments have to be performed.

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    3



Author Feedback

N/A



back to top