Authors

Taro Hatsutani, Akimichi Ichinose, Keigo Nakamura, Yoshiro Kitamura

Abstract

Many renal cancers are incidentally found on non-contrast CT (NCCT) images. On contrast-enhanced CT (CECT) images, most kidney tumors, especially renal cancers, have different intensity values compared to normal tissues. However, on NCCT images, some tumors called isodensity tumors, have similar intensity values to the surrounding normal tissues, and can only be detected through a change in organ shape. Several deep learning methods which segment kidney tumors from CECT images have been proposed and showed promising results. However, these methods fail to capture such changes in organ shape on NCCT images. In this paper, we present a novel framework, which can explicitly capture protruded regions in kidneys to enable a better segmentation of kidney tumors. We created a synthetic mask dataset that simulates a protuberance, and trained a segmentation network to separate the protruded regions from the normal kidney regions. To achieve the segmentation of whole tumors, our framework consists of three networks. The first network is a conventional semantic segmentation network which extracts a kidney region mask and an initial tumor region mask. The second network, which we name protuberance detection network, identifies the protruded regions from the kidney region mask. Given the initial tumor region mask and the protruded region mask, the last network fuses them and predicts the final kidney tumor mask accurately. The proposed method was evaluated on a publicly available KiTS19 dataset, which contains 108 NCCT images, and showed that our method achieved a higher dice score of 0.615 (+0.097) and sensitivity of 0.721 (+0.103) compared to 3D-UNet. To the best of our knowledge, this is the first deep learning method that is specifically designed for kidney tumor segmentation on NCCT images.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43990-2_2

SharedIt: https://rdcu.be/dnwLc

Link to the code repository

N/A

Link to the dataset(s)

https://github.com/neheller/kits19/blob/master/LICENSE

https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=61081171#61081171002c9b4515c84188a63a4440ca52abfa

Reviews

Review #3

Please describe the contribution of the paper

Deep learning is finding its application in all other fields including medical image analysis. This paper discuss a hybrid multi-step DL network to segment the kidney from the CT images of two modalities, CE and NC. Renal segmentation on CT images is one of the clinical task that needs little more attention due to the overlapping intensities which confuses the segmentation algorithms and hence results in either segmentation leak problems or improper boundary delineation. This paper is an attempt to this problem.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper describes a deep learning approach for segmenting the kidney from two modality images within CT. The organization of the paper content from the technical point of view and domain point of view is good. The methodology and results are discussed very well. The novelty of the work is convincing. Quantitative validation of the results are good. Seems the model can be re-executed by any researcher using the same data what the authors have used (I presume the code is deposited).
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Even though the method looks convincing, there is lack of subjective evaluation. Sufficient number of samples are not considered for training and testing, whether the volume based processing is done or slice based processing is done is not clear, there is no subjective evaluation in addition to objective evaluation.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

With the details provided by the author, it is possible to reproduce the results with the dataset of KiTS.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Few comments:
1. Do not use the word novel. It is over-optimistic.
2. You mentioned in page 2, point 2 that this framework can be extended to liver also. But in Liver we have 4 phases imaging. How is the proposed method which works on NCCT can be applied to multi-phase CT images.
3. As mentioned by you, after Figure 2, it is modification of the existing 3D UNet architecture. In all three networks it is the modification of existing networks. Then how is it possible to accept that it is novel architecture? Anyway all the three architectures are not fully written from scratch by you
4. First line in section 4 shows that the literature review is incorrect. There exists a paper in 2021 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7734203/).
5. Data augmentation is well described. What are the angular values? shear factor? what do you mean by random noise? is it Gaussian additive noise? or salt and pepper noise? How did you model the noise in-order to reduce it?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper content is well balanced and is convincing. I recommend for acceptance after minor revision
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

A method for segmentation of kidney tumors in the Computed Tomography scans without contrast is presented. This is a difficult problem because without contrast, tumors do not typically show a difference in radiodensity or texture in comparison with the surrounding parenchyma. It is therefore necessary to make use of deformations in the shape of the kidney to reason about where a tumor might reside. The authors produce a synthetic dataset of perturbed tumors added to noncontrast CTs to train a model to identify abnormal shapes, and then utilize this model to help inform a separate model which predicts tumor masks.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper presents a novel approach that does a good job utilizing domain-specific knowledge to enhance their method. The way in which synthetic data was created and used is clever in my opinion and its effectiveness is quite plausible. This is also a very under-studied problem, with virtually all attention in kidney tumor segmentation on contrast-enhanced CT as opposed to non-contrast.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I wish the authors would talk more about how they feel that solving this problem is clinically useful. The authors mention the early diagnosis of kidney tumors as motivation, but if diagnosis is the only goal, then surely formulating the problem as object detection would have been an easier machine learning problem with more suitable metrics. Segmentation has other benefits, but contrast-enhanced CT represents standard of care for kidney tumors, so a downstream CECT after the lesion was detected in NCCT could fulfill that need.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors make use of publicly available data for their experiments which greatly helps the reproducibility of their work, but as far as I can tell, their code has not been made publicly available. It would be helpful if the authors released this as well as the synthetic dataset that they generated.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

It would be very nice to see a figure with some examples of the perturbed kidneys that were synthetically generated.

Please see my comment in the “weaknesses” section about clinical utility and address this further in the paper.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper presents a novel and domain-knowledge-informed approach for a largely understudied problem, but the results are predictably somewhat poor due to the inherently high level of difficulty, and I feel that the clinical utility of the work could use better justification.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #1

Please describe the contribution of the paper

This paper presents a novel framework which can explicitly capture protuberances in kidneys to enable better kidney tumor segmentation from non-contrast CT (NCCT) scans. The pipeline consists of three networks: a conventional segmentation networks which extracts a kidney region and an initial tumor area, a protuberance detection networks which identifies the protruded regions from the kidney tumor mask and a final fusion network. The proposed framework, which is especially well suited to identify isodensity kidney tumors, makes use of a synthetic dataset that mimics kidneys with protrusions. Experiments on CT scans from the publicly-available KITS dataset show the relevancy of the proposed contributions.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- First deep learning method designed for kidney tumor segmentation on NCCT images
- Good positioning with respect to related works
- Innovative protuberance detection network and related synthetic dataset
- Methodological contributions rigorously assessed through an ablation study
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Comparisons with additional state-of-the-art methods could bring an added value to the experimental part
- The proposed pipeline increases the performance in Dice and sensitivity but amplifies the amount of false positives
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- The proposed method could be easily re-implemented based on provided architecture and training information
- The code is not provided
- Data comes from the publicly-available KITS dataset
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The submitted paper is innovative, well written and easy to follow. Major and minor comments provided below could be taken into account for further improvements.

Major comments: 1 - The fusion network combines the outputs from the base network and the protuberance detection network through a summation of the initial tumor and protruded region masks (Sect.3). However, the protruded areas may also be detected by the base network. If the output of the base network was binarized, it would be more appropriate to carry out the union of both masks. But, since the output is not binarized to keep gradient from flowing (Sect.4.1), can you have outputs larger than 1 for some voxels from protruded regions? 2 - Tab.2 and Sect.5.2 indicate that the fusion network could have amplified the false positives, which is in contradiction with the following (and redundant) statements from Sect.3: - “This fusion network […] is also responsible for removing false positives […]” - “[…] so that the last network can remove false positives […]” 3 - Comparions shown in Tab.1 could be extended using results from other related works including [11, 17, 21, 6, 17]. In addition, how do you explain the gap in tumor Dice results (more than 5%) between [13] and your baseline model. Mentioning “comparable scores” in Sect.5.1 for kidney segmentation is true but not for kidney tumors. 4 - Is missing in Tab.2 a configuration comprising the protuberance detection network without CECT images.

Minor comments: 5 - In Sect.3.3, you indicate that you train the complete network jointly but freeze the parameters of the second network. You could explicitly mention that the learnable parameters of the base network are updated during step 3. Moreover, one wonders if a one-pass end-to-end training could be possible? 6 - Both base and fusion networks employ a combination of Dice and cross-entropy as loss function. Why not using the same combination to train the protuberance detection network (instead of using cross-entropy only)? 7 - The justification for preferring summation to concatenation to avoid from ignoring all outputs from the protuberance detection network (Sect.3.3) does not seem detailed enough. 8 - When building the synthetic dataset, steps 1 and 2 could be grouped together (Sect.3.2). 9 - Are CECT and NCCT scans already registered to be able to simply transfer segmentation masks from CECT to NCCT? 10 - Adding CECT images contributed for the increase in performance because it allows to exploit a larger amount of CT scans. You may discuss about the learning transferability between CECT and NCCT segmentation tasks. 11 - Taking into account average symmetric surface (ASSD) and Hausdorff distances additionally to Dice, sensitivity and FPs/image could strengthen the experimental part. 12 - The comparisons between the different ablated versions of your pipeline could be confirmed using a statistical analysis through t-tests. 13 - Typos and bad formulations: - mentioning “compared to the previous method” in the abstract is enough precise
- “patient’s prognosis” instead of “patients’ prognosis” in Sect.1
- “protuberances in the kidneys” instead of “protuberances in the kidney” in Sect.1
- “from each challenge” instead of “from the each challenge” in Sect.2
- “cross-entropy” instead of “cross entropy in Sect.3.1, 3.2 and 3.3
- “To enable a segmentation of protruded regions only” instead of “To enable a segmentation of just protruded regions” in Sect.3.2
- The sentence “This is because other from the synthetic dataset, we will […]” should be reformulated
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Innovative way to extract protuberances from kidneys
- Pioneering work for kidney tumor segmentation on NCCT scans
- Encouraging quantitative and qualitative delineation results
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Strength: 1) A novel framework is designed to segment kidney tumors by developing protuberance detection network and related synthetic dataset. 2) It is probably the first work to apply deep learning methods for kidney tumor segmentation on non-contrast CT images. 3) Validation experiments are sufficient.

Weakness: Please follow review comments of all three reviewers to improve the clarification in the camera-ready version.

Author Feedback

N/A

back to top

Segmentation of Kidney Tumors on Non-Contrast CT Images using Protuberance Detection Network