Authors

Xiao Huang, Xiaodong Yue, Zhikang Xu, Yufei Chen

Abstract

Segmentation of bladder tumors from Magnetic Resonance (MR) images is important for early detection and auxiliary diagnosis of bladder cancer. Deep Convolutional Neural Networks (DCNNs) have been widely used for bladder tumor segmentation but the DCNN-based tumor segmentation over-depends on data training and neglects the clinical knowledge. From a clinical point of view, a bladder tumor must rely on the bladder wall to survive and grow, and the domain prior is very helpful for bladder tumor localization. Aiming at the problem, we propose a novel bladder tumor segmentation method in which the clinical logic rules of bladder tumor and wall are incorporated into DCNNs and make the segmentation of DCNN harnessed by the clinical rules. The logic rules provide a semantic and friendly knowledge representation for human clinicians, which are easy to set and understand. Moreover, fusing the logic rules of clinical knowledge facilitates to reduce the data dependency of the segmentation network and achieve precise segmentation results even with limited labeled training images. Experiments on the bladder MR images from the cooperative hospital validate the effectiveness of the proposed tumor segmentation method.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_69

SharedIt: https://rdcu.be/cVRwW

Link to the code repository

https://github.com/huangxiao1234/miccai2022-BTSLCK.git

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

1.This paper proposes a novel bladder tumor segmentation method by fusing clinical logic rules of bladder tumor and wall,the rules can guide the DCNN to produce precise segmentation results 2.This paper validate that fusing the logical clinical knowledge is helpful to reduce the data dependency of DCNNs for image segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The motivation of Integrating clinical rules into bladder tumor segmentation methods is suggestive. 2.The neural network is very cleverly designed, and the figures in this paper are well-drawn. 3.The transformation of graph neural network is described very carefully, and the definition of loss function is innovative.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I think the main weakness of this paper is, lack of literature review and comparative experiments on clinical logic rules fusing method.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The overall method is explained clearly, but the details of clinical knowledge integration into the network are not clear enough
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. The paper should be compared with more diagnosis rules embedded methods, for example “Integrating Diagnosis Rules into Deep Neural Networks for Bladder Cancer Staging”. 2.Describe the details of the methods by which clinical knowledge is integrated into the network.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

It is instructive to integrate clinical logic rules into segmentation tasks using GNN,so I vote to accept.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The authors propose a novel bladder tumor segmentation method in which the logic rules of clinical knowledge are incorporated into DCNNs.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The method is novel and provides an original way to use clinical data. The paper and the figures are clear. The authors provide comparison with other state-of-the-art methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The authors should provide more information about some parameters. In particular they should explain how the parameter beta was obtained. Was the value obtained empirically? (The narrow range 0.9-1 should be justified. The value 0.01 in the Lsegment formula is not justified. Were the experiments (groundtruth) validated with clinicians? There is a lack of information regarding to that issue.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is not available but the authors guarantee that they will provide it if the paper is accepted. In that case I do not have other way to proof the reproducibility. The method described looks clear and it looks as reproducible, but the results cannot be checked without the code and the dataset.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

The method is novel and provides an original way to use clinical data. The paper and the figures are clear. The authors provide comparison with other state-of-the-art methods. The authors should provide more information about some parameters. In particular they should explain how the parameter beta was obtained. Was the value obtained empirically? (The narrow range 0.9-1 should be justified. The value 0.01 in the Lsegment formula is not justified. Were the experiments (groundtruth) validated with clinicians? There is a lack of information regarding to that issue.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is novel and provides an original way to use clinical data. The paper and the figures are clear. The authors provide comparison with other state-of-the-art methods.
Number of papers in your stack

3
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The main idea of this paper is to merge the clinical logic rules with Deep Convolutional Neural Networks (DCNN) to enhance the segmentation of the bladder tumor. The clinical logic rules depend on the appearance view of a bladder tumor must rely on the bladder wall. While the standard U-net is used for DCNN to create semantic segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the paper is well-written and not organized. -There is a novelty, not big, for fusing DCNN with clinical logic rules.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-please mention the results in the abstract.
- the author should write precisely about the bladder tumor. What challenges did you face when segmenting the bladder tumor in the introduction?
- The introduction sections contain a limited number of the related work for bladder segmentation, only three. However, there are many publications for bladder segmentation in the top journals. Besides, it would help to write at least a paragraph about related work for merging the DCNN with clinical logic rules, not necessarily with bladder segmentation but with any other applications.
- You wrote in your manuscript, “To the best of our knowledge, the related works of involving logic rules into DCNNs for medical image segmentation are very limited.” You did indicate any current model in your manuscript. What is the difference between your model and the current one?
- How did you estimate the value of the variable β?
-construction logic rules for the bladder tumor are not clear enough for me. Would you please rewrite or reconstruct the figure for it, Fig. 2?
- many aspects of the method without references, which means you created everything equations, ideas!!
-The author compared the segmentation results with only the general techniques, not current state-of-the-art approaches for bladder tumor segmentation. They compared their results with only one reference with number 14 related to bladder segmentation. However, many recent works are published in the top tier.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

NA
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- If the author is the first one who applies this idea, it will be a good seed with more analysis for the top journal paper. -May be adding the wight for equation one be adding α before Lsegment and tuning the two available α and β will enhance the segmentation accuracy.
- The data for bladder segmentation is imbalance. I think you should select a loss function that can face this challenge, such as Tversky loss, which will be better than cross-entropy.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- I found a novelty if he is the first to create it with perfect organization.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposes a novel bladder tumor segmentation method which incorporates the logic rules of clinical knowledge into the segmentation task (using DCNN). The idea is novel and good. The network is very cleverly designed providing then an original way to use clinical data for helping the segmentation. Authors are advised to improve the article in the final version by enriching the literature review on bladder segmentation and comparative experiences on logic rule fusion method for clinical knowledge, and also by providing more information on the beta parameter and more details on the method by which clinical knowledge is integrated into the network. Could the authors explain why they compared segmentation results only with general techniques, and not with current state-of-the-art approaches for bladder tumor segmentation?

Minor problems: 1) “in (tumor, box)” in page 4, what is “box”? it is rather “wall” here? 2) In Fig.2. b, the parenthesis is missing for “(exist(tumor)=>exist(wall)) AND in(tumor, wall)” and for (p1 =>p2) AND p3. Without the paracentesis, the result is different.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

1

Author Feedback

We sincerely thank all the reviewers and AC for their time and constructive comments on our manuscript. Below we provide responses on the comments.

1.Questions about comparative experiments

Response: Meta Reviewer (MR), Reviewer 1 (R1) and 3 (R3) are concerned with the comparison of the proposed model with other SOTA methods for bladder tumor segmentation. In our current comparative experiments, we aim to validate the segmentation improvements caused by the logic clinical rules and thus focus on comparing the methods fusing and without the clinical logic rules. For more comprehensive analysis, we will add experiments to compare our method with other SOTA bladder tumor segmentation methods, including BW-Net (EMBC2020), and MD-Unet (CBC2021). In addition, R1 suggests adding a comparison experiment with the paper “Integrating Diagnosis Rules …”. However, this method is used to classify the stage of bladder cancer, and cannot be directly used for segmentation. We will add this paper in the related work of our paper.

2.Questions about parameter setting

Response: There are two main hyper-parameters in the paper. The parameter beta is used to control the proportion of rule loss in the total loss, and the value should not be too high to avoid affecting the segmentation performance. The other is the coefficient of dice loss. We determined the two parameters based on the parameter validation experiments which are not listed in the paper due to the length limitation. Moreover, in the validation experiments, we find that the two parameters are not sensitive to influence the segmentation performances.

3.Questions about the logic rule integration

Response: MR, R1 and R3 concern about the details of the logic rule integration into the network. Due to the paper length limitation, we just briefly introduce the strategy of logic rule integration. Here we further interpret the GCN pre-training process. Based on the truth table and the propositional formula, we labeled each input segmentation mask as positive or negative. If the segmentation mask corresponds to a true value, it is labeled as positive, otherwise it is labeled as negative. Besides the ground-truth segmentation masks in the training dataset which are labeled as positive, we augment a number of negative segmentation masks through making the masks violate the rules. Since the augmented masks are generated based on clinical logic rules, they are more interpretable than the traditional image augmentation methods. Finally, the GCN is pre-trained by classifying whether the input segmentation mask is true (positive) or not (negative). We will further clarify these details in the final version.

4.Other detailed questions

Response: (1) MR asks about two minor problems. First, it is an oversight on our part that it should be in(tumor,wall). Second, the proposition formula in the example should have no parentheses. (2) R3 suggests mentioning the results in the abstract and challenges of bladder tumor segmentation in the introduction. (3) MR, R1 and R3 suggest to enrich the related works. We will revise the paper according to the suggestions (1) to (3). (4) R3 asks about the difference between our model and the current methods combining logic rules. For example, the diagnosis rules constructed in the method “Integrating Diagnosis Rules …” only express the extent of tumor infiltration, and are used as the constraints of the optimization objective. In contrast, clinical logic rules constructed in our method can provide more semantic descriptions of bladder tumors and express various relationships between bladder tumor and wall, e.g. the positional and existential relationships between bladder tumor and wall. Moreover, based on GCN, we can map the clinical logic rules and segmentation results into the same feature subspace. Through measuring the distance between the feature vectors of logic rules and segmentation results, we can utilize the clinical logic rules to guide the segmentation.

back to top

Harnessing Deep Bladder Tumor Segmentation with Logical Clinical Knowledge