Authors

Seung Yeon Shin, Ronald M. Summers

Abstract

Small bowel path tracking is a challenging problem considering its many folds and contact along its course. For the same reason, it is very costly to achieve the ground-truth (GT) path of the small bowel in 3D. In this work, we propose to train a deep reinforcement learning tracker using datasets with different types of annotations. Specifically, we utilize CT scans that have only GT small bowel segmentation as well as ones with the GT path. It is enabled by designing a unique environment that is compatible for both, including a reward definable even without the GT path. The performed experiments proved the validity of the proposed method. The proposed method holds a high degree of usability in this problem by being able to utilize the scans with weak annotations, and thus by possibly reducing the required annotation cost.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_53

SharedIt: https://rdcu.be/cVRy7

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper introduces a RL based method to extract the path of the small intestine from ct images
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

the method appears to be performing well. to the best of my knowledge this is a novel application of RL for this problem set
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

there is little discussion on related literature - algo 1 could have been better analysed in text - It is not clear why RL is the solution here and not a wall aware graph method or topological method. Can the authors please expand on the motivation of their chosen methodology
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Paper appears to be at an acceptable level with regards to reproducibility
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

See above
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

There are no major errors in the approach - paper appears to be interesting for the community
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Somewhat Confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

A reinforcement learning framework has been setup that utilises ground truth segmentation data and optionally ground truth path data to learn path tracking. The reward is computable even without the ground truth path, thus making this optional during training. Experiments were performed using CT data and ground truth labelled by a human observer. The results show improvements compared to other techniques, however, the lack of statistical tests limits the conclusions that may be drawn.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The manuscript is well-organised and information is easy to find within the respective sections
- Sufficient details have been provided in order for the work to be reproducible
- The problem being solved is well-motivated and properly justified using previous work
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is not obvious what the task of ‘path tracking’ means (without reading other cited work) and a complete definition needs to be added in order to make the paper self-contained
- Some unclear sentences e.g. “They were done during the portal venous phase” (better to clarify what ‘they’ means); “to include from the diaphragm through the pelvis” (to ‘include’ what?)
- Some informal wordings e.g. “It would be nice if”
- Some undefined terminology e.g. “One more termination condition of zero movement” (the condition needs to be explained further; is it when the agent converges to a steady answer?); “wall detection as input” (I appreciate the explanation in the following sentences but what is exactly is used as input? Is it a mask of the image?)
- Some claims made without proper evidence e.g. “Euclidean distance to the closest point on the GT path, which is used in [20, 9], would be inappropriate for our problem” (I appreciate the explanation that follows but some preliminary experiments to demonstrate this would have been nice to see)
- Statistical test results for comparisons help to justify claims such as “increase the performance” and provide information about comparisons, however, these are not performed
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Sufficient details have been provided in order for the work to be reproducible
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

To make the manuscript more complete and to justify the conclusions that have been drawn, the authors should address the comments made. Specifically comments about statistical tests since without these tests drawing any conclusions is difficult. Moreover, some sentences need clarification and some definitions need to be added in order to make the paper self-contained. Please refer to the comments above for further details about the points mentioned here.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The major weakness of the paper is in justifying the results using statistical tests, if that is addressed along with some other comments made above, I think this paper would be more complete.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

3
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper presents a DRL framework to extract the centerline of the small bowel. The proposed framework can be trained on different annotations, i.e., bowel segmentation and bowel path.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. It is the first DRL based method to track small bowel path, and the idea of enabling the framework to train on different types of annotations is interesting.
2. A customized reward calculation algorithm that incoorperate the wll detection and geodesic distance transform.
3. Comparison to a wide variaty of other methods (graph-based, DRL-based, and supervised learning-based methods)
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The metrics to evaluate the methods are different from those in Ref. [16]. And the reason/motivation of using different metrics is not clear to me.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors will make the code available upon acceptance of paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

Overall, I think it is a good paper. The only concern is the way to evaluate the predicted paths as mentioned in the weakness part.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

It is a nice DRL paper in the application of path tracking . The authors adapted some related algorithms to make the DRL work well.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

A reinforcement learning framework is presented for small bowel path tracking from MRE data. This is an important and challenging problem highly relevant to the MICCAI community. The utilization of reinforcement learning is novel. Overall reviewers were positive about the paper. Some important issues include: Using evaluation metrics that are different from previous literature on this topic without proper justification makes it hard to compare methods. Lack of statistical test to justify the conclusions drawn from the experiments and improvement of the writing style.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

3

Author Feedback

We appreciate the reviewers and the area chair for their time and valuable comments. We address the issues raised regarding the motivation of using reinforcement learning (RL), evaluation metrics, statistical tests, and the detailed explanation of the method.

Regarding the motivation of using RL in the proposed method (R#1, ‘It is not clear why RL is the solution ~’), in view of the exceptionally high annotation cost of the ground-truth (GT) path, our objective is to utilize scans that have only GT segmentation as well as ones with GT path to train a tracker for the small bowel (Fig. 1). The same objective can be pursued in different formulations, and it was realized using RL in this work. More specifically, it was achieved by designing a unique environment that is compatible regardless of whether the GT path is available or not. Our formulation was validated by comparing with a supervised learning method [11] and a graph-based method [16] as well as a previous RL method [20] in Table 1.

Regarding the evaluation metrics used (R#3, ‘The metrics to evaluate the methods are different from those in Ref. [16].’), we used one of the metrics that are used in [16], which is the maximum length of the GT path that is tracked without making an error, which we believe is more relevant than the others. As explained in Section 2.4, we opted not to use the others, including the precision, recall, and curve-to-curve distance, since they are computed in disregard of the order of the tracking, and the error of crossing the walls. Despite they could indicate the coverage of the predicted path on the GT path, decent values can be obtained even for a predicted path that is totally incorrect in terms of the tracked order in some cases. Instead, we provided more statistics on the chosen metric in Table 1.

Regarding statistical tests (R#2, ‘Statistical test results for comparisons help ~’), we performed paired t-tests between the proposed method, ‘Ours (p+s)’, and the others. P-values of 6.109 x 10^-5, 1.747 x 10^-5, 0.003, 0.100, and 0.016 were obtained for ‘DT [11]’, ‘Zhang et al. [20]’, ‘TSP [16]’, ‘Ours (s) (0/20)’, and ‘Ours (p+s) w/o wall’, respectively.

Regarding the comment ‘the condition needs to be explained ~’ from R#2, the zero movement literally means that no movement or zero displacement was predicted from the actor network at the current position, as explained in Section 2.2 and Algorithm 1. Since the zero movement does not change the position of the tracker in test time and therefore the actor will predict zero movement constantly from the unchanged state (or position), we terminate the episode.

Regarding the comment ‘what is exactly used as input?’ from R#2, Fig. 2 visualizes example input patches for the networks. The wall detection map is computed using the Meijering filter [10] as explained in Section 2.2. The cumulative path and the GT path are binary, and the other inputs are non-binary.

Regarding the comment ‘Some claims made without proper evidence ~’ from R#2, our conjecture on the Euclidean-distance-based reward function was confirmed by the quantitative comparison in Table 1, which corresponds to ‘Zhang et al. [20]’.

We will clarify the relevant parts according to the reviewers’ comments in our camera-ready paper.

back to top

Deep Reinforcement Learning for Small Bowel Path Tracking using Different Types of Annotations