List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Aimon Rahman, Wele Gedara Chaminda Bandara, Jeya Maria Jose Valanarasu, Ilker Hacihaliloglu, Vishal M Patel
Abstract
Due to imaging artifacts and low signal-to-noise ratio in ultrasound images, automatic bone surface segmentation networks often produce fragmented predictions that can hinder the success of ultrasound (US)-guided computer-assisted surgical procedures. Existing pixel-wise predictions often fail to capture the accurate topology of bone tissues due to a lack of supervision to enforce connectivity. In this work, we propose an orientation-guided graph convolutional network to improve connectivity while segmenting the bone surface. We also propose an additional supervision on the orientation of the bone surface to further impose connectivity. We validated our approach on 1042 in vivo US scans of femur, knee, spine, and distal radius. Our approach improves over the state-of-the-art methods by 5.01% in connectivity metric.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_40
SharedIt: https://rdcu.be/cVRyU
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This study proposes a method to segment bone structures from ultrasound imaging. In contrast to widely used pixel-wise prediction, the proposed method relies on an orientation-guided graph neural network to perform the segmentation of bone surface. The proposed method was compared against a few other neural net based methods such as UNet and MFG-CNN. The proposed study uses 1042 images and 20% of the images were designated for testing the algorithms.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The author(s) proposed an orientation-guided graph convolutional network for bone surface segmentation.
- The proposed method outperforms UNet and a couple of other machine learning methods on the test set.
- Overall, the paper is written and organized very well.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The paper does not report inter or intra rate variability. Bone segmentation from ultrasound imaging is quite challenging and often results in a low inter-rater agreement. The impact of the uncertainty of the ground truth segmentation on the final quantitative evaluation is not clear.
- The study does not seem to utilize any data augmentation techniques. Other studies have shown that the performance of UNet and other machine learning algorithms could be improved through data augmentation.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Although the authors indicated that the data and code would be available publicly, no information is provided in the paper. In other words, the paper does not indicate when and how the code or data will be made available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- It might be worth including the inter or intra rater variability for the test set as it will allow the readers to better understand the performance of the proposed neural network.
- It might be worth assessing the impact of data augmentation on the overall performance of the proposed neural networks.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Overall, the study is designed well.
- The study clearly indicates the benefits of the proposed method in comparison to other machine learning methods
- The paper is well organized
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
A method for bone surface segmentation which addresses segmentation discontinuity is proposed, based on orientation-guided graph convolution network.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The dataset is fairly large. The method has been compared with the common UNet approach and 2 other methods used in the literature on bone surface segmentation. Evaluation is fairly thorough with suitable metrics.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors can strengthen the justification by describing the consequences of surface discontinuity. Also, some terminology needs to be better explained to the unfamiliar reader earlier in the text, such as orientation and connectivity.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors provide details of the method used, however it is not clear if the data itself is sharable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
It would help the reader if the authors explain what they mean by ‘bone connectivity’ or ‘bone shadow’ where it is first mentioned. Perhaps a figure would help. Also, it would help the reader to understand the effect of disjoint segmentation if authors describe how that would affect CAOS procedures. What level of discontinuity would affect the results? Looking at fig 3, row 2 method MFG-CNN the slight discontinuity does not appear harmful if guiding the surgeon is the goal. Perhaps explanation of the severity of the consequences can help the reader better understand the significance of the solution. Pg 2 “next we propose utilizing orientation as an …”. Can you please clarify the orientation of what with respect to what? Also in pg 3. “By tracing them in a specific orientation”. It is not clear what orientation means here. 2.1. Please clarify what portion of this is the authors’ novelty and which is existing work. Pg 3. “Existing bone segmentation networks only utilize…” Please provide reference. Please read the text to correct some typos. The dataset consists of images from ‘healthy’ volunteers (I assume this means healthy from an orthopedic point of view). Additionally, in the introduction, application for orthopedic surgery is mentioned. I’m curious to know whether and how images from patients requiring orthopedic surgery would differ from the dataset used. If it does differ (e.g. non smooth bone surface?) how the performance of the algorithm would be. In what situations does this method not work well?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Some sections need more clarification to better understand the work and its significance.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper describes a graph convolutional network approach to segmenting bone surfaces in ultrasound images, using orientation information to improve the performance and quantifying the connectivity of the resulting segmentations to ensure realistic results are produced. The paper compares results to three other existing approaches to this problem, showing improvements in connectivity and traditional Dice score.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-Experimental Rigour: The experiments presented in this paper are thorough, assessing the proposed approach with both the connectivity metric and traditional Dice score, including comparison to existing approaches and an ablation study, as well as providing many details about the implementation and hyperparameters used.
-Novelty: Although this problem has been investigated in other works, this paper provides a new approach with novel features, including the incorporation of orientation information.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Clinical Motivation: The unmet clinical need addressed by this work is not well-described or clear to the reader.
Discussion: The Discussion section does not provide much insight to the reader, beyond what has already been described elsewhere in the paper. The limitations of the proposed approach are not addressed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper has a reasonable level of repeatability, providing details about the hyperparameters and systems used. The authors also indicate that code will be made available. The dataset appears to be acquired internally but the authors indicate in the reproducibility checklist that the dataset could be accessed by others.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Overall, this paper was well done with detailed methodology and experiments described clearly.
Major revisions: 1) The clinical motivation for the task is not well-explained in the Introduction. Only the first two sentences introduce the clinical problem but the unmet clinical need is not clear and should be described more fully to allow the reader to understand the clinical relevance. 2) Limitations of the proposed approach should be acknowledged and described in the Discussion.
Minor Revisions:
Abstract: -Second last sentence: “in” is missing before “vivo” -Second last sentence: Define all acronyms on first use (US has not been defined)
Introduction: -Paragraph 2, Sentence 2: The sentence beginning with “Note that the difference…” is very unclear. Please rephrase and clarify to strengthen the rationale. -Paragraph 3, Sentence 3: Replace “good” with “better”. -Last sentence: “in” is missing before “vivo”, as in the Abstract.
Discussion: -Ablation Study should be with Experiments and Results rather than in the Discussion section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work presented in this paper was performed rigorously and is described with a thorough level of detail. This work would be of interest to the MICCAI community, appealing to both the MIC and CAI backgrounds. The paper could be strengthened by more complete descriptions of the clinical motivation and limitations of the proposed approach.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Strength
- An orientation-guided graph convolutional network (GCN) for segmentation of ultrasound images.
- A thorough investigation of the present method on images of different structures
Weakness
- Following Fig. 2, there is a clear class imbalanced problem that is not solved in the present work, i.e., the number of foreground pixels is significantly smaller than the number of background pixels. The authors should justify the usage of the Binary Cross-Entropy (BCE) loss in such a situation.
- The structure of the paper can be improved. For example, Eq. (6) – (8) may be better presented in Section 2 instead of Section 3
- Probability maps of both segmentation and orientation predictions should be presented.
- The distribution of the bony structures such as the knee, the femur, the spine and the radius should be presented because different structures may present different segmentation challenges.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
N/A