List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Aimon Rahman, Jeya Maria Jose Valanarasu, Ilker Hacihaliloglu, Vishal M Patel
Abstract
Segmenting both bone surface and the corresponding acoustic shadow are fundamental tasks in ultrasound (US) guided orthopedic procedures. However, these tasks are challenging due to minimal and blurred bone surface response in US images, cross-machine discrepancy, imaging artifacts, and low signal-to-noise ratio. Notably, bone shadows are caused by a significant acoustic impedance mismatch between the soft tissue and bone surfaces. To leverage this mutual information between these highly related tasks, we propose a single end-to-end network with a shared transformer-based encoder and task independent decoders for simultaneous bone and shadow segmentation. To share complementary features, we propose a cross task feature transfer block which learns to transfer meaningful features from decoder of shadow segmentation to that of bone segmentation and vice-versa. We also introduce a correspondence consistency loss which makes sure that network utilizes the inter-dependency between the bone surface and its corresponding shadow to refine the segmentation. Validation against expert annotations shows that the method outperforms the previous state-of-the-art for both bone surface and shadow segmentation.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_32
SharedIt: https://rdcu.be/cVRvX
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a incremental innovation for bone segmentation from US images. The main contribution is the cross task transfer block that improve the performance of the overall segmentation for both bone and shadow.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is well-written, clear and follow a proper methodology. As far as I can understand, the implementation is consistent and the results good after a proper comparison with previous algorithms.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Unfortunately, with my experience I cannot evaluate the novelty of this work in terms of programming. The CTFT looks for me similar to a GAN approach but a bit different. The rest of the proposed method is based on well-known previous studies.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No comments.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
No additional recommendations. US segmentation is a challenging task and this proposal will increase (not so much, but at least a bit) the current performance.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please, keep my recommendation under the others reviewers comments regarding to novelty. The paper is clear and the methodology consistent. An overall (small) improvement on the results are presented. All these together compile a good paper.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
- Multi-task segmentation with the use of a new cross-task feature transfer block
- Proposed a task correspondence consistency loss to improve multi-task learning
- Validation on images of different anatomies and different ultrasound scanners.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Novel framework for multi-task learning
- Thorough validation and ablation studies
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Small number of subjects
- Ground truths segmentations for curvilinear probes appear problematic
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Network architecture and training information are clear and helpful for reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The number of subjects is small. This can result in low anatomical variability, and it is unclear if the subjects are healthy controls or patients.
- It is not clear how Section 2.1 is related to the segmentation ground truth.
- For the curvilinear probe, the shadows should not be in the vertical direction parallel to the length of the image. Instead, they should follow a fan-shape along the transmission direction of the ultrasound beams. This can affect the validity of the trained network.
- Some discussion regarding the requirement of accuracy vs. the impact of the application will be appreciated.
- The term “mutual information” used in the article is not appropriate since it does not refer to the more commonly used information-theory-based metric.
- Figure 5: (d) and (e) are not consistent between the first and second rows regarding the surface and shadow segmentation.
- It will be good to mark which anatomical structures are shown in the figures.
- For the Ablation study in Table 2 and 3, it is unclear if the improvement with CTFT is statistically significant.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the framework is novel and interesting. The problematic ground truths can affect the validity of the work.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors should clearly state that the approximation/simplification made for the ground truth of the shadows related to the curvilinear probe in the text.
Review #3
- Please describe the contribution of the paper
Bone segmention from US images, taking into account the bone surface and shaow artiffact. The study is presente in the context of image-guided orthopaedic intervention, but it really is yet another segmentation paper.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method presented explicitly considers bone surface and shadow artifact, in the segmenation of bone structure in US images. The study involved IRB-cerfitified data collection, plus three additional virgin data not included in the network development. The wriging is clear and purposefully highlights the key items for sucessfull MICCAI submission.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The method is argued to be inteded for CAOS, but not specific condition, and strategy for clinical implementation is given. The subjects are all healthy subjects, not particulary represeting the image features of US images taken in CAOS procedures.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
OK. NO particular comment.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper can be improved by collecting data from disease subject.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The clarify of the paper is outstandig.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
LImited validation results, with respect to data size and the added clinical value due to the introduced novelty needs to be further justified, as suggested by the reviewers. Please look at comments from all reviewers, including those with positive overall recommendations.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
nr
Author Feedback
We sincerely thank reviewers for their valuable feedback. In what follows, we provide clarification to points raised by the reviewers.
1) Problematic ground truth (R2): R2 is correct in their assessment of the shadow regions for the curvilinear probe. However, the ultrasound probe used in our work is a convex 2-5 Mhz transducer with a shallow depth setting (5-6cm). For this transducer and with this depth setting the spine shadow regions resemble more of a linear image rather than a fully convex shape (Reviewer’s assumption would be more suitable if the image acquisitions were performed using a phased array transducer, micro convex transducer, or convex transducer with an increased depth setting). Therefore, our expert ultrasonographer segmented bone shadow regions similar to a linear transducer. Similar spine shadow segmentation can also be found in [12].
2) Accuracy vs. Clinical impact (Meta-Reviewer, R2): The overall system accuracy of CAOS procedures is a combination of system errors such as segmentation, registration, tracking, and US probe calibration. Therefore, any improvement in bone segmentation accuracy would have an impact on the overall accuracy of the US-guided CAOS system. Required accuracy limits will depend on the surgical procedures. For example, for spinal fusion surgeries an overall accuracy of <2mm would be required.
3) Dataset size and Validation (Meta-Reviewer, R2, R3): The dataset consists of 1042 bone images from 25 subjects, which is indeed an adequate dataset size for medical image segmentation. Previous works in literature also have a similar number of images in their dataset [1-3, 11-13].
4) Mutual information (R2): We apologize for the misuse of the term “mutual information”. We will replace it with the word “complementary features” which we believe is more appropriate.
5) Section 2.1 (R2): Section 2.1 is about enhancing the input image before sending it into the network. The extracted filtered images can be viewed as feature maps that provide different local information of bone surface in an US scan. Local phase tensor and images enhance the bone surfaces located deeper in the ultrasound image and mask out soft tissue interfaces. Bone shadow enhanced image maximizes the visibility of high intensity bone features inside a local region. Multi-feature guided networks bring ultrasound scans from different machines to a common independent domain.
6) Anatomical structure (R2): We will make sure to mark the anatomical structures in the revised version.
7) Importance of CTFT (R2): We would like to emphasize that adding the CTFT block improved the performance by 2% and 1.5% in terms of F1 score for surface and shadow in SSNet. Also for joint-Unet, we obtained a 1% and 3% improvement after adding CTFT. We believe these experiments show the effectiveness of CTFT as it shows a significant performance improvement for two different base networks over two different segmentation tasks.
8) Clarification of Figure 5 (R2): In Figure 5:(d) top row corresponds to output from an individual bone surface segmentation network and (e) corresponds to cascaded shadow segmentation output generated using the segmentation from individual network. In Figure 5: (d) bottom row corresponds to output from an individual bone shadow segmentation network and (e) corresponds to cascaded surface segmentation output generated using the segmentation from individual network. As we wanted to show how cascaded network can give errors in segmentation for both bone and shadow, we showcased one using surface segmentation network and one using shadow segmentation network in (d). We will add more explanation in the revised version to avoid the confusion.
9) Concern of Novelty (R1): We would like to emphasize that both R2 and R3 have pointed out the novelty of our framework and method as a strength in their reviews.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Citing other papers for proving the data set is sufficient may not be considered a good argument, however the work indeed reported meaningful statistical test results alone variance of the segmentation, which does show the usefulness of the improvement.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper certainly has a lot of merits and there is a strong clinical motivation for the paper. As pointed out by the reviewers, a diverse and larger dataset with the aberrant anatomy is important to train the networks. I suggest the authors include a more diverse dataset in their future publications. As such, this is a strong contribution to MICCAI.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The strength of the paper is the proposition of a new method for ultrasound image segmentation by considering both the bone surface and the corresponding acoustic shadow. Authors’ rebuttal appropriately addressed reviewers’ concerns.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1