List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Magdalena Bachmaier, Maximilian Rohleder, Benedict Swartman, Maxim Privalov, Andreas Maier, Holger Kunze
Abstract
Standardized image rotation is essential to improve reading
performance in interventional X-ray imaging. To minimize user inter-
action and streamline the 2D imaging workflow, we present a new au-
tomated image rotation method. Image rotation can follow two steps:
First, an anatomy specific centerline image is predicted which depicts
the desired anatomical axis to be aligned vertically after rotation. In a
second step, the necessary rotation angle is calculated from the orien-
tation of the predicted line image. We propose an end-to-end trainable
model with the Hough transform (HT) and a differentiable spatial-to-
angular transform (DSAT) embedded as known operators. This model
allows to robustly regress a rotation angle while maintaining an explain-
able inner structure and allows to be trained with both a centerline seg-
mentation and angle regression loss. The proposed method is compared
to a Hu moments-based method on anterior-posterior X-ray images of
spine, knee, and wrist. For the wrist images, the HT based method re-
duces the mean absolute angular error (MAE) from 9.28° using the Hu
moments-based method to 3.54°. Similar results for the spinal and knee
images can be reported. Furthermore, a large improvement of the 90th
percentile of absolute angular error by a factor of 3 indicates a better
robustness and reduction of outliers for the proposed method.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_42
SharedIt: https://rdcu.be/dnwLX
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #3
- Please describe the contribution of the paper
- The author proposed a line segmentation method with the Hough Transform and get better result compared to direct regression methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper explored better ways to calculate angles with Hough histogram and test method on datasets of multiple body regions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The method of the article lacks innovation, there are many good works to solve line segment parameter regression and extract line features, e.g. “Deep Hough Transform for Semantic Line Detection”. In addition, independent evaluation of angle and displacement is not enough, should consider both Euclidean distance and angular distance between a pair of lines.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The author should add more comparison methods.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The Hough space is discrete, the getting line precision is limited. The lines should be further refinement from line segmentation probability map, such as weighted least squares.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the paper is limited.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
Authors propose an E2E trainable model for robust transformation of Orthopedic X-Ray images using the Hough Transform
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Well organized
- Comprehensive analysis
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Authors do not comment on generalizability across vendors
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Not reproducible since training dataset is not publicly available
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Recommend the authors to demonstrate clinical feasibility by testing on prospectively acquired images, and reporting on inference durations on a clinical console-matched hardware spec
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Well motivated
- Well organized
- Claims in the title are realized
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #5
- Please describe the contribution of the paper
This paper proposes an end-to-end trainable model for rotation of interventional X-ray images, substituting Hu moments with a Hough transform. Authors validated their proposed method on three different body regions (spine, wrist, and knee) and showed a performance improvement and robustness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
This paper proposes an end-to-end trainable algorithm trained with the weighted sum of objectives: angle regression and segmentation. Even though this was inspired by the literature, results demonstrated the performance improvement of using HT instead of HM.
-
For the weighted sum of losses, they showed the ablation study using different weighting strategy, which is interesting.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
Even with a novel approach using HT with segmentation task for the indirect measurement for the rotation, it is not clear that the contribution of this study is novel, since the entire concept of using segmentation is from the literature.
-
It is not clear how improving the regression rotation task (MAE from 9.28 degree -> 3.54 degree) can affect the outcome in the interventional X-ray imaging.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It is clear that they provided experimental settings for training the models, such as computational resources, hyper parameters for training the models, details of augmentation, etc.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
In Table 1, it is unclear that the adapted loss function is better than non-adapted loss. As stated in the discussion section, I think it would be better to normalize the magnitude of each loss to see the impact of lambda.
-
How is the performance of segmentation task correlated to the performance of the rotation regression task? It is clear that the joint training of segmentation and rotation regression can help improving the rotation regression task, but not clear with the segmentation task.
-
In the first paragraph of section 2.1, there is a typo festures -> features (I guess).
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I’m not clear with the novelty of the method. However, the performance gap between HM and HT is significantly large.
- Reviewer confidence
Not confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes to predict the orientation angle necessary to orient interventional x-ray images in a standardized coordinate system for better reading during interventions via a deep learning based orientation angle estimation. It proposes to perform a hough transform based line estimation and derives the angle from this line w.r.t. one of the imaging axes.
A strength of this paper is the differentiable formulation of the Hough transform for line estimation, such that it can be included into a neural network in an end to end manner.
As identified by reviewers and meta reviewers, this work shows a number of shortcomings:
- The clinical motivation is vague. It is stated that standardized image rotation is important to “assess fracture reduction”. It is not clear what this means. Also referring to an “increase in overall outcome” is vague. Outcome of what? Moreover, the introduction is full of typos and misspellings making it hard to follow.
- Very importantly, this work is based on a framework of regressing the orientation angle to reorient interventional images quickly and robustly via a neural network architecture predicting a line in the image that corresponds to the main axis (up-down). However, it is not clear why the task of reorienting is not performed using standard computer vision techniques like image registration, either robust intensity based registration, or likely more promising feature and descriptor based robust image alignment. The optimization problem is actually simple after the features and descriptors are computed, since there is only one unkwnown parameter (2D rotation angle) and optimization can be done by exhaustive search for a global solution (or hierarchically using gradient descent on multiple scales). I am convinced that this will work well given a dataset from which features and feature descriptors can be learned (such a training dataset is needed in the proposed approach as well for the neural network, but in a registration setting, there is no manual annotation of lines required!). The basic premise of this work has to be to show that such an approach does not work, analyze why it does not work, and then propose a much more complex deep learning based solution. As a consequence, comparison to such simpler baselines is a crucial missing aspect of this work.
- The proposed method has questionable novelty, since the basic approach of heatmap regression based reorientation was already proposed in [10]. The current work is a modification that leads to slightly better results, however, it follows the same methodology as the previously published work. Authors have to justify more clearly why there is strong methodological contribution in this work.
- As one of the reviewers also noted, another very important issue is the notion of the clinical relevance of the error improvement in interventional X-ray imaging. The use case is to put arbitrarily oriented X ray images into an upright orientation, to enable improved reading during interventions. What is the error w.r.t. perfect upright orientation that is still deemed clinically suitable? I would assume the difference between a 10 degree or 4 degree misregistration is negligible, however, there is no discussion on this important aspect. Maybe an interactive tool where an operator quickly draws a line into the X-ray image for reorientation is already sufficient in terms of the error requirements?
- Parts of the document are confusing. For example, the loss has a segmentation loss component, however, there is no mentioning of this segmentation task prior to loss specification. It seems that this segmentation task should become clear after reading paper [10], however, that paper is behind a paywall and importantly, the submitted paper has to be self contained enough to understand this crucial aspect without having to refer to prior work. Overall, the submitted paper seems to depend heavily on the explanations from [10], which makes it hard to understand and follow.
Authors are asked to clarify in their rebuttal mentioned shortcomings and address additional reviewer comments that they find relevant.
Author Feedback
Dear Area Chair and Reviewers,
We would like to thank you for the review of our paper “Robust Rotation Estimation for X-Ray Images” and your valuable comments on it.
The main points of criticism about our paper are:
Lack of clinical motivation
Missing discussion, why standard image vision algorithms fail on the task
Weak motivation for the novelty of our method
Missing discussion on the benefits of the proposed method compared to state-of-the-art
Missing clarity as the work is based heavily upon [10].
As pointed out briefly in the introduction, the targeted modality for our solution is the mobile C-arm system, which is used in an operating room. Standardized alignment of the acquired images thus helps the surgeon correctly assess the current state of a surgery and subsequently increase the success rates of interventional procedures. In such a scenario, user interaction with the system must be minimized due to sterility considerations. Supporting staff is often not well trained in operating the vendor-specific systems and changes job positions frequently. Thus, even improved user interaction with the system does not improve the situation for the surgeon much. We will update the paper’s introduction with this clarifying information in the camera-ready version.
We acknowledge that in standardized acquisitions such as diagnostic thorax imaging standard vision algorithms are widely used. However, conventional vision algorithms fail to automate the image rotation of intraoperative images due to the enormous variety of images with different collimation settings, fracture types, and instruments/casts that obscure many image features. Thus, distinct feature extraction and registration is an even more complex task for such a scenario. Furthermore, the presented method is a general approach, overcoming the need for body region-dependent features and descriptor extraction. We will extend the introduction to emphasize this.
The novelty of the method is the proposition of a DSNT-like layer for angle regression based on the Hough Transform. As also discussed in the last section, this term helps to overcome label noise with respect to length and exact position of the labelled line: the usage of the Hough Transform instead of the Hu moments helps thus to generate thin, well-defined lines. Hence, the algorithm helps to reduce the number of wrong segmentations results. This feature especially helps to reduce the number of outliers. The resulting improvement of robustness will be added to the contributions section of the paper.
This increase in robustness is more important than improved accuracy. This aspect is already part of the discussion’s first paragraph. The paragraph is amended to reflect this. Additionally, we will insert a paragraph describing the P90 value in the Results section.
The most important part of the paper is the description of the DSAT part of the algorithm. To have sufficient space to motivate and describe the required steps to derive the angle from a Hough transform in a derivable manner, we reference state-of-the-art literature. In the training section, we will add a brief description that the Hough Transform layer and the DSAT layer are appended to the output of the segmentation network, so that an end-to-end training is available, giving the possibility of have a loss assigned to the segmentation network’s result and the angular value. A separate reference to the D-LinkNet paper will also be added decreasing the dependency on [10].
We hope our explanations and descriptions of proposed changes to the paper convince you that the improved version of the current paper is a valuable contribution to the MICCAI 2023 conference.
With kind regards,
The authors
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Authors have addressed the major concerns about this work in the rebuttal. Clinical motivation was significantly strengthened by the argument that interaction with screens is not easily possible in practice. The wide variety of intraoperative situations prevents classic computer vision methods. Also the contribution of the work was clarified and confusing sections in the document were promised to be improved. Given all these proposed revisions are included in the final manuscript, I tend to vote for acceptance - inline with the majority of reviewers - of this paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes a novel end-to-end trainable model using Hough Transform for the rotation of orthopedic X-Ray images. The authors validate their method with three body regions, showing performance improvement and robustness. The reviewers generally agree on the contribution, strengths, and weaknesses of the paper. The authors’ rebuttal addressed most of the raised concerns. The authors’ rebuttal successfully addresses most of the reviewers’ concerns, justifying acceptance with modifications. The contribution, though incremental, is solid and could benefit the medical image computing community.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Pros:
- Topic: The work proposes to predict the orientation angle necessary to orient interventional x-ray images.
- Experiments: Authors validated their proposed method on three different body regions and showed a performance improvement and robustness. Cons:
- Clarity: Parts of the document are confusing.
- Novelty: The basic approach of heatmap regression based reorientation was already proposed
- Movitation: The clinical movitation is vague. After Rebuttal
- the authors failed to convince the reviewer gave low score, but to me, the clinical need and novelty is sufficient for a conference paper;
- the two positive reviews are general consistent to acknowlege the contribution of this work