Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Mingyuan Luo, Xin Yang, Zhongnuo Yan, Junyu Li, Yuanji Zhang, Jiongquan Chen, Xindi Hu, Jikuan Qian, Jun Cheng, Dong Ni

Abstract

Ultrasound (US) imaging is a popular tool in clinical diagnosis, offering safety, repeatability, and real-time capabilities. Freehand 3D US is a technique that provides a deeper understanding of scanned regions without increasing complexity. However, estimating elevation displacement and accumulation error remains challenging, making it difficult to infer the relative position using images alone. The addition of external lightweight sensors has been proposed to enhance reconstruction performance without adding complexity, which has been shown to be beneficial. We propose a novel online self-consistency network (OSCNet) using multiple inertial measurement units (IMUs) to improve reconstruction performance. OSCNet utilizes a modal-level self-supervised strategy to fuse multiple IMU information and reduce differences between reconstruction results obtained from each IMU data. Additionally, a sequence-level self-consistency strategy is proposed to improve the hierarchical consistency of prediction results among the scanning sequence and its sub-sequences. Experiments on large-scale arm and carotid datasets with multiple scanning tactics demonstrate that our OSCNet outperforms previous methods, achieving state-of-the-art reconstruction performance.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43907-0_33

SharedIt: https://rdcu.be/dnwcK

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper presents an approach for tracking estimation of ultrasound sweeps. The work builds up on a previous work that integrated an IMU sensor for motion estimation and an online learning architecture.

    Here, one expands to usage of multiple IMU sensors by adding the sensors in a complimentary manner to each other and designing the learning process such that it takes advantage of this (modal-level self-supervised strategy (MSS)). Additionally, the authors add a self-consistency strategy (SCS). This is realized by randomly sampling and flipping sub-sequences from the original sweep.

    The paper includes appropriate comparison methods and ablation studies.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper tackles the important problem of freehand 3D ultrasound reconstruction. It demonstrates how to leverage multiple IMUs for better motion estimation. IMUs are low-cost, small-sized, easy-to-acquire sensors.
    It also demonstrates a self-consistency strategy that improves the robustness of estimations. The quantitative analysis is appropriate with ablations studies and method comparisons. The method has been tested on two different datasets.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    A statistical test analysis over the quantitative results would be appreciated. All numbers have improved, however, some only values only marginally e.g. the maximum drift in the Arm scans is actually not the lowest in median value. Additionally, standard deviation could be provided. In light of the suspicion that the benefits in accuracy are minor, the pipeline is quite complex.

    Were the CNN and DC2 trained on these datasets? It is not clear from the paper, technical details e.g. optimization with Adam is formulates as if it only applies to the proposed method, there are no details about the training of the rest. Additionally, the mentioned augmentations sound like the self-consistency strategy. Was the data for the other methods not augmented? And if the augmentations are just the self-consistency strategy, this could be briefly noted.

    A short discussion on limitation and difficult cases would be appreciated.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The dataset is not public and it seems it is not going to be made public. This is a downside.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    The method has been tested on two datasets. It would be interesting to see the performance on the combined dataset in addition to the separate ones. This would give inside to generalization ability to different anatomies.

    As mentioned, a short discussion on limitation and difficult cases would be appreciated. A short statistical analysis as well.

    It could be explicitly mentioned that the CNN and DC2 were also trained on the dataset. I assume the augmentations mentioned were not included in the training of those. It would be good to specify the commonalities and differences in the training settings.

    Lastly, one can provide a shorter running title in the latex code such that it is shown at the place of “Title Suppressed Due to Excessive Length”.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Using IMUs for 3D ultrasound reconstruction is a promising idea. The authors present a method how to integrate multiple of those to minimize the errors of their inaccuracies.

    The additional self-consistency strategy appears like a good augmentation strategy.

    There are some open questions that the paper does not answer in it’s current version, though.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes the use of multiple IMU (Inertial magnetic unit) sensors for linear sweep rotation and translation prediction of Ultrasound probes. They claim that redundant information can lead the network to reduce the acceleration noise generated by a single IMU. Besides introducing 3 more IMU, different placed and oriented, produce more signals for the network. They propose 2 online self-supervising strategies: the self-consistency strategy (SCS) and the modal supervise strategy (MSS). The SCS consists of a computational loss calculated between different ground-truth sampling (simulating different speeds and flipping). The MSS during training calculates the loss (MAE+Pearson) between GT and the average of the IMUs. But during test, each IMU data is used as a weak label, to estimate a matching acceleration between all IMU sensors.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The main strengths of this paper are:

    1. Its small error with respect to ground truth: The community seeks to return portability to 3D ultrasound made with 2d+ tracking images. The tracking systems apart from needing precalibration, force the patient to move to a precalibrated room. A method with a small error of less than 3cm in drift and less than 3 degrees in geodesic error is good. 2.The use of losses that efficiently integrate redundant information is very interesting. The authors compared the MSS with contrastive learning. I don’t see that the shape of the losses are similar, but I understand that they are looking to discard those IMU measures that are outliers. 3.The paper is well written and grammatically correct. It is pleasant to read and reflects a concise story.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    One minor weakness is:

    1. The lack of information on how much more the error must be decreased to be acceptable for use in practice. The paper argues that they decreases the drift and geodesic errors. Clearly a lower error generates a better volume reconstruction, but how much more should it be decreased? IMUs have an error integration for long sweeps. If the average sweep length of the arm and carotid sweeps is 32 cm and 20 cm respectively, an end-drift error of 10% implies about 3 cm and 2 cm respectively. Is this acceptable or does your method only work on short sweeps?

    Some mayor weaknes are: Authors do not make a clear comparison between their contribution and previous works.

    1. Did the authors use as baseline the 2 works of Luo et al? (MICCAI 2021 and MICCAI 2022). Because the paper of LUO et al, 2022 uses one IMU sensors for providing the network with additional information. Probably the dataset is different and that explain the difference in the values of the metrics. But the authors did not explain how did they train the comparison methods. How are they sure they train the MoNet as best as possible?
    2. The authors implement MoNet from Luo, but not ConvLSTM. Why?
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The method seems quite difficult to implement. There is not enough information in the paper to reproduce it, specially the network architecture. Authors claim they will free the code which is very positive but even if the code is available the use of 4 sensors introduces many sources of error during testing. Even if other reseachers achieve to get good pre-calibrated values for the IMU, how do they make a good training? Which is the quality of your Imu data?

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    The work is very interesting. The use of varoius IMU sensors to correct the errors is not new idea but is nice to apply it in such movements like the ultrasound sweeps.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    3

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is interesting, they perform better that the baseline methods they compare on and the paper is well written. Nevertheless the idea of using multiple imu sensors for correct the error is not new, the use of the IMU information in a network for freehand ultrasound is not new, the metrics are not new, the dataset is quite similar from the last LUO paper. If they free the data with the code the idea could be reproducible, in other case it seems hard to reproduce.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    6

  • [Post rebuttal] Please justify your decision

    My opinion of the paper has changed, thanks to the authors responding to all the concerns:

    1. They confirm that their method works for long sweeps even though the IMU sensors have an integral error.
    2. They added the statistical analysis of the t-test and also presented cases where the method is not better.
    3. The code they released and the page where you can see the quality of the images confirms that the method does work on valid images for doctors with many speckle.
    4. The realease of the network code will make the comunity able to reproduce the results and continue progressing in this direction from this point, without the need of re-inventing the wheel.



Review #3

  • Please describe the contribution of the paper

    A novel multi-IMU-based online self-consistency network for freehand 3D US reconstruction was proposed.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The multi-IMU-based freehand 3D US reconstruction method is interesting and novel.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The integration of the multi-IMU and the US probe seems too cumbersome, which may limit the convenience of the use of the US probe in clinical practice.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The reproducibility of the paper is unclear.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    This manuscript describes a novel online self-consistency network using multiple IMUs for freehand 3D US image reconstruction, with improved reconstruction performance comparing with related works.

    Major strengths: 1) The motivation of this manuscript is clear. The introduction of multiple IMUs for online reconstruction to reduce the influence of noise is reasonable. 2) The experimental results are satisfactory, demonstrating the SOTA performance of the proposed method. 3) The manuscript is well written and easy to follow.

    Major weaknesses: 1) The integration of the multi-IMU and the US probe seems too cumbersome, which may limit the convenience of the use of the US probe in clinical practice.

    Suggestions: 1) More details about the differences between this work and related IMU-based and online 3D reconstruction method could be added. 2) The code and implementation details could be open source.

    In summary, the manuscript is a good and novel. I suggest an accept of this manuscript.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The novelty of the method.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    6

  • [Post rebuttal] Please justify your decision

    I maintain my original decision



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes a method for freehand 3D ultrasound reconstruction using a custom 3D printed mount on the probe that holds multiple IMU’s. The method is demonstrated on carotid and arm scans showing competitive performance against baselines.

    Strengths:

    • Reviewers acknowledge the method accuracy is promising, and also that the multi-sensor fusion/self-consistency is an interesting and well implemented idea.

    Weaknesses:

    • The developed platform may be cumbersome to use. Comments on its limitations would be useful here
    • A reviewer questions whether all the relevant experimental comparisons are made

    I recommend this paper for rebuttal stage, where the authors can comment in more detail regarding 1) the limitations of ther platform in terms of design and sensor noise; 2) justify the choice of baselines; 3) clarify the movelty of the methodology



Author Feedback

We thank all the reviewers (R) for reviewing and recognizing our work. Our novelty and comparisons are clarified thoroughly. The required information has been provided. The code has been released.

Q1: Novelty (MetaR1, R2, R3) A1: Our work has remarkable novelty regarding the methodology and results. Luo et al. (2021) and Guo et al. (2022) use only images for reconstruction, challenged by the difficulty of estimating out-of-plane motion. Prevost et al. (2018) and Luo et al. (2022) use only a single IMU, neglecting the influence of IMU noise. Our novel designs include: 1) We introduce multiple IMUs for the first time to reduce the influence of noise in freehand 3D ultrasound reconstruction through online MSS. 2) We build online SCS to enhance network stability based on the consistency of prediction results. We hope to inspire the community with the released code.

Q2: Reproducibility (R1, R2, R3) A2: Our anonymized code and 3D-printed bracket model have been released via https://github.com/miccai1835/code. An anonymized interactive demo has been provided via https://miccai1835.github.io.

Q3: Comparisons (MetaR1, R1, R2) A3: We apologize for the lack of experimental details. Details will be added in the final version: 1) For all comparison methods, we used the same data augmentation methods as OSCNet. 2) All comparison methods were trained to convergence using the experimental settings given in the corresponding papers.

Q4: Baseline (MetaR1, R2) A4: We use the Luo et al. (2022) method as a baseline and do not implement the Luo et al. (2021) method for three reasons: 1) The Luo et al. (2022) method integrates IMU, while the Luo et al. (2021) method does not. 2) The sensorless SOTA method (Guo et al., 2022) and the IMU-based SOTA method (Luo et al., 2022) have been shown to outperform the Luo et al. (2021) method. 3) The Luo et al. (2021) method relies on the specific shape of anatomical structures, which is not suitable for anatomical structures (arm and carotid) with long sweeps and no prominent shape features.

Q5: Statistical analysis (R1) A5: The t-test results show that our OSCNet significantly outperforms the comparison methods CNN (Prevost et al., 2018), DC2-Net (Guo et al., 2022), MoNet (Luo et al., 2022), and our Backbone in all metrics for both arm and carotid datasets (p<0.05), except for MoNet’s ADR on the carotid scans. The standard deviations have been provided in Table 1. We will include the t-test results in the final version.

Q6: Performance (R2) A6: Our method can be used for long freehand sweeps. The community is committed to continuously improving the performance of freehand 3D ultrasound reconstruction, and our method improves the performance by about 10% compared to the current SOTA method (Luo et al., 2022). We will continue to improve the performance using methods such as integrating high-precision IMUs.

Q7: Platform design (MetaR1, R1, R3) A7: Our platform doesn’t limit the convenience of using the ultrasound probe. The current design is a prototype due to time and cost. The IMU chip is small (4×4×0.9mm^3) and we’re developing a next-gen platform that will integrate multiple IMUs into smaller modules. In the future, it may even be integrated inside the probe.

Q8: IMU noise (MetaR1, R1, R2) A8: The benefits of IMU integration exceed the drawbacks. Despite the noise of IMU, the introduction of IMU (Luo et al., 2022) can improve reconstruction performance by providing motion information beyond the image. Multiple IMUs (Ours) can also improve performance compared to a single IMU. In the future, we will use high-precision IMUs to further reduce the impact of noise.

Q9: Limitation (MetaR1, R1) A9: Our method is general for different anatomical structures but currently the model trained under this method is still anatomy-dependent. Incorporating multiple IMUs is actually our continuous attempt to enhance the generalizability by providing more structure-independent motion information for the construction.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The reviewers have reached a consensus post-rebuttal to accept this paper.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The authors have replied to comments regarding novelty, reproducibility and comparison baselines. I recommend accepting the paper now.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    R2 changed his review from “reject” to “accept” based on rebuttal. R1 and R2 recommended “accept”



back to top