Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Alexandre Cafaro, Quentin Spinat, Amaury Leroy, Pauline Maury, Alexandre Munoz, Guillaume Beldjoudi, Charlotte Robert, Eric Deutsch, Vincent Grégoire, Vincent Lepetit, Nikos Paragios

Abstract

We propose an unsupervised deep learning method to reconstruct a 3D tomographic image from biplanar X-rays, to reduce the number of required projections, the patient dose, and the acquisition time. To address this ill-posed problem, we introduce prior knowledge of anatomic structures by training a generative model on 3D CTs of head and neck. We optimize the latent vectors of the generative model to recover a volume that both integrates this prior knowledge and ensures consistency between the reconstructed image and input projections. Our method outperforms recent methods in terms of reconstruction error while being faster and less radiating than current clinical workflow. We evaluate our method in a clinical configuration for radiotherapy.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43999-5_66

SharedIt: https://rdcu.be/dnwxj

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #3

  • Please describe the contribution of the paper

    This paper presents a two-stage reconstruction pipeline for unprojecting sparse-view 2D X-ray images to 3D CT data. The first stage involves training a mapping network to produce an appropriate latent vector, which can then be decoded into a 3D representation. The second stage involves finding the most suitable latent vector and volume for the sparse-view projections.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • This paper’s approach to 3D cervical CT data reconstruction based on limited 2D image projections is a valuable contribution to the clinical domain since it reduces the radiation dose.
    • Using XraySyn to sample massive projections with known radiation poses randomly is also helpful.
    • The experiments were carefully designed and reported, highlighting the advantages of the generation and reconstruction modules. The proposed framework’s evaluation using commonly standard metrics is comprehensive.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Once again, as the view poses of the radiation sources are only sometimes available, reconstruction for arbitrary X-ray images with unknown poses cannot be guaranteed. Blind reconstruction or reconstruction in the wild requires additional conditions and assumptions.
    • The differentiable Cone-beam projection, adopted from XraySyn, requires further explanation and discussion. Because the ray tracing technique usually accumulates the sampling point from back-to-front objects as the screen is placed between the camera and the object, as opposed to the X-ray scan simulation.
    • Moreover, for manifold learning, comparison against the 3D brain MRI dataset is of limited interest since the entire module needs 3D cervical CT data, which leads to different domain distributions. Additionally, the 3D StyleGAN was used entirely, and a direct comparison with and without the mapping network to the latent vector would be worthwhile.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The reimplementation of this work is moderate since there was missing information on how the differentiable Cone-beam projection was set up.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    • Major comments:
      • Several NeRF-based methods work on extreme sparse-view projection (PixelNeRF) and advanced numerical optimization (Plenoxels). The statement “show good results when the number of input projections remains higher than a dozen but fail when very few projections are provided” is somewhat misleading, as PixelNeRF can employ the per-pixel feature vector to perform single view reconstruction of the scene. The authors may want to consider this technique for future work.
      • The latent vector space is of great interest. However, do the authors think that the projections can serve as a manifold of the 3D CT data and, therefore, can be used directly to train the 2D-3D decoder? Please discuss this matter for the design strategy to enhance the argument made in contribution (i) if it is not an implementation issue.
    • Minor comments:
      • There are still some typos in the manuscript, which can be fixed by some proofreading iterations. For example, “a ill-posed” -> “an ill-posed.”
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper fits well with the MICCAI theme for 2D image-based sparse-view 3D CT reconstruction. The design strategy and optimization scheme were carefully constructed to carry out the task of finding the most proper latent vector representation. However, there is still room to improve the paper regarding the above comments and its exposition/presentation. Hence, my current evaluation is borderline toward a weak acceptance. I am happy to adjust my score if the authors can effectively address these concerns.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #1

  • Please describe the contribution of the paper

    This paper has proposed a 3D reconstruction model from biplanar X-rays. The model first leans a style-based generator from Gaussian distribution, then reconstructs the 3D object by optimization on latent vector.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper has proposed an interesting way for 3D reconstruction from X-ray images by combining feed-forward models and optimization models.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. As the first step is to train a 3D StyleGAN, what is the performance, like FID, of the generative model? Are there any visualized results of the generation output? Are there any disentanglement studies for the generation results since it uses StyleGAN?

    2. During the second step, since the reconstruction image is obtained by optimization over latent vector w, what does the intermediate 3D image look like?

    3. In Fig. 3 and figures in the supplemental materials, the author mainly shows cases with healthy conditions. Since the method is based on the generative model, what is the reconstruction result when the test data is different from the training data? For example, if there is a tumor in the neck area or there are some fillings in the teeth, could these features be clearly shown in the reconstruction result?

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Training the generative model may not be easy for other people to reproduce the results. It would be better if the author could provide the code for training the generator or resease the weights.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    1. In section 2, what is Maximum A Posteriori?

    2. In section 2.1, I think the author what to use I_i instead of {Ii}i?

    3. Could this work be extend to the problem of 3D reconstruction from single panoramic X-ray image, such as [1], to show the generalization in different tasks?

    4. What are the settings for >2 beam projections? Could the author list some details?

    [1] Oral-3d: Reconstructing the 3d structure of oral cavity from panoramic x-ray.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My major concerns lies in the training of 3D StyleGAN and some diversity analysis of the dataset. I expect the author show some generation results of 3D CBCT by the well-trained StyleGAN.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    The study used biplanar projections to reconstruct CBCT images, aided by a manifold learned from a population-based training dataset. An optimization scheme has been introduced to optimize the latent vector to generate CBCT images matching the image features encoded on the biplanar projections.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The combined manifold learning and on-the-fly latent vector optimization scheme allows the reconstruction of patient-specific traits while also enabling the use of very sparse data (biplanar projections) to reconstruct high-quality images.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The model is trained on CTs but proposed to apply to CBCT reconstruction. The potential image quality differences between CT/CBCT can decrease the accuracy of the proposed method. It looks like the current method is evaluated on DRRs projected from the deformed CTs, rather than the available CBCT scans. Using the CBCT projections for evaluation can lend further support to the developed method.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    N.A.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. Use real cone-beam x-ray projections to evaluate the method. Or at least discuss the potential issues and methods to address these issues.
    2. Discuss the pros/cons of patient-specific and population-driven priors
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper proposes a novel method. The results are encouraging.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper received three relatively detailed reviews all recommending acceptance.

    The strengths are perceived as an interesting approach to an important problem. Further strenghts included a rigorous evaluation.

    Weaknesses included:

    • Assumptions of the method (for example, non poses or the evaluation on healthy subjects), which may not sufficiently well capture real world performance
    • Use of synthetic data instead of real CBCT projections (which would provide strong paired evaluations)
    • And the need for some additional clarifications

    These concerns should be addressed as possible in the updated version.



Author Feedback

N/A



back to top