Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Wei Peng, Ehsan Adeli, Tomas Bosschieter, Sang Hyun Park, Qingyu Zhao, Kilian M. Pohl

Abstract

As acquiring MRIs is expensive, neuroscience studies struggle to attain a sufficient number of them for properly training deep learning models. This challenge could be reduced by MRI synthesis, for which Generative Adversarial Networks (GANs) are popular. GANs, however, are commonly unstable and struggle with creating diverse and high-quality data. A more stable alternative is Diffusion Probabilistic Models (DPMs) with a fine-grained training strategy. To overcome their need for extensive computational resources, we propose a conditional DPM (cDPM) with a memory-efficient process that generates realistic-looking brain MRIs. To this end, we train a 2D cDPM to generate an MRI subvolume conditioned on another subset of slices from the same MRI. By generating slices using arbitrary combinations between condition and target slices, the model only requires limited computational resources to learn interdependencies between slices even if they are spatially far apart. After having learned these dependencies via an attention network, a new anatomy-consistent 3D brain MRI is generated by repeatedly applying the cDPM. Our experiments demonstrate that our method can generate high-quality 3D MRIs that share a similar distribution to real MRIs while still diversifying the training set. The code is available at https://github.com/xiaoiker/mask3DMRI_diffusion and also will be released as part of MONAI, at https://github.com/Project-MONAI/GenerativeModels .

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43993-3_2

SharedIt: https://rdcu.be/dnwM3

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #6

  • Please describe the contribution of the paper

    The article discusses the challenge of acquiring reliable MRI samples for training deep learning models in neuroimage analysis. The authors propose a new approach called conditional DPM (cDPM) that generates high-quality brain MRIs while using limited computational resources. The proposed method trains a model to generate a subset of 2D MRI slices (target slices) based on another subset of slices (conditional slices) from the same MRI. The cDPM generates realistic-looking MRIs that share a similar distribution to real MRIs while diversifying the training set.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • The experiments demonstrate that the proposed method can generate high-quality brain MRIs that share a similar distribution to real MRIs while diversifying the training set.

    • The proposed conditional DPM (cDPM) uses a memory-efficient training strategy that generates high-quality MRIs while using limited computational resources

    • The paper includes a nice analysis of sample diversity using t-SNE, which helps visualize the distribution of the generated samples and highlights the limitations of GANs in diversifying the training set. This provides valuable insights into the strengths and weaknesses of the proposed method.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • Although the proposed method is shown to outperform existing methods, the paper does not compare it to the current state-of-the-art methods for brain MRI synthesis, which may limit the significance of the reported results. For instance, the paper does not evaluate the proposed method against latent diffusion models (LDMs, 2022) which are also a computationally efficient approach using diffusion models for generating 3D brain MRIs (https://arxiv.org/pdf/2209.07162.pdf).

    • The paper does not include an analysis of computational and runtime performance. Such an analysis could provide valuable insights into the efficiency and practicality of the proposed method and help researchers compare it with other approaches, in particular with efficient 3D generative networks such as LDMs.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The paper provides sufficient details on the architecture and training procedures of the proposed cDPM method, enabling other researchers to replicate the experiments. The authors have also committed to releasing the code for their method as part of the MONAI library, which will enhance reproducibility. However, the paper lacks detailed information on hyper-parameter selection, mean and standard deviation trends, comparison to important baselines (e.g, LDMs), and significance analysis. These details are important for interpreting the results and comparing the method with the state-of-the-art. Additionally, including comparisons of computational and runtime performance could have further clarified the contributions of the proposed method.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    • It would be beneficial if the authors could explain why they did not compare their proposed method with LDMs [1], a similar approach that also performs efficient 3D brain synthesis. Including such a comparison could provide a more comprehensive evaluation of the proposed method. A comparison between the proposed cDPM and LDMs in terms of synthesis accuracy, computational cost, and runtime would be particularly insightful.

    • Since the proposed method relies on predicting 2D slices, it would be valuable to investigate the volumetric consistency of the brain regions generated by the method. For instance, it would be interesting to see 3D segmentations of regions of interest and understand how well the generated brain volumes align with the expected anatomical structures.

    [1] Pinaya, W.H., Tudosiu, P.D., Dafflon, J., Da Costa, P.F., Fernandez, V., Nachev, P., Ourselin, S., Cardoso, M.J.: Brain imaging generation with latent diffusion models. In: Deep Generative Models: DGM4MICCAI 2022. pp. 117–126 (2022)

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper lacks a comparison with a closely related method that also utilizes diffusion models for 3D brain synthesis. The referenced method proposes a computationally efficient latent diffusion model that generates 3D brain MRIs directly. Without a comprehensive evaluation of synthesis performance, computational efficiency, and runtime analysis for both methods, it is challenging to interpret the results and assess the significance of the contributions.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #1

  • Please describe the contribution of the paper

    The paper propose a conditional DPM based brain synthesis method which can generate a subset of slices conditioned on either random noise or another set of slices of the same subject. After training, the model can generate a 3D brain MRI starting from a random noise and iteratively synthesize successive MRI slices conditioned on previously generated sets. The design mitigates the usage of 3D networks, and is computationally efficient. The model shows better generative power than GANs, and it can also sample new independent MRIs from the same distribution of the training data.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper presents an interesting design for applying the 2D diffusion models to 3D volumetric MRI generation. The pipeline of generating subset of slices given another set of 2D slices from the same volume is a novel contribution.
    2. The method seems to be clinically relevant given that it is computational efficient, and it could be beneficial in rapid MRI imaging or compressed sensing as it can in principle perform imputation/synthesis from only sparse acquisition.
    3. The experiment and comparison conducted in the study are thorough and comprehensive. The results demonstrate realism and diversity of the generated samples.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Overall, I find the work innovative. However, I have two major concerns: (1) lack of details in the generation/inference stage, and (2) the slice inconsistency of the final 3D volume ensembled from the 2D slices. Please find detailed comments below.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Submission indicates that the code will be released as part of MONAI.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    Inference/generation details

    1. During training, the target slices are sampled at arbitrary locations conditioned on arbitrarily sampled conditional slices. However, it is unclear what is the case during the inference. Based on Fig.1, it seems the generation is ordered starting from slice 1 to slice N. Does such ordering matter during the generation?

    2. What is the total time for generating one 3D volume? If it takes multiple stages and each stage is a prolonged diffusion sampling process (known to be slow), then its practical usage may be concerned.

    3. How many denoising steps are used in the generation? Are there significant differences in terms of the image sharpness when using different number of steps?

    Results

    1. As the method is trained and evaluated with axial slices, slice inconsistency is a potential problem which is revealed in Fig.3. Both sagittal and coronal views exhibit intensity inhomogeneity (brightness shift) due to slice inconsistency. This could potentially affect its downstream usage. Please justify.

    2. Fig.3 v.s. Fig.4: the results of “Ours” seem to have different resolutions. Does the result in Fig.4 have higher resolution or it is just visualization differences?

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While there are a few unclear technical details and confusing result presentation, I think the paper has its merits over its weakness. Hopefully the concerns could be addressed during the rebuttal.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    This paper proposes a conditional diffusion probabilistic model (cDPM) for generating synthetic MRI images for multiple applications. The proposed cDPM features computational efficiency and high-realistic image synthesis performance. According to the results, the proposed model outperformed the comparison GAN-based methods. Overall, this paper is well organized and with sufficient results to demonstrate the results.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Strengths of the paper: Applications: This paper addresses the issue of limited medical image data for model training, which is useful for training high-performance medical models.

    Performance: This paper proposes a novel cDPM for synthesizing MRI images, and the results demonstrate that the performance of the proposed cDPM outperforms traditional GAN-based methods.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    A key feature of the proposed model is computational efficiency. However, the authors did not well demonstrate this improvement in results.

    In Fig. 3, the intensities of the synthetic MRI images look heterogenous, especially between the first (brighter) and the second MRI (darker) volumes. Could the authors explain about this?

    Typo: imputing.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Good reproducibility.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    I’m interested in how many training samples are needed to train a good performance cDPM model for virtual MRI images generation.

    The quality of this paper could be further improved if there is an example of application (for example, disease diagnosis as mentioned by the authors) to demonstrate or quantify the effectiveness of the proposed method.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Novelty and results.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The authors propose a 2D conditional diffusion probabilistic model for synthesizing 3D brain MRIs. The reviewers agree that the paper is well written and relevant. The method is memory-efficient, and the experiments demonstrate diverse and high-quality brain MRIs. However, the reviewers expressed concern about the slice inconsistency resulting from generating 2D slices for a 3D volume and one reviewer pointed out the lack of relevant state-of-the-art methods. Nevertheless, the paper is of high interest for the MICCAI community and of sufficient quality for an early accept.



Author Feedback

N/A



back to top