Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Yu Deng, Hao Xu, Sashya Rodrigo, Steven E. Williams, Michelle C. Williams, Steven A. Niederer, Kuberan Pushparajah, Alistair Young

Abstract

Anatomical heart mesh models created from cine cardiac images are useful for the evaluation and monitoring of cardiovascular diseases, but require challenging and time-consuming reconstruction processes. Errors due to reduced spatial resolution and motion artefacts limit the accuracy of 3D models. We proposed ModusGraph to produce a higher quality 3D and 4D (3D+time) heart models automatically, employing i) a voxel processing module with Modality Handles and a super-resolution decoder to define low-resolution and high-resolution segmentations and correct motion artefacts with multi-modal unpaired data, ii) a Residual Spatial-temporal Graph Convolution Network to generate mesh models by controlled and progressive spatial-temporal deformation to better capture the cardiac motion, and iii) a Signed Distance Sampling process to bridge those two parts for end-to-end training. ModusGraph was trained and evaluated on CT angiograms and cardiovascular MRI cines, showing superior performance compared to other mesh reconstruction methods. It creates well-defined meshes from sparse MRI cines, enabling vertex tracking across cardiac cycle frames. This process aids in analyzing myocardium function and conducting biomechanical analyses from imaging data. https://github.com/MalikTeng/ModusGraph

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43990-2_17

SharedIt: https://rdcu.be/dnwLs

Link to the code repository

https://github.com/MalikTeng/ModusGraph

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper describes a new method, called ModusGraph, for creating 3D and 4D (3D+time) models of the heart from cine cardiac images. The method employs a voxel processing module with Modality Handles and a super-resolution decoder to define low-resolution and high-resolution segmentations and correct motion artefacts with multi-modal data. It also uses a Residual Spatial-temporal Graph Convolution Network to generate mesh models by controlled and progressive spatial-temporal deformation to better capture the cardiac motion, and a Signed Distance Sampling process to bridge those two parts for end-to-end training. The authors trained and evaluated ModusGraph using CT angiograms and cardiovascular MRI cines, and found that it outperformed other mesh reconstruction methods. The method generated high-quality mesh models and improved the accuracy and efficiency of mesh model reconstruction from cine CMR.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Overall, this study proposes a method to reconstruct mesh from MRI and CT images, which can also handle low-resolution and high-resolution images and correct motion artefacts with multi-modal data.
    2. The reconstructed mesh for whole heart could be extended to other tasks as Biomechanical Simulation
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The paper lacks sufficient discussion and comparison of its proposed method’s performance. In Table 1, the results demonstrate that the ResNet decoder outperforms the proposed method, but the author merely attributes the inferior results to the challenge of registering between the template mesh and segmentation and the reduced number of vertices. However, this explanation does not seem convincing, as Paper [13] faced a similar challenge but achieved better performance. 2. In Table 2, the author justifies the exclusion of some methods from comparison by stating that their generated meshes are unsuitable for mechanical simulations without proper post-processing. However, this reason seems inadequate for this application, and the author should provide more detailed and comprehensive explanations for this decision.
    2. The paper fails to compare its proposed method with some of the state-of-the-art techniques in the same domain, despite the availability of relevant literature, such as [12][13].
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    It’s not mentioned in the paper.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. Better explanation of the training stage: it would help to have a clear separation of the training and test stage. Do both stages need both MRI and CT images?
    2. What does different color of arrows mean in Figure 1? please provide more detailed explanation.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The performance of this method and the discussion for the experiments.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper
    1. Whole-heart meshing on CT dataset (SCOT-HEART) and dynamic bi-ventricle meshing on CMR dataset (Cardiac Atlas Project).
    2. Method for addressing CT-MR resolution differences using a segmentation super-resolution decoder.
    3. Spatiotemporal graph conv net for template mesh deformation
    4. End-to-end training from image to mesh reconstruction enabled by differentiable SDF calculation
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Descriptive figures for methods and results
    2. Method motivation is clear, especially for CT-MR multi-modality and super-resolution.
    3. Promising surface mesh metrics
    4. Good baseline comparisons
    5. Considerable amount of data
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. Temporal deformation is a bit confusing. In the “Training scheme”, does T = 1 mean 1 time-frame? If so, why is the network trained that way if there are 48 frames for CMR? If not, what does T = 1 signify?

    2. Related to (1), what is the general strategy for temporal meshing? Based on the methods description, I could guess two ways: (1) template –> fit to first frame –> temporally deform from first frame or (2) template –> fit to each frame, using spatiotemporal features. This does not seem to be clearly stated,

    3. Why the choice of 16^3 for downsampled segmentation? Is it to maintain isotropic spacing of MR? This should be clarified.

    4. From my experience, it is the predicted segmentation –> SDF conversion that is hard to maintain differentiability, not the sampling part. This is explained in one short sentence, but I believe it should be explained in greater detail. Even the segmentation –> surface mesh conversion is difficult to maintain differentiability, so the SDF calculation seems non-trivial.

    5. Is Fig. 2 showing intersecting mesh elements? This would break most simulations. It would be helpful to quantify this if the goal is to demonstrate feasibility of generated meshes for simulations.

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Mostly seems good, but there are a few important details missing, as mentioned in the weaknesses section.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    1. “Geodesic distance” is the distance between two vertices on a graph. SDF is the signed distance of a point to the closest point on the modeled surface. To my knowledge, these are two different things, unless I’m missing something.

    2. What was the reasoning for bolding the MG row for Dice in Table 1? RES had better values in all columns.

    Minor:

    1. In the abstract, there is one sentence describing all contributions. I think it might make more sense to divide it up into multiple sentences for better delivery. Similar issue at the end of the intro.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This is by far the best paper in my batch, but it is still missing some important info that should be clarified. With those clarifications, I believe it has the potential to be useful for future heart modeling studies.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    6

  • [Post rebuttal] Please justify your decision

    There has been some meaningful clarification with respect to temporal meshing and geodesic distance transforms. Given that these changes can be incorporated into the final manuscript, I think the paper is ready for a conference publication.



Review #3

  • Please describe the contribution of the paper

    This paper proposed a deep-learning approach, ModusGraph, to produce 3D+time heart meshes from cine MRI images, which often have limited through-plan spatial resolution.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper has the following strengths:

    1. An effective method to predict high-resolution meshes from CMRI that often has limited through-plane spatial resolution and motion artifacts.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    I see the following two major weaknesses and please see the next section for detailed comments.

    1. The description of the proposed method is very unclear and possibly contains mistakes.
    2. The proposed method is rather complex and there is a lack of ablation studies to justify the design choices.
    3. A major novelty, from my point of view, is the ability to predict time-series meshes, since prior approaches have already worked on creating high-resolution meshes from CMRI (Duan 2019, IEEE TMI) and creating meshes suitable for biomechanical simulations (Kong 2022, IEEE TMI). However, the predicted time-series meshes were not validated (the accuracy of deformations was unknown). Also, it is unclear whether it is beneficial to directly predict a 3D+time mesh or predict a 3D mesh for each time frame of the image data as previously done in another cardiac mesh reconstruction paper (Kong 2022 TMI).
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Code will be provided and thus reproducible.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    A lot of places in the method need to be clarified.

    1. Page 4: “signed distance is computed as geodesic distances from each voxel to the surface boundary” - how do you compute the geodesic distance for points not on the same manifold? Should it be Euclidean distance?
    2. Section 3.2: What do you mean by topology? How come the topology (connectivity of vertices) is related to sharp edges (sharp angles between faces)? By learning the “topology” do you mean learning the mesh connectivity?
    3. Why is the CT data needed in addition to MR? Are MR and CT images paired?
    4. Do you need CT images for testing or just the MR?
    5. Do you need a ground truth point cloud for each time frame to train the method?
    6. Page 5: temporal deformation: Should it be T x 3 since it’s predicting displacements?
    7. Why do you need to predict a down-sampled segmentation first?

    For dataset selection, why do you use a Tetrallogy of Fallot dataset? The ToF patients often have defects in the heart that cannot be modeled by the template mesh, which might affect training.

    Results:

    1. Mesh prediction was compared with CorticalFlow but it doesn’t seem like transformations learned by the proposed method are guaranteed to be diffeomorphic. Surface intersections need to be quantified rather than qualitatively shown on a selected sample.
    2. Unit missing in table 1 and 2.
    3. The dice scores of the proposed method seem to be pretty low (0.51-0.77) - the authors could consider increasing the number of points sampled during training.
    4. Kong 2022 IEEE TMI seems to be a very relevant method for cardiac mesh construction. Authors could consider adding it as another baseline.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    1. The paper targets the challenging task of reconstructing high-resolution 4D meshes from CMRI and demonstrated preliminary success.
    2. Unclear method descriptions
    3. Lack of necessary ablation studies, and quantification of deformation to justify the novel components in the method.
    4. The accuracy of prediction needs to be improved.
  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This work tackles the challenging task of high-resolution 4D mesh reconstruction from cardiac MR images. It is an interesting paper. However, there are several major concerns raised by the reviewers, including:

    1) There is a lack of discussion of the method performance, including comparison to ResNet decoder, the unsuitability of the generated meshes etc, which are detailed in Reviewer 1’s comment.

    2) It would be more supportive, if the proposed method could be compared to existing methods in the field, such as [12, 13].

    3) The method description may need clarification, including the geodesic distance, topology and joint use for MR-CT data, which are detailed in the Reviewer 3’s comment.

    4) The general strategy for temporal meshing is not very clear, which is an important aspect of 4D mesh reconstruction.



Author Feedback

Thank you for the comprehensive feedback. We have carefully considered each point raised:

Performance and Accuracy (MR,R1,R2): ResNet’s high-performance benchmark in Table 1 can be attributed to its use of Marching Cubes for 3D segmentation, which benefits Dice scores but is not optimal for our requirement of quality meshes for simulations and atlasing. Lower Dice scores are due to three issues: information loss in the QuickHull-derived template mesh (see Appendix I), difficulties aligning deformed meshes with segmentation in a different coordinate system, and challenges capturing patient-specific geometric variations. We admit these challenges when using a single template mesh, but refining the template mesh and registration process may bring improvements. We gauge diffeomorphic performance through the intersections’ percentage in whole heart meshing (VM:9.98, CF:10.63, NNP:0.89, RES:0, MG:1.24). In 4D meshing, super-resolved CMRs/segmentation (Duan 2019, IEEE TMI) derived meshes do not facilitate myocardium function motion analysis or ensure a smooth surface with well-defined topology for biomechanical simulation. Meshes produced for this purpose (Kong 2022, IEEE TMI) are inapplicable to sparsely sampled CMRs without suitable network design. Thus, we created ModusGraph to produce meshes meeting these requirements. As nnU-Net and ResNet with post-processing do not guarantee these properties, they were omitted from comparison. Absence of high-quality 3D CMRs segmentation and aforementioned challenges hinder Dice evaluation of the deformed mesh. We will detail all above in our final version.

Comparison with Existing Methods (MR,R1,R3): We assessed Kong’s work [12, 13] but did not consider these as baselines. This is because their network structure is built on Voxel2Mesh with additions for CFD simulations and uses distinct techniques to learn a deformation field. These additions to Voxel2Mesh proved not replicable on our template mesh. Furthermore, their critical training losses code was open-sourced in April this year, leaving us with insufficient time for implementation. Additionally, integrating their proposed networks, which are based on the Tensorflow 1.14 framework, into our application is technically challenging. Hence, we deemed Voxel2Mesh as a more suitable baseline for comparison.

Training and Test Stages, Temporal Meshing (R1,R3,MR): We will elucidate task-specific schemes and template meshes alongside our upcoming code (github.com/MalikTeng/ModusGraph). In whole heart meshing, the CT modality handle and ResNet decoder train using CT images and segmentations, then combined by R-StGCN for further training (testing: CT Mod-handle -> ResNet -> R-StGCN, no CMR required). In 4D meshing, modality handles are trained with CT or CMR (not paired, from different datasets) images and segmentations at the ED frame. The ResNet decoder joined with R-StGCN, trained with CT images and derived point clouds at the ED frame. Subsequently, all modules fine-tune with CMR images and predicted segmentation point clouds at all frames (testing: MR Mod-handle -> ResNet -> R-StGCN, no CT required), deforming a distinct template mesh at each frame. These template meshes are formed by rescaling the template mesh (see Appendix II) mimicking myocardium contraction.

Clarity in Method Description (MR): Differentiable Geodesic Distance Transform (R2,R3): We’ll reference (Criminisi 2008, ECCV) and (Asad 2022, JOSS) for the definition and implementation details, which calculates the shortest voxel distance to the region of interest, indicating the voxel’s relative position. Topology (R3): To mitigate confusion, we’ll replace “topology” with “position relationship” in section 3.2, denoting that vertices remain on either the epicardial or endocardial myocardium surface throughout deformation, ensuring a watertight, non-intersecting mesh.

Minor Points: We appreciate your valuable suggestions and will duly incorporate them into our final version.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The rebuttal has addresed most of my concerns. Although there is some minor issue (discussion with Duan and Kong’s methods) requiring a bit more clarity, overall I think this is an interesting and strong medical imaging paper.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The lack of more relevant baselines such as the work by Kong is a great limitation. Contrary to the authors’ rebuttal, code for Kong MICCAI 2021 is available since March 2022 (see https://github.com/fkong7/HeartFFDNet). Additionally, in their reply the authors, breach anonymization by revealing their identity through the github url they provide. Lastly, the low DICE performance is underwhelming. Taking together, I believe additional work is required for publication.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The rebuttal addressed the reviewer concerns. Particularly, there was clarify with respect to temporal meshing and geodesic distance transforms. The authors should pay attention to the reviewer comments in detail while preparing the final submission and make sure the changes outlined in the rebuttal will feature in the final manuscript.



back to top