List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Matthew Ragoza, Kayhan Batmanghelich
Abstract
Magnetic resonance elastography (MRE) is a medical imaging modality that non-invasively quantifies tissue stiffness (elasticity) and is commonly used for diagnosing liver fibrosis. Constructing an elasticity map of tissue requires solving an inverse problem involving a partial differential equation (PDE). Current numerical techniques to solve the inverse problem are noise-sensitive and require explicit specification of physical relationships. In this work, we apply physics-informed neural networks to solve the inverse problem of tissue elasticity reconstruction. Our method does not rely on numerical differentiation and can be extended to learn relevant correlations from anatomical images while respecting physical constraints. We evaluate our approach on simulated data and in vivo data from a cohort of patients with non-alcoholic fatty liver disease (NAFLD). Compared to numerical baselines, our method is more robust to noise and more accurate on realistic data, and its performance is further enhanced by incorporating anatomical information.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_32
SharedIt: https://rdcu.be/dnwwL
Link to the code repository
https://github.com/batmanlab/MRE-PINN
Link to the dataset(s)
https://bioqic-apps.charite.de/downloads
Reviews
Review #3
- Please describe the contribution of the paper
The authors present a physic informed neural network for elasticity reconstruction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The idea of physic informed network is novel and highly required for elastography society. The method is evaluated using simulation and clinical data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The idea of the paper sounds interesting but the paper has several drawbacks: In the simulation data, the elasticity is known so wahy MSE or RMSE error are not evaluated and CTE is used? The simulation results are not encouraging, the method did not outperform conventional methods. The authors use correlation for clinical data to demonstrates the effectiveness of the proposed method. Although correlation can be an indicator of the low variance, it cannot show the bias error hence the method’s performance cannot be verified. Incorporating anatomical information. More details about training and convergence could be provided in supplementary materials.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The network architecture was explained but dataset and code are not available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
more quantitative results on simulation.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Physic inspired idea is novel in elastography There are several weaknesses but it seems the method is performing well enough.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #5
- Please describe the contribution of the paper
The manuscript presents a particularly interesting and novel approach to solving the inverse problem of computing tissue elasticity from magnetic resonance elastography imaging. In essence, a combination of two physics-informed neural networks is trained to infer tissue elasticity. The method is compared against two traditional methods on both synthetic and clinical data. Even though the proposed method showed reduced contrast transfer efficiency compared to the other methods, it was less prone to added noise. Most notably, the method demonstrated a significantly more consistent solution to proprietary clinical elastography in the clinical evaluation than the other two methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method is extremely interesting since it is a novel way to solve a challenging inverse problem. The manuscript is very well written, easy to follow, and the tables and graphics are well designed. The evaluation on both synthetic and clinical data seems appropriate and the results are well presented.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Even though the achieved results are promising, it is not perfectly clear what the final conclusion is and when one should use each method (especially Helmholtz equation or heterogeneous PDE).
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper is providing all information to re-code the method.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This is an extermely comprehensive and interesting work, which shows a great example of applying physics-informed neural networks to a challenging problem. Even though everything is very well presented, I find it hard to draw a final conclusion. Could the authors please clarify when they would recommend to use any of these methods?
In addition, I would appreciate if the authors provide more information about the clinical ground truth. If possible, could the authors please share some information about the underlying assumptions with which the elasticity was computed?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript is interesting, easy to follow, and shows a novel solution to a challenging problem. I strongly believe that this will be of interest to the MICCAI community.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes an AI-based MRI elastography reconstruction method taking into account anatomical data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Proposing a robust MRE reconstruction method, based on AI, and including anatomical information seems to me an excellent approach, in particular to take into account the possible multiple wave reflections at the interfaces of the organs, heterogeneities, wave guiding via certain structures as well as anisotropy of wave propagations - in other words the physical boundary conditions of wave propagations, very difficult to manage by more traditional algorithms (even if all these aspects are visibly not integrated in this manuscript at this stage).
The reconstruction method, even if it suffers from some aspects detailed below, is globally well described in the aspects related to the framework and seems to me to be quite relevant.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It is extremely difficult to evaluate the robustness of the proposed algorithms as almost no details are given about the numerical model used to generate the simulated data. For example, what type of simulation is used? What are the assumptions about the mechanical properties? What are the bounded conditions?
Do they take into account or not the noise (Rician type on complex MRI images, and therefore Gaussian type on phase images) normally present on experimental MRI images (and to which all methods, including this one, are necessarily more or less sensitive)?
On the other hand, purely elastic reconstruction does not seem to be the most ambitious challenge for MRE reconstructions. Current algotrithms, such as LFE - or even other methods already proposed by AI methods since 2018 -, seem to work very well (and very quickly). The challenge is much more towards taking into account and measuring innovative mechanical parameters, such as viscosity or mechanical anisotropy. This does not seem to be the case here and somewhat tempers the interest and originality of the proposed results. And what does “we used proprietary elastograms collected during the study as the “gold standard” mean? Is it the LFE?
Then, one of the major problems in MRE is the phase folding step, extremely sensitive to noise. The proposed method, if I don’t omit anything when reading the manuscript, involves data on already reconstructed waveforms (thus phase) and this sensitivity to phase unwrapping (very variable from one machine to another, from one antenna to another, from one organ to another, and especially from one reconstruction method to another) does not seem to be taken into account.
Finally, it is difficult to understand how the boundary conditions are considered. They are certainly implemented as “loss functions”, but which conditions and which assumptions are involved? organ boundaries? wave reflections at interfaces? and how are experimental wave propagations arriving with multiple and often random directions in the images managed?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
For me, many elements are missing to guarantee the reproducibility of the results: the database used, the frequency used in MRE (or multi-frequency? then which one?), the boundary conditions used, the type of MRI and sequence used (changing considerably the visualizable wavelength, resolution, SNR, FOV,… and thus the result of the investigations and comparisons with existing algorithms), the direct models used,…
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
It would be very useful to give more details about the data used, hypothesis, frequencies, fields of application, sequence used experimentally, evaluation of SNR,… as well as to investigate the effects of interfaces and boundary conditions from the phase of test data on numerical phantoms. Also a step on experimental phantoms seems essential to evaluate all the biases related to interfaces and boundary conditions.
This same study, presented by reinforcing these aspects as well as the experimental part and by applying it with more advanced mechanical assumptions (as done in the recent MRE reconstruction methods by AI).
Another aspect unclear to me: are the anatomical data used specifically acquired T2 images (and lengthening the acquisition time per patient) or are the ERM magnitude images used in this sense (the phase ones being dedicated to the encoding of shear waves)? This is very important because many pathologies, tumors… in the liver may not show up on anatomical images (especially if the magnitude of the MRE images) and therefore may not be used to take into account the anatomy of the elastogram reconstruction and therefore result in “missing” heterogeneities, particularly pathological ones).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please see my previous comments.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a framework that uses physics-informed neural networks (PINNs) for magnetic resonance elastography (MRE) reconstruction. As physical formulations are used in the reconstruction process, the use of PINNs is appropriate and interesting. The main concerns of the reviewers are related to the experiments especially those of the simulation data.
Different from common deep learning approaches that learn a single model from a population of patients, the PINNs in this paper learn one displacement model and one elasticity model for a single patient. The authors should clarify this especially in Fig. 2. A proper definition of “n” in Fig. 2 (number of pixels?) can also help.
Author Feedback
We thank the reviewers for their detailed and constructive feedback. We address specific comments made by each reviewer below.
R1: Clarify model retraining per patient
PINNs must be retrained on each patient’s imaging data. In Fig 2, “n” represents the number of (image coordinate, image value) pairs in a batch. This contrasts with a typical training setup where a batch contains several images from different patients. We will clarify this point in the camera-ready version.
R2: Insufficient detail on numerical simulation
This data set was obtained from BIOQIC and was originally described in reference 28. The data were produced in ABAQUS using FEM with a hexahedral mesh. The material were isotropic, incompressible, and used a Voigt model with shear viscosity 1 Pa s. Wave frequencies were 50-100 Hz in 10 Hz increments. One boundary had a shear surface traction and the other boundaries were absorbent.
R2: Experimental MRI noise
Noise is pervasive in real MRI images, which is why we conducted a controlled noise experiment using the numerical data. We did not evaluate the experimental MRI SNR in this work, but we also did not filter the images. We believe that the better reconstruction ability of PINNs compared to baseline methods on the in vivo data provides some support for relative noise robustness on experimental MRI.
R2, R5: Insufficient detail on gold standard MRE
The gold standard MRE were generated by proprietary Resoundant software integrated directly into the MRI scanner, thus not public knowledge. However, Resoundant produces nearly all FDA-approved MRE hardware and software for GE, Philips, and Siemens scanners and, according to their 2022 technical report, uses this same inversion algorithm in all of its products.
R2: Phase unwrapping/unfolding
This study focuses on reconstruction from wave images that have already been phase unwrapped. Improved phase unwrapping is outside the project scope.
R2: Boundaries/boundary conditions
In the simulation, the boundaries are the rectangular borders. In the patient data, boundaries of the liver were determined by applying a pretrained segmentation model to identify a mask for the liver region. Models were trained only on image points within the liver mask.
The Helmholtz PDE does not require boundary conditions. The heterogeneous PDE typically does, but it is not feasible to determine exact values for stiffness or tractions on the liver boundary from in vivo data.
R2, R3: Insufficient details for reproducibility
The simulation data were downloaded from BIOQIC, as previously mentioned. The patient data are not public but were obtained from a published study at UPMC (reference 22). All inversion methods in this work are single-frequency. The frequencies were 50-100 Hz by 10 Hz increments for the simulation and 60 Hz for patient data. The anatomical images included T1, T2 sequences and MRE magnitude images. Full details of the acquisition parameters are found in reference 22. Baseline method formulations are described in the supplement. We ommitted the code link for anonymity, but will include it in the camera-ready version. A download script for the numerical data and jupyter notebooks for all experiments will also be included.
R2: Evaluation of experimental phantom
We omitted an experimental phantom evaluation due to space constraints.
R3: Lack of MSE metric
We used CTE and R2 because they better correlate with qualitative assessment of elastograms than MSE and we did not have space for more figures.
R5: Clarify the final conclusion
The best PDE depends on the type of data that are used. For a numerical simulation, the proper assumptions are those that were used to generate the simulation. For in vivo liver data, we believe that our results show that the homogeneity assumption harms the reconstruction accuracy, so the heterogeneous PDE is better than the Helmholtz PDE when reconstructing liver stiffness.