List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Francesca De Benetti, Walter Simson, Magdalini Paschali, Hasan Sari, Axel Rominger, Kuangyu Shi, Nassir Navab, Thomas Wendler
Abstract
Dynamic Positron Emission Tomography imaging (dPET) provides temporally resolved images of a tracer. Voxel-wise physiologically-based pharmacokinetic modeling of the Time Activity Curves (TAC) extracted from dPET can provide relevant diagnostic information for clinical workflow. Conventional fitting strategies for TACs are slow and ignore the spatial relation between neighboring voxels. We train a spatio-temporal UNet to estimate the kinetic parameters given TAC from dPET. This work introduces a self-supervised loss formulation to enforce the similarity between the measured TAC and those generated with the learned kinetic parameters. Our method provides quantitatively comparable results at organ level to the significantly slower conventional approaches while generating pixel-wise kinetic parametric images which are consistent with expected physiology. To the best of our knowledge, this is the first self-supervised network that allows voxel-wise computation of kinetic parameters consistent with a non-linear kinetic model.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_28
SharedIt: https://rdcu.be/dnwcF
Link to the code repository
https://github.com/FrancescaDB/self_supervised_PBPK_modelling
Link to the dataset(s)
N/A
Reviews
Review #3
- Please describe the contribution of the paper
In the paper, the authors proposed a self-supervised spatiotemporal deep neural network to estimate parametric maps (K1, k2, k3, and VB) for FDG PET in cancer patients at the voxel level. The proposed approach was much faster than the traditional curve-fitting approach and generated reasonable parametric maps.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of the proposed method is that it is the first self-supervised network that estimates PET kinetic micro-parameters at the voxel-level. This network is easy to train and apply, and is much faster than the traditional curve fitting methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- As shown in Figure 2, the regional micro-parameters estimated by the network and curve-fitting method are very different, although no statistical analysis was performed (which I believe is necessary). I understand that the curve-fitting method does not provide the ground truth. Still, it would be healful to compare the results of the network with the results of some traditional methods to prove the clinical usability of the proposed method.
- The authors did not perform sufficient comparison at the voxel level to support the advantage of the proposed method. Even without ground truth, I think a radiological review of the parametric maps outputted from different methods by one or two physicians would be very helpful to strengthen the message, especially considering the exitence of tumors in the study cohort.
- Maybe I did not understand this part correctly, but I am not sure why the authors used the TAC instead of micro-parameters to select the model. (Table 1) I think what eventually
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors provided sufficient information about the network architecture, training setup etc.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I think this paper can be further improved by:
- Compare the results from the proposed method to traditional/well-accepted method at the voxel-level or in smaller ROIs (maybe in the tumor lesion?) to better support the advantage of the method.
- Perform quantitative validation using simulation data.
- I noticed that the bond of parameters were set differently for the network and curve-fitting method. It would be good to make them the same or explain the reason for this choice for a more fair comparison.
- Maybe it is good to emphasize the strength of the proposed method in the discussion section.
- Minor clarity issue: I think the k4 in the model is assumed to be zero and therefore omitted. It would be good to add one sentence to clarify.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My main concern for the proposed method is the lack of quantitative comparison with the ground true/reference standard to ensure the validity of the network outputs.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
This work proposed a spatial-temporal network to perform self-supervised learning to generate kinetic parameters. The method was validated on whole-body oncology datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of this work is employing the 2-tissue compartment model as the kinetic model. The concept of self-supervised loss in this work is not new.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The major limitation of this work is that the final results as shown in Fig. 2 seem to have a large discrepancy with the curve fit result (orange). Further quantification of the results is needed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reasonable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I do not understand what Figure 4 means here. Why the difference between the left and right figures are so large? Further quantification (regarding figure 2) is needed to better understand the proposed method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Novelty of this work seems a little limited.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper presents a self-supervised NN for learning parametric images of a reversible 2 compartments compartmental model from dynamic FDG-PET data.
Explicitly inspired by physics-informed NN, the proposed method builds a UNet in which the predicted parameters are forced to satisfy the compartmental model, and are constrained within predefined ranges. Training, test and validation are performed on a small dataset of subjects, and there is no testing on simulated data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method is interesting, and PET parametric imaging is indeed a topic which draws attention, as parametric images convey clinically-relevant information (in oncology for instance) but it’s still difficult to overcome a number of technical problems (PV effects, motion, breathing…). The resulting parametric images appear to be realistically-looking, and with meaningful estimated microparameters.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
My major concerns wrt the proposed paper are as follows
-
The lack of synthetic data simulation.
-
The generalisability of their approach to different compartmental model.
-
Constraining the curve fit procedure.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It is OK.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I’m now expanding on the list of weakness which I listed above.
-
The lack of synthetic data simulation. I’m aware that is no easy task to simulate realistic phantom data, but still there is a growing body of literature on PET simulators (mostly based on Monte Carlo simulators, but not only, e.g. Pfaehler, et al EJNMMI Phys 2018) which the authors could use to test their model. Because otherwise the method is run on such a small number of subjects that it could be hard to draw conclusions on its performance.
-
The generalisability of their approach to different compartmental model. This is particularly important as there’s work that shows that for liver and kidneys in particular the reversible 2CM model used by the authors is inappropriate (e.g. Scussolini et al, Inverse Problems, 2017). I wonder because eq (2) is unlikely to still hold when the model is more complex (for instance, is reversible) and so it’d need to be replaced by a different equation which involves the data (sum of F and B) and the microparameters.
-
I’d suggest (if not done already, I have not found such information in the paper), to have a fairer comparison with standard curve fit, to constraint the curve fit in the bound the authors used for their approach (the bound suggested by Sari).
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think parametric imaging is an interesting topic, and the paper proposed a novel method for this research topic. I’m skeptical on the validation and generalisability of the approach, but still, I find the method interesting and the paper well written.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors introduce a self-supervised spatial-temporal DNN for kinetic parameters modelling for compartmental analysis of dynamic data. The method was validated on whole-body FDG PET datasets. Overall the work is interesting, focus on an important application, and fits the MICCAI readership. The method proposed is faster with reasonable parametric maps generated (R3, R2). The paper has some weak points raised by the reviewers and that should be addressed by the authors, such as clarification of the presented results (Figures 2 and 4, R1, R3), justification on the constraints imposed on the curve fitting, a discussion for the possible extension of other compartmental models (R2, R3), results/discussion regarding voxel level (R3). Work limitations should be outlined. Can be it be extended to other dynamical data sets for compartment analysis? Please also conduct statistical analysis test, especially for Figure 2 results. In addition, other important comments have been raised by the reviewers.
Author Feedback
We thank the reviewers for their useful and precise feedback. The reviewers acknowledged that we proposed a self-supervised spatial-temporal network (NN) to generate parametric images of the kinetic parameters of a 2-tissue-compartment (2TC) model for F18-FDG. R3 liked the self-supervised approach and the reduction of the computation time, if compared with the conventional curve fit approaches, and R2 and R3 agreed on the fact that the proposed NN generates realistic parametric maps.
The conventional curve fit is performed with different boundaries than what was used in the proposed NN (R2, R3). With the goal of allowing a better comparison, we will run the curve fit algorithm with the boundaries used in the NN and we will update Figure 2, Figure 4 and the Supplementary Material (SM) in the camera-ready version (CRV). Moreover, we will add in the SM a deeper comparison between the voxel-wise results obtained with the NN and with the conventional curve fit approach (R3), in terms of average and standard deviation of all kinetic parameters per organ. Due to the lack of space, a complete statistical evaluation (i.e., significance of the difference between the approaches) will be included in a journal extension of this work, which is already in development (R3).
We chose the 2TC because it is considered a good estimation of the kinetic of F18-FDG [1]. In the CRV, we will explicitly write that k4 was omitted because we assumed that the accumulation in the cell of F18-FDG is irreversible (R3) [2]. We are aware that 2TC is not ideal for all the organs [3] (R2). However, the same choice was made by [4], whose results are used in the evaluation. Moreover, the novelty of the proposed approach was in the architecture and loss definition, and we did not focus on finding the best compartmental model for each organ/voxel. Further experiments including different compartmental models would require DynamicPET acquisitions with a different tracer. They are already planned, yet the corresponding clinical trial has not started. We will clarify in the discussion how to modify the workflow/loss function when working with other compartmental models (R2) or when k4 is not assumed to be zero. In practical terms, the number of output channels should be modified according to the number of parameters to be predicted. In addition to that, equation (1) should be replaced with the equation describing any other compartment model.
R2 and R3 suggested using simulated data to validate the proposed method. We agree with them about the fact that this would make the evaluation of our proposed NN stronger. Due to the page limit of the submission, we preferred to include results on real world data only, which we believe are more interesting for the community and more difficult to address. Nonetheless, an evaluation on simulated data will be included in the planned journal extension.
The scanner used to acquire the data has been in use in our partner hospital only for a few months, therefore the dataset is small. An on-going clinical trial is recruiting more patients, which we will include in the journal extension (R2).
We will improve the captions of Figure 2 and 4 to clarify their meaning (R1), including the fact that Figure 2 is a comparison with a traditional method, which was applied to the same dataset by [4] (R3).
[1] Liu G, et al. Kinetic metrics of 18 F-FDG in normal human organs identified by systematic dynamic total-body positron emission tomography. EJNMMI 2021 [2] Watabe H. Compartmental Modeling in PET Kinetics. Basic Science of PET Imaging. 2017 [3] Sari H, et al. Kinetic modelling of dynamic F-18-FDG datasets from a long axial field-of-view PET scanner using model selection criteria with deep learning-based organ segmentations. Nuklearmedizin 2023 [4] Sari H, et al. First results on kinetic modelling and parametric imaging of dynamic 18 F-FDG datasets from a long axial FOV PET scanner in oncological patients. EJNMMI 2022