List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yongsheng Pan, Feihong Liu, Caiwen Jiang, Jiawei Huang, Yong Xia, Dinggang Shen
Abstract
Positron emission tomography (PET) is a molecular imaging technique relying on a step, namely attenuation correction, to correct radionuclide distribution based on pre-determined attenuation coefficients. Conventional AC techniques require additionally-acquired computed tomography (CT) or magnetic resonance (MR) images to calculate attenuation coefficients, which increases imaging expenses, time costs, or radiation hazards to patients, especially for whole-body scanners. In this paper, considering technological advances in acquiring more anatomical information in raw PET images, we propose to conduct attenuation correction to PET by itself. To achieve this, we design a deep learning based framework, namely anatomical skeleton-enhanced generation (ASEG), to generate pseudo CT images from non-attenuation corrected PET images for attenuation correction. Specifically, ASEG contains two sequential modules, i.e., a skeleton prediction module and a volume rendering module. The former module first delineates anatomical skeleton and the latter module then renders tissue volume. Both modules are trained collaboratively with specific anatomical-consistency constraint to guarantee tissue generation fidelity. Experiments on four public PET/CT datasets demonstrate that our ASEG outperforms existing methods by achieving better consistency of anatomical structures in generated CT images, which are further employed to conduct PET attenuation correction with better similarity to real ones. This work verifies the feasibility of generating pseudo CT from raw PET for attenuation correction without acquisition of additional CT.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_3
SharedIt: https://rdcu.be/dnwjd
Link to the code repository
https://github.com/YongshengPan/ASEG-for-PET2CT
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This work proposed a learning based framework, namely anatomical skeleton-enhanced generation (ASEG), to generate pseudo CT images from non-attenuation corrected PET images for attenuation correction, with two sequential modules, i.e., a skeleton prediction module to delineate skeleton and a volume rendering module.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper realizes the major role the skeleton component is taking in AC and dedicated a module for that purpose. Effectively, it tailors the training objective towards the task, which is a plausible.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors have argued that CT in the current context does not offer much value in its own other than for AC correction. If that is the case, then why focusing on CT quality as the endpoint? Modality synthesis or translation typically have a quite strong motivation to reach the target domain, and if that is not the case, the perspective should either be a more ‘generic” definition of task, or to really align the training or optimization objective to the task of, say identify nuclear uptake.
If we look beyond the narrow scope of trying to come up with an artificial CT for correction, then there is much more one may do. Also, in the modeling line, there is much work in direct estimation starting from MLAA (ML recon strution of attenuation and activity) and TOF info.
In addition, there is also domain knowledge to appreciate the component related to tracer and disease, and the one relevant to anatomy (incorporated in the current work for skeleton delineation etc)
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
OK
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please at least provide comparison to one MLAA benchmark.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I appreciate the authors’ effort. It is a nice application and design of DL to synthesize CT from PET with thoughts on driving forces (structure and intensity). I am afraid that I feel its level of technical contribution and novelty is slightly lower than the typical MICCAI papers.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
Motivated by the easiness of using conventional AC processes, the document describes a method for generating CT scans from their PET counterparts.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper presents a novel formulation of an image generation method, with two sequential main modules, the first for skeleton prediction and the second for tissue rendering.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I am not sure the motivation of using conventional AC processes is strong enough to justify the generation of CT scans from PET scans to then correct the PET scan itself.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
There is no mention of making the source code available. Althought the dataset is public, the specific patients used are not.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I would like to congratulate the authors on their nice work. Some specific comments include:
- Make the source code publicly available.
- Add an ablation study (e.g. with and without thresholding the CT)
- Why the unbalance in the distribution of the selected data per pathology?
- Methods used for comparison range from 2016 to 2021. No more recent works of interest exist?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper has an acceptable degree of novelty and applicability.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This work proposed to perform attenuation correction with PET image alone. In order to achieve that, pseudo CT image is generated from non-attenuation corrected PET images via a two-step simulation process bridged by a segmentation map. PET image is first mapped to a CT-segmentation map, then further mapped to the CT image. The benefit of adding this bridge is that the two steps can be decoupled and utilize different loss functions for each task, which makes it easier for achieving structural fidelity. The generated CT image can then be used for attenuation correction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method introducing segmentation map as a bridge, although simple, is reasonable and effective for the candidate task. Sufficient evaluations and comparisons are performed to validate the method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Local metrics related to PET analysis can be further added, also some phantom image results with known ground truth. Model generalizability can be a potential challenge for practical application
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Network training, especially GAN part, can be challgenging for reproduciblity
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In several places “volume rendering” is used rather than “tissue rendering” in other places, I think tissue rendering is a better choice, since volume rending has a particular meaning for data visualization, which could cause some confusion Thresholding is used to generate the segmentation map, although I like this simple and deterministic design, I wonder if the accuracy of this step can have any impact on the final accuracy. For example, what if all masks in this works are generated by the TotalSegmentator?
Effectiveness in PET attenuation correction is currently illustrated by “overall metrics” – MSE, PSNR, etc. However, one major goal of AC is to recover local metrics for PET analysis – uptake values, SUVs, etc. Some such local stats can be included to illustrate the capability of this method.
To further support the method, phantom images with known ground truth uptake distributions can be included. This can also help illustrating the generalizability of the proposed method
Note that some of the above recommendations are more for an “extended/journal” version of this work, I fully understand they may not be fitted within the page limit of MICCAI .
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Simple yet effective design, works well for the candidate task, comprehensive evaluation.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
Rebuttal further addressed some of my questions, I decide to keep my initial rating.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Assessment: The paper addresses an interesting problem of generating synthetic CT from PET images with the goal of avoiding the obtaining a CT scan together with the PET scans. The authors focus specifically on the problem of CT generating for attenuation correction for improving the PET image. While technically interesting, this is also highly restricted in scope and its use in the clinic could be motivated better. Furthermore, as pointed out by reviewers, the metrics to evaluate the accuracy of synthesized CTs may not be sufficient. This is particularly important as the translation is done from a functional imaging modality that captures glucose uptake in this instance to an anatomic modality such as CT, which captures very different tissue characteristics. How reliable are the generated skeletons and CT densities for use in the attenuation correction? Also comparison to other methods are lacking, which is necessary to demonstrate accuracy and performance improvement.
To address in rebuttal: Please explain what is the clinical application for this problem. The lack of metrics to evaluate the accuracy of generated CT images (tissue densities as well as bony anatomy) should be addressed with local patch based metrics. In particular, it is also important to include a discussion of the limitation of the approach as the translation happens from a functional imaging modality to an anatomic modality. Discussion of the chosen baselines for comparison should be presented.
Author Feedback
Clinical application In typical PET/CT machines, CT can provide both attenuation coefficients for PET imaging and anatomical structure for location verification. This requires pseudo CTs to be similar to real CTs, thus it is necessary to focus on CT quality. We agree that modality synthesis or translation can directly generate AC-PET images from NAC-PET images. But such a strategy can neither provide anatomical structure nor generate better AC-PET images as demonstrated by our experiments (“no CT” in Table 1(b)).
Metrics to evaluate AC-PET We previously calculated PSNR, NCC, and SSIM for AC-PET because these metrics lead to the same conclusion on intensity comparison with the averaged metrics on anatomical structures. Meanwhile, we use segmentation results to evaluate the anatomy of CT since tissue types are highly related to attenuation coefficients. To efficiently evaluate pseudo CTs in correcting PET, we further calculated the local densities on each local structure of the corrected PET, where our method outperforms others in most structures as well as the overall performance.
-Limitation We agree that there are some limitations of our approach, the most challenging is that our synthetic CT images cannot compare with real CT images, as verified by our experiments. This is because the deficiency of structure information in functional PET can hardly be restored, although our methods can infer the most realistic anatomic structures. The second is that there may be different PET tracers, which have not been considered currently.
Justify CT generation by AC process The goal of AC process is to correct NAC-PET to AC-PET, which has attracted some CT-free attempts and is simulated by a generative approach in our study. It is proper since the better performance in AC process demonstrates that the generated CTs are more similar to real CTs.
Baselines for comparison We know that there are some MLAA-based techniques for PET correction. We did not follow MLAA for the following reasons. (1) MLAA methods are mainly applied to single tomogram, thus cannot utilize the contextual relationship among several contiguous tomograms as our study. (2) We can only perform simulation experiment for MLAA on our used datasets because they do not contain necessary raw TOF-PET data. On the contrary, our study can directly utilize real data. (3) Based on our preliminary experiments of MLAA methods according to recent studies [arXiv:2303.17042, DOI:10.1186/s12880-023-00987-7], these MLAA methods still need further improvement before applicable. (4) MLAAs are iterative methods that are more time-consuming than our direct inferring framework.
More recent works of interest It is right that there also exist some recent related works. For example, R Guo, et al. [5] published a method on Oct. 2022 that estimated AC-PET directly from NAC-PET by GAN, and its results are similar to the “no CT” method in our study. A Singh, et al. [J Nucl Cardiol, Apr. 2023] used pseudo-CT generated by GAN for registration of coronary PET. I Shiri, et al. [Eur J Nucl Med Mol Imaging, Mar 2023] integrated a modified U-Net in a federated learning framework for multi-institutional PET correction.
Reproducibility As mentioned by reviewers, our method is easy to implement. We will release code by our tradition, in which the datasheets will also be included.
Ablation study of thresholding In our unreported ablation study, our method with thresholding CT performs better than without. We do not use masks generated by TotalSegmentator because (1) it cannot provide whole-body segmentation and (2) storage of one-hot 3D labels will lead to overflow of GPU memory.
Imbalanced data and model generalizability. We selected all available samples in these datasets, the imbalance is due to the count of subjects in each dataset. When separately considered each dataset, same conclusions are derived.
Better term We agree “tissue rendering” is more proper than “volume rendering”.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper addresses an interesting problem of generating synthetic CT from PET images with the goal of avoiding the obtaining a CT scan together with the PET scans. The authors focus specifically on the problem of CT generating for attenuation correction for improving the PET image. While technically interesting, this is also highly restricted in scope. The authors’ response still doesn’t convincingly clarify the use case for “PET only” scans, especially because the approach cannot produce CT of similar quality as real CTs. The authors clarified the rationale for not comparing against current MLAA methods as well as the metrics used. The paper is potentially interesting but does have a limited application and limitations in the comparisons performed which should be clearly clarified in the discussion.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
No major weakness yet some concerns about performance and application remain. Will the source code be made available?
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes a method, anatomical skeleton-enhanced generation (ASEG), to generate pseudo CT images from non-attenuation corrected PET images for attenuation correction. Reviewer #2 questioned the motivation behind the work which was effectively addressed in the author’s rebuttal. Reviewer #1’s concern regarding the paper’s focus on CT quality was well-addressed by the authors, explaining the importance of structural fidelity in clinical applications. Overall, the authors successfully addressed the concerns raised by reviewers in their rebuttal. The innovative nature of the work and robust evaluations provide strong support for its acceptance. However, potential issues regarding the generalizability of the model and lack of MLAA benchmarks should be addressed in future work. Hence, I recommend acceptance.