Authors

Zhiwen Yang, Yang Zhou, Hui Zhang, Bingzheng Wei, Yubo Fan, Yan Xu

Abstract

Multi-center positron emission tomography (PET) image synthesis aims at recovering low-dose PET images from multiple different centers. The generalizability of existing methods can still be suboptimal for a multi-center study due to domain shifts, which result from non-identical data distribution among centers with different imaging systems/protocols. While some approaches address domain shifts by training specialized models for each center, they are parameter inefficient and do not well exploit the shared knowledge across centers. To address this, we develop a generalist model that shares architecture and parameters across centers to utilize the shared knowledge. However, the generalist model can suffer from the center interference issue, i.e. the gradient directions of different centers can be inconsistent or even opposite owing to the non-identical data distribution. To mitigate such interference, we introduce a novel dynamic routing strategy with cross-layer connections that routes data from different centers to different experts. Experiments show that our generalist model with dynamic routing (DRMC) exhibits excellent generalizability across centers. Code and data are available at: https://github.com/Yaziwel/Multi-Center-PET-Image-Synthesis.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43898-1_4

SharedIt: https://rdcu.be/dnwAB

Link to the code repository

https://github.com/Yaziwel/Multi-Center-PET-Image-Synthesis

Link to the dataset(s)

N/A

Reviews

Review #4

Please describe the contribution of the paper

The authors proposed a novel dynamic routing strategy with cross-layer connection for multi-center PET data image synthesis. The proposed DRMC model achieved excellent generalizability and outperformed state-of-the-art methods when applied to both known and unknown datasets. The paper is well-written and the proposed strategy is described in detail.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. Novelty: The authors proposed a flexible and novel dynamic routing strategy and successfully applied it to a transformer block for the PET image synthesis task.
2. Clarity: The authors did an excellent job describing the proposed strategy in detail.
3. Experiments: The authors conducted extensive experiments that demonstrated the excellent generalizability of the proposed DRMC model when applied to known and unknown datasets. Additionally, the selection of hyperparameters and activation functions was well-justified.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Generalizability: While the proposed DRMC model exhibited excellent generalizability, the use of the same DRFs for both known and unknown centers could limit the evaluation of the model’s generalizability. It would be beneficial to evaluate the model’s performance using “unknown” datasets with different DRFs.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Good. The authors described the method in detail and will share the code and data.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Please see weakness section. I think it would also be interesting to evaluate the generalizability of the proposed model using different PET tracers.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is well written, with novel method proposed and extensive experiments performed.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

In this paper, the authors present a novel strategy for dynamic routing with cross-layer connections in a generalist model, aimed at addressing the challenge of multi-center PET image synthesis. The experimental results demonstrate that the proposed method surpasses existing approaches and demonstrates exceptional generalizability across centers.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed dynamic routing method establishes cross-layer connections to facilitate more precise expert decisions, thereby enhancing the generalizability of the network for both known and unknown centers. In view of the multi-center data characteristics, multi-GPU collaboration is leveraged for training.

In this paper, a comparative analysis of the proposed model and existing methods is conducted for both known and unknown centers, demonstrating the superiority of the proposed generalist model. Furthermore, an ablation study, including a hyperparameter analysis, is designed and executed to thoroughly evaluate the model’s performance.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The results presented in Table 5 highlight that the performance of DRMC is comparatively weaker for some datasets when compared to Specialized methods. This observation merits further discussion to elucidate the potential reasons for the suboptimal performance of DRMC and identify opportunities for future improvements.

The language and organization of the manuscript should be improved to better convey the significance of the results and to avoid typographical errors or figure indication errors.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors have given enough information to reproduce the work.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Introduction section, Second paragraph, Line 3: Fig.2 should be Fig.1. The introduction section of the manuscript should include a discussion of the clinical significance of PET image synthesis. Typo: Section 3.3 “Unkown”. Proofread the paper. The results presented in Table 5 indicate that the performance of DRMC is inferior to that of specialized methods for certain datasets. This finding warrants further discussion to better understand the reasons behind the observed differences in performance and to identify potential avenues for improvement.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The novel dynamic routing approach proposed in this study utilizes cross-layer connections to improve the accuracy of expert decisions and enhance the generalizability of the network to both known and unknown imaging centers. Given the unique characteristics of multi-center data, the authors employed multi-GPU collaboration during the training process. The organization of the paper could benefit from minor improvements.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #5

Please describe the contribution of the paper

The authors proposed an innovative generalist model with dynamic routing (DRMC) for multi-center PET image synthesis addressing domain shifts and reported improved generalizability compared to other specialized and generalist models.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The novel dynamic routing strategy in the generalist model could successfully address the center interference issue and enable multi-center full-dose PET image synthesis with a single unified model.

The evaluation is comprehensive with both comparisons for known and unknown centers. The ablation study shows the effectiveness of the routing strategy.

The paper is well-written and clearly organized.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

It looks like the authors only addressed the domain shift across different institutions but didn’t evaluate the different low dose levels (the low-dose data were generated under quarter dose only). It will be of great interest to also evaluate on different low dose levels.

The authors only reported PSNR and the SUV biases. Including more metrics such as MSE and SSIM is more helpful. Also, the significance test results are missing.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors included implementation details and also plan to publish the code and data.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

In Fig. 3 the captions are not quite descriptive. For Fig. 3(a), suggest first showing the low-dose image as the reference of the input and adding the color bar for the second row. I assume the third row is the error maps, and please also add corresponding descriptions to it.

If possible, adding a comparison of the training/inference time and parameters.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed generalist model with dynamic routing is novel in multi-center PET denoising and the paper is well-written, though with some minor concerns.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper proposes a novel dynamic routing strategy with cross-layer connection for multi-center PET data image synthesis with strong experimental results and thorough evaluation. And the proposed strategy is described in detail. The main weaknesses identified include weaker performance for some datasets, lack of evaluation on different low-dose levels, missing significance test results, and a need for further evaluation of generalizability. Overall, this paper is well-written and clearly organized, further improvement is also welcomed and expected.

Author Feedback

We sincerely appreciate the reviewers for acknowledging our methodological contribution and providing constructive comments for further clarification. Our feedback is as described in the following text. Q1(MetaR&R3): Weaker performance for some datasets when compared to specialized models. A1: We anticipate that if there is sufficient data available in each center, the performance of the specialized model in the specific center will surpass that of the generalist model. This is because the specialized model is well-trained and can better preserve domain-specific information. Moving forward, our focus will be on enhancing our algorithm to further improve the preservation of domain-specific information. Q2(MetaR&R4&R5): Lack of evaluation on different low-dose levels. A2: We tested the performance of the models on the C6 dataset with different DRF values of 3, 4, 6, and 12. The average PSNR values for each model are 3D-cGAN 47.84, 3D CVT-GAN 47.83, FedAVG 47.69, FL-MRCM 49.11, FTL 49.85, and DRMC 49.96. Q3(MetaR&R3&R5): Missing significance test results. A3: We conducted a significance test, and the results demonstrate that our improvements are statistically significant (p<0.05). The results will be presented in our final paper. Q4(MetaR&R5): Further evaluation of generalizability. A4: We evaluated the SSIM metric of the models on multi-center data, and the results are as follows: 3D-cGAN 0.8210, 3D CVT-GAN 0.8934, FedAVG 0.9020, FL-MRCM 0.8758, FTL 0.8952, and DRMC 0.9160. Other minor issues will be addressed in our final paper.

back to top

DRMC: A Generalist Model with Dynamic Routing for Multi-Center PET Image Synthesis