Authors

Tianxu Lv, Yuan Liu, Kai Miao, Lihua Li, Xiang Pan

Abstract

Recent researches on cancer segmentation in dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) usually resort to the combination of temporal kinetic characteristics and deep learning to improve segmentation performance. However, the difficulty in accessing complete temporal sequences, especially post-contrast images, hinders segmentation performance, generalization ability and clinical application of existing methods. In this work, we propose a diffusion kinetic model (DKM) that implicitly exploits hemodynamic priors in DCE-MRI and effectively generates high-quality segmentation maps only requiring pre-contrast images. We specifically consider the underlying relation between hemodynamic response function (HRF) and denoising diffusion process (DDP), which displays remarkable results for realistic image generation. Our proposed DKM consists of a diffusion module (DM) and segmentation module (SM) so that DKM is able to learn cancer hemodynamic information and provide a latent kinetic code to facilitate segmentation performance. Once the DM is pretrained, the latent code estimated from the DM is simply incorporated into the SM, which enables DKM to automatically and accurately annotate cancers with pre-contrast images. To our best knowledge, this is the first work exploring the relationship between HRF and DDP for dynamic MRI segmentation. We evaluate the proposed method for tumor segmentation on public breast cancer DCE-MRI dataset. Compared to the existing state-of-the-art approaches with complete sequences, our method yields higher segmentation performance even with pre-contrast images. The source code will be available on https://github.com/Medical-AI-Lab-of-JNU/DKM.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43901-8_10

SharedIt: https://rdcu.be/dnwCU

Link to the code repository

https://github.com/Medical-AI-Lab-of-JNU/DKM

Link to the dataset(s)

https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=22513764

Reviews

Review #3

Please describe the contribution of the paper

The paper propose a learning based method refered as diffusion kinetic model for the segmentation of breast tumors on DCE-MRI. The paper reports better results compared with the state of the art and reports and ablation study that provides reasoning for the described options.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The results. The developed model.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Lack of supportive references. A large part of the work is based on previous works without suitable references expressing that. Lack of explanation on the division between training ad testing set.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Problems on the description of the training model make impossible to reproduce the work.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Issues found in the paper:

1) The abstract should be corrected. There are several language issues. For instance “generality ability”. 2) Acronyms are not always defined in order of appearance. E.g. DKM appears defined in section 2, but appears in section 1. DDPM multiple times defined. Furthermore, the paper use too many acronyms making the readable of the paper difficult. 3) Some definitions are not appropriate. E.g. “$x_K$ a sequence of images representing the DCE-MRI…”, should have “… the result of a DCE-MRI…” or better “… the result of the application of the DCE-MRI…” 4) Symbols should also be defined, which is not always the case. 5) Fig. 2 caption is also somehow strange as repeats what is in the text. However, no explication n of the modules is given. 6) All the mathematics formulation of (1) to (4) does not have appropriate references. Moreover, the explanation of the model is very vague. This is not appropriate because it seems the paper introduces this formulation (which is not true) and the explanation provided is not self-explanatory. 7) DDPM is described as new, and without any reference, while has been published. 8) Eq (7) of the cost function has no explanation, no comparison with previous work. This is not appropriate. The loss function is one of the most important parts of a system like the reported, and as the paper describes it seems something that come from a true mysterious force! Why lambda equal 0.5? 9) The description of the training does not exist, which is very important for reproducibility. For instance, how was data augmentation performed? 10) The division of the dataset is not appropriately described. The paper does not describe if the division into training and testing was of the scans or of the slices. This is mandatory because there are “published” works where the division of the MR dataset was of the slices instead of the scans. Of course, having slices of the same scan in the test and training dataset results in high performance, but that is not appropriate at all. Unfortunately, the paper is not objective on that description implying that I have problems accepting the paper as it is.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is in general good, but has this two problems that might be solved with the rebuttal: 1) training details for easy reproduction 2) better explanation of the division between training and testing sets 3) adding suitable references on the model explanation in the many parts that are not new.

I would also recommend a 10 fold cross validation as the paper uses a division between training and testing set.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #5

Please describe the contribution of the paper

Auhtors propose a diffusion kinetic model (DKM) that implicitly exploits hemodynamic priors in DCE-MRI and effectively generates high-quality segmentation maps only requiring precontrast images. A diffusion module is in charge of encoding hemodynamic response function and thus with pre-contrast images and encoded hemodynamic response function the system can achieve inspiring performance without sequential DCE-MRI nor post-contrast MRI.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The manuscript is easy to read and follow. The system design is neat and elegent.
2. The proposal is promising to me. Have one encoder to encode the hemodynamic response function so that we don’t need sequential data or post-contrast MRI.
3. The results look promising to me and it outperform other state-of-the-art system.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Relatively, the system was trained and tested in a small dataset which has only 64 patients. It would be interesting to check its performance in larger datasets.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors claim they would share the codes in the future. It would be easy to reproduce the results with codes public. Otherwise, it is hard to cover all architecture and training details in the manuscript for reproducing.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

1.In Table. 2, the best model is the one whose has only f3 and f4 from DM to be fed into segmentation module. But in Fig. 2 (b), it shows four features from DM are used in segmentation part. I believe this figure is just for illustration as an example. Please clarify.
2.How was the two constant in Eq. 7 set?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The system design is neat and logic is well-illustrated. Results look promising and it outperform other state-of-the-art systems. It is a good and standard manuscript to me.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #6

Please describe the contribution of the paper

This paper aims at improving the breast cancer segmentation on dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) using the proposed diffusion kinetic model (DKM). DKM learns hemodynamic priors in precontrast images DCE-MRI by a diffusion module, then annotates cancers from the latent kinetic code with a segmentation module. Claimed by the authors, the proposal is the first work that segments on precontrast breast DCE-MRI with diffusion model, and it outperforms existing state-of-the-art methods with complete sequences.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The idea is nouvel. The authors design a diffusion model to learn hemodynamic priors of breast tumor, i.e. from pre-contrast images to post-contrast images then use it as additional information to improve the segmentation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The forward and backward processes are not clear for me. As shown in Fig.2, only the latent kinetic code of the first diffusion step is shared with the segmentation model while I did not find its justification. Moreover, once the model is trained, do the diffusion steps from xt-1 to xk are still necessary?
- Equ. 7 lacks clarity. E.g., what is the dimension of µS and µG, why C1 and C2 can be different, etc.
- The reliability of the results. As far as I understand, one crucial characteristics of the tumour diagnostic in DCE-MRI is the different hemodynamics between normal and tumoral tissues while without the complete sequence, such information will be lost. Even though, the proposal still significantly outperforms the complete-sequence method using less semantics. I wonder if the comparison in Tab. 1 was carefully realized although the authors have discussed its potential reason.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors admit to publish the code in future. The employed dataset is public.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- How the authors justify the forward direction from x0 to xk instead of from xk to x0 when training the diffusion model?
- Fig. 1 and its illustration should be improved, e.g., legend of the time intensity curve, the meaning of the green flash, etc.
- Some typos, e.g., in Section 2.1 “DDPM approximate(s) the data distribution”; in Section 2.3 “varphiG represents the variance of G”
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The work is nouvel and shows great improvement against SOTA methods according to the authors’ comparison. However, the method lacks clarity, and I doubt the superior performance since the hemodynamic is not available in sole precontrast images.
Reviewer confidence

Somewhat confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper presents a learning based method refered as diffusion kinetic model for the segmentation of breast tumors on DCE-MRI. There was a concensus among the reviewers that the paper is clear and easy to follow, the model proposed is intersting and the results quality is good. Reviewers also noted the authors’ promise to make the code available upon acceptance. In the camera ready version the authords should address reviewers concerns. In particular, method’s details should be clarified, training-test split should be stated and motivated. In addtion, the results (Table 1) should be further discussed. Specifically, the improved segmentation results despite not having the complete DCE-MRI sequence should be explained. Relevant references should be added.

Author Feedback

We sincerely thank the meta-reviewer and all reviewers for their valuable feedback that we have used to improve the quality of our manuscript. Specific concerns are listed below and our response is given.

Q1: There are several language issues, inappropriate acronyms and definitions, as well as typos. A1: We will carefully check the gramma and content of the manuscript and solve these in the camera ready version. Besides, relevant references will be added.

Q2: The division of the dataset is not appropriately described. A2: The dataset was divided into training and testing sets based on the scans. Relevant description will be added in the camera ready version.

Q3: The loss function has detailed explanation. A3: Relevant explanation will be added in the camera ready version. The structural similarity loss is used to exploit the tumor structural information and strengthen inter-voxel similarity within the same tissue for segmentation. For the hyper-parameter, the lambda is set as 0.5 empirically. C1 and C2 are constants to avoid instability [1] when the denominator is close to zero. Following [2], C1 = (K1*L)2, C2 = (K2*L)2, where K1 = 0.01, K2 = 0.03 and L is the range of voxel values. More detailed explanation and experimental results will be added in our extended version.

Q4: The training details. A4: The training details will be added in the camera ready version according to the reviewer’s suggestions.

Q5: The dataset only has 64 patients. A5: We are conducting experiments on other larger datasets to verify the performance of the proposed method, which will be presented in the extended version.

Q6: The improved segmentation results despite not having the complete DCE-MRI sequence. A6: The relevant experimental results will be discussed in the camera ready version. The improved performance can be attributed to two aspects. 1) The proposed method utilizes the diffusion module to exploit hemodynamic knowledge from pre-contrast MRI, which is useful for tumor segmentation. 2) As reported in [3], the intermediate activations from diffusion models effectively capture the semantic information and appear to be excellent pixel-level representations for the segmentation problem. Thus, combining the intermediate features can further promote the segmentation performance.

Q7: The figures and their illustrations should be improved. A7: We will improve the figures and illustrations according to the reviewers’ suggestions.

Q8: The training diffusion process. A8: We employ the denoising diffusion process to transmute pre-contrast MRI into post-contrast MRI because this process is consistent with hemodynamic response function, which we think is helpful to segment tumors. Besides, pre-contrast MRI is more difficult to access compared to post-contrast MRI. So we consider extracting additional information from pre-contrast MRI instead of post-contrast MRI. In addition, we think that transforming post-contrast MRI into pre-contrast MRI can probably capture some useful information and we will conduct experiments to explore it.

[1] Zhou Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. [2] Jielian Lin, Hongbin Lin, Zhichen Zhang, Yiwen Xu, and Tiesong Zhao. Ssim-variation-based complexity optimization for versatile video coding. IEEE Signal Processing Letters, 29:2617–2621, 2022. [3] Dmitry Baranchuk, Ivan Rubachev, Andrey Voynov, Valentin Khrulkov, and Artem Babenko. Label-efficient semantic segmentation with diffusion models. In International Conference on Learning Representation (ICLR), 2022.

back to top

Diffusion Kinetic Model for Breast Cancer Segmentation in Incomplete DCE-MRI