Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Tianshu Zheng, Ruicheng Ba, Xiaoli Wang, Chuyang Ye, Dan Wu

Abstract

Diffusion MRI (dMRI) is a well-established tool for probing tissue microstruc-ture properties. However, advanced dMRI models commonly have multiple compartments that are highly nonlinear and complex, and also require dense sampling in q-space. These problems have been investigated using deep learning based techniques. In existing approaches, the labels were calculated from the fully sampled q-space as the ground truth. However, for some of the dMRI models, dense sampling is hard to achieve due to the long scan time, and the low signal-to-noise ratio could lead to noisy labels that make it hard for the network to learn the relationship between the signals and labels. A good example is the time-dependent dMRI (TD-dMRI), which captures the microstructural size and transmembrane exchange by measuring the signal at varying diffusion times but requires dense sampling in both q-space and t-space. To overcome the noisy label problem and accelerate the acquisition, in this work, we proposed an adaptive uncertainty guided attention for diffusion MRI models estimation (AUA-dE) to estimate the microstructural parameters in the TD-dMRI model. We evaluated our proposed method with three different downsampling strategies, including q-space downsampling, t-space downsampling, and q-t space downsampling, on two different datasets: a simulation dataset and an experimental dataset from normal and injured rat brains. Our proposed method achieved the best performance compared to the previous q-space learning methods and the conventional optimization methods in terms of accuracy and robustness.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43993-3_14

SharedIt: https://rdcu.be/dnwNf

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper considers an essential topic in diffusion MRI, i.e., parameter estimation under noisy labels. The article goes far beyond a standard protocol used in diffusion MRI and considers the joint q-t space, i.e., sampling in the q-space (on the sphere) and t-space, which refers to the effective diffusion time. The Authors proposed an end-to-end estimation method that could be used in a subsampled q-t space scenario.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper has good motivation, and the problem is up to date. The method has the potential to be used in clinical conditions.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    After reading the abstract, I felt this was an excellent job. Then, starting from section 2, the article is poorly written, including broken sentences, beginning with capital letters in the middle of the sentence, and non-symbolic math. Another example: sometimes the same symbol is written in a standard font and sometimes in bold, e.g. in Eq. (9) and the first line of page 5.

    The paper is very difficult to follow. It needs to be clarified what the contributions of the article regarding Zheng et al., MICCAI, 2022 are.

    It is unclear how the authors evaluated the method (see section 9 for some comments).

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    It is difficult to determine the reproducibility of the paper.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    The symbol t_d has not been defined.

    In Eq. (15) the kurtosis K is missing.

    It is not clear how the dictionary is constructed.

    The Authors write: “we plugged the varying parameters (K0, τm) into Eq. 15” – I think, the correct reference should be Eq. 14. However, I still need clarification on how the authors generated a synthetic data set.

    How did the Authors measure the SNR of the signal?

    It is not clear what Table 1 shows. Did you aggregate the kurtoses from all gradients? Did the Authors fit Eq. (15) to the data in each diffusion gradient?

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is poorly written except for the abstract. The paper is difficult to follow. It is not clear what are the contributions of the paper compared to Zheng et al., MICCAI, 2022.

  • Reviewer confidence

    Somewhat confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposed an adaptive uncertainty guided attention model dedicated to diffusion MRI models estimation (AUA-dE) and applied it to time-dependent diffusion MRI (TD-dMRI) to estimate the microstructural parameters of the TD-dMRI model.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Formulate an adaptive uncertainty guided attention mechanism and incorporate it into a deep neural network dedicated to the estimation of dMRI model parameters.
    • Interesting results of estimation improvement with respect to existing methods.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The effective link between the analytical formulation and the black-box mapping (so, highly nonlinear) was not clearly explained.
    • The key aspect “adaptive” was not explicitly and clearly explained.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    N/A.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    This paper proposed an adaptive uncertainty guided attention model dedicated to diffusion MRI models estimation (AUA-dE) and applied it to time-dependent diffusion MRI (TD-dMRI) to estimate the microstructural parameters of the TD-dMRI model. The originality of the work is that the authors have mathematically formulated the adaptive uncertainty guided attention mechanism and incorporated the latter into a deep neural network dedicated to the estimation of dMRI model parameters. The results of estimation improvement with respect to existing methods are interesting.

    Although the network architecture in the present work is similar to that reported in “An adaptive network with extragradient for diffusion MRI-based microstructure estimation. MICCAI 2022”, the new “adaptive” attention mechanism is still interesting. However, this key “adaptive” aspect was not explicitly and clearly explained. In the same sense, the effective link between the analytical formulation and the black-box mapping ((so, highly nonlinear) was not clearly elucidated.

    Other points:

    • The authors claimed “We brought up an important problem of the noisy label in dMRI model estimation which was not addressed before.” However, they didn’t mention what the label noise is in their application. Is it the noise present in real acquired signals in dMRI?
    • The authors said “we used the parameters estimated from fully sampled q-space as the “gold standard”. How is “fully sampled q-space” defined? The downsampling proposed by the authors might not make sense if the choice of the “gold standard” itself were not appropriately justified.
    • Is something missing in Eq. (15) ? (the parameter K?)
    • The authors wrote “ 5 td (50, 100, and 200 ms) with the following acquisition parameters”, but I didn’t find the 5 td!
    • The authors said “Fig. 2 showed that network structures”, but Fig.2 does not show network.
    • Please cite the refs in the order of their appearance.
    • The sentence after Eq. 4: I didn’t see where the adaptive uncertainty mechanism is reflected.
    • Throughout the paper, lower case of the letter” Where” after the formulae should be left aligned.
    • “the fully sampled q-space”: to specify what the “fully sampled” means for q-space, t-space and q-t space.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    • Solid mathematical formulation of the proposed adaptive attention mechanism.
    • Unclear link between the above analytical formulation and the black-box mapping of the used network architecture.
  • Reviewer confidence

    Somewhat confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The authors proposed an adaptive uncertainty guided attention (AUA) model to estimate TD-dMRI parameters. The proposed network has been evaluated using simulation data and a rat brain dataset. In addition, the authors have demonstrated the model’s outstanding performance on jointly downsampled q-t space data, which previous algorithms could not handle effectively.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Novelty: the authors introduce a novel deep learning network called AUA-dE that utilizes attention-based sparse encoding and uncertainty-based reweighting to enable end-to-end parameter estimation in both q-space and t-space, even with data that has been downsampled in both q- and t-space.
    2. Clarity: the authors have done an exceptional job of clearly describing the AUA module and its underlying theory in great detail, making it easy to understand and follow.
    3. Experiment: the proposed method was rigorously tested through extensive experiments on both simulated and in-vivo datasets, providing solid evidence for the effectiveness of the approach.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. I have some concerns about the performance of the proposed model illustrated in Fig. 2. While it does outperform the other three baseline methods, I am still apprehensive about the ~10% median K0 error and ~25% tau_m error with SNR=10. Additionally, the relative error shows significant variability, with a maximum of ~45% for K0 and 125% for tau_m. Such errors seem to exceed acceptable levels for clinical settings. As the performance test results are presented as MSE instead of relative error, I am not sure about the error range for the rat data. Could the authors please comment on this? Please correct me if I misunderstood sth.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    OK. Despite not sharing the code/data, the authors described the network architecture clearly with detailed theoretical analysis.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    Please see the weakness section.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the paper is well written and presents a novel method along with extensive experiments. However, I have some concerns regarding the performance of the proposed methods in clinical or research applications.

  • Reviewer confidence

    Somewhat confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    5

  • [Post rebuttal] Please justify your decision

    The authors have addressed my concerns satisfactorily. I will maintain my weak accept (5) rating, but lean towards accepting it.



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    Reviewers appreciated that this manuscript deals with a timely and relevant problem. Even though presentation still requires substantial improvements, this might be feasible within a revision. In the rebuttal, it would be helpful if the authors could focus on

    1. the relative contribution with respect to a related MICCAI paper from last year [14] (R1/2)
    2. if and how, in their final version, they would make a stronger link between the presented mathematical foundation and the actual implementation (R2)
    3. if and why they consider current results to be acceptable for clinical use (R3). They should also answer some of the more detailed reviewer questions as space permits.



Author Feedback

For all reviewers and Meta-reviewer: We will check for typos and improve readability. R1/2: Both articles use an extragradient framework and application in microstructure estimation. Previous work focused on downsampling q-space and the gold standard was considered satisfactory. This work focuses the noisy label (inaccurate gold standard) problem, which is particularly important in TD-dMRI, due to the low SNR and less dense sampling in q-t space. Therefore, we extend the previous model with an adaptive uncertainty to mitigate noise effects. The uncertainty threshold is trained on data, not empirical. Results show our network counteracts noise effects. R2: Due to length constraints, we can’t elaborate on the link between the equations and implementations here. In the final version, we’ll provide detailed descriptions and examples, e.g., annotating Fig.1 with corresponding equations and indicating in legend which network part corresponds to which equation. Further, we’ll explain equations in text, e.g., threshold function corresponds to threshold layer and dictionary matrices to dense layers (W, S). We’ll also provide implementation references for more information. R3: In practice, SNR is generally above 30, where the median estimated relative errors are 5% and 12%. Fig.2 gives an extreme case at SNR=10 to show the noise resistance of network, which is at least 14% better than AEME. The clinical utility is indicated in Fig.3, where only our method captures the permeability (1/tm) rise in ischemia brain injury. Our recent study on human brain indicates that the median relative error is below 3%, at 4x acceleration in q-t space. To R1 1) See meta-review 1 2) td=diffusion time 3) Eq.15 is modified. 4) The dictionary is constructed by assuming that diffusion signals can be sparsely represented by two separate dictionaries corresponding to the spatial and q-t domain. We construct dictionaries to obtain a sparse representation X of the original signal Y. This better reflects the signal’s characteristics and train the mapping network. See Ye et al, MedIA 2021 for details. 5) Correct reference: Eq.14. In section 2.3, synthetic data are generated according to Eq.14 using preset parameters (td, K0, tm) and signal formula. 6) SNR is defined from 10~30 and we follow ref[1] and randomly generate a normal distribution with a mean of 0 and a standard deviation of 1/SNR with respect to a normalized signal ([0-1]). 7) Table 1 shows kurtosis results for downsampled q-space (2 b-values with 9 gradients each) at different diffusion times (50, 100, 200ms) using three methods and averaged across all subjects. The gold standard was computed using the fully sampled q-space with the diffusion kurtosis method from ref[11], using data in each diffusion direction. We’ll remove Eq.15. To R2 1,3) See meta-review 2 2) “Adaptive” means that uncertainty threshold is trained by network, not empirical. Unlike AEME where “adaptive” refers to selecting the number of iterative blocks adaptively. 4) Noisy labels can be caused by real signal noise as well as fitting errors (when fitting with limited samples). In TD-dMRI, the STEAM sequence has low SNR, and dense sampling in q-t space is challenging due to limited scan time, so labels are noisy. 5) “Fully sampled q-space” is acquired with 18 directions per b-value at 3 b-values (0.8, 1.5, 2.5 ms/μm2). Ref[11] states that signals acquired with b-values greater than 2 with at least 15 gradients each are sufficient for kurtosis fitting. 6) We’ll modify Eq.15. 7) 5 td: 50, 80, 100, 150, 200 ms. Will fix. 8) Should be Fig.1. 9) We’ll reorder refs by appearance. 10) The adaptive uncertainty mechanism is referred to as AUA in Eq.2-3 and is illustrated in section 2.2. 11) We’ll fix “where” after the formula and left aligned. 12) For fully sampled q-space: see 5). For fully sampled t-space, we use 5 td, according to ref[10]. Fully sampled q-t space implies joint full sampling of q-space and t-space. To R3 See meta-review 3



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The rebuttal provides satisfactory answers to the main questions that were raised in the original reviews, and indicates that authors will adequately address the reviewer concerns in their final version.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    According to reviews and authors’ feedback, this work requires further enhancement. The paper’s current state does not meet the standard set by MICCAI. Hence, I suggest rejecting this work.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    In the rebuttal, the authors have addressed the comments satisfactorily. The different contributions of the current work and the existing relevant paper have been clarified. Necessary demonstrations have also been provided.



back to top