List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Junqing Yang, Haotian Jiang, Tewodros Tassew, Peng Sun, Jiquan Ma, Yong Xia, Pew-Thian Yap, Geng Chen
Abstract
Deep learning has drawn increasing attention in microstructure estimation with undersampled diffusion MRI (dMRI) data. A representative method is the hybrid graph transformer (HGT), which achieves promising performance by integrating q-space graph learning and x-space transformer learning into a unified framework. However, this method overlooks the 3D spatial information as it relies on training with 2D slices. To address this limitation, we propose 3D hybrid graph transformer (3D-HGT), an advanced microstructure estimation model capable of making full use of 3D spatial information and angular information. To tackle the large computation burden associated with 3D x-space learning, we propose an efficient q-space learning model based on simplified graph neural networks. Furthermore, we propose a 3D x-space learning module based on the transformer. Extensive experiments on data from the human connectome project show that our 3D-HGT outperforms state-of-the-art methods, including HGT, in both quantitative and qualitative evaluations.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_3
SharedIt: https://rdcu.be/dnwM4
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a new microstructure estimation model called 3D-HGT, which utilizes both 3D spatial information and q-space information jointly. The proposed model incorporates an efficient q-space learning module based on simplified graph neural networks and a 3D x-space learning module based on a U-shape transformer. The experiments on data from the Human Connectome Project demonstrate that 3D-HGT outperforms cutting-edge methods, including HGT, both quantitatively and qualitatively.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper proposes a novel microstructure estimation model, 3D-HGT, that makes full use of 3D spatial information and q-space information jointly. The proposed model consists of an efficient q-space learning module and a 3D x-space learning module. The q-space learning module is built with Simplifying Graph Convolutional Networks, and the x-space learning module consists of a U-shaped network composed of Local Volume-based Multi-head Self-attention and Wide Volume-based Multi-head Self-attention. The proposed model outperforms the state-of-the-art methods in terms of quantitative and qualitative evaluation. The paper is significant as it introduces an efficient and effective model for microstructure estimation, which can find its applications in various clinical and research settings.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper claims that the proposed model outperforms previous models in terms of both efficiency and effectiveness. However, the paper lacks sufficient comparisons to existing state-of-the-art methods and does not provide a thorough analysis of the model’s clinical feasibility.
Main points:
- The paper does not provide a clear explanation of its novelty compared to existing models.
- The paper does not provide an in-depth analysis of the limitations of the proposed model, such as its performance on more complex datasets, its robustness to noise and artifacts, and its generalizability to different populations.
- Insufficient description of the datasets and experimental setup.
Minor:
- Some of the terminology used in on the q-space learning module may not be clear to the reader.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Overall, the paper seems to be well-documented and provides the necessary information to reproduce the results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Overall, the paper appears to be well-written and well-organized, with a clear description of the proposed method and experiments.
However, there are a few areas that could be improved:
-
The authors should provide more details on the hyper-parameters used in the experiments. This information is important for reproducibility.
-
The authors compare their method with previous work, but the comparison is limited. It would be helpful to compare with more previous work to provide a more comprehensive comparison.
-
The authors could also provide more details on the limitations of their proposed method and potential future directions for improvement.
-
The figures could be improved by providing more clear and informative captions.
Overall, the paper presents a promising method for dMRI microstructure analysis, and with a few improvements, it could be even more impactful.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well-written, has a good method with some novelty, and it is interesting to the community. I suggest moderate improvements overall.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors propose an advanced microstructure estimation model in the form of a 3D-hybrid graph transformer which leverages 3D x-space information and 2D q-space angular information jointly. To this end, this paper proposes an efficient q-space learning module based on simplified graph networks and a 3D x-space learning module based on the U-shaped transformer. This architecture also helps to deal with the additional computational load due to the processing of 3D spatial information and improves processing speed. End-to-end training of 2 modules is done to predict microstructure index maps.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is concise, communicates the core ideas well and supports the experiments with suitable validation metrics, tables and visual representations.
- The tables and figures are well done especially Fig.2 which gives a visual comparison of the results from the different methods.
- The 3D-HGT method is definitely novel and tackles an important problem of incorporating 3d spatial information for microstructure estimation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
Were there any improvements to the results upon changing the batch-size or the size of the training data ? (given the high-volume of the data). Also, were there any noticeable differences in the results upon changing the diffusion directions ?
-
Sec 3.3, pg. 7, the term ‘ALL’ does not appear to be clearly defined. At first, I assumed it is the average of icvf, isovf and odi for a given method, but the numbers mentioned for PSNR (ALL) on page 7 versus the numbers in the table do not correlate. Please clarify this.
-
Table 1: While there are improvements in PSNR and SSIM, the improvements appear to be not significant compared to the previous best-performing method. A 1.1 % SSIM improvement given the increase in architecture size and computational complexity seems to be modest .
-
Table 2: the ablation study is quite detailed, though did the authors also validate this on a model involving SGC + 2D Trans.
-
Table 2: Please add the metrics used for measuring time (it’s mentioned in the text, but it would be nice to have this in the table too). Also, it would have been great to see the corresponding SSIM values as shown in Table 1.
-
The results shown in Table 2 again show an improvement of 0.5% over C (with a time reduction of 30 s) and 4% over A (with an increase in time of nearly 75 seconds). Again, the improvements shown by 3D-HGT do not appear significant enough especially when one considers the tradeoff with computational time.
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
the authors meet all the criteria on the reproducibility checklist.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please refer to the comments mention in Pt.6 - it would be great to get details on effects of batch-size, training data size and variation in number of diffusion directions.
Also, the following method from a recent paper was not included for comparison - it is a recent MedIA paper though the arxiv version seems to be there since 2021. Out of curiosity, it would have been great to see how the 3D HGT method compares against this - https://www.sciencedirect.com/science/article/abs/pii/S136184152300049X
Discussion point: On page 8, “The improvement is due to the 3D x-space learning module, which is equipped with LSAs and WSAs, allowing the model to capture long-term dependencies with a large 3D receptive field.” - looking at Table 2, if one compares B (only 3D Trans) with C (TAGCN + 3D Trans) and D (SGC + 3D Trans), the effect of adding a graph network on the PSNR seems very modest. So yes, in that sense, the highlighted statement holds true but then again brings into question the overall effectiveness of adding a graph network (or if these graphs can be optimized further to improve the results). I would love to hear the authors’ view on this.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The 3D-HGT is definitely novel and addresses the important point of incorporating 3D q-space information while estimating microstructures. However, the overall improvements in PSNR and SSIM are not significant enough, and do not justify the tradeoff with the increase in computational load. Moreover, it has also not been shown if any changes in batch-size/training data size would have affected the performance.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper introduces a 3d hybrid graph transformer for microstructure reconstruction with undersampled dMRI data. With dMRI data as the input, the proposed 3d-hgt method incroporates a pipeline with efficient q-space learning 3d x-space learning and convolutional/linear layers to output microstructure indices of recovered 3d images. Transformers and self-attention based modules have been intensively used in q-space learning and x-space learning phases. Authors also evaluate the proposed method with Human Connectome Project (HCP) datasets. The results demonstrate the effectiveness of the proposed method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
S1. Interesting technical problem with strong research motivation (recovering 3d microstructures from undersampled dMRI data) S2. Comprehensive pipeline with intuitive design to solve the problem S3. Extensive evaluation with real datasets.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
W1. Lack of novelty, To model 3D spatial information, the proposed method leverages 3D hybrid graph transformers, where geometric information, such as angular information, has been used to model the 3d spatial spaces.
W2. The proposed method has been evaluate using only one set of data (HCP for brain). More comparisons are desired to understand the effectiveness of 3d-hgt for other datasets or other body parts.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I think the paper should be reproducible upon the code releases.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Authors should consider to incorporate additional results on other datasets.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript is well-written with high-quality contents. The problem to solve is well motivated and the proposed method is straightforward.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The proposed model, called 3D-HGT, aims to improve microstructure estimation by utilizing both 3D x-space information and q-space information together. The model consists of two key components: a q-space learning module based on simplified graph neural networks for efficient computation and a 3D x-space learning module based on a U-shape transformer for accurate segmentation of 3D medical images. These modules are trained end-to-end to predict microstructure index maps. Experimental results using data from the Human Connectome Project show that the 3D-HGT model outperforms state-of-the-art methods, including HGT, in terms of both quantitative and qualitative evaluation. The model effectively leverages 3D spatial information and 2D angular information to improve microstructure estimation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper’s writing is effective in communicating the research findings, and the figures are well-designed to illustrate the main concepts. The proposed idea extends previous SOTA HGT from 2D to 3D.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The level of novelty in this paper may be perceived as limited since it builds upon the existing 2D HGT by extending it to the 3D domain. It may not deviate drastically from the existing framework.
-
Compared to HGT, regarding the experimental setup and the authors’ decision to change from 30 gradient directions to 60 and b=1000s/mm^2 to 2000 when comparing with previous methods, there could be several reasons behind this modification. Could you explain more on this?
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Looks reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please see weaknesses.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The level of novelty in this paper may be perceived as limited.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The proposed 3D-HGT method addresses the challenge of reconstructing microstructures using undersampled dMRI (diffusion magnetic resonance imaging) data. It employs a pipeline that combines efficient q-space learning, 3D x-space learning, and convolutional/linear layers. By taking dMRI data as input, the method generates 3D images and outputs microstructure indices, enabling the recovery of detailed microstructural information from the undersampled data.
Author Feedback
We sincerely thank AC and all reviewers for the valuable comments.
Novelty (R1, R3, R4) The novelty of our model lies in two aspects. First, it effectively addresses the limitation of the cutting-edge model, HGT, which overlooks the valuable 3D spatial information, by proposing an effective x-space learning module based on the 3D transformer. Second, a 3D network design involves significant computational burden. This issue is addressed with an efficient q-space learning model designed based on a simplified GNN. The effectiveness of these designs is verified by extensive experiments and ablation studies. Our model achieves SOTA performance in comparison with existing models, including HGT. Although our model is not the first to integrate GNN with transformer techniques, it addresses the limitation of existing hybrid designs with brand new x-space and q-space learning modules.
Experimental Details, Limitation, and Clarifications (R1) 1) Dataset: Consisted of patches with the same 3D spatial dimension of 32×32×32. 2) Model settings: 4 LSA layers and 3 WSA layers. 3) Details on limitations and clarifications will be included in the final paper.
Hyperparamters (R1, R2) Our model is not sensitive to the hyperparameters, e.g., batch size (BS), training sample size (TSS), and the number of gradient directions (GDs). The results show that there is no significant difference between the results given by different BSs (1 to 3), i.e., PSNR of ALL: 24.00, 24.01, 23.98. The performance improves slightly with the increase of TSS, i.e., PSNR of ALL: 24.01 (10 subjects) vs. 24.15 (15 subjects). With the increase in the number of GDs, the performance increases, but not very significantly, i.e., PSNR of ALL: 24.01 (30 GDs) vs. 24.60 (45 GDs). However, both increasing TSS and GDs significantly prolong the time cost, therefore are not adopted.
Comparison Methods (R1, R2) We compared our model with eight existing methods. We thank R2 for pointing out this valuable work and will include it in the final paper. This work was not officially published (i.e., arXiv preprint) and the code was not released when we submitted our work to MICCAI.
Meaning of ALL (R2) We use “ALL” to denote a combination of three NODDI-derived indices. Two results, 14% and 15.2%, should be switched, which will be corrected in the final paper.
SSIM (R2) SSIM is a normalized metric (0 to 1), therefore 1.1 % is a reasonable improvement for SSIM. It is also worth noting that the SSIM of ICVF has been improved by a large margin of 2.3%.
Ablation Study (R2) Thank R2 for the constructive comments. “SGC + 2D Trans.” outperforms (A) with a significant reduction of 17% computational time and a PSNR improvement of 0.12 dB. We acknowledge that the performance improvement of (D) over (C) is not large, however it is worth noting that (D) effectively improves the efficiency by reducing 13% computational time. Similarly, (D) improves the performance by a large margin of 4% over (A) without significantly extending the computational time. GNN can effectively improve the performance by explicitly modeling the q-space geometric structure, i.e., (D) vs. (B): 0.22/0.13 dB PSNR improvement on ICVF/ALL. We perform ablation experiments with low angular resolution undersampled dMRI data in the paper. Our results on undersampled dMRI data with 45 gradients show that GNN improves the performance more effectively with a PSNR improvement of 0.53/0.35 dB (ICVF/ALL). Missing details (metrics and SSIM results) will be added in the final paper.
Performance on Additional Data (R3) We have evaluation results on a patient dataset, called high-quality diffusion-weighted imaging of Parkinson’s disease. The results indicate that our model consistently provides the best performance, i.e., HGT vs. 3D-HGT: 19.30 vs. 20.07.
Two-Shell Undersampling (R4) The two-shell case is used to mimic the minimum requirement for fitting a NODDI model, which needs a muti-shell sampling.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Based on the reviews and the rebuttal the paper need more improvement and may be more suitable for a workshop.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Reviewers were split on this paper and I would say that, even after the rebuttal, it remains at the borderline: The proposed 3D extension of the previously described 2D hybrid graph transformer is plausibly motivated, but somewhat incremental. There is an additional benefit, but it is somewhat marginal, especially when considering the additional computational effort. In my opinion, the rebuttal contains useful additional information that should be integrated in the manuscript and will bring it slightly above the bar for getting accepted. In case this should not be the final AC/PC consensus, I would hope to see this paper submitted to the MICCAI CDMRI workshop.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper received 2 accepts (R1 and R3) and 2 rejects (R2 and R4) in the initial review phase. The authors provided their rebuttal to address the reviewers’ concerns. The reviewers did not give any inputs in the post-rebuttal evaluations. I looked at the paper and rebuttal. I agreed with R2 and R4 on the issues such as the novelty, implementation details and justifications, and result interpretations. Unfortunately, the authors did not respond to those issues well in their rebuttal. So I do not think the paper is currently in a good shape until those big concerns are cleared. I recommend to reject the paper.