List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Hong Wang, Minghao Zhou, Dong Wei, Yuexiang Li, Yefeng Zheng
Abstract
Sparse-view computed tomography (CT) has been adopted as an important technique for speeding up data acquisition and decreasing radiation dose. However, due to the lack of sufficient projection data, the reconstructed CT images often present severe artifacts, which will be further amplified when patients carry metallic implants. For this joint sparse-view reconstruction and metal artifact reduction task, most of the existing methods are generally confronted with two main limitations: 1) They are almost built based on common network modules without fully embedding the physical imaging geometry constraint of this specific task into the dual-domain learning; 2) Some important prior knowledge is not deeply explored and sufficiently utilized. Against these issues, we specifically construct a dual-domain reconstruction model and propose a model-driven equivariant proximal network, called MEPNet. The main characteristics of MEPNet are: 1) It is optimization-inspired and has a clear working mechanism; 2) The involved proximal operator is modeled via a rotation equivariant convolutional neural network, which finely represents the inherent rotational prior underlying the CT scanning that the same organ can be imaged at different angles. Extensive experiments conducted on several datasets comprehensively substantiate that compared with the conventional convolution-based proximal network, such a rotation equivariance mechanism enables our proposed method to achieve better reconstruction performance with fewer network parameters. We will release the code once the paper is accepted.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_11
SharedIt: https://rdcu.be/dnwwp
Link to the code repository
https://github.com/hongwang01/MEPNet
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposed an unrolled neural network algorithm for the joint sparse-view reconstruction and metal artifact reduction. It also leverages the advanced equivariant neural network architecture to embed the physical imaging geometry constraints in the neural network design, which improves the imaging quality and reduce the model parameters.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper comprise two major contributions: 1) a deep learning formulation that simultaneously considers reconstruction and metal artifact reduction, 2) a good application of equivariant neural network architecture to solve the reconstruction problem. It also reports massive experimental results that validates the effectiveness of the new method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
However, the paper is not that novel from the deep learning perspective. Both unrolled neural network and equivariant neural network are widely used in biomedical imaging applications. The paper only offers a smart combination of these methods. Also, the paper is not tested on real clinical data.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors promise to publicly release the codes after the paper being accepted.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper is clearly written and a good amount of validation experiments are included. However, all the validation experiments use simulation data. It would be good to see whether it performs well in real clinical conditions.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The strengths and weakness of the paper are both significant (see Q5 & Q6). However, given the significant improvements in simulation data, massive validation experiments and good writing quality, I would suggest weak acceptance.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The author doesn’t address all my concerns about technical novelty. However, I’d like to keep my rating.
Review #2
- Please describe the contribution of the paper
In this paper, the authors propose a joint numerical method that combines iterative solutions and deep learning networks to solve an optimization problem for reconstructing Sparse-view and Metal Artifact Reduction CT Data. The deep network is further enhanced with Rotation Equivariance to compress the encoded information and parameters. This novel framework addresses the challenging problem of Sparse-view reconstruction and Metal Artifact Reduction in CT imaging. The authors demonstrate that their approach outperforms state-of-the-art methods regarding image quality and computational efficiency.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Using Equivariant CNN to model two proximal networks is a unique and interesting idea that aligns well with CT imaging.
- The authors provide a detailed account of their experimental design, including standard metrics such as PSNR and SSIM, which allows for a comprehensive evaluation of the proposed framework’s performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The derivation largely overlaps with InDuDoNet, except for some additions to the Sparse-View Reconstruction.
- The authors must discuss the advantages of adding Group Equivariance to the CNN, and the ablation study on this topic needs to be included.
- The input shape and output shapes of the two CNN proximal networks are not clearly defined, which leaves questions about whether the networks are fully CNNs (FCN, autoencoder, UNet, etc.) for equivariant-related tasks such as reconstruction, as opposed to regular CNNs used for invariant classification tasks.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Based on the text, it is challenging to replicate the implementation without looking at the code. Releasing the source code would be an effective way to ensure reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Major comments:
- The chosen group (p8) is somewhat ad-hoc. It would be beneficial to make it complete SO(2) or SE(2) with zero translation.
- Fig.3: The results for under-sampling rates (a) x8, (b) x4, (c) x2 show decreasing PSNRs from 42.02 / 40.91 / 36.98. It is unclear why the PSNRs decrease as the under-sampling rate (or acceleration factor) decreases. According to Table 1, a lower under-sampling rate should result in higher PSNRs.
- Minor comments:
- In Equations (5) and (6), it would be helpful to distribute Y^\bar into the operand.
- A table summarizing the dataset statistics and train-val-test splits would be helpful in the dataset description.
- The resizing of images to 416x416 seems unusual. Clarification on whether pixel spacing was used for image normalization would be helpful.
- Major comments:
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper fits the MICCAI theme and offers a unique approach to Sparse-view reconstruction in CT imaging. However, the direct contribution of the Rotation Equivariant CNN (p8) is not well-discussed, and as a result, the evaluation of the paper is borderline toward weak acceptance. Nevertheless, the paper is well-researched, and the author’s efforts are admirable. The paper is engaging and enjoyable to read, and it is a rare submission that combines classical optimization techniques using proximal methods with deep learning networks.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors addressed my comments in the rebuttal.
Review #3
- Please describe the contribution of the paper
This paper proposed a model-driven equivariant proximal network used for joint sparse-view reconstruction and metal artifact reduction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The topic is interesting and practical. The paper is well organized. The proposed method is reasonable and solid.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The workflow of proposed method wasn’t clear shown and described.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I think it can be reproduced.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
It is better to provide a figure to display the framework. Could current method be used for 3D sparse-view reconstruction? Maybe it is more significant.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well organized. The proposed method is reasonable and solid. The application scenario is limited for 2D slice reconstruction.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors present an approach to joint sparse-view reconstruction and metal artifact reduction for CT imaging with a deep method. Although the three reviewers agreed to recommend the paper with weak accept, many major concerns were still raised, which are important to the quality of the paper and critical to decide if accept the paper finally. In particular, these concerns are about the novelty, ablation studies, and demonstrations on clinical data. Therefore, I invite the authors to address the concerns from all reviewers in the rebuttal stage.
Author Feedback
Thank all reviewers for the helpful and positive comments, e.g., clearly written (R1&R2&R3); significant improvements (R1); unique idea, engaging (R2); reasonable, solid (R3).
Q1: Not that novel (Meta-R1&R1) A1: We want to kindly emphasize that our method contains specific and challenging designs: 1) For the SVMAR task, we specifically construct the physical-imaging-based reconstruction model with clear working mechanism; 2) We carefully encode the inherent rotation equivariance prior via advanced filter parametrization strategy, which consistently promotes superior performance; 3) Our rotation equivariant CNN is a general tool that can be used for more tasks, e.g., limited-angle and low-dose CT reconstruction, which is valuable.
Q2: Test on real data (Meta-R1&R1) A2: From Sec. 6, due to lack of publicly available real data, we follow [31] and only execute simulated test. We’ll try to collect real data for clinical analysis in the future. Thanks.
Q3: Some repetitive derivations with InDuDoNet (R2) A3: Our MEPNet is indeed inspired by InDuDoNet. However, MEPNet contains novel and challenging designs: 1) InDuDoNet is a special case of MEPNet. Besides metal artifact reduction, MEPNet also handles sparse-view reconstruction, which is more practical; 2) We embed more intrinsic rotation equivariance prior via advanced filter parametrization method, making MEPNet outperform InDuDoNet obviously; 3) The proposed rotation equivariant CNN is a general tool that can be applied to more tasks, e.g., limited-angle and low-dose CT reconstruction, which is valuable. In revision, we’ll do further clarifications. Thanks.
Q4: Ablation study about group equivariance (Meta-R1&R2) A4: From Footnote 2, InDuDoNet re-implemented in our paper for the SVMAR task is exactly an ablation study, which is the degenerated form of MEPNet with removing group equivariance. The results in Sec. 5.2 clearly show that using rotation equivariance makes MEPNet outperform InDuDoNet obviously in preserving rotation symmetry (see Fig. 4) and obtain better generalization ability with fewer parameters.
Q5: Shapes of input and output of proximal networks (R2) A5: From Sec. 4, the two proximal networks both consist of FCNs without size changes. For sinogram, input and output are both with size Nb×Np×C. For CT image, the size is H×W×C. Please see [21] and its code for details about the channel dimension C. We’ll explain it in the final version.
Q6: Why p8 group? Make it complete SO(2) or SE(2) with zero translation (R2) A6: Reason: The p8 group [23] is obtained by discretizing a continuous E(2) group and it preserves rotation symmetry well. This finely complies with the rotation equivariance prior underlying CT imaging. Considering performance and efficiency, we followed [23] and chose p8 group for discretized equivariance convolution on CT images. Extension: To achieve complete SO(2) or SE(2) with zero translation, [LieConv, Finzi et al., ICML2020] computes the intractable group convolution by discretizing the integral and sampling from local neighborhood of each group element. To reduce computation cost, only limited elements are sampled, which might harm the global rotation equivariance. Clearly, they also need to balance between performance and efficiency.
Q7: Decreasing PSNRs in Fig. 3. Resize image (R2) A7: Unlike Table 1 which is averagely computed on the same set of metals, the sizes of metals are different in Fig. 3, which also causes different artifact intensities, so there is no comparability among (a)(b)(c). We follow [21][31] and resize CT images to 416×416 where pixel spacing is used for normalization.
Q8: Show the framework. Application scenario (R3) A8: By unfolding Eq. (6), we can easily construct the workflow which will be added in the final version. Besides 2D, our method can also be used for 3D, where we need to carefully balance between network equivariance and computation efficiency. We’ll specifically explore this topic in the future. Thanks.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper received consistent recommendations for the acceptance in both initial and post-rebuttal phase. The authors provided a nice rebuttal to eliminate the reviewers’ concerns. I am happy to recommend to aceppt the paper for the publication of MICCAI23. The authors should also update their paper accordingly as promised in the rebuttal.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper is a solid submission with an interesting approach to solve a complex problem in the medical imaging domain. It uses existing deep learning techniques to propose a novel joint numerical method for sparse-view reconstruction and metal artifact reduction in CT imaging. The authors responded well to the concerns raised by the reviewers. They clarified their choices regarding methodology, emphasizing the novel designs that differentiate their model from existing ones, and pledged to include more detailed descriptions in the final version of the paper. Given the practical implications of the work, the depth of the experiments, and the authors’ comprehensive responses to the reviewers’ concerns, I recommend this paper for acceptance. The authors have clearly addressed the issues identified by the reviewers, providing strong justification for their methodological choices.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All reviewers recommend that this work should be accepted. The strongest criticism pre-rebuttal was lack of technical novelty (R1) lack of testing on real data (R1). The rebuttal responses are adequate, and there do not appear to be any public real data test sets. Since all reviewers are in agreement that this work should be accepted, I will not go against this agreed consensus, and recommend acceptance.