List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Hong Wang, Qi Xie, Yuexiang Li, Yawen Huang, Deyu Meng, Yefeng Zheng
Abstract
During X-ray computed tomography (CT) scanning, metallic implants carrying with patients often lead to adverse artifacts in the captured CT images and then impair the clinical treatment. Against this metal artifact reduction (MAR) task, the existing deep-learning-based methods have gained promising reconstruction performance. Nevertheless, there is still some room for further improvement of MAR performance and generalization ability, since some important prior knowledge underlying this specific task has not been fully exploited. Hereby, in this paper, we carefully analyze the characteristics of metal artifacts and propose an orientation-shared convolution representation strategy to adapt the physical prior structures of artifacts, \emph{i.e.}, rotationally symmetrical streaking patterns. The proposed method rationally adopts Fourier-series-expansion-based filter parametrization in artifact modeling, which can better separate artifacts from anatomical tissues and boost the model generalizability. Comprehensive experiments executed on synthesized and clinical datasets show the superiority of our method in detail preservation beyond the current representative MAR methods. Code will be available at \url{https://github.com/hongwang01/OSCNet}.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_63
SharedIt: https://rdcu.be/cVRT7
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a method for a deep-learning-based metal artifact reduction algorithm using the orientation-shared convolution representation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The methodology has a solid theoretical background and shows promising results in both synthetic and real image data. The unnecessity of the sinogram is preferable from a practical point of view, where sinograms are not available. The proposed image-domain algorithm can also reduce the secondary artifacts due to the error in the geometric calibration (i.e., slight inconsistency between the physical imaging geometry and the geometry calculated by the calibration.)
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The experimental materials involve limited anatomy. I am very interested to see various other anatomies, including total hip arthroplasty.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors will release the code and the trained model after acceptance, which is very helpful for the community.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This is excellent work. It would be even better if the authors also discussed the limitation of the proposed method more in detail.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Nice work.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
I do not change my assessment.
Review #2
- Please describe the contribution of the paper
Improve the DICDNet by integrating rotationally symmetrical streaking (RSS) property and filter parameterization. Contribution is incremental.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novel formulation for the existing DICDNet by integrating the rotationally symmetrical streaking property and filter parameterization. The method is also demonstrated on clinical data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Compared with NMAR and DuDoNet, large streaks between metals are not reduced. The backbone of the method is based on the existing DICDNet method.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This paper addresses the metal artifact reduction problem in CT reconstruction. The main method is based on a recent method called DICDNet. In this work, based on the prior fact that metal streak artifacts are rotational because of the back-projection algorithm along different angles, the authors improve the existing DICDNet by integrating an orientation-shared convolution representation into their network design, which can reduce rotationally symmetric metal artifacts better. The overall structure of the manuscript is clear and the method has been compared with state-of-the-art methods. Not only qualitative results are displayed, but also quantitative evaluations are performed. The clinical feasibility is also demonstrated by the experiments on clinical data. These are all the good points from the paper. However, in Fig. 5, we can see that compared with NMAR and DuDoNet, large streaks between metals are not reduced, which is a major disadvantage of the method. The authors should find a way to overcome this limitation.
Since the backbone method, i.e., the DICDNet, has already been published. This work is an improved/modified version. Hence, the contribution is incremental.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novel formulation using rotational symmetry property.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Correct one mistake from my previous review: this paper ranks 2nd instead of 4th among my 5 reviewed papers.
The major drawback of DICDNet is the remaining large streaks, while it is successful in reducing other tiny streaks caused by metals, geometry inconsistency, or beam hardening. The proposed OSCNet is based on DICDNet by introducing the orientation-shared convolutions. It has a nice mathematic formulation and the quantitative evaluation shows its superiority to DICDNet. However, visually large streaks still remain in the OSCNet results. The model complexities of DICDNet and OSCNet are also similar, both around 1,275,000 parameters. Therefore, I still think that the proposed OSCNet has no essential improvement compared with DICDNet, while I think that using such orientation-shared convolution is interesting.
Review #3
- Please describe the contribution of the paper
This work introduces a new image-based neural network architecture for metal artifact reduction in X-ray CT. The architecture is built around an additive artifact reduction model which aims to predict an artifact layer which is to be subtracted from the input image. The artifact layer is estimated by using a formulation of steerable convolutional filters to represent rotationally symmetric streaking artifacts. Those filters are learned together with networks interpreted in this work as proximal operators which together predict the decomposition in artifact layer and artifact-free image. The method is evaluated on simulated data against multiple state-of-the-art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Evaluation against multiple competitive modern methods.
Novel and innovative, independent ideas.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The prior knowledge exploited does not hold true for all image artifacts caused by metal. E.g. in Fig. 5 the black area inside the mouth cavity can clearly be seen still contain the black band between the metal implants in the presented method, while other methods can remove it. In general the visual impression of the results does not match very well with the quantitative evaluation. CNNMAR leaves a very convincing visual impression compared to the proposed method.
Only simulated data for evaluation.
The method section is extremely hard to follow for me.
No ablation studies. This work introduces steerable convolutional filters and a network architecture inspired by an unrolled optimization scheme. Without an ablation study it is unclear what parts of this method are essential.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper will not be reproducible without code. The description lacks to many details. However, the authors promise to release their code which would render this point moot. If the code is published the method should be well reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
“Furthermore, it is difficult to collect the sinogram data in realistic applications [9].” This sentence is factually incorrect. In order to reconstruct a CT image this raw data is necessarily acquired. I believe this sentence is supposed to express the practical difficulty of acquiring this data if one does not collaborate with CT manufacturers or builds an own CT. Please reformulate this sentence if this is the intention. This is especially important because classically approaches in projection-domain where favored since they allow dealing with the rotational streaking artifacts easier, before they occur. In addition for deep-learning approaches different prior works have shown that correcting metal artifacts in projection domain is advantageous e.g. in https://ieeexplore.ieee.org/document/8331163.
I don’t understand equation 5. Why are all the terms multiplied element-wise by “I”? Doesn’t that make the term superfluous? From equation 9 I get the impression that “I” serves to direct the loss at image areas which are metal artifact affected. If that is the intention “I” could just be introduced in equation 9 and described like this.
I also don’t understand equation 9. Isn’t the objective minimized if X + C * M = Y ? That would just mean that any additive artifact model automatically minimizes this model. So simply setting up a global residual connection in a network where Y - A = X, where A is predicted by a network (A = f(Y)) would automatically minimize this. What am I missing here?
I am generally unconvinced about the necessity to use the presented prior knowledge. The rotational artifact structure stems from the fact that it is caused by inconsistency in projection domain, between different projections which causes such artifacts on circular trajectories. Therefore, projection-domain (sinogram-based) methods implicitly also use this prior knowledge. Even better, if more complicated trajectories are used, which can cause different artifact patterns, the methods in projection-domain would still exploit this appropriately.
The evaluation should be improved by providing results on measured data. The method description should be simplified, e.g. by providing a sketch of how it works. In addition an ablation study should be done to show that all aspects of the method are relevant.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is extremely hard to follow and the proposed prior knowledge is unconvincing. The mathematical justification of the network architecture seems superfluous. Once those motivations are stripped, the paper essentially proposes a novel network architecture. The evaluation is only done on simulated data and the results are visually inferior to other methods evaluated. Therefore in summary I am unconvinced this method improves over the state of the art.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
The rebuttal mainly tried to argue with my review instead of clarifying or changing points. I therefore stand by my recommendation that the method will not be useful for a broader community and therefore not be of greater interest.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes a deep-learning method for metal artifact reduction from CT data. The work is of interest to the MICCAI community as several groups have tried to address this challenging task. The code will be released if the work is accepted which is encouraging. The reviewers found the proposed method interesting and novel. However, several weaknesses were also noted. I invite the authors for rebuttal to address the concerns raised by the reviewers which are summarized below.
- Reviewer 2 mentions that compared with NMAR and DuDoNet, large streaks between metals are not reduced. The authors should explain why the NMAR and DuDoNet are performing better for this example.
- Qualitative and quantitative results between the DICDNet and the proposed method are very similar. The authors should justify and explain in which clinical scenarios the proposed method will have an advantage to use over DICDNet.
- Improved explanation of the equations and mathematical justification of the network architecture (Rev3)
- A better justification of why rotational artifact structure is necessary is needed (Rev 3)
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Author Feedback
Thank all reviewers for the constructive comments.
Q1: More anatomies (R1) A1: We’ll try to collect more metal-affected anatomies. Thanks.
Q2: Leaving large streaks between metals in Fig.5 (R2) A2: 1) Although leaving a black band, our method still outperforms others in reducing the artifacts around the dental region of interest which is meaningful for clinical scene; 2) From Fig. 4 in SM, our OSCNet removes more black bands and streaks, superior to other methods; 3) Even in this case, our method still obtains the best PSNR for fine detail fidelity. This should benefit from the rational constraint of our artifact model. To further reduce artifacts, we’ll add several ResNets to refine the image restored at the last iterative stage.
Q3: Contribution is incremental (R2) A3: We want to kindly clarify that OSCNet contains novel and challenging design. Specifically, 1) Eq. (6) should be the first orientation shared convolution coding model, which is hard to achieve without the filter parametrization strategy we adopt; 2) The OSC model is very suitable for encoding the rotational prior of artifacts ignored by DICDNet; 3) DICDNet is a special case of OSCNet (L=1, K=32). Such rotational design makes OSCNet outperform DICDNet obviously; 4) The OSC model is a general tool for modelling any rotational structure, which is valuable.
Q4: Comparison on clinical data (R3) A4: Fig. 6 shows good feasibility of our method on clinical data (agreed by R1 and R2).
Q5: Why multiplying I in Eq. (5)? Isn’t the objective minimized if X + C * M = Y in Eq. (9)? (R3) A5: (1) “I” denotes the non-metal region not the artifact-affected area, and “A” only represents the artifact in this region. In the metal region (i.e., I_ij=0), metals have higher CT values than tissues [16], leading to that artifacts may not comply with the model in Eq. (6) and Y_ij-X_ij≠A_ij. Thus, it is necessary to multiply I in Eq.(5). (2) Only X + C * M = Y doesn’t minimize the objective in Eq.(9). The regularizers f_1(M) and f_2(X) are also the minimization objectives, which connect network and the OSC model. Besides, the correct equation is I \odot (X+C*M)=I \odot Y.
Q6: Why not simply using a global residual connection where Y - A = X and A = f(Y)?(R3) A6: The reasons are four-folds: 1) Our OSC model can finely constrain the extracted artifact and better distinguish artifact from details (refer to Fig. 2 of SM). Yet, a general network with high flexibility works as a black box and the prediction f(Y) may confuse X and A; 2) Directly predicting A is very hard and needs more network parameters, thus reducing the generalizability; 3) If simply learning A by a network, it’s hard to adopt the powerful deep unfolding tool to build an interpretable structure. Instead, OSCNet is built based on the mathematical derivation and has clear iterative process as shown in Fig. 2 of SM; 4) With ResNet only, the PSNR on synthetic data is 37.29dB, evidently inferior to our method as 43.30dB.
Q7: Necessity and rationality of using rotational prior (R3) A7: Necessity: 1) For such ill-posed task, prior can shrink the solution space and help identify artifacts; 2) OSCNet evidently outperforms DICDNet, showing the effectiveness of rotational prior; 3) Many model-driven works [16,17] have validated the merits of embedding prior into networks.
Rationality: 1) Such rotational prior is intrinsic due to the back-projection process for CT reconstruction (agreed by R2); 2) Since projection-domain method may bring secondary artifacts and it’s hard to collect sinogram data without close collaboration with the scanner vendor (agreed by R1), it is worth focusing on prior design in image domain.
Q8: Without ablation study, it’s unclear what parts of the method are essential(R3) A8: DICDNet is exactly our backbone and represents ablation study, which shows that the essential embedding of rotational prior helps OSCNet outperform DICDNet evidently. Please refer to 3) and 4) in A6 for more details.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I believe the authors have provided a strong rebuttal and answered most of the critical questions. The authors should try their best to include as much as information provided in the rebuttal in their final camera ready paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a deep network based on orientation shared convolution for removing metal artifacts from CT images. The reviewers expressed concerns about the methodological and experimental design aspects of the paper. After rebuttal, two of the reviewers accepted the paper, while R3 insisted on rejecting the paper. AC thinks the methodology is interesting and the paper provides a relative proof of principle. Overall, the feedback letter has addressed the main issues raised and the strengths of the paper outweigh the weaknesses. AC recommends acceptance of the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors proposed a deep-learning-based algorithm using the orientation-shared convolution representation for metal artifact reduction in X-ray CT. However, the reviewers have consistent concerns about its improvement compared with state-of-the-arts. Moreover, as one reviewer pointed out, the paper was hard to follow, and the explanations were insufficient. For example, equation 9 is a foundation for the proposed method while it is confusing. These concerns are not well addressed in the rebuttal. But, with two reviewers’ support, I would let it pass and wish the authors can address all the issues in the final manuscript.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
12