List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Weiwen Zhang, Dawei Yang, Carol Y. Cheung, Hao Chen
Abstract
Optical Coherence Tomography Angiography (OCTA) is a novel imaging modality that captures the retinal and choroidal microvasculature in a non-invasive way. So far, 3mm×3mm and 6mm×6mm scanning protocols have been the two most widely-used field-of-views. Nevertheless, since both are acquired with the same number of A-scans, resolution of 6mm×6mm image is inadequately sampled, compared with 3mm×3mm. Moreover, conventional supervised super-resolution methods for OCTA images are trained with pixel-wise registered data, while clinical data is mostly unpaired. This paper proposes an inverse-consistent generative adversarial network (GAN) for archiving 6mm×6mm OCTA images with super-resolution. Our method is designed to be trained with unpaired 3mm×3mm and 6mm×6mm OCTA image datasets. To further enhance the super-resolution performance, we introduce frequency transformations to refine high-frequency information while retaining low-frequency information. Compared with other state-of-the-art methods, our approach outperforms them on various performance metrics.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_62
SharedIt: https://rdcu.be/cVRsu
Link to the code repository
https://github.com/KevynUtopia/Frequency-Aware-Inverse-Consistent-OCTA-Super-Resolution
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
- This paper proposes an inverse-consistent deep learning based method to enhance the unpaired OCTA images.
- To enhance the OCTA image, Fast Fourier Transformation, Gaussian filters and Discrete Wavelet Transform are used to decompose frequency information.
- The proposed method outperforms the compared methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The novelty is clear by integrate spatial and frequency domain information in the GAN.
- This paper has several Architecture and illustration figures to enhance readability
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
More experiments need to add in section 3. (1) The OCTA image enhancement algorithms [4,5] should be compared. (2) The ablation experiments of total loss should be added based on formula (7). (2) Some public OCTA datasets are available and can be used in section 3, such as: [1] Ma Y , Hao H , Fu H , et al. ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model[J]. 2020. [2]Mingchao Li, Yerui Chen, Zexuan Ji, Keren Xie, Songtao Yuan, Qiang Chen, Shuo Li. Image projection network: 3D to 2D image segmentation in OCTA images. IEEE Transactions on Medical Imaging, 39(11): 3343-3354, 2020
-
The authors should give more details of the related works, especially the work [4,5].
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The network could be reproduced only based on the manuscript without major difficulties.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- In page 6, the authors set a parameter α = 0.7. However, I could not find this parameter in any formula of this paper.
- The evaluation metrics PSNR and SSIM need the ground truth of OCTA image. Please give the way of getting the ground truth.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty is clear but needs more experiments to support the novelty.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
This paper proposes inverse-consistent generative adversarial networks (GAN) for unpaired OCTA image restoration and degradation. In the proposed framework, a frequency domain decomposition module is introduced to enhance high-frequency information at the frequency domain level.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper proposes an unpaired OCTA image enhancement framework to enhance 6x6-mm image quality by learning high-frequency information from 3x3-mm images.
- The paper constructs super-resolution networks to enhance OCTA image quality from the perspective of frequency domain decomposition.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The image evaluation methods are not reasonable. SSIM and PSNR are used only for paired data, and there are obvious structural and domain differences in the low/high-resolution images presented in the experiments.
- As can be seen from the experimental results, the reconstructed image quality is not significantly improved compared with the original image.
- In the Introduction, the authors do not present the clinical significance and value of 6x6-mm OCTA enhancement.
- The authors used unpaired generative networks to enhance 6x6-mm OCTA images, which may lead to missing structural features or generating pseudo-vessels. The method lacks effective supervision of the vascular coherence loss function.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The network details of the paper are not described clearly enough, leading to difficulties in reproducing the method.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- This paper is essentially a super-resolution task, and unpaired super-resolution methods should be added to the experiment for comparison.
- The paper may consider proposing indirect image quality evaluation metrics. The paired metrics used in the current paper are not convincing.
- The resulting image of the non-rigid alignment is missing in the paper.
- The purpose of the paper is to enhance the whole 6x6-mm image, but the comparison result image of the whole 6x6-mm image is missing in the experiment.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The innovation of the paper and the rationality of the evaluation index
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
The author has responded to most of the questions raised. But I have one more concern. Although the authors mentioned that real pairwise data were collected to verify the performance, the images under the two fields of view may produce small structural changes due to the time interval. In this case, SSIM and PSNR would not be suitable as validation indicators. The authors ignored this situation.
Review #4
- Please describe the contribution of the paper
The authors proposed an inverse-consistent generative adversarial network with frequency awareness for unpaired image enhancement of low-resolution optical coherence tomography angiography. In this work, the authors regarded the enhancement task as the mapping from low resolution domain to high resolution domain. Furthermore, frequency domain information including low- and high-frequency components was integrated into the proposed network. Qualitive and quantitative results demonstrate that their approach outperforms other methods, especially in terms of the balance between low- and high-frequency information.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
It is interesting that this work decomposes and processes the data and thus optimize the network from the perspective of frequency domain.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The experiments seem a little inadequate and have not verified the clinical value of resolution enhancement on optical coherence tomography angiography.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper includes information that would make it possible to reproduce the methods and experiments.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- I think the architecture presented in Fig. 2 could hardly reveal inverse-consistency and may lead to readers’ misunderstanding.
- I get confused about the cross-entropy adopted in feature distribution loss and f() in Eq. 5. In addition, the value settings of β1 and β2 are also puzzled me.
- Some mini comments: (a) clinis in the second sentence of the introduction should be written as clinics. (b) In the sentence “Weights of each components are set as α = 0.7, …”, what does α represent? (c) UnpairedSR using pseudo pairs in the first paragraph of Section 3.3 should cite the reference [14].
- Compared methods seem somewhat limited and all of them are unpaired learning frameworks. I think the authors could supply some supervised learning methods due to have paired images.
- In ablation studies, does the method wo HFB indicates that authors replace HFB with mere high-frequency components? Please clarify it.
- Only adopting PSNR and SSIM for evaluation is limited. It is of great clinical interest if authors prove that their approach could benifit subsequent vessel analysis or diagnosis tasks.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The integration of frequency components into unpaired learning frameworks for resolution enhancement of optical coherence tomography angiography seem interesting. However, this work should be further improved in terms of method description and experiments.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
I still have a little confusion primarily about the elaboration of equations and symbols. I think that there could exist a symbol that is more suitable than f() to represent the features maps of high-frequency branches from two models. For feature distribution loss, Eq.5 seems to be different from the typical cross entropy loss that is usually written as -ylogp-(1-y)log(1-p). Furthermore, cross entropy rather than other distances between two distributions such as KL divergence is adopted as this loss, which also confuses me. For beta1 and beta2, I am unclear the impact of their different settings, e.g., how could the performance change if beta1=beta2. I still expected that the authors could make these clear in the final version.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes an inverse-consistent generative adversarial network for unpaired OCTA image restoration and degradation. It aims to enhance 6x6mm OCTA enface image quality by learning high-frequency information from 3x3mm images. Several major concerns have been raised, despite all the reviewers found the merit of this work. 1) R3 and R4 considered the evaluation approach is not reasonable or not sufficient enough, authors should give the explanation why SSIM and PSNR were used. 2) Both R3 and R4 raised that the clinical value of the proposed work is not well-motivated. 3) R2 provided some valuable suggestions, hope these would improve your paper quality and benefit your future submission.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
We appreciate the AC and reviewers for their valuable comments and merit affirmation of our paper including methodological novelty on frequency domain enhancement and competitive results outperforming other methods (R2 and R4). We will address the major concerns below: Q1. Rationality of evaluation and unpaired methods for comparison (R3): Thanks for comments and we apologize for the confusion. But this is a clear misunderstanding, and we would like to clarify it clearly here. For training, our inputs are indeed unpaired data, which is much easier to be collected in the clinical practice. To achieve objective and quantitative evaluation, we actually obtained paired data for testing evaluation, which takes more efforts for imaging and alignment. For details of obtaining ground truth, please refer to Q2. As R3 agreed, PSNR and SSIM are suitable for paired data evaluation. Thus, the evaluation in our study is reasonable. Since training setting is unpaired data, most comparison results are from unpaired methods in our study. To facilitate easy comparison and enable reproducibility, we will release our codes upon acceptance.
Q2. Ground truth acquisition (R2): Our testing data has high-resolution paired images as ground truth. During imaging, both low-(6x6-mm) and high-resolution (3x3-mm) images from the same eye are collected and have the same image size while the ratio of field-of-view is 4:1. One foveal center and four parafoveal sub-images are cropped from each low-resolution image, and we upscale them x2 to have the same image size as the high-resolution version (see Fig. 1). Then the upscaled low-resolution images are registered with high-resolution images using non-rigid registration, such that the low-resolution images are properly aligned and paired with its high-resolution version. We will refine Fig.1 to clearly illustrate it in the final version.
Q3. More clear results comparison (R3): Thanks for suggestion. The quality improvement can be seen in the clearer capillary and vessel texture while noises are also suppressed in our results. We will provide clearer results comparison using error maps in the final version.
Q4. Clinical significance and validation (R3, R4): Ophthalmic studies have shown high-resolution image can provide better diagnosis performance on diabetic retinopathy progression and visual acuity deterioration. However, high-resolution imaging is impeded due to the trade-off between field-of-view and image-resolution. By tackling this gap, it will enable ophthalmologists to evaluate the capillary loss in the fundus area and perform personalized interventional treatment to prevent visual loss. We will detail the clinical significance and discuss clinical validation in the final version.
Q5. Elaboration of equations and symbols (R2, R4): In the total loss, alpha is the fraction of discriminator of DWT components (see Fig. 3). f() in Eq.5 denotes the feature maps (shown in Fig.2). The beta1 and beta2 were determined by validation.
Q6. Experiments on other datasets (R2): Thank R2 for suggesting other two datasets. However, the first dataset is unpaired, and they did not provide paired data for testing, while the second one only contains low-resolution images. Thus, these datasets cannot be evaluated directly in our setting, and we will explore the first dataset for enriching training in the future.
Q7. Lack of vascular coherence supervision (R3) Our proposed method is more data-driven and did not incorporate prior of vascular structures. We will investigate vascular coherence supervision to achieve better results in the future.
Q8: Ablation studies (R2, R4) “wo HFB” denotes replacing high-frequency boosting with pure high-frequency. For missing ablation studies, we will provide them in the final version.
Q9. Other minor comments, e.g., non-rigid alignment, related work, figure, etc. Thanks for the suggestions and we will address them thoroughly in the final version.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes an inverse consistent generative adversarial network to perform unpaired OCTA image enhancement. The reviewers raised concerns about the clinical significance of the paper and the evaluation method. After the rebuttal, R3 still rejected the paper and stated that there were fundamental problems with the evaluation methods, which were not effectively explained in the feedback letter. AC considers that the issues raised by R3 are of some significance. Changes in small vessels can cause problems with the evaluation, which makes the experimental results unconvincing. Therefore, AC recommends rejecting this paper and hopes that the authors can take this issue seriously in the future and add clinically relevant proof.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I find the paper proposing an innovative solution to an important problem of enhancing the OCTA scans, which take long to acquire. After reading the rebuttal, I find the experimental setup and the PSNR/SSIM metrics used convincing, and are hence addressing the main original remarks raised by R3. Considering the time interval between two acquisitions to be short, I would not expect structural changes. The confusion in the equations mentioned by R4 should be carefully addressed in the final version.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors proposed an inverse-consistent deep-learning-based method to enhance the unpaired OCTA images. The reviewers agreed that most concerns had been addressed after the rebuttal, lifting the paper over the acceptance threshold. However, the final version of the paper should make some details clear as mentioned by the reviewers, e.g., explain the effect of small structural changes due to the time interval on the validation, clarify the symbols, and Eq.5, discuss the performance when beta1=beta2, etc.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
11