List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ellen Jieun Oh, Yechan Hwang, Yubin Han, Taegeun Choi, Geunyoung Lee, Won Hwa Kim
Abstract
Retina images are non-invasive and highly effective in the diagnosis of various diseases such as cardiovascular and ophthalmological diseases. Accurate diagnosis depends on the quality of the retina images, however, obtaining high-quality images can be challenging due to various factors, such as noise, artifacts, and eye movement. Methods for enhancing retina images are therefore in high demand for clinical purposes, yet the problem remains challenging as there is a natural trade-off between preserving anatomical details (e.g., vessels) and increasing overall image quality other than the content in it. Moreover, training an enhancement model often requires paired images that map low-quality images to high-quality images, which may not be available in practice. In this regime, we propose a novel Retina image Enhancement framework using Scattering Transform (REST). REST uses unpaired retina image sets and does not require prior knowledge of the degraded factors. The generator in REST enhances retina images by utilizing the Anatomy Preserving Branch (APB) and the Tone Transferring Branch (TTB) with different roles. Our model successfully enhances low-quality retina images demonstrating commendable results on two independent datasets.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_45
SharedIt: https://rdcu.be/dnwwY
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a Retina image Enhancement framework using Scattering Transform (REST) to enhance low-quality retina images using unpaired retina image sets. It develops two core designs: the Anatomy Preserving Branch (APB) and the Tone Transferring Branch (TTB). The experiments demonstrate that the REST can adequately enhance retina images by restoring dark and uncertain regions without compromising anatomical structures on UK Biobank and EyeQ datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
S1. The illustrations in this paper make the concepts easier to grasp.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
W1. The novelty of this paper is quite limited. The overall quality of this paper is more like a technical report.
W2. Why use the scattering transform in the Anatomy Preserving Branch (APB) can effectively capture anatomic structures?
W3. The technical details need to be clear. For instance, In the Anatomy Preserving Branch (APB), the authors utilized interpolation with a factor of 2 and convolution with the kernel instead of transpose convolution. Why not use Sub-pixel convolution? And, in the Tone Transferring Branch (TTB), why the authors used transpose convolution?
W4. The Tone Transferring Branch (TTB) is designed to ensure that the synthetic images resemble the tone of high-quality images. Why not use the HSV image as the input of TTB?
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reasonable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
The authors needs to provide reasons why use kernel and transpose convolution in the APB and TTB, separately.
-
The overall quality (writing, presentation and results) needs to be improved.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty and main contribution of this paper are absolutely not enough.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The rebuttal addressed my concerns well. Considering other reviews, I tent to change my score to Weak Accept.
Review #2
- Please describe the contribution of the paper
This paper proposed an image enhancement framework for retinal fundus images based on unpaired image-to-image translation network CycleGAN and Wavelet scattering transform, which liberates from manual simulation and captures the anatomical structures. The proposed method was evaluated on two public datasets with a retinal quality assessment model and visualization.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The paper was written and explained decently, and thus it is easy to follow. (2) This paper adopted Wavelet scattering transform, containing diverse angles and frequencies, to modify CycleGAN in order to promote the enhancement of structural information.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) The explanation in figures, formulas, and tables might confuse the reader. It could be noticed that: A) the symbol of concatenation ‘||’ might lead to misunderstanding, although it was explained before written; B) the legends in Fig. 1 are confusing on account of lacking details; C) the arrangement in Table 1 is unclear leading to puzzle, e.g. the signal of # is confusing and cGAN and CutGAN are introduced in the table without explanation, and thus it is difficult for readers to understand the significance. (2) The paper provides limited contributions to this field because it applies CycleGAN with the combination of Wavelet scattering transform, but the transform is common-used in image processing. (3) The experiment lacks comprehensive evaluation: A) it adopts UKB and EyeQ for evaluation, but with only the metric of FIQA; B) although it could be noticed that from the visualization result, the proposed method achieved the best structure-preserving performance, the quantity comparison in structure-preservation should be provided to further demonstrate the effectiveness.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It could be reproduced if the source code is provided.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
(1) The figures, formulas, and tables should be rearranged to further improve the readability of the articles. (2) Extra verification experiments should be conducted to evaluate the effectiveness of the proposed algorithm. And some extra frequently-used metrics should be introduced to prove the effectiveness of the algorithms. (3) The reason why CutGAN and cGAN are adopted in the comparison experiments should be demonstrated, although the other methods are fully introduced. (4) More details are acquired to illustrate the fairness of the experiments. (5) The novelty of the paper could be further improved by a more dialectical and comprehensive explanation of the anatomy-preserving branch which is the most prominent novelty of this article.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the article is decently arranged and relatively comprehensible, the limited novelty and insufficient experiments are the major causes for the overall score.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
The author’s response did not meet my expectations.
Review #3
- Please describe the contribution of the paper
This paper proposes an unpaired Retina image Enhancement with Scattering Transform (REST) which preserves anatomical structure and maps the tone through the Anatomy Preserving Branch (APB) and Tone Transferring Branch (TTB), respectively. With qualitative and quantitative evaluation of two independent datasets, it is shown that REST can strike a balance between preserving anatomical details and improving overall image quality.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper effectively preserves anatomical structures during enhancement. The authors conduct a thorough literature study and demonstrated the effectiveness of the proposed REST through extensive experiments on two different datasets with encouraging results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The C, SC, e and d in Fig. 1 should be given a more detailed description, do these correspond to the ones in the method?
- The descriptions in methods section are a bit confusing, and it would be better to give relevant definitions for each abbreviation.
- The two branches may be inadequately described in words only. Some visualizations can be added to facilitate the readers’ understanding.
- The real high-quality images should be added to the comparison experiments in Fig. 3 and other figures in Supplementary file.
- Reference 6 misses page numbers.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Upon release of the code, the reproducibility can be verified.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Comments mentioned clearly in the strength and the weakness section of the papers.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The relevant weakness and comments are mentioned before.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The authors have addressed my concerns well.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents an approach for image enhancement in retinal images. Overall, the paper is easy to be followed and the results show some improvement. Strength: 1) The paper is well organised and easy to follow. 2) The results show some improvement compared with prior art. Weakness: 1) The experimental validation seems to be limited and not comprehensive. 2) The presentation of the papers shall be improved, especially in details of the method and experiments.
Author Feedback
We will address all concerns and hope the reviewers reconsider some of the ratings.
Q[R1,R2] Contribution and description of APB The wavelet scattering transform in APB effectively captures high-frequency information by employing diverse wavelets. Thus, APB is able to capture anatomical structures such as vessels, optic discs, and cups which exhibit high-frequency characteristics. While the use of scattering transform has been limited to image processing and classification, REST introduces a novel approach that leverages the scattering operation within a generative model.
Q[R1] Different up-samplings for APB and TTB As APB captures high-frequency features, it is affected by the checkerboard artifact caused by transpose convolution. Hence, we employed a combination of interpolation and convolution for APB. However, TTB deals with contrast and illumination that are robust to the artifact, and transpose convolution worked well. We appreciate suggestions such as sub-pixel convolution which may benefit our method.
Q[R1] HSV as input to TTB We agree that HSV will help deal with the tone and will consider using it in future work.
Q[R2,R3] Using only FIQA for quantitative evaluation / Absence of high-quality images in Fig.3 We additionally computed WFQA (weighted FIQA), introduced in [2], which provides a measure of performance where a higher WFQA corresponds to better results. The WFQA scores for UKB data were 1.85±0.07 (CycleGAN w/ ResNet), 1.76±0.09 (CycleGAN w/ Unet), 1.76±0.09 (ISECRET), 0.78±0.12 (PCENet), and 1.88±0.03 (REST) where REST was the best. Additionally, we apologize for an error in the FIQA result of PCENet in the UKB experiment, which should be 0.39±0.06. As we deal with ‘unpaired’ images, the dataset does not provide genuine high-quality paired images corresponding to each low-quality image for Fig.3. In this context, PSNR and SSIM, which require paired images, are inapplicable. Instead, we adopted FIQA for evaluation as in other studies [1,2]. Notice that PSNR/SSIM reported in [1,2] are based on synthetically degraded images used in training, however, we do not have any of such augmentation components and PSNR/SSIM cannot be used.
Q[R2] Why use cGAN and CutGAN in Tab.1 as Baselines? In the EyeQ experiment, we adopted the setup and results reported in [1] which include cGAN and CutGAN. However, for the UKB dataset, we ran all the experiments with cross-validation and deliberately chose not to use cGAN since our model does not rely on ‘paired images’. Instead, we utilized the latest models designed for unpaired retina image enhancement [1,2]. In addition, we included two cycleGAN variants with different backbones as traditional approaches.
Q[R2] The quantitative comparison in structure-preservation The FIQA score quantitatively assesses structure preservation, including key structures such as the optic disc and macula, as well as uniformity in illuminance and contrast. The high FIQA score for REST indicates excellent preservation of fine structure and even distribution of illumination and contrast.
Q[R2] Fairness of the experiments For UKB experiments, we employed cross-validation to acquire unbiased results, which is the most common practice in Machine Learning. We also performed sufficient tuning for each baseline. For the EyeQ results, we adopted the metrics reported in [1], ensuring fairness in all experiments.
Q[R3] Abbreviations in the method section and legends in Fig.1 The symbols ‘E’, ‘e’, ‘D’, ‘d’, ‘C’, and ‘SC’ represent the encoding process, encoding output, decoding process, decoding output, convolution output, and scattering transform output, respectively. These symbols correspond to the legends in Fig.1 and will be clarified in the paper.
Q[R2] Symbol || in Eq and # in Tab.1 We used || for concatenation as a common practice in computer science. # represents ‘the number,’ and we will clarify it in the revision.
[1] Cheng et al., MICCAI 2021 (ISECRET) [2] Liu et al., MICCAI 2022 (PCENet)
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After the rebuttal, the paper still has some weakness and some strength, which makes it a borderline paper. Weakness: the overall presentation of the paper can be improved. The rank of this paper in the three reviewers’ stacks are not high. Strength: the authors have proposed some new blocks. Although there are some concerns on the design of the blocks, the rebuttal seem to address the concern and Reviewer 1 raised the score. Considering these factors, I would recommend to accept this paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have partially addressed the reviewers’ concerns. It is suggestion to provide results of statistical significance tests. The paper is generally well written with the methods being easy to understand and follow. It reaches the minimum requirement for publication.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a framework for unpaired retinal image enhancement based on the scattering transform. The reviewers provide positive acknowledgement of the organization and writing of the paper, but raise concerns about the methods contribution and experimental evaluation. The authors have clarified these issues in their rebuttal. However, the authors’ rebuttal on the experimental evaluation did not convince me. I recommend rejecting this paper.