List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Haofeng Liu, Heng Li, Huazhu Fu, Ruoxiu Xiao, Yunshu Gao, Yan Hu, Jiang Liu
Abstract
As an economical and efficient fundus imaging modality, retinal fundus images have been widely adopted in clinical fundus examination. Unfortunately, fundus images often suffer from quality degradation caused by imaging interferences, leading to misdiagnosis. Despite impressive enhancement performances that state-of-the-art methods have achieved, challenges remain in clinical scenarios. For boosting the clinical deployment of fundus image enhancement, this paper proposes the pyramid constraint to develop a degradation-invariant enhancement network (PCE-Net), which mitigates the demand for clinical data and stably enhances unknown data. Firstly, high-quality images are randomly degraded to form sequences of low-quality ones sharing the same content (SeqLCs). Then individual low-quality images are decomposed to Laplacian pyramid features (LPF) as the multi-level input for the enhancement. Subsequently, a feature pyramid constraint (FPC) for the sequence is introduced to enforce the PCE-Net to learn a degradation-invariant model. Extensive experiments have been conducted under the evaluation metrics of enhancement and segmentation. The effectiveness of the PCE-Net was demonstrated in comparison with state-of-the-art methods and the ablation study. The source code of this study is publicly available at https://github.com/HeverLaw/PCENet-Image-Enhancement.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_49
SharedIt: https://rdcu.be/cVRsh
Link to the code repository
https://github.com/HeverLaw/PCENet-Image-Enhancement
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposed a network architecture and loss functions to enhance the fundus images. It contains three modules, one for augmentation, one for Laplacian pyramid extraction and one for autoencoding.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the paper written and explained well.
- the contributions are clear.
- the experimental results contain well methods, data and metrics
- the method sounds reasonable
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- the main issue if the paper is in the results section and comparison to other methods. Since the proposed method has a augmentation module (SeqLC), it is not fair the competitor methods did not have any data augmentation step. Or similarly, we need to have the performance of the proposed method without the SeqLC. Therefore the contribution of the paper is not well examined.
- make table 1 easier to read
- it seems that figure 1 missed the cnn layers. Perhaps the orange arrows need to be denoted as conv/downsampling. same for gray arrows.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors intend to share the source codes. one part of data is publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
see section 5
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
comparison to state of the art methods is not fair.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The authors proposed PCE-Net for the enhancement of fundus images. Moreover, a Laplacian pyramid is introduced to exploit the retinal structure in PCE-Net.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The organization of this paper is good and clearly describes the pipeline of the task. The structure of PCE-Net is technically sound. As shown in table 2, the proposed method outperforms the existing method by a margin in terms of restoration, segmentation, and diagnosis.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The novelty is limited, for example, the feature pyramid structure is a common strategy [1] and the constraint is contrastive learning conducted on features of each level. Hence, the FPC may not be qualified a contribution to this work. The authors claim that the straight constant on feature maps is too inflexible. However, they do not demonstrate the convergence of FPC.
[1] Lin, Tsung-Yi, et al. “Feature pyramid networks for object detection.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility is well sound.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The FPC shows superior performance as shown in table 2. However, the discussion on it is not sufficient. More experiments on it would make FPC more convincing.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well written and easy to follow. The proposed PCE-Net and the Feature pyramid constraint are technically sound. Some drawbacks exist such as the inadequate discussion on FPC. considering all things, I recommend the acceptance of this paper.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This manuscript proposes a pyramid constraint to develop a novel model called PCE-Net for fundus image enhancement. The work is of interest. The experimental results verifies the effectiveness of the model. I think it would be a good enhancement approach for fundus image analysis.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main idea of the work is to form SeqLCs (image sequences) and LPF (Laplacian pyramid features) from degraded images. The PCE-Net is to learn a degradation invariant model. The manuscript proposed a novel loss function to train the PCE-Net.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The experimental results focus on the vessel segmentation after enhancement. How about the results on classification tasks after enhancement.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The description of the method is clear. I think the proposed method could be reproduced.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This manuscript proposes a pyramid constraint to develop a novel model called PCE-Net for fundus image enhancement. The main idea of the work is to form SeqLCs and LPF from degraded images. The PCE-Net is to learn a degradation invariant model. The experimental results verifies the effectiveness of the model. Suggestions and questions are as follows:
- In ablation study, the authors didn’t consider the effectiveness of the enhanced images for classification tasks.
- How about the segmentation results on SE, EX, MA in enhanced fundus images?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The idea combines the neural network models and image analysis methods. It is pretty intesting.
- The organization and writing of the manuscript is satisfactory.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
In this paper, a fundus image enhancement network was proposed to mitigate the impact of quality degradation caused by complex imaging interference. To collect plenty of training data, SeqLC is randomly degraded from identical high-quality images. Then the low-quality images are decomposed to Laplacian pyramid features (LPF) as the multi-level input for the PCE-Net. To constrain the consistency of embeddings, the FPC was introduced to learn the representations invariant to degradations. Extensive experiments have been conducted under the evaluation metrics of enhancement( full reference and non-reference) and segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is very well written and the problem concerned in this paper is of great significance for clinical fundus diagnosis. Experiments are abundant and reasonable evaluation metrics are introduced to verify the effectiveness of the network.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Insufficient references, a broader survey of related work is expected.
- Medical image enhancement based on fundus is relatively few, but work based on photographs enhancement is considerable and optimized, both of which are indeed homologous tasks. The author is expected to conduct extensive research on the related work. If possible, mainstream photographs enhancement methods with superior performance should be trained to perform lateral comparisons.
- FIQ was used in the experiment but there is little information about this private fundus dataset, which makes the experiment carried out on FIQ less convincing and explainable. A brief introduction to this dataset is expected.
- Segmentation effect is one of the three evaluations in the Experiments part. However, the author failed to specify the method used to obtain the segmentation result, a traditional one or a deep learning network? More details should be given.
- In part 3, comparison and ablation study results are shown in Table 2. However, there is no numerical analysis of the data in the table but only some qualitative analyses. To give a clearer and explicit description of the experimental results and the superiority of the proposed model, precise numerical analyses should be given.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The source code of the paper will be publicly available. The reproducibility of the paper is recognized.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
For the formulas, detailed description of the symbols(for example ‘E’ in formula(1) and formula(3)) is required.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well written and the work is solid.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposed a network architecture with SeqLC (image sequences) and LPF (Laplacian pyramid features) to enhance the fundus images. While given three positive and one negative reviews, there are many key concerns mainly focusing on 1. the fairness of the comparison experiments with/without SeqLC augmentation module. 2. limited experiments only conducted on vessel segmentation, why not on classification or other segmentation tasks. 3. limited novelty. the SeqLC and LPC are common strategy in exisiting methods. The authors should address the concerns in rebuttal.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
We sincerely thank the reviewers for their high-quality reviews and constructive comments. In the following responses, we classified the issues and concerns of the reviewers and provided responses, which will be integrated into our final version.
[Q1] Fairness of the comparison experiments (R#1). [A] Many thanks for the comment. We apologize for not specifically describing the details of the comparison experiments, leading the reviewers to misunderstand the experimental fairness. In fact, the comparison methods used the same augmentation data as the proposed method. The only difference is that PCE-Net loads the data in sequences via SeqLC to learn a degradation-invariant enhancement model. Therefore, we think the comparison experiments should be concerned as fair.
[Q2] Application to other segmentation tasks or classification (R#2). [A] We appreciate the reviewers’ recommendation. We further applied PCE-Net to the optic disc and cup segmentation and disease diagnosis, and the added experiments are described as follows. 1) In the optic disc and cup segmentation experiment, we trained the segmentation model with the REFUGE dataset [MedIA 2020]. Then we applied the model to the images enhanced by PCE-Net, and the segmentation results of high-quality images were used as the reference. Results show that PCE-Net increases the IoU of low-quality images from 0.544 to 0.79. 2) In the diagnosis experiment, the ODIR-2019 dataset was adopted. We trained a ResNet-50 classification model and selected Kappa as the evaluation metric. Results show that the Kappa in the original test set is 0.598, degraded one is 0.469, and PCE-Net enhanced one is 0.494.
[Q3] Ambiguous novelty in LPF, and similarity to FPN [CVPR 2017] and contrastive learning (R#3). [A] Thanks for the comment. PCE-Net applies LPF to decompose images into multi-level features for constraining retinal structures and uses FPC to learn the degradation-invariant representations based on the content consistency in SeqLC. 1) LPF boosts the structure preservation in PCE-Net but is not the only contribution of PCE-Net. Furthermore, the motivation of LPF is different from FPN, which was proposed for object detection. 2) FPC adopts the consistency constraint in SeqLC to boost the learning of a degradation-invariant model, rather than positive and negative samples, which are the foundation of contrastive learning. Certainly, contrastive learning is also a potential paradigm for our future study.
[Q4] FPC with straightly constraining on the feature maps is too inflexible to converge (R#3). [A] We tried to constrain the numerical consistency straightly on feature maps without pooling, but it could not converge.
[Q5] Writing details in Table 1, Fig. 1 and formula (1), and more information on the mainstream enhancement methods (R#1, R#4). [A] The modified table, figure, and related method description will be integrated into our final manuscript.
[Q6] More description in FIQ dataset and segmentation experiments (R#4). [A] 1) The FIQ dataset was collected from healthy people with a mean age of 41.5 years and a variance of 6.36. The male to female ratio was 5:6. 2) The segmentation model was a U-Net trained on the DRIVE dataset and was applied to the enhanced images.
[Q7] Numerical analyses for PCE-Net (R#4). [A] We conducted a T-test between PCE-Net and comparison methods on the IoU of vessel segmentation (p<0.001 for all experiments), and the max p-value is between CofeNet (p=0.00086).
[Q8] Results of lesion segmentation (R#2). [A] The full-reference dataset we used in this study was collected from healthy people and lacks lesions, so it cannot be used for lesion segmentation. Alternatively, lesion preservation was validated by the added diagnosis experiment on a non-reference dataset in Q2. And we also checked the lesions in enhanced images, which are more clear after the enhancement.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a network architecture with SeqLC (image sequences) and LPF (Laplacian pyramid features) to enhance the fundus images. After the rebuttal, the key concers are clearified, especially the fair comprison of experiments.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I believe the rebuttal addressed most of the concerns. The authors should update their final camera-ready version based on the important information provided in their rebuttal to avoid misunderstandings.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addresses the concern of fair comparison in the experiments. The paper can be accepted after revision.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5