List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Zipei Yan, Dong Liang, Linchuan Xu, Jiahang Li, Zhengji Liu, Shuai Wang, Jiannong Cao, Chea-su Kee
Abstract
High myopia (HM) is a leading cause of irreversible vision loss due to its association with various ocular complications including myopic maculopathy (MM). Visual field (VF) sensitivity systematically quantifies visual function, thereby revealing vision loss, and is integral to the evaluation of HM-related complications. However, measuring VF is subjective and time-consuming as it highly relies on patient compliance. Conversely, fundus photographs provide an objective measurement of retinal morphology, which reflects visual function. Therefore, utilizing machine learning models to estimate VF from fundus photographs becomes a feasible alternative. Yet, estimating VF with regression models using fundus photographs fails to predict local vision loss, producing stationary nonsense predictions. To tackle this challenge, we propose a novel method for VF estimation that incorporates VF properties and is additionally regularized by an auxiliary task. Specifically, we first formulate VF estimation as an ordinal classification problem, where each VF point is interpreted as an ordinal variable rather than a continuous one, given that any VF point is a discrete integer with a relative ordering. Besides, we introduce an auxiliary task for MM severity classification to assist the generalization of VF estimation, as MM is strongly associated with vision loss in HM. Our method outperforms conventional regression by 16.61% in MAE metric on a real-world dataset. Moreover, our method is the first work for VF estimation using fundus photographs in HM, allowing for more convenient and accurate detection of vision loss in HM, which could be useful for not only clinics but also large-scale vision screenings.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_61
SharedIt: https://rdcu.be/dnwMl
Link to the code repository
https://github.com/yanzipei/VF-HM
Link to the dataset(s)
N/A
Reviews
Review #5
- Please describe the contribution of the paper
The authors tried to predict the visual field test results from fundus images. An axillary task with staging myopic maculopathy regularizes the primary task. The loss function is a combination of two loss metrics, and a cosine similarity value helps to modify the gradient of the loss. Ordinal classification treated visual fields as discrete numbers. Predicting visual field from fundus images is of great value to clinics and if successful, it can be used for large population-based screening.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The abstract is well-written and easy to follow. Proper ablation and comparison studies. An auxiliary task regularized the prediction. Evaluation metrics were properly designed and better performance of the proposed method was clearly shown.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Fig 1 still shows some disagreement between the ground truth and the predicted results. The sample size is quite limited. Especially given the fact that the visual field test is subjective and the reproducibility is not very high, more subjects should be included.
Lambda tuning seems to be coarse as the changes from 0.01 to 0.1 and from 0.1 to 1 are too abrupt. - Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code is not available and the parameters of the network are not provided. The ground truth is visual field test which is not highly reliable. Myopic maculopathy stage is labelled by an ophthalmologist.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Overall, it is a very interesting study that the authors aimed to correlate structure to function using auxiliary learning. The study is well designed. The main concern is the sample size. The limited number is critical here as the visual field test is not highly reliable. From the results shown in Figure, the variance between GT and prediction is still large.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A good application of recent methods to a clinical problem but the sample size is limited.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper presents a novel approach that utilizes color fundus photos to predict visual field for high myopia management. The prediction of VF is formulated as a ordinal classification problem and incorporate an auxiliary task for MM severity classification to assist the generalization of VF estimation. The results show major improvement compared to a simple regression approach.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is very well written. Clear structure and relatively easy to follow.
- The formulation of the problem is sounding, with details provided to elaborate the methodology as well as rationale despite of page limits.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The major weakness to me is the clinical utility. The authors stated in the introduction that one of the major contribution is “VF-HM is the first work for VF estimation using fundus for HM, allowing for more convenient and cost-efficient detection of vision loss in HM, which could be useful for not only clinics but also large-scale vision screenings.” However, this is not reflected in the results. The authors showed the improvement w.r.t. RMSE and MAE compared to a simple regression model. But what does this mean in terms of clinical utility? Is it good enough? Is there any other perspective to consider? etc. It would be great if the authors could elaborate more on it.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
There is no code repo + data access provided, whereas the implementation details as well as parameters are given.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please refer to the weakness of the paper.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents a novel approach for VF estimation using imaging. Though the clinical utility is still unclear, but it definitely opens a new path way of clinical application for HM management.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a approach called VF-HM for estimating VF using fundus, which can help diagnose and monitor ocular complications associated with high myopia. The paper’s contribution lies in providing a more efficient and effective way of estimating visual field sensitivity in patients with high myopia, which can aid in the diagnosis and management of ocular complications.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper proposes a novel method, VF-HM, for estimating VF from fundus in high myopia patients. This is the first work to use fundus photographs for VF estimation in HM patients, which could be a more convenient and cost-efficient alternative to other methods.
- The figure, table, and organization are clear and well-structured, making it easy for readers to follow the proposed method and results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The technical novelty of the method may be limited. The use of multi-scale fusion and auxiliary tasks in machine learning models is not a novel concept.
- While the proposed approach using fundus photographs provides an objective and convenient way of estimating visual field sensitivity, there are some studies that have proposed estimating VF from OCT. It is not entirely clear what the advantages and disadvantages of using fundus photographs versus OCT for estimating VF.
- The paper should clearly present the clinical contribution of the proposed approach for estimating VF using fundus, as its technical novelty may be limited. Additionally, a comparison with other methods, such as estimating VF from optical coherence tomography (OCT), is needed to identify which method is more accurate and feasible in clinical practice.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper provides a detailed description of the proposed method and the experimental setup, including the dataset used, evaluation metrics, and implementation details. However, it does not metion to the code or any information on how to access the implementation.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In the future, it would be interesting to extend this approach to patients with other types of diseases, such as glaucoma. The proposed method could potentially be adapted and optimized for use in other patient populations, which could have significant clinical implications.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the technical novelty is limited, the clinical relevance is reasonable.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes a approach called VFHM for estimating VF using fundus, which can help diagnose and monitor ocular complications associated with high myopia. The paper’s contribution lies in providing a more efficient and effective way of estimating visual field sensitivity in patients with high myopia, which can aid in the diagnosis and management of ocular complications. The significance of this work and the structure and integrity of the manuscript have been praised by the reviewers. However, a serious problem in this paper is that the training data are few and the distribution of the visual field value of the experimental data is lacking. Moreover, the innovation of the method is limited. Experimental results were evaluated with RMSE and MAE. If the visual field loss of test data was not serious, that is, the visual field value distribution of most test points was concentrated around 20-30, then the RMSE and MAE would be small after the predicted results tended to the average, so it was not convincing to use only these two indicators. I think it is necessary for the author to supplement the distribution of predicted visual field values in the camera-ready manuscript, and to add evaluation indicators such as R2, MAPE or SMAPE, so as to demonstrate the significance of predicting regression values.
Author Feedback
We sincerely appreciate all constructive comments from reviewers and meta-reviewer. We will respond to each reviewer’s comments in the following.
[R#2] “Using fundus vs OCT” [Re] Both fundus and OCT provide an objective measurement of retinal morphology. In brief, fundus provides two-dimensional images of the surface structure of retina captured by a specialized camara, while OCT provides cross-sectional analysis of the retina (three-dimensional). Although OCT offers better depth information, fundus is most commonly used for the diagnosis and evaluation of retinal disease as it is relatively quick, simple and non-invasive. In particular, the cost-effectiveness and convenience of fundus can provide significant advantages when establishing community screening and rural programs.
[R#2] “Comparison with other methods” [Re] Several studies have proposed to estimate VF using OCT, in which the thickness maps of different retinal layers were calculated from OCT. In this study, we also have compared our approach to methods estimating VF from thickness (in Table 1).
[R#2] “Future direction” [Re] We agree that our approach has the potential to be extended to other types of retinal diseases, such as glaucoma, with the underlying hypothesis of the “structure-function relationship” in retina. We also believe such an adaptive idea could greatly enhance future research on VF estimation in different patient populations, while deepening our understanding of the structure-function relationship across different diseases.
[R#4] “Clinical utility” [Re] Sorry for the confusion. Measuring VF by standard automated perimetry is prohibitively time-consuming and subjective, while fundus is a quick and simple imaging technique capturing retinal morphology, which likely associates with VF sensitivity. Therefore, using machine learning models to estimate VF from fundus is a cost-effective and convenient alternative, particularly when establishing community screening and rural programs. In this study, our proposed VF-HM provides a more precise estimation of VF with respect to RMSE and MAE, which are commonly used in other VF estimation studies. We will elaborate more on the clinical utility of this study, and include other evaluation indicators such as SMAPE, to demonstrate the significance of prediction.
[R#5] “Disagreement in Fig 1” We agree that there are some disagreements. In this study, we show that our proposed model significantly outperforms baselines, and show the potentials to predict local vision loss, which is important for clinical practice. Therefore, although the predicted results do not perfectly align with the ground truth, we believe our research could enhance the management of high myopia by facilitating early evaluation of vision loss. In future study, we will carefully analyze the areas of disagreement and investigate potential sources of error (e.g., sample size or data diversity), to enhance the accuracy and consistency of our predictions.
[R#5] “Limited sample size” [Re] We agree with that the sample size is limited and more subjects should be included. In this study, both fundus and VF are filtered by registered ophthalmologists following standard procedure, and only those data with high quality are included for analysis. For instance, those visual field test with excessive false positive losses, false negative losses or fixation losses have been removed. In addition, we are collecting more data from different clinical sites, and we will follow all constructive comments to evaluation our method more rigorously in the future study.
[MR] [Re] Thanks for your constructive comments. We agree that the sample size is limited, and we are collecting more data from different clinical sites, and we will follow all constructive comments to evaluation our method more rigorously in the future study. As for camera-ready manuscript, we will include the distribution of predicted VF values, add evaluation indicator, and attach code link.