List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Azade Farshad, Yousef Yeganeh, Peter Gehlbach, Nassir Navab
Abstract
Automated segmentation of retinal optical coherence tomography (OCT) images has become an important recent direction in machine learning for medical applications. We hypothesize that the anatomic structure of layers and their high-frequency variation in OCT images make retinal OCT a fitting choice for extracting spectral domain features and combining them with spatial domain features. In this work, we present Υ-Net, an architecture that combines the frequency domain features with the image domain to improve the segmentation performance of OCT images. The results of this work demonstrate that the introduction of two branches, one for spectral and one for spatial domain features, brings very significant improvement in fluid segmentation performance and allows outperformance as compared to the well-known U-Net model. Our improvement was 13% on the fluid segmentation dice score and 1.8% on the average dice score. Finally, removing selected frequency ranges in the spectral domain demonstrates the impact of these features on the fluid segmentation outperformance.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_56
SharedIt: https://rdcu.be/cVRso
Link to the code repository
https://github.com/azadef/ynet
Link to the dataset(s)
http://www.duke.edu/~sf59/Datasets/2015_BOE_Chiu2.zip
http://people.ece.umn.edu/users/parhi/.DATA/OCT/DME/UMNDataset.mat
Reviews
Review #1
- Please describe the contribution of the paper
— The paper proposes a novel framework for combining spectral and spatial features for OCT layer and fluid segmentation. — The method is evaluated on public dataset with performance gain of 1.9% over other methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
— The main idea of the paper to combine both the spectral and spatial features extracted from the OCT volumes for segmentation tasks. — The authors present the dice for the proposed method in relation to the results on public dataset. — Ablation study on different setting of the proposed framework strengthens the method. — The paper is well written and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There are some minor questions that could help better understand and justify the contribution of the work — Since there is global and local features extracted from the spectral encoder, how important/good is the segmentation without the spatial path? in the is the depth of the network chosen in terms of applying CNN/GCN network? A U-Net is only a spatial model and Y-Net with only spectral model can help to highlight the spectral path — Was any data augmentation used for this work? U-Net requires data augmentation to learn the data distribution and could gain some performance with this. — The numbers for fluid segmentation is very less for RelayNet compared to the results reported in the paper. The authors of RelayNet report performance >0.75 dice for the fluid region. — When the spectral features range is altered, how does the performance correlate to the SNR of the OCT images?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
— The authors are providing the code in the supplementary materials. — Providing the requirement and the hyper-parameters for the framework in the paper is a plus.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
— Adding couple of lines on why x_g = 0 for the initial block will be helpful. — What is the number of learnable parameters in the proposed framework? The spectral domain part is proportional to the dimension of the input image? — Devil advocate: What could potentially downplaying the performance when skipped connection are added between the the two domains?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
— The authors evaluate a the proposed method with proper comparison to existed methods in the literature. The validation in the paper justifies the contribution of the proposed framework.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The paper proposes an end-to-end conditional OCT layer segmentation network, which actually is an ‘”U-net” like model with an additional Fast Fourier Block branches. Furthermore, the segmentation experimental result does reach the state-of-the-art performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper seems to be orgainized well. And all training details are well adressed.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Overall, it is lack of novelty, and its experiments are far more satisfied.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The network is not too complicated to reproduce. The dataset is open access.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Comments: I): The experiments are not sufficient. Only U-net is included in the experimental comparison, which makes me doubt whether it is an ablation study or methods comparison. II): The dice score has many advantages on training a model by materializing it as a loss. And it could be utilised as an excellent metric in segmentation. However, a network trained by dice loss should be evaluated by not only dice but also other metrics. For instance, M-IOU also is a popular segmentation metric which could be involved in the experiments. III): The work lacks novelty. FFT blocks can not be considered as a newly implemented technique for segmentation, while there is no other claimed contribution for methods.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
The authors proposed an architecture for retinal OCT segmentation, which was consists of a spatial encoder, a spectral encoder and a spatial decoder. The proposed method was evaluated on a public dataset, experimental results showed that proposed model outperforms existing models in fluid segmentation retinal layer segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-The paper is well written and organized.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-In this paper, the authors have used only one dataset to evaluate the proposed method. For an architecture paper, I think that additional datasets should be used to evaluate the generality of the model. -I think the design of the comparison experiment is not reasonable. Compared to U-Net, the proposed method has an additional encoder. How can we determine whether the performance improvement is due to the extraction of spectral domain features or to the increase in the number of parameters? -The novelty and contribution of this paper are limited.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The computation cost is not analyzed. It is better to provide the computation cost for each method in this paper. -The author mentioned that spectral encoder can make the model pay more attention to high-frequency information. How to prove this and whether the corresponding features can be visualized？ -If Spectral Norm is replaced by a general convolution operation, what will be the impact on the results?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the proposed method and rationality of experiments.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The author’s feedback answered my main concern, such as model parameters, and comparative experimental design, so I am willing to improve my score.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a U-Net like model with an additional Fast Fourier Block branches framework for OCT layer and fluid segmentation. All reviewers agree that the paper was well-written and easy to follow. However, two reviewers raised major issues about the novelty and experiments. 1) The main contribution of the proposed work is not well-summarized; it is hard to reveal the highlights of the given work. 2) The current form of the manuscript does not illustrate the novelty of the proposed method, particularly given that the Fourier unit introduced in this work are not new, as several works have been published based on it. 3) Please justify whether the performance improvement is due to the extraction of spectral domain features or to the increase in the number of parameters; and how to prove that spectral encoder can make the model pay more attention to high-frequency information, as Reviewer#4 raised?
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Author Feedback
We would like to thank all reviewers for their constructive feedback.
We are delighted that the reviewers found our method novel, our experiments/comparisons justified (R1), and the text well-written, easy to follow and well-organized (R1,R3,R4).
Number of parameters in U-Net vs Y-Net (R1, R4): Our model has less parameters compared to U-Net (U-Net: 7.76M, Y-Net: 7.46M); this is due to the fact that both models start with 32 filters in the first layer and reach 256 filters in the bottleneck. In U-Net, the last layer before the bottleneck should provide all 256 filters, while in Y-Net, the output of the two branches with 128 filters each are concatenated to reach the dimension of 256 in the bottleneck; hence, this small change causes our model to have less number of total parameters.
Effect of spatial vs spectral path (R1, R4): In Tab.2, we presented a variation of Y-Net with two spatial encoders with similar number of parameters as Y-Net with FFC modules. Although this modification improves the performance compared to U-Net, the fluid segmentation is further improved with FFC modules. In addition, alpha=0 replicates the case of replacing spectral norms with conv layers by only attending to local features.
High-frequency information (R4): As mentioned in the text, one of the goals of our model is to learn the high-frequency speckle distribution in the OCT images; the existence of these speckles can harm the model performance when using only spatial features[1]. Nonetheless, by extracting spectral features, our model would be able to learn to disentangle features from different frequency distributions, as demonstrated in Tab.3. This enables the model to attend to more important frequency ranges in the features using adaptive learnable kernels in FFT Convolutions. A visualization of high-contrast speckle in OCT images is demonstrated in [1, Fig. 3].
[1] Schmitt, J. et al. “Speckle in optical coherence tomography.” Journal of biomedical optics (1999).
Experimental comparison (R3): In Tab.1, we compare our method to [18, 24, 25, 29]. To the best of our knowledge, these are the most relevant works to ours. We would have appreciated the reviewer’s suggestion for more comparisons.
Evaluation metrics and datasets (R3, R4): We report the DSC on Duke dataset to be consistent with the previous SOTA [18]. However, we omitted the other metrics and the results on the UMN OCT fluid segmentation dataset since they are in line with the reported results using DSC on Duke (Duke, mIoU: U-Net: 72.71, Y-Net: 73.91; UMN: U-Net DSC: 0.91, mIoU: 0.80, Y-Net DSC: 1.0, mIoU: 0.86). If the reviewers find these of value, they could also be reported.
Novelty (R3, R4): We believe that our contribution lies mainly in the choice of extracting spectral features from OCT images based on our assumption that the global information from high-frequency information in the OCT images enables modeling these features using FFT convolutions. Our ablation study on the frequency ranges supports the claim that the fluid segmentation performance is correlated with the spectral features. Thus, the main novelty lies in modeling the OCT segmentation/diagnosis based on a novel model that simultaneously focuses on both spatial and frequency domain.
Augmentation (R1): We tried different types of augmentation (random flip, color jitter, random rotation, etc.) in our experiments with U-Net, but they caused some loss of accuracy. Therefore, the current reported values are based on the normalization of the data and no extra data augmentation.
Minor Issues (R1): The RelayNet results reported in our paper are taken from [18,29] that follow the 6-2-2 split for the evaluation. However, the results reported in RelayNet are based on cross-validation across different folds, which yields higher performance. We will make this clear in our final version. We also will add more discussions on the reasoning behind x_g=0 and low model performance with skip connections.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents an OCT layer segmentation network combining frequency domain and image domain features, which mainly includes a spatial encoder, a spatial decoder, and a spectral decoder. The reviewers raised major issues regarding the innovative nature of the paper and comparative experiments. After rebuttal, R4 revised the score and indicated that the feedback from the authors addressed the main issues. AC considers that the feedback letter answers the main concerns. The overall strengths of the paper outweigh the weaknesses and AC recommends acceptance of the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have clearly clarified the issues raised by the reviewers. The explanations in the rebuttal look reasonable and correct to me.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal is well written and organized. The author answers the experimental comparison and evaluation metrics. Further, experiment details are well explained.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6