List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Wentian Zhang, Xu Sun, Yuexiang Li, Haozhe Liu, Nanjun He, Feng Liu, Yefeng Zheng
Abstract
By introducing the skip connection to bridge the semantic gap between encoder and decoder, U-shape architecture has been proven to be effective for recovering fine-grained details in dense prediction tasks. However, such a mechanism cannot be directly applied to reconstruction-based anomaly detection, since the skip connection might lead the model overfitting to an identity mapping between the input and output. In this paper, we propose a weight decay training strategy to progressively mute the skip connections of U-Net, which effectively adapts U-shape network to anomaly detection task. Thus, we are able to leverage the modeling capabilities of U-Net architecture, and meanwhile prevent the trained model from bypassing low-level features. Furthermore, we formulate an auxiliary task, namely histograms of oriented gradients (HOG) prediction, to encourage the framework to deeply exploit contextual information from fundus images. The HOG feature descriptors with three different resolutions are adopted as the auxiliary supervision signals. The multi-task framework is dedicated to enforce the model to aggregate shared significant commonalities and eventually improve the performance of anomaly detection. Experimental results on Indian Diabetic Retinopathy image Dataset (IDRiD) and Automatic Detection challenge on Age-related Macular degeneration dataset (ADAM) validate the superiority of our method for detecting abnormalities in retinal fundus images. The source code is available at https://github.com/WentianZhang-ML/WDMT-Net.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_63
SharedIt: https://rdcu.be/cVRsv
Link to the code repository
https://github.com/WentianZhang-ML/WDMT-Net
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
This paper proposed a new encoder-decoder based architecture that uses (scheduled) skip gating (what the authors called weight decay) and multi-task learning with HOG feature prediction. The proposed method has been evaluated on one public dataset for anomaly detection.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
Exploration of HOG feature prediction for medical image reconstruction is indeed interesting and novel.
-
The proposed method achieved the SOTA results on the dataset.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The technical contributions are not sufficient for a good MICCAI paper. The intuitions brought by this paper are limited, and there are many aspects that the authors could study further. (1) In Eq. 1, weighted feature fusion has already been well studied in the vision domain (a similar study can be traced back to “Identity Mappings in Deep Residual Networks”, He etal, ECCV2016) and many following U-Net based papers have tried such weighted skip connections since then. However, the only interesting part is that \alpha is gradually decreased in this paper instead of being learnt as others. In this way, the authors should spend more efforts to study the behavior of such decreasing schedule: linearly scheduled, cosine schedulerd piece-wise scheduled etc. (2) The HOG prediction in this work seems like a direct combination of [13] and [17]. Although there is indeed a new attempt (using different resolution HOG as GT), it is more like a training trick instead of a major technical contribution.
-
Experiment on one single dataset does not seem sufficient to fully evaluate the proposed method.
-
Since the primiary contribution claim is the network with skip connections, the authors should compare with more SOTA skip-connected networks (e.g. Attn U-Net, BiO-Net etc.) in Table 1. Also, these works should be cited in the related work section.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Codes are promised to be released soon.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I hope the authors can have a more thorough study on the proposed components, and conduct more convincing experiments on more datasets and compare with more advanced skip-connected networks.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, this paper indeed proposed a new approach in this domain, however the contributions are limited and the experiments are not enough.
I will consider to improve my rating if the authors can address most of my concerns as stated in the weakness section.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
I have seen authors’ rebuttal and they have answered most of my concerns. I choose to improve my rating. Please include the changes (evaluations on dataset, comparisons with other advanced methods) in the camera ready version if this paper gets accepted.
Review #5
- Please describe the contribution of the paper
This paper explores the applicability of skip connection and multi-task learning to anomaly detection tasks and presents a multi-task encoder-decoder network with weight decay skip connection (WDMT-Net). The authors first design a weight decay training strategy to alleviate the identity mapping problem of U-Net architecture as well as leveraging its strong capacity in feature representation learning. Then they integrate an auxiliary histograms of oriented gradients (HOG) prediction task to the anomaly detection framework for fully exploiting the significant commonalities of normal fundus images. The quantitative and qualitative experimental results indicate the advancement of the proposed method compared with other state-of-the-art method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
a) Different from the previous reconstruction-based approaches which are built upon auto-encoder architecture without skip connection, this paper proposes a novel WDMT-Net with an auxiliary HOG prediction task that adapts U-shape network to anomaly detection task. b) A weight decay training strategy and an auxiliary HOG prediction task are respectively designed to suppress the identity mapping problem and make full use of the shared commonalities of normal fundus images. c) Experimental results demonstrate that the proposed method outperforms other state-of-the-art anomaly detection methods. d) The detailed description of the model and experiment in this paper is accurate and clear. The writing of this paper is coherent and easy to understand.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
a) In “Introduction” Section, the description of the motivation for the proposed auxiliary HOG prediction task is unclear and insufficient. The authors only stated the effectiveness of HOG prediction for self-supervised representation learning, without explaining the correlation between this auxiliary task and the main anomaly detection task. b) In the comparison experiments, except Sparse-GAN and Proxy Ano respectively published in 2020 and 2021, the other competing methods are somewhat outdated. Comparing with these methods can not fully demonstrate the advancement of the proposed method, though it achieves the best performance. In addition to methods designed for anomaly detection, some classic medical image reconstruction framework and method should also be reviewed and re-implemented as comparison methods.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Since the authors described their method and implementation details clearly, this paper has good reproducibility. Besides, they also intend to release their source code related to this work.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The authors are supposed to clearly explain the motivation for the proposed auxiliary HOG prediction task and illustrate the correlation between the auxiliary task and the main anomaly detection task in more detail. In addition to the reconstruction results and the prediction A_M displayed in Fig.3, we hope that the authors can provide visualization results of the auxiliary HOG prediction task to verify its effectiveness. The authors should evaluate the advancement of the proposed method by further comparing it with the appropriate state-of-the-art approaches for medical image reconstruction. - Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper is well and logically written. The main contribution of this paper is that the authors proposed a novel U-Net architecture-based WDMT-Net with a weight decay training strategy and an auxiliary HOG prediction task to simultaneously combat the identity mapping problem and fully exploit the shared commonalities of normal fundus images. However, the description of the motivation for the auxiliary HOG prediction task is unclear and insufficient. Besides, appropriate state-of-the-art approaches for medical image reconstruction should also be reviewed and re-implemented as comparison methods.
- Number of papers in your stack
7
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors have tried to answer the questions raised by reviewers, but there are still some puzzling points. I list them as follows:
-
Unconvincing experiments. In the comparison experiment, except Sparse-GAN and Proxy Ano, the other competing methods are somewhat outdated. The authors could evaluate the advancement of their proposed method by further comparing it with the appropriate state-of-the-art approaches for medical image reconstruction. However, they did not. This greatly compromises the value of this work.
-
Unclear description of motivations. The description of the motivation for the proposed auxiliary HOG prediction task is unclear and insufficient in the original paper. Although the authors state their thought in the rebuttal. I think they still need to organize their narrative well.
To sum up, I cannot improve my score.
-
Review #4
- Please describe the contribution of the paper
In this paper, the author explore the applicability of skip connection to reconstruction-based anomaly detection.Specifically,a weight decay skip connection training strategy is presented to mitigate the identity mapping problem of the U-Net architecture and meanwhile leverage its advantage on feature representation learning. Then, the author integrate an auxiliary task, i.e.,HOG prediction,to the anomaly detection framework, which can fully exploit the significant commonalities of normal fundus images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.This paper comes up with two questions and do validation experiments : whether the skip connection can be helpful for improving the anomaly detection performance?whether the HOG prediction task can serve the image reconstruction (main task) as auxiliary and assist the anomaly detection? 2.This paper’s method demonstrates its effectiveness for detecting abnormal regions in retinal fundus images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.“The skip connections at higher levels tend to mislead the model to bypass the lower levels of features and essentially learn an identity mapping function.”How do you come to the conclusion that the model bypass the lower levels of features? Can you give any theoretical or experimental support it? 2.This paper proposes a weight decay skip connection training strategy. You’ve not give any references. What is the purpose of the strategy? What is the advantage of the weight factor α?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper seems reproductive.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1.In Section 2.3,“with the normal images Taking a gray image”.You miss a period before the word “Taking”. 2.In the introduction part of the paper, you mentioned the reconstruction-based and non-reconstructionbased methods. References of the reconstruction-based method you’ve not mentioned.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The weak reject has been made till current manuscript due to the ambiguity of the motivation and importance of the discounted shortcut. Moreover, the experiments cannot justify the effectiveness of the proposed method.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #6
- Please describe the contribution of the paper
The manuscript presents a multi-task network with weight decay skip connection training for anomaly detection in retinal fundus images. The method is of novelty and improves the classification performance significantly.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The idea of the auxiliary task, namely histograms of oriented gradients (HOG) prediction, is new, which could encourage the framework to deeply exploit contextual information from fundus images. Besides, the authors conducted comprehensive experiments and ablation study on a public dataset to verify the effectiveness and efficiency of the proposed method, achieving the state-of-the-art performance. The manuscript is well organized and written, which is easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The split of the dataset might be questionable, since the authors did not test the proposed method on the normal images. Some details of the method are missing.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the paper might be good since the authors provide us almost of all important implementation details.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
(1) When alpha is 0, M_{i} is D_{i}, so there is redundancy in Eq. (2). The authors mentioned that it can be removed by re-organizing the weights but did not introduce the details. Please elaborate on it in the manuscript. (2) A test image can be classified into normal or abnormal by comparing the reconstruction error with a pre-defined threshold. What is the value of the threshold? In addition, the prediction of the proposed method is a binary vaule, 0 or 1, instead of probabilities in the range of [0,1]. In this case, how can we obtain the AUC value? (3) The authors used only abnormal images of the IDRiD dataset for testing. In this case, how did the authors calculate the AUC and F1-socre? Why not use both normal and abnormal images for testing? (4) In “However, the same rule does not holds for anomaly detection at the test stage.” on Page 6, “holds” should be “hold”. (5) How does the proposed method process a test image? Please add the comparison of the computational cost with the state-of-the-art methods in Table 4.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is novel. The experiments are comprehensive enough and the results demonstrate the superiority of the proposed method.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
The authors have addressed all of my concerns, and I have no further questions.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper is reviewed by three experts in the field. The reviews of this work are quite divergent. The authors are suggested to provide a rebuttal to clarify main issues raised by reviewers, including: 1) The technical contributions is limited. 2) Insufficient experiment. 3) More theoretical or experimental supports for skip connection. 4) More details for weight decay skip connection.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
Thanks for the comments from reviewers. It is pleased to see that the reviewers agree that the proposed pretext task is novel (R2, R5 and R6) and the experiments are preferable (R4 and R5). Even so, some major criticisms are listed to address in the followings.
Validation of proposed method on other datasets. (R2, R4) We have conducted experiment on an extra dataset (ADAM [1]). Our framework achieved an AUC of 0.687, which outperforms the existing approaches, e.g., Proxy Ano (0.675), GANoamly (0.673) and AnoGAN (0.677). The experimental results will be added to the supplementary materials of final version.
Contribution over the SOTA skip-connected studies. (R2, R4, R5) Existing studies stated that skip connections can alleviate gradient vanishing (ResNet) and boost information transmission (U-Net). However, we found such a mechanism is not appropriate for anomaly detection task (as can be observed from Fig. 2 and Table 1). This paper addresses the challenge by implementing a weight decay strategy for the skip connections. In the last paragraph of Sec. 2.1, we explained the mechanism of the weighted factor (alpha) affecting the whole model training process. Moreover, according to the suggestion raised by R2 and R5, the two advanced skip-connected networks, i.e., Attn U-Net [2] and BiO-Net [3], are evaluated. The Attn U-Net and BiO-Net achieve AUCs of 0.581 and 0.606 on IDRiD, respectively, which are lower than that of our method (0.748).
Explanation of ``U-Net model bypass the lower levels of features’’. (R4) We expect to kindly clarify that the ‘higher levels skip connections’ are the ones at the early stages of U-Net transmitting the low level features from encoder to decoder. The lower level features can go through those short-cuts, instead of fusing into the high level semantic features, which degrades the anomaly detection performance. To validate the statement, we removed the skip connections of U-Net to formulate an auto-encoder and evaluated it. The results in Table 1 show the superiority of the auto-encoder, which demonstrate the loss of lower level features caused by skip connections. We notice that the representation ‘higher levels skip connections’ may confuse the readers and will accordingly revise the statement.
The motivation for the auxiliary HOG prediction task. (R2, R5) For normal fundus images, the areas around the blood vessels and optic disc yield the large HOG values, which contain the anatomical structure of the retina. Thereby, formulating HOG prediction as an auxiliary task can encourage the framework to deeply exploit the contextual information from normal fundus images during the training process, which improves its capacity for anomaly detection. The motivation for designing an auxiliary HOG prediction task will be illustrated in the final version. Moreover, instead of simply combining the HOG prediction with the anomaly detection task and form a multi-task architecture, we further developed a novel training strategy, which learns the multi-resolution HOG features from a label pool. The effectiveness of our training strategy has been verified in Table 3. Such a simple-yet-effective method can also be seen as a technical contribution of our study. Note that the similar simple-yet-effective training strategy (random erasing [4]) is well received by the community and inspires several follow-up studies.
Details of test image processing and evaluation metrics calculation (R6) Due to the page limitation, the detailed calculation process is not presented in the manuscript. The related materials, including source code and trained model, will be released once the paper is accepted.
[1] ADAM Challenge: Detecting age-related aacular degeneration from fundus images. TMI 2022. [2] Attention U-Net: Learning where to look for the pancreas. [3] BiO-Net: Learning recurrent bi-directional connections for encoder-decoder architecture. MICCAI 2020. [4] Random Erasing Data Augmentation. AAAI 2020.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have clearly clarified the issues raised by the reviewers. The explanations in the rebuttal look reasonable and correct to me.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I agree that the authors clarufied the questions
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper investigates the applicability of skip connection and multi-task learning to anomaly detection tasks. Experimental results on the public fundus datasets demonstrated better performance over some existing anomaly detection methods. Overall, I agree with majority of reviewers that this paper has merits and can be interesting to the MICCAI community. Thus I lean to vote for acceptance. However, a significant revision is required for including comparison with SOTA anomaly detection methods and illustration of motivation for HOG prediction task.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4