List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Amir Reza Sadri, Thomas DeSilvio, Prathyush Chirra, Sneha Singh, Satish E. Viswanath
Abstract
Wavelets have shown significant promise for medical image decomposition and artifact pre-processing by representing inputs via shifted and scaled components of a specified mother wavelet function. However, wavelets could also be leveraged within deep neural networks as activation functions for neurons (called wavelons) in the hidden layer. Integrating wavelons into a convolutional neural network architecture (termed a ``wavelon network” (WN)) offers additional flexibility and stability during optimization, but the resulting model complexity has caused it to be limited to low-dimensional applications. Towards addressing these issues, we present the Residual Wavelon Convolutional Network (RWCN), a novel integrated WN architecture that employs weighted skip connections (to enable residual learning) together with image convolutions and wavelet activation functions to more efficiently capture high-dimensional disease response-specific patterns from medical imaging data. In addition to developing the analytical basis for wavelet activation functions as used in this work, we implemented RWCNs by adapting the popular VGG and ResNet architectures. Evaluation was conducted within three different challenging clinical problems: (a) predicting pathologic complete response (pCR) to neoadjuvant chemoradiation via 153 pre-treatment T2-weighted (T2w) MRI scans in rectal cancers, (b) evaluating pCR after chemoradiation via 100 post-treatment T2w MRIs in rectal cancers, as well as (c) risk stratifying patients who will or will not require surgery after aggressive medication in Crohn’s disease using 73 baseline MRI scans. In comparison to 4 state-of-the-art alternative models (VGG-16, VGG-19, ResNet-18, ResNet-50), RWCN architectures yielded significantly improved and more efficient classifier performance on unseen data in multi-institutional validation cohorts (hold-out accuracies of 0.82, 0.85, and 0.88, respectively).
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_35
SharedIt: https://rdcu.be/cVRtl
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The main contribution of the paper is a deep learning framework using wavelets as activation functions and short-cuts within wavelon network blocks for residual learning. The authors applied the framework to three data sets, two for prediction tasks in rectal cancer, and one for a prediction task in Crohn’s disease.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strengths of the paper are that it is written very clearly, the rationale for the approach seems fundamentally sound, and there is a dedicated experiment and set of results on the optimization of the skip connection weights, which is critical for understanding the impact of those connections.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There is already a body of literature on wavelon networks, and the addition of short-cuts for residual learn is not very novel, but is worth investigating.
Overall, the results produced by the best wavelon residual network (RWCN-ResNet-15) are only slightly better than the best conventional CNN (ResNet-50). Also, the numbers in Table 2 and Table S-1 are generally quite high, which indicates the chosen clinical tasks may not be challenging enough for the RWCNs to demonstrate their theoretical advantages.
The three data sets used for the experiments are small and the results presented cannot be assumed to be representative of the distributions of rectal cancer and Crohn’s disease patients.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data sets seem to be private, but the main algorithmic concepts of paper are described in sufficient detail that a reader has a reasonable chance of implementing the framework and testing it on their own data.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Looking at the main weaknesses noted above, the greatest improvement that can be made to the paper is to expand the range of data and clinical tasks used in order to explore how RWCNs compare to ResNets under a wider range of conditions. I realize this would be difficult to accomplish during the rebuttal period, so it is not something I would insist on.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, this is a nicely written paper exploring a concept that, while not dramatically novel, is worth investigating. Wavelon networks have not made a major impact on medical image analysis so far, and based on results of this paper, it is unclear whether adding residual links will increase the overall impact. If larger data sets and/or a wider range of data and clinical tasks were used, it could be more convincing.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors’ rebuttal does not argue the fact that residual networks and wavelon networks are not new, but does provide a bit more clarity on how they are integrated, which does not significantly change the novelty of their contribution. The rebuttal’s claim that the “approach was rigorously tested on highly complex and heterogeneous MRI cohorts (326 datasets, 2 diseases, 7 institutions)” is not strongly supported, because the 326 datasets were actually three different cohorts, the largest of which had 153 subjects, while the other two had 100 and 73 subjects respectively. Overall, my opinion is that he samples are still too small for any rigorous validation and demonstration of the benefits of RWCNs that could be considered generalizable. Therefore, my recommendation stays the same (weak accept).
Review #5
- Please describe the contribution of the paper
This paper proposes a RWCN method, and it is an efficient utilization of wavelet functions as activation unit for convolution response.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
RWCN solves the problem of gradient disappearance
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) Some description in section 2 is not correct. (2) The contributions of this paper are not clearly described, and lack of innovation. (3) The experimental part is not compared with the improved method, nor with the advanced method in recent three years; (4) Too little validation data.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Easy to reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
(1) “We also present the theoretical basis for the technical advances offered by wavelet activation functions being utilized within our unique RWCN formulation”. This is only the theoretical description of the corresponding method in the paper. It is the workload, not the contribution. (2) The innovation point is only the proposed RWCN method, the innovation is single, and it is suggested to add innovation points. (3) “To our knowledge, no previous work has specifically examined the properties of residual skip connections within WNs in conjunction with CNNs.” The description of is not accurate. As early as ten years ago, there was an article on the combination of CNN activation function and wavelet. The article only focused on the architecture of wavelon integrated into convolutional neural network, abbreviated as WN. Pay attention to full literature research before expression. (4) VGG was proposed in 2014 and RESNET was proposed in 2015. These two methods were proposed seven years ago. It is suggested to conduct a comparative experiment with the latest in-depth learning method proposed in the current three years. At least compare the effect without combining wavelet with that after combining wavelet, so as to highlight the advantages of your method. (5) There is too little data in the experimental verification part. For the results after training and testing, multiple groups of experimental comparison should be carried out in the verification experiment. If the image display is required due to the length limitation of the article, the list can display as much data as possible.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Poor innovation and experimental results
- Number of papers in your stack
1
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #6
- Please describe the contribution of the paper
The author develope a new architecture form networks, layers and activation perspective using wavelet theory. They evaluate their outperform in different cohorts and topics. They deliver a variation of different network comparison in each topic.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelty in architecture, design in network to layer and activation level. A very well organise manuscript with very sientific evaluation of the hypothesis. Nice Figures and verification of the ideas.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No obvious weakness. Just I suggest a source code of the deveolped layers and networks.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Need of source code url link.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Very nice study. Just include the source code please.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The author develope a new architecture form networks, layers and activation perspective using wavelet theory. They evaluate their outperform in different cohorts and topics. They deliver a variation of different network comparison in each topic. Novelty in architecture, design in network to layer and activation level. A very well organise manuscript with very sientific evaluation of the hypothesis. Nice Figures and verification of the ideas. No obvious weakness. Just I suggest a source code of the deveolped layers and networks.
- Number of papers in your stack
8
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The author develope a new architecture form networks, layers and activation perspective using wavelet theory. They evaluate their outperform in different cohorts and topics. They deliver a variation of different network comparison in each topic.
advantages: Novelty in architecture, design in network to layer and activation level. A very well organise manuscript with very sientific evaluation of the hypothesis. Nice Figures and verification of the ideas. The author will deliver source code.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a deep learning framework using wavelets as activation functions and short-cuts for residual learning. The reviewers expressed mixed opinion on the novelty of the work, ranging from incremental to not novel at all. The authors are invited to provide a rebuttal. The reviewers also pointed out the performance improvement of the proposed method seems to be marginal. Please clarify that, given the context of relatively simple tasks and datasets. The reproducibility of the work was also a concern of R6.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Author Feedback
- Clarify novelty Our proposed RWCN is a novel integrated wavelet network (WN) architecture that employs weighted skip connections (to enable residual learning) together with image convolutions and wavelet activation functions to capture high-dimensional disease response-specific patterns more efficiently from medical imaging data. While wavelets have been examined in the context of CNNs (see below), the unique contributions of our current work are as follows:
- Using a residual connection inside the WN layers within a CNN architecture. The network can take advantage of wavelet properties for capturing complex patterns from the convolutional maps together with residual learning for efficient training optimization. For e.g., in Table 2, RWCN+VGG or RWCN+ResNet yields better performance than VGG or ResNet alone (which are convolutional-only architectures).
- Residual connections to enable learning at multiple layers: not only between blocks but also within each WN block as well. This allows the network to scale to very complex patterns with relatively few parameters. Note ResNet (23M parameters) and VGG (138M parameters) perform worse and are slower than RWCN-ResNet (~1.8M parameters) and RCWN-VGG (~800K parameters).
- A multidimensional wavelet network with shift and scale parameters that are integrated into a convolutional architecture directly, allowing them to be optimized together with the rest of the network parameters. To our knowledge, this is a unique effort compared to previous work which has primarily focused on wavelet decomposition of filter responses (e.g., Zhou et al, IEEE TMI 2020 and Atto et al, IEEE NNLS 2020), running CNN on the output of wavelet filters (e.g. Maggiora et al, IEEE TPAMI 2022 and Liu et al, IEEE Access 2019), or concatenating wavelet transforms together with convolution responses as inputs to a network. Notably, none of these works have leveraged residual learning in conjunction with WNs in the fashion that we have presented here.
Performance improvement vs complexity of tasks/datasets Our approach was rigorously tested on highly complex and heterogeneous MRI cohorts (326 datasets, 2 diseases, 7 institutions). These cohorts suffered different sources of variation (batch effects, scanner variations, and institutional differences) which allowed us to evaluate method robustness. We also examined two different clinical problems (pathologic response vs non-response in rectal cancer, and risk stratification based on the need for surgery in Crohn’s disease). As a result, while well-understood and widely used VGG and ResNet approaches yielded good AUC in discovery, their performance drops significantly in hold-out unseen validation. However, the best performing RWCN approach yielded consistent performance in discovery and validation, which was 5-10% improvement in AUC compared to other approaches.
- Reproducibility Source code will be released together with the full publication to enable reproducibility. Comprehensive performance evaluation of RWCN reproducibility included 10-fold cross-validation as well as hold-out validation for all 3 cohorts, detailed ablation studies of the impact of skip connections, and Shapley evaluation of model output; in total, nearly 850 experiments were performed to evaluate network performance.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal helps clarify certain concerns on novelty and reproducibility of the work. Given the overall optimistic review of the work about its innovation and quality, the paper is recommended for an accept.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
17
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The main contribution of this work is a new deep learning framework called Residual Wavelon Convolutional Network (RWCN) that uses convolutions with wavelets as activation function and weighted skip connections (for residual learning) with the aim capture high-dimensional disease patters and increase flexibility and stability during optimisation. They adapt VGG and ResNet to use RWCN and results are evaluated on 3 tasks: two for rectal cancer prediction from MRI (pathologic response vs non-response) and a risk stratification for Crohn’s disease also from MRI.
Key strengths:
- The use of RWCN has better generalisation to unseen data when compared to classic models (VGG and ResNet)
- The use of wavelets is considered technically novel.
Key weaknesses:
- The choice of dataset to demonstrate this methodology may not be the best
Review comments & Scores: The main concern from reviewers is the choice of the dataset and the novelty.
Rebuttal: Authors have provided a good rebuttal clarifying the novelty, performance improvement and reproducibility, which are the main points raised by the MR. This is also confirmed by some reviewers.
Evaluation & Justification: Although the dataset may not be the most adequate to demonstrate the potential of the method, I believe this is an interesting study proposing an interesting idea worth publishing at MICCAI.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed some concerns on novelty and reproducibility of this paper. Given that the majority of reviewers are in favor of this paper. I recommend to accept it.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR