Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top

List of Papers By topics Author List

Paper Info

Reviews

Meta-review

Author Feedback

Post-Rebuttal Meta-reviews

Authors

Zewen Liu, Timothy F. Cootes

Abstract

We describe an approach to making a model be aware of not only intensity but also properties such as direction and scale during forward propagation. Such properties are important in when analysing images containing curvilinear structures such as vessels or fibres. We propose the General Multi-Angle Scale Convolution (G-MASC), whose kernels are arbitrarily rotatable and also fully differentiable like normal convolution. The model manages its directional detectors in sets, and supervises a set’s rotationality with a novel rotation penalty called PoRE. The algorithm works on pyramid representations to enable scale search. Direction and scale can be extracted from the output maps, encoded and analysed separately. Tests were conducted on three public datasets, MoNuSeg, DRIVE, and CHASE-DB1. Good performance is observed while the model requires 1% or fewer parameters than benchmark approaches.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_8

SharedIt: https://rdcu.be/cVRye

Link to the code repository

https://github.com/Zewen-Liu/MASC-Unit

Link to the dataset(s)

https://blogs.kingston.ac.uk/retinal/chasedb1/

https://drive.grand-challenge.org/

https://monuseg.grand-challenge.org/Data/

Reviews

Review #1

Please describe the contribution of the paper

The authors propose an interesting convolutional module that combines local rotation and local scale encoding. They build upon [1] and propose three novelties to improve the design of [1]: A calibrated Response shaping instead of the response shaping proposed in [1]. An additional loss term that promotes the rotational symmetry of the learned kernels by computing a distance from the kernels at different orientations to a set of reference Gabor functions. An additional module, called index map featurization allows encoding the local scale and local orientation of the input and propagates this information to downward layers.

[1] Liu, Zewen, and Timothy Cootes. “MASC-Units: Training Oriented Filters for Segmenting Curvilinear Structures.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2021.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The two strengths of the paper are the novelty and relevance of the approach as well as the experimental evaluation which is thorough.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The authors failed to convey clearly and precisely their methodology. A lot of details are missing to fully understand the approach.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors mention that the code and dataset will be made available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

My main comment is that the current manuscript is not clear enough to understand the methodology, and a lot of important details are missing.

More precisely: A better motivation/introduction on the calibrated response shaping is necessary. In the current version of the paper, it is hard to understand without reading [1] what is the motivation of this and what is the novelty in comparison to [1]. The paper is written as if the response shaping was something obvious. Without a better explanation, I would remove Fig. 2 (a) since it is impossible to comprehend without context (furthermore, in the legend the authors use the terms “stress” and “node” which are not defined). The parametrization of the kernels is not clear and if they used weight sharing between the different orientations of the same kernel. It is also not that clear why they need two base kernels to cover 8 directions, I think it is to avoid rotating the kernels on smaller angles than pi/2, a comment on that point would be welcome since I think it is a strength of the method to avoid relying on steerability which can lead to resampling issue.

Furthermore, it is not clear from Section 2, what is the position of the proposed approach toward the state of the art (especially toward [1]) and also how the current approach compares to the steerable CNNs. I understand that the information of the local orientation is encoded and propagated throughout the network, but what would be the advantages/disadvantages in comparison to a fully rotation equivariant network?

I have other minor points and questions: Probably that I am missing the point, but If the authors used weight sharing of the two base kernels for the 8 orientations. Why does the PoRE loss need to be applied to the n=16 kernels? If they are rotated/mirrored versions of each other, I would expect Eq. 5 to reduce to something like 4 times the distances to the two base kernels. A quick definition of the metrics would be welcome, for instance, the F1-score can be computed the instance-wise for the MoNuseg dataset, but I think the authors used it pixel-wise. The authors mention doing an experiment with non-Hybrid G-MASC. First, the term hybrid was not clearly defined (either change the formulation or clearly define it). Second, I would be interested to see the result of the same experiment but keep only the Gabor part. I would like to see a discussion on the computation efficiency of the current approach compared to the other baseline. I expect it to be a strong selling point. What is the architecture of the U-Net (Double Conv)? In Fig 3. f) the rescale is not explained.

To conclude, the proposed method is very interesting in terms of novelty. Furthermore, the experiments are convincing. However, the authors fail to clearly explain and motivate the approach. I had to read [1] to have a better grasp of what the method was.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper could be accepted on the basis of the novelty of the approach and the satisfying evaluation. However, it would greatly benefit from clearer writing.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

5
[Post rebuttal] Please justify your decision

The rebuttal discussion did partially clarified issues raised in my first review. It is not 100% clear whether the authors plan to include these clarifications in the final version of their paper.

Review #2

Please describe the contribution of the paper

The paper presents an improvement of the MASC method which introduced computation of oriented filters in CNN networks to detect curvilinear structures while significantly reducing the needed parameters compared to having to learn each orientation individually as in standard CNNs. The main improvements of the method proposed in this paper are a new regularisation function added to the loss which aims at better balancing orientational relationships between kernels as well as integrating pyramid representations to handle responses at different scales.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- it improves an already well performing state of the art method
- a comprehensive evaluation has been done on public datasets which highlights improvements and limitations of the method
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- on the tested datasets, there are only minor differences between MASC and the presented method which raises the question whether the addressed limitations of MASC are major
- the new formulations to handle scale differences do not result in significant performance improvements, at least not on the used dataset
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper generally presents sufficient detail and references for reimplementation.

“In practice we use a set of Gabor reference functions..” It would be good to provide some details about the parameters used.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

The paper is well written and provides a very good evaluation. The MASC approach is an interesting way to incorporate orientation to CNN filters in order to reduce redundancy and parameters and an improvement would definitely be of interest to the community in my view. The presented extensions, in particular the regularisation function could have been better motivated, though. There are some indicators like “..the response shaping function which we found to be unstable in some cases” and “..it helps maintain a stable orientation relationship amongst its kernels”, but the evaluation does not show major differences between MASC and G-MASC. So I am wondering whether the original formulation really has significant drawbacks. At least these drawbacks do not materialise in the performance on the tested datasets. As for the integration of scale representations, the ablation study does not show significant differences compared to not having them. The authors note that the tested dataset may not be suitable as there are only minor scale differences. However, I think as the scale integration is one of the proposed novelties, it would have been good to provide an example to demonstrate an improvement.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

4
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I think the paper is very solid in its overall presentation, writing and evaluation. However, the improvements over MASC seem to be minor and as presented on the tested datasets, the proposed method is more of an alternative to MASC rather than addressing major drawbacks.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

5
[Post rebuttal] Please justify your decision

After reading the author‘s comments, I got a better picture about the differences to the original MASC approach. I still think the evaluation should better highlight the claimed improvements and add significance tests where possible. Overall, the additional explanations are reasonable in my opinion, though, and I think there is enough added value in the presented approach. I therefore change my recommendation by 1 point.

Review #3

Please describe the contribution of the paper

This paper proposes a network, called G-MASC, to encode rotation and scale into the convolution to adapt the model to vessel segmentation and histology segmentation tasks. This explicit encoding can reduce the redundancy of kernels in traditional CNN, thus largely lowing the size of the network. Information on the direction and scale can benefit the analysis of curvilinear structure, e.g., retinal vasculature, and authors evaluate the proposed model with MoNuSeg, DRIVE, and CHASEDB1 datasets.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. This work points out that MASC [8] sometimes is unstable and proposes to solve it by calibrated response shaping and a phase-offset rotational error (PoRE).
2. This paper combines the convolution operations with the orientation information, like using Gabor filters. The proposed model has about 1% parameters to U-Net and shows comparable performance.
3. Experiment datasets contain two retinal fundus image datasets and one histology image dataset. The proposed G-MASC shows comparable performance on DRIVE, and works better on MoNuSeg, comparing to MASC.
4. PoRE design is reasonable to regularise the filters to keep local relationships, which is also verified in the ablation study.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. It is claimed that the original MASC used the response shaping function which authors found to be unstable in some cases, but this has not been verified or clarified in which scenarios. From the experiment results, it is hard to say that MASC is unstable (MASC even performs slightly better than the proposed G-MASC on CHASEDB1).
2. The difference between the original response sharing and the proposed calibrated response sharing is unclear to me. The original one calculates the response across different filters w, while the proposed one seems successively update M with learnt kernels W. What is the benefit brought by the new way?
3. In the ablation study, the performance degrades only when IMF and PoRE are both removed. With either one, the performances are close to the final performance. This raises a question - Do IMF and PoRE substantively or functionally overlap?
4. I think this paper requires a clearer comparison and analysis between MASC and G-MASC, both theoretically and experimentally. For example, why does G-MASC only show considerable improvement over MASC on MoNuSeg (the authors claim G-MASC work better when using fewer directional kernel sets and smaller kernels MoNuSeg, but this is an observation rather than an informative analysis)?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Although some information is missing, such as training epoch, learning rate, and footprint, the authors claimed in abstract that they will release the code and data, so the reproducibility is good.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. Those names across figures and main text should be consistent. For example, “Rotational penalty reference node” in Figure 2 has not been mentioned in the main text. Also, some illustration is missed. What do (c) in Figure 3(b) and Figure 3(c) mean? What does “c kernel sets” in Figure 3(f) correspond to?
2. The authors mention that “a smaller set of training patches (4000 samples) was used in the ablation experiments to show differences more clearly”. Does it mean the proposed modules only show obvious efficacy when the dataset is small? What will happen if the data size is large?
3. I appreciate authors have reported the parameters, but considering the proposed network involves some matrix calculations, I believe some metrics like FLOPS can complementarily show the efficiency of the proposed method.
4. Supplement more comparison between G-MASC and MASC, e.g., it would be good if the authors compares some directional kernels of G-MASC to those of MASC in Figure 1.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper integrates the information of the rotation and scale into a convolution neural network, which largely lightens the model parameters while achieving comparable or even better performance on three datasets. Nevertheless, a few points summarised in weaknesses and comments need clarification, e.g., what does the calibrated response shaping bring and how?
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Not Answered
[Post rebuttal] Please justify your decision

Not Answered

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper contributes a convolutional module that combines local rotation and local scale encoding, which is an extension of the previous method MASC. While all reviewers find the paper pleasant and interesting to read, they also all raise similar concerns regarding the experimental validation, which shows that the proposed method and MASC indeed perform rather similarly. It is important for the authors to address these concerns in their rebuttal: Is their method indeed providing a significant improvement over MASC?
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

6

Author Feedback

We thank all the reviewers for their insightful comments and positive evaluation. We would like to address the concerns from the reviewers as follow.

Explain the improvement of G-MASC compared to the original MASC? [Meta, R1 R2 R3]

The original model relies on its special initialisation for tackling vessel-like objects. We found that the rotational symmetries of the kernels in MASC become more difficult to maintain if training on something very different from the initial kernel pattern (longer updating trajectory), or with orientation-biased training samples.

This makes it hard to generalise MASC. Our G-MASC approach strengthens the ability to retain the rotational symmetry of the kernels.

The G-MASC model achieves 2.6% improvement in accuracy on the MoNoSeg cell nucleus dataset compared to the original model, with significantly better class balance. In the ablation experiment, the full version is found 7.6% better in F1 score and 8.7% better in sensitivity than the original model when the training volume is halved on the DRIVE dataset.

The improvement of G-MASC can be better demonstrated by adding visualised MASC kernels trained on MoNoSeg, and comparing them to the G-MASC kernels. We will include this in supplementary material.

Similar results on the retina vessel datasets? [Meta, R1 R2 R3]

The comparison shows that the new method performs at least as well as the old method, which is what we would expect. Putting the retina vessel experiments together with the cell nucleus experiment can illustrate when the additional rotational symmetry protection is required and the significance of its positive impact.

For vessel images, the prior knowledge carried by the initial kernel has been found sufficient to encourage a stable rotational training. In this case, the additional protection is not as effective as it was on other common tasks.

Also, comparing the two retina vessel datasets, the original model works worse on DRIVE than CHASE-DB1 due to the offset vessel centre. The influence can be greater when using fewer training samples. G-MASC is able to deal with this more effectively.

We will revise the paper to make these points clearer.

Motivation for the new algorithm. [R1, R2]

The motivation for G-MASC is to generalise MASC, which was developed for ridge-like structures, and to explore new ways of imposing constraints to encourage rotational symmetries in the kernels.

We realised the value of the label maps as a side product and their benefits are shown in Table.2. With IMF modules, the new model’s sensitivity increased 8.7% compared to the baseline.

“Why they need two base kernels to cover 8 directions …” [R1]

For patterns which do not have a 180 degree symmetry we can’t use the transpose operation to simulate a rotation. We can only rotate them by multiples of 90 degrees. Thus we use a pair of kernels (and their 90 degree rotations) to cover 8 directions.

Why does the PoRE loss need to be applied to the n=16 kernels? [R1]

In practice, we applied PoRE only on the base kernels rather than all kernels, which is equivalent to Eq.5 up to a scaling factor.

What is the benefit brought by the calibrated response shaping with momentum? [R3]

Matrix M is an estimate of the expected responses from orientations. In early experiments, it was found that large changes in M can encourage rotational inconsistency between base kernels. A solution is the proposed new calibrated response shaping function.

Do IMF and PoRE substantively or functionally overlap? [R3]

The reviewer is right, IMF and PoRE share some information. The point of IMF is for discussing the potential of using property labels for image tasks. Space constraints prevent us from covering this in more detail.

Missing definition and ambiguity.

We will fix the description consistency problem and the missing explanation, and improve the figures in the final version.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors provided a well reasoned rebuttal which answered most of the questions asked. The methodological novelty is still limited, making the paper borderline, but the authors argue convincingly that in practice the extension was not trivial, and I recommend acceptance.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

8

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

All reviewers recommend acceptance after the rebuttal, which made adequate technical clarifications. The final version should take into consideration reviewer comments and include all requested experimental details.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

4

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The paper is well written and provides a very good evaluation. The MASC approach is an interesting way to incorporate orientation to CNN filters in order to reduce redundancy and parameters and an improvement would definitely be of interest to the community.

Reviewers agree to a borderline accept.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

10

back to top

A Sense of Direction in Biomedical Neural Networks