Authors

Junlong Cheng, Chengrui Gao, Fengjie Wang, Min Zhu

Abstract

Recently, U-shaped networks have dominated the field of medical image segmentation due to their simple and easily tuned structure. However, existing U-shaped segmentation networks: 1) mostly focus on designing complex self-attention modules to compensate for the lack of long-term dependence based on convolution operation, which increases the overall number of parameters and computational complexity of the network; 2) simply fuse the features of encoder and decoder, ignoring the connection between their spatial locations. In this paper, we rethink the above problem and build a lightweight medical image segmentation network, called SegNetr. Specifically, we introduce a novel SegNetr block that can perform local-global interactions dynamically at any stage and with only linear complexity. At the same time, we design a general information retention skip connection (IRSC) to preserve the spatial location information of encoder features and achieve accurate fusion with the decoder features. We validate the effectiveness of SegNetr on four mainstream medical image segmentation datasets, with 59% and 76% fewer parameters and GFLOPs than vanilla U-Net, while achieving segmentation performance comparable to state-of-the-art methods. Notably, the components proposed in this paper can be applied to other U-shaped networks to improve their segmentation performance.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43987-2_7

SharedIt: https://rdcu.be/dnwJp

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes a lightweight medical image segmentation network called SegNetr, which introduces a novel SegNetr block for local-global interactions and a general information retention skip connection (IRSC) for accurate fusion of encoder and decoder features. SegNetr achieves comparable segmentation performance to state-of-the-art methods while having 59% to 76% fewer parameters and GFLOPs than vanilla U-Net. The proposed components can also be applied to other U-shaped networks to improve their segmentation performance. This is a promising work for medical image segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

the author demonstrated a particularly clear presentation of the ideas and graphical representations in this paper, which is highly commendable and serves as an excellent example for colleagues to learn from.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

There remain deficiencies in certain specifics, particularly with regard to layout requirements and the rigor of the content. Therefore, it is recommended that modifications be made.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

This paper does not provide sufficient information to evaluate its reproducibility. The source code is not available, and the method section lacks the corresponding equations.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

1) The article does not fully explain why the Title is named SegNetr. While the “r” in Unetr represents a transformer for 3D medical image segmentation, what does the “r” in SegNetr represent? Was SegNetr modified based on SegNet? If so, why was there no comparison with SegNet? Please provide a reasonable explanation.

2) The background section does not introduce specific application scenarios or applicable data types, which lacks specificity.

3) In the introduction, the article does not clearly explain why transformers are introduced. Is it due to the weak long-term dependence and insufficient global feature extraction in Unet? Similarly, the article does not express why window-based local-global interaction mechanisms are introduced or the logical relationships expressed in the article are not clear enough.

4) “Skip connection is the most basic operation for fusing shallow and deep features in U-shaped networks.” Is this sentence accurate? Is it really “the most”?

5) In the method, “P” is sometimes italicized and sometimes not, please remain consistent.

6) The article header reads “Rethinking the Local-Global Interactions and Skip Connections in U-Nets” while the title states “U-shaped networks”. Please be consistent.

7) The method part lacks specific formula descriptions, which fails to express the method’s details clearly.

8) In the experiment, most of the experimental results in the article are presented in the form of charts and tables. Please supplement other visual experimental results related to this method.

9) Some expressions in the experiment section are not rigorous, such as “Row3-4 SegNetr and TransUNet obtained the highest IoU (0.775),” which conflicts with the numerical results in the table.

10) Additionally, I noticed that “Tab 1” was used in the table header, while the text uses “Table 1,” which is inconsistent. I suggest using “Table 1” or “Tab 1” consistently as the table name.

11) The “Effect of patch size” section is not part of the ablation experiment and should not be placed in section 3.2.

12) The experiments are not rich enough, “The components proposed in this paper can be applied to other U-shaped networks to improve their segmentation performance” is not verified in the experimental part, which the author says in the abstract.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

In conclusion, my overall recommendation for this paper is based on several key factors. Firstly, the clarity and organization of the paper, including its adherence to formatting guidelines and logical flow of ideas, were crucial in determining my evaluation. Secondly, the novelty and significance of the research findings were important in determining the value of the paper. Additionally, the methodology and analysis employed in the study, as well as the quality and relevance of the references cited, were also considered in my assessment. It is my belief that addressing the areas where the paper fell short, particularly in terms of formatting and logical flow, would greatly improve its quality and impact. In addition, providing more detailed information on the methodology and analysis, as well as including the source code, would enhance the reproducibility of the study. Overall, I recommend that the authors revise the paper to address these issues and improve its overall quality.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The authors designed a self-attention module (SegNetr block) and a general skip-connection (IRSC) for U-shaped networks used in medical image segmentation. Unlike existing works, the proposed SegNetr block enables local and global spatial intersections in images while maintaining a lightweight network size. Additionally, the proposed IRSC can preserve high-resolution detail by balancing location features from the encoder and semantic information in the decoder.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors designed a noval self-attention module (SegNetr block) and a general skip-connection (IRSC) for U-shaped networks used in medical image segmentation. Unlike existing works, the proposed SegNetr block enables local and global spatial intersections in images while maintaining a lightweight network size. Additionally, the proposed IRSC can preserve high-resolution detail by balancing location features from the encoder and semantic information in the decoder. Extensive experimental studies on four medical image datasets demonstrated the superiority of the proposed method compared to existing state-of-the-art (SOTA) approaches. The paper writting is pretty good.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

For some comparable results, statisical significance may be needed.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Implementation details are clear and sufficient.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Can the authors analyze whether the improvement is statistically significant in Table 4?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The authors designed a self-attention module (SegNetr block) and a general skip-connection (IRSC) for U-shaped networks used in medical image segmentation. Unlike existing works, the proposed SegNetr block enables local and global spatial intersections in images while maintaining a lightweight network size. Additionally, the proposed IRSC can preserve high-resolution detail by balancing location features from the encoder and semantic information in the decoder. Extensive experimental studies on four medical image datasets demonstrated the superiority of the proposed method compared to existing state-of-the-art (SOTA) approaches. The paper writting is pretty good.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

A lightweight SegNetr block is proposed to dynamically learn local-global information. An information retention skip connection is proposed to preserve the spatial location information of encoder features.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The manuscript is well-written and technically sound. The proposed method is a general framework that can be transferred to other U-shape networks. The effectiveness of the proposed module is verified legitimately.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I wonder to see the results of the other datasets in the ablation study, it the space is limited, the results can be provided in the appendix during the rebuttal session.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

There seems no problem about the reproducibility of the paper, but the authors are strongly suggested to open their codes during the rebuttal session.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

1.The authors are suggested to explain why there is no result at ‘——’ in Table 1. 2.If the proposed method can be transferred to 3D medical image segmentation? 3.I wonder to see the results of the other datasets in the ablation study, it the space is limited, the results can be provided in the appendix during the rebuttal session.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The manuscript is well-written and technically sound. The proposed method is a general framework that can be transferred to other U-shape networks. The contributions are clearly explained and the effectiveness of the proposed module is verified legitimately.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All three reviewers agree to accept this work. Hence, this work can be accepted. Please prepare the final paper based on review comments, including the resonable explanation of the “r” in SegNetr, an explaination why transformers are introduced, writing issues raised by Reviewer#1, and so on.

Author Feedback

Dear Area Chair,

Thank you and the reviewers for recognizing our work. We are thrilled to learn that our paper has been awarded the status of “early accept”. We will make every effort to improve and optimize our research according to your and the reviewers’ feedback.

Next, we will address the questions raised by you and the reviewers:

(1) The “r” in SegNetr stands for Transformer. For dense prediction tasks, it is crucial to have interaction between local and global information, as previous studies have demonstrated. Transformer enables the model to globally attend to the entire image, thereby capturing global semantic information in the image better and improving the accuracy of segmentation results. In this paper, we introduce a window attention mechanism similar to that in Transformer and construct a SegNetr block that performs dynamic local-global interaction between non-overlapping windows at any input resolution with only linear complexity. It is worth noting that SegNetr is not an improvement over SegNet, but rather an improvement over the generalized U-shaped network. In Section 3.2, we compare the skip connections in U-Net, SegNet, and our SegNetr method, and conduct comparative experiments with various U-shaped networks and SegNet.

(2) Regarding the application scenario and data type used, our research focuses on a medical image segmentation model. In the paper, we emphasize the importance of medical image segmentation in medical diagnosis and treatment and propose that our research work aims to address the problem of medical image segmentation. Furthermore, in the experimental section, we compare four different medical image segmentation datasets and conduct experiments and analysis on the proposed method on all four datasets. Through experimental results, we demonstrate the effectiveness and robustness of the proposed method across multiple application scenarios and data types.

(3) Regarding the visual experimental results, we were unable to include all the comparative results on all datasets in the initial draft due to space limitations. However, we have provided these visual comparison results in the supplementary material to provide the reviewers with a more comprehensive understanding of our experimental results. We hope for the understanding and support of the reviewers.

(4) The symbol “-“ in Table 1 indicates that we were unable to obtain that information. The experimental results presented in Table 1 are either obtained from experiments conducted on the same test set in previously published papers, or from experiments that we reproduced using the author’s open-source code under the same experimental conditions.

(5) We would like to express our gratitude to the anonymous reviewers for carefully reviewing our paper and providing valuable comments. With regard to the formatting issue, we will make the necessary revisions in the final paper to ensure that it meets the requirements of the conference. We greatly appreciate the constructive comments provided by the reviewers, as they are instrumental in improving the quality and accuracy of our paper.

Thank you and the reviewers once again for your recognition and support. We remain committed to working diligently to continuously improve the level and quality of our research.

Best wishes, On behalf of all the authors of the paper.

back to top

SegNetr: Rethinking the local-global interactions and skip connections in U-shaped networks