List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Wenqi Zhou, Xiao Zhang, Dongdong Gu, Sheng Wang, Jiayu Huo, Rui Zhang, Zhihao Jiang, Feng Shi, Zhong Xue, Yiqiang Zhan, Xi Ouyang, Dinggang Shen
Abstract
Pulmonary vessel segmentation in computerized tomography (CT) images is essential for pulmonary vascular disease and surgical navigation. However, the existing methods were generally designed for contrast-enhanced images, their performance is limited by the low contrast and the non-uniformity of Hounsfield Unit (HU) in non-contrast CT images, meanwhile, the varying size of the vessel structures are not well considered in current pulmonary vessel segmentation methods. To address this issue, we propose a hierarchical enhancement network (HENet) for better image- and feature-level vascular representation learning in the pulmonary vessel segmentation task. Specifically, we first design an Auto Contrast Enhancement (ACE) module to adjust the vessel contrast dynamically. Then, we propose a Cross-Scale Non-local Block (CSNB) to effectively fuse multi-scale features by utilizing both local and global semantic information. Experimental results show that our approach achieves better pulmonary vessel segmentation outcomes compared to other state-of-the-art methods, demonstrating the efficacy of the proposed ACE and CSNB module. Our code is available at https://github.com/CODESofWenqi/HENet.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43898-1_53
SharedIt: https://rdcu.be/dnwBN
Link to the code repository
https://github.com/CODESofWenqi/HENet
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a method to segment pulmonary vessel trees in CT images with low contrast. Two crucial components are proposed: Auto contrast enhancement for setting window level and window width automatically, and cross-scale non-local block for merging multi-scale features. Using a private dataset, effects of individual components are examined in ablation study, and the proposed method is also compared with previous methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is well structured, and the network architecture is also clearly illustrated.
- The ablation study clearly shows that U-Net can be enhanced when combined with the proposed components.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The comparison is only done with a private dataset
- In the ablation study, the baseline method is too weak (plain U-Net)
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- The results are difficult to reproduce, since the dataset is private
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
Description for Auto Contrast Enhancement (Sec. 2.1) is intuitive, but some details are hard to follow. For example, the authors state “Then, it passes through a convolution layer to be downsampled into half-size of the original shape, which is utilized to derive the following shift map and scale map”, but it is not described how the shift map and scale map are derived.
-
The method is evaluated on a private dataset, for which the ground truth is created by two radiologists. The diversity of the data and annotator is small. It is unclear how well this method generalized on other datasets. An evaluation on more public datasets would be far more convincing.
-
In the ablation study, U-Net is used as baseline, but as Table 2 shows, U-Net is much weaker than some other methods. What would the results of the ablation study look like if ResUNet++ or CS^2-Net is combined with ACE or CSNB?
-
Results in Table 2: I suppose ASD and HD are both surface-based metrics. Why are the values of HD more than one magnitude larger than values of ASD in all compared methods? Does this mean that there are some regions with errors which are significantly larger than errors in other regions?
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is interesting, but the description is occasionally unclear, and the baseline method in the ablation study is too weak.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper presented a novel HENet architecture for accurate pulmonary vessel segmentation. To address the issue of the diverse intensity values and morphological distributions of vessels, an ACE module and a CSNB are integrated into the U-Net respectively. Experimental results have demonstrated the superiority of both modules in segmentation performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses the challenges in vascular segmentation tasks and proposes innovative solutions to tackle them. Specifically, an ACE module and a CSNB are utilized to enhance the vascular representation learning at the image- and feature-level respectively. The former is based on the idea of radiologists using different WW/WL settings to observe CT images, automatically adjusting the intensity value distribution of the images to better segment vessels at different regions. The latter uses ANB to fuse features of different semantic levels, enabling better segmentation of vascular structures with complex morphologies.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This paper lacks validation on publicly available datasets. There are several vascular segmentation challenges in the Grand Challenge Website that can be used for further validation, such as Vessel12, CARVE14, Parse22.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of this paper is good. The proposed network architecture, parameter settings, and experimental details are all described in great detail.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- In the ACS module, the input image is down-sampled by one convolution with stride of 2. What is the consideration of down-sampling? If down-sampling is necessary, can other down-sampling methods be used instead?
- In the CSBN, a top-down multi-scale fusion method is used. I am interested in the bottom-up method, which is similar to the skip connection in UNet++. Further experiments can be conducted to verify this.
- In the implementation details, a two-stage training method is used. Do both stages of the training use vascular segmentation task? Why is it necessary to separately train a U-Net model with the ACE module first?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents novel solutions to address the challenges faced in vascular segmentation tasks and the experimental results demonstrate the effectiveness of the proposed method.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper tackles an important yet challenging task of pulmonary vessel segmentation. More importantly, they propose a hierarchical enhancement network for vessel segmentation in non-contrast CT images. An important contribution of this work is development of the auto-contrast enhancement (ACE) module. The authors conduct an exhaustive ablation study and comparison with other state-of-the-art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strengths of this work are:
- The most important contribution of the proposed framework is the ability to learn contrast enhancement at different window levels. This could potentially reduce the time taken by radiologists to manually set WW/WL for detecting subtle pulmonary vessel structures
- The authors also propose a CSNB (cross-scale non-local block) to model long range dependencies and aggregate the local-global features for segmentation
- The ablation study provides good insights into the performance of different model components
- The comparison with state-of-the-art is exhaustive
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some weaknesses to be noted are:
- Did the authors conduct statistical comparison of evaluation metrics in Table 2?
- The authors should present more qualitative results further highlighting model performance
- Does the model learn bias in radiological ground truth assessments of pulmonary vessels? If so, would the models be generalizable to larger cohorts?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Fair
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The authors should conduct statistical tests for all metric comparisons in Table 2. We recommend that the authors use a non-parametric Mann Whitney U test or a t-test (only if assumptions are satisfied)
- We suggest authors provide more elaborate qualitative evaluation results
- Please comment about model bias to radiological ground truths in the results section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper tackles a challenging problem and achieves reliable performance which is better than other state-of-the-art methods. Another important contribution is the ACE module, that learn the contrast enhancement levels and could significantly reduce radiologist intervention. The integration of NCSB also improves model performance. The results sections includes ablation study and an elaborate comparison with other methods.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors have attempted to address most of the concerns, but have not conducted a statistical analysis to assess if their results are significant or not. This raises a pertinent concern regarding the main contribution of the paper.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes a deep learning based method for segmentation of pulmonary vessel trees from CT images which are not contrast enhanced. It consists of a UNet like overall structure where skip connections are replaced with a cross-scale block to merge multi-scale features. Further, it introduces a contrast enhancement block as a first convolutional block to mimic radiologists setting window level (WL) and window width (WW) transformations. Evaluation is performed on an in house dataset of 160 non contrast enhanced images with a ground truth annotation from two radiologists. Ablation studies and comparison with other methods are performed.
A major contribution that I would see from this paper would be if the dataset with its manual annotation from two radiologists could be made publicly available.
Reviewers are in favor of the paper, however, a few shortcomings are mentioned, which should be treated in the author rebuttal. It is not clear how the manual ground truth, coming from two radiologists, was finalized. Was there a consensus step between raters, or was the intersection of the two annotations taken as the ground truth annotation? The evaluation can not be easily reproduced due to the lack of access to the private dataset. In the comparison to related work, the compared methods are mostly generic architectures, only two approaches are actually dedicated vessel segmentation approaches, which limits the conclusion that can be drawn from this experiment. Also the differences to CS2Net are small, or even slightly worse as in the important false positive rate. Differences in performance are not analyzed in terms of statistical significance (e.g. using AutoRank: Herbold2020).
Furthermore, I would like to add that the CSNB module for multi-scale connection is not a very original idea, this has been proposed in other works. Most importantly, however, I have to question the motivation and reasoning behind the Contrast Enhancement module (ACE). It is not entirely clear why this is necessary. The argument made is, that the network should mimic how radiologists visualize data using different WL and WW settings. However, this has nothing to do with inherent contrast of the scanned image data. Image data is 12 bit and vessel structures show contrast to the neighbouring lung tissue. WL and WW transformations are only used by radiologists to overcome shortcomings of monitors not being able to simultaneously display hundreds of different intensity values, and the human eyes can only distinguish a limited number of greyvalues as well. Therefore, the networks trained on the 12 bit input data (or a floating point representation of the same range) will never have the need for dedicated contrast enhancement. In case intensity values should be normalized across a dataset, techniques like CLAHE could be used, but in practice data augmentation also is useful for that. Summing up, I do not see a reason for a dedicated contrast enhancement module and I would assume the boost in performance comes from the additional network parameters that are invested in this module, and not so much from this specific design. Therefore, I am not convinced that ACE is a meaningful contribution, and I invite the authors to convince me of the opposite in their rebuttal.
Author Feedback
We appreciate the constructive feedback from all reviewers. Responses to major comments are given below: The rationale of ACE module (MR): Reviewers are concerned that neural networks may not need image enhancement inputs given deep learning’s great capability for feature extraction, i.e., HU values (whether using WW/WL or not) provide adequate contrast for vessels in the learning procedure. However, we argue that global and local image contrasts do affect the results in both traditional image processing and deep learning. Therefore, a learnable ACE could automatically adjust dynamic and local contrast thereby helping distinct subtle and/or large vessels from their backgrounds. Herein, ACE is trained based on the segmentation tasks coupled with the following networks. ACE can also be regarded as a novel contrast augmentation and adaptation for convolutional networks. Other traditional image enhancements are available such as histogram equalization, adaptive filters, etc. However, CLAHE as a classic enhancement algorithm cannot handle the highly variable HU ranges and dynamic proportions of vascular voxels at different locations. Additionally, they do not enhance image contrast based on the target objects, and cannot be learned from the segmentation tasks, while our proposed ACE can. Details of ACE (MR, R1, R2): The ACE module only consists of 3 convolutional blocks, with the first one down-sampling the input data to avoid rigid change in the up-sampled shift/scale map, and the subsequent two blocks deriving the shift and scale maps. Down-sampling operations can be varied and convolution kernel size can also be adjusted as per task needs. Fig. 3 illustrates that the contrast of vessels could be significantly improved by ACE, with minimal network parameter increase. The results in Tab. 1 shows significant improvement of the proposed lightweight module. Clarification of CSNB (MR, R2): The CSNB is designed to address the scale variations of pulmonary vessels by fusing features across different scales and forming local-to-global information connections. We innovatively use the ANB to compose the CSNB in a top-down way, taking advantage of the fine-grained information in the top-level feature maps to guide the lower-level feature maps accurately. The bottom-up approach is also an interesting aspect that we plan to investigate further. Justification on “weak” baseline (R1): We selected the simple UNet as our baseline to give priority to prioritize the development of our 2 proposed modules, and to avoid any interference from other modules present in more complex networks. By using this baseline, we can ensure a clear and lightweight structure and still outperform other methods, e.g., our model has only about half of parameters compared to ResUNet++. Advanced networks will be evaluated in future applications. Comparison with related works (MR, R3): We compared the proposed model with the most related SOTA deep learning-based methods (Tab. 2). Results demonstrate the best performance of our method, although not all the improvements are significant. The FPR of our method is slightly higher than that of CS2Net, but with better Dice and mIoU, indicating that CS2Net could potentially exhibit under-segmentation of vessels. Overall, the proposed lightweight model shows the best performance for segmenting entire pulmonary vessels in an end-to-end fashion. More statistical analysis and elaborate qualitative results will be included in the final paper to highlight the effectiveness of our method. Dataset (MR, R1, R2, R3): There are no publicly available datasets of non-contrast CT scans labeled with both pulmonary arteries and veins. We collected and annotated a private dataset by two radiologists, and a senior radiologist finalized the manual ground truth where consensus could not be reached. We acknowledge that making the dataset publicly available would be beneficial for the community and will release a portion of the data after publication.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Authors have addressed some of the concerns in the rebuttal. The ACE module was better justified, however, I am still convinced that performing operations like CLAHE together with a much more sophisticated intensity augmentation than the one the authors proposed in the paper would have a similar effect. But the argument that the learned contrast enhancement is task specific (i.e. vessel segmentation specific) due to the inclusion into the end to end training still has some merit. Another concern was the lack of significant improvement to state of the art, which the authors confirmed from their tests in table 2. Authors promised to include this limitation in a final manuscript. Authors also clarified the procedure how ground truth annotation was derived (two radiologists plus a senior radiologist for consensus, which is a strong setup), and promised to make a part of the annotated dataset publicly available, which could be an important contribution of this paper. After discussion phase, reviewers all still agree on paper acceptance, therefore, I slightly tend to vote for acceptance as well, despite some shortcomings with the experimental setup and results. In case of confirmed acceptance, I trust the authors will take all mentioned criticisms into account to revise the paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have sufficiently addressed the comments regarding ACE and CSNB modules, as well as the experimental settings. They have noted that the reviewers’ comments will be incorporated into the final version of the paper. Hence, I suggest accepting this paper for MICCAI.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I have read the comments and rebuttal. This paper is about pulmonary vessel tree segmentation in CT images. The method is developed based on a hierarchical enhancement network, including an auto contrast enhancement module and cross-scale non-local block. Parts of the concerns raised by the reviewers have been addressed. There is still some confusion about the ACE component (number of convolutional blocks, and the need and performance of the ACE component).