Authors

Minghui Zhang, Hanxiao Zhang, Guang-Zhong Yang, Yun Gu

Abstract

Detailed modeling of the airway tree from CT scan is important for 3D navigation involved in endobronchial intervention including for those patients infected with the novel coronavirus. Deep learning methods have the potential for automatic airway segmentation but require large annotated datasets for training, which is difficult for a small patient population and rare cases. Due to the unique attributes of noisy COVID-19 CTs (e.g., ground-glass opacity and consolidation), vanilla 3D Convolutional Neural Networks (CNNs) trained on clean CTs are difficult to be generalized to noisy CTs. In this work, a Collaborative Feature Disentanglement and Augmentation framework (CFDA) is proposed to harness the intrinsic topological knowledge of the airway tree from clean CTs incorporated with unique bias features extracted from the noisy CTs. Firstly, we utilize the clean CT scans and a small amount of labeled noisy CT scans to jointly acquire a bias-discriminative encoder. Feature-level augmentation is then designed to perform feature sharing and augmentation, which diversifies the training samples and increases the generalization ability. Detailed evaluation results on patient datasets demonstrated considerable improvements in the CFDA network. It has been shown that the proposed method achieves superior segmentation performance of airway in COVID-19 CTs against other state-of-the-art transfer learning methods.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_48

SharedIt: https://rdcu.be/cVD64

Link to the code repository

https://github.com/Puzzled-Hui/CFDA

Link to the dataset(s)

http://www.pami.sjtu.edu.cn/Show/56/126

Reviews

Review #1

Please describe the contribution of the paper

Authors propose to augment features in a siamese-like Unet between healthy and COVID-19 affected CT images. Specifically, healthy and COVID-19 image crops are passed into the neural network and their feature representations are permuted and concatenated at different resolution levels. This approach allows to learn the structure of the airway tree and improve segmentation quality for damaged lungs.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Authors propose novel method to introduce structural information from healthy lungs’ images to noisy damaged by COVID-19 lungs. The pipeline improves airway tree segmentation quality compared to multiple baseline methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Authors claim about transferring topological structure from healthy to noisy images seems optimistic, especially considering the absence of any registration between noisy and healthy patches and their random flipping/rotation.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors complete experiments on publicly available datasets. Authors claim to publish code and pretrained models, however no github link is provided in the text.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

For possible future research direction I would suggest testing this approach to improve segmentation for other similar problems, e.g. Airway tree for patients with pneumonia and/or for noisy images, e.g. low-dose CT.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Novel method; experiments on publicly available data; thorough comparison with existing solution to the problem
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

A collaborative feature disentanglement and augmentation framework is proposed for the segmentation of airway trees. This method can jointly exploit labelled clean CT scans and a small amount of labelled noisy CT scans to train a bias-discriminative encoder for the segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) It is interesting to exploit information from both clean and noisy CT scans, and deal with the domain difference problem of the airway segmentation for these two kinds of CT scans. 2) The disentangle technique is novel and reasonable.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The augmentation and training procedure is not described quite clearly.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

This work could be reproducible
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

It could be helpful for readers if the authors could explain more details about the augmentation and the training procedure.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The motivation of this work and the technique about the disentangle are the two major factors make me give the very positive evaluation on this work. However, the unclear description related to the augmentation and training procedure slightly lower the evaluation.
Number of papers in your stack

7
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This paper utilizes a disentanglement way to tackle clean and noisy domains, aiming to synergistically learn intrinsic features and independently learn unique features for airway segmentation of noisy COVID-19 CTs.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. It is an interesting task to consider generalization to noisy domain datasets.
2. Comparison methods are rich to demonstrate the improvements.
3. The total framework is very clear with Feature Disentanglement and Augmentation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. It’s better to visualize the disentangled features to support the idea.
2. The feature disentanglement approach is widely used in domain adaptation segmentation, what’s the difference compared with them.
3. For feature disentanglement, domain separation network (DSN) [1] designs shared encoder (similar to Ec in your work) and private encoder (similar to En in your work) to capture sharable features and bias features. A reconstruction network is proposed to ensure the feature completeness, which means that both share and private features are useful (avoiding trivial solutions). However, in your proposed CFDA network, there are no constrains to ensure feature completeness. How to prove the learned features are shared or private.
[1] Bousmalis K, Trigeorgis G, Silberman N, et al. Domain separation networks[J]. Advances in neural information processing systems, 2016, 29.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Sufficient details for reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. FLA is applied in every layer of the feature extractor. What if it is applied on only one layer or some layers?
2. More visualizations are needed to demonstrate the effectiveness of disentanglement.
3. What if the network input is clean + clean and noisy + noisy? I’m curious about the performance of them.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Interesting idea and good technical quality.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All reviewers recognized the novelty and value of the proposed feature disentanglement method and the topic of noise domain generalization. The evaluation is also sufficient. Some additional examples such as disentangled feature visualization will further help readers and promote the quality of the paper.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

1

Author Feedback

N/A

back to top

CFDA: Collaborative Feature Disentanglement and Augmentation for Pulmonary Airway Tree Modeling of COVID-19 CTs