List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Wei Xiong, Kaiwei Yu, Liang Yang, Baiying Lei
Abstract
In recent years, the emergence and rapid spread of multi-drug resistant bacteria has become a serious threat to global public health. Antibiotic susceptibility testing (AST) is used clinically to determine the susceptibility of bacteria to antibiotics, thereby guiding physicians in the rational use of drugs as well as slowing down the process of bacterial resistance. However, traditional phenotypic AST methods based on bacterial culture are time-consuming and laborious (usually 24-72 hours). Because delayed identification of drug-resistant bacteria increases patient morbidity and mortality, there is an urgent clinical need for a rapid AST method that allows physicians to prescribe ap-propriate antibiotics promptly. In this paper, we present a parallel dual-branch network (i.e, PAS-Net) to predict bacterial antibiotic susceptibility from fluorescent images. Specifically, we use the feature interaction unit (FIU) as a connecting bridge to align and fuse the local features from the convolutional neural network (CNN) branch (C-branch) and the global repre-sentations from the Transformer branch (T-branch) interactively and effectively. Moreover, we propose a new hierarchical multi-head self-attention (HMSA) module that reduces the computational overhead while maintaining the global relationship modeling capability of the T-branch. PAS-Net is experimented on a fluorescent image dataset of clinically isolated Pseudomonas aeruginosa (PA) with promising prediciton performance. Also, we verify the generalization performance of our algorithm in fluorescence image classification on two HEp-2 cell public datasets.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_56
SharedIt: https://rdcu.be/dnwN1
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents PAS-Net, a new method that uses a parallel dual-branch network to predict antibiotic susceptibility from fluorescent images. The method incorporates feature interaction and hierarchical multi-head self-attention modules, and is evaluated on an in house dataset and HEp-2 cell dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A relevant task is analyzed
- Several contributions are made (T and C branches, FIU, and HMSA)
- Performed an ablation study to motivate different components in their design.
- extensive evaluation against baseline methods
- proposed method is tested against another dataset (HEp-2)
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Clarity of the figure captions can improve, maybe further explanations can help the reader.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Not sure if the work is reproducible, the paper is clearly written but authors do not provide the code. It would require some effort to code it from scratch and probably make a few assumptions.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- in fig 5: what does “ours” show? is it the gradcam on the whole network? or is it the attention visualization?
- in fig 6: It remained a bit unclear, how can we measure a better disentanglement in this figure? I think in a 2D setting, no significant difference can be seen between the maps.
- Not a weakness, I was wondering how does online augmentation affect the training? So each time during the training one of the operators are randomly applied on the images rather than having offline augmentation.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
the paper is well written addressing a relevant problem. The method seems to be interesting.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper proposed a parallel dual-branch classification network (PAS-Net) to rapidly predict Antibiotic Susceptibility from Fluorescence Images. The proposed PAS-Net uses a feature interaction unit (FIU) to integrate local features from the CNN branch and global representations from the Transformer branch and proposes an efficient hierarchical multi-head self-attention (HMSA) module to simulate the global information of an image while effectively reducing computational costs and memory consumption. Experimental results on two HEp-2 cell public datasets demonstrate its effectiveness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The summary of the challenges in the prediction of Antibiotic Susceptibility from Fluorescence Images is sound and clear.
-
The paper provided multiple figures with enough technical details make the paper easy to follow.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
My major concern is that the Vision Tranformer (ViT) has already been proven to be able to capture both global and local features [ref 1, 2]. However, the paper stated that “its self-attention module is good at capturing feature dependencies over long distances, but ignores local feature information”. Please give proper citations of this statement and justify this statement accordingly, as the entire paper is built on this statement.
-
The paper compared the computation complexity of the original multi-head self-attention and the proposed hierarchical multi-head self-attention (HMSA) in equation (3) and (4), but it is not very clear how much the reduction is. It is recommended to add several sentences to further discuss this, as one of the paper’s main contribution claims that the proposed HMSA can ‘effectively reducing the computational costs and memory consumption.’
Ref:
-
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., … & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
-
https://ai.googleblog.com/2020/12/transformers-for-image-recognition-at.html “At higher layers (depths of 10-20) only global features are used (i.e., large attention distances), but the lower layers (depths 0-5) capture both global and local features, as indicated by a large range in the mean attention distance. By contrast, only local features are present in the lower layers of a CNN. These experiments indicate that ViT can learn features hard-coded into CNNs (such as awareness of grid structure), but is also free to learn more generic patterns, such as a mix of local and global features at lower layers, that can aid generalization.”
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
According to the answers to the reproducibility checklist, there will be no code to be released, which makes the reproducibility of the paper a little challenge.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
see the weaknesses above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See the weaknesses above. I would rise my score if the authors can solve my concerns.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper presents PAS-Net to predict bacterial antibiotic susceptibility from fluorescent images. Authors did rigorous validations on different cohorts to substantiate the results that showed the superiority of their approach compared to exiting approaches.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-This paper is well written. -The approach can be considered as novel. -The study did extensive comparison analyses to determine the robustness of the novel approach. -Multiple datasets were utilized. -The post-hoc explanation is provided. -The investigation of the feature space is also available.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
None
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Acceptable
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
None
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
a paper that validated the novel approach throughly.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a method for predicting antibiotic susceptibility. Overall, the reviewers recognize the novelty in the method design and extensive ablation studies. There are however questions regarding the design motivation and computational complexity. More clarifications are needed.
Author Feedback
Thanks for the rebuttal invitation. We itemize our responses to major points as follows: (1) Design motivation (Meta R+R2): To address the challenges mentioned in the text, we exploit the advantages of CNN in extracting fine-grained features to learn the contextual information of the neighboring spatial locations of the feature maps, so as to capture the shape features of bacterial aggregates holding clusters, and the powerful modeling ability of the self-attention mechanism to capture the feature dependencies at long distances, so as to mine the distribution features of bacterial aggregates within the images. To achieve the complementary advantages of both, we propose a parallel dual-branch network for implementing automatic classification of bacterial fluorescence images for rapid antibiotic susceptibility prediction. (2) Computational complexity (Meta R+R2): The original MSA module performs attention computation on the entire input feature map, and its computational complexity is proportional to the square of the number of image blocks (N). In contrast, our proposed HMSA module computes attention in a hierarchical manner, first computing local attention within an image block and then computing global attention among all image blocks. With this approach, only a limited number of image blocks need to be processed in each step, thus significantly reducing the computational effort of the module from (𝑁2) to 𝑂(𝑁𝐺2), where 𝐺2 is much smaller than 𝑁. For example, the size of the input image is 224×224, if the patch is divided according to the size of 4×4, the division will get (224/4)2=3136 patches, i.e., N=3136. However, we set 𝐺 to 4, so the computational complexity of the HMSA module is greatly reduced. (3) Fig. 5 (R1): In Fig.5, “ours” represents the Grad-cam visualization results on our parallel dual-branch network. Grad-cam is used to highlight the discriminant regions of interest for predicting sensitive and multi-drug resistant bacteria. (4) Fig. 6 (R1): In Fig. 6, the small circles in each 2D t-SNE map represent a test sample, and we can observe the clustering of the image features extracted by the different networks. Compared to other models, the features extracted by our model can better distinguish sensitive bacteria (blue) from multi-drug resistant bacteria (pink), with less overlap between the blue and pink circles in the figure. (5) Online augmentation (R1): We use offline enhancement. Specifically, to better train the network model and prevent overfitting, five data enhancement operations are performed for each image, including horizontal flipping, vertical flipping, and rotation by different angles (90°, 180°, 270°).
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal has clarified the design motivation and computational complexity. However, the method novelty seems quite limited and the rebuttal hasn’t directly answered R2’s question.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper develops a PAS-Net framework for rapid prediction of antibiotic susceptibility from bacterial fluorescence images, and the experimental results have validated the effectiveness of the proposed model. Besides, the authors have addressed the main issues raised by reviewers.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors proposed to integrate CNN and Transformer for better predicting antibiotic susceptibility in fluorescent images. Reviewers concerns on the clarity of paper as well as the motivation for the integration were answered in the rebuttal.