List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Zhongyi Shui, Shichuan Zhang, Chenglu Zhu, Bingchuan Wang, Pingyi Chen, Sunyi Zheng, Lin Yang
Abstract
Reliable quantitative analysis of immunohistochemical staining images requires accurate and robust cell detection and classification. Recent weakly-supervised methods usually estimate probability density maps for cell recognition. However, in dense cell scenarios, their performance can be limited by pre- and post-processing as it is impossible to find a universal parameter setting. In this paper, we introduce an end-to-end framework that applies direct regression and classification for preset anchor points. Specifically, we propose a pyramidal feature aggregation strategy to combine low-level features and high-level semantics simultaneously, which provides accurate cell recognition for our purely point-based model. In addition, an optimized cost function is designed to adapt our multi-task learning framework by matching ground truth and predicted points. The experimental results demonstrate the superior accuracy and efficiency of the proposed method, which reveals the high potentiality in assisting pathologist assessments.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_11
SharedIt: https://rdcu.be/cVRvv
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes an end-to-end framework that applies direct regression and classification for preset anchor points. The pyramidal features aggregation module provides low-level and high-level features for multi-task learning framework.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1 The proposed methdod achieves competitive results on PD-L1 IHC stained images of tumor tissue.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1 the novelty of this paper is quite limited. For pyramidal feature aggregation, please refer to “Feature Pyramid Networks for Object Detection” and “Richer Convolutional Features for Edge Detection” . 2 Inadequate experimentation. Experiments were performed on only one dataset. The generalizability of the method cannot be well assessed. The division of the test set will have a certain impact on the experimental results.The paper does not conduct multiple experiments for statistical description. 3 The formulas are confusing.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Can be reproduced based on some related knowledge.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1 Please conduct experiments on multiple datasets to verify the effectiveness of the proposed method. 2 Please conduct multiple trials on the same dataset to get statistical description of the results.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
novelty and experiments.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper proposes a neural network structure that predicts the anchor point for each cell without generating the density map. Two main components introduced in this framework are the pyramidal features aggregation and the optimized cost function.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is easy to follow and the cost function is explained in detail.
Then contribution is sufficient. It deploys the pyramidal features aggregation and the cost function integrate the detection and regression and classification loss.
The experiments are sufficient and clearly shows the improvements.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some notation is confusing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
All the hyper-parameters are given. It is possible to reproduce the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Overall it is a good paper, but some definitions are confusing. What is the definition of the preset anchor point is confusing. Several variables, e.x. x, y, y* are not explained. How does the network get the center point (x,y)?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The contribution is sufficient. There are few things can be improved. The introduction of the point-based problem. What is the preset anchor point. How to get the center point, etc.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The authors proposed a novel framework for dense cell recognition by using multi-task learning and one-to-one matching strategy. The experiment results demonstrate the effectiveness of proposed modules.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors proposed a novel multi-stage cell recognition method, using the point labels. The framework consists of several key improvements, Pyramidal Feature Aggregation, Multi-task Learning and Proposal Matching.
- Experiments suggest the proposed method can provide better performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The motivation for the proposed loss functions needs to be highlighted.
- The selection of hyper-parameters is unclear , such as lambda.
- Some typos make this paper hard to follow and less convincing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code is not provided. All methods used for the proposed framework are depicted clearly and noted with appropriate references.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The dataset is only divided into training and test sets. Does the method employ the early stop training?
- Some cells in IHC slides were not fully membrane stained. How to define whether such cells are PD-L1 positive or negative? What is the standard? example: 50% membrane stained as PD-L1 positive.
- “Regression models” mean which models? It should be clearly described in this section.
- “7times7” is a latex error? “The overall framework of the proposed model is depicted in Fig. 3”. I suppose it should be Fig .1.
- Does Time/s refer to the average time per sample? Seconds or minutes?
- What does ‘IC’ mean in Table 2? The various abbreviations in the paper need to be explained clearly.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is novel and effective. The data and experiments section are somewhat vague.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a cell recognition method in IHC stained images. The method integrates cell detection and classification via multi-task learning. A private dataset is collected for evaluation and results show improved performance over compared approaches. The reviewers have noted commented related to lack of novelty, insufficient evaluation and some method details. In addition, in Table 1, in the single-task setting, it’s not clear how to apply U-Net or other models for the classification task. Is it basically a semantic segmentation formulation with multiple classes? If there’s only point annotations, how were those models trained? What loss functions are used? Need to provide details. Also, this cell detection and classification is an interesting application. The authors should consider releasing the dataset. There are also several other cell detection/counting datasets available. Can the proposed method work well?
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
Q1: Novelty of this paper. (MR, R1) A: This work is the first attempt that uses the point-to-point paradigm in the cell recognition field. Our method eliminates the pre- and post-processing that lead to significant performance degradation in dense cell recognition in the membrane-stained IHC images in recent studies [1,2,16]. Q2: Training details of the regression models. (MR, R3) A: In Tab.1, the models except for P2PNet and the proposed model are regarded as regression models. We first generated the multi-class reference density maps (RDMs) by applying a 2D Gaussian filter on the point annotations as described in [1]. Then, the ground truth masks are generated by binarizing RDMs with a small threshold value (10). Finally, we trained all regression models (e.g., U-Net) by a well-designed loss function, which comprises a binary cross entropy (BCE) loss and an intersection over union (IOU) loss. The weights of BCE loss and IOU loss are 0.8 and 0.2, respectively (see Implementation Details). Q3: Generalizability of our method. (MR, R1) A: The dataset was gathered from seven centers at different times. To be specific, it comprises 485 PD-L1 IHC stained images and 353,883 cell annotations. Fig.2 presents the high diversity of our data. Without bells and whistles, we randomly divided the data from each center into the training and testing parts. Therefore, our experimental results are sufficient to demonstrate the generality of our method. We will also validate the effectiveness of our method in other datasets and report related results in our future work. Q4: Definition of the preset anchor point. (R2) A: {(xi, yi) | i=1,2,…,M} denotes anchor points, whereas {(xj, yj) | j=1,2,…,N} denotes ground truth points. Following the mainstream object detection methods [20,21] and P2PNet [12], we preset five anchor points in each 32×32 pixel region on the original image. One is in the center and the other four are generated by shifting the center point with (-8,-8), (-8,8), (8,8) and (8,-8) pixels respectively. Q5: Typos and formulas in this paper. (R1, R3) A: For “Time /s” in Tab.1, ‘Time’ refers to the average inference time per test image, and ‘s’ denotes the unit is seconds. “IC” is the acronym for independent classification branch. Unlike P2PNet [12], we formulated the cell recognition task as a binary classification problem (i.e., cell detection) and a multi-classification problem (i.e., cell classification). Since the category labels are noisy due to intra-reader variability, we applied the generalized cross entropy (GCE) loss [18] to supervise the classification learning. We will check the manuscript to correct the typos and improve the readability of our formulas in the future. Q6: Motivation for the loss functions. (R3) A: The motivation for applying CE loss to supervise detection learning can be found in the Loss Function Design paragraph. Besides, since the category labels are noisy due to intra-reader variability, we applied the noise-robust GCE loss to improve the performance and generalizability of our model. Q7: Selection of hyper-parameters. (R3) A: α and λ were set in line with [12]. We tuned β in [0.1, 1.0] to maximize the F1 score in our application. To be specific, if the precision score is higher than the recall score, β should be decreased. Conversely, if the recall score is higher than the precision score, β should be increased. We adjusted q from 0.1 to 0.9 with a step size of 0.1, and the results are shown in Fig.3. Q8: The definition of PD-L1 positive cells. (R3) A: Our pathologists annotated PD-L1 positive cells following rules in [22].
[20] Ren S, Faster r-cnn: Towards real-time object detection with region proposal networks, NIPS, 2015. [21] Cai Z, Cascade r-cnn: Delving into high quality object detection, CVPR. 2018. [22] Han B, Chinese Medical Association Lung Cancer Clinical Diagnosis and Treatment Guidelines (2021 Edition). Cancer Research and Clinical, 2021.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a cell recognition method in IHC stained images. Overall, it’s an interesting paper particularly addressing a different problem. The rebuttal has clarified some confusing points. On the other hand, without making the dataset public, contribution from this work is lower and it’s unclear how the method would work on existing datasets. It’s a borderline paper. The final version should be revised carefully to describe the method details and experimental setup more clearly.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes an end-to-end framework for dense cell recognition by integrating multiple components, such as pyramidal features aggregation and multi-task learning. Although some details regarding the experiments are missing, the reviewers believe the idea has merit and would benefit the community. It is suggested that the authors include the rebuttal discussions and consider the reviewers’ suggestions when revising the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
10
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents an end-to-end cell recognition algorithm by point annotation. In the first-round of review, two reviewers provide positive recommendations. I think the rebuttal have addressed the concerns about the novelty of the method, rigor of experimental design, and unclearness in method. In my opinion, this method would address the pressing need of high-throughput cell image analysis in the MICCAI community. For these reasons, the recommendation is toward acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5