List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Hansheng Li, Zhengyang Xu, Mo Zhou, Xiaoshuang Shi, Yuxin Kang, Qirong Bu, Hong Lv, Ming Li, Mingzhen Lin, Lei Cui, Jun Feng, Wentao Yang, Lin Yang
Abstract
Accurate segmentation and analysis of membranes from immunohistochemical (IHC) images are crucial for cancer diagnosis and prognosis. Although several fully-supervised deep learning methods for membrane segmentation from IHC images have been proposed recently, the high demand for pixel-level annotations makes this process time-consuming and labor-intensive. To overcome this issue, we propose a novel deep framework for membrane segmentation that utilizes nuclei point-level supervision. Our framework consists of two networks: a Seg-Net that generates segmentation results for membranes and nuclei, and a Tran-Net that transforms the segmentation into semantic points. In this way, the accuracy of the semantic points is closely related to the segmentation quality. Thus, the inconsistency between the semantic points and the point annotations can be used as effective supervision for cell segmentation. We evaluated the proposed method on two IHC membrane-stained datasets and achieved an 81.36% IoU and 85.51% F_1 score of the fully supervised method. All source codes are available at here.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43987-2_52
SharedIt: https://rdcu.be/dnwJ7
Link to the code repository
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
This paper introduces a new method of nuclei and membrane segmentation from IHC images with point-level supervision. A key difference between the point supervsion and existing point-supervision methods is that in this paper, the point supervision is used for both membrane and nuclei segmentation while the existing methods are mostly for nuclei segmentation. Given that the membranes are not always present together with nuclei and sometimes may also be imcomplete, the difference between this method and previous work is really significant.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
To me, the highlight of this paper is how to use the point supervision for membrane segmentation. Fig 2 is very informative, which clearly explains how this point level supervision is used.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
One of the key issues is about Fig 1. The lower left part of Fig 1 shows pixel-level annotation for training. Is pixel-level annotation part of the proposed methods? I think it is not. But, I don’t understand Fig 1. Maybe Fig 1 is trying to compare the difference between point-level supervsion and pixel level supervision? If so, this Fig should be significantly revised to make it more clear.
Section 3.4 and Section 3.4 are not quite clear. First, “… which can result in nuclei and membranes being inseparably segmented.” I don’t quite understand this part. Can a single pixel belong to both nuclei and membranes? My guess is not. I think a regular SegNet should be able to enforce this. Correct? Then, what is the exact reason why the two channels have to be used in a decoupled way and making two different decoupled predictions? I didn’t quite understand the motivation of decoupling. Second, about the contraints for membrane segmentation, how exactly the model can learn to localize the membrane stains? As a toy example, suppose we have a imcomplete membrane like a half-circle (only the left half of a circle). If the initial model makes a predicion of a half-circle, but only the right half of a circle, I feeel like none of the existing loss term can penalize this error. Is that correct? If so, how can the model learn to make the correct prediction, i.e., the right half of the circle, in this toy example?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
“For existing datasets, citations as well as descriptions if they are not publicly available.” Selected as “Yes”. But, I think it should be “N/A”, since no existing dataset is used.
“For all code related to this work …”, there is not enough information for me to verify
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Section 3.3 and Section 3.4 need to be greatly revised, e.g., better explaining the rational of decoupling and more explaination of why the employed loss will be sufficient.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think the overall paper is good and the proposed idea is very interesting. But, Section 3.3 and Section 3.4 (two important parts) are not very clear to me. Upon clarificaiton, I would endorse for acceptance.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposed a novel framework for the segmentation of membranes and nuclei under the supervision of point-level supervision. It devised a Seg-Net tosegment the nuclei and membranes respectively, followed by a Trans-Net to to convert the segmentation results into semantic points. Four types of loss functions are combined to train the networks. This pipeline can largely reduce the workload of annotating and achieve the comparable performance compared to fully-supervised methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- use point-level annotations to achieve membranes/nuclei segmentation
- comparable performance with those fully-supervised methods
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- This framework is composed of two nets: Seg-Net and Trans-Net which are optimized by four losses. The ablation study of them is missing.
- The format of tables and figs should be paid more attention to.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Since the methods are depicted clearly, it is not hard to follow this work.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Please contain experiments to validate each component in your pipeline and loss function.
- Writing should be improved since there are some typos in the paper.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
novel and effective.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
The author focused on the membrane segmentation method which utilizes nuclei point-level annotation because the process of making pixel-level annotations is time-consuming and labor-intensive. The experiment demonstrated the effectiveness of their proposed method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The motivation of this paper is very important in this field. The explanation of the proposed method is sufficient and persuasive. The experiment demonstrated the effectiveness of their proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some explanation is not clear for me. There is no description of the limitation of the proposed method.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Nothing.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In section 3.4, the author mentioned “For instance, if τ is set to 1 or 2, the expected values of the result would be 1 or 0.5, respectively.”. However, I’m not sure why the result would be 1 or 0.5. Please explain the detail. In Table1 and Table 2, the best scores should be bold for clearly visible for the reader.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please answer my comments above.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers appreciate the contribution of the paper by proposing a novel method for both nuclei and membrane segmentation using point-level annotations. R1 and R3 are already recommend paper acceptance. R2 is a bit conservative at this stage but also endorse paper acceptance upon clarification of Section 3.3 and Section 3.4. So I would recommend acceptance yet still advise authors to go over reviewers comments and resolve their concerns (particularly R1 and R2).
Author Feedback
We sincerely appreciate the constructive suggestions from reviewers. Our responses are shown as follows:
1.The limitation (@R1) In the discussion section, we analyzed the limitations of our approach and acknowledged that our current method lacks precision in segmenting cell nuclei. Further, we emphasized the need for future validation of our approach on Whole Slide Images (WSI) to ensure its effectiveness.
2.Figures and Tables (@R1 and R2) 2.1 About the best scores should be bold in Tables(@R1) Thanks for your reminder. We will make the necessary revisions based on your suggestions. 2.2 About Fig.1. (@R2) We apologize for any misunderstanding caused by our figures, and we appreciate your reminder. In response to your feedback, we have added textual annotations in Figure 1 to highlight the distinctive aspects of our proposed method compared to the conventional fully supervised approach.
3.Details(@R1 and R2) 3.1 About tau and the value(@R1) Due to the presence of the sigmoid function, the output of our model is constrained between 0 and 1. When tau=1, a hinge loss of 0 is achieved when the output value is 1. This indicates that the model strongly prefers an output of 1. Similarly, when tau=2, a hinge loss of 0 is obtained as long as the output value is 0.5. 3.2 Why need decouple (@R2) Indeed, since each pixel can only belong to one of the categories, namely background, cell membrane, or cell nuclei, the model tends to output both the segmentation results for cell membrane and cell membrane in a single channel. This doesn’t affect the subsequent Tran-Net process, but for our purposes, we need to separate the results of cell membrane and cell membrane into different channels. Thus, decoupling is necessary. 3.3 Toy example (@R2) Thanks for your interesting question about whether the model may not accurately identify cell membranes based on the staining patterns. Indeed, this scenario is unlikely to occur, because deep learning is fundamentally a function mapping process. When considering unstained or completely stained cells, they should have no or full response, as otherwise Tran-Net would suffer a loss. Therefore, the model inherently learns to respond to regions where cell membranes are truly stained rather than acting as an inverse function of the staining process. Consequently, for incomplete cells, the corresponding stained membranes should still be appropriately stained as per the model’s learning.
4.Ablation Study (@R3) Due to the limited space in the paper, we included both the comparative experiments and ablation experiments in Table 1. In the ablation experiments, we removed the hinge loss, and removed both the hinge loss and norm loss. The results indicate that the introduced constraint terms effectively ensure the accurate segmentation of cells.
5.Grammatical mistakes and typos (@R3) We sincerely appreciate your valuable suggestions. We have once again reviewed the paper to avoid any typo errors.
The task of segmenting cell membranes from immunohistochemistry (IHC) images solely based on point-level supervision has long been an aspirational goal in the industry. In our pioneering work, we have made significant strides by successfully addressing this challenge, simultaneously segmenting both membranes and nuclei using only point-level supervision. This breakthrough will undoubtedly contribute to the advancement of clinically precise analysis of IHC membrane-stained images. To foster further progress in the field of point-level supervised membrane segmentation, we are making our source code openly accessible for the community, serving as a valuable benchmark for future research endeavors.