List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Bowei Zeng, Yiyang Lin, Yifeng Wang, Yang Chen, Jiuyang Dong, Xi Li, Yongbing Zhang
Abstract
Progesterone receptor (PR) plays a vital role in diagnosing and treating breast cancer, but PR staining is costly and time-consuming, seriously hindering its application in clinical practice. The recent rapid development of deep learning technology provides an opportunity to address this problem by virtual staining. However, supervised methods acquire pixel-level paired H&E and PR images, which almost cannot be implemented clinically. In addition, unsupervised methods lack effective constraint information, and the staining results are not reliable sometimes. In this paper, we propose a semi-supervised PR virtual staining method without any pathologist annotation. Firstly, we register the consecutive slides and obtain the patch-level labels of H&E images from the registered consecutive PR images. Furthermore, by designing a Pos/Neg classifier and corresponding constraints, the output images maintain the Pos/Neg consistency with the input images, enabling the output images to be more accurate. Experimental results show that our method can effectively generate PR images from H&E images and maintain structural and pathological consistency with the reference. Compared with existing methods, our approach achieves the best performance.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_23
SharedIt: https://rdcu.be/cVRrF
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
the authors designed a Pos/neg classification module for consistency. Also they achieved the transformation of H&E staining to PR staining for breast cancer for the first time.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Well-written, use of significant metrics for evaluating achieved results
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There is no clear if the dataset is free and public
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
As said before, the dataset not seems to be free and public
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
As said before, the dataset not seems to be free and public
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is a very good scientific contribution in the field. The experimental framework is robust and denotes a lot of experimental work and analysis
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper describes a method of virtual staining that transfers between H&E stain and progesteron stain in breast cancer histopathological slides. The training is based on consecutive slides with different staining. Te stain transfer of un-paired patches is achieved based on the cycle-consistency framework.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- interesting idea with pos/neg classifier to improve consistency
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- more qualitative than quantitative results
- the metod works on 5x magnification (big FOV) with such magnification clinical usefulness of such processing is questionable
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
the model and additional classifiers are vaguely described, without source code the study is not reproducible
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Comments:
- (p.3) “It is a pity for traditional unsupervised method to discard the information of consecutive slides completely” - this is unclear, please explain/rephrase
- the description of generator on p.4 is vague. If the specific architecture is not crucial and the metod was tested with several different architectures yielding similar results it should be clearly stated in the paper. Alternatively, the specific architecture used should be give in supplementary materials.
- Nash equilibrium (p.5) is not properly introduced of referenced - it could be misleading for non-expert readers - please provide short explanation or reference
- (p.5) “[…] auxiliary classifiers are used to introduce attention for finding the focus region […]” - the auxiliary classifiers are not fully described, please provide further explanation in either mian text or supplement
- (p.5) I believe the Nvidia model 3090 should be RTX instead of GTX, please double check on that
- Results presented in Table 1 are not cross-validated and based on overall only 8 WSI slides, this poses a question about reliability of these numbers
- slide-level performance and ablation test has only qualitative results, that might be biased
- the metod works on 5x magnification (big FOV) with such magnification clinical usefulness of such processing is questionable, please provide some confirmation or reference/citations
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
All in all it is a well structured paper providing an interesting idea in the field of digital pathology. The reliability of the results might be questionable and majority of evaluation is qualitative (hence subjective), but nevertheless the performed study seems to be improving the knowledge in the domain.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors propose a smart system for IHC staining out of a H&E staining.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well written about a smart architecture to virtually stain histopathological tissue in IHC (PRC) from H&E slides taking into account classification into PRC+ and PRC- consistency in the the architecture.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The image processing part about labelling the PR+ patches based on color analysis is tricky as brown is a difficult color to model. The post analysis with pathologists that could be interesting to explain why there is discordance is not done.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reproducible
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Dear authors, very interesting work. I would go further on the “brown” color extraction for PR+ annotation and will add an explanation by pathologist about their perception error in the brownish grading. Proof read again ; “Refenence” instead of “Reference” and Paragraph 2.1 and 2.2 are not always very clear at first reading.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is interesting and well written but deserve to be slightly improved (English and explanation) and perhaps more technical details about color analysis in part 2.1 as well as pathologist perception of brown color in the virtual stained slides.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors propose a method to generate synthetic PR IHC stained images from H&E images. The main innovation is the incorporation of an additional constraint in the cycle-consistency framework that ensures that H&E images are mapped to the correct positive or negative class. Additional information about the pathologists’ evaluation is requested. In addition, it would be helpful to add some information about why one would expect H&E to contain sufficient information to generate synthetic PR images – this is not obvious since presumably if the information were already apparent in the H&E images, pathologists would not have developed IHC stains. It would also be useful to explain why they have chosen to generate a synthetic IHC image rather simply predict the score (for example see https://www.nature.com/articles/s41467-020-19334-3) – presumably interpretability is a factor.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
Thank you for all the invaluable comments. Common questions and responses for each reviewer are listed as follows. Common Questions: 1.Pathologist evaluation: The pathologist makes judgments based on the staining results at slide-level. According to the latest guideline, if the proportion of PR positive cells (nuclei stained brown) is more than or equal to 1%, it can be judged as positive, and less than 1% is negative. Therefore, the pathologist can evaluate easily and accurately. 2.Detailed description of model: The descriptions of specific structures (such as generators, discriminators, classifiers, and auxiliary classifiers) will be rewritten with more details, and our codes will be publicly available. 3.Typos: We will rewrite the “GTX3090” as “RTX3090” and the “Refenence” as “Reference” in our final manuscript.
Meta-Reviewer: 1.Pathologist evaluation: Please refer to Common Questions 1. 2.Why using H&E to generate PR: The positive/negative information of PR is indeed included in H&E so that the existing work can predict the positive/negative of PR based on H&E. However, it is contained in the subtle structural difference of tissues and cells, which is not obvious and pathologists cannot directly observe it. In the clinic, the commonly practice is using IHC to stain the expression of PR protein, which is costly. So it makes sense to generate PR images virtually based on H&E. 3.Why not simply predicting scores: By generating IHC images, the distribution of PR protein expression can be displayed, so localization, qualitative and relative quantitative studies can be carried out, thereby providing a basis for judging whether the whole slide is positive or negative, which is more interpretable than simply predicting the score.
Reviewer 1: 1.Dataset: It will be free and public.
Reviewer 2: 1.Why using more qualitative than quantitative results: The generated images do not have pixel-level ground truth, and most of researchers use the pathologist evaluation to measure the generated results. As seen in Common Question 1, the pathologist evaluation is based on guideline and is credible. Moreover, we are working on building a more effective evaluation system. 2.5x magnification (big FOV): Compared with higher magnification, a patch (256x256) at 5x can contain more tissues and cells, and the positive/negative can be classified based on more tissue information. Moreover, if the generated image of 5x is not clear enough, we can solve it by super-resolution. 3.Detailed description of model: Please refer to Common Questions 2. 4.Explanation of “It is a pity for traditional unsupervised method to discard the information of consecutive slides completely”: Consecutive slides (H&E and PR) contain associated information, while the traditional unsupervised method treats them as independent slides. We utilize associated information through registration to make staining more accurate. 5.Nash equilibrium: The generator and the discriminator fight against and make progress together. Finally, the generated images and the real images come from a similar distribution, and it is difficult for the discriminator to distinguish the real and fake. 6.Cross-validation: With 4-fold cross-validation, our method achieves MAE(5.264), SSIM(0.976), MS-SSIM(0.941), PSNR(30.775), CSS(0.712). The experimental results are consistent with the results in the paper.
Reviewer 3: 1.Details of colour extraction in “brown” regions: For each PR patch, the hematoxylin channel (blue) and the IHC channel (brown) are separated by colour deconvolution, and the proportion of the brown area in the IHC channel is calculated. We set the proportion higher than 0.1 as positive; below 0.02 is negative. 2.The post analysis with pathologists: Thanks for your valuable comments; it will be researched in this area in the future. 3.Pathologist perception of brown colour: Please refer to Common Questions 1. 4.English and explanation: They will be improved, as Common Questions 2 says.