List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Gabriel Jimenez, Anuradha Kar, Mehdi Ounissi, Léa Ingrassia, Susana Boluda, Benoît Delatour, Lev Stimmer, Daniel Racoceanu
Abstract
Quantifying the distribution and morphology of tau protein structures in brain tissues is key to diagnosing Alzheimer’s Disease (AD) and its subtypes. Recently, deep learning (DL) models such as UNet have been successfully used for automatic segmentation of histopathological whole slide images (WSI) of biological tissues. In this study, we propose a DL-based methodology for semantic segmentation of tau lesions (i.e., neuritic plaques) in WSI of postmortem patients with AD. The state of the art in semantic segmentation of neuritic plaques in human WSI is very limited. Our study proposes a baseline able to generate a significant advantage for morphological analysis of these tauopathies for further stratification of AD patients. Essential discussions concerning biomarkers (ALZ50 versus AT8 tau antibodies), the imaging modality (different slide scanner resolutions), and the challenge of weak annotations are addressed within this seminal study. The analysis of the impact of context in plaque segmentation is important to understand the role of the micro-environment for reliable tau protein segmentation. In addition, by integrating visual interpretability, we are able to explain how the network focuses on a region of interest (ROI), giving additional insights to pathologists. Finally, the release of a new expert-annotated database and the code (\url{https://github.com/aramis-lab/miccai2022-stratifiad.git}) will be helpful for the scientific community to accelerate the development of new pipelines for human WSI processing in AD.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_33
SharedIt: https://rdcu.be/cVRrQ
Link to the code repository
https://github.com/aramis-lab/miccai2022-stratifiad.git
Link to the dataset(s)
https://github.com/aramis-lab/miccai2022-stratifiad.git
Reviews
Review #2
- Please describe the contribution of the paper
The authors propose a deep-learning based method for semantic segmentation of taupathies for AD patients. They present a baseline to generate a significant advantage for morphological analysis.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors propose a deep learning framework for plaque segmentation in brain WSIs.
- The authors explore the interpretability and explainability of the deep features.
- Integrates domain knowledge in the segmentation method.
- The observations and the dataset provided can be helpful to improve the AD study wrt plaques.
- The proposed method shows interesting application for quantitative estimation of plaques.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- No major contribution in the method of the study but the application is interesting.
- The dataset size is small (8 whole slide images).
- The dataset is not publicly available so the merit of the method cannot be established and compared to any state of the art method.
- Quantitative analysis of different factors like different modalities, context effect and attention maps are missing. A table showing the contribution of each of the factors will be helpful.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The training/testing code and the dataset for the study will be provided. This will help in reproducing the results and use the method as a baseline.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The quantitative estimation using the trivial software can be included to compare the results.
- More dataset could be included. Deep learning methods on limited dataset could be over fitting.
- Clinical value of the techniques is not clearly discussed.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Deep learning models with explainability features have not yet been applied in tau segmentation from WSIs. The authors develops baseline Deep learning method for this application. The clinical significance of the method is not thoroughly discussed and compared with trivial methods.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
The authors of this paper describe neuritic plaque segmentation using 8 brain whole slide images. The authors try different patch size, different scanners, different stain normalization, and different models (UNet and attention UNet) to segment plaques and improve human annotations.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper uses deep learning to analyze AD tauopathy which has been rarely explored.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I am concerned for the technical novelty because the proposed method uses attention UNet for segmentation. Perhaps emphasizing more on why this problem is important and why others have not done this would highlight this work.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Code is provided as supplementary material.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Emphasizing more on why this problem is important and why others have not done this would highlight this work.
- Including results from UNet architecture (Section 3.1) for scanners and normalization in Table 1 would be helpful.
- According to Table 2, attention UNet provides better results than UNet. In addition, Figures 3-5 show results of attention UNet. Then what is the message from UNet?
- From Figure 5, the authors claim that attention map can improve human annotation. I think the changes in delineation is minor – what do the authors expect to see clinical improvements using this attention map for plaque segmentation?
- There are some typos to be fixed: and -> an (page 4), tunning -> tuning (page 5), diffult -> difficult (page 8), assitive -> assistive (page 8)
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I would like to see more description of why this work is important and why others have not previously done it.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
After reading author feedback, I understand the WSIs are rare and can only be obtained from specialized sources under strict regulations, and the authors plan to release high-quality samples. The authors also emphasized that the attention-UNet can guide pathologists in improving their annotation quality. However, I am still not convinced how much the improved annotation quality with attention-UNet can contribute to AD patient stratification.
Review #5
- Please describe the contribution of the paper
In this paper, a baseline for semantic segmentation of neuritic plaques in human WSI is proposed. Also, the release of a new expert annotated database as well as the code will be useful for the scientific community to accelerate the development of new pipelines for human WSI processing in AD.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A baseline for neuritic plaques is established;
- The expert annotated database is released as well as the code.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- I could not see any technical novelty in terms of the approaches and methods used even though the methods used are solid and well accepted within the community;
- A bit more of explainability exploration could have helped to make the paper stronger, what exactly the model learned.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
good assuming the code and the data will be shared.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- I could not see any technical novelty in terms of the approaches and methods used even though the methods used are solid and well accepted within the community;
- A bit more of explainability exploration could have helped to make the paper stronger, what exactly the model learned.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think it is important to facilitate data and code sharing and this paper can help to this.
- Number of papers in your stack
7
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes a neuritic plaque segmentation for brain WSI. The authors try different patch size, different scanners, different stain normalization, and different models (UNet and attention UNet) to segment plaques and improve human annotations. All reviewers find the application interesting and well-motivated, they have major concerns for the methodological novelty as the paper uses mostly mature methods. Besides, the dataset (8 WSI) is also relatively small. In the rebuttal, authors should address why this problem is important and why others have not done this would highlight this work (R2), and more explainability exploration (R3) , e.g. tontribution of each of the factors (R1).
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6 to 7
Author Feedback
Our work is the first to deal with automated neuritic plaque (NP) segmentation: a very challenging and essential task for Alzheimer’s Disease (AD) patients’ stratification. For now, only manually-assessed neuritic plaque counting by expert pathologists was performed. An unbiased automated plaques detection and segmentation methods for AD are still lacking. Therefore, this study aims to analyze NPs, quantify its morphological characteristics, and address the challenge of weak annotations provided by pathologists.
A major aspect of the problem addressed is the irregular shape of plaques and the proximity of non-plaques tau-positive objects (e.g., tangles and neuropil threads), which are prone to erroneous annotation by pathologists. Indeed, as stated in the introduction, the most advanced work in AI-based plaques detection did not report segmentation measures. Our attention-UNet results show that explainability visual features can guide pathologists in improving their annotation quality. Furthermore, pathologists can use these results to improve/redefine the annotation boundaries, which could be fed back to the DL models for improved predictions.
Related to the high medical impact of our project, selecting mature and robust architectures is essential to create a solid baseline in this domain. Also, presenting results for both methodologies (UNet and Attention-UNet) is important to show the added value of explanations, as the increase of the quantitative results within the pattern analysis process. Eventually, in our research project, these patterns will be used for a morphology and topology pipeline toward a robust AD patient’s stratification proof.
Regarding the dataset and the experiments’ preparation, the WSIs used are rare and can only be obtained from specialized sources under strict regulations. Therefore, we would contribute by releasing all the samples used for training, validation, and testing, a major contribution to the state-of-the-art, as reported in the introduction. After a rigorous quality-check process, we selected the best 8 whole slide images (WSI) to extract 15725 patches (all uploaded to supplementary material) using an object-guided sampling method described in the data acquisition section. It is the first professional (hospital-delivered and checked) high-quality release of this kind, able to become a milestone in this field. Having such post-mortem human brain data required crossing several constraining regulations, duly respected in our case. Also, the code was released and will be publicly available after publication.
Another concern is the clinical value of the techniques presented. As specified in the abstract, AD patients’ stratification is a clear added value and final objective of this study. Also, targeting the AT8 biomarker is precisely related to its relevance for the clinical protocols. As specified in the first point of the discussion section, AT8 is majorly used in clinics because it helps highlight all structures in the tissue. In particular, in AD studies, AT8 will detect most tau pathologies needed to understand AD. To our knowledge, even if some tau PET tracers are available for in vivo imaging, they have poor spatial resolution and often lack specificity in comparison to microscopic histological measurements that still constitute the gold standard. Therefore, having such a pipeline to segment and differentiate tau aggregates will become essential to getting reliable diagnostic, prognostic, and treatment feedback.
Finally, related to quantitative analysis, attention maps are presented in Figures 2 and 3. Also, context effects (i.e., increasing the environment) are presented in Tables 2 and 3. We could also build the suggested summary table showing each factor’s contribution as all data is already presented. Results from VisioPharm commercial software can also be added in the final version as a comparison.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Although the authors explain the challenging of the problem and motivation of using an attention U-net. Although the method itself, e.g. attention U-net is not novel, it is a interesting application with well-motivated methodological choice. The performance is also not bad given how challenging the problem is and the amount of uncertainty of the expert-provided “ground-truth”. Authors also promise to release the data and code which could attract further research in the field. Therefore, I recommend to accept the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes an attention U-net method for semantic segmentation of plaque on WSI. This is a new application and the author is about to release the first set of plaque WSIs with expert annotation. This would be a good contribution to the society.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
9
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a deep learning based neurotic plaques segmentation method. In the first round of review, the reviewers’ concerns focus on the novelty of the method, clinical value, and rigor of the experiments. The author provided a detailed response to partially address such concerns.
In my opinion, the value of a new dataset and open-source code are beneficial as the early work in this subfield. However, my major concern is the validity of using such explainable AI algorithm given the relatively low Dice performance. Moreover, the attention based approach is not innovative, even is empirical without solid validation. Therefore, it is not clear whether this approach could actually bring positive clinical values for the current practice. Moreover, it seems the format of supplementary material does not follow this year’s guideline:
“Authors will be able to submit supplementary materials in the form of additional images, tables, and proof of equations at the time of paper submission in the MICCAI submission format. These materials must not exceed two pages and must NOT bear any identification markers. Authors should not submit text materials beyond figure and table captions, definition of variables in equations, or detailed proof of a theorem. Captions should not exceed 100 words. Additionally, authors may submit supplementary videos without any identification markers. All supplementary material must be self-contained and zipped into a single file. Only the following formats are allowed: avi, doc, docx, mp4, pdf, wmv. We encourage authors to submit videos using an MP4 codec such as DivX contained in an AVI. Also, please submit a README text file with each video specifying the exact codec used and a URL where the codec can be downloaded.” For these reasons, my recommendation towards rejection.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
13