Authors

Renato Hermoza, Gabriel Maicas, Jacinto C. Nascimento, Gustavo Carneiro

Abstract

Survival time prediction from medical images is important for treatment planning, where accurate estimations can improve healthcare quality. One issue affecting the training of survival models is censored data. Most of the current survival prediction approaches are based on Cox models that can deal with censored data, but their application scope is limited because they output a hazard function instead of a survival time. On the other hand, methods that predict survival time usually ignore censored data, resulting in an under-utilization of the training set. In this work, we propose a new training method that predicts survival time using all censored and uncensored data. We propose to treat censored data as samples with a lower-bound time to death and estimate pseudo labels to semi-supervise a censor-aware survival time regressor. We evaluate our method on pathology and x-ray images from the TCGA-GM and NLST datasets. Our results establish the state-of-the-art survival prediction accuracy on both datasets.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_21

SharedIt: https://rdcu.be/cVRU1

Link to the code repository

https://github.com/renato145/CASurv

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper presents a deep learning method for survival predictions based on images. It presents a methodology to leverage the use of censored data which is often ignores or sub optimally used.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
*The paper is very well written and it is easy to follow. *The authors proposed a pseudo labelling approach to leverage the use of censored data, which seems to be novel and very useful.
- The paper shows a good ablation study of the proposed model as well as a good comparison to other methods.
- The paper claims to achieve SOTA in both datasets used.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The methodology of the images used in the pathology case is not clear. The paper cites the work in https://www.pnas.org/doi/epdf/10.1073/pnas.1717139115 mentioning 1505 patches. However, in the cited work, there is no mention of these 1505 patches, and the only data available are the whole slide images.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors mention that the code will be shared upon acceptance. The raw data used is public; however, the preprocessing to obtain the patches in the pathology case is not described at all, making it challenging to fully reproduce the work.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

This a very well written paper with some very interesting results. I think it would benefit from a clear explanation on the preprocessing of the images e.g. how to go from whole slide images to patches instead of just citing the work where this was done. If there is space problem, this can even go in the supplemental material.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This is a very well written paper with some strong results in public datasets. It can improve by including more details about the data.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The paper addresses the challenge of survival time prediction with censored data, either by patients leaving the study or by patients living longer than the study ran. The authors suggest a pseudo-label approach to estimate labels for the missing data and establish different loss functions that can be used in this case. The proposed approach is evaluated on two datasets and compared to other methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

• The proposed solution is straightforward, comparatively simple, and can be combined with other technical approaches. • The proposed solution seems to be superior to existing solutions. • The problem that is addressed in this paper is under-investigated.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

• The authors use only two datasets for their evaluation. I would have expected more. • The authors reported only the best run when comparing their approach to others. • The authors used only image data, while the approach should be usable with other data types as well.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper is clearly written and most of the relevant information is within the paper. The authors also promised to publish their code and conducted their experiments on public data. The results should therefore be reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Thank you for the interesting paper. I enjoyed reading it. However, I found a few small points that could be improved for the revision of the paper:
1. Not for MICCAI but generally: Include more datasets in the evaluation.
2. Further clarification is needed for the combination of the losses: Are all losses weighted equally? If not, how were the weights estimated?
3. Table 1 lists results from the best runs. I found this slightly irritating as it seems to give the proposed approach an unfair advantage (Which should not be necessary, based on the table)
4. Please comment on how the configuration selected for Table 1 was chosen. It seems that this had been done after the ablation study, which then could lead to methodical overfitting.
5. In the discussion, the authors claim that l_ca_rnk and l_elr are usually beneficial or not worse than not using it. Judging from the results, the same is true for not using them. The corresponding comments should be rewritten.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is quite strong. The presented idea is straight forward and easy to combine with other approaches. A real problem is addressed and the solution is well evaluated. However, while the paper is solid research, I’ve missed the surprising element or the additional effort to make it an outstanding paper.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper proposes a semi-supervised learning method for patient survival time regression with censorsed and uncensored pathology and x-ray images based on the lower-bound time to death and peudo labels. The proposed method is evaluated on two public datasets and achieve improved performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The introduced ELR loss and censor-aware ranking loss shows promissing results and could be adopted in related tasks.
2. Rich ablative studies are conducted and show detailed performance impacts of th e proposed modules.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. For the CA-MSE loss, why the authors adopt MSE loss instead of MAE loss considering the evaluation metric is based on MAE?
2. It would be better to conduct more ablative studies on the impact of the amount of noisy labels on the proposed method.
3. How to overcome the influence brought by the difference of proportions of censored/uncensored cases in different datasets which causes different performances in this paper?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors declare the code will be publically available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

Please refer to the previous questions.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, I think this is an interesting paper. The proposed method leverage censored data for survival prediction and show improved results. Please be able to address my concerns.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Somewhat Confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper proposes a novel semi-supervised learning method that utilizes sensor data for survival prediction from medical images by introducing a pseudo labeling approach. Extensive experiments on two datasets validate the effectiveness of the proposed method. The proposed method is simple and effective. More clarifications about the technical details would help understand the paper better. I suggest the authors consider the reviewers’ suggestions when preparing the final version.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

1

Author Feedback

To improve the clarity of the paper we will update Table 1 to include the configuration details of our method. Also, reviewer 1 mentions some concerns about the details and preprocessing steps for the dataset from [13]. We would like to clarify that [13] has published the preprocessed dataset; in their paper they show the number whole-slide images (1061) and number of unique patients (769), we just added the information about the number of patches from their files. For the second dataset (NLST), we will include the preprocessing steps together with the code to reproduce the paper.

back to top

Censor-aware Semi-supervised Learning for Survival Time Prediction from Medical Images