Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Taha Emre, Arunava Chakravarty, Antoine Rivail, Sophie Riedl, Ursula Schmidt-Erfurth, Hrvoje Bogunović

Abstract

Recent contrastive learning methods achieved state-of-the-art in low label regimes. However, the training requires large batch sizes and heavy augmentations to create multiple views of an image. With non-contrastive methods, the negatives are implicitly incorporated in the loss, allowing different images and modalities as pairs. Although the meta-information (i.e., age, sex) in medical imaging is abundant, the annotations are noisy and prone to class imbalance. In this work, we exploited already existing temporal information (different visits from a patient) in a longitudinal optical coherence tomography (OCT) dataset using temporally informed non-contrastive loss (TINC) without increasing complexity and need for negative pairs. Moreover, our novel pair-forming scheme can avoid heavy augmentations and implicitly incorporates the temporal information in the pairs. Finally, these representations learned from the pretraining are more successful in predicting disease progression where the temporal information is crucial for the downstream task. More specifically, our model outperforms existing models in predicting the risk of conversion within a time frame from intermediate age-related macular degeneration (AMD) to the late wet-AMD stage.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_60

SharedIt: https://rdcu.be/cVRss

Link to the code repository

https://github.com/EmreTaha/TINC

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    The authors presents a modification of the recent VICReg self-supervised method for longitudinal imaging data. The modification is (1) using pairs of images that are randomly sampled from different time point of a patient, and (2) changing the first loss, which corresponds to the invariance of related presentations, to depend on the time between the two images, such that for smaller time difference will lead to stronger enforcement of similarity between the two representations.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed method is a simple yet interesting extension/adaptation of VICReg to longitudinal medical images. To the best of my knowledge the method is novel, and I’m not familiar with other demonstrations of the VICReg method to medical images.
    2. The experiments and results are satisfactory, in terms of showing that the proposed method can have superior performance over other self-supervised non-contrastive methods: including vanilla VICReg, and simpler adaptations of it (which can be viewed as ablation study).
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The paper is not self-contained. I had to go to the original VICReg paper to understand the technical details of the method, and the intuition behind it. Also, there are additional unclear issues, which I detailed in Q8 below.
    2. The evaluation is done on a single dataset.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Apart of few technical issues that I detailed below, and after reading the original VICReg paper, I think that the method is clear and the results can be reproduced.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
    1. Th presentation of VICReg is incomplete and lacking a clear motivation for the three losses. I had to read the original VICReg paper in order to understand this paper. Specifically std(z, epsilon) and “d” are not defined. I suggest to provide a short introduction for the three loss terms, for example: the first loss is for reducing the distance between related presentations (e.g. invariance to data transformations), second loss is for increasing the variance of the different elements in the presentation to prevent norm collapse, and third loss is for de-correlating the different elements in the representation, to reduce redundancy.
    2. In the original VICReg method, the batch of n patients is composed of n images, which undergo two separate transformation, t1 and t2. The following sentence was unclear to me: “Given a batch of n patients with multiple visits, let visits v1 and v2 be the n tuple of time points randomly sampled from each available patients’ visit dates within a certain time interval.” Does this mean that for each patient we randomly sample a pair of images from two different points? In other words, does tuple=pair, and each sample contains a single image?
    3. In Table 1, the vanilla VICReg had “representational collapse”. The authors mention that in medical images a larger area-ratio crop should be taken. It would be interesting to see the results of VICReg with larger area-ratio crop but without the two visits input. Was there still a representation collapse? I suggest the authors add this result, to demonstrate the contribution of two-visits input.
    4. To assess whether the difference in the performance (AUROC, PRAUC) is statistically significant, I suggest to add p-values based on bootstrapping.
    5. I did not understand this sentence “This can be explained by the fact that in Barlow Twins, the fine-tuning reached the peak validation score within 10 epochs, same as the number of linear training epochs.” Did you try increasing the number of epochs, maybe the fine-tuning takes longer to converge to a better model?
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The novel and interesting extension of the recent VICReg method to longitudinal medical images and the demonstration of its utility.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This is an interesting paper. The authors propose a modified loss function which they call TINC (Temporally Informed Non-Contrastive Loss) to be used with VICReg to predict whether an eye is going to convert to wet-AMD within 6 months time-frame.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Modified the VICReg’s invariance term by constraining it with the normalized absolute time difference between the input images

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    discussion around misclassifications and possible ways to improve missing

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Not reproducible

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    This is an interesting paper. The authors propose a modified loss function which they call TINC (Temporally Informed Non-Contrastive Loss) to be used with VICReg to predict whether an eye is going to convert to wet-AMD within 6 months time-frame

    Comments are below: Include and discuss in a section to explain misclassifications. Are there images which were classified correctly by other methods and not by the proposed method. If yes, why? And also vice versa, why the proposed methods works better based on images as example.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    novelty and few other weaknesses as given above

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    2

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #5

  • Please describe the contribution of the paper

    This paper introduces a new loss function (TINC) to exploit existing temporal information in longitudinal context of OCT data. The proposed model outperforms the compared methods in terms of AUROC and PRAUC.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. This paper addresses a interesting clinical question, i.e., Disease progression in the longitudinal contexts. This problem is not only existed in Eye, but also other organs.

    2. Superior performance has been achieved when comparing with some baselines.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. Only one dataset has been evaluated in this paper, which may limit its generability evaluation. If the authors only has one dataset available, cross-validation would be more appreciated.

    2. The written can be improved. The author states that they have exploited the meta-information. However, what kind of meta information has included, and how they are included is not clear.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The authors state that they will release training code, pre-trained model and evaluation code. I believe the reproducibility of this paper is good.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    1. I would suggest a more comprehensive study, e.g., more datasets if possible, cross-validation on one dataset. Consider the time is limited during rebuttal. I will not say this is a mandatory requirement, but the authors may consider it.

    2. Improve the writing to make the readers easier to follow. As the weakness 2, the readers may be confused about that. So other parts including: Is the evaluation metrics (AUROC, PRAUC) computed based on volumes level or eye-level? Without this information, it is not friendly to readers who is not a expert on OCT. 2)

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    A new loss function to address a clinical-interested task. The authors also show promising results

  • Number of papers in your stack

    6

  • What is the ranking of this paper in your review stack?

    3

  • Reviewer confidence

    Somewhat Confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper addresses an interesting clinical question, i.e., Disease progression in the longitudinal contexts. The author introduces a novel loss function, which calls Temporally Informed Non-Contrastive Loss (TINC), to exploit existing temporal information in longitudinal context of OCT data. The proposed model outperforms the compared methods in terms of AUROC and PRAUC.

    The method is novel, the experimental results are promising, and the paper is well written. I recommend accepting this submission. The authors should address the detailed comments from the reviewers in the camera-ready manuscript.

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    2



Author Feedback

N/A



back to top