List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Li-Hsin Cheng, Xiaowu Sun, Rob J. van der Geest
Abstract
In this work, we aimed to tackle the challenge of fusing information from multiple echocardiographic views, mimicking cardiologists making diagnoses with an integrative approach. For this purpose, we used the available information provided in the CAMUS dataset to experiment combining 2D complementary views to derive 3D information of left ventricular (LV) volume. We proposed intra-subject and inter-subject volume contrastive losses with varying margin to encode heterogeneous input views to a shared view-invariant volume-relevant feature space, where feature fusion can be facilitated. The results demonstrated that the proposed contrastive losses successfully improved the integration of complementary information from the input views, achieving significantly better volume predictive performance (MAE: 10.96 ml, RMSE: 14.75 ml, R2: 0.88) than that of the late-fusion baseline without contrastive losses (MAE: 13.17 ml, RMSE: 17.91 ml, R2: 0.83). Code available at: https://github.com/LishinC/VCN.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_33
SharedIt: https://rdcu.be/cVRvY
Link to the code repository
https://github.com/LishinC/VCN
Link to the dataset(s)
https://www.creatis.insa-lyon.fr/Challenge/camus/index.html
Reviews
Review #1
- Please describe the contribution of the paper
In this work they fuse two Echo views (A2C and A4C) to estimate the LV volume. The main contributions are the intra-subject contrastive loss and inter-subject contrastive loss which are proven to improve the performance of the model.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Well-written; clear story-line, and reliable results. Rigorous experiments with ablations studis which independently assess the impact of each component and idea. To the extent of my knowledge, the ideas around intra-subject contrastive loss and inter-subject contrastive loss are novel. Experiments are conducted on public datasets and source code will be released, hence, a more prominant impact is expected.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The work seems to have failed to cite other LV volume estimation works, and they are a plenty. In turn, they have not compared their method with such methods. Yet, to be fair, they have compared their method with the ablated versions of their own model which highlgihts the impact of each component.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Source code will be included; all good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Question: In the “Inter-subject volume contrstive loss”, it is stated that “we find a random subject i’from the batch in each training iteration which has a similar volume at phase p to form a positive pair”. How do you find this random subject from the batch? How do you guarantee that such sample is available in the batch for the given positive sample?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea, its explanation, and execution of experiments are complete and close to perfect. The paper would add value to the community.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
- This paper proposed a contrastive learning method for view fusion to estimate LV volume in echocardiographic examination. Results on CAMUS dataset illustrate the effectiveness of the proposed approach.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper is well-written and easy to follow. The figure is illustrative.
- The idea of contrastive learning makes sense especially considering the physical characteristic of ED/ES in the two views of A2C/A4C.
- The ablation experiments are good to illustrate the effectiveness of each proposed loss function.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The experimental results are not solid. Results are performed on only one dataset without comparison to the existing methods. I’m not familiar with the targeted task but I wonder if there’s any existing work on this task for the purpose of comparison.
- The technical implementation of the contrastive loss is trivial. Direct maximizing and minimizing the eculidian distance is not optimal and barely used. In this scenario, I think triplet loss or regular contrastive loss (in self-supervised learning) are better alternatives to try and compare with simple distance-based losses.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good if code is released.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please see weaknesses.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea is interesting and the paper is well-written. However, the experimental drawbacks and the technical implementation remains problematic.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The rebuttal addressed my concern. I recommend acceptance.
Review #3
- Please describe the contribution of the paper
This work presents a volume contrastive network used to derive the left ventricle volume (a 3D measurement) from Apical-2-Chamber and Apical-4-Chamber 2D echocardiographic views. The main contributions are tackling the 2D fusion information to generate 3D measuements and introducing intra and inter volume contrastive losses to improve this fusion
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors propose a novel method that uses a volume contrastive network for 2D information fusion. The network itself is not novel but the application is. Furthermore, the authors propose using intra and inter subject constrastive losses by maximixing/minimizing positive/negative distances between pairs, which shows a deep understanding of the problem and a novel method to improve the solution. The evaluation is very detailed, with ablation studies showing the contribution of each of the features added to the base architecture and statistical analysis shoing the statistical significance of the improvement.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors claim to be the first to apply volume contrastive networks to solve this problem but the evaluation lacks other methods in the literature that have been used to combine the views. The only comparison provided is the ablation study, which shows very convincing results. It would also be helpful to show some clinical correlation rather than just the MSE. For example, how does this improvement in the volume estimation help diagnosis, is this improvement relevant to the final task for which the volume estimation is used?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors have provided enough detail for the paper to be reproducible and have used a public dataset
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This is a well written manuscript that clearly describes the problem at hand and the solution presented as well as the logic behind it. It shows a good understanding of the problem and presents some simple additions to a known architecture (contrastive networks) for a novel application. It would be helpful to do a more in depth comparison of other state of the art methods to solve the LV volume from the 2 views the authors use and show the actual clinical impact of the error reduction the authors are showing. Nonetheless the results from the ablation study show that each part that has been introduced contributes to the performance of the network and it would be useful to the community to have this published.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the paper is enough to be accepted and the ablation study evaluation provides convincing results that show that the proposed architecture and loss does indeed improve the model performance.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper was received positively in terms of method and presentation. One major issue remains that the results are performed without comparison to existing methods. I think this aspect should be addressed by the authors and discussed in their work. Otherwise it is difficult for the reader to understand the performance results of the approach.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Author Feedback
We thank the reviewers for the valuable feedback.
Q1. Comparison with existing methods [R1-R3]: Firstly, the major goal of our work is to investigate how to better incorporate information from multiple views to benefit automated diagnostic tasks. LV volume, a 3D information, can only be accurately predicted by incorporating the complementary 2D views, and happens to be available in the CAMUS dataset. Therefore, we use volume prediction accuracy as a way to assess the performance of view fusion. However, there is no prior literature reporting on regression based volume prediction methods focusing on view fusion in echocardiographic data. Early- and Late-fusion methods [13-15] have been proposed to fuse information from multiple views for other tasks in other imaging modalities, and therefore we compared our methods with those two classic multi-view models. The results are reported in Table 1, showing our proposed VCN outperforming both classic approaches. Based on reviewers’ comments, in the revised version of the paper we have included a comparison of our VCN to a recent method [15], which uses Transformer to aid feature fusion. We trained their proposed model on CAMUS, and the performance (MAE: 12.57 ml, RMSE: 18.67 ml, R2: 0.81) was significantly lower than that of our proposed VCN (MAE: 10.96 ml, RMSE: 14.75 ml, R2: 0.88). In summary, regarding our major goal of improving view fusion, the results demonstrated that VCN fuses the information better than other methods. Secondly, as the VCN predicts LV volume, it can be compared to other LV volume quantification methods as well. Only one previous work [16] reports a regression based method for LV volume prediction from 2D echocardiograms. However, it uses the EchoNet dataset [16] which only includes a single A4C view, and due to the unavailable pixel spacing information, the performance cannot be compared. On the other hand, the majority of methods deriving LV volume are based on segmentation. However, we argue that segmentation and regression are two distinct approaches, each with different purposes and strengths. Therefore, the volume prediction performance should not be directly compared. The regression based framework can be expanded to fuse more views, given segmentation ground truth or not. It can also learn to predict a target not relying on segmentation (such as diagnosis), or a more accurate target derived from other modalities (such as volume derived from short-axis MRI). Therefore, exploring volume prediction via a regression approach is also an added value of this work.
Q2. Technical implementation [R2]: We were not directly maximizing and minimizing the distances between the embeddings. Instead, the losses we developed stem from the triplet loss, exactly as R2 pointed out in the question, and as cited [5,7] (p.4 line 15). Under the same spirit as the triplet loss, the variation we developed encourages the distance of the positive pairs to be at least smaller than that of the negative pairs by a certain margin (p. 4 line 10), and the losses are clamped at zero to avoid pushing the negative pairs infinitely apart unnecessarily (p.4 line 19). We made two adaptations to the triplet loss to tailor it for our problem. First, the original triplet loss was constraint to contrast among an anchor, a positive, and a negative sample. We modified the loss for an efficient contrast between 4 samples in one go. Secondly, we proposed a varying margin based on volume difference, which we demonstrated to result in a more delicate disentanglement of embeddings.
Additional References: [13] DOI: 10.1007/978-3-030-32239-7_37 [14] DOI: 10.1109/TMI.2019.2945514 [15] DOI: 10.1007/978-3-030-87199-4_10 [16] Ouyang, D., et al. “Echonet-dynamic: a large new cardiac motion video data resource for medical machine learning.” NeurIPS2019 ML4H Workshop.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have responded to the reviewers concerns in a detailed way, especially w.r.t. differences to SOTA/comparison to SOTA. All seem to agree that the paper has the merit to be accepted.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have addressed main concerns in the rebuttal satisfactorily and reviewers are unanimous about acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper seems interesting; R2 raised their score after reading the rebuttal. My only concern is that the literature review especially regarding the used task is very limited and that there would be a lot of space left to acknowledge a lot of previous work that has been done to provide automated means for the cardiac ultrasound.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
9