List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Louis Blankemeier, Isabel Gallegos, Juan Manuel Zambrano Chavez, David Maron, Alexander Sandhu, Fatima Rodriguez, Daniel Rubin, Bhavik Patel, Marc Willis, Robert Boutin, Akshay S. Chaudhari
Abstract
Opportunistic computed tomography (CT) analysis is a paradigm where CT scans that have already been acquired for routine clinical questions are reanalyzed for disease prognostication, typically aided by machine learning. While such techniques for opportunistic use of abdominal CT scans have been implemented for assessing the risk of a handful of individual disorders, their prognostic power in simultaneously assessing multiple chronic disorders has not yet been evaluated. In this retrospective study of 9,154 patients, we demonstrate that we can effectively assess 5-year incidence of chronic kidney disease (CKD), diabetes mellitus (DM), hypertension (HT), ischemic heart disease (IHD), and osteoporosis (OST) using single already-acquired abdominal CT scans. We demonstrate that a shared multi-planar CT input, consisting of an axial CT slice occurring at the L3 vertebral level, as well as carefully selected sagittal and coronal slices, enables accurate future disease incidence prediction. Furthermore, we demonstrate that casting this shared CT input into a multi-task approach is particularly valuable in the low-label regime. With just 10\% of labels for our diseases of interest, we recover nearly 99% of fully supervised AUROC performance, representing an improvement over single-task learning.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_30
SharedIt: https://rdcu.be/cVRVi
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors present a method to predict the incidence of a set of chronic diseases in the five years post abdominal CT acquisition. CT input is sliced in an axial, a coronal and a sagittal plane using landmarks to homogenise them. Outcome data is obtained from the medical records by analysing the ICD codes. A neural network (ResNet-18) is used to predict outcomes, either one at a time or all together. Results are presented for the network having only one slice, having multiple slices or having multiple slices and multi-task learning. Results with respect to the number of training samples are also presented. All measured as AUC of ROC curves. The dataset used for training and evaluation is large (>14,000 CT scans from >9,000 patients).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The strengths of the paper are related to the problem statement. Once we have a CT, we have a clear view of the status of the patient and a lot of information can be extracted. The method presented, planar reformats guided by anatomy, and outcomes obtained from ICD codes are an ‘easy’ way to obtain both the input and the output of the system.
The evaluation with respect to the percentage of training points is interesting and valuable. The more training points, the better performance, as one could imagine. - Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weak point is the doubts that the results pose. If we look at table 2, there is almost the same result for ischemic heart disease (IHD) when using the axial slice, that does not contain information about the heart, as when using the coronal or the sagittal, that have at least the lower part of the heart on them. Similarly, the coronal slice does not have information on the vertebrae, but osteoporosis is almost as well predicted as with the sagittal, that has most of the column or the coronal. Multi-slice representation has almost as good performance as the others. These data suggests that the network is focusing on information that is not of relevance for the task at hand.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper will only be reproducible if the database is made publicly available. Otherwise, it is straightforward to implement.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please assure that scans from the same cases are not used for training and testing.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The problem is interesting and the training and validation are well performed, even though doubts about the results arise (see above).
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper designed a multi-task low-label learning method for opportunistic incidence prediction of multiple chronic diseases from abdominal CT imaging. A multi-planar 2D CT processing method is designed to extract useful information for five diseases, which reduces the dimensionality of the volumetric 3D data and outperforms 2D single-plane approaches. The proposed method achieve outperformance in 5-year incidence prediction of CKD, DM, HT, IHD and OST.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A multi-planar 2D CT processing method is designed to extract useful information for five diseases, which reduces the dimensionality of the volumetric 3D data and outperforms 2D single-plane approaches.
- A multi-task low-label learning method is designed for opportunistic incidence prediction of multiple chronic diseases from abdominal CT imaging, and achieves outperformance in 5-year incidence prediction of CKD, DM, HT, IHD and OST.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The segmentation performance of aorta is slightly unsatisfactory. Whether the segmentation results would affect the prediction accuracy? If that’s the case, other SOTA segmentation methods can be consider to perform segmentation tasks.
- The reason of concatenating three slices laterally did not explain clearly, dose concatenating three slices in channel dimension can also work?
- The training process using sparse labels did not explain clearly, more details should be supplement in the article.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code of this work was not provided and the detail of proposed network have not been explained clearly. The reproducibility is slightly worse.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This paper designed a multi-task low-label learning method for opportunistic incidence prediction of multiple chronic diseases from abdominal CT imaging. A multi-planar 2D CT processing method is designed to extract useful information for five diseases, which reduces the dimensionality of the volumetric 3D data and outperforms 2D single-plane approaches. The proposed method achieve outperformance in 5-year incidence prediction of CKD, DM, HT, IHD and OST.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This study provided a meaningful approach for opportunistic incidence prediction, and the multi-planar 2D CT processing method can provide reference for other researches. If the details can be added after major revision, this is a meaningful work. Thus, I suggest receiving it after major revision.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
Development of a multi-task DL model, levarging on reanalysed CT scans for a 5-year multiple chronic disease detection tool.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The concept is an interesting one, trying to utilise all information available on an image for predcitions - mimicing a real world situation - as most DL models focus on a particular singlar disease/prediction. Therefore the idea of having a source and target disease is interesting and novel for this type of application when used together with a multi-planar apparoach.
I like that you include a statistical test into your work - always appreciate seeing this in comparative anlaysis.
- The paper is nicely laid out and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- I don’t see the clincial aspect/translation - does the source and target combinations/disease affinities make sense from a clinical perspective? (apologies if missed)
- I feel the dice score for the segmentation is quite low? Any comments about this?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I think the details wrt the methodology is there but I think the actual data utilised after slice selections is an important aspect to share as often the methods are clear but when one tries to replicate the data ithe outcomes may create vastly different results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I enjoyed reviewing your paper, well done on your work. I have some points to kindly raise:
- How do you discard the predictions of the secondary disease? Not clear to me.
- The optimal parameters chosen - how did you conclude this, trial and error? or the strongest associations/lowest losses drove these choices?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I feel the concept and work related to multi-task learning and multiple disease prediction is the way forward for DL models in healthcare, and I found the paper enjoyable to read. There are some aspects that might require some working - perhaps to answer some of the questions I have and any other comments from other reviewers.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper is nicely laid out and easy to follow.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Author Feedback
We appreciate Reviewer 1’s critique that “data suggests that the network is focusing on information that is not of relevance for the task at hand”, as well as Reviewer 2’s question - “does the source and target combinations/disease affinities make sense from a clinical perspective?” To interpret our results from a clinical perspective, we chose to predict cardiometabolic disorders that share several common risk factors and have co-incidence of disease. Specifically, imaging biomarkers, such as abdominal calcifications and the extent of muscle and fat, and hepatic steatosis (fat infiltration in the liver) are surrogates for other well established risk factors, such as body mass index, blood glucose, and blood pressure. Furthermore, mechanisms of muscle, fat, and bone are highly interconnected, and these tissues are thus increasingly considered a single unit. So, even when bone is not present in the CT view, it is not surprising that we can predict osteoporosis as it is recently thought to be a disease of the bone-muscle-adipose tissue triad [1, 2]. Regarding Reviewer 1’s comment - “please assure that scans from the same cases are not used for training and testing”, we carefully ensured that scans belonging to the same patient were in a single split only. Regarding low performance on aorta segmentation (R2 and R3): it is challenging to segment the aorta in the L3 axial plane. However, for our work, the segmentation was only a surrogate for choosing an optimal coronal slice for our multi-planar classification. Despite the low segmentation performance, we achieved a mean absolute error of 0.46 mm for the chosen coronal slice, which we concluded was sufficient for our downstream task. Regarding Reviewer 2’s note - “the reason of concatenating three slices laterally did not explain clearly”. Our initial intuition here was that concatenating in the channel dimension would perform suboptimally as there is little spatial correlation between features in these various planes. In future work, we will investigate whether slice-concatenation or other techniques may be a better way to combine the multiple planes. We appreciate Reviewer 3’s question - “how do you discard the predictions of the secondary disease?”. Discarding predictions for the secondary disease can be understood with an analogy to pretraining. Predictions for the pretraining task are not utilized, and the pretraining strategy simply aids with downstream task training. Since each primary disease has a secondary ‘pretext’ disease, we can still predict progression to all chosen disorders. We appreciate Reviewer 3’s question - “the optimal parameters chosen - how did you conclude this, trial and error?” We selected hyperparameters that worked sufficiently on validation runs, relying on network architectures and training paradigms that have been validated previously. Our primary focus was determining whether there was sufficient information in our CT images to harness for multi-task learning and to predict disease progression with adequate accuracy. Future work will investigate optimal hyperparameters now that we are aware of the efficacy of the underlying approach. In accordance with the comments that we did not explain various aspects of our method sufficiently, we added explanations as pertinent.
[1] Ching-Ti Liu et al. Journal of Bone and Mineral Research, 2016 [2] Jean-Yves Reginster et al. Current Opinion in Clinical Nutrition and Metabolic Care, 2016