Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Qingqing Zhu, Tejas Sudharshan Mathai, Pritam Mukherjee, Yifan Peng, Ronald M. Summers, Zhiyong Lu

Abstract

Despite the reduction in turn-around times in radiology reporting with the use of speech recognition software, persistent communication errors can significantly impact the interpretation of radiology reports. Pre-filling a radiology report holds promise in mitigating reporting errors, and despite multiple efforts in literature to generate comprehensive medical reports, there lacks approaches that exploit the longitudinal nature of patient visit records in the MIMIC-CXR dataset. To address this gap, we propose to use longitudinal multi-modal data, i.e., previous patient visit CXR, current visit CXR, and the previous visit report, to pre-fill the “findings” section of the patient’s current visit. We first gathered the longitudinal visit information for 26,625 patients from the MIMIC-CXR dataset, and created a new dataset called \textit{Longitudinal-MIMIC}. With this new dataset, a transformer-based model was trained to capture the multi-modal longitudinal information from patient visit records (CXR images + reports) via a cross-attention-based multi-modal fusion module and a hierarchical memory-driven decoder. In contrast to previous works that only uses current visit data as input to train a model, our work exploits the longitudinal information available to pre-fill the “findings” section of radiology reports. Experiments show that our approach outperforms several recent approaches by ≥3% on F1 score, and ≥2% for BLEU-4, METEOR and ROUGE-L respectively.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43904-9_19

SharedIt: https://rdcu.be/dnwGW

Link to the code repository

https://github.com/CelestialShine/Longitudinal-Chest-X-Ray

Link to the dataset(s)

N/A

Reviews

Review #5

  • Please describe the contribution of the paper

    The authors proposed an approach that uses longitudinal multimodal data, including previous patient visit chest X-rays, previous patient visit reports, and current patient visit chest X-rays, to pre-fill the findings for the current patient visit chest X-ray. They trained a transformer-based model that incorporates a cross-attention-based multi-modal fusion module and a hierarchical memory-driven decoder. To evaluate the approach, the authors created a new dataset called ‘Longitudinal MIMIC’.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed approach utilizes Chest X-rays and reports for pre-filling radiology reports.
    2. The approach outperforms baseline methods.
    3. The paper introduces a new dataset called ‘Longitudinal MIMIC’ to facilitate research utilizing multimodal data from longitudinal patient visits.
    4. Clinical relevance: Pre-filling radiology reports has the potential to assist radiologists in their work.
    5. The paper proposes a transformer-based model that utilizes cross-attention-based multi-modal fusion module and a hierarchical memory-driven decoder.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The paper lacks detailed descriptions about the implementation of the baseline methods.
    2. The proposed approach only slightly outperforms the best performing baseline, and no significance tests was conducted to evaluate the differences.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The paper seems to be reproducible since it is clearly written, and the authors have shown the implementation details. Also, the authors intend to release the code and dataset used for the evaluation.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. The evaluation in the paper is reasonable, and the authors acknowledge a weakness in their proposed method. However, the paper lacks implementation details for the baseline methods, which could be helpful to include.
    2. The authors contribute to an important problem in report generation, and their created dataset (which they intend to make publicly available) can inspire further research in this area.
    3. There is a grammatical error in the first line of the ‘Encoder’ subsection under the ‘Method’ section. The authors wrote ‘Our model uses a Image Encoder’ instead of ‘Our model uses an Image Encoder’.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Utilizing longitudinal multimodal data to pre-fill radiology reports has clinical relevance and potential practical implications. Additionally, the paper’s approach presents technical novelty and reproducibility.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper
    1. A new longitudinal MIMIC dataset for chest X-ray image report generation is introduced.
    2. A novel model is proposed for the longitudinal chest x-ray report pre-filling.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. This paper introduces a new dataset and task setting for chest x-ray report generation. (1) The new setting takes the historical chest x-ray reports and the current chest x-ray image as input and generate the report for the current chest x-ray image. (2) The new dataset is built upon the MIXMIC-CXR benchmark.
    2. This paper introduces a novel pipeline for the new dataset and setting.
    3. The presentation is clear.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. There are only a few samples in the validation and test splits.
    2. It is recommended that the authors conduct more empirical analysis in the future.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Implementation details are provided and thus the reproducibility looks good.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. There are only a few samples in the validation and test splits. The authors may consider adding more samples in the future.
    2. It is suggested that the authors conduct more empirical analysis. For example: what’s the models’ performance for cases with 2, 3, 4, 5 historical reports respectively?
    3. Missing reference: please also consider to cite [1][2].

    [1] Jing, Baoyu, Zeya Wang, and Eric Xing. “Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-ray Reports.” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. [2] Li, Yuan, et al. “Hybrid retrieval-generation reinforced agent for medical image report generation.” Advances in neural information processing systems 31 (2018).

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper introduces a new dataset and new setting for chest x-ray report generation.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    Proposes a method to prefill the finding section of a radiology report based on the current image, and the set of previous images and reports. A longitudinal data set is generated from the MIMIC-CXR dataset for those cases where multiple images from the same patient are available. Ablation studies of using the previous report only, image only, or dropping the cross attention module are explored.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Derives and releases a new dataset from MIMIC-CXR Addresses the sequence of exams, rather than a single exam, which is relevant to medical practice Ablation studies showing performance with and without the use of the longitudinal data

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    It is stated that the implementation of the compared methods is detailed in the supplementary material, but I am unable to find it.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Dataset and code will be made publicly available.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    P3. “Patients with >2 2 visits” should be “Patients with > 2 visits”

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The fact that a patient’s clinical journey is often a sequence of examinations is frequently not investigated. This investigation of longitudinal data is a good contribution. In addition, an new dataset is constructed and released. A few points were lost because I did not find the implementation of the other methods.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper proposes a method for prefilling radiology reports by utilizing longitudinal multimodal data, including previous patient visit chest X-rays and reports. The authors introduce a new dataset called ‘Longitudinal MIMIC’ and develop a transformer-based model for the prefiling task. The strengths of the paper include the novel dataset, addressing the sequence of exams in medical practice, and conducting ablation studies to analyze the impact of different inputs and modules. The lack of implementation details for the compared methods and the limited sample size in the validation and test splits are weaknesses pointed out by the reviewers. Overall, the paper is recommended for acceptance, as it presents a solid contribution in investigating longitudinal data for chest X-ray report generation, and the reviewers acknowledge the clinical relevance and potential practical implications of the proposed approach.



Author Feedback

N/A



back to top