List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Jinho Joo, Gihun Joo, Yeji Kim, Moo-Nyun Jin, Junbeom Park, Hyeonseung Im
Abstract
Recent advances in wearable healthcare devices such as smartwatches allow us to monitor and manage our health condition more actively, for example, by measuring our electrocardiogram (ECG) and predicting cardiovascular diseases (CVDs) such as atrial fibrillation in real-time. Nevertheless, most smart devices can only measure single-lead signals, such as Lead I, while multichannel ECGs, such as twelve-lead signals, are necessary to identify more intricate CVDs such as left and right bundle branch blocks. In this paper, to address this problem, we propose a novel generative adversarial network (GAN) that can faithfully reconstruct 12-lead ECG signals from single-lead signals, which consists of two generators and one 1D U-Net discriminator. Experimental results show that it outperforms other representative generative models. Moreover, we also validate our method’s ability to effectively reconstruct CVD-related characteristics by evaluating reconstructed ECGs with a highly accurate 12-lead ECG-based prediction model and three cardiologists.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_18
SharedIt: https://rdcu.be/dnwLt
Link to the code repository
https://github.com/knu-plml/ecg-recon
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper is proposed to use single-lead ECG to synthesize multi-lead ECG, so as to provide more information (for CVD, for example).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1- A considerable reconstruction performances and new lead synthesis performances. 2- A dual-generator approach, and I consider the approximation requirement of hidden states of these two generator is sound and interesting. 3- The authors gave a good introduction of background in the Introduction Section.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1- The statement in the abstract, “for the first time”. To my best knowledge, it seems that this is not the first time to use single lead ECG to synthesize multi-lead ECG. For example, the electrocardio panorama [6]. Besides, maybe the approach in [6] should be compared (but it is not compared now; you can give the results in the rebuttal, and I may raise the scores).
2- The evaluation: The authors proved the superiority of the proposed approach by MMD. It is encouraging to give some visualization. You can give it in the appendix, or supplementary materials, or additional pages in the final version, or even a future version in arxiv.
3- the framework illustration is quite confusing. What is the meaning of single-lead ECG * 12? 12-lead label ECG? The “label ECG” seems to be a grammatical mistake. Since there are some space (left and right) in the illustration, you should carefully give the details and make the illustration intuitive and clear.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good.
The authors said:
For all code related to this work that you have made available or will release if this work is accepted, check if you include: YES (for Specification of dependencies, Training code, Evaluation code, (Pre-)trained model(s)).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Authors provided a good paper. It could be better if （see weakness）:
-
compared with [6].
-
provided some visualization.
-
showed the codes.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The background and the method.
I consider the dual-generator approach can make the synthesized results more trusty.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #5
- Please describe the contribution of the paper
This paper proposes a GAN for reconstructing 12-lead ECG signals from single-lead ECG signals. The model consists of two generators (one inference generator and one label generator) and one 1D U-Net discriminator. Experiments on an internal dataset suggest that the proposed model results in higher quality reconstruction, as well as improved cardiovascular disease classification compared to existing methods. In addition, the reconstructed ECGs are evaluated by three cardiologists and concordance rates are reported.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper tackles an important and largely unaddressed problem — reconstructing 12-lead ECG from single-lead ECG.
- The model is a combination of existing methods and relatively new.
- Experiments are extensive, including comparison to existing generative models and evaluation by three cardiologists.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- There lacks details on cardiologists’ analyses. Did the cardiologists review the reconstructed 12-lead ECGs? If so, how many cases were reviewed? How many cases were good reconstructions? Similarly, it’s not clear to me how the concordance rates in Table 5 are computed.
- No confidence interval or standard deviation of the results are provided. Also, the authors say that “For all metrics, EKGAN significantly outperforms other models…” in Section 3.2, but no statistical analysis is provided to support this.
- Figure 2 is not straightforward to understand by audience who cannot read ECGs. It would be great to indicate areas where other methods reconstruct poorly, but EKGAN reconstructs well. And vice versa.
- In Table 4, the results of the first baseline (results reported in reference [18]) are not from the same dataset as EKGAN (please correct me if my understanding is wrong). This may not be a fair comparison. Why not re-run the model in [18] on the same dataset and report the results?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Dataset is not publicly available. No information about code release in the manuscript.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Please provide more details on how cardiologists analyzed the ECGs. See my comments above.
- Please provide confidence interval or standard deviation in results. Also, please only use “significantly outperforms” if it’s confirmed by statistical analyses.
- In Figure 2, please highlight areas where other methods reconstruct poorly, but EKGAN reconstructs well. And vice versa. Also, it would be great to show a few more reconstructed ECGs (can be in Supplement) to confirm that good reconstruction is general and not cherry-picked.
- In Table 4, please provide results of the first baseline (reference [18]) using the same dataset as EKGAN.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper addresses an important and open problem. The methods are relatively novel. Experimental design is sound and results are strong.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The study introduces a GAN-based method to simultaneously (re-)construct all 12 lead channels in a standard ECG from measuring only single Lead 1. The introduced method is tested by means of comparing synthesized signals to real ones, of comparing synthesized signals to those generated by other existing GAN-based methods, and by demonstrating the feasibility of using synthesized signals for use in predictive CVD models. The introduced GAN outperforms other synthetic data generation models and yields predictive CVD results comparable to those obtained from using real ECG data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The strength of the paper is the fact that for the first time it introduces a method to simultaneously construct all 12 channel signals from measuring only single Lead I whereas the synthesized data is of a quality that allows to run advanced predictive models on it which so far have only been shown to be applicable for 12 channel ECG data. This has major applications for the functionality of wearable monitoring systems capable of measuring only one single Lead (p.ex. the Apple Watch) as it enables those systems to run advanced 12 channel predictive diagnostic and prognostic deep learning models which rely on 12 channel ECG data. Furthermore, the introduced EKGAN architecture in using the additional label generator is to be seen as novel.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The study has no obvious weaknesses. Possible ways to strengthen its impact are, as also indicated by the authors, additional work that tests the usability of synthesized data for other CVD prediction models.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reproducibility is low. The authors neither provide the source code for their novel EKGAN model nor seems the dataset they used for validation and testing to be publicly available. It is highly desirable for this information to be made publicly available should the paper be published.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The work introduces a method for synthesizing ECG data that is highly relevant to the field of wearable health monitoring using commercial single-lead ECG sensors. It would be highly interesting to investigate how other time-series modalities (such as for example EEG data in the epilepsy space) could be reconstructed through this method. The findings, if reproducible for EEG data, would be highly relevant to enabling seizure monitoring (detection and forecasting) using advanced seizure detection and prediction models (such as for example SeizureNet, ChronoNet) that rely on multi-lead monitoring input data traditionally produced through muilti-channel sensor setups (caps in the case of epilepsy) which, using EKGAN, could be reduced to single, or few-lead sensors (such as EpiMinder for example).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces a novel model architecture, tests its validity and performance on a representatively large dataset and shows applicability of the synthesized data to run advanced 12 Lead predictive CVD models from measuring only 1 single Lead. The experimental methods are sound and the impact of the outcomes is highly relevant to the field of health monitoring using wearable sensors and devices.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This work considers that most wearable device only detects single-lead signal for ECG while there are effective multi channel CVDs detectors based on deep networks. Therefore, the authors propose a GAN-based method for reconstructing twelve-lead ECG. They not only utilize the reconstruction loss but also incorporate label information from Lead I. They compare the performance with various GANs, and the reconstructed signals are validated by three cardiologists.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors propose, for the first time, the use of a GAN to generate 12 signals from Lead I.
Furthermore, they present a variety of experimental results. For instance, they compare the performance with different GANs. Additionally, the authors conduct an ablation study to validate the components of their GANs. Moreover, the reconstructed signals are verified by three cardiologists.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some aspects of the writing are difficult to follow. For instance, in Section 2.3, it is unclear whether both e_1 and e_12 indicate the label of the signal, or if only e_1 indicates the label while e_12 is the actual signal. Consequently, the second equation and the third equation on page 4 do not match. In the third equation, calculating the L1 distance between e_1 and the output of label generator G_L is not possible because the vector sizes would be different. Additionally, the third equation is not utilized in the objective of EKGAN G_I.
The baselines used in the study are somewhat outdated. While CardioGAN, an ECG-specialized GAN, was published in 2021, Pix2Pix and CycleGAN were published in 2017. The experimental results demonstrate that Pix2Pix outperforms the ECG-specialized GAN. Therefore, it is reasonable to assume that the latest image generation specialized GAN would perform even better than Pix2Pix. Limited comparisons make it difficult to convincingly argue that the proposed method is the most effective.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method is reproducible since it provides the hyper-parameters for the network structures and objective function. However, it is worth noting that despite the authors’ claim of providing the range of hyper-parameters in answer 4 of the reproducibility checklist, no such range is actually mentioned in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper includes various excellent experiments such as performance comparisons, reconstructed samples, ablation study, and concordance rate comparisons. However, the number of experiments conducted is insufficient. As mentioned earlier, the baselines used in the study are outdated. Despite the authors’ claim that the 2D convolution-based method may result in degraded performance, Pix2Pix performs better than CardioGAN. It is reasonable to assume that the latest 2D convolution-based methods would outperform Pix2Pix. It would be beneficial if the authors compared their method with state-of-the-art approaches. Furthermore, they could have easily converted the 2D convolutional layers of image-specialized methods into 1D convolutional layers.
As mentioned earlier, the clarity of the paper needs improvement, especially regarding notations such as e1 and e12. Furthermore, does the number 16 in the input size of (16, 512, 1) indicate the batch size? Additionally, the authors did not provide information on how they selected the values of lambda=50 and alpha=1. Did they perform a grid search to determine these values?
Furthermore, if the authors included classification results using only Lead I, it would strengthen the credibility of the proposed method. If the classification performance using a single signal is significantly better than the performance of the generated 12 signals, it would suggest that a single signal classifier is sufficient for the task.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is somewhat challenging to recommend this paper for MICCAI 2023. While the defined problem and proposed method are interesting, the clarity of the paper needs improvement, as mentioned in my comments in 6. Furthermore, although the experiments cover various aspects, the volume of the experiments is quite limited, and the baselines used are outdated methods, as stated in answers 6 and 9.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper describes a GAN model to synthesize 12-lead ECG data from single-lead ECG with 1D convolutional neural networks. The work would have implications for the use of wearable devices. This is the kind of work that we don’t typically see at MICCAI, but it’s an interesting approach with good results that is appreciated by the reviewers. Reviewers raise several small issues in the clarity of the paper that can easily be addressed in a camera-ready version without performing additional experiments.
Strengths
- Interesting problem setting.
- Experiments are extensive and even include a qualitative evaluation by cardiologists.
Weaknesses
- Reproducibility is limited.
- Reviewers indicate that not all methods in the comparison are state-of-the-art.
Author Feedback
We sincerely appreciate the reviewers’ valuable feedback, which will greatly enhance the quality of our work. We will incorporate all constructive suggestions and address any concerns raised during the review process. In particular, to ensure reproducibility, we will release our prototype code alongside the final version. For detailed information on comparisons with SOTA models, please refer to our response to Reviewer #4. While we primarily address the comments of Reviewers #1 and #4 below, we want to assure that we will also address all constructive comments of Reviewers #2 and #5 in the revised manuscript. Reviewer #1’s comments on [6]: Answer) Thank you for your valuable suggestion. Although the problem addressed in [6] and their motivation differ from ours, as the reviewer pointed out, they can indeed generate multi-channel ECGs from a single lead, albeit in an incremental one-by-one manner. We will more carefully use the phrase “for the first time” in the revised version. Meanwhile, while their model only operates on a single bit of the QRS segment, our model targets longer ECG segments of about 8 seconds, encompassing multiple QRS segments. This enables our model to be used for predicting cardiovascular diseases that require analysis of consecutive QRS segments. Furthermore, the differences in data normalization and preprocessing may impact the performance measures, posing challenges in conducting a fair comparison. However, we acknowledge the need for such a comparison and will conduct one in our future research. Reviewer #1’s comments on visualization and our framework illustration: Answer) We will provide more visualization in supplementary material and improve the illustration. Reviewer #4’s comments on loss functions and the notation in Section 2.3: Answer) We appreciate the reviewer’s observation regarding the potentially misleading notation used in our paper. We will make it clearer in the revised manuscript. Meanwhile, it is correct that the third equation is not used in the objective of EKGAN G_I. It is only used for training G_L. To clarify this, we will provide a clearer explanation in the revised version. Reviewer #4’s comments on the comparisons with SOTA models: Answer) In this study, we conducted a comparison between conditional GAN approaches (including our own and Pix2Pix) and unconditional GAN approaches (specifically CycleGAN and CardioGAN) for ECG reconstruction. Our findings indicate that the conditional approach outperforms the unconditional approach in this task. We also observed that the more recent and ECG-specialized model, CardioGAN, demonstrates superior performance compared to the older CycleGAN, as expected. However, it is worth mentioning that current research trends in image synthesis and generation more focus on unconditional and diffusion-based models, resulting in relatively less exploration of conditional GANs. While our study highlights the effectiveness of conditional GANs for ECG reconstruction, there is still room for further improvement, as the reviewer pointed out. In our future research, we aim to explore more advanced conditional GAN approaches to enhance the capabilities of ECG reconstruction. Reviewer #4’s comments on the input size of (16, 512, 1), how we selected the values of lambda and alpha, and including classification results using only Lead I: Answer) 16 represents the number of leads plus zero paddings. The values of lambda and alpha were chosen through grid search, and we will provide clearer explanations in the revised version. Furthermore, in clinical practice, cardiovascular diseases (CVDs) such as LBBB and RBBB necessitate the simultaneous analysis of multiple leads. Consequently, we believe that a classification based solely on a single lead has limited clinical applicability. It is essential for a physician to examine the (multi-channel) ECG comprehensively when an AI model predicts CVDs, as a follow-up check.