List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Keming Tang, Zhenyi Ge, Rongbo Ling, Jun Cheng, Wufeng Xue, Cuizhen Pan, Xianhong Shu, Dong Ni
Abstract
Mitral regurgitation (MR) is the most common heart valve disease. Prolonged regurgitation can cause changes in the heart size, lead to impaired systolic and diastolic capacity, and even threaten life. In clinical practice, MR is evaluated by the proximal isovelocity surface area (PISA) method, where manual measurements of the regurgitation velocity and the value of PISA radius from multiple ultrasound images are required to obtain the mitral regurgitant stroke volume (MRSV) and effective regurgitant orifice area (EROA). In this paper, we propose a fully automatic method for MR quantification, which follows the pipeline of ECG-based cycle detection, Doppler spectrum segmentation, PISA radius segmentation, and MR quantification. Specifically, for the Doppler spectrum segmentation, we proposed a novel adaptive-weighting multi-channel segmentation network, PISA-net, to accurately identify the upper and lower contours of the PISA radius from a pair of coupled M-mode PISA image and corresponding M-mode decolored image. Using the complementary information of the two coupled images and combing with the spatial attention module, the proposed PISA-net can well identify the contours of the PISA radius and therefore lead to accurate quantification of MR parameters. To the best of our knowledge, this is the first study of automatic MR quantification. Experimental results demonstrated the effectiveness of the whole pipeline, especially the PISA-net for PISA radius segmentation. The full method achieves a high Pearson correlation of 0.994 for both MRSV and EROA, implying its great potential in the clinical application of MR diagnosis.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43987-2_22
SharedIt: https://rdcu.be/dnwJE
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #3
- Please describe the contribution of the paper
Authors propose a Mitral Regurgitation (MR) quantification method based on signal extraction and image segmentation. The proposal uses a deep architecture for image segmentation to extract the MR stroke volume and effective orifice area from multi-channel ultrasound images. Compared to other methods, the authors combined different types of images in order to extract an accurate information, obtaining good validation results.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Originality: the main strength is the combination of different information sources to quantify the proximal isovelocity surface area. They combined a CD, M2D, and MCD ultrasound images in an unique pipeline to extract the desired measurements.
- Significative performance: the method was tested providing good results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Limited extendibility: the proposed work is only useful for the combination of the above-mentioned images, so if one of them is not available, the algorithm can’t be applied.
- Limited dataset: used data is good but more data is needed to validate the results.
- Insufficient validation: cross-validation is necessary to provide a realistic performance of the method and to be compared with others.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Models and algorithms: it would be good to include an algorithm to show the steps of the pipeline.
- Datasets: authors said ‘Yes’ for a downloadable version of the dataset, but it is not present in the paper and not referenced.
- Code: authors said ‘Yes’ for the inclusion of code, but in the submission there is no code to run and test.
- Reported experimental results: The authors chose ‘Yes’ for every statement, which is false. The average runtime, the number of run, the analysis of situations where the method failed, etcetera, are not presented.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Improve results section: authors should make a better description of the experiments and results. The plots are presented but not described with detail. The error measures were mentioned at the beginning but not remarked in every plot and report.
- Code availability: it would be good if the code can be open to doctors and researchers.
- Further works: I would recommend test the method with other datasets, and also compare other deep architectures. It would be also interesting to see if this method can be applied to 3D heart images.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The contribution of the paper is interesting. The problem they need to solve is very specific, and there are not many works related to. The fusion of different information sources seems to be very effective in medical quantification. More experiments are needed to validate the algorithm, but I understand that there not enough data and state-of-art methods to compare with.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
An automatic pipeline for MR quantification from multi-channel ultrasound images was proposed in the present work. The proposed method shows well segmentation results and quantification of MR parameters, and may contribute to the MR diagnosis in clinic.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main work of the paper is as following: 1) the authors propose an automatic pipeline for MR quantification from multi-channel ultra- sound images; 2) an adaptive-weighting multi-channel segmentation network based on Unet is proposed to identify the lower and upper contours of the PISA radius from the images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
In clinical procedure, the regurgitant flow rate is measured as the product from the surface area of the hemisphere and the aliasing velocity. The radius is firstly measured through the color doppler during the mid-systolic frame. The MR jet velocity/VTI is susquently calculated by CW doppler MR and color Doppler.The main contribution of this work is the development of the automatic pipeline of MR quantification and the modified UNet to segment the contour of PISA. In general, the idea of the present work is not novel and the employed deep learning methods has been widely used.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The model is difficult to be reproduced.The dataset and code are not available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
More experiments are needed to prove the robustness of the proposed pipeline. For instantce, how is the performance when there are heavy noise in the ultrasound images. Besides, what is the calculation time of the proposed pipeline compared to that in clinical procedure.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is not novel. But the results are generally promising and the developed automatic pipeline may benefit for the clinic.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The paper presents a fully automatic method for quantifying mitral regurgitation (MR), a common heart valve disease. The method follows a pipeline consisting of ECG-based cycle detection, Doppler spectrum segmentation, PISA radius segmentation, and MR quantification. The authors introduce PISA-net, an adaptive-weighting multi-channel segmentation network that effectively identifies the upper and lower contours of the PISA radius from coupled M-mode PISA images and corresponding M-mode decoloured images. By leveraging the complementary information of the coupled images and a spatial attention module, the proposed PISA-net accurately quantifies MR parameters. The results demonstrate the effectiveness of the pipeline, particularly PISA-net for PISA radius segmentation, achieving a high Pearson correlation of 0.994 for both MRSV and EROA, highlighting its potential for clinical application in MR diagnosis.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The study addresses a clinically important problem, mitral regurgitation (MR), and proposes a fully automatic pipeline for MR quantification using multi-channel ultrasound images. The approach is valuable for improving the efficiency and accuracy of MR diagnosis in clinical settings.
-
The proposed PISA-net incorporates a novel adaptive-weighting mechanism and a spatial attention mechanism to effectively combine features from multi-channel inputs and enhance local features using a global context. This innovative approach contributes to the accurate identification of the upper and lower contours of the PISA radius and leads to precise quantification of MR parameters.
-
The authors have carried out extensive experiments and evaluations to validate their method, including comparisons with other state-of-the-art segmentation techniques such as Unet, Unet+SE, and Attention Unet. These comparisons demonstrate the effectiveness of the proposed PISA-net in achieving better segmentation accuracy.
-
The full method achieves a high Pearson correlation of 0.994 for both MRSV and EROA, which indicates its potential for successful clinical application in MR diagnosis. The strong performance of the proposed pipeline in quantifying MR parameters is a significant strength of this work.
-
The inclusion of various figures and tables provides a comprehensive visual representation of the proposed method, which aids in understanding the concepts and evaluating the results.
-
The proposed method is applicable to images collected from different ultrasound machines (GE VividE95 and PHILIPS CX50), suggesting its adaptability and practicality in real-world clinical environments. This feature highlights the robustness and generalisability of the proposed method.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The literature review needs to be updated to include recent advancements in the field. The referenced methods are outdated, and the authors should explore and compare their work with more recent studies, particularly those that address automation in MR quantification.
-
The claim of being the first attempt to automate the process is not fully substantiated. There are existing automated methods for deriving these metrics. The authors should provide a thorough comparison of their contributions with the latest published work in the field.
-
While the results in Tables 1 and 2 demonstrate numerical superiority, the significance of these improvements remains unclear. The authors should conduct a statistical analysis to assess the significance of the results and provide information on the number of parameters used in each model to ensure a balanced comparison.
-
An inter-observer variability analysis would support the effectiveness of the proposed model.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors did not mention or include any link to a repository in the manuscript, but relevant code will be publicly available, as stated in the reproducibility check. Uploading the trained models may improve the reproducibility and transparency of the manuscript.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
The motivation of the study is clearly defined. However, the referenced works on the background and the proposed methods for this problem are about 20 years old. While the background could be justified, the referenced proposed methods might be deprecated by now. Please revise the literature review exploring recent work.
-
Similar to the previous point, the claim that this work is the first attempt to automate the process is not fully supported. There have already been methods that automate the derivation of these metrics, please compare the current contribution with the already published work.
-
Please avoid directly referring to figures to explain something: “Figure X shows that A is larger than B”. Instead, state a general finding, and illustrate this with a reference to a figure “A is larger than B (Figure X)”.
-
In section 2.1, please clarify how these steps are performed. Could there be any source error from this phase that can propagate to the next steps?
-
As the proposed method is segmenting the area under the curves, is there any influence of the pixel spacing? Is the size 256 x 256 the whole field of view?
-
While the binary cross-entropy loss is one of the most widely used approaches, I would suggest using focal loss to enforce the model to handle complex cases.
-
In the (sub)section of results, further evaluation techniques should not be outlined, these should be in a prior (sub)section.
-
For the comparison analysis, the Swin-U-Net should also be used to further assess the benefits of the proposed model.
-
While the results in both tables 1 and 2 are numerically superior, are these significant? The extra benefit is marginal. Can the authors also show the number of parameters to balance the comparison?
-
Please discuss the limitation of the single-centred dataset.
-
Minor detail for the graphics: Please increase the font size to improve readability. Also, the range of the y-axis of the Bland-Altman plots is too wide.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The study’s motivation is clearly defined, and it presents a novel method for automating mitral regurgitation quantification using a pipeline and PISA-net. However, the literature review requires revision to include recent work, and the claim of being the first attempt at automation is not fully supported. Additionally, there are concerns about the segmentation process, the use of binary cross-entropy loss, and the need for a more comprehensive comparison with other models. While the results show numerical superiority, their significance needs further analysis. The paper has potential, but addressing these issues is necessary for a stronger recommendation.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents an automatic pipeline for Mitral Regurgitation (MR) quantification from multi-channel ultrasound images, offering a unique combination for more precise quantification. The authors propose an adaptive-weighting multi-channel segmentation network based on Unet, called PISA-net. The results, although preliminary, seem promising.
However, some weaknesses need addressing to elevate the work’s overall quality. The literature review requires an update. The claims of the study being a first attempt at automation need better substantiation. Further, the manuscript lacks information about inter-observer variability analysis and parameter details for each model to ensure balanced comparisons.
In addition, the technical depth of the paper could be improved by providing more information about the pipeline’s steps and their potential error propagation. The authors should also consider discussing the influence of pixel spacing and potential limitations related to the single-centered dataset. More experiments, including those accounting for noise in ultrasound images, are suggested for better validation. Moreover, the authors should make their dataset and code publicly available for better transparency and reproducibility.
Author Feedback
N/A