List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ang Zhang, Guijuan Peng, Jialan Zheng, Jun Cheng, Xiaohua Liu, Qian Liu, Yuanyuan Sheng, Yingqi Zheng, Yumei Yang, Jie Deng, Yingying Liu, Wufeng Xue, Dong Ni
Abstract
Structural parameters of the heart, such as left ventricular wall thickness (LVWT), have important clinical significance for cardiac disease. In clinical practice, it requires tedious labor work to be obtained manually from ultrasound images, and results in large variation between experts. Great challenges exist to automatize this procedure: the myocardium boundary is sensitive to the heavy noise and can lead to irregular boundaries; the temporal dynamics in the ultrasound video are not well retained.In this paper, we propose a Temporally Compatible Deformation learning network, named TC-Deformer, to detect the myocardium boundaries and estimate LVWT automatically. Specifically, we first propose a two-stage deformation learning network to estimate the myocardium boundaries by deforming a prior myocardium template. A global affine transformation is first learned to shift and scale the template. Then a dense deformation filed is learned to adjust locally the template to match the myocardium boundaries. Second, to make the deformation learning of different frames become compatible in the temporal dynamics, we adopt the mean parameters of affine transformation for all frames and propose a bi-direction deformation learning to guarantee that the deformation fields across the whole sequences can be applied to both the myocardium boundaries and the ultrasound images. Experimental results on a ultrasound dataset of 201 participants show that the proposed method can achieve good boundary detection of basal, middle, and apical myocardium, and lead to accurate estimation of the LVWT, with a mean absolute error of less than 1.00 mm. When compared with human methods, our TC-Deformer performs better than the junior cardiologists and on par with the senior cardiologists.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43987-2_21
SharedIt: https://rdcu.be/dnwJD
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a 2-stage deformation framework for estimating wall thickness. Compared to the previous segmentation-based and direction-regression methods, this paper can handle the noisy boundaries and explicitly temporal modeling for the whole sequence. The error is within 1mm.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is easy to follow. The methods clearly address the summarized challenges. Instead of a pure CNN model, this paper smartly uses ring shape as prior knowledge to constrain the segmentation correctness.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Smoothness: The paper did not discuss the smoothness of the generated deformation field. For example, in VoxelMorph [https://arxiv.org/abs/1809.05231], a regularization loss must be applied. Otherwise, the boundary of deformed objects can be noisy. In this paper, I am curious if such a noisy/non-smooth deformation field exists. If not, which module/loss function avoids such an error? Experiments: The paper only used segmentation-based methods as a baseline (i.e., UNet, CLAS). It would be better if compared to one of the direct regression-based methods [12,5,4,13].
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
reproducibility is good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
If the authors would like to extend it as a journal paper, they may consider discussing their work with a larger scope. For example, how the proposed method is related to techniques used in image registration/atlas matching. And show extensive experiments.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper does not have obvious flaws except lack of experiments.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a deformable registration solution to solve the detection of the myocardium boundaries, which is mostly modeled a segmentation problem. It aims to warp a predefined template to fit the input image via registration, in which idea is similar with atlas-based segmentation. Compared with human methods, the proposed TC-Deformer performs better than the junior cardiologist and is on par with the middle-level cardiologist.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is of certain novelty and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Please refer to the comments.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper lacks of details about how to measure LVWT based on the obtained segmentations. Is the estimation of LVWT automatic or manual? After clearly presented that, it may be reproducible. In addition, training details should be presented to prove the reproducible capability. Since there are two sub modules for rigid and deformable registration and no deformation field is provided for directly supervision, I think there are tricks to train the model, i.e. maybe the two modules are not trained simultaneously. Something like that should be referred to make the paper convincing.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- In the abstract, it is declared that the paper proposes a deformation network and could estimate LVWT automatically. However, there is no information of estimate LVWT in the rest part of the paper. It’s clear that the estimation could be finished once one gets the contour, but it should be clearly presented for the sake of the completeness.
- How exactly are the global affine transformation and local deformation learning trained? 2) If the AT prior in the global affine transformation initially makes inaccurate predictions, it may introduce incorrect prior knowledge and fail to guide the training of local deformation learning. How to address this issue?
- Why is the AT loss not included in the final total loss function?
- In result tables, the best results should be highlighted.
- The comparison methods are too limited, which makes it difficult to demonstrate the novelty of the proposed method.
- It’s unfair to compare results from people and model like table 1. It’s unclear about the setting of expert groups and from Table 1 it’s not always the mid-level group beat the junior group. Maybe remove the table and provide other evidences to prove the advantages.
- Deformable registration is a kind of resource sensitive solution comparing with segmentation. The computational resource consumption should be discussed.
- Why is Unet compared in Table 2 but neglected in Fig 6?
- In the context of manuscript, Fig 6 is cited before Fig 5, which means it would be better to switch the two tables.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation is clear and the designed method is targeting the shortages described.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Although written could be improved, technology corresponding concerns have been solved. No more questions now.
Review #4
- Please describe the contribution of the paper
This paper deals with the automatic assessment of LV wall thickness from short axis views in 2D echocardiography, a subject very litle studied in the literature. This work demonstrates the feasibility of satisfactory measurements comparable to those done by the cardiologists. A two stage learning algorithn is proposed for segmenting the myocardium. In the first step a global affine transform model is learned based on the template of the myocardium and then a dense deformation field is acquired using a U-net.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) It is one of the first works considering the problem of myocardial wall thickness estimation from short axis ultrasound images. (2) The wall thickness is measured at all AHA segments. (3) It is shown that the use of a ring-type template for the myocardium permits to obtain a satisfactory estimation of a simple affine deformation model. (4) It is demonstrated that using a U-net the myocardium can be localized given the estimated ellipse type initial shape. (5) The temporal consistency is taken into account. (6) Good results are reported on a dataset owned by the authors.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) The main algorithmic steps are not novel. In reference [8] of the paper is introduced the concept of template transformer networks, where a shape template is deformed to match the underlying structure of interest, and is also introduced a way of incorporating priors in binary classification methods such as U-net. The application on myocardial SAX segmentation in echocardiography is novel, as well as the introduction of the Temporally Compatible Transformer. (2) For the last module the motivation of the loss function in Equation (3) is not given. (3) The standard deviation of the estimation error given in Fig. 5 seems to be strong. (4) While two other segmentation methods are implemented, the corresponding wall thickness estimation is not given.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data collection process is briefly described and the dataset is not provided.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please verify that the structural similarity in Equation (3) is a loss. Please give the exact formulation of the wall thickness estimation. Do you consider a section of the ring for each segment, or just two corresponding points as suggested in Fig. 1? Please give not only the absolute errors on wall thickness, but also the relative errors. It would be preferable to give the HD in Table 2 in mm. Please give more details on the implementation of methods U-net and CLAS of Table 2.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
An interesting application of template based segmentation is given on myocardioal SAX echocardiography, but a more extensive comparison to other methods is needed, in particular concerning wall thickness estimation, as the dataset is not publicly available.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
R1 considers this paper is easy to follow, the method clearly address the summarized challenges, but lacks the direct regression-based methods.
R2 also considers it is easy to follow with some novelty, but shows many concerns about it.
R4 considers it is one of the first studies for myocardial wall thickness estimation from short axis ultrasound images, with good results.
Although three reviewers tends to show the positive attitude for this paper. However, there are some serious concerns abou it
-
It is not new to consider a ring shape prior as the circle-like object segmentation/detection in medical image analysis, e.g. Segmentation of intravascular ultrasound images: a knowledge-based approach, IEEE TMI, 1995.
-
Although the prior seems to be applied to address the challenge of heavy noise, is it possible to over-smooth the border because of the prior. How to balance it ?
-
Reference [8] has introduced the templated transformer networks. It seems similar to the proposed method.
-
Little comparison with the sota method for short axis ultrasound cardiac images. It is difficult to support the superiority of the proposed method in this field.
-
Some problems in paper writing and illustration making
- it is inappropriate to make the reference number as the subject of a sentence.
- The figures and tables in Page 8 look awkward.
- Some grammatical errors.
-
Author Feedback
We sincerely thank the AC and reviewers for their efforts of reviewing our work and appreciate those positive comments about the writing and organization (R1, R2, R4), the novelty of the work (R1, R2, R4). We would like to make clarifications below regarding the other main concerns of the reviewers. META-1: The IEEE TMI 1995 paper that considers a ring shape prior. Thanks for reminding this interesting paper to us. We will add this reference in the introduction of the revised paper. We would like to clarify that the usage of the ring shape prior in our work is clearly different from that in the TMI paper. First, we use a general prior template which is generated by the mean shape of the myocardium from the training data. The TMI paper made an ellipse assumption of the vessel and required four points to be manually marked to define the ellipse. Second, the prior template is transformed first by a global affine transform to match the scale and position, and then followed by a local deformation to adjust the boundaries in a fine-grained manner. In the TMI paper, the ellipse prior was used to resample the image (coordinate transformation) to straighten the boundary and to combine with the edge information to identify the external lamina using a graph search algorithm. META-2: Is it possible to over-smooth the border because of the prior. How to balance it? No, our method won’t over-smooth the border. The prior template provides a reasonable starting point for our method. With the following local deformation learning, the prior circular shape will be adjusted to match the myocardium boundaries during the supervised deformation learning procedure. META-3: Reference [8]. Our method clearly differs from [8] in two ways: two-stage transformation learning and temporal-compatible deformation learning. The two-stage transformation learning first learns the global affine transformation and then learns the local deformation, where a novel bi-directional temporal compatible constraint is added for the video data. In [8], the transformation is conducted between two static images, with no temporal constraints. Besides, it’s a single-step operation, without the global affine transformation. META-4: Comparison with SOTA methods and other direct regression methods. In Table 2, we have compared out method with the classic Unet and one SOTA method CLAS, which achieves the best performance for segmentation of A4C and A2C ultrasound images. For direct regression methods, it may be unfair to compare with them since they do not require the segmentation mask for supervision and probably leads to inferior performance. We can add the results in the revised version. R1: Smoothness. In our method, we experimentally find that a smoothness regularization does not lead to improved results. This may be because that the prior template provides a reasonable and smooth starting point for the deformation learning, and the supervised information ensures a good direction of the learning. R2: Training details. Yes, the training is in two stages, i.e., global affine transformation and local deformation learning. We will make this clearly clarified in the revised version. R2, R4: Calculation of LVWT. We follow the procedure in [12] to calculate LVWTs from the segmented myocardium mask. R4: Motivation of Eq. 3. As shown in the right of Fig. 3, the motivation of Eq. 3 is to encourage the learned deformation field to be compatible with the changes of the image content in the ultrasound sequence. Specifically, when one frame X_t is deformed first forwardly and then backwardly, the resulted image content should be the same as X_t. We use SSIM to measure the similarity of two images. It should be (1-SSIM) in the loss function.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The author doesn’t address my concerns. The innovation of the method is limited, and the technical details are not clear enough. The authors do not explain the details of affine transformation and local deformation learning during trainingg. The typesetting format of the rebuttal is poor, which affects the reading experience.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All reviewers agreed on the novelty of the paper. The authors have answered some of the questions in the rebuttal. Please still check the language and grammer in the final version for the readability of the paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The proposed method is a deformable template-based approach for image segmentation. A global transformation followed by a local deformation is quite common before deep learning becomes popular. The TC-Deformer is relatively novel, though I wonder why it is necessary as only the wall thicknesses at ED and ES are of interest. The experiments were only performed on a private dataset, and the performance of the proposed framework was similar to other tested methods.
As pointed out by meta-reviewer #1, there are discrepancies between the comments and ratings of the reviewers, especially reviewer #2. Overall, I think this is a weak-reject paper even after the rebuttal.