Authors

Xiaojiao Xiao, Qinmin Vivian Hu, Guanghui Wang

Abstract

Simultaneous multi-index quantification, segmentation, and uncertainty estimation of liver tumors on multi-modality non-contrast magnetic resonance imaging (NCMRI) are crucial for accurate diagnosis. However, current methods do not have an effective mechanism for multi-modality NCMRI fusion and accurate boundary information capture, which makes these tasks challenging. To address these issues, this paper proposes a unified framework, called edge-aware multi-task network (EaMtNet), to associate multi-index quantification, segmentation, and uncertainty of liver tumors on multi-modality NCMRI. The EaMtNet is made edge-aware by using the newly designed edge-aware feature aggregation module (EaFA) for feature fusion and selection, which captures long-range dependency between feature and edge maps. Multi-tasking improves segmentation and quantification performance by leveraging prediction discrepancy to estimate uncertainty. Extensive experiments are performed on multi-modality NCMRI with 250 clinical subjects. The proposed model outperforms the state-of-the-art by a large margin, which demonstrate the potential of EaMtNet as a reliable clinical-aided tool for medical image analysis.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43901-8_62

SharedIt: https://rdcu.be/dnwEa

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #2

Please describe the contribution of the paper

The authors propose a multi-task and multi-modal approach with edge awareness (i.e. edges are accounted for in training and testing time) for liver tumors where they quantify multiple index (e.g. area), segmentation quality and uncertainty around the results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The article has a clear structure and is easy to follow. Multi-modal and multi-task approaches are not novel, but present novelty in the context of the assessment (liver tumors). The idea of using uncertainty quantification together with other tasks seems a reasonable and interesting approach. The evaluation is clear and presents evidence that the approach is effective.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Despite the good and interesting job carried out by the authors, the article could benefit from a more diverse range of metrics, specially in the segmentation part. It is well-known that DSC can be misleading as a segmentation metric and it is commonly recommended to be presented together with other complementary metrics such as Hausdorff distance or relative volume difference. Addition of those metric would strengthen the evaluation of the authors when seen from a more clinical perspective and utility.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors comply with the checklist. No mention to code or github repository is available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Despite the good and interesting job carried out by the authors, the article could benefit from a more diverse range of metrics, specially in the segmentation part. It is well-known that DSC can be misleading as a segmentation metric and it is commonly recommended to be presented together with other complementary metrics such as Hausdorff distance or relative volume difference. Addition of those metric would strengthen the evaluation of the authors when seen from a more clinical perspective and utility.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The authors present a clear organisation and the work has clear and well-defined objectives. The development of the work is also clear as it is. One of my main concerns and recommendations would be to consider adding more metrics in the segmentation results. DSC is commonly presented together with Hausdorff distance and Relative Volume Differences given the drawbacks of DSC and the utility in clinical practice of measures such as the volume. Good job!
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #5

Please describe the contribution of the paper

1.The paper proposed an edge-aware multi-task network for segmentation and quantification on liver tumor based on multi-modality non-contrast MRI.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The authors pointed out that T2FS and DWI is complementary and proposed a novel edge-aware feature fusion model to extract the import information from different modality data.
2. The paper is well structured with clear visual illustration on the proposed model.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. To extract two modality input data, the authors devised two encoders to extract different information. It seems inflexible for that if the model introduce other modality data, the number of corresponding encoders will also increase, which may cause the parameters explosion.
2. In this paper, the Sobel filters are applied to extract the boundary information of segmented objects. However, the MR images appear blurred or even invisible. Under this circumstance, the Soble filters tend to be ineffective.
3. In experiment part, the authors compare the performance of EaMtNet w/ and w/o EaFA module. EaFA module aggregate features extracted from two encoders and edge features derived from Sobel filters. If EaFA module only make the fusion on features extracted from two encoders, the authors should demonstrate the its influences on the segmentation results.
4. In EaFA part, different kinds of features are aggregated, including the low-level features (edges) and high-level features (from encoders). Author should discuss the necessity of feature alignment before feature fusion.
5. The EaMtNet uses linear regression to obtain the center point, max-diameter and area of liver tumor. It is wonder that whether the linear regression parts are of benefit to segmentation results.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper provided the source code of proposed model.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1. The authors should provided more experimental details to verify the effectiveness of Sobel filters in dealing with blurred MR images. 2.The authors should demonstrate the relationship between the segmentation and quantification tasks.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed model obtains well segmentation results and the structure of paper is well organized.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This paper proposes a unified framework, called EaMtNet, to associate multi-index quantification, segmentation, and uncertainty of liver tumors on multi-modality NCMRI. To enhance the weight of tumor boundary, Sobel filter extracts edge information and the connecting local feature as prior knowledge. The novel EaFA module makes our EaMtNet edge-aware by capturing the long-range dependency of features maps and edge maps for feature fusion.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper proposes a unified framework, called EaMtNet, to associate multi-index quantification, segmentation, and uncertainty of liver tumors on multi-modality NCMRI. To enhance the weight of tumor boundary, the edge information extracted via the Sobel filter is introduced into the model. The EaFA module further refines the original feature maps using the extracted edge features via the transformer-like strategy.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In this paper, the authors claim that the existing methods ignore the complementary information between multi-modality NCMRI of T2FS and DWI, and mainly focus on designing a model to process multi-modality data. In my opinion, the authors should further conduct experiments to evaluate the differences between using only the single modality and the multi-modality inputs to better verify the necessity of using multi-modality information.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors claim that the codes will be released.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

The author should discuss more about the complementary information between multi-modality NCMRI of T2FS and DWI. The author can conduct experiments to verify the effectiveness of using multi-modalities.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The motivation and the proposed modules in this paper are reasonable, and the experimental results verify the effectiveness of the proposed modules. But the authors should discuss more about the complementary information between the multi-modalities to further improve this paper.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

R2 considers this paper is easy to follow, clear structure, and the approach is reasonable and interesting.

R4 considers it has reasonable method and effective results.

R5 considers it is novel, well structured, and clear figures.

All reviewers tend to accept this paper.

Author Feedback

N/A

back to top

Edge-aware Multi-task Network for Integrating Quantification Segmentation and Uncertainty Prediction of Liver Tumor on Multi-modality Non-contrast MRI