List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Xukun Zhang, Yang Liu, Sharib Ali, Xiao Zhao, Mingyang Sun, Minghao Han, Tao Liu, Peng Zhai, Zhiming Cui, Peixuan Zhang, Xiaoying Wang, Lihua Zhang
Abstract
Accurately segmenting the liver into anatomical segments is crucial for surgical planning and lesion monitoring in CT imaging. However, this is a challenging task as it is defined based on vessel structures, and there is no intensity contrast between adjacent segments in CT images.
In this paper, we propose a novel point-voxel fusion framework to address this challenge. Specifically, we first segment the liver and vessels from the CT image, and generate 3D liver point clouds and voxel grids embedded with vessel structure prior.
Then, we design a multi-scale point-voxel fusion network to capture the anatomical structure and semantic information of the liver and vessels, respectively, while also increasing important data access through vessel structure prior. Finally, the network outputs the classification of Couinaud segments in the continuous liver space, producing a more accurate and smooth 3D Couinaud segmentation mask. Our proposed method outperforms several state-of-the-art methods, both point-based and voxel-based, as demonstrated by our experimental results on two public liver datasets. Code, datasets, and models are released at https://github.com/xukun-zhang/Couinaud-Segmentation.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43898-1_45
SharedIt: https://rdcu.be/dnwBF
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
This paper targets the Couinaud segmentation problem in Liver CT scans. The proposed framework is a combination of point-based and voxel-based approaches, which effectively addresses their individual limitations and enhances their benefits. The overall framework is designed in principle and demonstrates robustness against potential corner cases. Trained and validated on 3Dircadb and LiTS (with additionally manually annotated Couinaud segments), the proposed framework outperforms the leading approaches by a large margin.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The strengths of the paper are summarized as follows: 1) This paper targets a challenging problem in medical imaging, i.e., precisely parsing segments using spatial priors. This challenge is particularly prevalent when the segments are primarily determined by the surrounding anatomical structures rather than the imaging context. The proposed framework presents a multi-scale fusion that effectively combines voxel features and point locality for Couinaud segmentation.
2) The proposed framework is inspirational. I assume the proposed framework could work well for other tasks, e.g., brain tissue segmentation, lymph node station segmentation, vessel segment parsing, and even multi-modality fusion.
3) While the existing fusion approaches might share similar ideas, the proposed framework is carefully designed and targets 3D medical imaging problems, which I assume the (given relatively reasonable supporting priors) proposed framework stands robustness against corner cases.
Some minor good aspects: 1) The paper is in good shape and the convey of the paper is well-organized. Most of the content is self-contained. 2) I appreciate the authors would publish the code if the paper gets accepted.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I couldn’t find major weakness of this paper. I list some minor questions/suggestions as follows: 1) Concerning the point-cloud computation efficiency, will it possible to have a region-wise module that could “stitch” the region patches for overall prediction?
2) It would be great if the authors could report the training and test time of the proposed framework. The authors might want to discuss the potential inference acceleration directions.
3) The author might want to report the surface-dice and highlight the boundary differences for better illustration.
4) The Figure font are too small (including the one in the supplementary material)
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Yes
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper is in good shape. Please refer to Section 6 for suggestions.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
As I mentioned in Section 5, precisely parsing segments using spatial priors in medical imaging is challenging. To further improve the details requires effort. I appreciate the authors’ devotion to this problem and present a framework to tackle it. The proposed framework is designed in principle and demonstrates good performance. I, therefore, give a “strong accept” for this paper.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
This paper addresses a challenging segmentation task. The proposed method casts the segmentation task as a point cloud segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper addresses a challenging Couinaud segmentation task.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
• The aim of the paper is not clear (see 9) • The baseline UNet model is too weak and not applicable. (see 9) • The data and experimental settings are not clearly described (see 8)
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
-
The intra- and inter-user variability between multiple annotators are not reported. This variability is essential to clarify the desired resolution for the output!
-
The split for the training/validation is not mentioned.
-
The network is trained for 400 epochs, but how the final weight is chosen? Best on the validation loss?
-
The training and validation loss of the selected epoch for each method is not mentioned.
-
The robustness of the Couinaud segmentation with respect to the performance of the liver/vessel segmentation is not reported.
-
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
In the introduction, it is claimed that “Unfortunately, the CNN models treat all voxel-wise features in the CT image equally, cannot effectively capture key anatomical regions useful for Couinaud segmentation.” without providing any proof or reference! By utilizing the attention block or the transformer, CNN do not treat all voxel-wise feature equally!
-
In the introduction, it is mentioned that “In addition, all these methods deal with the 3D voxels of the liver directly without considering the spatial relationship of the different Couinaud segments, even if this relationship is very important in Couinaud segmentation.” However, all spatial relation can be considered with a higher receptive field or by utilizing a transformer!
-
The aim for converting a uniform grid to a sparse point cloud, then sampling from that, training a network, and finally achieving an arbitrary resolution cloud/mesh for Couinaud segmentation is not stated. Is the subpixel accuracy important in this study? How large is the intra- and inter-user variability of the ground truth masks/ contours? What is the reason of converting a uniform grid to a sparse point cloud while the entire 3D image (or downsampled) can be easily given to an advanced voxel-based network using a normal GPU! Point-based approaches are applicable in the input data is non-uniform and sparse such as LiDAR. Explicit 3D representation is an exciting field. However, those methods are only applicable if a mesh output or subpixel accuracy is intended.
-
The baseline should be an advanced segmentation model like TransUNet [1] or a simple one but with a large receptive field.
Minor comments:
-
The MLP block only using Conv1d(1,) as mentioned in the appendix, is a 3D convolution with a kernel size of 1 and stride of 1. It is not clear how it relates to MLP!
-
The receptive field of the Basic-Res-Sig block seems to be 7. That might not be sufficient even for the third point-voxel fusion block with the size of 16.
-
Page 3, image coordinate point is written as I = {i1, i2, …, it, it} ∈ R3. The one to the last element should be it-1? {i1, i2, …, it-1, it}
-
It is not mentioned if the input of 3D UNet is the image or the vessel-liver masks
Minor typo:
Introduction #5: helps surgeons for make surgical planning 2.2 #1 Based on the above work, we first use the M′ and the L to sample get point data
Additional comments for extending this publication in the future (not for this conference paper):
-
It is possible to train a single UNet for segmenting the liver and the vessels! In theory, the UNet is capable enough to predict multiple labels. The vessel masks can be filtered in a post-processing step by the liver mask. The vessel loss can be simply ignored for the LiTS dataset.
-
The pretrained liver-vessel segmentation network can be in theory, used in extracting features for the point cloud. This idea is used in [2, 3].
[1] Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L. and Zhou, Y., 2021. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306.
[2] Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S. and Geiger, A., 2019. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4460-4470).
[3] Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M. and Geiger, A., 2020. Convolutional occupancy networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16 (pp. 523-540). Springer International Publishing.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The aim of the paper is not clear!
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper presented a couinaud segmentation method based on a point-voxel fusion framework. The fusion was claimed to be able combine topological relationship of coordinate points learned from pointcloud with semantic information learned from voxels. Both the liver and vessel were pre-segmented and fed as input to the network.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper was well written and easy to read
- It used open access datasets which made it easy to reproduce
- Pretty diagram with promising results
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Not clear how senstive the method is to the vessel segmentation performance
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- The used hyperparameters were well documented
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
Please list the annotation SOP, e.g. how many annotators were recruited, what tool was used for the annotation. It is also beneficial to show/discuss the inter-observer variance for this particular task.
-
Randomly drop off part of the vessel and analyse the impact on the segmentation and final clinical output would be even better
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Novel method with convincing results,
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Work addresses an important and challenging segmentation task. A method is proposed and results are shown on 2 public liver datasets. The reviewers indicated strengths of the work and found it of interest. Reviewers 2 and 3 gave the work very high scores. I agree with their reviews and recommend an Accept. Reviewer 1 raises major concerns regarding the baseline used and overall data and the experimental settings. I recommend that the authors take these remarks intoc onsideration in the final paper and presentation.
Author Feedback
N/A