List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Zixu Zhuang, Xin Wang, Sheng Wang, Zhenrong Shen, Xiangyu Zhao, Mengjun Liu, Zhong Xue, Dinggang Shen, Lichi Zhang, Qian Wang
Abstract
Magnetic Resonance Imaging (MRI) has become an essential tool for clinical knee examinations. In clinical practice, knee scans are acquired from multiple views with stacked 2D slices, ensuring diagnosis accuracy while saving scanning time. However, obtaining fine 3D knee segmentation from multi-view 2D scans is challenging, which is yet necessary for morphological analysis. Moreover, radiologists need to annotate the knee segmentation in multiple 2D scans for medical studies, bringing additional labor. In this paper, we propose the Cross-view Aligned Segmentation Network (CAS-Net) to produce 3D knee segmentation from multi-view 2D MRI scans and annotations of sagittal views only. Specifically, a knee graph representation is firstly built in a 3D isotropic space after the super-resolution of multi-view 2D scans. Then, we utilize a graph-based network to segment individual multi-view patches along the knee surface, and piece together these patch segmentations into a complete knee segmentation with help of the knee graph. Experiments conducted on the Osteoarthritis Initiative (OAI) dataset demonstrate the validity of the CAS-Net to generate accurate 3D segmentation.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_11
SharedIt: https://rdcu.be/dnwCV
Link to the code repository
N/A
Link to the dataset(s)
https://github.com/zixuzhuang/OAI_seg
Reviews
Review #1
- Please describe the contribution of the paper
In this paper, the authors proposed a stacked segmentation on MR knee tissue. The proposed method first builds a knee graph representation in a 3D isotropic space, and then the authors utilize a graph-based network to extract the local patches around the knee bone surface, and in final, the segmented patches are combined to a whole knee tissue segmentation. The experiments are tested on the OAI dataset (knee bone, cartilage, meniscus).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The two-step framework may be useful for the refinement of knee tissue segmentation. Based on the knee tissue graph representation, the algorithm could segment the tissue in a local area. From the visual results, some local segmentation errors could be reduced.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
From the experimental part, the proposed method does not show good performance. First, the knee bone may be not the key tissue for the OA analysis, but cartilage and meniscus are. Second, lots of papers have shown better performance, e.g., Automated Segmentation of Knee Bone and Cartilage combining Statistical Shape Knowledge and Convolutional Neural Networks, Automatic segmentation of meniscus based on MAE self-supervision and point-line weak supervision paradigm and etc.
-
The method is not novel. First, the two-step framework has already been proposed by previous approaches (e.g., the above paper “Automated Segmentation of Knee Bone and Cartilage”). Second, Zhuang [21, 22] have shown the method to build the knee tissue’s mesh representation, and the main framework in this paper is very similar to the previous ones. Third, from the experimental part, the GTN does not show very good results, why the authors do not test the Liu [10] or other related networks in this work?
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code is not provided, but the dataset is accessible. The reproducibility of this paper is good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I think the proposed method is mainly focusing on the refinement of segmentation, and thus the authors should report the metrics in the local area of knee tissue, and also use some distance metrics to prove the performance.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The segmentation performance in this paper is not good, and thus it cannot show its method’s effectiveness.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
Thanks for the response in the rebuttal.
The knee graph construction in multi-view slices of 3D data may be a novel usage in this point, but a good novelty requires some effective results to support it.
In the response from “Q2: Segmentation performance (R1)”, it shows that the proposed method is sensitive to the motion issue, yet the previous approaches (e.g., 3D segmentation) are stable and reported their results on the whole dataset.
If the authors are going to prove the performance of the proposed method, besides the DICE value, some distance metrics on the segmented results should also be provided. The authors should focus on the results of cartilage and meniscus which are much challenging. The authors should provide the detailed segmentation results of the cartilage and meniscus, their local detail segmentation results could prove the segmentation ability of proposed model.
The authors could also compare the running time on the same 3D data with other segmentation approaches (e.g., some previous 3D segmentation on cartilage or meniscus).
Review #2
- Please describe the contribution of the paper
The paper describes a system which uses a variety of deep learning approaches to segment the bones and cartilage in the knee from multi-view 2D MR slices. Multi-view 2D slices are more commonly used in the clinic than full 3D volumetric scans. A 3D volume is estimated from the 2D slices by super-resolution methods. A 2D U-Net is used to segment the sagittal slices to obtain an initial segmentation and associated knee mesh (graph). The nodes of the graph correspond to points on tissue boundaries, with associated centre cropped patches. A graph transformer network is used to integrate information across the graph, combined with a 3D U-Net-like structure, aiming to generate improved predictions for labels in the voxels in each patch. The algorithm is trained on 210 images and tested on 70 images. Ablation studies are included.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Interesting approach. Ablation studies demonstrate that it works and that each component makes a contribution to the overall performance. The proposed method seems to perform better than a 3D nnU-Net on full 3D volumes for the bones, though is significantly worse than that for the cartilage. However, it is better than alternatives on the 2D multi-view data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Space limitations mean that the description of the Graph-based U-Net is quite terse, though sufficient to give the general idea.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Assuming that the code is made available, and details of which of the images from the OAI dataset were used, this should be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
P2 “scans in local for” - what does this mean? P2 “utilize” -> “use” ? P4 “distributed nearby bone” -> “distributed around bone” ? P5 What is meant by “added to the origin feature H by broadcasting” ? P5 “is utilized to our method” – unclear. Do you mean “is used in our method” or “is used to [do something] in our method”? P5 “And the 2D … “ -> “The 2D …” (no need to start the sentence with “And” Table 1: “Cononal” -> “Coronal”
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Nice work, demonstrating that good results can be achieved even on multi-view 2D slices with the proposed method.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposes a MRI knee segmentation algorithm, which makes use of multi-view 2D scans of the knee to perform a segmentation. To that, the 2D slices are resampled in a 3D volume using super-resolution network. Then the high resolution patches are used to construct a graph covering the knee, which is used in a 3D segmentation network.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Presentation of a novel idea to generate knee segmentation from multi-view scans Good presentation and visualization of the idea Extensive examination of the algorithm
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
A nice algorithm is presented, which seems to work on good datasets An example of a bad segmentation result would give the reader more insight into the problems of the proposed algorithm.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It should be possible to reproduce the results with the given description
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Add images of bad results, which show the problem of the proposed algorithm, so that the explaination you are giving are more comprehensable
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall a good paper, I think the algorithm can be used in many differnent body regions and thus could be interesting for many scientists
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Strengths:
- New segmentation algorithm from multi-view 2D scans to perform knee bone segmentation, with good results
- Clear presentations, with good images
Weaknesses:
- Method may not be novel relative to literature cited by Reviewer #1
- Knee bone segmentation performance is good but cartilage is not. Reviewer #1 makes the case that clinically cartilage is more important.
In the rebuttal, please make sure to address the difference between your approach and the references cited by Reviewer #1; clarify issue of cartilage segmentation performance. In the revised paper, you need to provide visual examples of “poor” segmentation.
Author Feedback
Q1: Novelty of our method relative to literature cited by R1. A: The major novelty of our paper is to generate 3D knee segmentation from multi-view 2D MRI scans. To attain this goal, we process multi-view MRIs to isotropic 3D volume by super-resolution. Then, we build a knee graph and conduct graph-based segmentation in 3D.
Our method has significant novelty compared to the papers mentioned by R1.
Existing knee segmentation methods such as Ambellan et al. (“Automated Segmentation of Knee Bone and Cartilage…”, MedIA 2019) focused on 3D MRIs, whereas we aim to generate 3D segmentation from multi-view 2D scans. Note that 3D knee scans are relatively limited concerning the cost and time. As 2D MRIs are very common in clinical practice, our method has good potential to apply to many real data.
Zhuang et al. built knee graph for disease classification. Our work differ as we target on 3D segmentation, and thus we propose new way to build the graph. Particularly, we need to fuse multi-view MRIs for fine-grained segmentation. So we apply super-resolution to convert each of the multi-view MRIs to be isotropic. Then we sample a knee graph in the isotropic space, where the number of vertices is fewer than Zhuang et al. and each vertex has a more powerful 3D signature to incorporate all views. Our experiments show that the new graph leads to high performance in 3D segmentation.
Liu et al. produced multiple segmentation masks corresponding to multi-view MRIs, which significantly deviates from our method where only a single 3D segmentation is acquired from multi-view MRIs. Furthermore, Liu et al. required annotations from multiple views to supervise training the segmentation network, while our method only needs single-view (sagittal) supervision to generate 3D isotropic segmentation.
Q2: Segmentation performance (R1). A: The OAI dataset is research-oriented, and it has one 3D scan and three 2D scans for each subject. The 3D scan has ground-truth segmentation in OAI, and we recruited a radiologist for manual labeling of 2D scans in this work. The scanning time is very long (30-40 minutes, especially due to 3D DESS sequence), which easily induces subject motion.
In Table 2, our segmentation ground-truth comes from 3D scan. If the input multi-view 2D scans have motion with respect to 3D scan, then the Dice ratio will be relatively low. Specifically, we have checked all data, and found ~20% subjects in our dataset have noticeable motion across different sequences. If we exclude these subjects, the Dice ratio for femur cartilage (FC) can rise to 87.3% compared to 81.7% in Table 2.
Note that our score is based on training from multi-view 2D scans with sagittal supervision only. On the contrary, Ambellan et al. worked on 3D scans directly, and their Dice ratio for FC was 89.4%. The above results show that subject motion is key factor that downscales our reported performance. We will release the indices of these subjects with motion in OAI as supplementary to our paper, to help other researchers pay special attention to them.
Q3: Visualization of “poor” segmentation (AC, R3). A: The bad case is related to the subject motion issue above. We will provide visualization results in the revised paper. And we will also provide the list of subjects in OAI with non-negligible motion.
Q4: Writing misunderstandings and mistakes. (R2) A:
- P2 “scans in local for” -> “Zhuang et al. introduced a unified knee graph architecture to fuse the multi-view MRIs in a local manner for knee osteoarthritis diagnosis.”
- P5 “added to the origin feature H by broadcasting” - > “H3 is reshaped to match the shape of H and then summed up, serving as the input to the UNet decoder.”
- P5 “is utilized to our method” - > “ is used in our method”
- The rest of writing mistakes will be corrected as R2 suggested, and we appreciate for the valuable feedback.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I believe that the authors have addressed reasonably the concerns raised initially by Reviewer 1, and they outline the novelty in approach and labeling requirements.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a high-res 3D super-resolution segmentation from low-res 2D MRI scans from multiple orientations by exploiting a surfacic knee graph representation. The reviews highlight the applicative approach reusing multiple concepts, but also critize novelty and question the clinical performance of key kneee segmentation. The rebuttal partially clarifies novelty but may be insufficient to convince on the clinical performance. The multiview fusion to produce a super-resolution segmentation is appreciated, but the novelty remains the main weakness of the paper, which also miss a validatoin support with sufficient clinical performance. These aspects should be strenghtened in a follow-up submission. For these reasons, and situating the work to other submissions, the recommendation is towards Rejection.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The major concern of R1 is the novelty of the work. The authors have made substantial efforts in the rebuttal to clarify this point. The authors have also promised to add bad-quality images. For the final version, it is advisable to add the distance metrics that R1 mentioned in the response.