Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

David Lüdke, Tamaz Amiranashvili, Felix Ambellan, Ivan Ezhov, Bjoern H. Menze, Stefan Zachow

Abstract

Statistical shape modeling aims at capturing shape variations of an anatomical structure that occur within a given population. Shape models are employed in many tasks, such as shape reconstruction and image segmentation, but also shape generation and classification. Existing shape priors either require dense correspondence between training examples or lack robustness and topological guarantees. We present FlowSSM, a novel shape modeling approach that learns shape variability without requiring dense correspondence between training instances. It relies on a hierarchy of continuous deformation flows, which are parametrized by a neural network. Our model outperforms state-of-the-art methods in providing an expressive and robust shape prior for distal femur and liver. We show that the emerging latent representation is discriminative by separating healthy from pathological shapes. Ultimately, we demonstrate its effectiveness on two shape reconstruction tasks from partial data. Our source code is publicly available.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_44

SharedIt: https://rdcu.be/cVRsc

Link to the code repository

https://github.com/davecasp/flowssm

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper propose a novel shape model based on continuous neural flows, which produce natural shape deformations without relying on dense correspondence between training shapes

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    flow parametrization and latent representation

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    the comparative experiment is lack.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Yes

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
    1. The innovation of the paper is weak
    2. How about the time complexity about your model compared other methods.
    3. Comparative experiments are slightly inadequate
    4. There are many expressions in the full text that I cannot understand Such as the last two lines in page 7, and so on. Pls revise the manuscript carefully.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The experiment results may be satisfying although the experiment is not sufficient.

  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    2

  • Reviewer confidence

    Somewhat Confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Not Answered

  • [Post rebuttal] Please justify your decision

    Not Answered



Review #4

  • Please describe the contribution of the paper

    This paper proposes a novel shape space representation based on diffeomorphic deformation of templates parameterized by the PCA codes of the latent space of a MLP. The authors show on 3 examples that this shape space is expressive and is suitable to discriminate between healthy and pathological cases. The representation generalizes the shapeflow approach by considering multiscale deformations through the addition of a local latent representation

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper is well written
    • It introduces a novel representation of shape spaces (compared to shapeflow) by adding a local neural flow deformer to a global one. This local NFD is based on RBF with fixed control points.
    • The authors provide examples of their approach on 2 datasets liver and femur and show the discriminative power of the latent space on one test case.
    • The author provide a comparison with 3 other techniques one of them being also based on LDDMM. They show that their technique leads to improved results.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The novelty introduced in this paper is somewhat limited to the i) addition of the local NFD compared to the shapeflow approach and ii) the use of PCA modes in the latent space instead of the whole latent space.
    • The authors do not compare their approach with that of shapeflow from which it is based on. It is therefore difficult to check whether the additional shape space refinement are important or not.
    • The proposed approach for NFD is fairly complicated as it corresponds to the PCA modes of the control points of an RBF function describing a latent space which is itself parameterizing a velocity flow.
    • The authors do not provide a lot of implementation details. For instance, they do not describe the optimization technique to perform inference of the shape representation. Why choose an encoder free approach ?
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Many implementation details are missing (see above) such as : the number of RBF basis considered, the optimization methods, the number of PCA modes considered and how this number was chosen, the number of sampling points…

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    To improve their paper, the authors should i) compare their approach with shapeflow ii) detail more the implementation parameters of their method iii) the computation times of the method

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Albeit its complexity and the lack of implementation details, the introduced shape space is of high interest for the community.

  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    4

  • [Post rebuttal] Please justify your decision

    The authors have not addressed the issue of comparison with shapeflow. Their approach is fundamentally complex and overengineered and the paper may provide more noise than information to the MICCAI audience



Review #3

  • Please describe the contribution of the paper

    Authors present a novel method for shape modeling based on neural flow deformations.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The proposal is based on a continuous model of a velocity field in 3D. The differential equation is solved via a parametrization model solved by a flow net. Results seem correct and applications to shape reconstruction, segmentation and classification.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The mathematical method is described very superficially. It is difficult to check correctness and novelty. The idea of solving a differential equation by parametrization, using a neural network is not novel, but the application to shape modeling is interesting.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    It is very difficult to reproduce the paper because of its shortness explaining the method. The supplement is not enough either for this purpose.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    Authors should provide more details that allow fully reproduction of the method. The method is very well described in a general overview. It is elegant, but more details are needed to fully undersand the math background.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The apllication to shape modeling is interesting. It is difficult to judge novelty beacuse of the shortness of the paper, but the proposal is interesting. I think the paper would turn into to a good topic for discussion in the conference.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Not Answered

  • [Post rebuttal] Please justify your decision

    Not Answered



Review #5

  • Please describe the contribution of the paper

    This paper claims to propose a novel shape modeling approach based on neural flow deformations that can eliminate the need for correspondences within training samples. The approach is evaluated using lever and distal femur dataset publicly available and reports the comparison of proposed method with the state-of-the-art methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper is well written with adequate citations and explanations at places. The proposed formulation is well explained in terms of its parameters and in terms of their usage. The evaluation performed with the publicly available dataset is clear and strongly presented. Overall, the manuscript is very well organized with appropriate supplementary material.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    While the proposed approach seems novel at the first glance, this impression fades away as one digs through the citations provided. The authors claim that the approach is novel based on continuous neural flows. However, continuous neural flows have been previously defined and established by Chen RTQ (Reference 7 in the manuscript) and further explored by Jiang et al (Reference 17 in the manuscript). To this reviewer, there is no novelty involved in the study and it is a adaptation for specific dataset of lever and distal femur. The elimination of correspondence is not the target of the method but comes as an intrinsic nature of the continuous neural flows. The paper uses “hub-and-spokes” approach already established by Jiang et al (Reference 17) to generate correspondence free template. Paper uses IM-Net established (and made available) by Chen et al (Reference 8 in the manuscript) for multilayer perceptron set-up. So, in short, authors have put-up multiple previously published methods in this paper in an attempt to call it a novel approach, however, this reviewer strongly disagree.

    Other weaknesses include the lack of explanations on the usage of particular method or parameters at a particular step. For example, it was not clear how control points are selected or why the specific Gaussian kernel was selected. It was also not mentioned why the latent vector size was 128 which was similar to the previous publication.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Authors have used publicly available dataset, hence it would be available for anyone else as well. Authors claim that they would provide the code upon acceptance of the manuscript, so it would be easier to reproduce the experiments.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    This is a very nicely written and organized manuscript. However, the lack of novelty is my major concern. As already explained in the weakness of the paper, the proposed approach is developed by combining multiple existing techniques and methods from the literature. The authors should have focused on the application side of it by reducing the methodological content and increasing its focus on “Computer Assisted Intervention”.

    Because authors are trying to fit in many methods within limited space, many explanations pertaining to each adopted method have been missing. For example – 1) How do you decide/control the flow magnitude? What is its sensitivity? 2) If you have global and local level deformers, then why they are used sequentially? What happened to the global and local PCA models (you claimed to develop separate PCA models)? 3) How was your z-vector defined in terms of its size? 4) Adding Guassian kernel makes your model Heuristic. How did you control or explore the heuristic nature of your model? 5) Template selection was done using “hub-and-spokes’ approach in a canonical space. How did you deal with the bias that may get induced in selecting the template? 6) For evaluating specificity, you use PCA to generate random surfaces, but specificity was reported using chamfer distance with the closest example – details onto how this was done were not provided.

    These and many other such questions remain unanswered.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the manuscript is written and organized well, it lacks the required novelty.

  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    3

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Not Answered

  • [Post rebuttal] Please justify your decision

    Not Answered



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper introduced several novel ideas. However, there are several limitations. The method was not described in detail. Details about the neural network architecture and implementation are missing. The experimental evaluations are lacking. The authors incorporate ShapeFlow in their work, but comparison to ShapeFlow is missing. PDM and FCM give comparable results. Further, the performance of a deep neural network implementation isn’t that significantly lower from PDM, which is a linear approach. One assumes the PDMs just used a dense linear correspondence without identifying any key landmarks? This should be clarified. Otherwise the results are not overall convincing.

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    8



Author Feedback

We thank the reviewers for their appreciation of our “elegant [method]” [R3], producing “natural shape deformations without relying on dense correspondence” [R1], introducing “several novel ideas” [MR], being “of high interest to the community” [R4], leading “to improved results” [R4], with a “clear and strongly presented” evaluation [R5] and introducing “a good topic for discussion in the conference” [R3].

The novelty of our approach is the improved solution to the following MIC problems:

  • building a shape model without relying on correspondence (unlike PDM/FCM) that accurately represents anatomical structures (better than PDM/FCM/LDDMM/ShapeFlow)
  • osteoarthritis classification, reaching state-of-the-art accuracy, but without relying on correspondence

  1. We’re better than PDM/FCM [MR] The novelty of our solution for the shape modeling task w.r.t. PDM & FCM is that we do not require given dense correspondence. As MR rightfully points out, reliance on correspondence represents a significant ambiguity for PDM/FCM and is the exact problem we solve in our approach. For the PDM/FCM, we followed the common but tedious practice of manually annotating key landmarks [1,4,18]. Moreover, our method strongly outperforms the linear PDM by ~20% on the liver in Generality & Specificity (Tab 1). An even larger improvement is in the number of self-intersections, revealing limitations of PDM & FCM. The femur, as a bone, has less variation in shape and therefore shows a smaller improvement over baselines. Overall, the proposed model outperforms all baselines significantly on both datasets in Generality & Specificity (paired t-test, p < 0.05) and is on par with state-of-the-art [4] in osteoarthritis classification without relying on dense correspondence.

  2. We’re better than ShapeFlow (SF) [MR,R4,R5] The main novelty of our method compared to SF is the increased expressiveness – essential for anatomical structures. We achieve this by extending SF’s globally parametrized deformation by a consecutive local deformation. This extension nearly halves the error in approximating unseen shapes compared to SF’s global parametrization (Tab S3 suppl.). Moreover, the added local latent representation significantly improves osteoarthritis classification (72.3% vs. 93.4% for 90/10 train/test ratio). Furthermore, SF deforms a source shape from the training set into a target shape. Their formulation disentangles style from structure, retaining details of the source shape throughout the deformation. However, keeping patient-specific details of the source shape, when reconstructing new shapes, is not desirable for anatomical shapes. Hence, we have opted for a common template as source shape.

  3. Method Details [MR,R3,R4,R5] We regret that we had to be brief in our description. Architecture, optimizer, and more details are listed in Suppl. (Fig S1, Tab S1+S2). Given the limited space, we aimed for a concise description in favor of experiments. We felt that these are valuable for the reader, showcasing the method’s strengths from various angles. To ensure reproducibility, we will publish the source code (upon acceptance).

  4. Our model is fast [R1,R4] The training takes ~2h & 4h (liver & femur); generating new shapes is ~0.14s per shape.

  5. Template Bias [R5] Our setting exhibits a natural relationship to SF, esp. in the global deformation. Hence, we employ SF to approximate the mean w.r.t. the data distribution. Although technically not a mean shape w.r.t. our full model, this shape can be assumed to be very well-centered, allowing practically for an unbiased deformation setup.

  6. Gaussian Kernel in RBF [R5] While any positive definite function could be used, the Gaussian kernel is a natural and common choice [12]. The kernel performs well on both datasets and arguably generalizes to further anatomical structures. The dataset-specific kernel width is determined based on the training loss.

We will revise the manuscript to clarify points 2, 3 & 4 adequately.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    This was a borderline paper with the rating fluctuating between weak reject to weak accept. The paper introduces several novel ideas. The authors do a good job of addressing concerns in the rebuttal. While the authors have not included the comparison to shapeflow in the main paper, they comment on the results from shapeflow in an application to osteoarthritis classification.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    11



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The shape modeling application is interesting. This work builds heavily on existing methods and hence methological novelty is considered limited to the introduction of correspondence-free shape modeling that is demonstrated in a classification downstream task that achieves SOTA accuracy. This work can be of interest to the MICCAI community.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    4



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The overall evaluation of this paper is more negative than positive. One of the reviewers downgraded the original recommendation from ‘weak accept’ to ‘weak reject’ after rebuttal.

    While I believe the idea of learning latent shape space that captures anatomical shape variations through neural network is interesting, several major concerns (including but not limited to those brought up by reviewers) make the current manuscript below the minimal bar of MICCAI conference:

    • Limited technical novelty. All reviewers agreed that the proposed network is a combination of previously published work without substantial innovations, including velocity parameterizations, IM-Net, and the loss function based on Chamfer distance. Meanwhile, it is not entirely true when the authors claim their network loss function is correspondence-free. The sampled surface points of a deformed template $\P_{\phi}$ requires correspondence learned by a previously published network IM-Net.

    • The need to perform PCA on the latent representations of training shapes after the training is done seems awkward. If the designed network is only intended to extract shape representations, then why particularly using the proposed approach? There are many deep learning based approaches in the literature that learn shape representations with nicely encoded deformation models.

    • The need to re-optimize the loss function in Eq. (5) during testing is not optimal. Does this mean the learned latent representations is not able to capture the underlying shape variability well? In this case, the model does not have a good generalizability to unseen shapes. In addition, the performance of the re-optimization may be dependent on the number of testing data. It seems a large sample of testing data will be in favor of the inference.

    • Another big question is the number of principal components (PC) selected in this work. None of the experiments includes information of how many PCs were chosen for both the proposed method and baseline algorithms. This is critical since the number of PCs can drastically affect the model accuracy in shape reconstruction and shape classification. It is difficult to justify the proposed work without detailed information on this part.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Reject

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    NR



back to top