List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Chi Zhang, Qihua Chen, Xuejin Chen
Abstract
Morphological analysis of various cells is essential for understanding brain functions. However, the massive data volume of electronic microscopy (EM) images brings significant challenges for cell segmentation and analysis. While obtaining sufficient data annotation for supervised deep learning methods is laborious and tedious, we propose the first self-supervised approach for learning 3D morphology representations from ultra-scale EM segments without any data annotations. Our approach, MorphConNet, leverages contrastive learning in both instance level and cluster level to enforce similarity between two augmented versions of the same segment and the compactness of representation distributions within clusters. Through experiments on the dense segmentation of the full-brain EM volume of an adult fly FAFB-FFN1, our MorphConNet shows effectiveness in learning morphological representation for accurate classification of cellular subcompartments such as somas, neurites, and glia. The self-supervised morphological representation will also facilitate other morphological analysis tasks in neuroscience.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16452-1_10
SharedIt: https://rdcu.be/cVRYN
Link to the code repository
https://github.com/zhangchih/MorphConNet
Link to the dataset(s)
https://github.com/zhangchih/MorphConNet
http://fafb-ffn1.storage.googleapis.com/data.html
Reviews
Review #1
- Please describe the contribution of the paper
Submission 2634 proposes a novel method to learn a representation of the morphology of 3D objects segmented from EM volumes. The method is self-supervised, validation is performed on a large manually annotated dataset of neurite fragments. Comparison to state-of-the-art shows the advantages of the method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Strengths:
- As far as I’m aware, the idea of class-instance contrast is novel
- Excellent validation results, proper comparison to state-of-the-art
- The paper is well written and well illustrated
- The method itself is not specific to EM and is likely to be applicable to other problems which require a representation of morphology.
- A new dataset will be made available to the community
- Failure cases are demonstrated, limitations are discussed
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Weaknesses:
- Only one (although very big) dataset is used. Will the learned representations transfer to another dataset without retraining?
- The classification problem of soma/neurite/glia is not that difficult. It would interesting to test the method on the more common axon/dendrite/soma classification problem in mammalian brain data.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
New dataset will be made available!
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I realize these are impossible to achieve within the rebuttal timeline and would still recommend acceptance. However, if the authors want to improve the next extension of the method, I would recommend the following:
- As noted above, I think it would be important to evaluate if the learned representations have to be retrained when a different dataset is used for input.
- Besides the evaluation on a different dataset, preferably a mammalian one with more difficult classes, I think it would be interesting to compare more directly to the methods of [15] and [20].
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A new method, full evaluation, very good results.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper proposes a new self-supervision method for point cloud representation with the application for neuron subcompartment classification. The proposed method combines existing self-supervised methods for images, BYOL and SwAV, with minor modification. On its own dataset, the proposed method outperforms baseline methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The technical writing is clear.
-
Contribution of a new dataset.
-
The proposed method outperforms previous baselines.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
Lack of novelty: The proposed method is a direct combination of BYOL and SwAV with minor modification. It’ll be good if the paper can directly re-use the terminology from the previous work and put things in context. Otherwise, the paper reads as if it proposes all these new modules.
-
Lack of reference and comparison with state-of-the-art point cloud self-supervision methods [A]. BYOL and SwAV were designed for image input, while [A] is more proper for comparison.
-
Lack of comparison with prior work. The proposed method can be more convincing if compared with prior work [15]. (1) Although the input field-of-view of [15] is much smaller, it can still be trained on the proposed dataset. (2) Also, it’ll be great to test out the proposed method on [15] dataset.
-
Unclear how hard the proposed benchmark is. It’ll be good to use handcrafted 3D point cloud features as a comparison. For example, the scale of the soma is vastly different from the shown neurites and glias, which makes it a simple task with even handcrafted features.
[A] Xie et al. PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding. ECCV 2020
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The supplementary material provides enough details for reproduction.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
Writing: The paper can be clearer if it can directly attribute modules to previous works and emphasize on the proposed new pseudo cluster label supervision. In its current form, it’s unclear to novice readers which is this paper’s contribution.
-
Experiment: The paper only provides ablation study comparisons. It’s unclear how it compares with previous neuron subcompartment classification methods or state-of-the-art point cloud contrastive learning methods.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I really like the topic of this paper, which applies contrastive leaning approach to learn point cloud features for 3D neuron morphology. However, the writing does not make it clear which part is from the previous work and which is new contribution. In addition, the experiment section lacks comparison with prior work on either the same task or the point cloud contrastive learning methods.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
-
Novelty. The proposed method uses the linear combination of two losses, L_CIL and L_IIL. (1) L_IIL is exactly the same as BOYL, and (2) L_CIL is a simpler version of SwAV., i.e. use NN classifier instead of the optimization in SwAV to estimate the code q. I understand that the proposed L_CIL is different from SwAV, but there is little justification on why not directly using SwAV’s formulation. It’ll be convincing to have a head-to-head comparison to motivate the choice of NN classifier instead of SwAV’s approach for code q prediction. Conceptually, there
-
Lack of comparison. As mentioned above, the proposed method is a simple change of BOYL+SWAV. The improvement upon BYOL is small 0.003 and BOYL+SwAV may have similar/better results without the need of the new L_CIL. Also, the previous methods [15,20] can be run without the additional organelle channels and the skeleton extraction is standard.
-
The writing in its current form directly take the formulations from BOYL and SwAV without proper reference, e.g. L_IIL is exactly the same as BOYL.
-
Review #3
- Please describe the contribution of the paper
The authors present a self-supervised approach for learning 3D morphology representations from ultra-scale EM segments. The proposed method leverage contrastive learning at both instance level and cluster level to learn the representation. Experiments on the over-segmented results FAFB-FFN1 outperforms other contrastive learning methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well-written and easy to follow. The first self-supervised method for 3D EM segments. The motivation is clear. The motivation is clear to integrate the feature correlations at the instance level and cluster level.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Missing the quantitative ablation study for the cluster/instance level contrastive learning. Missing the description of the choice for the hyper-parameters.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method part is clear. But it lacks the discussion for the choice of hyper-parameters.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
In general, the paper is well written and easy to follow. The motivation is clear. The motivation is clear to integrate the feature correlations at the instance level and cluster level. However, it misses the quantitative ablation study for the cluster/instance level contrastive learning. Although BYOL and SwAV can be seen as the instance-level and cluster-level methods, the training set may be different from the proposed method. It is better to add a comparison of the proposed method w/o L_{CIL}/ L_{IIL}.
Others:
- For the cluster initialization methods. Do KMEANs and DBSCAN have a large difference?
- Why choose K-means instead of the Sinkhorn-Knopp algorithm?
- Does the choose of /beta affect the performance?
- How to set K?
- How to choose the size of the memory bank?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation is clear. The motivation is clear to integrate the feature correlations at the instance level and cluster level.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
There are non-converging review recommendations. The authors are encouraged to address esp. the issues raised by the reviewers including the novelties & technical contributions (clarify how it is not a combination of existing methods of BYOL and SwAV), empirical evaluations (e.g. work with more than one dataset), presentation (e.g. clarify the motivation and challenges of the task), use references, among others.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Author Feedback
We thank all reviewers for their thoughtful feedback, especially their recognition of the novelty and generality of our method. Here we will first clarify some common concerns and then answer individual questions of each reviewer. [MR] The novelties. [A] In this paper, we propose an unsupervised contrastive-learning method to learn representations of neuron morphologies from massive unlabeled EM segments. Our contrastive learning method is not a simple combination of the BYOL and SwAV. The proposed cluster-instance module is fundamentally different from the clustering-based instance contrast in SwAV. SwAV employs clustering to enforce the consistency between cluster assignments produced for two augmentations. However, we do NOT compare the cluster assignments of TWO augmentations. Instead, we use clustering in the feature space to enforce the in-cluster concentration and between-cluster dispersion in the representation vector space. Our cluster-instance contrastive loss compares the hard cluster assignment and the soft cluster assignment of the same augmentation. There is no “swapped prediction” in our cluster-instance module. [MR] Challenges of the task. [A] Our motivation is to learn an effective morphological representation for massive over-segmented 3D pieces in EM images. We agree that distinguishing somas from other categories is not that hard. However, the classification of neurites and glia according to their 3D morphology is much more challenging, as Table 1 shows. It is non-trivial to manually design features. Our method is effective in representing the diverse and complex morphology on the whole-brain scale. [MR] Comparisons with more related works and more datasets. [A] In terms of the task, we have a similar goal with [15][20]. They both take the data in a local volume (884 μm^3) at each node on extracted segment skeletons as input. They also require prediction results of cell organelles, such as mitochondria and vesicles. In contrast, we only use the surface points and learn the morphological representation of the entire segments (up to 300 μm in one dimension). In terms of methodology, PointContrast [A] is related to our work. However, PointContrast aims to learn point-level embedding in indoor scenes. It takes point cloud scans at different views as data augmentation. The code for the classification task is not provided. Due to the time limitation of rebuttal, we tested our model trained on FAFB-FFN1 directly on ShapeNet with linear evaluation and obtained favorable results: (OA/availability of training data) 0.634/1%, 0.782/10%, compared to baseline model trained from scratch: 0.622/1%, 0.779/10%. It shows that even pre-trained on a distinct dataset, our model can be well generalized to another domain. Compared with the finetuned results of PointContrast pre-trained on ScanNet with PointInfoNCE loss: 0.658/1%, 0.788/10%, our results are slightly worse. However, our model is pre-trained on a biological dataset which is more distinct from ShapeNet. Besides, our results are based on linear protocol. [R3] Hyper-parameters and quantitative ablation. [A] We set beta=0.9 and K=16 empirically. We set beta as 0.9 following many existing studies that use momentum for parameter updating. According to the review comments, we conducted experiments on different values for beta (0.0, 0.5, 0.9, 1.0) and obtained OAs (0.911, 0.926, 0.935, 0.614) respectively. We choose N_Q=4000 which is the upper bounder allowed by our hardware memory. In Table 1, we reported the accuracy of BYOL, which is identical to our method w/o L_{CIL}. [R3] Choice of the clustering algorithm. [A] Our framework does not have specific requirements for the clustering algorithm. Since online clustering for each mini-batch is not needed, we do not use Sinkhorn-Knopp. We choose the k-means for simplicity. We conducted experiments by using K-Means and DBSCAN for clustering and obtained similar results (OA: 0.935 for K-Means, 0.934 for DBSCAN).
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper deals with segmentation of 3D EM images with self-Supervised contrastive learning, with good empirical results. There has been a major concern on the novelty. With the rebuttal, I feel it could be justified based on the new application to EM data, as well as the detailed differences. Overall it could be an interesting contribution to the community.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
11
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Two of the reviewers voted for acceptance of this paper and one gave a weak reject. The main criticism from the negative reviewer is on lack of technical novelty. While there are similarities to other self-supervised learning methods, I think there is sufficient difference and more so when considering the application. I think the rebuttal did a good job of addressing the concerns.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper studies the problem of learning 3D morphology representations from ultra-scale EM segments by a self-supervised method. While the studied problem is interesting, the novelty of the paper is concerning. While the authors’ rebuttal has provided some additional support, I concur with Reviewer #2 in that the formulation is very close to the existing works and insufficient justification on novelty is provided. In addition, as pointed out by the reviewers, some experimental details should be better provided.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
10