List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Sunyi Zheng, Jingxiong Li, Zhongyi Shui, Chenglu Zhu, Yunlong Zhang, Pingyi Chen, Lin Yang
Abstract
Karyotyping is an important procedure to assess the possible existence of chromosomal abnormalities. However, because of the non-rigid nature, chromosomes are usually heavily curved in microscopic images and such deformed shapes hinder the chromosome analysis for cytogeneticists. In this paper, we present a self-attention guided framework to erase the curvature of chromosomes. The proposed framework extracts spatial information and local textures to preserve banding patterns in a regression module. With complementary information from the bent chromosome, a refinement module is designed to further improve fine details. In addition, we propose two dedicated geometric constraints to maintain the length and restore the distortion of chromosomes. To train our framework, we create a synthetic dataset where curved chromosomes are generated from the real-world straight chromosomes by grid-deformation. Quantitative and qualitative experiments are conducted on synthetic and real-world data. Experimental results show that our proposed method can effectively straighten bent chromosomes while keeping banding details and length.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_12
SharedIt: https://rdcu.be/cVRvw
Link to the code repository
https://github.com/lijx1996/ChrSNet
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The proposed method straightens images of curved chromosomes, to allow easier analysis of their properties. It applies a DNN, self-attention layers, a U-Net refinement layer, and a loss function tailored to the use-case. Ironically, a key finding (though not acknowledged as such) is that the extra architectures beyond the basic DNN have little-to-no effect (table 2). To the authors’ credit, they do not obscure this finding.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The motivation is well-stated. The prior work is well and succinctly covered (I do not have expertise to assess if it is complete). The loss function is well-described and thoughtfully incorporates domain-specific metrics. The clarity of the paper is good.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The use of self-attention is not well-motivated. In particular, it is not clear that CNNs’ inability to model long-range relationships matters in this use-case, and the division of spatial patches into a 1-D sequence (to enable application of attention methods) seems like a backwards step - throwing out highly salient spatial (neigbor) relationships.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The main text makes no mention of code being available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The loss function is well-described and thoughtfully incorporates domain-specific metrics. Reading the results, I wondered whether length is an important metric - does it affect the later analysis of chromosomes, or is the most important point clear banding?
As mentioned above, is the use of attention networks indicated for this use-case? (a) are there any salient long-range relationships that a CNN will not encode? (b) Putting the patches into a sequence (pg 4) seems to lose ground, since relevant spatial neighbor relationships are lost.
In the results (table 1), evaluating length and straightness is arguably a bit unfair, since this is exactly what ChrSNet explicitly optimizes. LPIPS is a more neutral metric - does it relate meaningfully to the needs of later analysis? (I don’t know). On a related note: Are there metrics about clarity of banding that derive from the needs of later analysis, that would make good assessment metrics?
Providing uncertainty intervals in the tables is well done - a vital element for assessing the findings.
A key finding, seen in table 2, is that the various added-on architectural blocks do not add value vs using just RG (all accuracies vary by less than 1 std dev). It seems that this should be discussed. In fact, I would argue that the paper could be restructured to decrease the description of the other blocks, because they were experimental deadends (in the best sense) - worth reporting, but in the context of “ideas that did not work out”.
Miscellaneous: -deep features: what does this mean? -a type from 1 to 12: what does this mean (unclear to a domain outsider) Fig 2: what is the stippling artifact in UNet Real-world (bottom row, second from right)? Fig 1: y is supposed to be a straightened version of x. Perhaps modify the figure to show this, for clarity.
Miscellaneous typos: complimentary -> complementary flat it into -> flatten it into agained -> again slop -> slope -Unet -> U-Net, or UNet the from -> from -parts[19] -> add space A careful review of grammar would be good (eg some definite articles are dropped). As an aside: if English is not your first language, I congratulate you on a far better command of English than I will ever have of your native tongue, whatever it is :)
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Much of the paper describes architectures that were shown by the experiments to be of little value, and the lack of added value of certain architectures is not discussed. While negative findings are useful to report, the paper should be shaped accordingly. There were quite a few typos.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The paper proposes a novel chromosome straightening approach using self-attention guided networks. The method combines low-level details and global contexts to recover banding patterns. This study creates mappings between straight and curved chromosomes for chromosome straightening.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelty: This study creates mappings between straight and curved chromosomes for chromosome straightening for the first time. Experimental evaluation: well conducted and presented. Much appreciated the comparison.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I would suggest to the authors to improve the introduction so that it provides a deep overview of the study. In fact, I think it is unclear what unique challenges are associated with this task.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibily is adequate. Perhaps, the authors could give more details regarding the key parameters involvedin their method.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Dear Authors, I read your manuscript with great interest and I found it of very good quality. Also the results are quite impressive and opens the field for further improvements. I congratulate with you also for the conduction of the experimental evaluation: very clear and well presented. I have no major concerns. My only suggestion is the following: I would suggest to improve the introduction so that it provides a deep overview of the study. In fact, I think it is unclear what unique challenges are associated with this task.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Novelty of the proposal, and novelty of the task. Extremely well conducted experimental evaluation.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
this method introduces a new neural network based machine learning method for chromosome straightening, with good contribution of its machine learning method and its biological impacts.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
the beauty of this paper is about formulating an important biological application into a pixel-to-pixel prediction problem, solved by a simple but effective learning paradigm.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
One major issue is that for real-world validation, it is not clear if ground truth is available or not. I am a little confused. Meanwhile I have two minor comments:
-
Fig 2 is a little bit contradictary to the results in Table 1, or at least may not fully represent the evaluation in Table 1. For example, to my eyes, the Unet result on synthetic data seems to have better S score and L score than ChrSNet, at least for this example.
-
Why Unet performs so badly on real-world example, but does a very good job on synthetic data?
-
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
In the authors’ answer to the reproducibilty checkliost, the authors claim to include all the code. But, I don’t see any placeholder in the manuscript.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Both predictions from Unet and ChrSNet are more or less very noisy, comparing to real images. I think adding some adversarial training to the learning step would help further imcrease the performance.
For Fig 2, I would suggest report the S score, L score and LPIPS for the example.
In general, I would suggest to do more investigation on why Unet performs so badly on real-world example, but does a very good job on synthetic data
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is good. I am willing to change from 6 to 7 or 8, if the AC can verify on the code release and reproducibility.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a deep learning based method for straightening curved chromosome images to enable better analysis. All three reviewers agree that this is of interest to the the MICCAI audience. However, R1 brings up several concerns regarding clarity and motivation. R3 also brings up a concern about contradicting results and figures. Please clarify these in your rebuttal.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Author Feedback
We would like to thank the reviewers and the AC for their constructive comments. Please find our responses below: Q: Motivation for the use of self-attention (R1). A: Banding patterns at the bottom and top of a chromosome are in the distance but correlated. So, we use self-attention (SA) to learn long-range relationships (LRR) between them. Results show that without SA, CNN fails to recover patterns at the bottom and top on real-world data. Division of patches did not throw out spatial info as patches including patterns in the 1D sequence still have LRR. Our results show adding SA with division of patches is effective as more pattern details are restored compared to UNet (LPIPS: RG [SA+UNet] in Table2 and UNet in Table1). Q: Importance of chromosome length and banding patterns (R1). A: Length is an important metric that affects chromosome classification. If length is accurately persevered, types of autosomes can be acquired since autosomes are arranged in order of length. Banding patterns are also important for later analysis, especially for sex chromosome classification and abnormality identification which cannot benefit much from the info of length. Q: Evaluation metrics (R1). A: Length can determine types of autosomes and straightness reflects deformation levels which affects difficulties for pattern analysis. Both metrics are important to be evaluated for later analysis. LPIPS is neutral and useful for later analysis as a higher value means more banding patterns restored. Although LPIPS slightly decreases in ChrSNet, it is still comparable to the baseline result and therefore, will not significantly affect later analysis. In our future work, we will improve pattern restoration. Besides, we find another metric, density profile which records pixel values along the chromosome skeleton. We consider this metric is a good one to evaluate clarity of patterns in specific regions. Q: Added value of extra architectures (R1). A: In Table 2, the results always show an increasing trend for S and L scores on both two sets after adding extra architectures. Based on a relatively high baseline, the S score improves by 2.3% at most, while the L score increases by 3.0% at most. Of note, standard deviations are smaller with more added architectures, which suggests the results become more stable. Therefore, adding extra architectures is helpful to improve model performance and robustness. Q: Unclear associated challenges with this task (R2). A: Main challenges of this task include: 1.Lack of images of the same chromosome with straight and curved shapes for training; 2.Limited model generalizability; 3.Inconsistent length and inaccurate patterns after straightening. Q: Explanation for the bad performance of UNet on real-world data (R1,R3). A: The reason is that real-world data is more complex than synthetic data and generalizability of Unet is poor without SA on complex data. To verify this, we did several experiments. We found when taking UNet with various model sizes for straightening, the performance on real-world data is always poor except when real-world data is relatively close to synthetic data. But applying both SA and the same UNet produce results without stippling artifact on real-world data. Q: “Contradictory” results of UNet between Table 1 and Fig. 2 (R3). A: The results on synthetic data in Table 1 are based on the curved chromosomes. We did results based on the ground truth and will add them to the table. We found ChrSNet performs worse compared to UNet based on the ground truth on synthetic data, however, it shows much better performance on real-world data. Q: Availability of ground truth in the real world (R3). A: The ground truth is not available in the real world since it is hard to collect images of the same chromosome with both straight and curved shapes. Q: Code availability and reproducibility(R1,R2,R3). A: We have released our code and trained model. The link is https://anonymous.4open.science/r/MICCAI-1588/README.md
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I think the authors address the issues that the reviewers bring up regarding code availability, clarity and motivation in the introduction and explain why UNET performs worse on real world data in the rebuttal. From the Reviewer discussion, it also appears that all three are now in favor of acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Strenghts of the work include the novel approach to an important problem, well crafted loss function that is derived from biological domain knowledge, and convincing evaluation. The weaknesses identified during review (motivation of the attention mechanism and some clarifications) were addressed sufficiently well in the response, making me vote for accepting this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper introduces a self-attention neural network for chromosome straightening, which can extract global textures and local details to straighten bent chromosomes. Given the importance of the problem, the proposed method potentially has a high biological impact. In addition, the rebuttal has addressed some reviewers’ major concerns, e.g., the motivation of using self-attention and the interpretation of experimental results.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6