List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Chao Xia, Jiyue Wang, Yulei Qin, Yun Gu, Bing Chen, Jie Yang
Abstract
Chromosome recognition is a critical and time-consuming process in karyotyping, especially for R-band chromosomes with poor visualization quality. Existing computer-aided chromosome recognition methods mainly focus on better feature representation of individual chromosomes while neglecting the fact that chromosomes from the same karyotype share some common distribution and are more related compared to chromosomes across different patients. In the light of such observation, we start from a global perspective and propose an end-to-end differential combinatorial optimization method for R-band chromosome recognition. To achieve this, a grouping guided feature interaction module (GFIM) is built for feature aggregation between similar chromosome instances. Specially, a mask matrix is built for self-attention computation according to chromosome length grouping information. Furthermore, a deep assignment module (DAM) is designed for flexible and differentiable label assignment. It exploits the aggregated features to infer class probability distributions between all chromosomes in a cell. Experimental results on both normal and numerically abnormal karyotypes confirmed that our method outperforms state-of-the-art chromosome recognition and label assignment methods. The code is available at: https://github.com/xiabc612/R-band-chromosome-recognition
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_1
SharedIt: https://rdcu.be/cVRvl
Link to the code repository
https://github.com/xiabc612/R-band-chromosome-recognition
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a novel chromosome recognition method to improve the existing recognition performance. To address the recognition issues of karyotypes with numerical abnormalities, deep assignment module is proposed. Also, a grouping guided feature interaction module is proposed for feature aggregation. These proposed modules address the current challenges in chromosome recognition and as a result the proposed method outperforms the state-of-the-art algorithms.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The problem analysis is done well. Then, the method is designed and described in great detail. The proposed DAM and GFIM modules address the problem very well. GFIM enhances the feature aggregation between similar chromosome samples. DAM enhances the recognition of chromosomes with numerical abnormalities.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The loss function used in the neural network is not given in full detail. It is a weighted combination of chromosome grouping loss and no other details are given. The authors should either provide with a reference or a clear formulation for this loss function. This is definitely needed for the reproducibility of the proposed method.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method is described in great detail. However, the loss function definition is missing. Also, the hyperparameters of the Bi-RNN training are missing.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The research is conducted in a proper way, from problem formulation to experimental results. It is nice to see an example of a properly-conducted research and well-written paper.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The problem is defined well. Then, the proposed modules address these problems directly and successfully. Method description is proper and clear. The paper is written and organized well. Finally, the experiments are conducted properly including comparison with others and ablation studies. The contribution of the paper is shown clearly. There are just some minor weaknesses that can be addressed during rebuttals.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
• This paper proposed an end-to-end deep learning method to recognize both normal and abnormal karyotypes, without needing any feature extraction backbone. • A grouping guided feature interaction module (GFIM) is built for feature aggregation between similar chromosome instances to reduce confusion between chromosomes with similar lengths. • A deep assignment module (DAM) is designed for flexible and differentiable label assignment. • An empirical study was performed on a large-scale R-band chromosome dataset collected and labeled by clinical cytogeneticists. The proposed method outperformed competing methods on both normal and abnormal karyotypes.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
• This paper studied an interesting and important imaging computing problem for chromosome identification in karyotyping. The application is relatively unique, in comparison with typical image computing problems studied in MICCAI. • The proposed end-to-end method is innovative. • The empirical study yielded improved prediction performance compared with competing methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
• The chromosome recognition problem is relatively new to the MICCAI community. It was not clearly specified in this manuscript. • It is unclear why the input data can be treated as a sequence. There is a lack of discussion on any spatial, temporal or other sequential information related to the chromosome data. I understand that chromosomes are numbers, but that is the output label the method aims to predict. Where is the sequential information for the input data? • In Table 1, the patient number is not equal to karyotype number. What is the relationship between them? • In Tables 2-3, the performance improvement is minor. The conventional Hungarian algorithm seems to be a good strategy. It is unclear whether the minor improvement would remain for other independent data sets. • There is a lack of ablation study. The individual contributions of GFIM and DAM were not quantified.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Data and code are not available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
• Please address the weaknesses mentioned above. • “one can hardly to identify” should read “one can hardly identify”.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
• The studied application is interesting and under-explored in the MICCAI community. • The computational problem should be clearly described. • The performance enhancement is modest.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors state that they propose a novel approach for chromosome recognition, specifically R-band chromosome recognition. R-band chromosome recognition is, according to the authors, understudied. The authors claim that the method outperforms state-of-the-art for this problem.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is generally well written and concise. The figures and table are mostly self-contained, meaning that they can be understood in isolation, without the context of the surrounding text. The authors are clearly well versed in the technicalities of both the biological side, and the technical side concerning deep learning, this is both a strength and a weakness of the manuscript.
The authors show promising results on a decently sized dataset. The authors compare with reasonable baseline methods and do an ablation study of their proposed approach. The performance is good and generally convincing that the method could indeed be better than what is compared to.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper’s main weakness is that it is telling a complex story in a very short format. The paper combines a couple of fields, (general problem of MICCAI submissions). Here we have a niche within a specific biological problem (karyotyping) and the technical solution (deep learning and graph matching). Quite a lot is assumed from the reader. The problem itself lacks a bit of motivation, but not much. It needs be clearly stated which diseases or disorders benefit from the R-band staining. It is stated that: “R-band chromosome of bone-marrow cells can help identify the abnormalities occurring at the end of chromosomes”. This is not convincing me that the problem is of clinical relevance, name at least one particular case where this is important.
There is an attempt to explain why this problem seems to be understudied compared to work on other staining methods. More blurred bands are mentioned. This also relates to the clinical relevance. I cannot understand whether this problem is interesting because it is challenging or because other staining approaches are superior and thus more data is available for those. The motivation for why R-band is used is also lacking here.
The reported performance is convincing from the tables in the experiments sections. Performance on another dataset, comparing to reported metrics would certainly aid here as well. Some repetitions to provide error bars on the metrics would also aid the reader in assessing the differences between the approaches. The performance needs to be summarized a bit at the end of the introduction, the only mention of results is in the end of the abstract. It is better to have numbers than just stating “state-of-the-art”.
The dataset itself looks like it is not open source. Maybe this is not very specific due to anonymity. The dataset lacks a description of the abnormal cases. Do these all have the same abnormality?
How is G=7 chosen in section 2.2 ?
Equation 4 looks more like programming assignment equal operator than an equation. Consider giving the normalised weights on the left hand side a different name.
An ablation study is required to convince the reader that the DAM is needed, this is done but not mentioned until section 3. This should be mentioned in the end of the introduction.
For the results, the dataset should ideally also be stratified by disease/disorder/abnormality to show whether the improvements are specific to a particular disorder. Maybe there are few cases in the abnormalities for doing this, but this should be addressed.
I cannot see that a specific test set is used for the normal karyotypes. The reported accuracy seems to be from a 5-fold cross-validation, this needs to be more clear, specifically in the table caption.
English language is generally good, one minor comment:
1) Sentence in first paragraph of section 2.1 does not sound right: “For example, one can hardly to identify and distinguish these two chromosomes…” Remove the “to”, and it makes sense.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The results are not reproducible without the code and the dataset. It seems that the authors intend to release the code, but the dataset does not seem to become available.
I do not agree with all the claims of the authors:
1) The authors mark yes to the following: “For new data collected, a complete description of the data collection process, such as descriptions of the experimental setup, device(s) used, image acquisition parameters, subjects/objects involved, instructions to annotators, and methods for quality control.” There is absolutely not sufficient description of this in the paper. The description is just from the point of receiving the images, there is no description of the acquisition process. 2) The authors also mark yes to the following: “Whether ethics approval was necessary for the data.”. I cannot find anything about ethical approvals in the manuscript. 3) I generally do not agree with the statements under section 4), e.g. there is no statistical test for comparing the significance of the difference between the methods, although the authors state that the method significantly outperforms state-of-the-art in the conclusions, this is simply misleading… Also I could not find a description of the hardware used.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please go over the weaknesses and reproducibility issues. The paper is generally good for a MICCAI paper. The reproducibility report does not accurately represent what is in the manuscript, and that needs to be addressed.
I don’t think that you need to run any extra experiments, just be a bit more clear on exactly what you did. Stating that the results are significantly better than state-of-the-art without a formal statistical test or comparison to results on a already reported dataset, is misleading.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is generally well written and the problem is relevant for MICCAI. Some motivation for the problem is lacking and the paper can be slightly better organized. The authors are a bit liberal in the reproducibility report.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a deep learning method for R-band chromosome recognition. The topic of great interest in the miccai community, the paper is clearly written and all reviewers agreed that there is a novel contribution that this paper makes.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Author Feedback
We will use C for comment and A for answer in the following.