Authors

Tianhong Quan, Ye Yuan, Yu Luo, Teng Zhou, Jing Qin

Abstract

The symptoms of neuropsychiatric systemic lupus erythematosus (NPSLE) are subtle and elusive at the early stages. 1H-MRS (proton magnetic resonance spectrum) imaging technology can detect more detailed early appearances of NPSLE compared with conventional ones. However, the noises in 1H-MRS data often bring bias in the diagnostic process. Moreover, the features of specific brain regions are positively correlated with a certain category but may be redundant for other categories. To overcome these issues, we propose a robust exclusive adaptive sparse feature selection (REASFS) algorithm for early diagnosis and biomarker discovery of NPSLE. Specifically, we employ generalized correntropic loss to address non-Gaussian noise and outliers. Then, we develop a generalized correntropy-induced exclusive ℓ2,1 regularization to adaptively accommodate various sparsity levels and preserve informative features. We conduct sufficient experiments on a benchmark NPSLE dataset, and the experimental results demonstrate the superiority of our proposed method compared with state-of-the-art ones.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43904-9_13

SharedIt: https://rdcu.be/dnwGQ

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #3

Please describe the contribution of the paper

The authors propose a robust exclusive adaptive sparse feature selection (REASFS) for use in early diagnosis of neuropsychiatric systemic lupus erythematosus (NPSLE) using proton magnetic resonance spectroscopy (H-MRS). Due the overlap of symptoms with other neurological conditions, NPSLE is a challenge to diagnose. Furthermore, the H-MRS, while suitable, has issues with low SNR and noise as a result of overlapping metabolites, further complication the process of biomarker discovery for early detection and clinical intervention. Authors claim current statistical/ML state-of-the-art is insufficient due to requirement of Gaussian noise, but H-MRS produces non-Gaussian outputs. The proposed method, coupled with correntropy loss, is meant to identify the most suitable biomarkers while minimizing the negative impact of noise and outliers.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Research needs and clinical impact are clearly outlined. The proposed method appears to be robust, and the generalized correntropy loss used minimizes the negative effects of noise and outliers.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Some of the justification for the work presents with mixed messages. Feature weights are not listed to show why metabolites are selected (with quantifiable results focused on proposed method performance as relative to the state-of-the-art). Selected metrics can produced biasedly high results, and therefore new metrics need to be considered.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

In its current state, there are gaps that need to be addressed, but no code or appropriate data accompanies this paper to test for reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Comment 1 : Typographical errors need to be sorted out. For example, in the first line of the abstract (e.g., systemic written twice). Also, the full names for some acronyms are missing. Recommend the authors proofread and make corrections accordingly.

Comment 2: The authors need to be careful with some of their statements. While clinical need is clearly outline, the statement regarding statistical and machine learning methods needs editing. For example, the authors state the issue with current statistical and machine learning techniques are that they assume Gaussian distribution, giving examples such as Mann-Whitney U Test, SVM, Ensemble models, and how these are derived from the minimum mean square error (MMSE). Not all of these methods mentioned necessarily rely on this assumption. For example, the Mann-Whitney U Test can be used to test differences between two samples but does not require normally distributed data. SVM is a regression and does not necessarily assume normality. Furthermore, with in some standard regression models, normality is applied to residuals (i.e., input data can be skewed or non-Gaussian, if the model’s residuals are normally distributed). Yes, some techniques do assume Gaussian noise, but there are others that account for non-Gaussian data. So, this is a very broad and incomplete generalization.

Comment 3: The discussion on the limitations of sparse coding-based methods is good, given that different brain regions have different metabolites and are thus distinct, and would therefore have different sparsity levels. Thus, using the same sparsity constraint across all brain regions would not be sensible. The authors propose the use of a robust exclusive adaptive sparse feature selection (REASFS) algorithm to overcome these issues and to be used in biomarker discovery and early diagnosis of NPSLE. This choice reflects the need for an adaptive method that can adjust according to these different regions.

Comment 4: Dataset acquisition and pre-processing outlined is comprehensive, including quality checks.

Comment 5: The sparse encoding framework is interesting. Its goal is to find the projection matrix, W, that minimizes the function J, which includes a loss function (correntropic loss), regularization term, and the hyperparameter which acts as a trade-off between loss and regularization. There is a focus on the LASSO method and multi-task feature learning, with the authors noting potential limitations with the l2,1. The authors propose the generalized correntropy to account for this limitation.

Comment 6: The loss function, correntropy, is used to minimize the negative effects of noise and outliers. Here, a generalized correntropy is presented, with the Gaussian kernel replaced with a Gaussian density function (GGD), which makes it more robust against outliers. Furthermore, the generalized correntropy acts as a feature ranker.

Comment 7: While the authors compare their method to state-of-the-art (e.g., MIC, Gini), I am not entirely convinced with accuracy as the most appropriate metric of choice. There is no real justification for the use of accuracy as a metric here and maybe the authors consider it a default to use. However, in my opinion, accuracy can be a biased metric, and can misleading provide high results even in situations a model is not performing optimally. I personally would like to see comparisons using recall, F-1, RMSE, etc., instead of accuracy. I would like the authors to re-run their experiments with more suitable metrics (I have given a handful of examples, but I am going to leave it up to the author’s discretion to re-examine and select new metrics, but also provide appropriate justification for their selection).

Comment 8: A crucial point that is also missing for me is that there is a list of metabolic features suggest to contribute to early diagnosis of NPSLE, and then the authors provide a list of these metabolites. In early parts of the paper, the authors focused on non-Gaussian noise as being a driving factor of using this proposed pipeline over other state-of-the-art statistical and machine learning methods. However, other considerations were overlooked, including how those methods can be used to not only extract but rank features based on weighting. The proposed Generalized Correntropy-induced Exclusive (GCIE) component applies different weights to the sparsity constraints on each feature. So, I would anticipate different brain regions, so each feature is expected to be enforced with a sparsity constraint of a different weight. But the quantified weights of these metabolites are not shown in the results. Perhaps this is not inferable in this study, but in existing statistical methods, relative importance can be used rank features and provide a list of ranked features and their weights. If these feature weights are not quantifiable in the proposed work, I would consider this a very real limitation.

Recommendations : The goal of this paper is biomarker discovery through identifying the most discriminative features from noise-prone H-MRS. Some positive points include the data acquisition and pre-processing being comprehensive, the clinical needs clearly outlined, the proposed method aligned with the needs, and the use of adaptive sparse coding framework coupled with a correntropic loss designed to minimize negative effects of noise seem sensible. It is about striking a balance between identifying and keeping important features while removing erroneous or redundant ones. However, disadvantages include generalized statements regarding the capabilities of current state-of-the-art, using metrics which are not sufficient to justify performance, and absence of quantifiable weights to show why the listed metabolites in the results were selected. I recommend new metrics are identified, and clear justification for the use of these metrics presented, and for the selection of the metabolites to be made more clear using quantifiable weights.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

There is merit to the paper, but additional experiments, resuls and justifications are required to convince me this methos is indeed superior to existing state-of-the-art statistical and machine learning methods.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The authors develop a novel, adaptitve feature extraction methodology using generalized correntropic loss formulation of the optimization function. Their method avoids some of the pitfalls of sparse normalization mechnanisms, by introducing the genaralized correntropy in the loss function, and l2,1 and l1,2 norms in the regularization terms to achieve an adaptive feature extractor.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novely includes:
- adaptive selection of terms in the projection matrix with controls to limit the impact of sparse data effects - the Generalized Correntropy-induced Exclusive (GCIE) norm 2,1
- application of a correntropy based loss function and adaptive means to rank and select features in an MR Spectroscopy extraction problem,
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

*single-center data set and relatively small sample size of data (noted, is always difficult for rarer diseases) *It is unclear the level of non-Gaussian the noise processes in this particular sample. *few details about the experimental settings in section 3 “the basic classifier and RBC”. Would be difficult, I think, to replicate the findings
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I think implementing the GCIE would be straight-forward. Reproducing, though, the specific results presented would be very difficult due to lack of details of the classifier and data set.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Exciting work and a strong method to improve feature detection for sparse and non-Gaussian MR spectroscopy data (with probable applications beyond H1I think). The Optimization section in Methods (section 2) needs reworking. The sentences are incomplete and/or confusing. “To this end, We use the close fodrmed solution to ….”. I think this section lends to be rate the reproducibility low in addition to details about the SVM used in the classification.

*What criteria was used to select the top m features? Were there unexpected results or findings?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Unclear how to reproduce the work, particularly executing the feature extraction and optimization.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #1

Please describe the contribution of the paper

This paper describes the use of MR spectroscopy of 9 brain regions for the early detection of neuropsychiatric systemic lupus erythematosus (NPSLE). The method proposed uses sparse feature selection with correntropic-based loss and regularization. Subjects (N=23 patients, N=16 controls) were studied using 13 MRS measures in each region. Support vector machine classification was used to evaluate the feature selection.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This is an interesting clinical application (NPSLE) with interesting MRS data. Introducing the use of generalized correntropy to this area for feature selection is also interesting. The formulation is sound and the evaluation is promising.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

There are many measures for feature similarity and generalized correntropy is certainly a good one. However, I would have preferred the inclusion of a straight-up comparison of related measures such as cross-entropy, K-L divergence and regular correntropy to see how they differed and their strengths and weaknesses. I would also like to see some indication of what features were selected and how that differed between the different methods. It would also be interesting to separate out the regularization in order to evaluate its contribution.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility is fine.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Nine brain regions are measured but the acronyms are not spelled out. Fig 1 (a-d) are conventional images; the caption indicates they are multi-voxel MRS. The English is poor.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The methods are interesting but I would prefer to see a more convincing evaluation.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper proposed a robust exclusive adaptive sparse feature selection (REASFS) for use in early diagnosis of neuropsychiatric systemic lupus erythematosus (NPSLE) using proton magnetic resonance spectroscopy (H-MRS). An interesting idea with correntropy adoption for system performance improvement. Good writing an good experiments. Although the reviewers have some concerns, they unanimously supported the paper acceptance. The authors should carefully study the reviewer comments, e.g. those from R#3, and improve their camera ready manuscript.

Author Feedback

N/A

back to top

Robust Exclusive Adaptive Sparse Feature Selection for Biomarker Discovery and Early Diagnosis of Neuropsychiatric Systemic Lupus Erythematosus