List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Boxiang Yun, Qingli Li, Lubov Mitrofanova, Chunhua Zhou, Yan Wang
Abstract
Medical Hyperspectral Imaging (MHSI) brings opportunities for computational pathology and precision medicine. Since MHSI is a 3D hypercube, building a 3D segmentation network is the most intuitive way for MHSI segmentation. But, high spatiospectral dimensions make it difficult to perform efficient and effective segmentation. In this study, in light of information correlation in MHSIs, we present a computationally efficient, plug-and-play space and spectrum factorization strategy based on 2D architectures. Drawing inspiration from the low-rank prior of MHSIs, we propose spectral matrix decomposition and low-rank decomposition modules for removing redundant spatiospectral information. By plugging our dual-stream strategy into 2D backbones, we can achieve state-of-the-art MHSI segmentation performances with 3~13 times faster compared with existing 3D networks in terms of inference speed. Experiments show our strategy leads to remarkable performance gains in different 2D architectures, reporting an improvement up to 7.7% compared with its 2D counterpart in terms of DSC on a public Multi-Dimensional Choledoch dataset. Code will be released.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_15
SharedIt: https://rdcu.be/dnwCZ
Link to the code repository
https://github.com/boxiangyun/Dual-Stream-MHSI
Link to the dataset(s)
https://www.kaggle.com/datasets/ethelzq/multidimensional-choledoch-database
Reviews
Review #2
- Please describe the contribution of the paper
The authors propose a novel space and spectrum factorization strategy based on 2D architectures to address the challenges in performing efficient and effective segmentation of Medical Hyperspectral Imaging (MHSI). The proposed strategy draws inspiration from the low-rank prior of MHSIs and employs spectral matrix decomposition and low-rank decomposition modules for removing redundant spatiospectral information. The authors demonstrate that their dual-stream strategy integrated into 2D backbones achieves state-of-the-art MHSI segmentation performance, with 3 to 13 times faster inference speed compared to existing 3D networks. The experiments show remarkable performance gains in different 2D architectures, reporting an improvement up to 7.7% in terms of DSC on a public Multi-Dimensional Choledoch dataset. Overall, the proposed strategy offers a computationally efficient and plug-and-play approach to MHSI segmentation with improved accuracy, speed, and generalizability.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
I find the proposed factorization strategy for medical hyperspectral image segmentation to be innovative and promising. The dual-stream strategy, which utilizes the low-rank prior of MHSIs, is not only computationally efficient but also highly adaptable, allowing it to be easily integrated into any 2D architecture. The authors provide thorough experimental results on two MHSI datasets, which demonstrate significant improvements in evaluation metrics, including a remarkable 7.7% improvement in DSC compared to its 2D counterpart. Moreover, the authors report state-of-the-art MHSI segmentation accuracy with 3 ∼ 13 times faster inference speed compared to existing 3D networks when their strategy is applied to ResNet-34 backbone. Overall, this work is a significant contribution to the field of medical image segmentation, and I recommend it for publication.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Need to pay attention to English presentation.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
reproducibility depends on the code release.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I find the proposed factorization strategy for medical hyperspectral image segmentation to be innovative and promising. The dual-stream strategy, which utilizes the low-rank prior of MHSIs, is not only computationally efficient but also highly adaptable, allowing it to be easily integrated into any 2D architecture. The authors provide thorough experimental results on two MHSI datasets, which demonstrate significant improvements in evaluation metrics, including a remarkable 7.7% improvement in DSC compared to its 2D counterpart. Moreover, the authors report state-of-the-art MHSI segmentation accuracy with 3 ∼ 13 times faster inference speed compared to existing 3D networks when their strategy is applied to ResNet-34 backbone. Overall, this work is a significant contribution to the field of medical image segmentation, and I recommend it for publication.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I find the proposed factorization strategy for medical hyperspectral image segmentation to be innovative and promising. The dual-stream strategy, which utilizes the low-rank prior of MHSIs, is not only computationally efficient but also highly adaptable, allowing it to be easily integrated into any 2D architecture. The authors provide thorough experimental results on two MHSI datasets, which demonstrate significant improvements in evaluation metrics, including a remarkable 7.7% improvement in DSC compared to its 2D counterpart. Moreover, the authors report state-of-the-art MHSI segmentation accuracy with 3 ∼ 13 times faster inference speed compared to existing 3D networks when their strategy is applied to ResNet-34 backbone. Overall, this work is a significant contribution to the field of medical image segmentation, and I recommend it for publication.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #6
- Please describe the contribution of the paper
This paper proposes to improve the efficiency of medical hyperspectral image (MHI) segmentation methods by exploiting the correlation information of MHI’s and removing redundancies in spatial and spectral information. Results show better performance and computational efficiency compared to other 2D or 3D methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Evaluation is performed on 2 datasets, with ablation studies. Results show better performance and better better computational efficiency compared to other 2D or 3D methods.
- The methodology is well-motivated
- The paper is well written
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
In Table 1, the ablation results for spectral stream are provided for L2 only and L4 only. Why aren’t they also provided for L1 only and L3 only?
-
There lacks a small description on how the mean and std results were obtained in Tables 1 and 2 (ie: were they obtained by changing the initialization seed, or also other elements?). This could help clarify why the standard deviation is so high. On the other hand, why were the experiments not repeated several times in Table 3? and would it be possible to evaluate the statistical significance of the results?
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors give hyperparameters, framework and machine used for experiments
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Could you define C_0^{spe} when it appears (paragraph before section 2.1)?
- Missing/Extra word in section 3.2: “Our quantitative and qualitative analysis demonstrated that effectively reduces ….”
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well motivated and well-written.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
This study proposes a computationally efficient, plug and play spatial and spectral decomposition strategy based on a two-dimensional architecture to alleviate the problem of low efficiency caused by the use of 3D networks to segment high spatial spectral dimensions and address information correlation in Medical Hyperspectral Imaging. It can achieve state-of-the-art Medical Hyperspectral Imaging segmentation performance, with significantly higher computational efficiency than 3D networks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
At present, the simplest method for classifying/segmenting MHSI is to use its two spatial dimensions as input spatial dimensions. These methods are not suitable for high spatial resolution MHSI, and they may bring noise in spatial features while reducing spectral dimensions. Although 3D networks can handle it, it will bring low computational efficiency. In order to better utilize the high computational efficiency of 2D networks and handle low rank spectral domains, the article proposes a separate and effective spatial dimensions and proposes an effective and effective dual stream strategy to “factor” the architecture. This novel approach has been validated in the article and can be applied as a plugin to other networks, with good application value.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Although this article has significant innovation in methodology, its explanatory viewpoint has not been proven in theory. Although the accuracy has been improved, the proposed viewpoint needs to be further proven. For example, the article states that “improving the representation ability of low rank priors in Medical Hyperspectral Imaging, designing a low rank decomposition module.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Encourage the authors to publicly disclose code, and it is recommended to write method details more clearly. It is recommended to use charts to indicate the structure and steps. This is helpful for future scholars studying this issue and can promote the development of this direction.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
the same as 6.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
There are some innovations in the proposed method, and the experiments in the text are sufficient to verify its effectiveness.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This submission proposes to speed up the segmentation of hyperspectral image segmentation by combining spectral decomposition and low-rank prior to remove redundancies in spatio-temporal information. The originality resides in the proposed general plug-n-play 2d decomposition modules for removing hyperspectral redundancy. The evaluation is on bile duct imaging.
All three reviewers recommend Acceptance. Their consensus is on significant innovation and promising in hyperspectral imaging. The methodology is described as novel enough for the topic and supported by the provided evaluation. Minor elements raised by the reviewers are likely to be improved in a final submission.
For all these reasons, the recommendation is towards Acceptance
Author Feedback
All reviewers and the AC confirm the merit of our paper to the HSI community. We will release the code for reproducibility of our work (@R1&R2) and improve our English presentation (@R2) . In the following, we will address the major concerns one by one.
1.1) @R1: More explanatory viewpoint should be proven in theory We visualized the utilization of low-rank characteristics in the design of LD modules for removing feature redundancy in Fig. 3. To prove the proposed viewpoint that our method improves the representation ability of low rank priors in Medical Hyperspectral Imaging, we visualize the heat maps of the learned sub-attention map A_i in the LD modules, and observe that distinct sub-attention maps activate different regions, some only activating a small portion of detailed areas while others activate a large range of regions. This indicates that the LD decomposition module is capable of capturing discriminative information of varying frequencies.
1.2) @R1: write method details more clearly. We will write more details and put charts to show steps in the supplementary.
3.1) @R3: In Table 1, Why aren’t they also provided for L1 only and L3 only? We have extensively explored the roles of L1~L4 in the ablation experiment. Since we find exhaustive experiment of traversing all the possible arrangements of spectral features inserted into spatial features is redundant, we only provide results for L2 and L4. Based on the R3’s suggestion, we supplement the experimental results using only L1 & L3, which yielded a dice score of 68.16 (the baseline L2 & L4 of dice is 69.73). This demonstrates that shallow connections are insufficient for merging spatiospectral features.
3.2) @R3: Description on how the mean and std results were obtained in Tables 1 and 2 and why it is so high. We calculate the mean and standard deviation of all testing data, i.e. by calculating the corresponding metrics for each testing HSI and taking the average and standard deviation of the metrics of all testing HSIs.
3.3) @R3: Table 3 is incomplete Due to the limitation of the main text length, we have presented the complete Table 3 including standard deviation in the supplementary. Taking the public MDC dataset as an example, the statistical significance (p-value) of our method and others is as follows: PCA-UNet: 1.46×10^(-5); CGRU-UNet: 3.02×10^(-3); 3D-UNet: 2.75×10^(-3); nnUNet: 1.45×10^(-2); Swin-UNETR: 9.33×10^(-3); HyperNet: 6.89×10^(-3); SpecTr: 5.87×10^(-2), showing our improvement is significant.
3.4) @R3: Define C_0^{spe} when it appears. As previously elucidated in section 2.1, C_0^{spe} refers to the dimensions of the spectral stream input features.
3.5) @R3: Missing/Extra word in section 3.2 The full sentence is, “Our quantitative and qualitative analysis demonstrated that the proposed MDC and LD modules effectively reduces the redundancy of output features”.