List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ming Kang, Chee-Ming Ting, Fung Fung Ting, Raphaël C. W. Phan
Abstract
With an excellent balance between speed and accuracy, cutting-edge YOLO frameworks have become one of the most efficient algorithms for object detection. However, the performance of using YOLO networks is scarcely investigated in brain tumor detection. We propose a novel YOLO architecture with Reparameterized Convolution based on channel Shuffle (RCS-YOLO). We present RCS and a One-Shot Aggregation of RCS (RCS-OSA), which link feature cascade and computation efficiency to extract richer information and reduce time consumption. Experimental results on the brain tumor dataset Br35H show that the proposed model surpasses YOLOv6, YOLOv7, and YOLOv8 in speed and accuracy. Notably, compared with YOLOv7, the precision of RCS-YOLO improves by 2.6\%, and the inference speed by 60\% at 114.8 images detected per second (FPS). Our proposed RCS-YOLO achieves state-of-the-art performance on the brain tumor detection task. The code is available at https://github.com/mkang315/RCS-YOLO.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_57
SharedIt: https://rdcu.be/dnwD5
Link to the code repository
https://github.com/mkang315/RCS-YOLO
Link to the dataset(s)
https://www.kaggle.com/datasets/ahmedhamada0/brain-tumor-detection
Reviews
Review #1
- Please describe the contribution of the paper
This paper demonstrate a modified 2D YOLO network and apply it to brain tumor detection. It shows improvment over other YOLO networks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
It is claimed to the first to apply YOLO on brain tumor detection
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) Ablation study is missing for each modification on YOLO. 2) It is a 2D detection network 3) Comparison with non-YOLO methods is missing
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Code is not provided
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1) Ablation study is missing for each modification on YOLO. 2) Comparison with non-YOLO methods is missing
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1) Ablation study is missing for each modification on YOLO. 2) Comparison with non-YOLO methods is missing
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors propsoed a novel YOLO framework with Reparameterized Convilution based on Channel Shuffle. There is a one shot aggregation of RCS used in the YOLO model.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The background and literatures review discussion are comprehensive. The method is clear and easy to follow. The proposed method considers the balance of accuracy and complexity. Overall, this is well-writen paper with clear description of methods and effectiveness evaluation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Method figure can be improved, it;s not very intuitive on how the RCS is achieved in the workflow.
- How the detection head are determined to reduce inference time. The selection of feature layers and scales can be empirical instead of determinitic numbers. The authors can add discussion of the parameter selection.
- The improvement on the results table can be further evaluated by statistical tests. Since the Recall, APs are not very obvious. Significancy evaluation can be helpful to justify the improvements.
- Evaluation metrics can be moved to supplementary as they are common equations familiar with the community.
- Are the YOLO the only well behaved detection networks? The authors can discuss this part, as the only comparison to YOLO based variant might not be sufficient to justify the effectiveness of the proposed methods.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors did not mention code release or potential links for access in the paper nor in the supplementary. If the paper is accepted, the authors can consider add code links.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The methods’ figures can be further improved and highlight the proposed RCS.
- The selection of hyperparameter is not very well described, the author can add clarification on parameter selection.
- More discussion/statement or potential comparison to medical detection models.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The clear innovation and comparison among YOLO methods, and the improvement on detection results.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The paper approaches the problem of brain tumor detection by the proposed YOLO architecture with Reparameterized Convolution based on Channel Shuffle (RCS-YOLO). The key contributions of proposed method are RepVGG/RepConv ShuffleNet (RCS) and One-Shot Aggregation (OSA). Author claims that the proposed RCS in tumor detection task can provide more feature information in the training stage and reduce the computation overhead in inference and the proposed OSA module can achieve semantic information extraction with low-cost memory consumption.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The topic is interesting and clinically significant.
-
The paper is well organized and easy to follow.
- There is no mention of validation strategy that would have guided the selection of many hyperparameters. Yet it seems that no proper tuning opportunity has been given to the proposed method. It appears that the selection was purely heuristic.
-
While Experiments results presented in Table 1 show merits of the proposed RCS-YOLO, there is not any ablation study in the manuscript and supp. materials to demonstrate the effectiveness of proposed modules in RCS-YOLO.
- The proposed RCS-YOLO seems effective in terms of computation cost and experiment results.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
There is no mention of validation strategy that would have guided the selection of many hyperparameters. Yet it seems that no proper tuning opportunity has been given to the proposed method. It appears that the selection was purely heuristic.
-
While Experiments results presented in Table 1 show merits of the proposed RCS-YOLO, there is not any ablation study in the manuscript or supp. materials to demonstrate the effectiveness of proposed modules in RCS-YOLO.
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors mentioned that the code will be made available upon acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In general, it would be nice to see the standard deviations of performance metrics across the test subjects in additional to their mean, and ideally when performance differences from baselines are claimed they should be backed up by a significance test. Authors may consider conducting statistical significance test and conducting additional ablation study for each proposed modules to support claims in the paper.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The rating reflects innovation of the approach for tumor detection and presented good performance. The merits slightly outweigh the weaknesses.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors’ rebuttal address my questions I would like to keep my rating.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents a modified 2D YOLO network, called RCS-YOLO, for brain tumor detection. It introduces Re-parameterized Convolution based on Channel Shuffle (RCS) and One-Shot Aggregation (OSA) as key components of the proposed method. The strengths of the paper include comprehensive background and literature review, clear methodology, and a balanced consideration of accuracy and complexity. However, weaknesses include the absence of an ablation study for each modification on YOLO, the lack of comparison with non-YOLO methods, and the need for statistical significance tests and further evaluation of the results. Improvements in the clarity of figures, explanation of hyper-parameter selection, and discussion of alternative detection networks are suggested. The reproducibility of the work would benefit from the release of code. Based on the initial reviews, the meta-reviewer invites this paper for a rebuttal.
Author Feedback
We appreciate the positive comments from reviewers and constructive suggestions for improvements. For reproducibility, the code, data, details of the model architecture and hyper-parameters can be found here for review purposes: https://anonymous.4open.science/r/RCS-YOLO-1F86 (expired on 06/24/2023). This will be made publicly available on GitHub upon paper acceptance. We group and summarize the major concerns of reviewers, which we carefully addressed as follows.
Missing ablation study for each modification on YOLO (Reviewer #1, #4) Response: We have conducted an ablation study to demonstrate the effectiveness of incorporating our proposed RCS-OSA module in YOLO-based detectors. Table 2 (See https://anonymous.4open.science/r/RCS-YOLO-1F86/README.md) shows that, compared to the RepVGG-CSP model (where RCS-OSA in the RCS-YOLO is replaced with the Cross Stage Partial Network (CSPNet) used in existing YOLOv4-CSP architecture; model architecture: https://anonymous.4open.science/r/RCS-YOLO-1F86/cfg/ablation/repvgg-csp.yaml), our RCS-YOLO (model architecture: https://anonymous.4open.science/r/RCS-YOLO-1F86/cfg/training/rcs-yolo.yaml) performs better in both the detection accuracy and speed on the Br35H dataset.
Lack of comparison with non-YOLO methods & discussion of alternative detection networks (Reviewer #1, #2) Response: We did not include performance comparison with other non-YOLO methods. This is because, from our literature survey, YOLO-based detectors (e.g., YOLOv5/6/7/8, see refs [28, 15, 30, 29]), are now well-known as state-of-art models of object detection both in accuracy and speed. From the results reported in previous object detection studies, it is shown that alternative non-YOLO methods, such as Transformer-based DETR-DC5 (https://github.com/facebookresearch/detr), are underperformed compared to YOLOv5/6/7/8 (both in speed and accuracy) on the benchmarking MS COCO dataset. Therefore, we focus our comparisons with YOLO-based detectors which give state-of-the-art performance. We will include a more comprehensive discussion of alternative detection networks in the revised paper.
Hyperparameter selection/tuning is not well described (Reviewer #2, #4) Response: The selection of hyper-parameters such as feature layers and scales was done empirically by evaluating a range of hyperparameters and selecting the optimal set of parameters that gives the best detection performance. The list of selected hyper-parameters of our proposed model is given here (https://anonymous.4open.science/r/RCS-YOLO-1F86/data/hyp_training.yaml)
Statistical significance test of results (Reviewer #2, #4) Response: We used standard performance metrics for object detection models (like in the YOLOv5/6/7/8 papers), which are typically computed over the aggregation of all testing samples. Moreover, the samples in the dataset used are not indexed/differentiated by subjects, therefore it is not feasible to report the means and standard deviations of the metrics over subjects or to conduct statistical tests of differences in performance between algorithms.
Clarity of figures can be further improved (Reviewer #2) Response: The proposed RCS-OSA module is incorporated in the backbone and neck of the object detector to shorten the information path between feature prediction layers to speed up inference. We will highlight this in the overview workflow in Fig. 1. The details of RCS-OSA are illustrated in Fig. 2 and Fig. 3. We will revise Fig. 2 to highlight the different stages/operators of RCS, i.e., channel split, structural parameterization, and channel shuffle. We will also revise Fig. 3 and its caption to describe more clearly the one-shot aggregation which concatenates all features only once in the last feature map.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The initial reviews highlighted several weaknesses in the paper, including the absence of an ablation study to analyze the impact of modifications on YOLO, the lack of comparison with non-YOLO methods, insufficient explanation of hyper-parameter selection, and the need to discuss alternative detection networks. In the rebuttal, the authors addressed these concerns by confirming the presence of the ablation study in the paper and stating that YOLO-based models outperform alternatives in MS-COCO. They also committed to adding the necessary discussions. Additionally, minor concerns were well addressed and acknowledged by the sole reviewer who responded. The authors’ provision of code further strengthens the reproducibility of their work. Considering the paper, the reviews, and the rebuttal, the meta-reviewer recommends accepting the paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The adoption of the YOLO framework and the reparameterization strategy seems to be relatively new in the MICCAI community. As mentioned in the reviews, the paper is overall clearly written. The result section is relatively weak but can demonstrate the advantages of the proposed method. As I think the reparameterization part can stimulate discussion and new ideas, I prefer to accept this paper.
Minor comments:
- The reparameterization strategy should be briefly described in the introduction, otherwise it is confusing to readers that have not read the RepVGG paper.
- The figures and tables should be at the top of a page.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The author addresses my concerns. This paper is innovative. There are mathematical theories to address the challenges, and technical details to support their method. The authors conduct more experiments and provide detailed numerical metrics. I think it can be accepted weakly.