List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Tianbao Liu, Zefeiyun Chen, Qingyuan Li, Yusi Wang, Ke Zhou, Weijie Xie, Yuxin Fang, Kaiyi Zheng, Zhanpeng Zhao, Side Liu, Wei Yang
Abstract
Super-resolution (SR) of wireless capsule endoscopy (WCE) images is challenging because paired high-resolution (HR) images are not available. An intuitive solution is to simulate paired low-resolution (LR) WCE images from HR electronic endoscopy images for supervised learning. However, the SR model obtained by this method cannot be well adapted to real WCE images due to the large domain gap between electronic endoscopy images and WCE images. To address this issue, we propose a Multi-level Domain Adaptation SR model (MDA-SR) in an unsupervised manner using arbitrary set of WCE images and HR electronic endoscopy images. Our approach implements domain adaptation at the image level and latent level during the degradation and SR processes, respectively. To the best of our knowledge, this is the first work to explore an unsupervised SR approach for WCE images. Furthermore, we design an Endoscopy Image Quality Evaluator (EIQE) based on the reference-free image evaluation metric NIQE, which is more suitable for evaluating WCE image quality. Extensive experiments demonstrate that our MDA-SR method outperforms state-of-the-art SR methods both quantitatively and qualitatively.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_50
SharedIt: https://rdcu.be/dnwdx
Link to the code repository
https://github.com/SMU-MedicalVision/MDA-SR
Link to the dataset(s)
N/A
Reviews
Review #7
- Please describe the contribution of the paper
In this paper, the authors present a domain adaptation SR model for WCE images. The proposed model is based on CycleGAN, which does not require paired HR-LR images for supervised training. The effectiveness of the model is validated through experiments, making this a strong paper with a clear motivation and convincing results.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors provide a detailed explanation of the motivation behind this work.
- The proposed model is unsupervised and thus has the potential to be applied to a variety of clinical cases.
- The proposed technique follows CycleGAN and includes some key designations, such as adaptive degradation.
- The authors conduct experiments on real data, and the related results demonstrate that the proposed model outperforms other SR models, especially in terms of the MOS metric evaluated by clinical doctors.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The model proposed by the authors is based on GAN, and in the method section, we can see that there are many loss terms in the entire model. Therefore, how stable is the training of the model? I think this part needs a more detailed explanation, such as the specific steps of the training, which is beneficial to the reproducibility of this work.
- The authors do not perform simulation experiments. Therefore, some objective metrics (e.g., PSNR and SSIM) cannot provided. Although this work focus on the SR problem under a domain shift setting, I think a basic simulation experiments also should be performed.
- The method section of this paper is somewhat difficult to follow.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors provide the source code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
A simulation experiment needs to be conducted to measure the objective metrics.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation behind this work is clear, and the proposed model shows promising performance. However, the presentation could be improved further to enhance the clarity and cohesiveness of the paper.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
In this paper, author has implemented domain adaptation at the image level and latent level during the degradation and SR processes, respectively. As per author, this is the first work to explore an unsupervised SR approach for WCE images. Furthermore, they have designed an Endoscopy Image Quality Evaluator (EIQE) based on the reference free image evaluation metric NIQE, which is more suitable for evaluating WCE image quality.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Some strength of the papers are mentioned below. 1) The paper is well written and categorized including wireless capsule endoscopy, super resolution and domain adaptation. 2) In this work, author propose a Multi-level Domain Adaptation Super-Resolution (MDA-SR) for WCE images to bridge the domain gap between electronic endoscopy images and WCE images 3) Through extensive experiments on real WCE images, author has demonstrate the superiority of our method over other state-of-the-art SR methods, and its efficacy in reality.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
the main motivation is not much discussed in the context of domain adaptation.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Author has convey, clear, specific and complete information about data, code, models and computational methods and analysis that support the contents and result presented in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The author has contributed for paper entitled “ MDA-SR: Multi-level Domain Adaptation Super-Resolution for Wireless Capsule Endoscopy Images”. The content of paper is very interesting and well written. Here I would like to provide some comments mentioned below as per the reviewers comment and response provided by the author
1) Author has added data loss as an additional supervision information . how is it affecting the overall performance in terms of adaptive degradation. 2) It is important to mention how author improved the domain adaptation at the latent level during the SR process. 3) It is advisable to create a table in the dataset heading so the dataset is more visible in the manuscript. 4) How the author has created the training and testing set for the validation of the algorithm with state-of-the-art method in terms of quantitative and qualitative assessment. 5) How mean opinion score is efficient to better illustrate the subject quality for evaluation.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Author has well written the manuscript and has done much research on related work in the past to support the content of the paper. Author has adopted the method and amend the techniques as per the need of this experimental setup, which makes the author’s contribution significant towards this submission.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #6
- Please describe the contribution of the paper
The paper’s main contribution is the proposal of the MDA-SR model for super-resolution of WCE images. It introduces adaptive degradation and domain adaptation techniques to bridge the domain gap between electronic endoscopy and WCE images. The use of unsupervised learning for WCE image super-resolution is a novel approach.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- It is a novel Approach which is a multi-level Domain Adaptation Super-Resolution (MDA-SR) model for WCE images, utilizing unsupervised learning to address the lack of paired high-resolution data.
- The paper conducts comprehensive experiments on real WCE images, demonstrating the superiority of the MDA-SR method over existing techniques through quantitative metrics and subjective evaluations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.While the paper discusses the potential clinical applications of the proposed method, it does not provide a thorough evaluation of its feasibility and effectiveness in real clinical scenarios or provide validation with expert clinicians. 2.The paper does not delve into the interpretability of the MDA-SR model, making it challenging to understand the underlying mechanisms and potential limitations of the approach.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper is reproducible with the help of available code and detailed experimental description.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Include a more detailed comparison with related works in the paper, discussing the pros and cons of current approaches and highlighting how the MDA-SR model addresses challenges unique to WCE images.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a novel approach and provides convincing experimental results. However, addressing the mentioned weaknesses would further strengthen the paper.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents an unsupervised super-resolution (SR) approach for Wireless Capsule Endoscopy (WCE) images using Multi-level Domain Adaptation (MDA-SR). This involves domain adaptation at the image and latent levels during degradation and SR processes. The authors also develop an Endoscopy Image Quality Evaluator (EIQE) based on the reference-free image evaluation metric NIQE, tailored for assessing WCE image quality.
The paper is clearly structured and offers the novel approach of using unsupervised learning for WCE image super-resolution. It introduces the MDA-SR model for super-resolution of WCE images, utilizing adaptive degradation and domain adaptation techniques to bridge the gap between electronic endoscopy and WCE images. The work validates the effectiveness of the model through comprehensive experiments on real WCE images, demonstrating the superiority of their method over other state-of-the-art SR methods.
The paper lacks a detailed discussion of its main motivation within the context of domain adaptation. The potential clinical applications of the proposed method are discussed but its feasibility and effectiveness in real clinical scenarios are not thoroughly evaluated. The authors do not delve into the interpretability of the MDA-SR model, making it difficult to understand its underlying mechanisms and limitations. The stability of the model’s training given the numerous loss terms is also a concern, with a call for more detailed explanation of the training process. Moreover, the absence of simulation experiments and the complexity of the method section are pointed out.
Author Feedback
Super-resolution (SR) in wireless capsule endoscopy (WCE) images presents a significant challenge due to the unavailability of corresponding high-resolution (HR) images. A seemingly straightforward approach involves simulating paired low-resolution (LR) images from HR electronic endoscopy images for supervised learning. However, a prominent domain gap exists between WCE images and electronic endoscopy images, as discussed extensively in our paper. Our main motivation is to utilize the prior knowledge of HR electronic endoscopy images to guide the SR process of WCE images by addressing and eliminating the domain gap between the two. To streamline the model training and reduce its complexity, we have divided the training process into two stages. In the first stage, we focus on training the adaptive degradation, which stabilizes the quality of generated LR images after 50 epochs. Following this, we incorporate the domain adaptation SR into the training process. Moreover, we have introduced specific data fidelity terms to ensure the stability of the generative adversarial network (GAN). These terms constrain both the adaptive degradation and the domain adaptation SR, including the adaptive data loss in the adaptive degradation and the pixel-wise content loss in the domain adaptation SR. For practical implementation of WCE SR, only the domain adaptation SR is needed for inference. This model has a relatively low parametric parameter of 1.407 M and achieves a rapid inference speed of 31 frames per second on a GPU (NVIDIA GeForce RTX 2080 Ti). This speed is significantly faster than the 3 frames per second generated during capsule endoscopy, making it an efficient solution for real-world applications.