List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Tomáš Chobola, Gesine Müller, Veit Dausmann, Anton Theileis, Jan Taucher, Jan Huisken, Tingying Peng
Abstract
The process of acquiring microscopic images in life sciences often results in image degradation and corruption, characterised by the presence of noise and blur, which poses significant challenges in accurately analysing and interpreting the obtained data. This paper proposes LUCYD, a novel method for the restoration of volumetric microscopy images that combines the Richardson-Lucy deconvolution formula and the fusion of deep features obtained by a fully convolutional network. By integrating the image formation process into a feature-driven restoration model, the proposed approach aims to enhance the quality of the restored images whilst reducing computational costs and maintaining a high degree of interpretability. Our results demonstrate that LUCYD outperforms the state-of-the-art methods in both synthetic and real microscopy images, achieving superior performance in terms of image quality and generalisability. We show that the model can handle various microscopy modalities and different imaging conditions by evaluating it on two different microscopy datasets, including volumetric widefield and light-sheet microscopy. Our experiments indicate that LUCYD can significantly improve resolution, contrast, and overall quality of microscopy images. Therefore, it can be a valuable tool for microscopy image restoration and can facilitate further research in various microscopy applications. We made the source code for the model accessible under https://github.com/ctom2/lucyd-deconvolution/.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_63
SharedIt: https://rdcu.be/dnwN8
Link to the code repository
https://github.com/ctom2/lucyd-deconvolution/
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The proposed method, LUCYD, is a well-designed and well-executed solution to the problem of image degradation and corruption in microscopy. The approach of combining the Richardson-Lucy deconvolution formula with a U-shaped network for feature fusion is innovative and effective. The authors have provided a thorough description of the method’s architecture and have demonstrated its effectiveness on both synthetic and real datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
LUCYD is a novel method proposed for restoring volumetric microscopy images that combines the Richardson-Lucy deconvolution formula and the fusion of deep features obtained by a fully convolutional network. The model outperforms existing deconvolution methods on both synthetic and real datasets in terms of image quality and generalizability.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is suggestable to include more detailed discussion of the limitations of the proposal.
- The authors compared their proposed model, LUCYD, with two advanced models: CARE and RLN. But I noticed that CARE is outdated. It is recommendable to show recent related methods for fair comparison. It may be useful to compare LUCYD to other state-of-the-art methods for a wider range of microscopy imaging tasks to show better understand its strengths and limitations.
- LUCYD has been shown to outperform other state-of-the-art methods on both synthetic and real microscopy data sets, but its performance may be varied depending on the specific characteristics of the data being used. It may be necessary to experiment with more diverse data.
- It may be important to further investigate the interpretability of LUCYD’s feature-based reconstruction model and explore ways to make it more transparent and explainable to end users.
- In the abstract section, the author mention that LUCYD reduces the cost of computation, compared to CARE, LUCYD performs better with fewer parameters, but it is recommendable to show recent related methods for fair comparison.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It provides the source code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please refer the comments of No. 6.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
LUCYD effectively combines the strengths of U-Net and Richardson-Lucy deconvolution. The experiments conducted on both synthetic and real microscopy datasets show that LUCYD exhibits strong generalization capabilities and robustness to noise.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
By carefully reading AUTHOR FEEDBACK, the author has answered most of the reviewers’ doubts and concerns. But I read that the authors replied that RLN has more significant performance than DenseDeconNet (Guo et al., Nature Biotech 2020) and RCAN (Chen et al., Nature Methods 2021). And the authors proposed that LUCYD outperforms RLN in both quantitative and qualitative evaluations. However, when I read the RCAN (Chen et al., Nature Methods 2021) paper, I found that the RCAN proposed as early as 2021 already outperformed CARE (2018), therefore, I think that the authors’ choice of CARE vs. LUCYD as an experiment is not meaningful and its explanation is not sufficient.
Review #2
- Please describe the contribution of the paper
A novel deep learning model based on Richardson-Lucy algorithm was proposed. Better restoration performance compared with CARE (Nat. Method 2022) was achieved.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A novel deep learning approach to formulate the RL algorithm in deep learning.
- The experimental results in terms of PSNR and SSIM are impressive.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The method is not well explained when reformulating the RL algorithm.
- The comparison with other models is not given in terms of the number of network parameters.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of this work is good with code publicized.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
The problem is formulated mathematically in Eq. (1). Since it is not different from other deconvolution problems, then what is the main difference between the deconvolution of the microscopy images and other medical images or natural images?
-
It would be great if you can provide some explanation on why the correction module works. What is the $z_{tilde}$ is Eq. (3), and what are FP, DV in Eq. (4). I don’t think it is appropriate to using abbreviation where these notations are first used.
-
What is the difference between the proposed RL-based algorithm and other RL-based deep learning works such as “DEEP-URL: A MODEL-AWARE APPROACH TO BLIND DECONVOLUTION BASED ON DEEP UNFOLDED RICHARDSON-LUCY NETWORK”?
-
How do you prevent numeric problem (like divided-by-zero) in the Division Block?
-
It would be great to provide the comparison of the model size in Table 2, which validates the fair comparison.
-
Are all the compared methods based on blind-deconvolution or known kernels?
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The lack of clear presentation and explanation of the model design and the experiment section.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors propose a CNN-based deconvolution method to effectively recover microscopic images with blurring and noise.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The classical deconvolution methods are iterative with high computation time. The authors propose a network that learns the iterative procedure of a classical deconvolution method to effectively generate the results. They used several synthetic datasets for training and test. They also used two real microscopic images for demonstration. The test results indicated their method can generate quality images without blurring and noise.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Fig 1. should be improved. The colors of lines and Residual blocks are too closed. What are the details of Conv layer and Residual block?
- The dimensions of test volume are small (128^3), could you test a larger volume? For example, 512^3.
- The experiments did not consider microcopy image analysis.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This work can be reproduced.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
For real microcopy images, it is difficult to acquire a sample with ground truth. However, we can examinate that by a post analysis. For example, can you verify that the microscopic images produced by your method can detect a certain symptom as positive or negative? Further, is that possible to apply your method for super resolution imaging? For example, 128^3 to 512^3.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work is novel and effective.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors formulate a deep learning approach for image deconvolution that uses richardson-lucy steps as submodules. The results on simulated and real datasets is convincing and outperforms SOTA in several metrics. Some reviewers have pointed out that there are newer deep deconv. techniques out there that need mention. R2 has noted some lack of clarity and intuition about the model design. Please try to address these concerns in a rebuttal.
Author Feedback
Dear Area Chair,
We thank you and the reviewers for the valuable feedback given on our manuscript. We appreciate the chance to address the raised points and provide further clarification.
We are pleased that the reviewers consider our work “a well-designed and well-executed solution to the problem of image degradation and corruption in microscopy” (R1). They describe LUCYD as “novel” (R2, R3), “innovative” (R1) and “effective” (R1,R3) due to its incorporation of “the Richardson-Lucy (RL) deconvolution formula with a U-shaped network for feature fusion” (R1), and highlight its ability to “generate quality images without blurring and noise” (R3). They unanimously agree that our experimental evaluation demonstrated “effectiveness on both synthetic and real datasets” (R1) while yielding “impressive” (R2) and “convincing” (AC) results that “outperform SOTA” (AC).
Yet, they have concerns, as pointed out by AC, regarding i) comparison with new deep deconvolution methods including Deep-URL (Agarwal et al., 2020) (R1,R2), ii) clarification of the intuition behind the design of the model (R2) and iii) dependency on input variety (R1,R3).
i) In our study, RLN (Li et al., Nature Methods 2022) was primarily utilized as a benchmark due to its recent state-of-the-art performance in deconvolution and denoising for microscopy imaging, surpassing techniques such as DenseDeconNet (Guo et al., Nature Biotech 2020) and RCAN (Chen et al., Nature Methods 2021). Since LUCYD outperforms RLN in both quantitative and qualitative evaluations, it implies potential superiority over these recent methodologies. We include CARE (Weigert et al., Nature Methods 2018) in our comparison, despite it not being the latest method, because it remains a widely used microscopy image deconvolution technique and has been incorporated in many established image restoration toolboxes. Deep-URL, as a self-supervised iterative model, is unsuitable for LUCYD’s objectives, due to its substantial computational burden and excessively long processing times. Additionally, Deep-URL can only address spatially invariant blur, whereas LUCYD can handle spatially variant blur. Lastly, existing state-of-the-art deblurring and denoising methods for natural images often sacrifice information for visually pleasing results, which falls short in preserving crucial details in life sciences applications.
ii) The intuition behind the LUCYD model design is to augment the image formation process described by the RL formula by incorporating a trained convolutional network, thereby increasing the capacity and flexibility of the method. The original RL formula iteratively multiplies the previous iteration’s sharp image estimation with an update term for N steps. In contrast, LUCYD employs a U-shaped network as a correction module to predict a tensor that, when added to the raw input, provides an estimation equivalent to the N-1 iteration and bypasses the iterative process. Because the model is trained on simulated paired data, the module learns how the estimation is supposed to look like. Then, the update module replicates the RL formula’s convolution steps using trained convolutional layers while incorporating deep features from the correction module to produce the update term. Same as in the RL formula, the image formation process is finished by multiplying those two terms.
iii) Due to the fully-convolutional nature of our method, the input size does not affect LUCYD’s performance.
Besides the three issues above, we acknowledge the comment about the parameter comparison, and we would like to clarify that the information is indeed provided in Table 1. We will also follow the suggestions and evaluate the model on more diverse data and tasks in the journal extension of the manuscript.
We hope we clarified the concerns of the reviewers and expanded on the “thorough description of the method” (R3) to advance the research of image restoration tasks within the MICCAI community.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The author have adequately addressed most of the reviewer concerns. I therefore recommend acceptance.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have addressed some of the concerns of the reviewers specifically in clarifying methodology but R1 points out an important issue that CARE might not be an appropriate SOFA to compare to… so I am a bit on the fence but would lean towards rejecting the paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a microscopy image recovery framework based on Richardson-Lucy deconvolution. The reviewers positively acknowledge the innovation of the paper, but raise concerns about the details of the model design and comparative experiments. The authors clarify most of the issues in their rebuttal. In particular, the authors provide a detailed response to the intuition of the model design, which is a concern of mine. Therefore, I think the authors’ rebuttal meets my expectations and I recommend accepting the paper.