List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Calvin-Khang Ta, Abhishek Aich, Akash Gupta, Amit K. Roy-Chowdhury
Abstract
Image enhancement approaches often assume that the noise is signal independent, and approximate the degradation model as zero-mean additive Gaussian. However, this assumption does not hold for biomedical imaging systems where sensor-based sources of noise are proportional to signal strengths, and the noise is better represented as a Poisson process. In this work, we explore a sparsity and dictionary learning-based approach and present a novel self-supervised learning method for single-image denoising where the noise is approximated as a Poisson process, requiring no clean ground-truth data. Specifically, we approximate traditional iterative optimization algorithms for image denoising with a recurrent neural network that enforces sparsity with respect to the weights of the network. Since the sparse representations are based on the underlying image, it is able to suppress the spurious components (noise) in the image patches, thereby introducing implicit regularization for denoising tasks through the network structure. Experiments on two bio-imaging datasets demonstrate that our method outperforms the state-of-the-art approaches in terms of PSNR and SSIM. Our qualitative results demonstrate that, in addition to higher performance on standard quantitative metrics, we are able to recover much more subtle details than other compared approaches. Our code is made publicly available at https://github.com/tacalvin/Poisson2Sparse
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16452-1_53
SharedIt: https://rdcu.be/cVVp9
Link to the code repository
https://github.com/tacalvin/Poisson2Sparse
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposed a novel approach to denoise given noisy image. The considered approach in this paper is very relevant in the domain of medical image processing. Self supervised de-noising approach does not require ground truth image to learn the de-noising model. The objective is achieved by using sparse representation based approach. Here the dictionary, to obtain the sparse representation, is learned using convolution sparse coding network. A framework is worked to denoise image, where it is assumed the noise has poisson distribution. The experimental results show that the proposed approach is performing better in comparison with the existing approaches.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Use of convolution sparse coding network to denoise the image by maximizing the likelihood function for poisson distributed noise
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The solution proposed in this work, is being studied in the literature. The only novelty comes here is work out the solution for poisson noise.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Convincing
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1) The assumption of noise to be poisson distribution is very restrictive. It will be good to devise an approach where there is no assumption on noise. Or specifically , when the signal strengths is comparable to noise.
2) It will be good to observe the behaviour of alpha and check it susceptibility for different type of noise.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed approach is devised to use the DNN in the framework of analytical method
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The paper has introduced a modification of the ISTA iterative optimisation dictionary learning algorithm which uses a recurrent neural network. The authors demonstrate this shows improved denoising performance compared to stat of the art self supervised methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper leverages existing methods and combines them with neural networks. This is both interpretable and in this case may actually reduce the number of free parameters in the process.
The challenge dealt with is about self-supervised learning, which is a well-known problem in biomedical imaging. The authors have demonstrated their results on a variety of problems, which are not limited to a single modality.
The mathematics of the paper are very strong and equations are clear and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This paper operates under the primary assumption that noise in medical images is Poisson distributed. This is incorrect, noise in the collected medical image projections is Poisson distributed, however, the reconstructed volumes no longer follow the Poisson distributed noise. The noise in reconstructed volumes is in fact very non-stationary and follows no distribution at all.
The FMD dataset has no ground truth. The authors write that the ground truth is obtained by averaging. While qualitative results may be relevant, I don’t think such a dataset should be considered for quantitative analysis.
The paper is fairly light on experiments and is heavily theoretical.
Figure 2 states that the ribs have been recovered. To my untrained eye, the ribs have not been recovered, they are little better than random noise.
Table 1 shows the highest SSIM in the DIP for the FMD dataset. In Fig. 2 and Supplementary Fig. 3, The DIP image is visually the most blurry image, which usually implies a very low SSIM. In my opinion, the results have not been reported correctly.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility is good. Datasets and code are available. Only possibility for major improvement is the inclusion of the pre-trained model.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
More experimental detail and better quality of data is needed.
The optimization and denoising needs to take place in the projection/k-space domain, rather than the reconstructed volume domain.
A comparison to the original ISTA optimiser needs to be shown, to demonstrate the superiority of the neural network based modification.
The advantage of this method to standard supervised learning approaches needs to be explained and experiments need to be conducted to show it.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
2
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The scientific premise of the paper is incorrect.
The experimental depth is insufficient. The paper results may be reported incorrectly. The data used is of questionable quality.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
While the theoretical framework might be the relevant part of this paper, the results do not show this theory to have much merit. One of the experiments is scientifically incorrect. The other does not have a ground truth for comparison. I will stand by my current review.
Review #3
- Please describe the contribution of the paper
This paper presents a self-supervised approach for single image denoising for Poisson corrupted images, which requires only one noisy image to generate the clean version. The method is extremely practical in situations where the acquisition of clean data can be difficult. Meanwhile, embedding deep neural networks into the framework of traditional iterative optimization methods provides a new inspiration for related research.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) This paper presented a novel self-supervised approach to denoise biomedical images that follow the Poisson noise model. 2) The proposed method only needs a single noisy image which is desirable in many practical applications like biomedical imaging. 3) This work unrolls traditional iterative optimization methods into neural networks-based methods, which provides a new perspective of using modern deep learning technologies to deal with the traditional learning methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) The authors proposed to approximate traditional iterative optimization algorithms for image denoising with a recurrent neural network. However, the application of the recurrent neural network is not reflected well in the paper. 2) Some mathematical expressions are not standardized. For example, the coefficient of loss L_N in Eqn. (12), \mu_N, is inconsistent with that in training details. 3) The description of Fig. 1 is inconsistent with Eqn. (10), please check carefully. 4) The method overview in Fig. 1 is not described in sufficient detail, and does not correspond to the text well. 5) The visual results shown in Fig. 2 and Fig. 3 should be accompanied with quantitative evaluations such as PSNR and SSIM to compare different approaches better.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good reproducibility
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1) The method overview presented in Fig. 1 should be more clearly described to highlight the innovation of the proposed method. 2) Why the structure of autoencoders is described as a recurrent neural network requires further clarification. 3) In the experimental section, the format of table titles of Tab. 1 and Tab. 2 seems to be unconventional. 4) As the proposed method is an extension of iterative optimization methods, using deep neural networks, it would be better to compare it with the iterative ISTA method. When it comes to practical applications, computational efficiency should also be considered in the experiments.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Good novelty, but not clear writing and illustrations.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The main strengths of this work are clear and obvious, contrbuting to the community with valuable reference. However, the hypothesis of this paper is slightly strict, and there are many detailed issues in writing.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
There are non-converging review recommendations. The authors are encouraged to address the issues raised by the reviewers including esp. the novelties & technical contributions (is it solely about dealing with Poison noise, and how about other imaging noise types), the empirical evaluations, presentation (esp. figures and tables), among others.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Author Feedback
We thank the reviewers for their comments. All responses will be added to the final version. R1: Different types of noise; behavior of alpha. – Thanks for the suggestion. We will explore our method (eg. behavior of alpha) for denoising natural images which pertain to different kinds of noise in our future studies. R2: Assumption that noise in medical images is Poisson distributed; denoising in the projection/k-space domain – We believe that there is a misunderstanding here. Our focus is on the method of denoising images with Poisson noise in multiple datasets. In Tab. 1, we show results on microscopy images that are NOT reconstructed volumes but 2D images, and the noise is usually Poisson [17]. We also show results on MRI images following the exact setting used in a recent MICCAI paper [25], ensuring consistency with past work. The results on such different datasets show that our method works for a variety of settings with Poisson noise, and hence has the potential to be generalized to other applications. The suggestion of the reviewer would lead to a method that is MRI-specific, rather than a general Poisson denoising framework - that would be a completely different line of work. R2: Ground truth for FMD dataset – The ground truth is derived from an averaging of static samples [27]. This exact same setting was used to evaluate denoising performance by prior work [13]. To keep a fair comparison, we follow the same practice. R2: Paper is light on experiments and is heavily theoretical – Our proposed theoretical framework is a major contribution to the paper, and should not be viewed as a weakness. The experiments show that our approach works in a synthetic case (eg. PINCAT dataset) as well as with natural Poisson-Gaussian noise (eg. FMD dataset). Along with a detailed ablation analysis of the proposed method (Tab. 2), and multiple qualitative analyses on the PINCAT and FMD dataset, we believe that the experiments conducted are adequate to show the effectiveness of the approach. R2: In Fig. 2, the ribs have not been recovered – To prove that the presented examples in Fig. 2 have been denoised, we compute the MAE between the restored and the ground-truth image for all methods. We observed that our method results in the smallest difference w.r.t. the ground-truth with the following MAE scores: (starting from the top row) ours: 0.0208, 0.0181, 0.0241; noisy image: 0.1017, 0.0945, 0.1000. This implies there is a 79.547%, 80.84%, and 75.90% decrease, respectively, in noise level clearly indicating the recovered images show lower noise. R2: DIP example for the FMD dataset is poor despite having a higher average SSIM. – The SSIM values in Tab. 1 are similar for DIP and our method when averaged over the entire dataset. However, there are interesting cases where our method does significantly better than DIP, and we highlighted that in Fig. 3. This provides qualitative examples of when our proposed method works better. R2, R3: Comparison to the ISTA optimiser and its computational efficiency – Running ISTA on the PINCAT dataset with three noise levels (denoted by lambda) gives us the following PSNR/SSIM results. With lambda = 40: ISTA (19.732/0.719), ours (34.309/0.957); lambda = 20: ISTA (19.642/0.661), ours (32.202/0.937); lambda = 10: ISTA (19.181/0.499), ours (30.005/0.898). Clearly, our method outperforms ISTA in all cases by large margins and although ISTA is slightly faster than deep learning-based methods like ours, the time vs. performance trade-off is extremely poor as shown above. R3: Fig. 1, Tab. 1 and Tab. 2 should be more clearly described – We will update these in the final version to make them clearer. R3: Autoencoders as a recurrent neural network – At each ‘i’th iteration, updating alpha requires us to keep the previous ‘i-1’th iteration alpha in memory as shown in Eqn. 10. Such an iterative update rule naturally maps to a recurrent network as an autoencoder.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper deals with single noisy images with Poisson noise via a novel self-supervised convolutional sparse coding pipeline. The experimental results are demonstrated on a variety of problems with reasonable performance. Meanwhile, the reviewers have raised a number of concerns including better justification of the used method (e.g. why RNN here), the limited practical use concerning only focusing on Poisson noise, insufficient empirical evaluations with possible cherry-picking results, and issues with data quality, as well as issues in presentation. The authors need to seriously go through the issues raised by the reviewers and address them properly.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
11
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper addresses the problem of single image denoising for images affected by poisson noise. The proposed method combines iterative optimization approaches with deep architectures leading to a self-supervised convolutional sparse coding approach. The theoretical derivations have raised interest among reviewers, and some clarity issues have been answered in the rebutal, which both justify my suport for acceptance. However there are several clarity issues remaining, in particular regarding the experimental setup and results which should be adressed in a revised version.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I think the authors have done a good job in addressing the concerns of reviewers on novelty, evaulations and presentation. I recommend acceptance of the work.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8