List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Riccardo Taiello, Melek Önen, Olivier Humbert, Marco Lorenzi
Abstract
Image registration is a key task in medical imaging applications, allowing to represent medical images in a common spatial reference frame. Current literature on image registration is generally based on the assumption that images are usually accessible to the researcher, from which the spatial transformation is subsequently estimated. This common assumption may not be met in current practical applications, since the sensitive nature of medical images may ultimately require their analysis under privacy constraints, preventing to share the image content in clear form.
In this work, we formulate the problem of image registration under a privacy preserving regime, where images are assumed to be confidential and cannot be disclosed in clear.
We derive our privacy preserving image registration framework by extending classical registration paradigms to account for advanced cryptographic tools, such as secure multi-party computation and homomorphic encryption, that enable the execution of operations without leaking the underlying data. To overcome the problem of performance and scalability of cryptographic tools in high dimensions, we first propose to optimize the underlying image registration operations using gradient approximations. We further revisit the use of homomorphic encryption and use a packing method to allow the encryption and multiplication of large matrices more efficiently.
We demonstrate our privacy preserving framework in linear and non-linear registration problems, evaluating its accuracy and scalability with respect to standard image registration. Our results show that privacy preserving image registration is feasible and can be adopted in sensitive medical imaging applications.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_13
SharedIt: https://rdcu.be/cVRST
Link to the code repository
https://github.com/rtaiello/pp_image_registration
Link to the dataset(s)
Reviews
Review #1
- Please describe the contribution of the paper
The authors describe a distributed image registration, where two parties provide a fixed an moving image, without wanting to share their actual image data; they show that some established image registration algorithms can be approximated using full encryption, with significant performance loss, but comparable accuracy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This is a pretty cool idea! In light of all the hype around federated learning, it is quite interesting to see a work that addresses the low-level implementation details of image registration algorithms using encryption.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It is not straightforward to come up with a real-world use case of this, outside of research/academia (but then after all, this is an academic conference), this could be motivated better in the manuscript. Tailoring the image registration algorithms to suit the encryption is done by classical steps one would do on a very slow or old computer, i.e. sub-sample with smart schemes as much as possible without compromising quality - there is not much novelty there.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good enough, a bit more details about the software implementation framework & programming languages could be mentioned.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Regarding the motivation of the work, the authors could make a bit more effort to come up with a realistic use case. The SSD metric itself is mostly suited for intra-patient registration, and this would pretty never require your PPIR. For inter-patient, atlas- or multi-modal scenarios on the other hand, this might become relevant, mostly in academic & research scenarios similar to federated learning, i.e. where sophisticated registration-based anatomical atlases can be added to, with data from multiple privacy-preserving sources. The limitation of simple SSD metrics & gradients might be overcome to do something like pre-processing on each party’s computer (self-similarity, a modality synthesis GAN etc.).
Please improve the presentation of the results a bit. Show the computation times in total (not per iteration), also showing the time of the original method (it won’t be 0.0 seconds as in the table now), to allow to estimate the relative performance loss. Maybe you can also print the required network bandwidth for the original scenario, i.e. if one were to send the pixel data for each iteration in an unencrypted fashion.
Can you come up with anything more innovative than subsampling? Are there specific mathematical properties of the encryption methods that lend themselves to more specific algorithm changes during the registration?
If the overall description of the MPC and FHE approaches can be further improved for people not familiar with such encryption methods, this becomes a really nice manuscript that bridges the gap between two otherwise quite disconnected technical domains.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
All described above already.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The paper shows how to perform image registration while preserving privacy using cryptographic tools.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the problem of preserving privacy is pratically valid
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The contribution is not clear. Like, is the cryptographic tools are adapted for the registration task?
- Not sure why the evaluation metric would concern about the image similarity. Is there any difference for the results among CLEAR and other methods? Or the encrypting way will make the decoded image different with the original one?
- What is other baseline methods? In reviewer’s view, such proposed framework is use the cryptographic tools to perform registration task. And why not apply the framework to other tasks, i.e., segmentation?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility should be good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Authors are suggested to apply for other tasks, i.e., segmentation, detection.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The problem is pratically valid with little discussion.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The authors present a privacy preserving image registration algorithm using cryptographic tools such as secure multi-party computation and homomorphic encryption.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strengths of the paper lies in the fact that the authors have used cryptographic tools to develop an image registration algorithm. The paper is well written and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness of this paper lies in the evaluation and time effectiveness. The authors motivation to develop a PPIR algorithm is weak.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors have indicated the source code will not be available (not aplicable). I am not very sure regarding the reproducibility of the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The authors should provide clear evidence motivating the proposed algorithm.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
2
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work is interesting, but the authors do not show evidence of the need to develop such an algorithm.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
2
- [Post rebuttal] Please justify your decision
I am still not convinced by the motivation of the work and would stick to my original rating.
Review #4
- Please describe the contribution of the paper
This paper proposes privacy-preserving image registration (PPIR), allowing image registration when images are confidential.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper reformulates the typical image registration problem to integrate cryptographic tools, enabling preserving the privacy of the image data. Experiments on linear and non-linear registration problems show its accuracy with respect to standard image registration.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) Unclear motivations and insufficient explanations. In the introduction part, the authors assume that people possibly could not share the image content in clear form, based on [1-2]. Could you give more specific and clinical applications? Cause it is unimaginable to diagnose diseases without clear images. The assumption seems confusing and impractical. It would be much better to concretize sensitive medical imaging applications mentioned in the Abstract and Introduction.
(2) Limited novelty. The authors focus too much on existing secure computation techniques [30] [11] [24] [9], which should not be their contribution. Also, the choice of gradient sampling techniques [19][29][26] and matrix partitioning [7] takes a prominent role in the Method section, although none of these aspects are novel contributions, making this paper has limited technical novelty and originality.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Minor point: (1) Eq. (1) should be improved. (2) In section 3.1, ‘… collaborating parties .’ should be improved. (3) The font size in Fig.1 should be adjusted.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Limited novelty. Also, this paper needs to clarify more clear motivations and sufficient explanations.
- Number of papers in your stack
7
- What is the ranking of this paper in your review stack?
7
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors addressed my concerns about the motivation and clinical use cases by rebuttals. As for the novelty, this paper uses insights from existing encryption works and designs tailored MPC and FHE for Privacy-Preserving Image Registration. Though having no idea about professional encryption technologies that fall outside the scope of MICCAI, I think that this work may be a real-world use case and promote the development of federated learning. As such, I tend to raise my score (weak accept).
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Contribution
Presents a privacy preserving image registration algorithm using cryptographic tools, such as secure multi-party computation and homomorphic encryption.
- Cool idea.
- Interesting to see a work addressing low-level implementation details of image registration algorithms using encryption.
- Well written and easy to follow.
Weaknesses to address
- Needs more motivation for the work. Present some clinical use cases.
- Lack of novelty in the techniques involved, which all appear to be based on existing secure computation techniques. Perhaps there is novelty in the way the techniques have been combined?
- Address the reviewers’ remaining concerns about reproducibility.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
12
Author Feedback
We thank the Reviewers (R1,2,3,4) and Meta-reviewer (MR) for their feedbacks. Please below you can find our point-to-point responses to the questions. The main comments are on: 1) clarify the need for PPIR in real-world applications (R1,2,3,4,MR). PPIR is inspired by real world use cases, and tackles open problems of the MICCAI community. Several groups have provided in the past web-based tools for automated segmentation and labeling of medical images, based on image registration and multi-atlas segmentation propagation. We list here some free tools which can be readily found on the web: NiftyWeb (atlas based segmentation methods for multiple organs), iBEAT (infant brain extraction), volBrain (Automated brain volumetry), BrainBox (online brain labeling). These tools require to upload a medical image in clear over the Internet and perform image registration with respect to one or multiple (potentially proprietary) atlases, for example for multi-atlas propagation. These services are generally not compliant with current privacy regulations existing in many countries. PPIR contributes to the domain by allowing these services to be compliant with the privacy-preserving requirements in the real world. Another application is correctly identified by R1, and consists of federated learning (FL) analyses with medical images, in which multiple centers want to align their scans and create a common anatomical template prior to the FL analysis. We will clarify these key applications of PPIR in the introduction. 2) Novelty (R3,4,MR). As stated by R1, our work “ bridges the gap between two disconnected technical domains”: encryption and image registration. The problem we tackle is not straightforward and contributes to the domain of image registration with novel techniques from cryptography. To this end, our methodological framework “revised the low-level implementation of standard registration protocols” (R1), and adapted the formulation of MPC and FHE accordingly. To the best of our knowledge, no work ever has ever addressed this problem. More specifically, while this work doesn’t provide a new cryptographic primitive (which falls outside the scope of MICCAI), we propose a new algebraic approach to scale existing multi-party computation and homomorphic encryption schemes to the problem of image registration (Sec. 3.2)(MR comment 2). 3) reproducibility (R2, MR). We believe that there was a misunderstanding in our statement about reproducibility: our source code and the public dataset used will be made readily available, and the link to them (through GitHub) will be provided in the paper. Additionally, we also plan to publish all metric logs and execution times of our experiments through the wandb.ai platform. Other questions: 4) SSD metric (R1). We completely agree with R1 that an extension of the present framework consists in applying the cryptographic formulation to registration metrics beyond SSD (such as mutual information). We are currently working on this aspect, but we believe that the concept and results of PPIR shown in the manuscript already contribute nicely from both methodological and application perspectives. SSD is still a useful metric for voxel based morphometry, registration of quantitative/probabilistic images, and time series alignment. 5) Computation time (R1,2). We will provide actual computation time as recommended. We still note that our current implantation of the registration framework is based on a Python implementation which is not optimised. Therefore, we feel that the interest is rather on the relative slowdown due the encryption operations (as shown in the paper). The current registration time on the clear images is of few seconds (linear) and <2 minutes (non-linear). 6) Use of image similarity among the evaluation metrics (R3). Please note that the encryption operation requires to approximate the data through quantisation. This operation does not impact the quality of the final image (Supplementary Fig. 1,2).
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors provided a good example of a use-case, as there are indeed several web-based applications where people upload images for processing (without too much consideration of privacy issues). Most reviewers suggested accepting the manuscript, although one reviewer still gives a low score.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
10
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
While there is certainly merit in the detail on the description of methods for privacy preservation and the proposed adaptation to medical image registration. The initial submission had a slightly vague motivation, but the rebuttal resolved that issue. Some reviewers felt that the used cost function (RMSE) would cause a serious limitation, however, when employing modality invariant descriptors (MIND or similar) the same cost function could be used without much effort for more complex problems. Some other outstanding extensions to sparse-registration methods are mentioned in the paper. I am convinced that the approach constitutes a useful addition to the conference. I would however strongly encourage to make source-code public to enable reproducibility.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
9
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Seems important aspect of registration, however the presentation lacks clarity in the registration-specific challenges and the asscoiated contributions from this work.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR