List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Mohammad M. R. Khan, Yubo Fan, Benoit M. Dawant, Jack H. Noble
Abstract
In cochlear implant (CI) procedures, an electrode array is surgically inserted into the cochlea. The electrodes are used to stimulate the auditory nerve and restore hearing sensation for the recipient. If the array folds inside the coch-lea during the insertion procedure, it can lead to trauma, damage to the resid-ual hearing, and poor hearing restoration. Intraoperative detection of such a case can allow a surgeon to perform reimplantation. However, this intraoper-ative detection requires experience and electrophysiological tests sometimes fail to detect an array folding. Due to the low incidence of array folding, we generated a dataset of CT images with folded synthetic electrode arrays with realistic metal artifact. The dataset was used to train a multitask custom 3D-UNet model for array fold detection. We tested the trained model on real post-operative CTs (7 with folded arrays and 200 without). Our model could correctly classify all the fold-over cases while misclassifying only 3 non fold-over cases. Therefore, the model is a promising option for array fold detection.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43996-4_24
SharedIt: https://rdcu.be/dnwOY
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #3
- Please describe the contribution of the paper
In this work, the authors propose a DL-based technique to detect cochlear electrode array fold-overs from CT images, a relatively rare but not unusual complication (1-8%) of cochlear implant surgeries. The fold-over ar typically detected intra-operatively from electrical measurements or visually from post-operative imaging. The authors propose a way to automate the visual inspection.
They train multi-task U-Net that simultaneously segments the inserted electrode array and while classifying the presence or absence of a folding from the U-Net’s bottleneck features from tight CT image crops.
The authors circumvent the limited availability of positive examples and generate synthetic post-operative images by sampling different electrode array configurations, simulating their appearance on CT images, and ultimately compositing them with the pre-operative images.
The real images with fold-overs are kept for testing and the proposed method achieves perfect classification on all 7 examples.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novel application of imaging:
-
This is likely the first paper that describes how to automatically detect tip electrode array fold-overs from post-operative CT images. The method is computationally efficient and achieves perfect prediction performance on the limited test dataset.
-
The study confirms previous findings that auxiliary tasks (such as the electrode array segmentation) in the multi-task learning set-up can be a useful technique for extra performance gains.
-
The authors show how additional images with metallic artifacts can be composed with normal images to create “good enough” synthetic images that improve the training. This is particularly interesting in the event of the absence of a large training set, and is another tool that rightfully belongs to the arsenal of data augmentations and for self-supervised learning.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Limited reproducibility:
- Sampling of the electrode array geometries with fold-overs is crucial to the method, yet, it is not described with sufficient detail. What variability of shape or electrode array models was considered?
- The authors do not describe any sensitivity of the work to the selection of the crop or scanner acquisition parameters. What parameters were used for the backprojection?
- Parameters of the methods used for comparison are not clear
Limited novelty of the technique and comparison with other approaches:
- A very similar post-operative cochlear image generation with metallic artifacts was done previously by Wang et al: Deep Learning based Metal Artifacts Reduction in post-operative Cochlear Implant CT Imaging
- A multi-task network with a 2D U-Net was trained in Marullo et al: A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery
- The study shows that even more straightforward single task classification networks (e.g., ResNet18) can achieve classification performance of 80% and more on real images. It is not clear, whether the choice of parameters for the other techniques has been tuned for their best performance and whether the compared models had at least comparable capacity (similar “number of weights”), and whether more augmenation, or more time to learn the task would not make them perform as well as the proposed Multi-task U-Net approach.
- The paper shows no comparison with the multitude of traditional methods detecting the individual contacts of the electrode arrays that with could be then used for a geometry-defined fold-over detection such as: Zhao et al. - Automatic Localization of Cochlear Implant Electrodes in CT Benink et al. - Automatic Localization of Cochlear Implant Electrode Contacts in CT Hachmann et al. - Localization of Cochlear Implant Electrodes from Cone Beam Computed Tomography using Particle Belief Propagation Zhao et al. - Automatic localization of closely spaced cochlear implant electrode arrays in clinical CTs
- Wouldn’t any pre-trained U-Net trained on the CT images or feeding the network a thresholded image be just as useful to extract the bottleneck layers?
Limited clinical application
- This technique detects the tip fold-over only once it happens, yet that usually means that the trauma has already been caused. Doing a full ionizing CT intra-operatively several times is likely not the best strategy for the patient’s safety
Limited validation:
- The testing dataset if fairly small. Did it contain also images from multiple electrode
- The solution requires additional data annotation
- Testing on different electrode arrays?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper is fairly difficult to reproduce as is. One of the main components (generation of realistic images with metallic artifacts) is not described in sufficient detail),
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Add more details on the electrode geometry generation. Without it, the paper cannot be reproduced. It is not clear how this would behave for different electrode vendors or images from different acquisition machines.
Please, add more details about the filters used in the reconstruction of the synthetic image.
Typo “Therefore, Therefore”
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work addresses an important complication in electrode array insertion during cochlear implant surgery. The main challenge is that this occurs relatively rarely and is not well represented in clinical datasets and the “heavy-weight” ML models do not perform as well as they could.
The authors chose an interesting way to tackle it - to simulate a synthetic dataset of positive cases and add an auxiliary task to help the model. They do comparison with alternative approaches and show that their approach achieves perfect classification performance on a limited test set. This work can serve to us all as an inspiration to always seek additional ways of doing data augmentation or finding auxiliary tasks for our models, especially when the data is missing.
This paper offers incremental methodological innovation and there are remaining concerns about reproducibility of the work due to missing key information, however, it merits MICCAI publication.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors address a surgical complication in cochlear implant surgery - the electrode array tip fold-over. A dataset of 155 CT images with folded synthetic electrode array with realistic metal artifacts has been generated to circumvent the data scarcity of fold-over cases and is further used to train a weakly supervised customized U-net. The network performs the segmentation of the electrodes and the classification of fold-over cases.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper introduces a novel approach to circumvent the data scarcity in electrode array fold-over by the generation of 155 synthetic fold-over cases with realistic metal artifacts on real CT images data.
- The topic is clinically relevant, and an early detection of the tip fold-over is required.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
It is unclear whether shape-based features have been considered in addition to the intensity-based features in the process of creating the synthesized electrodes and the tip fold-overs.
-
The authors suggest using their technique to reposition/reinsert the electrode array intraoperatively in case a fold-over is detected. However, intra-operative CT/CBCT are rarely acquired in current conventional cochlear implant surgeries, limiting the usability of such an image-based detection to post-operative corrective actions during the CI fitting/programming session.
-
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Partially reproducible - the geometric aspects and constraints used in the process of creating the artificial electrodes are provided at a very high-level detail. This hinders the reproducibility of the datasets. In case this paper is going to be extended to a journal paper, I strongly emphasize on the importance of clarifying this part being the prominent contribution of the work.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors address a relevant surgical complication in cochlear implant surgery - the electrode array tip fold-over. A dataset of 155 CT images with folded synthetic electrode array with realistic metal artifacts has been generated to circumvent the data scarcity of fold-over cases and is further used to train a weakly supervised customized U-net to perform the segmentation of the electrodes and the classification of fold-over cases. The paper is well-structured and written in a consistent manner. The work is well-positioned wrt. the comprehensive literature summary. Please have a look at two related works [Ref.1, Ref.2] My main concern is related to the generation of the synthetic dataset. Given that the datasets are not accessible, a detailed description of the generation process is very crucial for reproducibility purposes. While a large attention has been attributed to the benchmarking against different SOTA methods, less details were presented on the dataset generation algorithm.
- Did the authors considered any shape-based features? If not, this might be an oversimplification of the existing large portfolio of electrodes from different CI manufacturers.
- What kind of geometric constraints have been imposed? centerline, length, insertion angle, curvature, path (avoiding deviation to the scala vestibuli), etc.
- Does the order of the electrodes matter in the tip fold-over detection? Is there any topological information which allow ordering the electrodes/contacts?
- What is the length of the electrodes array? closely/distantly spaced electrodes? Are those constants and the same for the 155 pseudo electrodes?
- In case only one geometric configuration is selected? Is the method expected to generalize on different electrodes sizes, length, spacing, and number of contacts?
- Are the real test data with fold-over cases implanted with the same type of electrode arrays? Are those types like the synthesized ones? In case there is no room to briefly address the above-listed points in the present paper, I strongly recommended to address them in an extended journal paper.
Minor:
- Page 2, line 18: duplication of “Therefore”
- Page 4 - For the sake of clarity, please rephrase “The combined dataset including 192 real CTs was divided into …” to “the combined dataset including 192 real CTs and 215 synthetic CTs was divided into …”
- Duplication of references 5 & 12
[Ref.1] Mom T, Puechmaille M, El Yagoubi M, Lère A, Petersen JE, Bécaud J, Saroul N, Gilain L, Mirafzal S, Chabrot P. Robotized Cochlear Implantation under Fluoroscopy: A Preliminary Series. J Clin Med. 2022 Dec 27;12(1):211. doi: 10.3390/jcm12010211. PMID: 36615012; PMCID: PMC9820833.
[Ref.2] Perazzini C, Puechmaille M, Saroul N, Plainfossé O, Montrieul L, Bécaud J, Gilain L, Chabrot P, Boyer L, Mom T. Fluoroscopy guided electrode-array insertion for cochlear implantation with straight electrode-arrays: a valuable tool in most cases. Eur Arch Otorhinolaryngol. 2021 Apr;278(4):965-975. doi: 10.1007/s00405-020-06151-z. Epub 2020 Jun 25. PMID: 32588170.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Clinical relevance - this work addresses an under-estimated surgical complication which can severely impact the restored hearing quality. Image-based approaches are well-positioned to address the problem.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
The paper describes a network to classify whether cochlear implants undergo unwanted folding during implantation. Because the incidence of the event is rare, synthetic data were generated to train the network. In an early stage evaluation of the algorithm, testing was done using real CT images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Synthetic data, simple network, comparative evaluation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Small test dataset size, no variance of estimates.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The network is simple and hyperparameters are reported to reproduce it. No information on accessibility of the synthetic and/or real dataset.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
Text in legend for Figure 3 is too small to read.
-
The reasoning behind choosing the comparator networks is not clear.
-
Why was the synthetic data so small in size?
-
The test is small. It is ok for an early stage evaluation of the algorithm. However, details on the test set are missing. How were patients in the test set selected? Did each patient give one image in the test set? What were the characteristics of patients in the test set?
-
The conclusion refers to clinical deployment. I’m not sure the findings support a clinical deployment. The Discussion section is repetitive.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The problem is important, use of synthetic data is novel, early stage testing shows a good signal of performance. However, the test set is small and it is not clear how the patients in the test set were selected to assess potential for conserved performance in a larger and heterogeneous test set.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a novel approach for generating synthetic data to train a deep neural network for detecting the electrode array folding for cochlear implants. The clinical impact is significant. However, one reviewer rightly pointed out that intraoperative CT or C-arm CT is not the standard of care for guiding the placement of the electrodes intraoperatively. This algorithm would only be useful in the postoperative setting to detect electrode folding on postoperative CT imaging. Providing intraoperative guidance would then be difficult based on the results of this algorithm. The authors must be commended for their innovative approach to generating a synthetic dataset for training the deep neural network due to the paucity of clinical cases showing electrode folding. As one of the reviewers pointed out, this is an innovative method of augmenting the dataset. The reviewers also pointed out that there is a lack of detail on the generation of the synthetic dataset, the applicability of the method to implants from different manufacturers, and comparisons to other methods in the literature. It would be good for the authors to address these concerns of the reviewers.
Author Feedback
We would like to thank the reviewers for their thorough and insightful comments. We will modify our manuscript according to many of the suggestions, as described in detail below. Reviewer #1:
- The reasoning behind choosing the comparator networks is not clear. Response: As the objective of the study was fold-over detection, we evaluated several common image classification approaches. For example, we started with a popular classification network (ResNet-18). However, as the model was failing to classify fold-over cases we considered different networks architectures that would permit encouraging more focus more on extracting features of the electrode array rather than the whole CT image. Thus, we used VAE, GAN and multi-task 3D U-Net to extract the array features while performing the classification task and compared to these commonly used approaches.
- The test is small. What were the characteristics of patients in the test set? Response: Thank you for the suggestion. Since our original submission, we’ve added another 187 real CT images to our test dataset to more comprehensively analyze the performance of the proposed approach. Therefore, the updated testing dataset has 207 real postoperative CT images (200 non fold-over cases and 7 fold-over cases). The test CTs are selected randomly, and each CT image corresponds to a unique implant case. For this study, we are considering only the adult patients with array implants with 22 closely spaced electrodes. Reviewer #2:
- It is unclear whether shape-based features have been considered in the process of creating the synthesized electrodes. Response: During the process for generation of the synthetic electrodes, we indeed accounted for different a priori shape based features (such as fixed electrode-to-electrode distance, total electrode array length) as constraints in addition to constraining the array to a realistic intra-cochlear position (fully within the Scala tympani lumen versus including a realistic scalar translocation).
- Details about the synthetic electrode array generation process are missing. Are the real test data implanted with the same type of electrode arrays? Response: We synthesized CI532 electrode arrays in the synthetic CTs, as studies have found this type of array to be most at risk for fold-over. With this array, the active array length when straightened is around 14mm. The array has 22 closely spaced electrodes (about 0.6 mm of electrode-to-electrode distance). The active array length and electrode-to-electrode distance is randomly varied in a small range to create variability among the synthetic CTs. The test data has fold-over cases for 3 types of arrays: CI532, CI522 and Contour advance, which are fairly similar in terms of electrode spacing. As we do not have data of real fold-over cases for arrays with significantly varied geometric configurations, we are not sure whether our model is applicable for other array types, currently. Reviewer #3:
- The authors do not describe any sensitivity of the work to the selection of the crop or scanner acquisition parameters. What parameters were used for the backprojection? Response: We have not tested varying CT acquisition parameters for this study. That can be introduced in our future work where we will generate images with varying resolution and blurring effects. For the backprojection, we have used the Hamming filter with a frequency scaling of 0.8 using MATLAB’s iradon function.
- Doing a full ionizing CT intra-operatively several times is likely not the best strategy for the patient’s safety. Response: We agree that the intraoperative CT is not the ideal solution, however, at the moment it remains the gold standard for fold-over detection. Further, despite the fact that trauma has already been caused, many clinical experts we work with prefer the option to detect a fold and reinsert the electrode array within one procedure rather than requiring a subsequent re-implantation procedure.