List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Wei-Lun Huang, Davood Tashayyod, Jun Kang, Amir Gandjbakhche, Michael Kazhdan, Mehran Armand
Abstract
Longitudinal tracking of skin lesions - finding correspondence, changes in morphology, and texture - is beneficial to the early detection of melanoma. However, it has not been well investigated in the context of full-body imaging. We propose a novel framework combining geometric and texture information to localize skin lesion correspondence from a source scan to a target scan in total body photography (TBP). Body landmarks or sparse correspondence are first created on the source and target 3D textured meshes. Every vertex on each of the meshes is then mapped to a feature vector characterizing the geodesic distances to the landmarks on that mesh. Then, for each lesion of interest (LOI) on the source, its corresponding location on the target is first coarsely estimated using the geometric information encoded in the feature vectors and then refined using the texture information. We evaluated the framework quantitatively on both a public and a private dataset, for which our success rates (at 10 mm criterion) are comparable to the only reported longitudinal study. As full-body 3D capture becomes more prevalent and has higher quality, we expect the proposed method to constitute a valuable step in the longitudinal tracking of skin lesions.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_25
SharedIt: https://rdcu.be/dnwLA
Link to the code repository
https://github.com/weilunhuang-jhu/LesionCorrespondenceTBP3D
Link to the dataset(s)
Reviews
Review #1
- Please describe the contribution of the paper
A method is proposed for establishing correspondence between skin lesions in whole-body images taken at different times. It is evaluated on a public and a private dataset and compared quantitaively with a previous work.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Addresses establishing correspondence between lesions using whole body imaging, an important component in envisaged systems for longitudinal analysis (important because lesion changes over time can be diagnostically important). There is relatively little published on this setting.
- Quantiative evaluation on private and public datasets with comparison to a previous work.
- Results are reasonable and there is good discussion of the limitations of the method and the approach with suggestions for what remains to be addressed.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The method is not especially novel in the context of computer vision, but its formulation in this setting is novel enough.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The experiment on the public dataset should be reproducible given that the method is described in a clear manner.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In Section 2, an assumption is stated that there is the same number of landmarks in the source as in the target. In practice I imagine this is often not the case (as alluded to in the final paragraph of the paper). I suggest commenting on the strength of this assumption at the point where it is introduced in Section 2.
In Fig. 3 please state what the whiskers in the plots denote.
In Table 1, I suggest putting “[26]” after “Skin3D” to make it easy for the reader to see that this is the pre-existing method being compared.
Fig. 2 should be moved so not immediately following the subsection title (e.g., to bottom of page). Define acronym LOI on first use (Section 2)
I found the phrases “localize skin lesion correspondence” and “lesion correspondence localization” a little strange. Perhaps just say “lesion correspondence”?
In Section 3: “insure” -> “ensure”
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A well-written paper on longitudinal skin lesion matching in whole body imaging, a topic without much prior literature. The method is presented clearly and motivated appropriately. Good evaluation and discussion.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The dataset size is a limitation and this work could certainly be strengthened by acquiring and using larger datasets. But on balance I think this paper represents progress on this application and is sufficiently interesting and well written for acceptance.
Review #2
- Please describe the contribution of the paper
This paper introduced an iterative based approach to track lesions across longitudinal scans. The proposed approach is based on using landmarks in together with textures to iteratively identify the local correspondence in the following scan that has a high confidence. The experimental results with a public dataset and a private dataset show that the proposed approach achieved good performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Tracking and identification of skin lesions in longitudinal total body photography is an important problem addressing significant clinical needs.
- The manuscript is well structured and nicely presented, which made easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The technical novelty seems to be limited. Tracking across longitudinal scans has been well investigated for medical image analysis. It’s relatively difficult to understand the technical novelties of the proposed method, when compared to the current methods.
- The experimental materials seem to be quite small. Only 13 subjects were used for comparison. It is true that individual patient may have multiple lesions. However, this individual lesion can hardly be considered as individual samples, as correctly identify one lesion will also affect the accuracy of other lesions of the same subject.
- The experimental setup seems to be very simple. There is no comparison with the current methods. There should be some ablation studies to illustrate the improvement of individual component.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Source codes were not provided. However, the experiments were conducted on a public dataset, which may allow to reproduce the results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The clinical application is important. However, the technical novelty seems to be limited. The comparison and experiments are also quite limited.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Authors claimed that existing methods on skin lesion tracking have limitations in only applicable in well-controlled environments and not allowing to track lesions across scans at different visits. It is challenging to understand that the proposed method will allow to address these limitations.
- It seems that there exists quite number of hyper-parameters / rules, which were not properly justified.
- In fig. 4, the solid and transparent sphere are difficult to distinguish.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
Authors have addressed some of the concerns. However, the limited dataset problem still remains. Therefore, I slightly increased my decision.
Review #3
- Please describe the contribution of the paper
This paper proposes a framework for establishing correspondences for longitudinal tracking of skin lesions in 3D textured meshes from total body photography. An iterative framework first establishes landmark-based correspondences and then refines it using texture information. Evaluation on samples from a public and a private dataset show that the results are similar to the competing work, Skin3D. The proposed method relies on the assumptions that the lesions do not change dramatically between 2 meshes and that there are no new appearing/disappearing lesions.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
A well-written paper with a clear motivation towards solving a problem of clinical relevance - tracking skin lesions from total body photography over time.
-
The methodology is intuitive and explained clearly, with textual descriptions accompanying the mathematical formulations for the most parts, and is therefore easy to follow.
-
I really appreciated the authors providing details on all the hyperparameters for the method. It would have been better if the time taken per mesh-pair was also reported (see weakness #3).
-
The authors provide a good discussion of the limitations of the proposed method and scenarios where the correspondence matching could yield sub-optimal results.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
It is not clear why the ECHO descriptors were computed over 3 different radii. Is it solely for the purpose of robust matching? What are the advantages of using 3 radii values? And aside from the extra computation, are there any downsides to using multiple radii values? How would the results be affected if a single radius was chosen?
-
Were the values of epsilon_{3, 4, 5} in Eqn. 6 empirically chosen? It is not mentioned. Similarly, there is no explanation for the formulation of the geometric score expression in Eqn. 7.
-
Since it is an iterative method (K = 10 iterations), it would be helpful to report how long (runtime) the entire correspondence localization framework takes for a pair of meshes on average.
-
What are the scenarios where this method fails or produces sub-optimal matches? The authors write that this method may not hold when either the appearance or the count of the lesions change between 2 longitudinal meshes. It might be very valuable to show qualitative examples of lesions where the correspondence matching was not perfect because of the aforementioned reasons or otherwise.
-
Please mention the exact count of evaluation samples from both the datasets. Looking at Fig. 3, it looks like there were 20 scans from Skin3D and 3 from IRTBP, the latter of which is very small. Similarly. please provide details about the resolution of the meshes/the number of vertices in the meshes.
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of this paper is good. The authors have reported the values of all the hyperparameters in Supplementary Table 1. However, both the datasets are quite small, with the IRTBP only containing 3 subjects (going by Fig. 3 (b)).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
For Eqn. 2 and 3, C and D are implied but not explicitly defined. Please do so for better readability.
-
In the sentence above Eqn. 4, please consider rephrasing the sentence “In practice, we compute …” because at the moment, it is not very clear if epsilons denote the radii or the descriptors.
-
Some typos were spotted in the paper:
-
Sec. 2.1, page 2: “We define an initial dense correspondences” -> “… correspondence”.
-
Fig. 3 caption: “included in the paratheses” -> “… parentheses”.
-
Sec. 3.3, page 7: “measured the result in both to insure consistency” -> ensure.
-
Supplementary Fig. 3 caption: “textuer similarity” -> “texture similarity”.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a good paper with a sound methodology. The formulation has been explained in detail and is easy to follow. The only major weakness might be that the evaluation has been performed on an extremely small sample size, but it could possibly also be attributed to the lack of large public datasets for this task.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
I thank the authors for a detailed rebuttal and for clarifying several of my questions. My key concern about the (very) small size of the datasets evaluated upon: 10 subjects (Skin3D) and 3 subjects (IRTBP), still remains.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper in review presents a method for establishing correspondence between skin lesions in whole-body images taken at different times. This method is assessed on a public and private dataset and compared with a previous work quantitatively. Despite this, the reviewers express several concerns and areas for improvement.
Firstly, reviewers noted the limited technical novelty of the proposed method. They pointed out that tracking across longitudinal scans has already been well-investigated in the field of medical image analysis, making the unique contributions of the proposed method unclear. They suggested a more detailed comparison with existing methods to highlight the paper’s novelty and contribution.
Secondly, the experimental setup and materials were perceived as inadequate. The reviewers remarked that only 13 subjects were used for the comparison, and individual lesions could not necessarily be considered individual samples due to interdependencies. They also criticized the lack of comparison with current methods and the absence of ablation studies to demonstrate the effectiveness of individual components.
The reviewers also raised questions about the clarity and justifications provided in the paper. There were doubts about how the proposed method would address the limitations of existing methods on skin lesion tracking. The paper was perceived to have many hyperparameters/rules that were not properly justified. The visual presentation, such as figure 4, was also unclear, making it difficult to distinguish between solid and transparent spheres.
Regarding the method’s specific details, questions were raised about the use of ECHO descriptors over 3 different radii, the selection of values for epsilon_{3, 4, 5} in Eqn. 6, the formulation of the geometric score expression in Eqn. 7, and the overall runtime of the iterative method. The reviewers also requested more information about the method’s limitations and potential scenarios where it might fail or produce sub-optimal matches.
Lastly, they requested more specific details about the evaluation samples and the resolution of the meshes or the number of vertices in the meshes.
Author Feedback
R1 and R2 noted limited technical novelty. We propose to find lesion correspondences by using shape and texture feature vectors defined directly on “the 3D textured mesh” in an iterative manner. We appreciate R1’s recognition of the novelty in our work. To our knowledge, there are only two other references, [2] and [26], tackling longitudinal tracking of skin lesions using a 3D mesh. Both propose using a template mesh. However, accurately deforming a template mesh to fit varying body shapes is challenging when the scanned shape deviates from the template, leading to large errors in downstream tasks such as establishing shape correspondence. Additionally, [26] does not take advantage of texture, while [2] uses texture in a common UV map that may lead to failures when geodesically close locations on the surface are mapped to distant sites in the texture map (e.g. when the two locations are on opposite sides of a texture seam). R2 pointed out that tracking across longitudinal scans has been well investigated. However, most methods (e.g. [7,8,21]) assume a well-controlled environment. In contrast to using 2D images, where the representation of lesions is done in the frame of the camera and hence dependent on body pose and camera parameters ([8] and [21]), our approach represents the lesions in the frame of the surface itself and is stable in the presence of isometric deformations. R2 noted inadequate experimental setup and materials. We have evaluated the proposed method on the only publicly available dataset and compared our results to [26] in Table 1. We emphasize that comparison with 2D images is not applicable since such public datasets are not available. We have also evaluated our method on a private dataset that is comparatively small but represents 3D meshes from different sources. R2 mentioned that, because of interdependency, individual lesions should not be considered as individual samples. We agree. Our reliance on interdependency is a core component of our method, allowing us to reach a higher success rate. The success rate, with higher clinical significance, is found by evaluating the percentage of correctly localized correspondence. R2 noted the need for illustration of improvement from individual components. The effectiveness of individual components is demonstrated in 3.3 and Fig. 5. In particular, we show them in different scenarios: 1) correspondence with texture-based features searched within different sizes of regions associated with landmark-based correspondence, 2) correspondence combining landmark-based and texture-based features, and 3) correspondence using the iterative anchoring mechanism. R2 and R3 raised questions about the parameters. We have used a number of parameters and have attempted to explain them in the associated equations, as also noted by R3. The values are empirically determined and provided in the supplement. Additional refinement and/or investigation of the parameters is beyond the scope of this study. The selection of three different radii in ECHO descriptors is done to accommodate different sizes of lesions and their surrounding texture. To combine geodesic distances with textural similarity we use a standard transformation to turn distances into similarities. While this proved to be effective in practice, we would certainly like to consider other combinations in future work. R3 mentioned the runtime. The runtime is several minutes (on average), which we believe is adequate for the setup. R3 requested examples of failure cases. Examples of failure cases due to inconsistent local texture with large errors are visualized in Fig. 4. R3 requested details about the evaluation samples. The evaluation samples (the number of subjects and the number of lesions per subject) are shown in Fig. 3. The number of vertices is on average 300K and 600K for the public and the private dataset respectively.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper under consideration introduces a pioneering framework designed to longitudinally track skin lesions in 3D textured meshes, derived from total body photography. The authors establish an iterative approach that initially creates landmark-based correspondences, subsequently refining them by assimilating texture information. While the paper makes certain assumptions - that lesions don’t significantly change between two meshes and no new lesions appear or disappear - the evaluation of the method using both a public and private dataset demonstrates results comparable to the rival Skin3D work.
The manuscript is remarkably well-written and explicit, showcasing a clear intent to address a clinically significant issue - monitoring skin lesions over time using total body photography. The methodology, which pairs textual descriptions with mathematical formulas, is intuitive and straightforward to comprehend. The authors’ meticulous explanation of all hyperparameters is particularly laudable, although the inclusion of time taken per mesh-pair would have further enhanced this section.
However, a few concerns still linger. The technical novelty of the paper seems somewhat constrained, making it difficult to discern its distinct contributions compared to existing methods. The size of the experimental materials appears somewhat limited, as the study includes only 13 subjects. Additionally, the experimental setup might appear too simplistic, lacking comparison with current methodologies or any ablation studies.
In their rebuttal, the authors address these concerns by arguing that their approach of finding lesion correspondences using shape and texture feature vectors on the 3D textured mesh iteratively, is indeed an innovation. They also emphasize that while most existing methods for tracking across longitudinal scans usually assume a tightly controlled environment, their method isn’t predicated on such a condition.
Responding to concerns about the experimental setup and materials, the authors underscore that their evaluation encompassed the only publicly available dataset and a private dataset comprising different source 3D meshes. They illuminate that their method is designed to leverage the interdependency between lesions, which is crucial for achieving a higher success rate. They also indicate that the effectiveness of individual components is lucidly illustrated in Section 3.3 and Figure 5.
Addressing concerns about parameters, the authors maintain that they have made a concerted effort to explain their choices in the corresponding equations, but concede that further refinement of parameters was beyond the scope of this study. They, however, express their willingness to consider different combinations in future research.
Given the clinical importance of the problem, the robustness of the proposed method, and the authors’ comprehensive and thoughtful rebuttal, I am persuaded to recommend the acceptance of this paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The application and methodology of the paper are interesting. Despite the limited dataset size, I think the paper is worth presenting at the conference and the methodology could be transferable to other domains as well. I would recommend to add the clarifications provided in the rebuttal to the camera-ready version, increase the resolution of the figures and plots, and release the code to increase reproducibility.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a method for tracking skin lesions in whole body phtography where images are taken progressively over time.
The paper is very well written, and most of the reveiwers’ concerns are addressed in the rebuttal. One important concern, which is shared among all reviewers, remains and is not really commented on in the rebuttal, namely the very small sample size.
I will recommend acceptance because of the interesting use case and the fact that after all, conferences should also be a place to discuss work at an earlier stage than what you see in journals – but it is a very borderline case due to the low sample size.