Abstract
Background: Access to dermatologists is limited in parts of the US, making primary care clinicians (PCCs) integral for early detection of skin cancers. A handheld device using elastic scattering spectroscopy (ESS) was developed to aid PCCs in their clinical assessment of skin lesions.
Methods: In this prospective study, 3 PCCs evaluated skin lesions reported by patients as concerning and scanned each lesion with the handheld ESS device. The comparison was pathology results or a 3-dermatologist panel examining high resolution dermatoscopic and clinical images. PCCs reported their diagnosis, management decision, and confidence level for each lesion. Evaluation of results included sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and Area Under the Curve (AUC).
Results: A total of 155 patients and 178 lesions were included in the final analysis. The most commonly patient-reported concerning feature was “new or changing lesion” (91.6%). Device diagnostic sensitivity and specificity were 90.0% and 60.7%, respectively, based on biopsy result or dermatologist panel reference standard; comparatively, PCC sensitivity was 40.0% and 84.8% specificity without the use of the device. Device NPV was 98.9%, and device PPV was 13.6%. The device recommended patient referral to dermatology with 88.2% concordance with the dermatologist panel. AUC for the device and PCCs were 0.815 and 0.643, respectively.
Conclusions: The use of the ESS device by PCCs can improve diagnostic and management sensitivity for select malignant skin lesions by correctly classifying most benign lesions of patient concern. This may increase skin cancer detection while improving access to specialist care.
- Artificial Intelligence
- Family Medicine
- Dermatology
- Internal Medicine
- Primary Care Physicians
- Prospective Studies
- Skin Cancer
- Spectroscopy
- Technology
Introduction
Over 40% of the US population resides in areas with a shortage of dermatologists, making primary care clinicians (PCCs) crucial in early skin cancer detection.1 With rising skin cancer rates globally, especially among the aging, patients frequently consult PCCs about suspicious lesions to decide if dermatologist referral is necessary.2 If PCCs evaluated all US skin cancer cases, each could identify an average of 14 basal cell carcinomas, 7 squamous cell carcinomas, and 0.7 melanomas annually.3,4 Research indicates that about half of melanoma cases are first noticed by patients or their family members.5,6 Therefore, evaluating concerning lesions is a vital aspect of primary care.
PCCs manage a wide variety of diseases. A recent clinical study estimated that to manage the range of diseases presenting PCCs, their daily workload would range from 9.3 to 26.7 hours, highlighting the need for prioritization in their clinics.7 Addressing every worrisome lesion from patients places additional time constraints on both the patient and the physician. In addition, PCCs may not be very comfortable in the evaluation of skin lesions.2,8 Dermoscopy is useful for evaluating skin lesions, but training in this technique varies across primary care residency programs. Furthermore, continuing education courses frequently lack mandatory practical training in dermoscopy due to the significant time required for hands-on learning. This lack of standardized and comprehensive training contributes to the relatively low utilization of dermoscopy among US primary care physicians, with only about 8.3% regularly employing this method in their practice.2,9,10
These factors contribute to a rise in dermatology consultations for assessment of concerning lesions.11,12 In Europe, there are approximately 1 million dermatology referrals per year.13 About 50% of those referrals are for skin cancer concerns, of those referrals, only 6.5% of the referrals are malignancies.12,14 Thus, there is significant opportunity to improve PCC referral accuracy to detect skin cancer earlier and to triage urgent dermatology referrals.
When assessing a concerning lesion, PCCs must quickly decide if it warrants further evaluation and dermatology referral, considering various lesion and patient risk factors, clinical experience, and time constraints. To streamline this process, a handheld device using elastic scattering spectroscopy (ESS) has been developed. ESS, an optical tissue sampling technique, analyzes light scattering properties to distinguish between benign and malignant tissue. By measuring the spectra of skin lesions, this device provides objective data to help PCCs decide whether to monitor the patient or refer them for specialist evaluation, thus addressing efficiency and training limitations in lesion assessment.15,16
The purpose of this study was to evaluate whether the handheld ESS device can be a valuable and accurate tool for PCCs to assess lesions of patient concern by appropriately differentiating benign lesions from malignant lesions needing further evaluation.
Materials and Methods
Study Overview
This comparative effectiveness study aimed to examine the performance of the ESS device (DermaSensor device, manufactured by DermaSensor Inc., Miami, FL, USA) and the enrolling PCCs in correctly identifying skin lesions that patients reported as concerning during their visit with their PCC. All lesions enrolled were reviewed by a panel of dermatologists (DL, DS, NCZ) and many were diagnosed through biopsy with histopathology. The study's additional objective was to determine the device's specificity in correctly classifying benign lesions that patients believed were concerning for skin cancer. The study followed the Declaration of Helsinki and was approved by the WIRB-Copernicus Group Review Board (IRB). All study participants provided written informed consent before participating.
Device
The ESS device (Figure 1) produces 200 μs-long pulses of light that span wavelengths ranging from near ultraviolet to near infrared. Less than 0.01 seconds of light is exposed to each lesion. It is not necessary to capture the optical spectra in a dark environment because the timing of the pulse allows system performance that is largely unaffected by ambient light.
Handheld Elastic Scattering Spectroscopy Created by DermaSensor, Inc.
A 2.5 mm biocompatible tip with 2 optical fibers inside is placed in contact with the lesion surface. The only part in contact with the patient's skin lesion is the sterile fiber optic tip, which only transmits and collects light and does not emit any electric energy. Using Monte Carlo simulations, the volume of tissue that the ESS device assesses with each spectral recording can be approximately estimated (modeled) to 0.7 mm (l) x 0.4 mm (w) x 0.5 mm (d). Five spectral scans are performed to record the light reflectance of the tissue structure and architecture at different lesion locations (eg, nuclear and chromatin characteristics).
More than 10,000 spectral scans from more than 2,000 lesions were used to develop and train the device’s machine learning algorithm. The algorithm development was independent of this study dataset, and the algorithm was selected and locked before completion of the study statistical analyses. Measuring the spectra of skin lesions, the ESS device classifies lesions as either low risk or high risk for malignancy with a binary output of “Monitor” or “Investigate Further,” respectively. A spectral score of 1 to 10 is provided for “Investigate Further”-classified lesions based on the degree of similarity to malignant lesions present in the device training dataset, with a score of 10 representing the greatest degree of similarity.
Study Design and Patients
This prospective, comparative effectiveness study was conducted in a single primary care study site in the United States. PCC investigators included a board-certified family physician, a board-certified internal medicine physician and a board-certified nurse practitioner. Study enrollment was conducted at the study site from April 6, 2021, to June 3, 2021. Eligible patients with self-identified concerning skin lesions were recruited, screened, and enrolled. After providing informed consent, the PCC investigator performed a clinical assessment of all patient-concerning lesions. The patient's medical history, including the identification of any risk factors for skin cancer, personal history of skin cancer and the patient’s level of concern about the lesion (on a scale of 1 to 10, with 10 representing the highest level of concern) were obtained. Furthermore, a lesion assessment with dermoscopy, objective measurements (anatomic location, size, and surface characteristics), clinical and dermatoscopic images, and the ABCDE characteristics for assessment for melanoma were recorded. PCCs reported their suspected diagnosis, as well as confidence level in their diagnosis for each lesion (“low” or “high”) providing physician comparison data. After PCC clinical assessment, the lesion was then scanned by the handheld ESS device with the device outputs blinded to both the patient and the PCC investigator. Based on their standard of care clinical judgment, PCC investigators chose whether or not to further evaluate (ie, refer or biopsy) the enrolled suspicious lesions. The standard comparison for performance of both the PCCs and the ESS device was histopathologic biopsy results when available and a panel of experts of 3 dermatologists when pathology was unavailable. The panel, which was blinded to ESS results, reviewed dermatoscopic and clinical images.
Inclusion and Exclusion Criteria
The patients included in the study had to be at least 18 years old, have at least 1 self-identified concerning skin lesion, and be able to read and sign the consent form. Lesions were excluded if the lesion in question was less than 2.5 mm or larger than 15 mm in diameter; the lesion was in the ear, under the nail, on acral skin, on an area of active sunburn, area of psoriasis, or other background eczematous conditions. Furthermore, if the lesion had an ulceration with no intact area greater than 2.5 mm, had foreign matter (eg, tattoo, splinter, etc.), or was within 1 cm of the eye, on mucosal surfaces, adjacent to scars, on a previously biopsied site or a previous surgical intervention, then the lesion was excluded from the study due to unknown performance of ESS device functionality. Patients were excluded if the patient had dementia or any other neurologic condition or physical or psychological limitation that would prevent them from signing an informed consent.
Outcome Measures
The primary aim of the study was to evaluate the performance of the device and PCCs in evaluating lesions suggestive of skin cancer to patients, using dermatologist panel assessment and biopsy results as the reference standard. Additional evaluated outcomes included negative predictive value (NPV), positive predictive value (PPV) and area under the receiver operating characteristic curve (AUROC). Further analyses included evaluation of PPV results with 1 to 10 spectral scores (evaluated independently and grouped). These scores or groupings can be used to provide an indication of the relative likelihood that a lesion with an “Investigate Further” result is cancerous.
Statistical Analysis
A sample of 100 subjects with at least 1 normal lesion (benign lesions, including actinic keratoses) was expected to yield a 2-sided 95% CI with a width of at most 19.7% around the specificity estimate. This calculation accounted for up to 5% of subjects with missing or unevaluable data.
Continuous outcome descriptive statistics were calculated with sample size (number of lesions or number of patients, depending on the variable), mean, median, standard deviation, quartiles, minimum and maximum. Descriptive statistics of categorical outcomes include sample size, frequency, and percentage. Confidence intervals around specificity and sensitivity estimates were calculated using the Wilson method to account for within-subject correlations due to subject with multiple lesions.17 Statistical analyses were conducted using SAS Enterprise guide version 7.1. P-values are 2-sided and were declared significant using a 0.05 level of significance, unless otherwise specified.
Results
A total of 157 patients were screened and consented to participate in the clinical trial. One patient withdrew, and 1 patient was excluded due to missing data. Twenty-two of the 23 completed biopsied lesions were included in all analyses, and 7 of 155 unbiopsied lesions were excluded from the effectiveness analysis due to a lack of dermatologist diagnostic consensus, as detailed below.
The 155 eligible enrolled patients who presented with 178 lesions of patient concern were included in the safety analysis. The majority of patients were female (63.9%), white (92.2%), non-Hispanic (94.2%), and Fitzpatrick skin types I-III (51.0%) with a mean age of 65.6 years (Table 1). The most reported concerning feature was a “new or changing lesion” (reported by 91.6% of patients). Of the 178 lesions enrolled in the study, the mean level of concern reported was 3.7 on a scale of 1 to 10. The average size of enrolled lesions was 5.9 mm by 4.9 mm. The 178 lesions had a mixture of surfaces (62.9% elevated vs 37.1% flat), textures (67.4% smooth vs 32.6% rough), and pigmentation (62.9% light vs 37.1% dark).
Description of Patient Characteristics (Based on Investigators Assessment)
PCCs evaluated 177 of the 178 lesions, with only 1 lesion not undergoing assessment (Table 2). PCC lesion assessments were that 155/177 (87.6%) of lesions were expected to be benign and 22/177 (12.4%) were clinically diagnosed as malignant, confidence level was high in 160/177 (90.4%) of the lesions. The most common suspected benign diagnoses were seborrheic keratosis (51/155, 32.9%), solar lentigo (31/155, 20.0%), and actinic keratosis (17/155, 11.0%). A majority (150/177, 84.7%) of lesions were not further evaluated (ie, referred or biopsied), while 26/177 (14.7%) were recommended for biopsy, and 1/177 (0.6%) were referred to dermatology.
Clinical Assessment of Lesions by Primary Care Clinicians (PCC) Investigators
The dermatologist panel evaluated the 155 unbiopsied lesions and could not reach consensus on 7 lesions (4.5%). Lesions with a lack of consensus were excluded from the primary effectiveness analyses, which were performed before the study data being unblinded. Dermatologist’s panel assessments reported that 37.1% of lesions were pigmented, 62.9% were nonpigmented, 95.5% were symmetrical, 4.5% were asymmetrical, 94.2% had normal borders, and 5.8% with abnormal borders. The color was reported as being uneven in 1.6% of lesions, with 98.4% having flesh-toned coloring. The panel assessed 12 of the 148 lesions to be malignant (8.1%), and 136 as benign (91.9%).
Of the 26 lesions recommended for biopsy, 1 biopsy was performed on a PCC-selected lesion (excluded for not meeting inclusion criteria), 2 were not performed, and 1 was lost to follow-up. A total of 22 lesions were biopsied per standard of care and met the inclusion criteria and therefore included in all analyses. Of those, histopathology diagnosed 3 lesions as malignant with the 19 remaining found to be nonmalignant. The most common diagnosis was seborrheic keratosis (36.4%), followed by SCC (9.1%), mildly atypical melanocytic nevus (9.1%), lichenoid keratosis (9.1%) and actinic keratosis (9.1%). A total of 10 cancers were identified via dermatopathology or dermatologist consensus panel, 4 BCC, 4 SCC, 1 melanoma, and 1 malignant other.
When the ESS device diagnostic results were compared with biopsy results (when available) and panel consensus results, the device sensitivity and specificity were 90.0% (95% CI: 71.4% - 100.0%) and 60.7% (95% CI:52.5% - 68.4%), respectively (Table 3). Device specificity in lesion sub-analyses was 76.9% (95% CI: 62.8% - 96.8%) for pigmented lesions, 70.2% (95% CI: 56.7% - 80.9%) for seborrheic keratoses, 41.2% (95% CI: 22.2%-86.2%) for actinic keratoses, and 66.7% (95% CI: 43.5% - 83.8%) for benign melanocytic nevi. In comparison, PCC management sensitivity and specificity when compared with the same reference standard were 40.0% (95% CI: 9.6%-70.4%) and 84.8% (95% CI: 78.2%-89.7%) respectively. Overall NPV of the device for a negative “Monitor” result, was 98.9% (95% CI: 93.4%-99.8%), while the overall PPV of a positive “Investigate Further” result was 13.6% (95% CI: 7.1%-24.6%), which equates to a Number Needed to Refer (NNR) of 7.4. Overall diagnostic performance, as measured by AUROC for the device was 0.815 (classified as very good), compared with PCC performance of 0.643 (classified as sufficient).18
Concordance Between Device and Reference Standard: Diagnosis
When compared with panel consensus alone, the device diagnostic sensitivity and specificity were 91.7% (95% CI:76.0% - 100.0%) and 62.5%, (95% CI: 54.0% to 70.3%) respectively, with a similar NPV of 98.8% (95% CI: 93.1%-99.8%) for a “Monitor” result and corresponding PPV of 17.7% (95% CI: 9.9%-29.6%, NNR: 5.6) for an “Investigate Further” result. When considering the 1 to 10 spectral scores provided by the device, the PPV increased with increasing scores. Spectral scores 1 to 3 had a PPV of 11.1%, which increased to 19.0% and 60.0% for scores of 4 to 7 and 8 to 10, respectively.
Management decision concordance between the device and panel consensus decision is summarized in Table 4. Using this reference standard, management sensitivity of the device was calculated to be 88.2% (95% CI: 64.1% - 96.9%) and device specificity was 70.4% (95% CI: 59.6% -79.3%). The associated NPV and PPV were 96.6% (95% CI: 87.7%-99.1%) and 38.5% (95% CI: 24.3% -54.9%), respectively.
Concordance Between Device and Panel: Management Decision
There were no adverse events related to device use reported during the conduct of this study.
Discussion
The main objective of this study was to determine whether the ESS device could aid PCCs in their clinical assessment of lesions of patient concern, allowing for increased detection and referral of skin cancers while correctly monitoring benign lesion. The study found that the diagnostic sensitivity of the device was 90% when compared with biopsy result or dermatologist panel consensus, and 92% when compared with dermatologist panel consensus alone. The high concordance of the handheld device with the dermatologist panel for benign lesions suggests that use of the device by PCCs may help alleviate patient concern for benign lesions, especially for benign pigmented lesions and seborrheic keratoses. Furthermore, specificity was maintained across skin types, from I-VI, with device specificity of 53.2% for skin types I-III and 69.1% for skin types IV-VI. These results suggest that the use of the handheld ESS device by PCCs can significantly improve diagnostic and management sensitivity for cancerous skin lesions. Furthermore, the accuracy of the device in correctly identifying benign but suspicious skin lesions indicates that its usage could decrease needless dermatology referrals and reduce patient concerns for malignancy.
The diagnostic sensitivity of the handheld ESS device at 90% was higher than the PCC's sensitivity of 40% when compared with the dermatologist panel consensus or biopsy result. This suggests the potential for improving PCCs detection capabilities with device availability and similar results have been seen across multiple clinical validation studies.19,20 In addition, the AUROC of the device was higher than that of the PCC, suggesting use of the device use should improve PCC overall performance. (Table 5) Furthermore, the 90% device sensitivity is comparable to that reported for dermatologists as found in current literature (83 to 96%).21⇓⇓–24 In addition, the NPV of 99% has potential to aid PCCs by giving them confidence in a “Monitor” result for lesions that were correctly classified as benign, allowing them to triage urgent referrals to dermatologists. This can help the patient, patient outcomes, and the health care system by decreasing wait times to see dermatology and obtain appropriate care. For lesions that the PCC finds challenging, the handheld ESS device also helps by providing an additional, objective test result to aid in their evaluation.
Device to Primary Care Clinicians (PCC) Comparison
With an aging population, there is increasing demand for PCCs and these providers are often assessing patients with many health care concerns. There is only one FDA-approved tool that provides primary care clinicians with a risk assessment for suspicious skin lesions. Moreover, there are no FDA-approved tools for patient self-assessment. Literature indicates that commercially available tools (e.g., those on smartphone app stores) are highly inaccurate.25⇓–27 There is growing popularity for genetic testing of lesions, however there are limitations to the applicability of such technology for screening purposes given their high cost. As such, providing PCCs with a tool that provides an immediate, additional point-of-care result to aid in their assessment may improve overall skin cancer detection capabilities and better inform referrals to dermatologists. In this study, it was observed that the PPV increased from 11% to 60% as the device spectral score increased from low scores (ie, 1 to 3) to high scores (ie, 8 to 10). Thus, the 1 to 10 spectral score provided by the device for “Investigate Further”-classified lesions can aid PCCs in prioritizing referrals, as clinical validation studies for the ESS device indicate higher likelihood of malignancy with increasing scores.28,29 Importantly, the publication of this smaller-scale study in a setting akin to a typical primary care office, this approach provides a practical perspective on how these dermatologic findings can be effectively integrated and utilized in everyday clinical practice.
This study has a few notable limitations. Overall study limitations include the blinding of investigators to the device output given the comparative effectiveness study design, which excluded the potential for clinical utility to directly be assessed. Other limitations of this study include those imposed by the exclusion criteria, such as excluding lesions covered in crust or with extensive erosions, lesions on mucosal and acral skin, or those less than 2.5 mm or greater than 15 mm size. In addition, the sample size in this study was relatively small at 177 evaluated lesions; however, additional large studies have been conducted evaluating device use in various settings. Merry et al. conducted a study at 22 primary care study sites, with 1005 patients with 1579 lesions (with 224 high risk lesions, including 48 melanomas, 90 BCCs and 86 SCCs.30 Device sensitivity in that study was 96% for all malignant lesions. Furthermore, in a study by Hartman et al., the authors investigated the ESS device performance in a study conducted at 10 dermatology study sites (440 enrolled lesions, 88 melanomas), to address the utilization of the device as an adjunctive tool for the evaluation of pigmented lesions suspicious for melanoma, and the device’s sensitivity for melanoma was found to be 96%.29 Future evaluations should include cost-benefit analyses of the device given its commercial cost; the device is reported by the manufacturer as being recently cleared by FDA.
This study did not provide a direct comparison of the diagnostic probability of the handheld ESS device to the conventional ABCDE criteria for melanoma since this study was not limited to lesions suggestive of melanoma. However, evidence shows that naked-eye inspections remain insufficient for the evaluation of skin lesions for cancer.22 Further research could demonstrate that a new visual or dermatoscopic approach, combined with the optical spectroscopy assessment, could result in a higher performance than either modality alone. Despite the small sample size, there was a high negative predictive value of 99%. Improving early recognition of melanoma is key to reducing morbidity, mortality, and cost to the patient.31 If melanoma is recognized and diagnosed early, where the disease state is localized to the skin, the 5-year survival rate is 99% versus being diagnosed in a state where the melanoma has spread to regional lymph nodes or distant sites, the 5-year survival rate drops to 68% and 30%, respectively.32
Conclusion
The implementation of the handheld ESS device in primary care settings can markedly enhance the sensitivity of diagnoses and management strategies while helping rule out suspicious yet benign skin lesions from further evaluation that were concerning for skin cancer. The high concordance with both pathologic findings and dermatologists' assessments can boost the detection of skin cancer and improve PCC referrals to dermatology. This is particularly beneficial in easing patient concerns regarding benign lesions, such as abnormal pigmented lesions and seborrheic keratoses. This approach effectively bridges the gap between specialized dermatologic care and the day-to-day requirements of primary care, highlighting the practicality and relevance of our findings in routine clinical practice in primary care.
Acknowledgments
We thank David J. Leffell, MD, and Daniel M. Siegel, MD, for serving as expert dermatology panelists for this study. We acknowledge Anish Scaria and Anna Maria Tablada Aran for their contributions to data management and statistical analyses.
Notes
This article was externally peer reviewed.
Funding: This study was sponsored by a grant from DermaSensor Inc.
Conflict of interest: Laurent Billot was a paid independent consultant who performed the biostatistical analysis. Miguel Tepedino and Nathalie C. Zeitouni were paid investigators for the study. All other authors report no financial conflicts of interests related to the presented work.
To see this article online, please go to: http://jabfm.org/content/37/3/427.full.
- Received for publication July 6, 2023.
- Revision received December 5, 2023.
- Revision received December 11, 2023.
- Accepted for publication January 2, 2024.
