VisioMel Challenge: Predicting Melanoma Relapse

Use digitized microscopic slides to predict the likelihood of melanoma relapse within the next five years. #health

€25,000 in prizes
may 2023
541 joined

alt-text


The results show that the algorithms are at least on par with the traditional prognostic factors. In the near future, it will undoubtedly be possible to have a predictive digital signature of melanoma recurrence.

— Frédéric Staroz, President of the Conseil National Professionnel des Pathologistes (CNPath)

Why

In 2020, over 325,000 people were diagnosed with skin melanoma, with 57,000 deaths in the same year.1 Melanomas represent 10% of all skin cancers and are the most dangerous due to high likelihood of metastasizing (spreading).2

Patients are initially diagnosed with melanoma after a pathologist examines a portion of the cancerous tissue under a microscope. At this stage, the pathologist assesses the risk of relapse—a return of cancerous cells after the melanoma has been treated—based on information such as the thickness of the tumor and the presence of an ulceration.

Combined with factors such as age, sex, and medical history of the patient, these microscopic observations can help a dermatologist assess the severity of the disease and determine appropriate surgical and medical treatment. Preventative treatments can be administered to patients with high likelihood for relapse. However, these are costly and expose patients to significant drug toxicity.

The Solution

Assessing the risk of relapse is a vital but difficult task. It requires specialized training and careful examination of microscopic tissue. Currently, machine learning approaches can help analyze whole slide images (WSIs) for basic tasks like measuring area. However, computer vision has also shown some potential in classifying tumor subtypes, and in time may serve as a powerful tool to aid pathologists in making the most accurate diagnosis and prognosis. 3, 4, 5

The goal in this challenge was to predict whether a relapse will occur in the 5 years following the initial diagnosis using digitized versions of microscopic slides and tabular clinical features. For each slide, participants had to predict a score between 0 and 1 indicating the likelihood of relapse.

The Results

Challenge participants generated over 600 submissions, and the winning solutions achieved log loss scores of 0.39-0.40 (lower is better) compared to 0.50 for a benchmark model that predicted relapse from tabular clinical features. In terms of area under the ROC curve, winning submissions substantially outperformed the benchmark with AUC scores surpassing 0.80.

Receiver-operating characteristic (ROC) analysis of the VisioMel winners

The results suggest that computer vision can detect signals in whole slide images that help make more accurate predictions of relapse months to years into the future.

See the results announcement for more information on the winning approaches. All of the prize-winning solutions from this competition are linked below and available for anyone to continue to use and learn from.


RESULTS ANNOUNCEMENT + MEET THE WINNERS

WINNING MODELS ON GITHUB