| Web: | mayocliniclabs.com |
|---|---|
| Email: | mcl@mayo.edu |
| Telephone: | 800-533-1710 |
| International: | +1 855-379-3115 |
| Values are valid only on day of printing | |
| Web: | mayocliniclabs.com |
|---|---|
| Email: | mcl@mayo.edu |
| Telephone: | 800-533-1710 |
| International: | +1 855-379-3115 |
| Values are valid only on day of printing | |
You are asked to test a computer vision application created to detect Anomaly X on digital pathology slides. Anomaly X is considered to be a rare event. You are provided the following:
Model Type: Logistic Regression Image Classifier
Images in training set: 56 positive for Anomaly X, 200 negative for Anomaly X
Accuracy of detecting Anomaly X on training set: 0.95
When you attempt to validate the application on 50 of your own digital pathology slides that are known to have anomaly X and 50 slides without, the application is only able to correctly identify 20% of the slides as positive.
A. The machine learning model has been overfitted to the training images.
B. Cross-validation and subsequent hyperparameter tuning was not performed on the training data.
C. Logistic regression is not a valid approach for image classification.
D. Choices A and B.
The correct answer is ...
Choices A and B.
Ensuring that a machine learning model trained on a specific set of data is adaptable to real world data, also known as generalization, is important. Data overfitting is a common issue that is seen with models that have excellent performance with training data but performs poorly on test data. Overfitting is when the model “learns” the unique characteristics of the data it was trained on, rather than on characteristics that would be more broadly applicable to its intended task.
The green line represents a model that is overfitted on the training data to classify between red and blue dots. The black line represents a model that has optimal fit on classifying the two. (Image adapted from https://en.wikipedia.org/wiki/Overfitting#/media/File:Overfitting.svg, licensed under BY-SA 4.0).
Common approaches to prevent overfitting include repeatedly training the model on different slices of the training set, while also leaving out a subset to test the accuracy of the model. This is known as cross-validation. Subsequent adjustment of the model after cross-validation can further help the model’s performance. This is also known as hyperparameter tuning.
Other ways to prevent overfitting include simplifying or decreasing the number of features being used for the model, also known as regularization, and increasing the amount of data available for the model to be trained on.
Choice C is incorrect; logistic regression is a very popular machine learning method for classification, including image classification.
Ray Qian, M.D.
Fellow, Clinical Informatics
Mayo Clinic
Chady Meroueh, M.D.
Senior Associate Consultant, Anatomic Pathology
Mayo Clinic
