Anomaly Detection#
We're excited to share LandingLens now supports Anomaly Detection Projects! In the event you don't have enough defective images, you can now create an Anomaly Detection model which is trained entirely on normal or ok images. Anomaly Detection allows users to quickly get to model building and deployment. This is useful especially when you're launching a new product and are unsure what sort of defect types to expect.
Data#
Our upload feature enables you to upload and classify pre-organized images, meaning there is no need to go through long labeling tasks. Unlike other models in LandingLens, Anomaly Detection is an Unsupervised Model (it isn't trained explicitly on labeled media). While this saves a lot of time, it means the accuracy of your folder organization is absolutely paramount to training a good model.
In an anomaly detection project, there are two classes:
Normal media are used to train the model. Object location and lighting should be as consistent as possible to ensure good anomaly detection.
Abnormal media are used to test your model once trained. The anomaly should be clearly visible to ensure detection.
To start, organize your images into folders based on whether they are "normal" representations of your item or "abnormal" defective examples of your item.
Next, navigate to the upload page and drag & drop each folder to the corresponding section.
- After uploading, you will see all the images in the data browser tagged with the corresponding class of the upload type.
- Finally, split the data as you would normally and export. As mentioned earlier, Anomaly Detection models are only trained on "normal" data - as such when you auto-split the data, you'll notice that you are unable to add "abnormal" data to the training set.
Model#
Launching anomaly models is easy - LandingLens automatically sets model type to the latest anomaly model. Simply select your exported data set, adjust any hyperparameters you want and launch the training job.
Once your model run completes, you'll see your new anomaly detection model in the overview page.
Error Analysis#
Anomaly detection models are much more sensitive than traditional models in LandingLens - you should expect a higher overkill (False Positive) rate. With this in mind, Anomaly Detection is a great approach when you're not sure what types of defects to expect.
Deployment & Model Drift Alerting#
We're pleased to announced that LandingLens Deployment now supports Alerts. Reliably monitor each devices' Defect Rate and Model Confidence
To create alerts, navigate to the Deployments module and choose "Manage" from the Alerts card.
Next click "create" to pull up the Alert creation pop-up. Give it a name, select the target device and choose one of the alert options. Today we support two types of alerts
- Average Defect Rate is the number of defective parts over the total number of parts inspected. Generally it is a good sign if this number is low.
- Average Confidence measures how confident the model is in all predictions made over the time range. Normally this number should be high.
Next choose a threshold at which you want the alert to be triggered.
For Average Defect rate, this is triggered whenever the average value breaches the set threshold.
Average Confidence Rate is triggered whenever the average value falls below the set threshold.
Next choose a time window over which you want your "average" to be calculated. In other words, alerts are monitored based on a rolling average. For example, if you were to monitor an average defect rate over a week - the Alert would only be triggered if the rolling average from the previous week breached your set threshold.
Finally, click the + and select the recipients of the alert, which will be delivered in the form of an email.
Human in the Loop (HiLo)#
We're excited to share that we've completed our initial release of Human in the Loop. Now customers can combine insight from both their models and their teammates to ensure complete recall of defective items. Human in the loop augments model performance by sending defective samples to human reviewers. It also can send a random sample of predictions to be double checked by a human. This is ideal for customers just getting started with augmenting their processes with models - HiLo helps your team build confidence in the system by manually checking LandingLens results.
More instruction on this feature to come...