LandingLens Defect Book Cheat Sheet

Use these tips to construct a high quality defect book!

Tip 1: Describe your defect's visual features

• Have clear and practical labeling rules. Avoid using words such as "big" or "small". Instead, describe in numerical terms like number of pixels or a reference object. It is especially helpful to write descriptions gives labelers visual clues in categories: color, size/length, shape, texture, density, contrast, location
• Include real labeled examples. Avoid having examples in which the defects are only pointed or highlighted, it's important for the labelers to see how the labeled defect should look like.

Tip 2: Provide counterexamples

• Provide counterexamples for visually similar classes
• Give instructions for how to distinguish classes in the defect description

Tip 3: Give instructions on defect boundaries:

• Explicitly addressing how to label multiple instances
• Giving pixel-wise rules for how tight to make bounding boxes or segmentation areas

Tip 4: Provide clear explanations of borderline cases

• Include an image spectrum of cases for each defect. This spectrum will help labelers differentiate between “OK” and “NG” cases, so be sure to include difficult or borderline cases, not just obvious ones!
• Determine a measurable threshold that samples can be evaluated against. Labelers should be able to easily decide if a case is “OK” or “NG” based on this threshold.

Tip 5: Give examples of rare cases

• 4 representative examples
• 3 counterexamples
• 3 borderline cases
• Some edge cases if needed