Visual Search
Assembly Line
Visual search: how to find manufacturing parts in a cinch
In the modern world, advanced recognition technologies play an increasingly important role in various areas of human life. Recognizing the characteristics of vehicle tires is one such area where deep learning is making a valuable difference. Solving the problem of recognizing tire parameters can help to simplify the process of selecting tire replacements when you don’t know which tires will fit. This recognition can be useful both for customer-facing ecommerce and in-store apps used by associates to quickly read necessary tire specs.
During the research process, we decided that online stores and bulletin boards would be the main data sources, since there were thousands of images and, most importantly, almost all of them had structured descriptions. Images from search engines could only be used for training segmentation, because they did not contain the necessary structured features.
In this blog post we have described the complete process of creating a tire lettering recognition system from start to finish. Despite the large number of existing methods, approaches and functions in the field of image recognition and processing, there remains a huge gap in available research and implementation for very complex and accurate visual search systems.
Visual search: how to find manufacturing parts in a cinch
The process of engineering a robust mechanical product, whether it’s an escalator or a car engine, requires many small parts. We accept that these parts wear out over time and require replacement to avoid breakdowns and to keep the mechanics of the product running smoothly.
During our analysis of the data that the client shared with us, we found a mix of photos of the parts themselves, photos of packages or only product labels. Serial numbers or easily distinguishable characters were clearly visible in some photographs, but not in all of them. One of the primary challenges we faced, therefore, was dealing with the differences between the photos the engineers were submitting compared to the images in the search catalog. For example, there were examples of visually indistinguishable images where only the model number differentiated the part, photos of a sticker with a serial number instead of an object itself, rulers alongside objects in photos to indicate scale, and drawings of the part in the catalog instead of photos.
For this use case we implemented the CNN model based on ResNeXt architecture (ResNeXt-50 (32×4d)) pre-trained on an ImageNet dataset. However, the manufacturing parts we were dealing with were not adequately available in the pre-trained dataset, which meant we had to enhance the training dataset with about 10 000 independently sourced manufacturing part images along with the client-supplied labeled dataset.