Machine vision and barcode reading technology have helped logistics professionals keep pace with increasing demand and customer expectations. For years, warehouse automation has relied on machine vision for barcode reading from inbound to outbound applications that sort packages based on shipping label or destination. But now, supplier demand is driving the need for new levels of automation that require further inspection of item attributes, which has been difficult to address with rule-based machine vision due to the high variation and frequency of changes in package designs. New machine vision technologies, such as deep learning and edge learning, are being developed that are more flexible and can better handle the challenges of sorting items based on attributes.
Machine vision systems are hard to program for item detection and classification applications. Traditional rule-based tools struggle to handle unpredictable variations in size or shape, and it is time-consuming to optimize a vision system for each line. Furthermore, changes in package design can make machine vision systems difficult to maintain because of the large number of thresholds required.
Artificial intelligence (AI) presents a promising solution. AI, or more precisely machine learning and deep learning, can detect, classify and inspect objects, even those that are difficult to define using predetermined rules based on fixed thresholds and logic to determine the resultant criteria. However, the process of gathering and labeling the large datasets necessary to train deep learning AI models can be a hindrance to widespread implementation. The investment is often not transferable within the same applications, and even experienced vision professionals find that deploying deep learning solutions is time-consuming, and not every application justifies the investment required.
This is where edge learning, a specialized form of deep learning that is designed for industrial automation, can make a huge difference. Edge learning improves the accuracy and efficiency of inspections while enabling logistics professionals to scale, optimize operations and minimize manual work and equipment downtime, with zero experience and minimal skillset. Furthermore, edge learning’s ease of use and quick implementation makes it practical for both low-tech facilities and fully automated warehouses.
Edge learning is already being used to execute tasks such as presence-absence detection on cluttered backgrounds, sortation of different items and packaging types, and process issue detection of stuck items that could cause equipment damage, conveyor jams and other shipment delays.
Package Variations Stymy Traditional Machine Vision
Traditional machine vision systems require skilled engineers to evaluate each vision task and create a specific rule to solve the problem. However, rule-based solutions are effective only when items of interest have consistent edges, intensities, or patterns. Conventional rule-based solutions fail to accurately identify features that are difficult to quantify or exhibit unpredictable variations from one item to another, as is the case with logistics packaging that includes boxes, poly bags, jiffy bags, flat packs and totes.
For example, some operations sort packages based on size, routing larger objects to specific conveyors to avoid tight bends while diverting smaller polybags away from roller conveyors, which tend to jam. However, determining whether a small item is a polybag or padded envelope still requires manual intervention, because size alone is generally not sufficient for differentiating between the two.
The maintenance and development of rule-based machine vision algorithms present ongoing challenges, often driven by variation and frequency of changing packaging designs, such as: changes in the appearance of apparel in polybags such as a shirt that gets crumpled and changes shape, or holiday packaging for Halloween, where the same item is wrapped in black and orange promotional packaging designs. Maintaining such machine vision systems necessitates highly skilled and expensive engineers with specialized knowledge, who are in high demand as automation becomes more prevalent across all industries.
Deep Learning Evolution
A decade ago, deep learning was only accessible to data scientists with big budgets. However, recent advancements in theory, computer hardware like GPUs, and the availability of data made it viable to apply deep learning in logistics and warehousing for machine vision applications and other operations. Deep learning has proven effective in scenarios that require subjective judgments, traditionally performed by human inspectors, and complex scenes with challenging image features that involve high complexity or extreme variability.
In contrast to rule-based machine vision, which relies on experts to develop new algorithms, deep learning relies on operators, line managers and other subject matter experts to label images as good or bad and classify the types of defects present. This data-centric, learning-based approach reduces the dependence on experienced machine vision specialists and minimizes the engineering resources needed for deploying and maintaining machine vision solutions. If any changes occur, individuals familiar with the application can potentially retrain the model by capturing and labeling new images.
Deep Learning Toolkit Challenges
While deep learning tools have simplified the implementation of learning-based machine vision systems, there are still challenges that need to be addressed. Successful deep learning projects often require significant financial and time resources, as well as expertise from vision engineers and data scientists to establish the initial system.
Some projects, despite offering improvements, may not provide enough savings to justify the costs. Development of models can be time-consuming, requiring the tagging of hundreds or even thousands of images. Additionally, it may not be practical to gather and label images for every possible variation.
Another challenge involves lighting and image formation. Just like in any machine vision application, the quality of image acquisition hardware is crucial for the success of a deep learning solution. A well-designed imaging system is essential, ensuring reliable and repeatable imaging techniques that can effectively distinguish features or objects of interest.
Edge Learning Bridges Logistics Gaps
For many logistics applications, rule-based machine vision solutions are inadequate, but the return on investment (ROI) may not justify the development and upkeep of a complete deep learning solution. To tackle this issue, machine vision companies have introduced edge learning, an AI-based approach that bridges the gap between deep learning and traditional rule-based vision.
Edge learning is a specialized form of deep learning designed for industrial automation. It comprises two stages: pretraining and specific use-case training. This process requires one to two orders of magnitude fewer images compared to deep learning. For example, showing an edge learning-based system pictures of a small white box and a medium-sized brown box can help it learn to recognize and classify all types of boxes based on their characteristics.
Edge learning also places fewer demands on the hardware, often eliminating the need for a GPU. Embedded smart camera solutions, which combine sensors, optics, processors and illumination, offer quicker and simpler setup and acquisition. As a result, they reduce the complexity of the system and minimize hardware integration issues.
Importantly, edge learning reduces costs and increases flexibility. High-end GPUs can significantly increase hardware expenses and may be challenging to retrofit into existing logistics processes that already have vision PCs without GPUs. In contrast, a smart camera equipped with an embedded NPU or even just a CPU may be capable of training and running edge learning tools.
Edge learning offers major benefits compared to conventional rule-based and deep learning solutions:
• By requiring fewer images, these systems enable users to assess the feasibility of a project with just a few example packages.
• Once a project is approved, the development cycle becomes significantly faster.
• The process becomes much simpler, eliminating the need for data science experts.
• This flexibility is advantageous since adjustments can be made directly on the warehouse floor using embedded smart camera hardware, to quickly adapt to changes in illumination or package type.
• Furthermore, it is very easy to modify; if a new scenario or unexpected result arises, simply add it to the existing model.
Edge Learning: User-Friendly and Scalable
Logistics professionals must increasingly rely on technology to ensure accurate sortation, quick processing, order accuracy and customer satisfaction. As companies strive for faster delivery options to stay competitive, embedded smart camera platforms with user-friendly edge learning capabilities offer a solution for warehouse applications that are too challenging for conventional rule-based machine vision, but do not warrant the investment of a complete deep learning solution.
Compared to traditional machine vision tools, edge learning can quickly and effectively perform accurate inspections using only a few examples. This can significantly reduce the time and investment required compared to traditional deep learning approaches. Anyone who knows what a defect looks like can now set up a model for the majority of sensing applications in as little as 15 minutes. Edge learning technology is effective for inbound sortation because it can classify objects based on their type without needing a large database of images. As a result, edge learning simplifies swift adaptation to high variation and frequency of changes in package designs, allowing for system adjustments with minimal images required for on-the-go model updates.
Edge learning technology can also be used to classify other types of objects, such as polybags, jiffy bags, totes, and even specific item classes with various permutations. The system can be taught to handle these objects differently based on their unique characteristics. Additional edge learning applications include detecting the quality of seals on packaging, checking if totes are empty before they are stacked, and detecting small items in totes, all of which are difficult to accomplish with traditional sensing methods.
The ease of training and setup that comes with edge learning can help logistics professionals understand and implement the technology to achieve fast results.
Edge learning enhances inspections, facilitating the expansion of logistics enterprises, streamlining operations, and reducing the need for manual labor and equipment downtime. Additionally, the user-friendly nature of edge learning allows for its practical implementation in both low-tech facilities and fully automated warehouses. It enables the execution of various tasks, including detecting the presence or absence of objects against complex backgrounds, sorting diverse items and packaging types, and identifying process issues such as stuck items that could lead to equipment damage, conveyor obstructions, and delays in shipments.
Ben Carey is director, logistics product management, with Cognex Corp., a provider of vision systems, software, sensors and industrial barcode readers.
