Machine vision and packaging inspection traditionally go hand in hand. Cameras have long been used on automated packaging lines to ensure labels are present and straight, or that bottle lids are in place – and the technology behind inspection systems has advanced over time to deal with increasingly complex applications. Vision systems must also be able to handle the high throughput of products associated with packaging lines.
Swedish company Sick IVP, part of the Sick Group, provides industrial vision cameras for factory automation. Its IVC-3D camera is used in numerous packaging applications including verifying that the caps of bottles and liquid containers are properly sealed.
The camera records the profile of a laser line as the lids of bottles move past on a conveyor belt, which provides a 3D image of the shape of bottle caps. The camera is used to measure the height of caps and make a comparison to a horizontal reference surface, within which there are certain tolerances to which bottle caps must adhere. The angle of the cap is also analysed to ensure lids are correctly threaded and secured.
Lid integrity is important to avoid spillage, which can cause damage to the container, and also to avoid ruining the contents of the bottle if the lid is not properly sealed.
‘Using a 2D inspection solution for ensuring lid integrity only supplies the operator with one perspective and, even though the camera can detect deviation in the height or angle of the lid, it all depends on the viewing angle,’ explains Peter Nilsson, business development manager at Sick IVP. If the product moves along the conveyor belt in a fixed position, any deviation can only be seen from one side with one camera. Other cameras can be added for 360° coverage of the lid, but using more than one camera leads to integration problems.
In addition, when dealing with liquids, there is the possibility of condensation on the bottles, which can interfere with the images captured by 2D cameras. ‘Sick’s 3D vision systems are much less sensitive to influences such as condensation or the colour of printed labels, as the sensors detect variations in the shape of the laser line rather than changes in image contrast. This allows for a much more robust and reliable inspection method for packaging lines,’ says Nilsson.
To provide full 360° coverage of labels or caps using a 2D inspection system, the product is simply rotated as it passes the camera. Sightech Vision Systems, based in Santa Clara, California, US, provides intelligent software for the machine vision industry and the company’s PC-Eyebot inspection system is used in various industries, from medical, pharmaceutical and biotech to semiconductor and solar panel inspection.
The intelligent, self-learning inspection system is also used heavily for packaging applications, such as fill levels, cap inspection and label inspection and incorporates the full line of FireDragon progressive scan cameras from Toshiba Teli. ‘The inspection system is taught to identify defects in a product by comparing the images of actual parts moving along a production line to those of model parts,’ explains Scott Skochenko, vice president of sales at Sightech.
In label inspection on canning or bottling lines, for instance, PC-Eyebot is used for 360° wraparound inspection of products on labelling equipment from German company Krones. The bottles or cans are rotated as they pass the PCEyebot camera, allowing the system to identify defects such as dents, shifts in label position, missing or torn labels, and overlapping label ends. The system defines multiple inspection areas on a rotating bottle by using frame-delay technology, which results in a pass/fail decision at production speeds of up to 1,400 parts per minute.
The Matrox Imaging Library is used to identify complex product shapes in pick-and-place applications with the Astor Assortment Placer from Swiss company Demaurex, part of the Bosch Packaging Group. Image courtesy of Demaurex.
‘Colour recognition is particularly important in labelling applications, where print runs can reach up to hundreds of thousands of labels per batch,’ says Skochenko. ‘Over the course of printing, dye levels within the printing machines will fluctuate and there will be variation in the hue and vividness of the colours.’ The Coloration mode of PC-Eyebot allows not only a spectrum of different colours to be recognised, but also colour patterns. Sightech is also developing a mode called Auto Learn, which will allow the system to accept slight changes by learning them over time.
Food packaging
Packaging food items is particularly difficult for vision systems due to the variability in the product. The Swiss company Demaurex, part of the Bosch Packaging Group and a provider of automated packaging solutions, uses vision in combination with its Delta robots for pick-and-place applications, mainly for companies producing chocolates and biscuits. For instance, its Astor Assortment Placer has been installed on production lines of a French biscuit and chocolate manufacturer to sort biscuits into different-sized blister packs.
The biscuit manufacturer produces 30 million assortment packs each year at one of its factories; the Astor Assortment Placer was employed to manage 15 different blister pack sizes filled with up to 20 different products. ‘Vision is closely linked to Delta robot applications. It is the eyes of the robots,’ says Dragan Dragojlovic, product manager at Demaurex. The systems use the Matrox Imaging Library (MIL) software from imaging hardware and software specialists Matrox Imaging.
The vision system was required to sort a variety of complex shapes and to correctly identify broken or upside-down biscuits to maintain the quality of items loaded into the blister packs. The images were processed in real time, ensuring the blister packs were filled without stopping production flow.
The PC-Eyebot from Sightech Vision Systems is used on canning or bottling lines to ensure labels are straight and caps are present. Image courtesy of Sightech Vision Systems.
‘MIL is used for recognition of complex product shapes, to find the product orientation and to differentiate between two products touching each other,’ explains Dragojlovic. Early vision systems were limited in their capabilities and the identification of two products touching each other, for instance, was difficult. ‘MIL increased our capabilities to see the products and also increased the kind of geometry we were able to find,’ he says.
‘One of the main challenges for vision systems inspecting packaging lines is the huge variation in the materials used,’ notes Nilsson of Sick IVP. ‘Artwork and the appearance of packaging change constantly and, even if the product stays the same, new promotional campaigns and decorative material are implemented, which changes the superficial appearance. Systems using 2D cameras identify contrast and colour features, and are effective in applications that make use of this, such as barcode reading,’ he says. As colour and contrast on many packaging lines regularly change, this can be problematic for 2D inspection systems. 3D vision, on the other hand, is less sensitive to these variations, but analyses surface shape, making it much more suited to applications such as box completeness or box integrity.
Inspection systems using 3D vision provide solutions that can deal with vast variations in external appearance and, according to Nilsson, Sick is receiving more and more enquiries into using 3D vision systems for applications where 2D systems were once commonly used.
Skochenko, from Sightech, says: ‘Customers look for robustness and repeatability in a vision system installed on packaging lines. The systems must also be easy to use.’ PC-Eyebot is very user-friendly as it is self-learning and requires no programming.
Optical character recognition (OCR), which, according to Bruno Menard, senior software engineer at digital imaging components provider, Dalsa, ‘is the translation of handwritten, typewritten or printed text images into machine-editable text’, has been used to sort mail since the 1960s. The technology is now also used in a variety of applications, including the semiconductor, automotive and pharmaceutical industries.
Menard states four criteria that should be taken into consideration when selecting an OCR system: flexibility, reliability, ease-of-use, and efficiency. Systems should be able to define multiple variations of a character font: ‘Whether it is solid, dotted or discontinuous (like Asian characters), or whether it is italicised or not, an OCR tool should be able to deal with all these kinds of font attributes at training time,’ he says. Menard cites pattern matching algorithms as one method commonly used for character recognition.
OCR is used in various industries, including pharmaceutical, where the systems are used to record information printed on vial labels. Image courtesy of Dalsa.
Menard says: ‘OCR is often used on images acquired in harsh, industrial environments … and scratches, spots, cracks, gaps, as well as variations in the background are common examples of defects.’
‘An OCR tool presenting an easy-to-learn Application Programming Interface (API) is likely to facilitate and speed-up integration with all other components of the application,’ Menard says. Dalsa provides its image acquisition boards for use in mail inspection and verification systems produced by Document Data Solutions (DDS), based in Connecticut, US.
Prime Vision, with headquarters in Delft, The Netherlands, is a specialist in OCR and automated identification technology. The company’s ParcelVision is an automated parcel handling system using barcode readers and OCR, and which also includes software recognition tools for handwriting. Mark Ryder, international sales manager at Prime Vision, makes the distinction between identifying machine written text and handwriting, the latter being sometimes referred to as intelligent character recognition (ICR).
TNT’s parcel division in The Netherlands has installed ParcelVision in three of its sorting houses. The system, installed at the parcel and freight delivery company’s warehouses, must manage between 17,500 and 18,000 parcels per hour.
As well as parcel handling systems for large courier and parcel delivery companies, Prime Vision also supplies systems for a major British retailer to manage its stock intake. The retailer has more than 400 stores and an extensive mail order service. It receives around 10,000 items per hour into its warehouses. Prime Vision’s ICRlite system was installed in one of its warehouses to register stock and automatically update stock levels on the Warehouse Management System (WMS).
‘A lot of stock comes from the Far East, where labels are often handwritten. These parcels are then shipped to the UK, where most of the storage and retrieval systems are automated. Handwritten labels can be problematic,’ notes Ryder.
‘Language is also a barrier and while most of the auto ID information can be supplied to the manufacturer by the requisitioning retailer, for instance in a barcode, the quantity of items in a box cannot be pre-coded and is often written or stamped on the side of the box,’ says Ryder. Therefore, prior to installing the ICRlite system, an operator would have to manually key in the number of items for each consignment of produce.
High-speed cameras capture images of the boxes as they enter the warehouse, and Prime Vision's software reads both machine-printed data and the handwritten product quantity, cataloguing the stock with a 95 per cent automation rate. The system has been very successful and Ryder sees other retailers employing this kind of ICR software in the future.
Prime Vision’s ParcelVision software uses OCR and ICR to handle the high throughput of packages in TNT’s parcel division warehouses. Image courtesy of Prime Vision.