Surface inspection, typically used for quality control of products ranging from paper to TFT monitors, has now found applications in galleries such as the Louvre, Paris, and the Zwinger, Dresden. Art historians have been using NIR sensors to detect fakes, alterations, and throw light on the creative process of the old masters.
The process is in many respects typical of any surface inspection application: a bright illumination highlights the surface to be studied while a highly sensitive camera captures the image, which is then processed by sophisticated software.
It is mainly used to study underdrawings: sketches, mostly in charcoal and pencil, or brushstrokes, lying directly under the paint layer, and which vary from simple outlines to detailed drawings. ‘You can see the starting point, and if the painter changed his mind,’ says Raf Vandersmissen, a sales engineer at XenICs. ‘It is also used to see if the painting is a real or fake, and if it has been restored.’
Possible underdrawings on the Mona Lisa, studied by a XenICs system.
IR reflectography is used to see through the paint, because it has low absorption and is the least damaging. Like all surface inspection applications, the wavelength depends on the surface being studied, particularly with thicker paint and darker colours. In this area, 900-1700nm is normally sufficient, but sometimes detection up to 2.2µm is necessary.
The cameras must have good IR detection and, to this end, XenICs produces InGaAs detectors. Because high resolution is particularly difficult in IR, many separate images are taken to produce a mosaic of the whole painting. ‘These cameras are easy to use, compact, and do not need extensive cooling,’ says Vandersmissen. ‘In the past, silicon CCD cameras were used. They had a little sensitivity up to 1050nm, but were a lot less efficient.’
In more typical surface inspection applications, visible light is used. Line scan cameras are particularly suitable for detection, because they can provide very high-resolution pictures (more than 12k wide) and are ideally suited for use on conveyor belts, as the image can be infinitely long.
David Hannaby, a vision systems sales specialist at Sick, described a typical setup including a camera, laser line and two ordinary white lights. ‘The Sick Ranger Multiscan sensor is partitioned,’ he says. ‘One part looks at the deflection of a laser line to define the profile of the surface. The other gives an image of the patterns and greyscale on the surface.’ This setup is particularly suitable for tile inspection to detect defects in printed patterns, as well as cracks and 3D defects. The 3D detection is useful for brake pad inspection, where dents are not normally visible using the 2D image alone.
It is also used in the processing of wood. Before a log is cut, its surface is profiled by three cameras to provide a map that will minimise wastage. Additionally, it can be used to inspect the surfaces of boards for dents and knots. These are detected by analysing the way the light scatters differently on knots compared to the rest of the wood.
Complications depend on the surface. ‘In 3D imaging, the intensity of the laser light has a great effect on the accuracy you can achieve,’ says Hannaby. ‘With wood, the laser light sits very well, but with metals there are changes in intensity across the laser line and with dark materials, the intensity decreases as the light would be absorbed.’
Another application is quality control of fabrics, to detect pilling and for crease grading. In the past this would have been judged by humans, but obviously an automated system can work at a greater speed, with greater accuracy.
Basler specialises in line scan cameras with high-speed processing boards that can process the images while scanning is still taking place. Their main applications lie in TFT display inspection. The cameras have to be very flexible, as Dr Michael Fuss, general manager of display inspection, explains: ‘Our products need to be integrated into product lines, with parameters defined by the customer, such as sheet size and the quality level required. Each customer wants to see different defects.’
TFT displays consist of three major components that need to be inspected: the polariser film, glass, and colour filters. The inspection of the polariser film is a continuous process – the plastic is pulled over rollers, checked, and then rolled up for the next stage of production. The process looks for any kinds of defects – dirt, bubbles created by the rollers, or scratches.
For glass inspection, there are two kinds of illumination. In brightfield illumination, the glass is used as a mirror. This is used to detect bumps, which would not reflect, and would show up darker on the image. In darkfield illumination, used to detect small scratches, the camera looks at the surface of the glass, and cannot see the illumination; they could be on different sides of the glass. Perfect glass would show up as black, and scratches would reflect the light to show up as white spots.
Examples of brightfield illumination, courtesy of Basler
The standard line scan rate is around 14k lines per second. Basler’s Sprint can manage five times this amount. Faster inspection clearly has its own benefits, but it does significantly reduce the integration time, which creates problems of its own. To solve these, the Sprint has two lines of pixels, to effectively give more time for integration and a greater sensitivity. Brighter illumination is also necessary, so more light will fall on the sensor at any given time.
Andy Falconer, a director of V Cubed, says: ‘The right lighting, and the right camera, can make the software’s processing job a lot easier.’ V Cubed’s line lights, the VLX1 and VLX2, solve many of the issues highlighted previously. Line lighting is necessary when using line scan cameras, and can be used for a number of different lighting geometries. LEDs are increasingly being used, because they are brighter than other sources such as neon tubes, and allow very fast switching between the different lighting schemes. In addition, V Cubed’s LEDs are water-cooled, meaning they can be even brighter, which allows faster scanning. Water-cooling will also ensure a longer lifespan, and open up more applications where the ambient temperature is naturally hotter, such as steel production. V Cubed’s lighting is also intelligent, so it can correct for variability in lighting; typically, as a result of optical geometry, illumination is brighter at the centre than at the edges.
Parsytec has produced the espresso SI, which is used in steel production. There are two methods to detect and mark defects in the production line. Direct defect marking marks the areas with defects before processing, where detection of the marks will trigger a designated action, such as a scrap cut, whereas indirect defect marking selects the defects automatically, both before and during material processing. In one application, manual inspectors could review the marked defects before the processing procedure, which would further guarantee successful detection.
The Fraunhofer Institute typically provides a full end product for customers. Paper inspection is a common application, to detect blemishes around 0.3mm in size, and dust and scratches in reflective coatings. At this level of precision, 200m of paper per minute may be inspected, but this speed is not possible when inspection at smaller scales is necessary.
It has also installed a system in Sweden to inspect tanned leather. The system detects spots or the marks of insect bites that would look unsightly. It is possible to inspect one hide in 90 seconds, which is then graded and priced depending on the quality. The leather could be used in furniture, or for car seats. Fraunhofer has also provided systems for the inspection of ceiling tiles, for breaks, scratches and rough edges. The tiles typically come out of production at 300°C, with a lot of dust, so the system has to be very robust to cope with the demanding environment.
Even with perfect physical conditions, surface inspection still requires complex algorithms to detect defects. MVTec has produced a software library, Halcon, with integrated development environment, for many applications, including surface inspection. In many cases they provide prototypes of solutions based on the customers demands.
The software contains different kinds of filters that highlight the defects. The surface may have an irregular texture, such as brushstrokes or the grain of wood, so the orientation of the faults may be important. Whether a product is defective may also depend on the number of faults, so a threshold level would need to be defined. Dr Wolfgang Eckstein, managing director of MVTec, says: ‘Some customers just want to know if it has defects, while others need to know where they are, and how big, so Halcon must find all the data, and provide statistics.’
More advanced applications would be the inspection of a surface with a texture or pattern, that has to be removed in the processing by Fourier transforms. ‘For about three years we have been able to do this in real time, which is surprising as it is a very complicated analysis,’ says Eckstein.
‘There is almost no limit to our customers. Halcon is used in medicine, where pills are typically coated and customers are inspecting for colour or to grade the value. The pill’s appearance is important, particularly for identification regulations, and the coating can affect the absorption of the drug into the body.’
Recent improvements to the software have meant that measurement is possible to sub-pixel levels, which could be used in grading scratches dependant on their width. It can also detect and extract the effect of noise – for example, the grain of wood. Eckstein also outlined a method of 3D inspection using just one camera. For very fine surfaces, the depth can be measured by changing the distance between the surface and the camera slightly, and capturing the details that are now in focus. If this is done many times, the user can obtain a full 3D image. ‘This is not high-speed, but provides a very dedicated, local analysis of the surface, for example in the analysis of semiconductor wafers for scratches,’ says Eckstein.
Many believe that 3D imaging will significantly broaden surface inspection applications. David Hannaby says: ‘Most people are aware of 2D inspection, but they are now becoming increasingly aware of 3D imaging too.’ The introduction of colour line scan cameras too seems to be causing quite a stir in the industry. Ronald Roesch, head of the image processing department at Fraunhofer, says: ‘There’s lots of potential for the future. One paper stated that only 10 per cent of applications are realised. We’re just at the beginning of where it will go.’
