Caption: Huron Digital Pathology's whole slide scanners can be used to create digital images of entire slides in less than a minute
Three years ago, camera manufacturer Basler recognised the medical and life science market as its second most important market for imaging after factory automation.
While historically there has been an overlap between the cameras used by the factory automation and medical and life science markets, according to Peter Behringer, team leader of Basler’s medical product market management, with features such as high reliability and robustness being valued by both, the requirements of the two markets are quite different. This has led to certain aspects in medical and life science imaging that can’t be covered by classical machine vision cameras.
From the beginning of 2018, Basler has been developing its first range of cameras dedicated to the medical and life science market. It has done this by not only equipping the devices with a more suitable range of features for medical and life science applications, but also by getting its quality management system (QMS) certified by the international standard ISO 13485:2016, which defines the requirements for quality control in the design and manufacture of medical devices. With the ISO 13485-compliant Med Ace series, released in March earlier this year, Basler intends to reduce the time and complexity it takes its customers to get a life science product using imaging to market.
‘For medical device manufacturers it is getting more and more difficult to bring systems to the market, due to the strict rules of legal authorities,’ said Behringer. ‘Typically, they have to go through between a six-month and one-year design end phase, followed by around a year of certification in which they need to perform testing, produce documentation and communicate with legal authorities. By using ISO 13485-compliant products for the components of their systems, these manufacturers can save time performing internal testing, due to the amount of testing that has already been done at the facilities producing the products.’
In order to be ISO 13485-compliant, a camera must be produced in a validated production facility, where not only the device itself is tested extensively, but where the tools being used to test it are also tested thoroughly.
‘It’s an ultra-safe production environment,’ Behringer explained. ‘The ISO 13485 certification also means that customers who purchase these products are notified if a change to the product needs to be made, such as if a new firmware update is needed, or if a component is reaching end-of-life and has to be replaced.’
He added that ISO 13485 is not only happening in Europe, but also in the US, with the FDA planning to integrate the ISO13485:2016 into its 21 CFR 820 – the quality system regulation for medical devices sold in the United States. Currently, the differences between FDA regulations and ISO 13485 are less than 5 per cent, with plans in place to reduce this further. The FDA plans to roll out its revised quality system regulations by April.
‘Next April there will be a deadline that the ISO 13485:2016 version needs to be implemented at medical device manufacturers,’ said Behringer. ‘This means from April on, these manufactures will have the need for a dedicated supplier management system. From April onwards there will be a lot more discussion about supplying ISO 13485-compliant products.’
According to Basler, the Med Ace is the first ‘standard’ camera on the market that is produced, distributed and serviced under ISO 13485:2016 certification.
While Behringer expects other camera manufacturers to follow suit in producing ISO 13485-compliant products, he believes this will be very challenging to achieve. ‘For medium-sized camera manufacturers, especially, this will not be a very easy process, because the ISO certification is very complex, and a very high-quality level will be needed, before it can even be implemented.’
A screen capture from a Huron slide scanner. Credit: Huron Digital Pathology
A range of new feature sets that target medical and life science applications have been developed for the Med Ace, each of which take advantage of the cameras’ Sony Pregius or On Semiconductor Python CMOS sensors. These include a low light imaging feature set, to capture weak light signals from fluorescence microscopy, and a colour calibration, which will benefit medical imaging in many clinical diagnostic applications that rely on exact colour information. Ophthalmology, for example, will benefit from the Med Ace’s high colour accuracy, according to Behringer.
Switching to CMOS
The increasing prominence of DNA sequencing has opened up new fields of research, such as cell therapy, synthetic biology and personalised medicine, into which imaging and machine vision distributor Multipix – a supplier of Basler’s Med Ace cameras – has been selling into, according to Simon Hickman, the firm’s director. He explained that CMOS sensors from Sony and On Semiconductor that offer a high dynamic range, exceptional low-light sensitivity, and very good quantum efficiency, are now driving a lot of these applications, and that they have been increasingly taking over from the cooled CCD technology still used by a lot of researcher customers.
‘Noise reduction is a key factor in life sciences. Lots of applications in the field involve looking at very small levels of fluorescence, where high sensitivity and low noise are the key parameters,’ Hickman said. ‘These new sensors also offer very low dark current noise in addition to low-read noise – which itself is less than 10 per cent of what would be expected from an equivalent CCD sensor. This enables these new CMOS sensors to be used in applications where you would have previously required a cooled CCD sensor, the setup for which is very expensive and considerably larger.’
According to Behringer, the CCD cameras still used in many research and high-end applications are in the range of €10,000, whereas cameras such as the Med Ace, based on new CMOS technology, can be found starting at around €600. These cameras can also be offered in a much smaller physical footprint – the Med Ace is around 29mm3 in volume and weighs around 80g – because they don’t require a cooling system, which CCD-based cameras typically do. Behringer expects CMOS technology will become even more powerful in the future, with upcoming sensors from Sony and other manufacturers, and that over the next two years the high-end market of cooled CCD technology will be replaced by very compact cameras using CMOS sensors.
One firm that has been reaping the benefits of advances in CMOS technology is Huron Digital Pathology, which incorporates area scan CMOS RGB cameras from Teledyne Dalsa into its scanning systems, which are used to image glass biopsy slides containing a patient’s tissue samples.
Basler's Med Ace cameras were shown attached to microscopes at the Analytica trade show
‘In the past, line scan cameras were commonly used in many whole slide scanning systems in digital pathology,’ said Patrick Myles, CEO of Huron. ‘By standardising on area scan cameras, we were not only able to increase the scan speed significantly – by being able to do a single pass versus three passes with line scan – but we were also able increase the perceived quality of the image being produced. We can image a 15mm2 area at 20 times magnification in less than a minute, and can resolve features as small as 0.2µm.’ Myles also said this adoption of area scan cameras into applications that have traditionally used line scanning is accelerating thanks to advances in CMOS technology.
Huron’s systems are digital microscopes known as automated whole slide scanners. These systems consist of an LED light source sitting beneath a stage that holds a slide containing a tissue sample. Over this is positioned an objective lens, additional optics, and then finally an area scan CMOS RGB camera from Teledyne Dalsa.
The scanners operate by moving the stage beneath the objective lens, while the camera captures multiple image strips of the slide, which are then combined by a computer to create a digital whole slide image. The scanner does this for each of the 120 slides that can be loaded into the system, outputting BigTIFF images of each slide that can then be viewed on a computer and zoomed in on if detail is required on the microscopic level.
A camera with a high frame rate is needed so that the stage can be moved quickly and a whole slide imaged within a minute. ‘The cameras we use provide 60fps at 12 megapixels. However, we run them in a windowed mode, so they’re actually running at thousands of frames per second.’
While the speed and quality of imaging in life science is increasing thanks to the capabilities of CMOS, a resulting challenge is that a larger amount of data is now being produced that needs to be handled and processed.
‘Digital pathology, particularly concerning whole slide scanners, is a very data-intensive application of machine vision,’ confirmed Myles. ‘We’re generating multiple gigabytes per second in the form of huge datasets.’
In general, according to Myles, the mainstream of the life science market – particularly for the large labs processing millions of slides – therefore relies on fast scanners with powerful workstation computers attached to them.
‘We’ve been able to take some of the image processing functions that we do – such as debayering – and perform them on a GPU inside the computer, rather than on a frame grabber,’ he explained. ‘Every scanner in our category … has a powerful workstation with a multicore processor, a GPU and lots of hard drive space.’
Myles feels there is a big drive towards ease of use, cost effectiveness, reliability, and ease of integration. The systems also need to be portable, so that they can be placed on pathologists’ desks and in multiple parts of the lab, or wherever the tissue is.
Embedded automation
Systems harnessing embedded imaging – where processing is moved closer to the image sensor and performed on-board the system – are already present in the life science market, and are becoming increasingly popular for automating applications in laboratory environments.
‘Laboratory automation is something that has really picked up significantly over the past two to three years. Big companies in the medical field have taken up this trend to improve their systems with vision,’ said Martin Klahr, head of the digital imaging division at Phytec, a manufacturer of embedded imaging systems. ‘Increased safety and security for patients is sought after in the medical field now, and vision will be used to help deliver healthcare automatically around the clock.’
Phytec has many different projects in the life science laboratory automation market, according to Klahr, not only in large hospitals, but also in smaller laboratories and even doctors’ offices. ‘These embedded imaging systems are becoming increasingly affordable, and could sell in the thousands per year. They could really increase the level of diagnosis in the field,’ he said.
One such project, for which Phytec built the image acquisition system and the control electronics, is the fully automated diagnostics system Unyvero, from molecular diagnostics firm Curetis. The system is capable of imaging DNA markers from a patient sample and processing the results in order to make an accurate diagnosis of disease within four hours – compared to days taken by past systems.
‘Using the Unyvero system, all microbiological processes for DNA analysis of pathogens are carried out in a fully computer-controlled laboratory cartridge,’ Klahr explained.
‘An integrated camera system captures the position and intensity of DNA markers in an image that is then processed on the machine – using image processing electronics based on modular embedded imaging components.’
A laboratory cartridge in which DNA markers are imaged in the Unyvero analyser. Credit: Phytec and Curetis
The camera used in the system, rather than requiring a high resolution, was instead developed to be sensitive to the particular wavelengths emitted by fluorescence of the DNA markers. The camera head was therefore designed – on a project-specific basis with Curetis – to contain a beam splitter and an intelligent LED lighting unit, as according to Klahr the wavelength of the fluorescence is different to that of the LEDs. The size of the camera was also important, as it had to be moved around inside the system in order to capture images of different markers in different positions within the laboratory cartridge.
‘This isn’t a standard USB camera,’ he commented. ‘The idea behind the embedded concept was that we could directly connect the CMOS sensor on our plane interface to the camera interface of the microprocessor that is used inside this embedded system.’
The embedded system designed for the Unyvero by Phytec contains both the camera and the complete control system that manages the multiple sensors, motors, heaters and pumps of the device, in addition to its data communication.
Phytec uses a modular concept for its embedded vision systems – it has modular microprocessor boards that can connect to its camera modules – which keeps design costs down.
While Phytec currently produces all the camera, boards and electronics of its embedded vision systems itself, for the future Klahr can see there being potential for collaboration with companies offering very sophisticated cameras for special applications, which Phytec could combine with its own embedded products.
‘For example, our current products do not cover applications requiring a camera with very high light sensitivity,’ Klahr said, ‘so we could search for companies who have been in this market for a long time and have special products that are able to connect to embedded systems.’
