Thanks for visiting Imaging and Machine Vision Europe.

You're trying to access an editorial feature that is only available to logged in, registered users of Imaging and Machine Vision Europe. Registering is completely free, so why not sign up with us?

By registering, as well as being able to browse all content on the site without further interruption, you'll also have the option to receive our magazine (multiple times a year) and our email newsletters.

Open source robotics gains traction in manufacturing

Share this on social media:

Tags: 

Credit: Drag&bot

Christoph Hellmann Santos, of the ROS Industrial Consortium Europe and Fraunhofer IPA, explains why more industrial users are now building robots through the open source software platform, Robot Operating System

In recent years, robot automation has tapped into the rise of open source software, notably with the success of the Robot Operating System (ROS). ROS gives developers control of a wide range of industrial robots using modern programming languages such as Python or C/C++. It is also a great for robot systems incorporating machine vision, as easy integration of different cameras, robots and image processing tools makes it ideal for these types of application.

Currently, more than 140 different robots can be controlled using ROS, including autonomous ground vehicles, drones, agricultural robots or even underwater robots. Developers originally used ROS mainly in research-oriented applications, but currently we see a strong trend towards industrial applications. In addition, thanks to the H2020 project, ROSIN, the quality of the ROS open source software has been brought close to industrial standards.

Why is the open source model becoming so popular? The answer lies partly in Marc Andreessen's famous statement that 'software is eating the world', which continues to be true including in robotics.

Progress in robot automation can only be achieved by creating better, more intelligent software, as robot kinematics have stayed the same for decades. However, creating smarter software is quite challenging and requires complex software engineering as, in addition to designing a completely new automation function, many other problems need to be solved. Without a large software development team, building robots for complex automation problems is close to impossible using conventional methods. One answer to this is to use open source software such as ROS.

ROS is a software platform that provides tools and components for solving most of the problems that are not directly related to an actual new automation function. Thus, it offers a fast track to new technology, by taking advantage of the common, shared R&D that has already been solved around the world. When developing a robot application based on ROS, the engineer can simply reuse the shared code, and focus entirely on putting their own, unique functionality on top. Resources can therefore be focused on value-added work, which gives a competitive advantage.

To pay back, the engineer or organisation contributes to the common open source platform, by providing improvements, bug fixes and new modules that they would like to be developed further by the open source community. ROS, for example, provides intra-process communication, support for all major industrial robots and path planning for industrial robots. Currently, more than 2,000 software packages are available for ROS.

One of the major advantages of using ROS for building robot automation solutions is the ease of integrating machine vision libraries such as OpenCV or Point Cloud Library (PCL). This means robot software developers can create systems that require complex machine vision. There's also the advantage that a vision solution can be switched easily to another type of robot when working with ROS.

A rising number of organisations providing robot solutions for manufacturing have realised this advantage and joined forces in the ROS Industrial Consortium, spreading ROS in industrial applications. Currently, the consortium has approximately 80 members. The open source community around ROS, consisting of developers, researchers and companies, is what makes ROS tick and enables fast development of new software components.

Recognising dirt

The robotics team of Fraunhofer IPA’s robot and assistive systems department uses ROS for industrial machine vision applications for robotics. Currently, the team is working on dirt recognition for mobile cleaning robots, programming the robot to only trigger cleaning when it identifies dirt. This saves energy and detergents, and reduces wear on the cleaning tools.

For its dirt recognition module, the team uses ROS because the ecosystem gives access to advanced vision libraries. In addition, support tools such as visualisation with RVIZ are available out of the box. Moreover, ROS enables the simple integration of the developed dirt detection modules into ROS-based mobile robots. By using bridging mechanisms, integration into robots running other control software is also possible. Metralabs and Bosch Engineering have tested the dirt detection module on their robots.

Richard Bormann, team manager at Fraunhofer IPA, commented: 'We develop using ROS because our modules can be easily integrated into a large number of mobile robots.'

Drag&bot's GUI for building robot applications

Martin Naumann, CEO of German software firm Drag&bot, added his reasons for working with ROS: 'ROS as reliable robotics middleware allows us to offer common interfaces to integrate all kinds of periphery and functionality – such as cameras and vision algorithms – into Drag&bot, and also to profit from new developments of the ROS community.'

Drag&bot offers a software suite designed to simplify and unify programming for industrial robots. Its software enables workers to program robots without special training, and reduces programming time, integration costs and automation project duration.

Drag&bot software provides a graphical user interface for building robot applications using function blocks. To program a robot, the worker only needs to put these function blocks in the right sequence using a drag and drop mechanism, and configure the function blocks' parameters in the graphical user interface.

The software also offers the integration of 2D and 3D vision function blocks, for example for workpiece pose detection or bin picking. Drag&bot relies on ROS to provide customers with a manufacturer-independent robot programming solution, and makes use of ROS' machine vision capabilities.

A third ROS user is the Norwegian robot systems integrator, PPM Robotics. PPM provides industrial robots for grinding, deburring, polishing and welding, and is the main developer of Nachi's program interface FlexGui 4.0, which is also released as open source for ROS.

PPM has built ROSweld, which is the first heavy robot welding system using machine learning in the welding process modelling for planning multipass welding. ROSweld also includes planning from CAD models, graphical multipass welding cross section handling, simulation, and welding camera integration.

The machine vision system uses FlexGui 4.0 as the user interface, with fully reprogrammable workpieces, filters, parameters, and playback. The vision system is robot and camera independent all through the ROS platform.

Trygve Thomessen, managing director of PPM Robotics, said: 'As a provider of the newest technology for industrial robotics, PPM would never survive by developing our solutions from scratch. Imagine, how to develop a cutting-edge 3D vision solution during three weeks, without ROS?'

PPM Robotics' ROSweld system

Vision system manufacturers are also recognising the growing demand for vision systems with an ROS interface. Today, many of the leading industrial vision equipment providers offer ROS interfaces for their products, such as Sick, Ensenso, Photoneo, and Zivid. The ROS Industrial 3D camera survey (https://rosindustrial.org/3d-camera-survey) provides a good overview of supported 3D cameras. Furthermore, a growing number of robot OEMs, above all UR, Pilz, ABB, Hanwha and Yaskawa, and component providers such as Keba or Bosch, acknowledge ROS as a de-facto standard for implementing complex robot applications.

Alexander Barth at Keba commented in 2019: 'Previously, integrating a sensor involved a huge amount of work. Now it’s all done by the community.'

Even though ROS has huge advantages for the development of robot automation applications, it also has drawbacks. Using open source software in products is different from using software provided under commercial license. In terms of liability, other rules apply, which the user needs to be aware of when planning to distribute ROS-based applications. Although the large ROS developer community enables fast innovation it sometimes creates short-lived software packages or those without maintenance support. Down the line, this can lead to problems in a ROS-based automation application. If you need help avoiding these drawbacks, contact your local ROS Industrial initiative.

Overall, ROS is a great software framework for machine vision-based robot applications. Robot OEMs, vision equipment and robot component providers, as well as robot application developers, are now using ROS, and we see more and more ROS-based robot applications being deployed into manufacturing plants.

About the author

Christoph Hellmann Santos is group manager in the robots and assistive systems department of Fraunhofer IPA. His group researches software engineering and system integration, and has a focus on open source robotics. He is also programme manager of the ROS Industrial Consortium Europe: Christoph.Hellmann.Santos@ipa.fraunhofer.de

Write for us

Are you using ROS or open source platforms to build vision systems? Let us know your experiences: greg.blackman@europascience.com

Related analysis & opinion

A point cloud of a National Research Council Canada artefact superimposed on a CAD model. Credit: NIST

31 July 2020

How do you choose a 3D vision system for a robot cell? Geraldine Cheok and Kamel Saidi at the National Institute of Standards and Technology in the USA discuss an initiative to define standards for industrial 3D imaging

Chinese hospitals have ordered 2,000 UVD robots from Blue Ocean Robotics since the start of the pandemic. Credit: Blue Ocean Robotics

09 April 2020

The Covid-19 pandemic has highlighted the importance of robotics and automation in many areas of manufacturing and society, according to CEOs of robotic firms in a webinar run by A3

28 February 2020

Paul Wilson, managing director of Scorpion Vision, describes what it takes to install a 3D robot vision system in a Chinese foundry

20 June 2019

The UK is up to 20 per cent less productive than its major competitor countries because it is not investing in automation, Mike Wilson at the British Automation and Robot Association said at UKIVA's machine vision conference in Milton Keynes. Greg Blackman reports

Related features and analysis & opinion

MVTec’s Halcon software library includes a deep learning OCR tool with pre-trained fonts from a wide range of industries. Credit: MVTec

03 August 2020

Matthew Dale explores vision solutions for code reading and inspection in pharmaceutical production

Engineers at KYB in front of a pick-and-place solution for handling steel metal cylinders. Credit: Pickit

03 August 2020

Car manufacturing has been hit hard by Covid-19, but the need for automation on production lines has not diminished, as Greg Blackman finds out

A point cloud of a National Research Council Canada artefact superimposed on a CAD model. Credit: NIST

31 July 2020

How do you choose a 3D vision system for a robot cell? Geraldine Cheok and Kamel Saidi at the National Institute of Standards and Technology in the USA discuss an initiative to define standards for industrial 3D imaging

Two robots have been installed at Aalborg University Hospital in Denmark. Credit: Kuka

04 June 2020

Keely Portway looks at how robots are automating procedures in hospital testing laboratories, and how imaging underpins this

Chinese hospitals have ordered 2,000 UVD robots from Blue Ocean Robotics since the start of the pandemic. Credit: Blue Ocean Robotics

09 April 2020

The Covid-19 pandemic has highlighted the importance of robotics and automation in many areas of manufacturing and society, according to CEOs of robotic firms in a webinar run by A3

28 February 2020

Paul Wilson, managing director of Scorpion Vision, describes what it takes to install a 3D robot vision system in a Chinese foundry