Thanks for visiting Imaging and Machine Vision Europe.

You're trying to access an editorial feature that is only available to logged in, registered users of Imaging and Machine Vision Europe. Registering is completely free, so why not sign up with us?

By registering, as well as being able to browse all content on the site without further interruption, you'll also have the option to receive our magazine (multiple times a year) and our email newsletters.

Retailer employs customer service robot that functions using imaging

Share this on social media:

Cameras, laser scanners and vision software are all allowing Fraunhofer IPA’s new customer service robot, Care-O-bot 4, to greet shoppers and show them around an electronics store in Germany.

The imaging technology enables the robot, alias Paul, to observe and respond to the mood of customers through observing their gestures and facial expressions. Paul has been greeting customers at electronics retailer Saturn's store in Ingolstadt, Germany since the end of October.

Paul, a member of the Care-O-bot 4 robot family, has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016. (Credit: Saturn)

For the Saturn assignment, which will run initially for six months, the Stuttgart-based Fraunhofer IPA team working on Care-O-bot 4 supplemented its software and hardware with additional features. ‘For example, we specified Paul’s navigation, dialogue communication and facial expressions,’ explained Ulrich Reiser, Fraunhofer IPA project manager.

A set of laser scanners implemented at shin level monitor the robot’s orientation, allowing it to navigate the store effectively. In addition, its front-facing camera, together with Fraunhofer IIS’ Shore software, enable it to ascertain the emotional state of the customer. The robot is then able to communicate with the customer through a system of microphones and cameras for speech and gesture recognition. If they ask it about a certain product, it accompanies the customer to the department and points them in the direction of the relevant shelf.

Fraunhofer IPA is now working with its spin-off company, Unity Robotics, to ensure the solution is suitable for use in various commercial environments. Saturn is the first retailer to make use of the robot platform in a real-life scenario. ‘With Paul, we are offering our customers the opportunity to get to know one of the most advanced robots in the world,’ said Martin Wild, chief digital officer at Media-Saturn-Holding.

The robot has a modular construction and an open software interface, meaning it can be used in many different application areas. It could, for example, conceivably be used as a mobile information kiosk in shops and airports, for pick-up and delivery services in homes and offices, for security applications, or as an attraction in a robotics museum.

‘We retain an interest in ensuring that as many scientists and companies as possible are able to work with the system in order to expand its potential areas of application,’ Reiser of Fraunhofer IPA explained. Within the framework of the Care-O-bot 4 research platform, experts from around the world can further develop the software and hardware components. Together with Unity Robotics, Fraunhofer IPA experts are intending to adapt applications to the needs of different companies, making Care-O-bot 4 available as a commercial product.

Recent News

07 January 2020

The machine is able to inspect a complete bearing at 80 parts per minute, at 5µm height resolution, and simultaneously in colour and 3D

18 December 2019

The engineering firm One Box Vision developed the vision system for quality control of printed sheets such as those used on dashboards

12 December 2019

Coffee bean supplier, Miko Coffee, has installed an optical sorting machine to reduce the amount of foreign material in its beans

15 November 2019

A time-lapse 3D video of a zebrafish heart growing over a day has been captured for the first time by a new microscope imaging technique