Thanks for visiting Imaging and Machine Vision Europe.

You're trying to access an editorial feature that is only available to logged in, registered users of Imaging and Machine Vision Europe. Registering is completely free, so why not sign up with us?

By registering, as well as being able to browse all content on the site without further interruption, you'll also have the option to receive our magazine (multiple times a year) and our email newsletters.

Retailer employs customer service robot that functions using imaging

Share this on social media:

Cameras, laser scanners and vision software are all allowing Fraunhofer IPA’s new customer service robot, Care-O-bot 4, to greet shoppers and show them around an electronics store in Germany.

The imaging technology enables the robot, alias Paul, to observe and respond to the mood of customers through observing their gestures and facial expressions. Paul has been greeting customers at electronics retailer Saturn's store in Ingolstadt, Germany since the end of October.

Paul, a member of the Care-O-bot 4 robot family, has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016. (Credit: Saturn)

For the Saturn assignment, which will run initially for six months, the Stuttgart-based Fraunhofer IPA team working on Care-O-bot 4 supplemented its software and hardware with additional features. ‘For example, we specified Paul’s navigation, dialogue communication and facial expressions,’ explained Ulrich Reiser, Fraunhofer IPA project manager.

A set of laser scanners implemented at shin level monitor the robot’s orientation, allowing it to navigate the store effectively. In addition, its front-facing camera, together with Fraunhofer IIS’ Shore software, enable it to ascertain the emotional state of the customer. The robot is then able to communicate with the customer through a system of microphones and cameras for speech and gesture recognition. If they ask it about a certain product, it accompanies the customer to the department and points them in the direction of the relevant shelf.

Fraunhofer IPA is now working with its spin-off company, Unity Robotics, to ensure the solution is suitable for use in various commercial environments. Saturn is the first retailer to make use of the robot platform in a real-life scenario. ‘With Paul, we are offering our customers the opportunity to get to know one of the most advanced robots in the world,’ said Martin Wild, chief digital officer at Media-Saturn-Holding.

The robot has a modular construction and an open software interface, meaning it can be used in many different application areas. It could, for example, conceivably be used as a mobile information kiosk in shops and airports, for pick-up and delivery services in homes and offices, for security applications, or as an attraction in a robotics museum.

‘We retain an interest in ensuring that as many scientists and companies as possible are able to work with the system in order to expand its potential areas of application,’ Reiser of Fraunhofer IPA explained. Within the framework of the Care-O-bot 4 research platform, experts from around the world can further develop the software and hardware components. Together with Unity Robotics, Fraunhofer IPA experts are intending to adapt applications to the needs of different companies, making Care-O-bot 4 available as a commercial product.

Recent News

15 November 2019

A time-lapse 3D video of a zebrafish heart growing over a day has been captured for the first time by a new microscope imaging technique

26 September 2019

Rugby fans are now able to watch highlights from the Rugby World Cup, currently taking place in Japan, from angles and viewpoints not possible with conventional cameras, thanks to a multi-camera system from Canon

13 September 2019

A hyperspectral imaging system built by US research centre Battelle, using Headwall sensors, has been chosen as a finalist for the Department of Homeland Security’s Opioid Detection Challenge

23 July 2019

On the 50th anniversary of the Moon landing on 20 July 1969, Zeiss has described how, in less than nine months, it built the camera lens used to capture the iconic images during the Apollo 11 mission