Retailer employs customer service robot that functions using imaging

Share this on social media:

Cameras, laser scanners and vision software are all allowing Fraunhofer IPA’s new customer service robot, Care-O-bot 4, to greet shoppers and show them around an electronics store in Germany.

The imaging technology enables the robot, alias Paul, to observe and respond to the mood of customers through observing their gestures and facial expressions. Paul has been greeting customers at electronics retailer Saturn's store in Ingolstadt, Germany since the end of October.

Paul, a member of the Care-O-bot 4 robot family, has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016. (Credit: Saturn)

For the Saturn assignment, which will run initially for six months, the Stuttgart-based Fraunhofer IPA team working on Care-O-bot 4 supplemented its software and hardware with additional features. ‘For example, we specified Paul’s navigation, dialogue communication and facial expressions,’ explained Ulrich Reiser, Fraunhofer IPA project manager.

A set of laser scanners implemented at shin level monitor the robot’s orientation, allowing it to navigate the store effectively. In addition, its front-facing camera, together with Fraunhofer IIS’ Shore software, enable it to ascertain the emotional state of the customer. The robot is then able to communicate with the customer through a system of microphones and cameras for speech and gesture recognition. If they ask it about a certain product, it accompanies the customer to the department and points them in the direction of the relevant shelf.

Fraunhofer IPA is now working with its spin-off company, Unity Robotics, to ensure the solution is suitable for use in various commercial environments. Saturn is the first retailer to make use of the robot platform in a real-life scenario. ‘With Paul, we are offering our customers the opportunity to get to know one of the most advanced robots in the world,’ said Martin Wild, chief digital officer at Media-Saturn-Holding.

The robot has a modular construction and an open software interface, meaning it can be used in many different application areas. It could, for example, conceivably be used as a mobile information kiosk in shops and airports, for pick-up and delivery services in homes and offices, for security applications, or as an attraction in a robotics museum.

‘We retain an interest in ensuring that as many scientists and companies as possible are able to work with the system in order to expand its potential areas of application,’ Reiser of Fraunhofer IPA explained. Within the framework of the Care-O-bot 4 research platform, experts from around the world can further develop the software and hardware components. Together with Unity Robotics, Fraunhofer IPA experts are intending to adapt applications to the needs of different companies, making Care-O-bot 4 available as a commercial product.

Recent News

16 April 2019

Robot bin picking is being employed by a leading toy manufacturer based in Denmark to separate plastic building blocks during production

03 April 2019

Scientists at the University of the West of England are using face recognition to detect different emotional states in pigs in order to alert farmers to any health and welfare problems

22 January 2019

New software called ChimpFace is helping conservationists identify illegal trade in chimpanzees using face recognition

11 January 2019

The World Wildlife Fund and Flir Systems are supplying rangers across 10 parks and game reserves in Kenya with thermal imaging cameras in an effort to stop illegal rhino poaching