Finding somewhere to live can be a time-consuming task, particularly when you spend time going around properties that turn out to be completely unsuitable. For people who plan to move to a new area for a few months for work, the task of viewing somewhere to rent can be almost impossible.
Germany-based corporate housing agency e-rent has a solution. The company uses a portable machine-vision system to film apartments that are available for rental, so potential tenants can view them online. The system generates spherical video using the Ladybug3 camera from Canada-based Point Grey, which mimics the way that people might look up at the ceiling, down at the floor or behind them when looking at a property. This combines the benefits of more traditional – although still not widely used in such applications – video and panoramic photography.
To film properties, the Ladbybug3 camera is housed on a small, remote-controlled vehicle developed by e-rent, along with a computer for recording the video and another for controlling the movements of the vehicle. Having all the computing on the vehicle and wirelessly controlled means that there are no cables to obstruct the resulting video.
The e-rent solution is not an isolated example. There are many innovative applications that call for capabilities that are not available from off-the-shelf consumer products.
For example, Mativision has used the Ladybug camera to record concerts from different angles. This application involves 10 cameras in various places around a concert venue and allows viewers to select to watch the event from different vantage points.
Stereoscopic vision using systems such as the Interactive Communication Unit from Sony Image Sensing Solutions and Atracsys have many possibilities for marketing. Credit: Sony
Another application has been developed – and is in the early stages of commercialisation – by David Hayden, a PhD student from MIT. Hayden has limited vision and was frustrated by the bulkiness and poor performance of existing note-taking technology for students and others with visual impairments. His solution, Note-Taker, combines optical and digital zoom with high-definition video to help people view classroom and business presentations, and take their own notes directly on their laptops. The system, which includes hardware for streaming live video and software in which users take notes while simultaneously viewing a distant presentation, uses Point Grey’s Firefly MV 1.3-megapixel USB 2.0 camera.
Sports stars
Another way that industrial machine vision tools have found their way into more consumer-facing applications is through sport.
Teledyne Dalsa’s machine vision systems are mostly used for non-contact industry inspection and biomedical imaging applications. However, the company has now taken its industrial technology into sports applications. A recent example is as a visual aid in American football, enabling reconstructions from different angles.
‘We have distributors throughout the world and in machine vision we get all types of requests. Football is played over a very large area so this forces a need for high resolution. We got into this because we have systems that can cope with it,’ explains Yvon Bouchard, director for OEM applications engineering at Asia Pacific of Teledyne Dalsa. ‘This technology would not have been possible 10 years ago. We didn’t have the mathematical horsepower on PCs.’
Stereoscopy has potential applications in gaming. Credit: Sony
He says that this is a new direction for the company. ‘A benefit is that this gives us exposure that we wouldn’t have had before. In addition, these industries have different cycles so it helps us to average out our production load. A lot of what we do uses our standard product so the cost is not too high,’ he adds.
He also notes that working with this type of application has marketing benefits for machine vision companies too because customers doing consumer applications are generally very happy to talk about innovative things that they are doing. In contrast, many industrial contracts are subject to confidentiality agreements.
However, there are some new challenges. One of these with the football application is the need to cover large distances. ‘Most industrial systems require a range of about 15 metres but the football application requires hundreds and hundreds of metres. We have to convert the data to go over optical fibre,’ says Bouchard.
Another challenge is that what looks good to a computer is not necessarily what looks good to the eye. ‘For good broadcast video we would need to process with a non-linear transformation, but that is not something we would do with machine vision in industry. For most of our clients what is most important is linearity. In this industry it is about broadcastability,’ he explains.
E-rent uses the Ladybug3 camera, from Canada-based Point Grey. Credit: e-rent.de
In addition, this type of application requires some additional capabilities. Unlike industrial inspection, football games are played outside and light levels can vary hugely during the course of a game. Bouchard notes: ‘The clients wanted to have the capability to adjust camera exposure automatically. 3D reconstructions need to have uniform light levels.’
Allied Vision Technologies (AVT) is also bringing its expertise into the sports arena. One application is with the US sports broadcasting technology company Sportvision, which has introduced FIELDf/x, a service that allows objective field data to be collected and analysed in baseball games.
FIELDf/x uses Sportvision’s baseball software technology to track and record digitally the position of players, pitches, hit balls and throws in near real-time. This is used to derive quantitative defensive statistics, including the difficulty of a catch, the probability of a particular fielder making that catch, and a comparison of the fielders. FIELDf/x uses four Prosilica GX cameras by AVT that are mounted on the stadium structure in weatherproof enclosures.
Similarly, Allied Vision Technologies’ Prosilica GE680 fast CCD camera is being used by Mobics, a spin-off company of the National and Kapodistrian University of Athens, Greece, to monitor and keep score in beach racket, a popular game in Greece.
This application presented some challenges for machine vision because temperatures can get very high on Greek beaches and cameras will be exposed to direct sunlight and the risk of damage from sand and dust.
‘We actually went far beyond AVT’s specifications for the camera, with operating temperatures above 50°C and a camera exposed to dust without any protective enclosure. The AVT Prosilica GE680 proved to be an extremely reliable camera for demanding applications,’ says Vassileios Tsetsos, chief technology officer of Mobics.
Stereoscopic vision
Stereoscopic vision is of particular interest to Sony Image Sensing Solutions and it is in this area that the company is partnering with Atracsys of Switzerland. The result is the Interactive Communication Unit (ICU), which includes two high-specification Sony cameras that track a user’s precise three-dimensional movements and even their emotions.
According to the two companies, the subtle characteristics that ICU is able to detect makes the vision system ideal for applications such as marketing. The system can determine a user’s age, gender and moods such as happiness, anger, sadness and surprise. This is done by tracking, for example, the upturn of the mouth or the widening of the eyes, and using two cameras allow these to be tracked in three dimensions.
Sporting applications include a requirement to cover long distances. Credit: Teledyne Dalsa
In addition, a stereoscopy vision system makes it possible to determine precisely where a person is pointing or looking. This could be used, for example, to bring up relevant reviews or additional information if somebody is looking at a product in a shop window.
It has many potential applications in areas such as gaming too. In addition, interfaces based on vision systems could play a role in the medical world. They could enable surgeons and others to interact with computing systems without touching the mouse or keyboard, which helps them to maintain sterile conditions.
Back to industry
And there are potential opportunities of such developments back in the industrial sector too. For example, people could operate robot arms remotely simply by moving their own arms.
Bouchard of Teledyne Dalsa also notes how ideas from consumer-facing applications can be fed back into industry. For example, he noted that the approaches used to record football games from many angles could be used for testing for accidents, enabling incidents to be reviewed from many different angles. He said that the company has customers who do this type of imaging of assembly lines to see, for example, what generated a fault. ‘That’s an example of a situation of where you don’t know at the time what you need to look for,’ he explains. He notes many other application areas for machine vision systems too – from photographing people on a roller coaster at a theme park to monitoring how attentive a car driver is.
‘These applications require huge amounts of computing power and storage but technology changes so fast. What you thought was impossible once may not seem so outrageous anymore. We have to re-evaluate everything as things improve,’ he says.
‘In this type of business you could say one week that something will never happen but by the next week that will have changed. The only real limit is the imagination of the people putting it together.
FreeD sports replay system using Teledyne Dalsa cameras: http://replay-technologies.com/portfolio
E-rent system using Point Grey cameras: www.e-rent.de/en/spherical-videos-apartments.htm
Atracsys marketing using Sony cameras: www.youtube.com/watch?v=URDPJ8LkotQ
Atracsys system used by Tissot luxury watchmaker: www.youtube.com/watch?v=mQ637Ry3PtQ
