Track action in blink of eye? A snap for this NTU camera

From left: Assistant Professor Chen Shoushun, 40, with researchers Guo Menghan, 28, Li Wan Long, 26, and Yang Wen Lei, 25, and the prototype camera named Celex. The ultra-fast camera mimics the capability of the human eye and can track minute changes
From left: Assistant Professor Chen Shoushun, 40, with researchers Guo Menghan, 28, Li Wan Long, 26, and Yang Wen Lei, 25, and the prototype camera named Celex. The ultra-fast camera mimics the capability of the human eye and can track minute changes in the scene being viewed, even in low light.ST PHOTO: LAU FOOK KONG

The human eye is adept at spotting and tracking moving objects in its field of vision, but not a camera sensor, which captures what it "sees" in its entirety, like a photograph.

But scientists at the Nanyang Technological University (NTU) have developed an ultra-fast camera which mimics the capability of the human eye, and can track minute changes in the scene being viewed, even in low light.

The prototype camera named Celex, which has a patent pending, was showcased to the media last Thursday, and its developers say the technology can be useful for autonomous vehicles and self-flying drones.

Work started in 2009 and involved a team of 10 scientists, led by Assistant Professor Chen Shoushun from NTU's School of Electrical and Electronic Engineering.

Prof Chen said, for a conven- tional camera, many frames of pictures have to be continuously taken to track a moving object. This generates a huge amount of data, much of which is wasted, he added.

A camera sensor has millions of pixels, or sensor sites, which record light information and collectively form a picture.

HELPING TO PREVENT COLLISIONS

With its continuous tracking feature and instant analysis of a scene, it complements existing optical and laser cameras and can help self-driving vehicles and drones avoid unexpected collisions that usually happen within seconds.

ASSISTANT PROFESSOR CHEN SHOUSHUN, from NTU's School of Electrical and Electronic Engineering, on the Celex prototype camera.

With the Celex, individual pixels record only information relevant to movement, in other words, when there is a change of light intensity.

It does this at nanosecond intervals, much faster than conventional video. Where there is no movement, the pixels are inactive and do not record information. This allows the Celex camera to process and track what it "sees" much faster.

This also helps the camera's processors to differentiate between objects in the foreground and the background. The data output from the Celex is between 100 and 1,000 times less than that of a normal camera, said Prof Chen, 40.

"With its continuous tracking feature and instant analysis of a scene, it complements existing optical and laser cameras and can help self-driving vehicles and drones avoid unexpected collisions that usually happen within seconds," he added.

The project has received $500,000 in funding, from grants awarded by the Ministry of Education and the Singapore-MIT Alliance for Research and Technology.

A start-up named Hillhouse Tech was set up in July 2015 to bring the camera to the market.

It is now in talks with auto- makers and a self-driving car company to use its technology, said Prof Chen, who declined to reveal names.

A version of this article appeared in the print edition of The Straits Times on February 21, 2017, with the headline 'Track action in blink of eye? A snap for this NTU camera'. Print Edition | Subscribe