Virtual Observer

Virtual Observer is a research collaboration project between DTI Group Ltd and Curtin University of Technology.

Application Background

Video surveillance systems usually consist of a network of fixed or static cameras. This requires cameras to be established close to locations of interest across a wide area in advance of their anticipated need.

Accelerated by the recent terrorist events, transportation agencies have become acutely aware of the need to improve passenger security through the use of new technology, including mobile video surveillance systems. These systems can monitor and record onboard events, collect footage from inside and outside the bus, store operational data and generally improve the quality and safety of a bus, train or plane ride.

Further, through the implementation of forward and rear facing cameras, each transit vehicle ( bus/tram) becomes a patrolling video surveillance unit capturing off-bus street activities as these video-enabled buses roam major city areas, up to 24 hours a day, 7 days a week.

This roaming city-wide surveillance function is an invaluable extension to existing “static” CCTV programs for crime prevention, policing, intelligence collection, counter terrorism and to assist in the assessment of insurance claims and court actions.

The Technology

The Virtual Observer technology consists of an integrated network of video streams captured by mobile cameras ( sensors) such as those installed on public transit vehicles. It represents an imaginary (fixed position) camera that can be placed anywhere with in the space traversed by a mobile camera network. A key differentiator of this technology is the fact that the source video streams are mobile and are constantly moving throughout space.

Each Virtual Observer is established using a position, radius, orientation, and field of view. The system then constructs the view as seen from that observation point by indexing, organising, and transforming images collected from the mobile camera network. Where possible, the system builds composite images by combining observations taken at different times.

The technology provides a two- fold innovation: 1) it uses a mobile network of cameras to synthesise a fixed camera view ; and 2) it provides query based video streaming for a more efficient use high cost transmission and storage infrastructure.

The technology works by integrating different source camera streams to create a single stream at a specific spatial location. Video images are shown to the user based on a) a selected spatial position; b) a selected orientation of view; and c) a selected time range. Additionally, users can enlarge or reduce the area observed (the field of view) by changing the radius of the observation point.

Information recorded by a network of mobile cameras represents a large collection of sample images across a large spatial area and often at low temporal resolution and as such views of a particular fixed location (the Virtual Observer) are distributed and derived from many different video streams.

By recalling the information stored from a network of discrete mobile cameras, Virtual Observer is able to re-create a view from an imaginary 'fixed' camera located at any point within the network. Further, multiple Virtual Observers can be located within the same vicinity and used to construct multiple views of the desired scene.

The Innovation

Video surveillance implementation is a labour/resource intensive service. Large networks of fixed cameras are used to capture data in major cities worldwide. At the same time mass transit vehicles are equipped with devices to capture video streams as they roam the busy city streets. The result is a vast quantity of video that require organization, storage and integration with the existing static camera surveillance network. The present methodology and technology available to deal with transit camera video streams is inefficient because it is ineffective in presenting the captured information in a coherent manner.

Virtual Observer represents a breakthrough innovation by integrating vast numbers of video streams in an orderly manner, so that the requested or queried video data can be selected, integrated and presented. By integrating the video streams captured by different moving sensors and thus recreating a “Virtual Observer”, users can now efficiently synthesize the vast amount of video streams in order to recover and view the information effectively and sensibly.

Additionally, Virtual Observer uses a proprietary method to ‘microstich’ or ‘mosaic’ different streams that are geographically related in a timely manner. The result is a chronological sequence of views without interruption, essentially recreating a time-lapse sequence from a fixed perspective derived from multiple moving sources.