Overview
The strong political drive within Europe to encourage use of public transport will lead to increased passenger flows through metro stations and other critical elements of the public transport infrastructure. To meet this demand, public transport operators must improve the efficiency and security of their operations. The use of computer vision techniques will enhance the role of CCTV in meeting these objectives. The ADVISOR system builds on recent advances in computer technology applied to CCTV, academic research into computer vision and, in particular, progress made in recent European funded programs which address the needs of public transport operators. As they plan to upgrade their installations there is a growing problem of 'information overload' with more CCTV cameras installed that can be effectively monitored by the staff available).
Furthermore, facilities for recording CCTV data are either non-existent or very rudimentary. There is considerable scope for better exploitation of CCTV data to meet the growing demands on public transport operators for improvements in safety and efficiency. The ADVISOR project started from some work on previous EC projects, but was the first to integrate the results of advanced video processing algorithms with an archive, search and retrieval database and a human computer interface.
The goal of ADVISOR was to improve the management of public transport networks through better exploitation of data from CCTV cameras.
The principal objective of ADVISOR was to demonstrate the benefits of computer vision for reducing the workload on metro staff who rely on CCTV surveillance to assist them in the management of the transport network.
Two further objectives were to develop new management services for the operators, based on CCTV surveillance, and to demonstrate how a system such as ADVISOR should be specified and procured in the future.
The specific goals for the ADVISOR project were to demonstrate the following capabilities:
- Computer-assisted incident detection
- Post-incident analysis
- Quantitative analysis of behaviours
- Assessment and evaluation
- Market acceptability.
The results of previous work in this area will be used to identify the 'technology gaps' where current systems cannot meet user expectations and indicate how the latest technologies can be integrated to address these gaps. Functional specifications for ADVISOR will be developed, implemented and evaluated through two test-beds and one demonstrator to provide:
- Real-time computer analysis of multiple video sequences to detect, track and locate individuals in the 3D environment. It will be based on techniques for motion detection, calibration of cameras, scene modelling and object tracking. In addition, crowd-monitoring algorithms developed by CROMATICA will be integrated.
- Real time interpretation of the behaviour of individuals and of crowds to recognise anomalous or dangerous situations (such as overcrowding or vandalism) which would require a prompt response from metro staff to maintain smooth running of the transport network. This will be achieved by techniques for event detection and scenario recognition. Moreover, learning techniques will be developed to set thresholds for detecting anomalous situations and prioritising alarms to minimise operator workload.
- Tools for the creation, maintenance and searching of a digital video database, using appropriate image coding standards for storing compressed video with automatic annotation according to content, and to assist both the extraction of statistical information on passenger behaviour and also analysis of serious incidents after they have occurred.
- An ergonomic human computer interface for the display of annotated live video and diagnostics relating to analysis of activities and events that have been detected. The interface will also provide means for the operator to define alarm conditions, verify alarms that are generated and to access the video database. The implementation will use standard commercial hardware configured in an open and scalable architecture.
Funding
Results
The ADVISOR project has demonstrated the feasibility of using computer vision algorithms to detect unusual human behaviour and to use this to improve the effectiveness of existing security operators. It is particularly effective in a metro environment but could be adapted to other situations such as railway stations, airports and shopping malls.
ADVISOR project has developed new operator services: Image Capture, Behaviour Recognition, Archiving of all camera captured images sequences, Search and Retrieval, Motion Detection, Crowd Monitoring, People Tracking, Human Computer interface and Communications over an IP infrastructure.
ADVISOR project advanced the state-of-the-art in the following important ways:
- adaptation of the existing algorithms to work with compressed digital video inputs
- extension of the algorithms to work with input from multiple cameras
- development of the algorithms to make them more robust, including 3-D modelling
- exploitation of the latest video workstation technology to achieve real-time operation
- use of the algorithms for annotating video sequences for efficient storage, search and retrieval
- improvement of the algorithms by providing learning feed-back from an operator
- encapsulation of the algorithms in a software 'wrapper' to provide industry standard interfaces for ease of system integration.
The measurable benefits were: reduced operator workload, faster response to incidents, more efficient management and retrieval of video data and improved means for analysing public use of transport systems. Specification and procurement of systems will be facilitated by the open architecture adopted.
Policy implications
The work that has been performed represents the current state of the art in machine vision technology for behaviour recognition of individuals, groups and crowds. The ADVISOR system addresses a market segment where there is an obvious and growing need for products.
The next challenge will be to exploit that need by taking the results of ADVISOR and disseminating them to end users, and planning and developing products that can be derived from ADVISOR.
Further work that will need to be performed to address real world applications includes:
- Behaviour recognition needs to be broadened to include more possible behaviours and to include supervised learning algorithms controlled by the operators;
- An improved person tracker, perhaps based on the Reading people tracker will need to be integrated to allow the recognition of more types of behaviours;
- More use needs to be made of multiple cameras, and to achieve this, better quality images will be required;
- The capture of images needs to be capable of inserting a digital watermark to ensure that the images can be used in legal proceedings;
- The size of the equipment needed to implement a system must be reduced, so that the cost of installation and maintenance will be more acceptable;
- Tools must be developed to reduce the installation and maintenance costs of scene modelling and camera calibration;
- Improvements will need to be made to the archive, search and retrieval to cope with a larger scale deployment.
Rail
Key findings
The ADVISOR project is particularly effective in a metro environment but could be adapted to other situations such as railway stations, airports and shopping malls.
Intelligent Transport Systems
Key findings
ADVISOR project advanced the state-of-the-art in the following important ways:
- adaptation of the existing algorithms to work with compressed digital video inputs;
- extension of the algorithms to work with input from multiple cameras;
- development of the algorithms to make them more robust, including 3-D modelling;
- exploitation of the latest video workstation technology to achieve real-