Overview
Future scenarios do not simply involve an increase in air traffic; in addition new forms of co-operation and co-ordination are expected to emerge. New IT technologies and platforms afford more data availability to many of those who run the whole ATM system. As the human factors literature suggests, the availability of more data implies new forms of task distribution and thus new co-operative strategies have to be anticipated. This implies a paradigm change when designing new technological support.
A novel 4D approach, that we call D4 - to differentiate from the existing 4D ATM concept - creates new opportunities in the way the controller will visualize and interact with information: new ways of representing the information, while not necessarily decreasing workload, provide an opportunity to reduce existing knowledge gaps, supporting optimal decision making (as demonstrated in recent works).
The AD4 project focused its investigations and technological developments on building an innovative Virtual Air-Space representation for the ATM system, providing a range of valuable benefits to support efficient control systems where 3D real time interaction with air traffic/airport space is accessible to the controllers.
The AD4 projects aimed at exploring the application and benefits of 3D (three dimensional) displays and interaction technologies with a view to determining the qualities required to produce an effective 3D information visualisation environment for the air traffic controller. The implemented system, targeted at semi-immersive three-dimensional displays, made use of VR (Virtual Reality) techniques to provide an environment within which an air traffic controller can observe and monitor a large number of aircraft over a wide area, being kept aware of the many complex factors about their planned routes which may affect the future planning of the flight paths. Furthermore, they can use 3D interaction methods to select and re-route aircraft interactively as the data are updated in real time.
The targeted AD4 system was designed and implemented in collaboration with experts in field of ATM systems, VR developers, Human factors and the Italian Agency for Air Navigation Services - ENAV, producing a specific demonstrator that enclosed real interfaces and data coming from a traffic centre and simulation environments. Such a demonstrator was extensively evaluated by air traffic control personnel. AD4 infrastructure enabled the identification of benefits, in terms of enhanced understanding and clarity of perception, 3D displays and/or 3D representations in 2D displays, combined with enhanced information presentation, provided to the controllers in Approach and Tower sectors. It was hoped - highlighted in recent studies - that improvements in this area would permit more efficiency and safe management of more aircraft over a wider airspace.
The key objective of the AD4 project was the enhancement of a 3D Virtual Reality system, called D3 (D-cube), for the real time visual representation and manipulation of data in the field of Air Traffic Management and Control, both in open space (particularly in the approach phase) as well as at the airport level. The D3 technology is capable of managing real-time 3D visualisation and navigation by means of the adoption of an open distributed infrastructure able to handle dynamic and scalable data elaboration and integration using auto-stereoscopic displays and 3D mouse devices.
To provide an effective test-bed for 3D VR interaction and visualization in air traffic management the consortium developed a robust, distributed and real-time system. The system provides a controller with a 3D environment showing all of the aircraft active in the controller's particular regio
The AD4 project addressed its objectives through:
- the analysis of operational concepts and human factors;
- the engineering of the IT infrastructure and its core Components (4D HMIs, Middleware, Predictive and Applicative Components,
- Interfaces to external data e.g. Meteo and ATM system integration);
- the development the working Demonstrator for an operative context;
- the validation by the use of the MAEVA methodology and the assessment and exploitation of results.
AD4 was based on extensive reuse of a technology called D3 (D-Cube), developed by NEXT in a successful National project co-funded by the Italian Space Agency ASI. Such technology supports dynamic management and scalable data elaboration for DEM, Meteorology, Pressure and Wind fields, Radar tracks and Telemetry data using auto-stereoscopic displays and 3D mouse devices.
Furthermore the AD4 project integrated 3D technologies and ATM components, driven by models by the use of OMG Model Driven Architectures and making use of Component Middleware (CORBA CCM).
A series of experiments were conducted to assess and demonstrate the portfolio of results achieved by the AD4 project to controllers and technical specialists:
- validations were conducted to establish the 'fitness for purpose' of the AD4 operational concept;
- demonstrations were performed to demonstrate results/capabilities achieved in all the fields covered by the AD4 project;
- proofs of concepts were conducted to assess ideas for future developments.
Validations
Validations were performed to establish the 'fitness-for-purpose' of the AD4 Operational Concept. The AD4 Operational Concept consisted of deploying 4D (Space plus Time) displays to enhance the presentation of the spatial-temporal information necessary for the controller's job. Following MAEVA methodology, high level evaluation objectives were identified and mapped to a set of low level objectives with associated hypotheses, metrics and data collection methods. In particular, safety (mapped to both workload and situation awareness low level objectives), usability and acceptability were identified as high level objectives of the AD4 evaluation activities.
Real-time simulation, with the involvement of Air Traffic Controllers, were selected as the most appropriate validation technique; and ATC simulation platforms, like ESCAPE/ACE and ATRES, were chosen as real time ATC simulators that feed
Funding
Results
The AD4 project developed a 3D air situation display (3D radar picture) based on the representation of visual elements within a purely synthetic 3D Virtual Environment. Such a virtual environment provides a 3D perspective display of the ATC controlled sector.
The implementation of such a 3D Virtual Environment was based on a 3D framework, called D3 (D-cube) developed by NEXT in a national research project. Such framework was successfully enhanced and tailored to the ATC domain in the course of the AD4 project.
D3 is a 3D virtual reality system for real time visual representation and manipulation of heterogeneous geo-referenced data such as DEM (terrain), meteorological data (clouds, pressure and wind fields), telemetry data, GNSS (satellite), surveillance data (radar tracks) and flight plan data. D3 framework is capable of both 3D visualisation and 3D navigation. 3D visualisation resulted from the integration of heterogeneous data, structured into layers. Throughput and elaboration limitations were overcome by a scalable and distributed architecture. 3D navigation was allowed by the use of both specific (3D Mouse, 6 degrees of freedom device) and standard (classical mouse) input devices.
A test-bed to experiment with and validate the use of 3D/4D displays in the ATM domain was constructed by the integration with well-known and consolidated simulation platforms in the ATC domain, like Eurocontrol ESCAPE and Vitrociset ATRES. Such an integrated environment was successfully used to evaluate the most relevant 4D HMI concepts and representations by Real-Time Human-in-the-loop Simulations with the involvement of controllers and simulation of the operational ATC environment. Interoperability with external systems was a targeted objective of the projects, achieved by the supports of standard exchange formats (e.g. ASTERIX) and CORBA IDL interfaces for the ATC domain (e.g. AVENUE).
An AR (Augmented Reality) D4 technology and a demonstrator were developed. The purpose of the AR D4 demonstrator was to prove applicability of the AR visualisation technology in the development of an HMI for tower controllers, while taking into account some of the main 3D/4D visual concepts that derive from the needs of the Operational Scenarios identified in the AD4 project. Those 3D/4D concepts are partially revisited and utilized for the AR HMI, which thus appears to be strictly related to the D3-based Tower HMI.
The capabilities of the demonstrator were mainly focused on visualisation, ra
Technical Implications
The AD4 project demonstrated the possible exploitation of 3D VR (Virtual Reality) and AR (Augmented Reality) technologies in the ATC context to support innovative Human Machine Interfaces aimed at improving controllers local situation awareness in specific scenarios.
Furthermore, the project promoted the interoperability among heterogeneous components and systems, first of all ATC simulation and operational systems, through the adoption of standard exchange formats and rules and the definition of proper adaption components. This policy led to the development of an open platform for 3D representations and interactions in the ATC domain.
The AD4 project can be also considered as a real test case of the application of MDA concepts to the development of systems in the ATC context. The MDA Tool Chain developed in the framework of the project to support the development of the AD4 3D Radar Display is a tangible result that can be exploited in model based engineering processes in the aeronautics domain.
Policy implications
The Single European Sky ATM Research (SESAR) initiative identified a set of strategic objectives to be pursued for the implementation of the european ATM of the future. Results of the AD4 project (in terms of both concepts and technology) could contribute to the achievements of the following strategic objectives of SESAR:
- improved safety and security levels
- reduced operating costs
Improved safety can be achieved by the use of 3D displays for air traffic control in order to improve local situation awareness thanks to the visual representation of third and fourth (time) dimensions. Furthermore use of Augmented Reality techniques in the airport environment can alleviate most of the visibility problems experienced by tower controllers (low visibility due to bad weather conditions, occlusion, etc.)
Improved security (security at the IT level) could be achieved by the adoption of technologies of the same type of the ones investigate and developed in the course of the project (Secure CORBA Middleware and enforcement of secure policies by OpenPMF).
Reduced operating costs could be achieved by the use of 3D displays and augmented reality techniques (i.e. by the use of synthetic visual elements superimposed to video images) as building blocks for the implementation of virtual/remote towers concept.
An effective use of 3D technologies for the air trafic control has to take into consideration the integration of 3D displays with (existing) 2D air situation display by the provision of 2D-3D combined air traffic control displays. Several possibilities exist and some of them were investigated in the course of the project.
The need of such combined display arose from the fact that while 3D representations improve local situation awareness, pure 3D visualisation tends to be disruptive for controllers global situation awareness. The solution to this problem consists then into conserving a global 2D display while proper integrating in it appropriate 3D displays.