DE10016765A1 - Display system for generation of hybrid two-dimensional or three-dimensional representation of conditions within airport environment has data server that provides input to interpretation model - Google Patents

Abstract

The system has a data server (1) that provides input to an interpretation model (2) and this together with a rules memory (3) allows a scene module (4) to be accessed. Together with a three-dimensional model data base (5), the system provides a two-dimensional/three-dimensional representation of the process, e.g. an airport operating system.

Description

The invention relates to a data processing system for generating hybri the 2D / 3D representations of process states that are on an airport apron stand, as well as a method for generating such representations.

The invention can be used in the context of electronic airport information systems Use for example for resource allocation or for resource planning.

A well-known electronic airport information system is, for example, the "Universal Flight Information System - UFIS" from ABB Airport Technologies, which supports airport operations with automatic resource allocation and 2D process visualization. This includes processes on the apron, such as

• - assignment and allocation of gates and parking positions by aircraft,
• - deployments of personnel groups (loading groups, cleaning staff, etc.),
• - Use of vehicles and other ground equipment (tank trucks, pushback, Stairs, etc.).

With this system, both planned assignments and the current one Condition textually and in two-dimensional graphics (e.g. tables and Gantt charts).  

Three-dimensional visualization of the apron including the aircraft and the floor unit is already used in IT systems in the airport area. Everything However, these visualizations serve to simulate vision and movement neither in architecture, space and traffic planning nor when training To advertising personnel (e.g. the Total Airspace and Airport Modeller from Preston Inc.). It is important to have the most realistic spatial and visual impression of the scene. A visualization of process states and there is no additional detailed information about the 3D objects.

Programming, for example, are general tools for 3D visualization Libraries like OpenGL or WorldToolKit are available, with the help of which one three-dimensional scene description including object geometry, colors, transparency border and textures in three-dimensional screen representations.

Disadvantages of known visualization methods and systems to support the Planning, monitoring and control of processes on the apron stand in that they are very abstract and also knowledge about the spatial Aus require the expansion of all individual components. Hence these representations use by less experienced personnel with restrictions. Also it is often difficult for the experienced user in stressful situations, with known systems to maintain the necessary overview in order to be able to react quickly.

The invention is therefore based on the object of a system and a method for Generation of hybrid 2D / 3D representations of process states by their application is the solution of planning, monitoring and control were better supported.

This task is made more hybrid by a data processing system 2D / 3D representations of process states that best on an airport apron hen, solved, which has the features specified in claim 1. Besides, will the task by a method with the features specified in claim 2 ge solves.  

The invention essentially proposes using virtual reality (VR) technologies to create an intuitive user interface which mixes three-dimensional information with two-dimensional information in order to support the execution of the following tasks in the context of the operation of an airport:

• - resource management,
• - Gate and position planning / allocation,
• - airport traffic planning,
• - work process validation, and
• - User training.

For this purpose, the user interface is connected to the flight and planning data and shows the information stored there in real time. The display method is depending on the application and is described below.

Three-dimensional representations show the surgeon the geometric ge following virtual world, which includes both the airport with all run and Representing taxiways as well as all buildings. Flight movements and vehicle movement are provided by virtual vehicles that measure the scale following the real vehicles, visualized.

The individual aforementioned tasks of airport operations and the advantages of Application of the invention to accomplish these tasks are as follows purifies.

Resource management

In this scenario, surgeons (operators) control and plan, so-called "Dispatcher" the handling of aircraft during their stay at the airport fen. Through the additional use of a three-dimensional VR user interface the dispatcher can get a spatial impression of the positioning of the individual vehicles at the respective position. Through different Representation methods of three-dimensional objects such as shading, color etc. conflicts can be recognized and prevented more quickly. More information like Processing status, resource utilization and other already planned orders  can be visualized. There is also the option of planning within the make and change three-dimensional user interface. To do this both two-dimensional Gantt charts and tables as well as three-dimensional ob made available between which the user can choose freely.

Gate and position planning / allocation

For this task, the surgeon makes an allocation and planning of the position (gates) on the aircraft. By the Be Using the VR user interface, the surgeon is able to control the spatial Dimensions of the aircraft must be taken into account when planning and can therefore plan more efficiently. He can also identify the intersection of wings like temporal overlaps between two aircraft at one position. Flight data are both color-coded and represented by different shading levels posed. Conflicts can easily be identified from this.

Airport traffic planning

In airport traffic planning, all vehicle movements are both from Airplanes as well as other vehicles are monitored, and specifications about the routes to be used at the respective time. The VR user super The area represents the entire airport in three dimensions, with a Any observer location can be chosen by the surgeon. All vehicles are represented with their respective movement, and also with their respective Departure and destination, connected by graphic objects. Both additional information information about the type and type of vehicles, as well as information relevant to flight data ions can be displayed as graphic objects and can be selected move wisely with the objects. Time-lapse functions also allow Si flow the simulation results into the representation.

Work process validation

Work processes are mostly predefined processes, work on machines or describe technical device. To ensure an optimal process, offers the VR user interface the ability to virtually complete these workflows run to reveal vulnerabilities. This property is achieved by the enables additional gain of the spatial dimension.  

User training

With conventional user interfaces, one operator gets the planning and Surveillance tasks done, no visual feedback as it is based on Tabel len and charts that his planning actions do not succeed with real actions gen can connect. Through the VR user interface he gets for all actions a visual feedback corresponding to virtual reality, wel ches he can intuitively - since it corresponds to the actual reality. There in addition, the spatial dimensions are available, he is able to lengths and estimate times.

A further description of the invention and its advantages follows below hand of an embodiment shown in drawing figures.

Show it:

Fig. 1 components of a visualization system, and

Fig. 2 is a table as a two-dimensional representation.

Fig. 1 shows the components of a data processing system according to the invention that it enables an online visualization of process conditions at an airport apron, wherein three-dimensional information to two-dimensional information ge mixed, in order to support the following tasks better during operation of an airport:

• - assignment of gate and parking position,
• - planning assignments for staff groups,
• - Planning the use of floor equipment.

Components of the system are a process data server 1 , an interpretation module 2 , a control memory 3 , a scene module 4 , a 3D model database 5 , a renderer 7 , and a configuration data memory 8 . The system generates a scene graph 6 and from it 2D / 3D representations 9 .

The components and operation of the system are explained below, distinguishing between a phase of initialization and an operating phase becomes.

In the initialization phase, the process data server 1 delivers the current state of the process, which is interpreted by the interpretation module 2 in a graphical representation. For this purpose, interpretation rules stored in the rule memory 3 are used which, for example, specify the color in which an object of a certain type is to be represented in the given state. In the initialization phase, the scene graph 6 is generated by the scene module 4 with the aid of information that is stored in the 3D model database 5 . The renderer 7 implements the description in the scene graph 6 by accessing configuration data stored in the configuration data memory 8 and correspondingly to this configuration data in representations 9 for display on a screen or on its output medium.

In the operating phase, the scene module 4 adjusts the scene graph 6 in accordance with the respectively changed process data, so that the desired graphic representation of the process in the scene graph 6 in each case by its structure, which can change in the operating phase, and the node content as a three-dimensional scene is described.

The system components are described further below.

The process data server 1 supplies the downstream system components with the current information about the processes on the apron. All automatic and manual entries can serve as data sources. The most important process data are:

• - flight data (flight schedule and current status of a flight),
• - The need for resources derived from the flight data (gate, parking position, per staff, equipment),
• - deployment plan of resources, and
• - Current status of the operations or the allocation of resources.

This data is dynamic. After the initialization phase of the system, each Event, d. H. Changes to the data are recorded and sent to the downstream system components forwarded.

Process events are (discrete) changes to the process data such as: the aircraft xyz has arrived; there is a conflict with the gate assignment; the planned loading group is late; a new loading group is assigned to the flight. These events are therefore changes in the states of the process or the objects, which are now also to be represented graphically. The interpretation of the process events therefore means the implementation of changes in state in graphic representations. The geometry of the objects, their display attributes (e.g. color, transparency) or two- or three-dimensional symbols can be graphic representations. The conversion of events into graphical representations takes place according to a set of predefined rules stored in the rule memory 3 .

To the saved rules for event interpretation

In the three-dimensional visualization of the airport apron, the geometry of the objects serves to illustrate the spatial and logical allocation of resources, not their exact positions. Graphic attributes such as color and transparency are representations of the states of the process and of objects. The assignment between these attributes and the states is essential. The rules of use of the airport or operating companies apply in some cases. A typical example of rules when using colors are:

• - red = conflicts,
• - Green = arrival / assignment confirmed,
• - Yellow = arrival / deployment expected but not yet confirmed.

Furthermore, the type of representation of the 3D objects (line representation, illuminated surface representation, transparent representation, etc.) can be used to differentiate between need and current use:

• - Transparent = there is a need,
• - Solid = resources in use at the position / location shown.

The use of a rule base in the form of the rules stored in the memory 3 instead of direct coding has the advantage that the rules can be easily adapted.

To module 4 for the generation and manipulation of scenes

For a three-dimensional representation of configuration data, a description of the scene is required, which contains both the physical objects such as the airport buildings, aircraft and equipment, and the symbolic representations, such as virtual text panels. This scene is described by the scene graph 6 . The module 4 for scene generation and scene manipulation has the task of generating the scene graph 6 during the initialization of the system and then updating it accordingly to the state of the process.

The structure of the scene graph is created during initialization. The geometry and representation attributes of the objects in the scene are generated from the basic information about the different types of objects, which is contained in the 3D model database 5 . For each process event, it is then checked whether the graphic representation needs to be changed. If this is the case, the corresponding entries in the scene graph 6 are adapted. This can be both a modification of the structure of the scene graph 6 and changes in the attributes of individual objects (nodes of the scene graph).

3D model database 5

Since the application to be supported is not based on a realistic visual simulation tion arrives, but much more on the representation of the temporal-spatial combination and conditions, the approach of the function-realistic representation of the Airport objects used. That means: it will be typical for the users of the Systems easily recognizable geometric models for any type of objects used. These basic models are stored in the 3D model database and managed. In addition to the geometry and the standard display attributes, who the identifiers, e.g. B. Logos of airlines and service companies, saved as textures.  

Above all, planes, vehicles, groups of people are functional objects Building parts and similar to take into account, also the model of the respective Airport apron including runways and buildings.

Scene graph 6

The scene graph 6 is a structuring and organization of the data describing the scene. It is typically hierarchical and contains different types of nodes. More about the general structure of a scene graph can be found in J. Werneke: "The Inventor Mentor, Programming Object-Oriented 3 D Graphics with OpenInventor", Addison-Wesley, 1994.

When displaying dynamic scenes, the nodes of the scene graph are grouped into:

• - Dynamic objects = objects that move or whose state changes changes
• - Static environmental objects = other objects,
• - Representation elements = light sources, cameras, etc., which are used to generate the dar serve positions.

This grouping enables efficient manipulation of the scene graph.

Renderer 7

The renderer 7 is the system component which converts the scene graph 6 to the representations 9 . This includes the 3D representation of the objects as well as the organization of the display area and display elements. For the latter, configuration data stored in the configuration data memory 8 are used. The mode of operation of a renderer is known and therefore does not require any further explanation. Configuration data could be, for example: show an overview of the entire airport and a close-up view of gate 10 . Menus and information windows are created in the same way.

Standard tools, e.g. B. 3D Pro programming libraries such as OpenGL or WorldToolKit can be used.

Configuration data for representations

The scene can be divided into multiple windows using as needed projection parameters. Also the projection pair meters of individual windows change. Furthermore, 2D-Ele elements such as menus and information windows for the objects.

Information on how all of these display elements are displayed on the display surface typical, the screen to be placed and parameterized summarized as configuration data. This data can be predefined from egg saved from a previous session, but also from the interactive one User input.

Representations 9

The representations are the result of the system described. This includes, as described above, both the 3D representations of the scenes as well additional 2D elements. Together they offer the user of the system the over view of the temporal and spatial state of the process at the airport apron, and enable better planning, monitoring and control of the pro zesses.

The functionality of the system will be further illustrated using an example the:

The information from an underlying flight plan results: flight LH999, planned arrival 10:00. For this purpose, the need for resources is determined taking into account all other flights, and the deployment (automatic or interactive) is planned in advance. The processes or processes on the apron are monitored and corrected if necessary. In the table shown as FIG. 2, the events, the representations and the associated scene graph manipulations are compiled.

Claims (2)

1. Data processing system for generating hybrid 2D / 3D representations ( 9 ) of process states that exist on an airport apron, wherein

• a) a process data server ( 1 ) is arranged, from which process data can be called up and which is set up to report process events,
• b) an interpretation module ( 2 ) is available and set up for this,

• - Retrieve process data representing the given process state from the process data server ( 1 ) in an initialization phase, and use interpretation rules that are stored in a rule memory ( 3 ) to convert the process data into a graphic representation and to supply a scene module ( 4 ) ,
• - to retrieve changed process data from the process data server ( 1 ) in an operating phase due to a reported process event and to convert it into changed or new graphic representations and to feed it to the scene module ( 4 ),
  • a) the scene module ( 4 ) is set up to generate a scene graph ( 6 ) in the initialization phase and to update it in the operating phase while accessing information in a 3D model database ( 5 ), and to supply it to a renderer ( 7 ), and
  • b) the renderer ( 7 ) is set up to convert the scene graph ( 6 ) into hybrid 2D / 3D representations ( 9 ).

2. Method for generating hybrid 2D / 3D representations ( 9 ) of process states that exist on an airport apron, whereby

• a) a process data server ( 1 ) is used, from which process data can be called up and which is set up to report process events,
• b) by means of an interpretation module ( 2 )

• - be in an initialization phase, the given process state representing process data from the process data server (1) retrieve and Ver application of interpretation rules stored in a rule storage (3), the process data is converted into a graphical representation and a scene module (4 ) are fed
• - In an operating phase, changed process data are called up from the process data server ( 1 ) on the basis of a reported process event and converted into changed or new graphic representations and fed to the scene module ( 4 ),
  • a) by means of the scene module ( 4 ) and with access to information in a 3D model database ( 5 ), a scene graph ( 6 ) is generated in the initialization phase and updated in the operating phase, and is supplied to a renderer ( 7 ), and
  • b) the renderer ( 7 ) converts the scene graph ( 6 ) into hybrid 2D / 3D representations ( 9 ).