Methodology of Pilot Performance Measurements

The article is devoted to the development of the methodology of measuring pilot performance under real flight conditions. It provides the basic information on a research project realized to obtain new information regarding training and education of pilots. The introduction is focused on the analytical part of the project and the outputs in terms of the current state of the art. Detailed view is cast on the issue of measuring pilot performance under specific conditions of the cockpit or the flight simulator. The article is zooming in on the two selected and developed methods of pilot performance in terms of the defined indicators evaluated, conditions of compliance for conducting research and procedures of the methodology of pilot performance measurements.


Introduction
The stimuli as above have turned into basic drivers pulses for the company of Education training & consulting company, j.s.c. and the Faculty of Aeronautics TUKE, which in the period of 2012 up to 2014 were solving project titled as " Research of training methods of pilots using flight simulators", ITMS code of the project: 26220220161, which was co financed from EU resources.
The strategic aim of the project was research into increasing air transportation safety and the follow-up transfer of scientific findings into economic practice [1].
The basic outline of the project was: • Analyzing the state of the art in preparation and training of pilots, • Identifying insufficiencies in this area, • Suggesting new methods of pilot training, which could increase the level of safety while simultaneously reducing financial burden of these trainings, • Formulating of hypotheses at the level of confirming the identified insufficiencies and verification of the contribution of suggested new methods of pilot training for improving the current status • Developing the methodology of research activities • Verifying new procedures for the purpose of practical applications.

Methods used to analyze the state of the art
The analytical part of the project involved mapping of the state of the art in the field of applied procedures as part of the preparation and training of the aircrew with emphasis on the efficiency of using flight simulators and on their contribution to increasing air traffic safety.
Stage one of the analytical activities was focused on the legislative framework, which is a controlled process of aircrew training and preparation.Special attention was paid to training procedures for pilots and the requirements for the training on flight simulators.
The next stage of the analysis focused on the individual subjects of air transportation in both the domestic and international environment, in civil and military sectors.For this activity, of good use were information obtained directly from the individual subjects of air transportation by way of discussions on the occasion of personal visits at selected firms and institutions.Further sources of information were the websites of those companies, which could not been visited in person.
As a matter of fact, the key issue of the project is in the focus of both local and international businesses.That is why, it was necessary to identify and analyze similar research projects with the aim to find out the state of the art in the issue so as to establish a follow-up on the outputs of similar projects.
The most important element of air transportation safety is human factor, of course.Consequently, an analysis of the causes of those aviation accidents was performed, which resulted from training procedures of pilots.
An important part of the analysis was made up by a questionnaire-based survey regarding the state of the art of training procedures for pilots and use of flight simulator as part of pilot training.It meant contacting aviation companies and institutions, which were directly or indirectly involved in air transportation.The survey was also freely available for the wide professional public in via a web application.

Analyzing the state of the art
The analysis was focused on the preparation and training of pilots with the purpose of identifying the insufficiencies in this field and obtain stimuli for the research part of the project.The following points are bringing closer the basic areas of interest of the analysis.

Legislative framework
Within the framework of analyzing air law directly specifying the conditions and requirements for pilot training as well as flight simulator applications in the training process, focus remained on the legislative environment of aircrew preparation in the Slovak republic, i.e. on the environment, which forms part of the European environment and is regulated by the European Aviation Safety Agency, the EASA, applicable to the European environment.

Aviation subjects directly involved in the process of the pilot training and preparation in the domestic and international environments
This stage of the analysis was focused on firms and companies representing all the elements of the air transportation system in the context of pilot training and preparation: • Air carriers, holders of the AOC in commercial air transportation, • Flight training schools and organizations, FTO/ATO, • Air force units, • Flight simulator manufacturers, • Operators of flight simulators.
The areas of interest covered information regarding brief characterization of the subject, its history, process of obtaining certificates, training procedures applied for pilots and use of flight simulators during pilot training.

Identification and analysis of similar research projects These activities involved conducting analyses of the available information regarding:
• Research projects focused on the procedures of pilot training and preparation, • Factors affecting pilot performance i.e. air transportation safety, • Methods of measuring pilot performance, • Use of flight simulators in pilot training and the efficiency of their utilization, • Wide spectrum of the issue of pilot training and preparation.

Causes of aviation accidents resulting from pilot training procedures
The causes of aviation accidents were analyzed with the purpose of identifying those cases, which resulted from pilot training and preparation procedures so as to identify the insufficiencies in this area on the basis of real consequences.

Questionnaire-based opinion poll regarding the state of the art in the pilot training procedures and utilization of flight simulators as part of pilot training
As par to the survey, the information obtained was related to the following areas [2]: • Identification of pilot procedures involving the most frequent use of flight simulators, • Using/not-using flight simulators to verify correctness/efficiency of the established flight procedures, • Identification operational and training procedures in which the respondent is aware of insufficiencies, which could result in changing the training procedures in order to eliminate the deficiencies, • Effect of changing flight data presentation on pilot performance (e.g.changing from analogue to glass cockpit and vice versa).
Figure 1 is expressing the basic ideas of the survey respondents regarding identification of flight operational and training procedures in which they are aware of insufficiencies that could result in changing the training procedures so as to have them eliminated.The current development in the field of basic flight, navigation and engine data presentation on the cockpit dash-board is heading towards the gradual replacement of classical analogue indicators by grass cockpit indicators, which in some cases bring about fundamental changes in the way of presenting the information needed for the navigation of an aircraft.We assume that the issue of changing presentation is symptomized differently at various categories of pilots in view of the proficiency, number of hours flown etc.The most important influence on pilot performance resulting from changing the presentation is assumed for the following combinations: • Changing from analogue presentation to glass cockpit for pilots who have flown only by analogue instruments so far, • Changing from glass cockpit presentation to analogue form for pilots who have flown only by glass cockpit instruments, • Changing presentation for pilots, who have flown alternatively and are flying by both forms of presentation, whereas the changes occur following a longer period of time.
On the basis of the research outputs, two hypotheses were defined for the purpose of the research activities within this project in order to be instrumental in obtaining new knowledge in the course of the research in the field specified as above.

Hypothesis 1
Based on the findings of the analysis, we assume that the influence of changing presentation from analogue to glass cockpit is negative regarding the performance of those pilots who during their short flight career have flown only by analogue data presentation spending only a small number of hours in the air (around up to 100 hours flown), or are utter beginners familiarized only with the basics of flying on instruments.This category of pilots will form the Test sample of pilots No.1, called as Beginners.We assume that if the change in presentation is proceeded by a training on flight simulators, then it will less negatively affect pilot performance [2,3].

Hypothesis 2
Based on the analysis we assume that the influence of changing from analogue to glass cockpit or vice versa is more apparent on the performance of those pilots, who have spent a greater number of hours flying on one form of data presentation or the change was proceeded by a greater the other way of presentation.In view of the fact that aircraft are operated offering both forms of presentation, these pilots are currently confronted with changing presentation, some of them within shorter periods of time (e.g. even within a day ) and some for a longer section of time.This category of experienced pilots will make up the Test sample of pilots No.2.We assume that if the change in presentation is proceeded by a new method of training and preparation, it will have lower negative impact on the performance of pilots [3].

Failure of the artificial horizon
The analyses of simulator-based training procedures concluded that the failure of the artificial horizon does influence pilot performance in view of flying on stand-by instruments, which is apparent by deviations of the actual aircraft position from the required flight and navigation parameters.At current forms of presenting flight and navigation information, the following basic situations may develop: When flying on instruments with analogue form of presentation, failure of the artificial horizon and the necessity to perform piloting on the basis of stand-by flight instruments such as turn and bank indicator with a lateral bank indicator (by the " ball") and the vertical speed indicator, or using the stand-by artificial horizon not situated directly in front of the pilot (resulting in violating of the established procedures of scanning flight and navigation information) and usually this back-up horizon of smaller by design, i.e. does not provide so precise information as the main artificial horizon.This situation does affect piloting in terms of maintaining the required bank and pitch of the aircraft but does not seriously affect navigational procedures.
When flying on glass cockpit presentation, with the digital information failed, there arises the need to fly on stand-by flight instruments such as accelerometer, altimeter, stand-by artificial horizon and a simple magnetic ball compass, usually an analogue one.The stand-by artificial horizon is not located directly in front of the pilot (a fact resulting in violating the established procedures of scanning the flight and navigation instruments for information) and this back-up artificial horizon is smaller by design, so it does not provide as accurate information as the main artificial horizon.This situation does affect piloting in terms of maintaining the required bank and pitch thus influencing the navigational procedures, too.

Hypothesis 3
Based on the analysis findings we assume that the effect of main artificial horizon failure and the need to perform piloting on stand-by instruments is more apparent on the performance of those pilots, who have not undergone training on stand-by instruments for a long time using a flight simulator, or a real aircraft.This category of experienced pilots will make up the Test sample of pilots No.3.We assume that if flying on stand-by instrument is proceeded by absolving a new training and preparation, then flying by stand-by flight instruments will be of less negative influence in terms of pilot performance [3].

Methodology of measuring pilot performance under real flight conditions
Changes in the form of presentation of flight and navigational data in terms of pilot performance was assessed within the project research by way of measuring precision of piloting and the level of load.Definitions of the mentioned indicators of assessment are as follows: Precision of the piloting techniques in terms of the research requirements was defined as deviation of the actual flight parameters in the determined phases of flight from the required flight parameters.
Level of load for the purpose of research was defined as deviations of the selected psycho-physiological parameters of the pilot from the initial level of the parameters of the same person.
In view of the external condition, the research flights were performed under the following conditions: • Sufficient visibility of the natural horizon, • Performed in the area below or above the cloud base, • Performed in an area with maximum wind speed up to around 7 ms −1 , • Non-turbulent environment, • With sufficient clearance above obstacles and lateral separation form them so as to prevent the pilot form unnecessary monitoring safe altitude or lateral separation, • Performed under suitable light conditions in terms of flight data instruments readability.
Within the framework of the research, all phases of flying on aircraft and on flight simulators related to the research (piloting in the working zone) were performed as flying on instruments.

Measuring pilot performance by way of defining
the deviations of parameters form the required parameters in the appropriate phases of flight The individual true deviations were defined with a time separation of 1 second for around 1 minute of the given modes of flight.For each mode, 3 measurements were made [4,5].
For the purpose of recording the true parameters of flying, the research made use of the following procedures:      Within the framework of the project we worked with the hypothesis by which we assumed that from a certain level of pilot's load, in compliance with the stated definition, this load is negatively influencing pilot's performance, e.g. also from the point of precision of piloting techniques.It means that by measuring pilot's load we are able to quantify pilot performance (for particular results, see [8,9]).For this purpose, the research project made use of measurement equipment of the Faculty of Biomedicine Engineering, CTU in Prague [10,11].
The basic parameters and the setup of the measurement instruments: • Telemetric sensing unit

Figure 1 .
Figure 1.Areas of training to be paid increased level of attention.

Figure 2
Figure 2 is expressing the basic ideas of respondents regarding the changes in flight data presentation affecting pilot performance when changing from analogue to glass cockpit form of presentation.

Figure 2 .
Figure 2. The issue of changing presentation from analogue to glass cockpit.

4 .
Outputs from the analytical part of the project 4.1 Changes in the presentation of basic flight, navigation and engine data The analyses of simulator-based pilot training procedures concluded that the issue of changing the form of basic flight, navigation and engine data presentation on the instrument panel in the cockpit does affect pilot performance in terms of piloting, with symptoms of deviations from the actual position of the aircraft and the actual flight trajectory from the required flight and navigation parameters.

•
Recording flight parameters on the TRD40 flight simulator by means of a SW Instructor station, function Performance, • Recording of flight parameters when flying on real aircraft using the flight data recorder supplied by TRADIP, • Recording flight parameters (maximal deviations from the required parameters) by the pilot-instructor.Deviations of the actual parameter of flights from those required were monitored during: • Straight-and-level-flight for an altitude and heading(with the pilot instructed to fly at a required altitude and heading), • Horizontal turn for an altitude, bank and heading (with the pilot instructed to fly at a required altitude, bank and radios of turn, or the final heading of the turn), • Climb or descent turns, for vertical speed, bank and heading (with the pilot instructed to maintain a required bank, vertical speed and radius of turn, or the final heading of the turn).

Figure 3
Figure 3 illustrates the deviation of the true vertical speed from the required vertical speed +300 f t/min.The performance of 10 pilots under Test sample of pilots No. 1, called as Beginners in changing presentation from

Figure 4 .
Figure 4.The change of pilot performance -Beginners -Real flying under the present training procedures.

Figure 5 .
Figure 5.The change of pilot performance -Beginners -Real flying under the new training procedures.

5. 2
Measuring pilot performance by way of recording his psycho-physiological parametersFor the purpose of quantifying the level of pilot load the following parameters were recorded:• Pulse frequency, • Breathing frequency, • Body temperature, • Body activity (3D actogram),• Intensity of muscular activity.

-
in on-line mode, wireless communication via a wireless XBee interface in off-line mode the possibility of recording the data on the SD card • Sensor module for sensing pulse frequency • Sensor module for sensing body temperature • Sensor module for sensing body activity (3D actogram) • Sensor module for sensing breathing frequency • Sensor module for sensing intensity of muscle activities • Intensity of muscular activity.6.Conclusion Use of simulators enables training of border-line situations, which cannot be performed under real training flight.Applying new training methods potentially reduces financial burden of training without degrading the level of air traffic safety.Furthermore, the new training methods may contribute to raising the level of air traffic.Shifting the main share of training from real aircraft to modern flight simulators brings about direct financial savings in terms of the costs of fuel and wear of aviation equipment, and the undisputable positive contribution for the living environment as well.