Analysis of Quality Indicators in ADS-B Messages

There is lack of available statistical data in the field of quality indicators in ADS-B messages, particularly in Czech airspace. Quality indicators in the ADS-B message define if this message can be used by ATM applications. This article is devoted to the analysis of quality indicators in the ADS-B messages. The data collected in the Czech airspace during six months were analysed. These data were statistically evaluated. The main objective is the statistical evaluation of the data and comparing them with quality requirements in Europe. The article deals with results of the statistics and assessment of the level of quality. Results are satisfactory and similar to the results published by EUROCONTROL and FAA. According to the examined data, 86.42% of them meets the EASA requirements in RAD environment. As ADS-B is not mandatory yet, it is possible to state that these results are satisfying.


Introduction
ADS-B 1090 ES system went through certain developments since its first specification.That evolution had an impact on transmitted messages and its structure.Beside to defining a completely new types of reports, the changes have marked the already defined messages.The most significant changes are related to the determination of the accuracy and quality of data obtained from satellite navigation systems, namely data used for positioning.Various ADS-B applications require for their operation a certain quality and these quality indicators went through evolution.Therefore, it is quite difficult to distinguish and identify data accuracy because the precision indicators vary for different types of certification.Interpretation of quality indicators varies by ICAO version 0, 1 and 2.
Implementing Regulation (EU) No 1028/2014 amended the Original implementation schedule set out in Regulation (EU) No 1207/2011.The European ADS-B Implementing regulation mandates that new aircraft with a maximum certified take-off mass exceeding 5 700 kg or having a maximum cruising true airspeed capability greater than 250 knots must be equipped with ADS-B 'Out' after 8 June 2016, and retrofit for existing aircraft is mandated from 7 June 2020 on.ICAO version 2 is required in Europe [1,2].
The changes in schedule were primary motivated by delays in certification and in availability of required equipment, as well as industrial capacity constraints for equipping aircraft [2].Current European mandate is harmonized with mandate published by United States Federal Aviation Administration (FAA) that should ensure smoother implementation.
From the viewpoint of current usage of ADS-B applications there are inevitable to know actual level of penetration particular ICAO ADS-B evolution versions and the level of quality nowadays provided information.Availability of different level of quality position information have indirect impact on implementation of new navigation applications and into its safety aspect as well [3].

Statistical Evaluation of the Data
ADS-B receivers of Department of Air Transport were used for data collection.There were three receivers available to collect data from aircraft equipped with mode S transponder.The data was received via a software tool.Receiving was continuously from 09.01.2015 to 24.02.2016, but there were some unexpected failures during this period.These failures are not considered relevant in terms of credibility and accuracy of the results, because they represent less than 5% of the total time of receiving.The total number of messages, coming within the area bounded by coordinates: lower limit 48.54 • N, the upper limit 51.06 • N, 18.86 • E eastern border and western border 12.1 • E, is 308 564 474.These coordinates form a rectangle, in which the Czech Republic is situated.The number of messages received and processed is shown in table 1.

Statistics Of ICAO Version 0
One parameter represents the requirements for the level of accuracy and integrity of data in the report.NUCp for the position and NUCR for the velocity.As can be seen from table .3, NUCp parameter is dependent on other parameters.They are: HPL (Horizontal Protection Limit) and µ/ν (95% containment radius).HPL is the radius of the circle in the horizontal plane (plane equal to the WGS-84 ellipsoid) with centre in true position of the aircraft.It describes a radius in which the indicated position is with probability of integrity.Fraction ν/µ defines the accuracy as a location area in which is 95% of the indicated positions [4].
Figure 1 shows the percentages for each NUCp value for the ICAO version 0. The chart includes all data, regardless of the type of the message.There is a predominance of NUCp 7, the resulting percentile is 62.41%.The meanings of the values can be found in table 3. On the graph (see Fig. 1) we can see percentile 10.44% which represents NUCp value of 0, which represents a useless statement of position, so it shows the number of useless messages.

Statistics Of ICAO Version 1
In this version, the parameters for accuracy and integrity are split.Defined new parameters are: NAC (NACP, NACV), NIC, SIL.In version 1, there is a dependence on VPL (vertical protection limit).This applies to the parameters NIC and SIL.If VPL cannot be determined, then the SIL must be set to 0. A similar dependence was declared for the parameter NIC.If it is not possible to determine the VPL, then we cannot take values greater than 8, although the available data could report more accurate indicator in a horizontal plane [5,8].
From the type of ADS-B message and from the NIC supplement code parameter NIC was decoded.It defines the radius of the occurrence of the integrity R c [5]. Figure 2 summarizes percentile of each NIC.Meaning of NIC values can be found in table 4. In the figure 3, there can be seen predominance of the SIL 2. This represents 95.57% of all SIL parameters.SIL value determines the probability of exceeding the R c (R c is defined by NIC) without detection.Accuracy is in the ICAO certification version 1 is defined by the parameter NACp, which can be found in the Aircraft Operational Status messages.Figure 4 shows the percentage of individual parameters.Superiority of the parameter 9 can be seen there.NACp 9 is shown in 59.52% of the messages.Meaning of NACp values can be found in table.5.

Statistics of ICAO Version 2
In version 2, there are the following quality parameters: NAC (NACp, NACv) to define accuracy, NIC and SIL with SIL-SUPP to define integrity, SDA determining the likelihood of system failure, NICBARO to determine the quality of altitude  In the ICAO certification version 2, the dependence of NIC on the vertical component was removed.From the type of ADS-B message and from the NIC Supplement A has been decoded the value of parameter NIC.NIC values are described in figure 5 and table 6.
Figure 6 shows the percentage of each SIL value.In the graph we can see superiority of the SIL value 3, namely 95.60% representation.This represents the highest level of integrity.This is the probability of exceeding the radius Rc (defined by parameter NIC) greater or equal to 1 × 10 −7 per sample or per hour.
If the probability is related to sample or hour, there is defined by SILSUPP parameter.In the sample of data, 99.96% of them are per hour.Rest is per sample.
NACp as the main indicator of the accuracy of the ICAO certification version 2 lost dependence on the vertical component.This allows more accurate assessment of the accuracy only in the horizontal plane [8].You can see statistics of NACp on the figure7.NACp values are described in table 7.

Quality Parameters Compared to Quality Requirements of ATM Applications
There is ADS-B working in Australia, Canada, East Asia and in some parts of Europe.The requirements for quality indicators are defined by ICAO in ICAO Circular 326.The European requirements are specified in the document EURO-CAE ED-161 for areas with radar coverage and in document EUROCAE ED-126 for areas without radar coverage.Thus, there are different requirements for areas with no radar coverage and radar coverage [10,11].

Non Radar Areas
For areas without radar coverage requirements are as follows.
For accuracy we use a 95% accuracy bound on horizontal position EPU (or VEPU in the vertical plane).EPU on the flight path (En Route), on which separation of 5 NM is applied, required EPU less than 0.5 NM.For ICAO version 0 it means NUCp≥4.For ICAO version 1 and 2 it means NACp≥5 [12,13].EPU in terminal control area (TMA), where separation of 3 NM is applied, is required less than 0.3 NM.For ICAO version 0 it means NUCp≥5, for version 1 and 2 it means NACp≥6 [12,13].
With ICAO version 0 there is a problem with parameter NUCp as it determines the accuracy and the integrity at the same time.Accuracy is limiting for it, so with integrity there are more limiting values for NUCp.For integrity it is for version 0 on the flight path (separation 5 NM) limiting NUCp≥4 (Rc<1.0NM).For versions 1 and 2 in the separation 5 NM is required NIC value≥4 (Rc<2.0NM) [12,13].
Versions 1 and 2 are defined by more parameters.Integrity is defined by parameter SIL, and for ICAO version 1 the requirement is SIL≥2.In version 2 the requirement is SIL≥3 [12,13].

Areas With Radar Coverage
For areas with radar coverage requirements are more limiting because it is necessary to reach ADS-B quality at a higher or equal level as the available radar technology.The requirements for the radar environment are shown in table 9.The table specifies only the requirements for version 2, as in Europe ICAO version 2 will be mandatory.We used equivalent values for the other versions.For version 1, we used SIL≥2, because in version 1 SIL determine multiple parameters (it is equivalent of the SDA, therefore probability 10 −5 is enough) and thus it is not reaching value of 3 [14,15].
In the Czech Republic, where the data were collected, we are in a radar environment, thus it is preferable to compare the data with RAD requirements.
For the comparison, Federal Aviation Authority (FAA) in USA requires similar performance [9].Therefore, it requires the ICAO version 2 and SIL 3, but for the parameters NACp and NIC the requirements are more limiting.Required is NIC<0.2NM and NACp<0.05NM for all data.

Integrity Compared to Requirements
In the figure 8-a, there can be seen the percentile of the data which meets the requirements on NIC or NUCp, respectively.The data are therefore directly assessed as required by EASA and EUROCONTROL.The key parameters were NIC and NUCp.We can see quite high percentile of data that meets the requirements.
In table 8, the data are clearly shown.There is shown the percentage and the particular number of data that met the requirements.
SIL requirements are different for version 1 and 2. It is due to the fact that SIL in version 2 only determines the integrity of Signal-In-Space.In version 2 is therefore required SIL parameter value 3.In version 1 is required SIL parameter value 2. Figure 9 shows the results.
Figure 8-b shows how much data meet the requirements for accuracy.Relatively high percentage of data that meet it can be seen.The number of messages is shown in table 8.
A key parameter was NACp or NUCp.

Conclusion
The results presented in this article are in line with the results of other studies.For the comparison, in the US, according to a survey conducted by the FAA published in 2014, about 20% Study conducted by EUROCONTROL states that approximately 73% of aircraft were equipped with ICAO version 0, 13% were equipped with version 1 and 14% were equipped with version 2. This result was published in 2016 and study took place in the airspace over Paris [10].
The purpose of the paper is to summarize the results of the statistics of quality indicators received in ADS-B messages over Czech Republic.Also the purpose is to compare actual results in Czech airspace with the EASA requirements.
As can be seen on the evaluation of the data in this article, requirements in Europe meets at least 86.42% of the data, while limiting factor is accuracy in RAD environments (collectively En-Route and TMA).RAD environment is an environment in which examined data were collected and therefore the result of this environment can be considered as decisive.In the NRA environment the least data meets the requirement for accuracy in the TMA area, namely 89.47%.In the En-Route NRA environment, the limiting factor is again accuracy, 89.63% of the data meets the requirement in this environment.The surprising fact is low percentage of ICAO certification version 2 (12.60%, see Tab. 2), as EASA requires it from 2016 for forward fit and from 2020 for retrofit.

Figure 8 .
Figure 8. Percentage of data that meets NUCp/NIC (a) and precision (b) requirement

Figure 9 .
Figure 9. Percentage of data met the SIL requirement

Table 1 .
Number of received messages

Table 8 .
Number of messages met the requirements