Application of Resilience Assessment Grid in production of aircraft components

The paper evaluates application of the Resilience Assessment Grid (RAG) method in aircraft components production. For the purpose of evaluation, specific RAG-based model for a change process was created. The paper contains description of the change and approval process of an engineering-related change proposal within an organization dealing with aircraft components production. For the purpose, this paper utilizes Business Process Modeling and Notation (BPMN) and the Functional Resonance Analysis Method (FRAM) to support RAG-based modeling, both helping with clarification of relations among the functions inside the management of change process. The paper presents an aviation-specific approach for the Resilience Assessment Grid application. The outcome is a model of a managed change process in a company producing aircraft components, with recording the current state of four resilience potentials that can be used by the company management to develop better safety awareness among the company employees and to increase the potential for the organization resilience performance.


Introduction
Safety in aviation industry is a mandatory element, especially in a production of aircraft components. Due to variation and combination of parts which form the system and human interactions with them, it is possible to characterize companies producing aircraft components as a complex socio-technical system. Socio-technical systems involve complex systems in which many people cooperate to achieve a common goal by using the technology and its tools [1]. Therefore, to manage the safety of socio-technical system it is important to find a proactive way which will provide not only the tools to ensure the freedom from unacceptable harm, as defined in Safety-I, but which will also provide the way how can organization's potentials of resilient performance be developed. This paper considers Resilience Assessment Grid (RAG) [2] as the fundamental method for the safety management in application of Safety-II. Whereas theories about safety are continuously explored, resilience engineering with the axioms of Safety-II lead to new definition of safety as the situation where as much as possible goes well [3]. This implies that organizations should focus on everyday activities that are necessary for acceptable outcomes and to support them or mitigate activities that lead to potentially unacceptable results or accidents.
Any organization can achieve resilient performance. In order to support this ability, an organization should be able to adapt its performance to the given conditions, it should be able to respond to changes (whether positive or negative) and to respond flexibly within reasonable time period [4]. Resilience engineering presents four abilities (potentials) to be considered for achieving resilient performance, namely the potential to respond, monitor, learn, and to anticipate. However, having these abilities does not mean that the organization will inevitably perform in a resilient manner. While the establishment and retention of these abilities does not itself guarantee the resilient performance, the lack of these potentials will certainly make the organizational potential for resilient performance very low [2].
For the measurement of resilience potentials, RAG method can be used. The RAG is normally carried by means of questionnaires to create a resilience profile of an organization and, to this end, the method uses analysis of individual functions in the assessed process, which can be supported by the establishment of a functional representation by means of the Functional Resonance Analysis Method (FRAM) [5]. Responses from the application of RAG provide the basis for assessment of the organizational potentials to perform in resilient manner. This paper describes application of the RAG in an organization producing aircraft components, assessing the change process regarding proposed engineering-related changes and their approval within the organization. Such application can be used as a starting point for the organization to recognize its performance regarding Safety-II compatible management and to improve its potential to perform in a resilient manner.

Background
Since the specific application of RAG presented in this paper is assessing the change process inside a company producing aircraft components (not disclosed due to confidentiality reasons), this section provides description of the change process and the proposed resilience potential measurement in the context.

Change process
Change process is one of the processes managed by configuration management. It provides identification and ensures traceability of the documentation needed for implementation of the engineering-related change proposal, while providing access to up-to-date data at all stages of the change process. The change itself regards modification of a product described in the technical documentation and it may be initiated by the design organization, final product customer or supplier itself. All design changes either during development of the product or during serial production must be identified, documented, controlled and their implementation must be managed and planned [6]. It is mandatory to document product configuration information such as identification number, name, revision, quantity, etc., and to keep information about each proposed change, including the actions of the units responsible for performing individual tasks, or tracking of task schedule fulfillment.
Build to print (BTP) process is a contract manufacturing process of a company manufacturing products, equipment or components according to the customer's requirements [7]. The process is selected in this work for resilience assessment. Next follows the list of individual activities of the change process control regarding BTP production of aircraft components: • request for change or creation of the engineering-related change proposal (ECP) • assessment, evaluation and registration of the ECP • production of changed product • verifying of the change implementation

Resilience Assessment Grid
Thinking about an organization's resilience performance implies the necessity to look at what the system does rather than what the system is or has [2]. In order to estimate the resilient performance, four potentials are proposed to determine how well a system performs regarding each one. Potential to respond is defined as "being able to respond to regular and irregular changes, disturbances and opportunities by activating prepared actions, by adjusting the current mode of functioning, or by inventing or creating new ways of doing things" [2]. In simple terms, the potential is about the ability of an individual or collective to react to emerging situations, whether expected or unexpected.
Potential to monitor means "being able to monitor that which affects or could affect an organization's performance in the near term -positively or negatively. The monitoring must cover an organization's own performance as well as what happens in the operating environment" [2]. This potential attempts to assess whether an individual or collective is capable of adequate monitoring of relevant variables with respect to its activity or operation and related risks.
Potential to learn is about "being able to learn from experience, in particular to learn the right lessons from the right experiences. This includes both single-loop learning from specific experiences and the double-loop learning that is used to modify the goals or objectives. It also includes changing the values or criteria used to tailor work to a situation" [2]. In simple terms, the potential measures how much an individual or collective can learn from both positive and negative experiences to improve own activity or operation.
Potential to anticipate assesses "being able to anticipate developments further into the future, such as potential disruptions, novel demands or constraints, new opportunities or changing operating conditions" [2]. This means the ability of an individual or collective to foresee based on the present state of variables and the history of experience.
These potentials are interdependent and usually do not operate as a separate functions. As already mentioned, RAG was developed to provide an accurate characteristic or an organization's profile about how well it performs in regard to the four potentials. This is accomplished by the creation of four sets of diagnostic questions that need to be developed for each specific organization. Number of questions for each set is not specified because each organization is unique and often of different size and complexity. It is recommended to apply RAG repeatedly over some time period to evaluate the organization's status over the given period or to review the organization's attitudes toward its defined objectives. Hence, it is helpful to address questions in a formalized way as it is recommended to re-apply them. This can be achieved by distributing questions via e-mail or through a website, which reduces the need for personal meetings with the respondents [2].
It is desirable that the results of one evaluation are easily comparable with the results from another. Such comparison can show the weight and direction of any changes that may have occurred. Therefore, representation of the applied RAG results should take into account its intended repeated application and one possible way is to present the results via radar chart. For assessing each potential, the usage of the Likerttype scale [8] is recommended. This way respondents can rate questions addressed by the RAG method. With regard to the system's particular potential, a clear signature of how responses are distributed is shown in a star-like polygon plotted on the radar chart [2].

Methods used to support creation of the specific RAG model
In order to assess the application of RAG in the particular company producing aircraft components, specific model for the change process was created. The RAG application was supported by the creation of Business Process Model and Notation (BPMN) [9] and Functional Resonance Analysis Method (FRAM) based models. Both clarify functions (activities) relations within the change process and examine in detail the relationships among particular processes of the ECP approval process. Using the FRAM model is recommended by the author of the RAG method [2]. The regarded processes represented by FRAM notation are depicted in Fig. 1. Based on the model, the individual functions were analyzed in detail to properly understand the whole functioning of the analyzed process. To verify the FRAM model, BPMN-based model was created and its representation is depicted in Fig. 2. By making use of both models it was demonstrated that results from RAG application can be related with the description of the company's organizational process via BPMN representation, relating the results to the business processes and supporting further decision-making process.

Results
The RAG-based questionnaires were derived from the characteristics of individual potentials and aimed to analyze relations in the specific process of a complex socio-technical system based on respondents' answers. Understanding how resilient a system can perform might valuably contribute to determining how effectively the company manages its processes. During the initial stage of the FRAM and BPMN-based modeling, the company's already known issues regarding the change process were discussed with two members of the approval process (whose answers are not included in the final statistics). The RAG questions were modified accordingly to accommodate the specifics of the assessed change process environment. At the same time, some questions were split in order to receive clear answers. Stylistic adjustments were made for better understanding of the questions meaning.
RAG-based questionnaires were distributed to 14 members of the change process in the company producing aircraft components via e-mail, with link to the questionnaire. Variety of the respondents helped to establish a wider picture of the process. Department membership of the respondents is depicted in Fig. 3. Respondents assessed each question by the 5-point Likert-type scale, presenting their attitude on the statements with their full agreement, agreement, neutral attitude, disagreement and full disagreement.  Table 1 represents final list of categories assessed by respondents regarding each potential.
Concerning the potential to respond, members of the process agree that the internal change process documentation clearly describes the process and they can respond adequately even in situations not described by the documentation. They also agree that they can ensure the necessary resources to start their activities. However, what they assess unsatisfactorily is the efficiency of the process, as their assessment shows that the change process generates need for extra efforts. Members of the process do not assess communication as effective and they also assess as unsatisfactory the provision of inputs to the change process at a specified time interval. The final radar chart for potential to respond is depicted in Fig. 4. The potential to monitor seems to be the most developed among all of the assessed. Respondents rate almost all the RAG questions satisfactorily. They express their agreement that the real processes in operation are as defined by the internal documentation. They satisfactorily evaluate members' ability to detect indicators leading to internal denial of a change proposal and agree that the system database is regularly maintained and updated to provide both current and historical data. The respondents assess the use of data from the internal system to define the commercial impact of the change proposal satisfactorily. Respondents evaluation of implementation monitoring and monitoring of lead time are considered appropriate. Respondents express full agreement to review individual members' inputs into the change process Well-maintained database A6 Regular review of internal approval processes --L7 Willingness to deal with possible malfunctioning of the system within the CAB commission meetings. The final radar chart for potential to monitor is depicted in Fig. 5. The results of the assessment of potential to learn are depicted in Fig. 6. Respondents evaluate as unsatisfactory the course of staff training for the change process. They consider their willingness to deal with the possible malfunction of the system and their ability to acquire the information they need in the internal system to be appropriate so as they consider the database to be transparent. Respondents assess the process, revision and regular incorporation of changes to the change process with the same opinion. They are satisfied with the subsequent presentation of process modifications and the exchange of information among them throughout the process.
Concerning the potential to anticipate, members of the process agree that the replacement of the members does not pose a long-term threat to the approval of change proposals. Respondents agree that by knowing the procedures they know who to contact in need of help. Respondents consider regular review of internal approval processes through existing change process oversight as appropriate. Respondents also assess the Assessment results for the potential to learn adaptation of the system to adjust its activity when the volume of the incoming proposals is changed as appropriate. They express disagreement with sufficient capacity adjustment, based on regular monitoring of changes. They also disagree with the general knowledge of the benefits of the program's strategic plans. The final radar chart for the potential to anticipate is depicted in Fig. 7.

Discussion
Resilience assessment is a difficult task, mainly due to the tools available being rather empirical and context-dependent [10]. By application of the RAG method in the company producing aircraft components, several observations can be concluded. Results show that the system is capable of modifying its behavior when operating conditions are changing. System is also ready to respond to the specific needs in certain areas, but clear shortcomings were identified in the learning ability. By creating a picture of how functions relate within the change process, the importance of communication and the essence of process clarity were highlighted. Individuals participating in the process contribute to the overall organizational resilience and, therefore, it is essential for them to understand the process.
The analysis of the RAG application results shows that the change process members perceive the application of theoretical procedures as adequate in real operations. However, this does not mean that internal documents should not be further adapted to current needs.
The analysis further shows that the company keeps a thorough record of data and their management is adequate. The system's ability to monitor is high. The results also show that the company has a well-arranged database in the internal system. Data from the internal system is effectively processed and easy to navigate. The analysis has shown that the individual potentials are interdependent and the overall degree of resilience of a company depends on the weight that organization attributes to it.
As for the very RAG application, the study demonstrated that it is feasible to apply the method in aircraft components production and that the results from the method application are useful for company management and decision-making. The method could identify both strong and weak points, and the both formative and diagnostic nature of the questions enabled not only diagnosis of a problem, but also provided basic guidance for the company management.

Conclusion
This paper describes application of the RAG method used for the assessment of the resilience performance of a company producing aircraft components. The work provides supporting evidence for further development of resilient measurement tools in the field of safety management. It is important to note that RAG should be applied differently for individual companies, accommodating the method to specific company environment. Furthermore, the outcome of the RAG application in this paper is establishment of a change process profile in a company producing aircraft components. It resulted in the company's profile of four potentials under the current conditions. Recording the current state of the four potentials can be used by the company management to develop wider safety awareness among employees.
Although it is recommended to apply the RAG repeatedly over some time period, this work applied it only once. Another limitation is that the method was applied only on one process, which is a small fraction of the entire assessed organization, so the results do not account for the company resilience as a whole.
Future research in this domain could consider repetitive application of RAG over some time period so as the extension of its application to more company processes. The results of this work look promising and support the industrial application in the aircraft components production.