Agent-based Simulation of the Maritime Domain

In this paper, a multi-agent based simulation platform is introduced that focuses on legitimate and illegitimate aspects of maritime traffic, mainly on intercontinental transport through piracy afflicted areas. The extensible architecture presented here comprises several modules controlling the simulation and the life-cycle of the agents, analyzing the simulation output and visualizing the entire simulated domain. The simulation control module is initialized by various configuration scenarios to simulate various real-world situations, such as a pirate ambush, coordinated transit through a transport corridor, or coastal fishing and local traffic. The environmental model provides a rich set of inputs for agents that use the geo-spatial data and the vessel operational characteristics for their reasoning. The agent behavior model based on finite state machines together with planning algorithms allows complex expression of agent behavior, so the resulting simulation output can serve as a substitution for real world data from the maritime domain.


Introduction
The maritime domain is a complex environment consisting of many independent, though often intensively interacting entities that have their own goals and intentions.The aim of this research is to simulate not only legitimate maritime traffic, such as an intercontinental transportation, coastal fishing and recreational traffic, but also illegitimate aspects, such as illegal fishing, waste dumping and maritime piracy.
In this paper, an agent-based simulation of maritime traffic is presented, more specifically the design of an architecture of the simulation platform is described (Section 2) and the simulation configuration and control flow are explained (Section 4).A model of the maritime environment (Section 5) with several types of independent agents striving to achieve their goals (Section 6) is visualized using the Google Earth1 platform (Section 7).
The simulation is used to provide a noise-free source of maritime data with the desired spatiotemporal resolution to algorithms analyzing the situation, and it serves for developing and prototyping agent-based techniques for understanding, detecting, anticipating and eventually preventing piracy and, possibly, other categories of maritime crime.Because of the recent surge in maritime piracy, insurance rates have increased more than 10-fold for vessels transiting known pirate waters in recent years, and the overall costs of piracy in the Pacific ocean and the Indian ocean alone were estimated at US$ 15 billion in 2006 and have continued to rise [7].It is therefore appropriate to develop a set of techniques and algorithms that are able reduce this threat.A description of these algorithms is beyond the scope of this paper, but it can be found in [5].
Although agent-based techniques have been successfully applied in other traffic and transportation domains and problems (see e.g.[4,8]), this is -to the best of our knowledge -the first integrated attempt at employing agent-based concepts and techniques in the domain of maritime transport security.

Architecture overview
The main objective in designing the simulation platform was modularity and extensibility; this objective has been met by employing a loosely coupled architecture with clearly defined data and control flows.A diagram depicting the main modules and data flows is depicted on Figure 1.As can be seen, the simulation platform consists of several modules that can be arranged into the following groups: • Simulation Control -these modules control the initialization and execution of the simulation and all related processes (see Section 4).• Data Sources -these modules provide the platform with the off-line data required for initializing and running of the simulation.For a description of the necessary data sources, see Section 3. • Simulation -containing modules responsible for representing and operating the simulation model, both the simulated vessels and the simulated environment in which they operate.• Analysis -containing modules analysing the data coming from the simulation and a module emulating the imperfect aspects of the real world (noisiness and incompleteness of data).A detailed description of algorithms and modules in this category is beyond the scope of this paper.• Planning and Coordination -containing modules responsible for more complex coordination and planning beyond the basic vessel behavior implemented as part of the simulation.Two different planners have so far been implemented.A geo-spatial planner plans the route of the vessel taking into account geographical obstacles (coastline, shallows, etc.) and a planner using the results of a game-theory formulation of the relation between the transport vessel and the agent (the formulation and results are briefly described in Section 6.3).• Presentation -containing modules responsible for presenting the output of the simulation and the analytical modules using the Google Earth KML2 format.

Data sources
As an essential feature, the platform incorporates several categories of real-world data and enables integrated analysis, specifically general geographical data comprising general information about the geography of the environment, in particular shorelines, ports and shallow waters.This data is supplied directly by Google (Earth); it is used primarily for general vessel navigation and for providing the background geographical context in the user frontend.Operational geographical data comprises geographical information specific to the operation of the simulated vessel types, in particular the location of the main piracy hubs [1], piracy zones, fishing zones and transit corridors.The operational geographical data governs the operation of individual vessel categories (see Section 6).
Vessel attributes describe vessel operational attributes such as vessel type, length, tonnage, max speed, etc.This data is extracted from vessel tracking servers, e.g.AISLive [2], and is used to provide realistic parameters for simulated vessels.Activity data comprises higher-level information about maritime activity.This data is typically provided by organizations observing the situation in relevant regions.The Maritime Security Centre, Horn of Africa (MSCHOA) [3] provides up-to-date information on piracy incidents and alerts in the Gulf of Aden and off the Somali coast, including guidelines on how to proceed when traversing these areas.This data is used for pirate behavior modeling, and also for the route planning of transport vessels.

Simulation control
The simulation is executed from a script with a predefined configuration.The loading of the data and parameters and the creation of the needed data structures is carried out in the Scenario Configurator module.The simulation time flow is synchronous and discrete, i.e. the simulation time is measured in steps.In each step, a sequence of synchronized actions is executed.

Scenario configurator
The Scenario Configurator handles the initialization of all modules and all agents present in the simulation.
The configuration of the simulation (e.g. the number of vessels, the files containing vessel and GIS data, etc.) as well as the parameters of each of the modules is specified in a single Groovy3 script file 4 .The main advantage of this approach is that the script can be both embedded into a pre-compiled package and kept separate (in the source code form) to be able to run different scenarios or to modify the parameters of the simulation without recompilation.
The modularity of our architecture enables us easily to initialize only a small subset of Analyzers or to initialize a simulation without support for KML-based visualization when running non-interactive batch experiments, and logging the results for later postprocessing.

Time flow
The main parameters governing the flow of the simulation are the simulation step size and simulation step delay.The simulation step size parameter specifies how much time one step takes, measured in simulation time (i.e.not related to the real-world wall clock time 5 ).The simulation step delay parameter specifies how much time a simulation thread sleeps after each step (to leave some computing time to other components, in particular the presentation and analysis component.This parameter is measured in wall clock time.The simulation duration parameter specifies the total simulation time of a particular scenario. The Simulator module holds information about the current simulation step and the total simulation time of the simulation run.This time is measured in simulation-time seconds (i.e. the maximal granularity of the simulation time is one second).The simulation time is related to date-anchored time by the Environment module, where the Environment module is initialized to a specific calendar time (i.e.24th of December, 2009).In each time step, the Environment module updates the simulation time so that the other modules can derive additional context information about relating to the given simulation date.
The amount of time in each time step (given by the simulation step size parameter) affects the simulation granularity, and the temporal resolution of the output data obtained from the simulation.If the simulation step size is 1 s, the granularity is the finest, and we can focus on vessel behavior in detail (e.g.vessel interaction in ports etc.).If the simulation step size is set to 1 hour, the simulation runs fast, and we can roughly estimate the future positions of the vessels and events that may happen in the future.

Simulation cycle
In each simulation step, the following sequence of actions is performed: 1. Environment module updates its simulation time.
2. Agent Container is notified and it sequentially sends the information about the new step to each agent.Each agent spends the amount of time on performing a part of its plan (see Section 6 for details).Generated events are sent to the Agent Container, which distributes the events to all relevant listeners and to the Environment that records the event.

If -as a result of actions of the agents -new data
become available, this is sent to the subscribed Analyzers for processing and analysis.The synchronous nature of the simulation control, together with seed-based randomization, guarantees the determinism and thus the repeatability of the simulation.Deadlocks cannot occur, as access to resources is sequential; this also enables the use of unsynchronized data structures, which are faster to work with.

Environment model
The state of the environment is maintained by two data modules -the GIS Data Provider and Vessel Data Provider and in variable storing the current simulation time/date.The Environment module is a central access point for the other modules to the environmental data including data about the vessels.
The temporal information is represented in standard time and date units.The main units of spatial information are nautical miles; speed is measured in nautical miles per hour or per second.

GIS data model
The GIS Data Provider stores the geographical data and provides access to this information.The following data is represented and is available for analysis and display: • Somali harbors -a list of Somali harbors with name, location, a description and approximate capacity for pirate vessels.This data is used for pirate vessel initialization and placement.The original file is a KML file. .This data is used by the transport vessels to plan their trajectory.

Vessel data model
The Vessel Data Provider provides data about the ships in the simulation.Each record is about one vessel and is identified with a unique identifier, Vessel ID (this ID is equal to the ship's call sign when real-world ship parameters are used).Vessel-related data can be categorized into two groups: • Static vessel information -static information stays the same throughout the simulation and cannot be modified.This information can be viewed as a database table where the rows repre- sents individual vessels and the columns represent the attributes of each vessel.The data is stored in an SQL database for easy access.• Vessel trace information -a dynamic part of vessel information, storing the actual position of the ship as well as the previous positions in the form of time-annotated location records.

Agent vessel model
The agents reside in the Agent Container, which distributes events gathered from the agents or from the Environment.Each agent controls one or more vessels.
The plans for each vessel are either created prior to the simulation (e.g. for transport vessels) or, typically, are generated dynamically during the simulation run (e.g. for pirate vessels).

Vessel types
The platform can simulate the simultaneous activity of a large number (thousands) of the following categories of vessels: Long-range transport vessels are large-to very large-size vessels transporting cargo over long distances (typically intercontinental); these are the vessels most often targeted by pirates.Shortrange transport vessels are small-to medium-size vessels which transport passengers and/or cargo close to the shore or across the Gulf of Aden.These ships are local, and are not attacked by pirates.Fishing vessels are small-to medium-size vessels which go fishing within designated fishing zones; fishing vessels launch from their home harbors and return back after the fishing is completed.These vessels can potentially be attacked by a pirate, but currently only local fishing vessels, which cannot be attacked, are simulated.Pirate vessels are small to medium-size vessels operating within designated piracy zones and seeking to attack a long-range transport vessel.The pirate control module supports several strategies, some of which can employ multiple vessels.Table 1 summarizes the main parameters of each type of vessel agent.Note that, in general, a vessel agent can control more than one vessel (e.g. a mothership pirate vessel agent controlling several small boats and a hijacked transport ship).
The behavioral models for individual categories of vessels have been manually synthesized on the basis of information about real strategies, which have been obtained from several sources including the IMB Piracy Reporting Centre7 and the Maritime Terrorism Research Center8 .

Executable behavior representation
Vessel agent behavior is implemented using finite state machines (FSM).Agent FSMs consist of states that represent an agent's principal mental states.Transitions between the states are defined by unconditional or by conditional transitions conditioned by external events.Implementation-wise, the simulator allocates a time slice to the agent and the agent delegates the quantum to the FSM.The current state may either use the whole time slice and stay in the current state, or it can utilize only part of the time slice and delegate the rest of the time to a following state or states.An example of a pirate FSM is depicted in Figure 2.

Path planning
A modular route planning architecture has been developed allowing us to combine general shortest-route point-to-point planners with specialized planners for specific areas.A more detailed description of the operation of both planners can be found in [5] and [6].

General shortest-route navigation
The basic route planner finds a shortest route between two locations on the Earth's surface considering ves-sel operational characteristics and environmental constraints, including minimum allowed distance to shore, which can differ between regions.The planner is based on the A* algorithm adapted for a spherical environment and polygonal obstacles.

Gulf of Aden transit planner
Factors other than route length need to be considered when planning a route through the Gulf of Aden.Two specialized planners have been therefore developed for this purpose.The simple corridor planner navigates the ship through the International Recommended Transport Corridor and mimics the current practice in the area.As a possible alternative, several alternative route planners through this area have been implemented.A risk-minimizing route planner uses a pre-generated risk map to avoid high risk areas.
A game-theory planner solves a two player zero-sum game between the transport ship and the attacker to find an optimal path through the gulf.A detailed description can be found in [5].

Visualization
The user front-end provides a geo-based visualization of the various outputs provided by the platform, using Google Earth, a KML capable viewer.The Google Earth-based front-end allows us to interactively visualize the various output of the testbed, both the static data described in Section 3 and the dynamically generated output of the advanced analysis and planning modules.A screenshot of the front-end is given in Figure 3.
In addition to ergonomic navigation and 3d camera control, the main advantage of the front-end is the ability to present structured data on varying lev-els of detail.The layer-based interface allows us to select different layers of information and compose an information picture with the aspects and the level of detail fit for the specific user's needs.To leverage the layer-based concept, the testbed output is organized into multiple information layers.The simulation itself provides a number of layers; each of the analysis and planning tools then also adds its own layer.
The integration with Google Earth is provided via dynamically constituted KML files served by an HTTP server running inside the platform.The KML files are read into the Google Earth application using its HTTP data link feature and automatically refreshed.This way, dynamic data can be displayed (such as a moving vessel), though for performance reasons, the refresh rate is limited to about once a second.

Fig. 1 :
Fig. 1: Main modules and data flows.The lines represent the main data flows, the dashed line represents the temporal synchronization controlled by the simulation loop

Fig. 2 :
Fig. 2: Finite state machine of the pirate vessel agent

Fig. 3 :
Fig. 3: Google Earth-based front-end showing vessels, their past trajectories and an output from dynamic incident risk analyzer over the Gulf of Aden

Table 1 :
Main parameters of different types of vessel agents.