CROSS-STUDY RESEARCH ON UTILITY AND VALIDITY OF DRIVING SIMULATOR FOR DRIVER BEHAVIOR ANALYSIS

Driving is one of the most ordinary and universal everyday tasks and, at the same time, one of the most complex and dangerous. It requires a full range of sensory, perceptual, cognitive, and motor functions, all of which can be affected by a wide range of stressors and experience levels. Therefore, exploring of human behaviour while controlling a vehicle is a crucial task in improving traffic safety. Experimental studies can always be conducted with on-road tests, however, using a simulator is safer and more cost-effective. The main goal of this paper is to demonstrate if and under what conditions could a driving simulator provide sufficient results required for a proper study of driver behavior. It discusses its limits and advantages. Overall, the research reviewed in this paper indicates that simulator driving behaviour approximates (relative validity of speed and lateral position of vehicle on road), but does not exactly replicate (absolute validity), on-road driving behaviour.


Introduction
With rapid growth of technological advancement, not only executive elements or traffic engineering like intelligent transport systems, is shifting toward more sophisticated and ecologically and economically valid solutions.Another example is research on the transport subjects which is reaching for advanced tools like driving simulators.However, although supported in many government and scientific organizations [1] the usefulness and validity of results achieved in a driving simulator remain a very hot topic in transport engineering -especially when driving behaviour and the psychology of the drivers are concerned.Although this topic seems to be of significant importance (either supports of diminishing effects and results of driving simulator experiment), it seems to be often forgotten during the execution of the experiment, which may, at times, lead to meaningless conclusions.Hence, it is very important to produce a literature review on the topic in search of significant data to utilize simulator results to make conclusions on real-world driver behaviour.
This document includes a table (Table 1) which summarizes the most important papers on the topic, as well as further analysis and description of chosen papers, all of which are divided into several categories depending on the kind of validity confirmed and parameters measured in the experiment.The table allows for quick familiarization with the most important research reviewed and introduces an in depth analysis of specific studies.Chapter two consists of subsections that are dedicated to particular kinds of validity which can and should be of concern when setting up a driving simulator experiment.This paper produced a cross-study review and analysis of confirmed and proven validity measures for driving simulator exper-iment measures, and is concluded by summarizing recommendations and predictions regarding the future of driving simulators as tools for various studies on driving behaviour.

Reviewed studies
Despite the importance that is placed on the simulator as a tool for investigating driver performance and behaviour, an extensive literature review reveals that only a very limited number of studies specifically evaluating the behavioural validity of simulators was conducted [3].Majority of the available literature on simulator validation has focused on measures of speed, lateral position, and braking responses (Table 1).The following summary of literature research reviews literature on the given topic with a specific emphasis on the description of performance measures, the tasks and conditions, and first of all the evidence provided for behavioural validity of each simulator and the behavior of the subject "driving" it.

Speed adjustment
Speed is one of the most studied measures of driver behaviour and, thus, one of most commonly analised with respect to simulator validity [2, 5-7, 9-11, 13, 15, 17].Overall, these studies rather consistently provide results confirming relative but not absolute validity of speed data in driving simulators.Most explanatory ones were for example Klee et al. [10] who examined the validity of a fixed-base simulator with respect to forward speed.30 drivers drove the same road section on both driving simulator and in an instrumented vehicle.The speed values were measured at 16 points, while measures in 10 of them were similar in both observed environments.While authors do not conclude relative validity of the simulator and focus on lack of
absolute validity, it can be observed that speeds in all 10 locations with similar results were 5-10 km/h slower for simulated driving.Moreover, described speed difference provided the same results in each case which implies relative validity of simulator.Another author providing evidence for the relative validity of the simulator maintained speed and its correlation with the on-road speed is Bella [2,15].Bella provides two studies in which the simulators were designed with identical road sections to those where real driving was performed by experiment participants.In real world experiments, speed measurement points were specified along the road instead of the instrumented vehicle driving record.The first experiment was conducted on road section with construction zone with speed limitation applied, whereas the second study described a highway in normal condition.A bilateral Z-test for non-matched samples was performed in both data sets to estimate whether the recorded driving speeds were significantly different from those in the simulator.Consistently with other researches Bella conducted that simulator speeds were different than in the real world, although in [2] mean speed in simulator was higher than in real world while in [15] it was on the contrary.Nevertheless, the differences in simulator speeds and in the field speeds for each point were not significantly different, demonstrating at minimum, interactive relative validity of static simulator tool for assessing speed.Unconventional way of assessing speed as simulator validity indicator was presented by Shinar & Ronen [17] where two speed related measures were observed: speed estimation (i.e.driver perception of the current speed of vehicle) and production of speed (i.e.adjustment of the vehicle's speed to reach predetermined speed).Sixteen participants took part in an experiment where both on-road driving and simulator (with shielded speedometer) drives were performed.During each drive, participants were repeatedly asked to estimate the current speed of vehicle and produce seven different speeds ranging from 40 to 100 km/h.Authors found that estimated speeds differed significantly in the simulator only for speeds of 40 and 100 km/h where the first speed was always estimated higher in simulator and latter one was always estimated lower.On the other hand, second investigated variable showed consistently higher produced speeds in simulator with a difference of approximately 25 km/h.Conclusions of multiple analysed studies pointed out towards absolute validity of most of the estimated speeds and vast majority of the studies suggests relative validity for speed produced in the simulator.This seems to form firm foundations for performing scientific research with the use of driving simulators.

Lateral position of vehicle
Second often validated measure in driving simulators is the lateral position of the vehicle on the simulated road.One of the most interesting studies in the matter of lateral positioning validity was performed by Wade and Hammond [8].The authors measured positioning deviation as distance from the centreline of the road.
In a tested sample of 26 participants, the mean deviation of the distance from centreline was statistically significantly larger in virtual driving when compared to the on-road conditions.Authors determined that in the real-world it is clear that drivers travel more accurately near the centreline when compared to virtual driving experience.This led to the conclusion that absolute validity was not demonstrated in performed research.Nevertheless, Mullen, Charlton, Devlin and Bedard [3] noticed that similar to results in speed researches mean deviations on different types of roads showed a familiar pattern when compared to the real environment (i.e. the road with the highest mean deviation in the real-world was also the road with the highest mean deviation in the virtual environment) which indicates relative validity.However, it is a noticeable fact that data were collected on straight road section with a relatively low speed limit of 35 mph (approx.56 km/h).Although Blana & Golias [12] tried to answer the question of speed influencing validity of lateral displacement measures, it remains unknown.
In their research scientists conducted experiment in a simulator where distance between front wheel of the vehicle on the passenger side and white line determining the edge of the road were measured.A sample of 100 on-road drivers in vehicle instrumented with video cameras pointing toward road surface drove on elaborated highway section, while corresponding road was also driven in simulator by 100 participants.The results clearly pointed towards higher average distance measured in the real-world conditions when compared to simulator on both straight sections and curves.This indicates that drivers in the on-road conditions position their vehicles closer to the centreline of the road lane.This conclusion was further confirmed by smaller standard deviations (SD) for the measures recorded in real-world conditions.Although the direction of displacement from the road edge was consistent for curved and straight road sections for all speeds measured, the magnitude of the difference in displacement and SD varied depending on speed, leading to inconclusive results in terms of absolute or relative validity of the simulator.Again, these results seem to provide strong argument supporting driving simulators as recommended research tools.

Road safety countermeasures
The simulator validity for evaluating the relation of a driver with road design and traffic control devices was another issue often examined.Riemersma et al. [6] performed two experiments aiming to understand, explain and describe these effects.First, the Daimler-Benz driving simulator in which the authors created a scenario where the driver entered a Dutch village area.The goal of the simulator was to register changes in desired speed adjustments.The drivers were divided into 2 groups and drove the car either on a normal road or on the same road with special speed reduction objects.Three infrastructure changes were implemented in the second scenario, those being a median strip, coloured asphalt and a portal gate.Analysis of the vehicle speeds in both scenarios revealed decreased speed values in the scenario with special objects installed.Moreover, faster drivers reduced their speeds more than average speed drivers.The second experiment involved both real-world driving and in-simulator ride, utilizing their university driving simulator to perform speed reducing infrastructure testing.The experiment was conducted on 24 participants, of which each drove through the village entrance 12 times.Infrastructure changes varied during the rides.Driving speed comparison between real-world and simulator revealed higher average speed of in-simulator driving than on road conditions at distance of approx.400 meters from village entrance.Nevertheless, an interesting finding was that, after the 400m distance point, simulator drivers reduced their speed to a greater extend, thus entering the village at a lower speed.The magnitude of speed reduction was evaluated as 3 times bigger for simulator than real-world (25,7% and 8,6% respectively) this can indicate similarly to chapter 2.5.2, relative but not absolute validity of the simulator results.Due to these results, the authors concluded that a simulator was an effective tool for evaluating speed reduction methods.Following research of McAvoy and Datta [16], found in their study that AMOS II simulator has contrary results for evaluating night traffic speed control devices in construction zones.A model construction zone on a highway with warning lights was designed and implemented into simulator conditions.35 drivers took a ride in simulator at the road described above, while in naturalistic study vehicles average speed was measured on road with the use of radar guns.The summary of the results led the researchers to conclude that the simulator failed to provide proof for relative or absolute validity standards.A suggesteed explanation for that fact was the subject's perception of risk associated with driving through real-world work zones at night in a simulator-provided experience.The final paper reviewed in relation to drivers' behaviour studied using a simulator was the work of Godley et al. [13].In this study again, similarly to Riemersma work [6], relative validity of simulator results was found for effect of rumble strips as a speed-reduction method.

Complex driving behaviours
Another examined validity in short history of driving simulator experiments is in relation to the complex behaviours of drivers such as tasks of divided attention.Reed & Green [9] investigated validity of the driving simulator constructed at the University of Michigan Transportation Research Institute.The goal of the study was to assess decrements in driving performance during a manual car phone task.Participants of the experiment were divided into 2 groups: young (20-30 years) and old (above 60 years old), and received instructions to dial a manual phone call during vehicle driving tasks both in the on-road setting and in a simulator.Conclusions of the work exhibited that speed parameters (SD of speed and throttle position) were similar in both studied conditions, whereas lane positioning (i.e.mean lateral speed and SD of steering wheel position) varied significantly more in the simulator.Overall, for seven of the ten variables measured in the experiments, a significant correlation was found between two environments (authors provide correlation factor r in range of 0.43 to 0.76).For all experiment participants regardless of the age group, the phone task resulted in a decrease in driving abilities in terms of speed control and lateral position of vehicle.Worth mentioning is the fact that, just like in the previous studies, the magnitude of impact of particular driving conditions was higher in a simulator than on the road.Additionally, older drivers seemed to suffer more from the phone dialing task than young drivers in terms of driving abilities.Few other studies in this area were performed, but ones worth mentioning are Lee, Lee and Cameron [14] (distinguishing between drivers with different abilities), Lee & Lee [18] (identifying drivers at risk of future violations), Reimer et al. [19] and Salvucci et al. [20] who investigated an iPhone, button style flip-phone and iPod distraction on driver performance.Results of these works however, are based on very few participants therefore, although similar to the ones described above, they will not be summoned in detail here.

Physiological measures
The final validity of importance investigated in this research are psychological responses.While the previous sections described a direct way of measuring driving behaviour, physiological responses are commonly used to determine driver awareness and thus produce some useful insight into driver behaviour and reactions across both simulator and real-world contexts.An important condition that is required from driving simulators all over the world is that simulations should induce and provoke similar physiological responses as real-world experience.First research conducted on that topic was Watts and Quimby [4], who used skin conductance measures for subjects in a Transport and Road Research laboratory simulator.Due to the technological limitations of the time, driving simulation was performed in form of film presented to the drivers that was placed to create the impression of being the driver of a vehicle while film from a drive on a road was displayed.The presented film was 30 minutes long, and consisted of 45 hazardous situations for which drivers had to evaluate the perceived danger on a 10-point scale.Conductance of skin was measured for all participant during the film.The measurements were then repeated for a real-world driving condition and the results were compared.In the presented results, the risk assessments were comparable for both environments, and moreover, the perceived risk ratings were positively associated with skin conductance levels (r = 0.78).A more recent study performed by Slick et al. [21] utilized measures of both skin conductance and heart rate in both simulator and real-world conditions.The experiment performed in DriveSafety DS-600c motionbased simulator was made of four specific driving tasks performed on both stop sign and intersection traffic lights.The study suggests that there are no significant differences between skin conductance and heart rate measures between the simulator and realworld driving conditions.The only exception for female drivers was that their mean change in heart rate was significantly smaller in the simulated condition than while driving on-road.The most recent study is one analyzed by Mullen, Charlton, Devlin and Bedard [3].Bedard in his research measured heart rate and respiration for participants exposed to threatening situations while driving STISIM 400 simulator.Two intersection scenarios were investigated in depth with the green light turning into orange immediately after driver approached the intersection, and vehicle appearing from the right shoulder of the road in front of the driver.Results indicated high measure of presence of the simulator.In each observed situation, the driver's heart rate increased significantly, and 65% of the participants showed increased respiration rates in risky situations.Overall, these studies suggest that physiological measures in general demonstrate the validity of simulators.
Overall, these studies suggest that physiological measures in general demonstrate the validity of simulators.

Summary and Conclusions
Many studies have investigated the validity of simulators for examining driving behaviour, leading to a conclusion that there is sufficient evidence to suggest that simulators provide a valid tool for assessing various driving performance measures.Some important findings on the topic of simulator implementation can be found in studies like [22][23][24], where overwhelming majority of the studies suggest that there are no obstacles in using driving simulators.These studies suggest that simulators provide approximated behaviour and reactionsof the drivers (relative validity) , despite the fact that it does not replicate on-road behaviour (absolute validity) exactly.The studies under review confirm a moderate to high correlation of the simulator-achieved results for a whole range of cognitive measures of drivers as well as physiological reactions.This implies that a driving simulator is a sufficient tool for majority of research purposes, and thus may be accepted as suitable for the collection of desired data.Nevertheless, it is of high importance to remember that the fact that a driving simulator was shown to be valid in a previous setting does not guarantee it will be suitable for the next one.Therefore, it is highly recommended to perform additional simulator validation of the results for each specific setting and the scenario designed for a given simulation experiment.This ensures the researchers are capable of utilising this data to create solutions applicable to real-world situations.On road measures will surely be necessary and prevail over the driving simulator method in research situations in which absolute validity of measured variables is required.Although there are certain drawbacks and limitations to current simulator deployment in transport research, there is a confident promise of overcoming this issue through technological advancement and, thus, broadening of simulator validity by creating better imitation and perceptual projection of on-road condition for simulator drivers.This will undoubtedly increase the role of simulators as a tool for driving performance measurements as well as for studying driving behaviour.Such studies are certainly important not only with respect to the traditional transportation research tasks (transportation planning, study of traffic flow characteristics, optimization of traffic control and others), but also for new and emerging tasks such as within the field of Smart Cities (resp.Smart Mobility), study of drivers using cooperative vehicles by Přibyl and Svítek [25].The authors of this paper place high importance on understanding which real-life conditions a driving simulator can replicate and aimed to provide guidance in this respect through this document.