APPLICATION OF MACHINE VISION TECHNOLOGY IN ROAD DETECTION

Machine vision was first applied in industrial manufacturing field, and now it is also used in road detection, with the rapid development and continuous innovation of computer technology and digital image processing technology. This study provides a detailed description of the application of machine vision technology in detection of pavement crack, such as crack image acquisition, preprocessing (image de-noising and enhancement), segmentation, and recognition technology. Further the application of machine vision technology in pavement compactness and evenness was introduced. Finally, based on the application of machine vision in other aspects of road detection, hopefully, this study can provide a reference for method of road detection.


INTRODUCTION
Since the end of 20 th century, with the rapid development and continuous innovation of computer technology and digital image processing technology, machine vision technology has been progressively applied to highway pavement detection.In this period, the United States, Japan, France and other developed countries have devoted extensive research efforts and development work on the main contents of the road automatic testing equipment and pavement management system, and published a number of related research results [1].Although the application of machine vision technology in the automatic detection of pavement in China has started later, it is developing rapidly, and gradually narrowing the gap with the world's advanced level [2].This study reviews the application of machine vision technology in detection of pavement cracks, pavement compaction, and pavement roughness, with primary focus on the application of machine vision in pavement crack detection.Finally, based on the application of machine vision in other aspects of road detection, hopefully, this study can provide a reference for method of road detection.

THE DEVELOPMENT HISTORY OF MACHINE VISION TECHNOLOGY
A machine vision system refers to the capturing of images through a machine vision device (i.e.image capture device).Then the image is transferred to the processing unit through digital processing.Based on the pixel distribution and brightness, colour, and other information, it can discriminate the size, shape, colour, and other parameters, and then according to the results it controls the operation of the field equipment.The study of machine vision began with the pattern recognition in two-dimensional images in 1950s.In 1960s American scholar Roberts put forward the concept of building blocks of polyhedron [3], preprocessing, edge detection, and object modelling which are still used in the field of machine vision.In 70s the visual computing theory proposed by David Marr provided an unified theoretical framework for machine vision research [4].At the same time, machine vision was incorporated in several important research branches such as target guided image processing, parallel algorithms for image processing and analysis, knowledge base of vision system, and so on.In 80s with the rapid development of machine vision technology, new technology and new knowledge were continuing without end, for example, the theoretical framework of object recognition based on perceptual feature group, the framework of active vision theory, and the framework of visual integration theory, etc.By the middle of the 90s, machine vision has entered a period of deep development and wide range of application and by now, machine vision has become a very active research field.This section should describe in detail the study material, procedures and methods used.

APPLICATION OF MACHINE VISION IN DETECTION OF PAVEMENT CRACK The composition of machine vision system for detection of pavement crack
The machine vision system for detection of pavement crack is mainly composed of two parts: hardware system and software system.The hardware system mainly includes power supply system, image acquisition system, and image lighting system.The software system mainly includes image preprocessing system, image segmentation and recognition system, and so on, as shown in Figure 1.

Pavement image acquisition
Pavement image acquisition system is the foundation and key of the entire machine vision system, which determines the accuracy and precision of image analysis.The pavement image acquisition system is mainly composed of industrial camera, lens, lighting device, speed sensor, and digital image capture card.At present, the pavement image acquisition is mainly acquired using the industrial camera.Based on the type of sensor, the industrial camera can be divided into charge-coupled device (CCD) camera and complementary metal oxide semiconductor (CMOS)  2. Owing to the high imaging quality of CCD camera, it is usually used in pavement detection.The linear array CCD camera can only obtain one line of information at a time.It has fast speed and high resolution, which can realize continuous detection of moving objects; therefore, the linear array CCD camera is especially suitable for image acquisition for pavement detection.At the same time, the change of the ambient light plays a very important role in the image acquisition; therefore, the selection of light source in the image acquisition system is also important.The light emitting diode is widely used as a lighting device in pavement acquisition system because of its small size, low power, fast response, high reliability, and long service life.

Pavement image preprocessing
Uneven traffic signs, road line spilled materials, oil, shadow, and other interference, as shown in Figure 3, leads to strong noise in the pavement image, which increases the difficulty of identification of pavement crack.Therefore, it is necessary to eliminate or weaken these interferences and preprocess the pavement image.Although there are various methods of image preprocessing, they can be divided into two categories according to different emphasis: image denoising and image enhancement.

Image de-noising
The noise in pavement image mainly comes from two aspects: one is the interference from the external environment, such as uneven illumination, shadow, etc.The other is road itself, due to the heterogeneity of the road surface, in particular, the size of the road material, particles, and the different roughness of the road.According to the actual situation of road noise, the noise can be divided into two types: random noise and interference noise.Random noise is commonly referred to as image noise; however, the interference noise is referred to the source information of the images that interfere with the detection of cracks, such as the line of traffic signs in the road surface image.
At present, the representative methods of image de-noising include mean filtering, median filtering, Wiener filtering, low-pass filtering, high-pass filtering, and morphological filtering.The pavement image mainly contains salt-and-pepper noise and particle noise, and median filtering and mean filtering methods are simple; therefore, they are widely used in pavement image denoising.
Median filtering is a nonlinear signal processing technology based on the sequencing theory, which can effectively suppress noise.In 1D situation, 1D median filter contains odd number of slide template, the template data is in the order from small to large, and the data in the middle position is considered as the final result.The core of the median filter aids in sorting the data in the template; thus, if a bright spot (dark spot) is noise, it is ranked in the sequencing process in the right (left) side of the data sequence.Therefore, the final value of the data sequence in the middle position is generally not the value of the noise point, which can achieve the purpose of noise suppression, as shown in Figure 4. Assuming that the signal contaminated by noise is ) (t h , the original signal is ) (t  , and the noise is The mean filter formula is as follows: Taking mathematical expectations on both sides of the equation, (  is obtained.We considered the noise to be the salt-and-pepper noise, thus ) (t  ≠0.According to the above mentioned definition, for the salt-and-pepper noise, amplitude basically remains the same.However, there are some points in the data without noise, so that the output noise can be equated to the point that the noise is 0, so  .As a result of average filtering, the noise intensity value of ) (t  is lower than  , as shown in Figure 5.The median filter and mean filter also have common defects, they cannot effectively filter various noises superimposed on the pavement image.Thus, in order to overcome this shortcoming, the scholars continue to improve.Combined with the advantages of the existing denoising algorithms and characteristics of the noise and crack information of pavement image, a novel multilevel de-noising model based on the detection of pavement crack was presented by Wang et al [5].In order to remove the isolated noises in pavement crack images and protect the fine features of cracking edges, a de-noising algorithm based on the connectivity checking of pixel level and cracking subimage level was designed by Peng et al [6].The traditional image de-noising algorithm suffers from difficulty in maintaining the edge and texture features, which seriously interrupts the reliability of the detection system.To overcome this issue, an algorithm for image denoising with line-type texture based on gradient enhanced diffusion was proposed by Zhang Yongqiang [7].The noise spectrum often superposes with the signal spectrum, and the threshold de-noising losses part of the information while de-noising image; therefore, a novel image denoising algorithm based on the statistical model in contourlet transform domain was proposed by Wang et al [8].A novel pavement image de-noising method based on shearlet transform was proposed by Wu et al [9], which could smooth most of the noisy spot, but well maintain the cracks details.Owing to without regard of the noise statistical distribution, and only using Fast Fourier Transform and wavelet transform to image noise reduction bringing an image distortion, Han et al. [10] offered a method of image de-noising based on the transform domain and noise estimation.This method could completely consider the advantages of wavelet transform and Fourier Transform.Considering the image characteristics on different scales, a new image de-noising method based on ContourBougie elements morphology was proposed by Huang et al [11].Based on the non-local means filter, an improved de-noising algorithm for synthetic aperture radar images was proposed by Zheng et al [12].

Image enhancement
At present, the enhancement technology of pavement crack image includes the histogram equalization, the image subtraction method, and the image enhancement method based on prior knowledge.In order to make the pavement crack image feature better protection and more obvious crack, Cui Hua [13] adopted the Partial Differential Equation (PDE) method for preprocessing the pavement images, and improved the P-M diffusion coefficient and fused it with coherence enhancing diffusion, thus forming the new PDE model.Aiming at overcoming the shortcoming of Retinex algorithm, an improved image enhancement algorithm based on improved Retinex algorithm combined with fuzzy algorithm was proposed by Luo Rui [14], as shown in Figure 6.With the objective of improving the traditional fuzzy enhancement algorithm, a fuzzy enhancement algorithm for pavement image based on gray relational order threshold judgment was proposed by Li Ziqiang [15].To improve the uneven illumination of the background in the pavement image, Liu Yuchen [16] used a fuzzy relation method to eliminate the non-uniform background illumination.An adaptive multiplicative factor removal method of non-uniform background was proposed by Wang Xiaoming [17].Liu Yiling [18] proposed an algorithm based on gray level correction algorithm of multiplicative models with some modifications.Chen Xianqiao [19] introduced an algorithm based on difference threshold to remove the shadow from the pavement images.In order to overcome the shortcomings of the traditional image fuzzy enhancement algorithm such as small enhanced intensity and poor gray level changes in the rich image processing effect, difficult to set control parameters and other problems, a new image enhancement algorithm was proposed by Tang Lei [20].Changxia Ma [21] offered a new approach for the detection of pavement cracks based on fractional differential and wavelet transform, and experimental results proved that the proposed detection was a valid method for the different road crack image even if any noise existed.

Segmentation and recognition of pavement crack image
Pavement crack image segmentation algorithm mainly includes three methods: threshold segmentation, region segmentation, and edge detection.
Threshold segmentation is one of the earliest research and application method.The basic principle is as follows: a gray level image size is set as MN  , the gray level is L, and ( , ) f x y represents the gray level of the pixel in coordinates of ( , ) xy, where . The purpose of single threshold segmentation is to determine a threshold value, and the gray level of all pixels is mapped as follows: The segmented image contains only two types of pixel: 0 and 1  L , also known as "binaryzation", which is suitable for the situation that the distribution of target and background pixels in the two different gray level ranges, for example, the global threshold method.
For the multi-threshold segmentation, it is assumed that the threshold number is n , then the gray level of pixels is mapped as follows: where are n +1 gray levels of the segmented image.Multi-threshold segmentation method is suitable for the situation when many targets are needed to be extracted, and distributed in different gray level intervals, for example, local threshold method.For the classic example, in 1979, OTSU proposed a threshold segmentation algorithm based on gray histogram that was used widely, which divided the image into the target area and background area.A maximum entropy image segmentation method was proposed by Liu Na [22], and the adaptive threshold segmentation was proposed by Zhang Lei [23].In order to overcome the deficiency of OSTU threshold method and maximum entropy method, a threshold segmentation method based on histogram estimation and shape analysis was proposed by Xu Zhigang [24].Moreover, the experimental results showed that the performance was better than the minimum error method, the OSTU threshold method, and the maximum entropy method.
The 1986 Canny algorithm is the classic algorithm based on edge detection.Based on these classical segmentation algorithms, the later researchers constructed their own segmentation algorithms, such as Sobel, Prewitt, Kirsch, Roberts, LOG, gradient operator, and Laplasse operator were studied based on edge detection, as shown in Figure 7.According to the crack edge, a gradient may exist at various angles.Li Jinhui [25] proposed the detection of image by Sobel edge detection on templates in eight directions, combined with the noise filtering algorithm of neighbourhood weighted averaging and OSTU segmentation algorithm to segment the crack images.

-Segmentation algorithms for pavement cracks image
In recent years, some new algorithms have been developed in the field of pavement crack image recognition, which do not belong to the above mentioned three types of crack identification algorithms.For example, an asphalt pavement cracks extraction method using Fuzzy C-Means clustering (FCM) segmentation and morphology was proposed by Song Beibei [26].Xiao Xiaoming [27] presented an improved method based on cluster analysis and region growing for road detection.An automatic detection method based on binocular vision and digital image processing was proposed by Li Zhun [ 28].Xu Wei [29] presented a novel pavement crack detection method through integrating multi-features fusion and Gestalt principles.According to the data characteristics of 3D pavement cracks combined with the digital image processing technology, Yuan Mengxia [30] presented a new method for 3D pavement crack identification based on the background subtraction method.The existing algorithms exhibit lack of stability, reliability, and timeliness in large-scale applications, in particular, in wide geographical areas and varying road conditions.Therefore, in order to overcome these drawbacks, Zhang Dejin [31] proposed a new algorithm for pavement crack detection based on spatial clustering characteristics.Moreover, Li Wei [32] proposed a pavement crack detection algorithm based on 3D data and double scale clustering algorithm.Inspired by recent success on applying deep learning to computer vision and medical problems, a deep-learning based method for crack detection was proposed by Lei Zhang [33].Wang Xiaoming [34] presented a pavement crack detection method based on multi-image and multi-resolution treatment.The application of 3D imaging technique for the detection of pavement cracks was proposed by Hu Kebo [35] and Senthan Mathavan [36].Roberto Medina [37] presented a pavement crack detection method combining 2D/3D image processing techniques.

THE APPLICATION OF MACHINE VISION TECHNOLOGY IN PAVEMENT EVENNESS
At the end of last century, measuring equipment for road surface roughness in China mainly included the spirit level, the 3m ruler, continuum flatness instrument, and so on.These equipment sets have many disadvantages, such as low efficiency and slow speed.At present, road surface roughness measuring equipment used in China mainly consists of direct contact type mechanical device and intelligent road inspection vehicle.The direct contact mechanical device has the disadvantages of low detection accuracy, low detection efficiency, and bulky instrument.Intelligent road inspection vehicle has complex structure and high price, thus it is difficult to popularize it in practice.In recent years, with the development of machine vision technology, some researchers began to apply machine vision technology to road surface evenness detection, and achieved good results.For example, a method of pavement surface evenness detection based on image moire method was proposed by Ou Yangaiguo [38], as shown in Figure 8. Jiang Xin [39] presented a method of pavement surface evenness detection based on photogrammetry.For the cement concrete pavement, Ma Yukun [40] presented a laser vision measuring system for pavement evenness.Xie Jian [41] presented the method of pavement evenness detection based on machine vision, as shown in Figure 9. Zhang Xuelian [42] reported an on-line measurement method of road evenness based on weak calibration laser structured light.

THE APPLICATION OF MACHINE VISION TECHNOLOGY IN PAVEMENT COMPACTNESS
With the rapid development of China's transportation industry, the construction standards for the construction of the traffic line are getting higher and higher.The compaction quality of highway pavement is related to the quality of entire project, thus it is very important to detect the compactness of pavement.At present, the methods for detection of pavement compaction are divided into destructive and non-destructive testing methods.The destructive testing method includes cutting ring method, wax sealing method, sand replacement method, and water-filling method, etc. Non-destructive testing methods include vibration acceleration method, Rayleigh wave, the nuclear density instrument, and the vehicular compactness method, etc.However, research on application of machine vision technology on pavement compaction has rarely been reported yet.According to the Monte Carlo model, Li Xirong [43] offered a method of laser image processing of compacted soil based on Matlab Gui, as shown in Figure 10.

OUTLOOK
With the emergence of new machine vision theory and new measurement theory, digital image acquisition equipment with higher resolution, faster acquisition, better image quality, and lower cost has become more popular.In this case, machine vision technology will be more widely used in other aspects of road pavement detection, such as road marking damage of intelligent detection and detection of road congestion.Some scholars have reported the use of the machine vision technology in highway construction machinery.Moreover, based on machine vision technology, some application systems, such as road construction machinery navigation and automatic levelling, construction machinery condition monitoring, and fault diagnosis, etc. have been designed.Undeniably, the application of machine vision technology in road detection will become a signature technology.

Fig. 4 -
Fig. 4 -Median filtering of pavement imageMean filtering uses the average pixel value to replace each pixel value of the original image.