AUTOMATIC CLASSIFICATION OF POINT CLOUDS FOR HIGHWAY DOCUMENTATION

Mobile laser scanning systems confirmed the capability for detailed roadway documentation. Hand in hand with enormous datasets acquired by these systems is the increase in the demands on the fast and effective processing of these datasets. The crucial part of the roadway datasets processing, as well as in many other applications, is the extraction of objects of interest from point clouds. In this work, an approach to the rough classification of mobile laser scanning data based on raster image processing techniques is presented. The developed method offers a solution for a computationally low demanding classification of the highway environment. The aim of this method is to provide a background for the easier use of more sophisticated algorithms and a specific analysis. The method is evaluated using different metrics on a 1.8km long dataset obtained by LYNX Mobile Mapper over a highway.


Introduction
Mobile laser scanning (MLS), a highly efficient tool for acquiring dense point clouds, has been accepted as a standard for the mapping of road corridors.It is applicable in areas, such as project planning, project development, construction, operations, maintenance, safety, research, and asset management [1].Most of these applications demand a complete description of the road geometry and all objects on, above or near the road surface.For an efficient use of the acquired datasets, a tool for extraction of all objects in the scene is needed.Despite the fact that the concept of the method for overall structure recognition from mobile laser scanned point clouds was presented in [2], most of the later presented works still focuses on the detection of a sole object type.Among other things, the detection of road markings from road surface points [3], detection of vertical traffic signs and extraction of its main geometric parameters [4], detection of tunnel luminaires [5] and detection of pole-like objects in urban scenes [6] can be noted.The high accuracy of these narrowly focused sophisticated methods is often related to high computational demands.However, these demands can be decreased by decreasing the number of input points.The solution can be found in the rough classification of the point cloud into several basic classes, which serve as a background for a further sophisticated analysis.

Concept
The concept of the method for a rough point cloud classification of roadway scenes is based on the following premises: (1.)The point cloud classification problem can be transformed into an image classification problem, which enables the application of techniques for the image processing of the classification and reduce the computational demands of spatial queries (much lower on raster images than on unorganized/organized point cloud).
( .)Characteristics of classified objects are known and can be used for the classification (refers to knowledge based methods). (3.)Many of the formerly presented algorithms for ground point filtering have reached a satisfying performance and the presented method can profit from their outputs (a prior knowledge of the terrain position is demanded). (4.)The basic concept of the method prefers low computational demands to absolute accuracy.

Dataset
The point cloud used for the method development and evaluation was acquired in 2009 by LYNX Mobile Mapper and it represents the highway D11 in the Czech Republic.Ten blocks of a size of 200 × 200 m were selected for the development phase and a 1.8 km long continuous strip (divided into 9 blocks) for the evaluation.Both datasets (for development and evaluation) contain objects from all classes and reflect the variability of the scenes (dense and sparse vegetation, cutting and embankment).
The Input file has a form of a text file (.txt) and contains the following information: class, x, y, z over ground , intensity, z (in this order, blank space is used as a delimiter).The method uses both the absolute altitude of a point (z) and the relative height of the point over the terrain (z over ground ) for classification, therefore, a prior knowledge of the terrain position is demanded.The estimation of the terrain position and computation of the height over the terrain were accomplished by the automatic functions of the TerraScan software.

Description of the method
The process of classification consists of three steps: the creation of images displaying selected point cloud characteristics, the classification of images and applying the results of the classification on the point cloud (Figure 1).
The presented method focuses on highway scenes outside the city and the selection of the classes corresponds to the common objects present in these scenes.Points are divided into ten classes, see Table 1.
The dataset used for the testing and detailed process of classification are described in the following paragraphs.

Point clouds into raster images
In order to arrange the easy and fast production of raster images for the classification, the data in the point cloud were divided into voxels and a matrix summarizing the information about individual voxels (later called the voxel matrix) was created to serve as an intermediate between the point cloud and raster image.The size of a voxel was 0.20 × 0.20 × 0.10 m.The density in used dataset varies between units and hundreds per square meter -the density of point clouds from the MLS steeply decreases with the distance from the scanning system.The size of the voxel was limited by the lowest density in the observed scene.The resolution in the first two dimensions correspond 01 Unclass (points, which cannot be assigned to any other class) 02 Ground 03 Roadway 04 Vehicles (temporary objects on the roadway) 05 Gates (all objects above the roadwaytoll gates, power lines, ecoducts etc.) 06 Crash-barriers 07 Poles (pole-like objects, e.g., utility poles and delineator posts) 08 Signs (vertical traffic signs and other boards) 09 Walls ((noise) walls and (wire) fences) 10 Vegetation Table 1.Classes of points.
to the former pixel size in raster images.A higher resolution leads to an abundance of empty pixels and a lower resolution causes that many pixels contain points from more than one object.The third dimension has a higher resolution to ensure the classification of low objects.
The voxel matrix is a four dimensional matrixthe first three dimensions refer to the position of individual voxels (x, y and z over ground , transformed into the voxel matrix coordinate system) and the last dimension refers to four parameters describing points, which belong to the corresponding voxel -number of points belonging to the voxel, minimal height of the point over terrain, maximal intensity (reflectivity), and minimal altitude.This voxel matrix serves as a source for the raster image creation.For instance, the raster image of potential crash barrier points is based on a number of points included in voxels between 0.5 m and 1.0 m height over terrain (based on minimal height set for crash barriers in the Czech Republic -0.75 m [7]).

Raster image classification
Objects belonging to different classes are situated in different heights over the terrain.When points are projected on a horizontal plane to create a raster image, one pixel can comprise of points from different classes (objects from different classes can have overlaps); hence the classification cannot be done on one raster image, but each class should be classified separately.
For each class, a raster image (or images), which displays the parameter that is significant for the class, is derived from the voxel matrix.These parameters are selected according to specific characteristics of the class.For instance, the roadway is a smooth surface of a low slope and it is situated near the mobile mapping system during the data capture, so the point density on the roadway is higher than on other surfaces.Therefore, the roadway class is based on a combination of three raster images, which display maximal height of points over terrain, local slope, and point density.Another example is crash barriers, whose parameters fulfil specified criteria (e.g., in height, size, position).The crash barriers class is based on an image displaying the point density in a specific height over terrain.
The created image (or combination of images) displays points potentially belonging to the selected class.The next steps have to confirm which pixels/group of pixels really belong to the class and which do not.For that purpose, image processing techniques from the following categories are applied to the image (usually in the same order, but some of them repeatedly): • image segmentation by thresholding (image binarization); • image filtering (to emphasize certain features or remove other features); • object-oriented classification (based on objects shape, size, and context).
The creation of a primary image displaying potential class pixels and the application of previously mentioned image processing techniques are repeated for each class (from 03 -roadway to 10 -vegetation) and lead to production of images representing individual classes.The class gates is represented by three raster images -the first, called gates, represents parts of objects in height 4.5 m over the terrain and higher, the others, called gate end and gate column, represent lower parts of the same objects.The resulting raster images serve for the classification of the point cloud.The process of the image creation and classification was developed and evaluated on selected datasets.Regardless how sophisticated the automatic classification is, it sometimes fails.This failure manifests itself in the form of two types of errors -FN (false negativepoint should belong to the class but it is not classified) and FP (false positive -point should not belong to the class but it is classified).The efforts made in order to decrease one error type lead usually to an increase of the other error type, therefore, a decision has to be made between minimizing FN and FP errors for every class.

Raster image into point cloud
It is clear, from the previous example, that the pixels of a same position on classified raster images can belong to more than one class.The transmission of classification results from raster images into the point cloud is realized in a way presented by the diagram in Figure 2.
The points, in the first step, are divided into two groups according to the height over the terrain -the points lower than 0.025 m over the terrain are assumed to be a part of the class roadway (in case that the point belongs to a pixel classified as the roadway class) or ground (in the opposite case).From the second group,   points are tested to be higher than 4.5 m over the terrain and incident to the gates class or not.If not, each pixel is tested to raster images of other classes, respectively to their importance.First comes the smaller objects connected to the traffic management, such as vertical traffic signs and delineator posts (signs and poles), then other objects connected to the traffic safety (crash barriers), other manmade structures (walls), vegetation and afterwards, all temporary objects (vehicles) and objects or parts of objects, which stay unclassified (unclass).The described workflow is very fast and simple, but offers a limited possibility of overlapping the objects classification.

Results
The presented method was tested on a 1.8 km long part of the D11 highway divided into nine blocks (200 × 200 m).The reference data were manually classified.The manual classification was conducted using TerraScan software -at first, tools for a semiautomatic classification were used.Then, the classified dataset was carefully inspected by a human expert and classification was manually improved when needed.The quantitative evaluation of the method is based on a comparison of an automatically (by the presented method) and manually classified dataset; it includes a confusion matrix generation (Table 2) and a summary of FN and FP errors (Table 3).Names of classes in tables are replaced by codes: 1 -unclass, 2 -ground, 3 -roadway, 4 -vehicles, 5gates, 6 -crash barriers, 7 -poles, 8 -signs, 9 -walls, 10 -vegetation.
The tested dataset consists of more than 32 million points and 94.5 % of the points were classified correctly (Figure 3).The detailed results of the classification are presented in the confusion matrix (Tab.1).The highest classification accuracy was achieved for roadway, gates, and vegetation classes (classification accuracy over 96 % and amount of both error types lower than 4 %).Despite of the simple principle of the presented method, the results achieved by this method (Table 4) are comparable to the results of more sophisticated classification methods presented in the last year five years (Table 5).

Examples of errors
Despite of the relatively high accuracy of the presented method, there were still a number of situations in which the algorithm fails.It was highly problematic to distinguish between tree trunks and utility poles.The main difference between these two objects is the tree crown; however, in dense forest stands, it is usual that the number of points describing individual trees (especially tree crowns) strongly decrease with the distance from the mobile scanning system (Figure 4).
Highly reflective surfaces (typical for vertical traffic signs) can cause multiple reflections, which appear in a form of a shadow behind the surface (Figure 5).The reflecting object, together with the shadow, creates a  complex object, which has different parameters than the original one and can be wrongly classified.
Steep slopes, terrain discontinuities and bridges typically cause difficulties for ground filtering algorithms.Classification of objects on terrain discontinuities, such as retaining walls or ends of bridges, remains problematic (Figure 6).
More details about the presented method and its evaluation can be found in [11].

Conclusions
Both the manual and automatic classification of a tested dataset were executed on a standard office computer.The manual classification lasted 19 hours while the automatic classification lasted only 54 minutes.The automatic classification achieved the overall classification accuracy of 94.5 %, however the accuracy of individual classes significantly varies.Despite the achieved accuracy of the presented method, which exceeded the expectations and did not stay far behind other methods for the point cloud classification, there are still opportunities for a further enhancement.The misclassification of tree trunks to the poles class and long vehicles to crash barriers class remains especially challenging.The basic concept of the presented method -the transfer of a point cloud classification problem into a raster classification and the execution of spatial queries on a pixel to pixel basis instead of the point to point basis -is connected to low computational demands and high efficiency, which compensate the error rate of a method given by the method principle.

Figure 1 .
Figure 1.The diagram of the presented method.

Figure 2 .
Figure 2. Classification -from raster image to point cloud.

Table 4 .
The lowest classification accuracy was achieved Completeness and correctness of proposed method.
*results from five datasets

Table 5 .
Examples of classification algorithms.