UTILIZATION OF IMAGE AND SIGNAL PROCESSING TECHNIQUES FOR ASSESSMENT OF BUILT HERITAGE CONDITION

Historical buildings represent invaluable heritage from the past and therefore their protection is a very important task. Assessment of their condition must not cause damage accumulation, thus the least possible volume removed from the structure is essential. As many historical buildings in the Czech Republic are built using sandstone that can be considered as a typical heterogeneous system, statistical signal processing is a promising approach for determination of the representative volume element (RVE) dimensions. Such calculations can be carried out on the domain of logical arrays representing binary images of the materials microstructure. This paper deals with processing of image data obtained using SEM-BSE and high resolution flatbed scanner for determination of RVE dimensions. Advanced image processing techniques are employed and results from calculation using grayscale data are presented and compared with results calculated on the basis of color input images.


Introduction
Conservation, rehabilitation and strengthening of the built heritage are all claimed by modern societies requiring development of appropriate intervention techniques [1].In order to facilitate a proper maintenance, historic buildings' condition has to be assessed on regular basis.Assessment itself must not cause damage accumulation and has to be carried out with caution [2].Therefore, in the case of material parameters required for evaluation, the least possible volume of removed sample is essential.The extracted specimen should represent the material at the level of a constructional element and thus be suitable for investigation of material properties leading to determination of representative volume element (RVE).
There are several methods for quantification of porosity and/or volume fractions of constituting phases ranging from radiography imaging to experimental methods such as mercury porosimetry.Among available approaches, image processing routines offer balanced ratio between equipment or time demands and reliability, eventually reproducibility of results.In this paper, possibility of utilization of image processing techniques for consequent spectral analysis of the specimens' microstructures is demonstrated on a set of natural stone samples commonly used in the Czech historical buildings.

Materials
In the Czech Republic, many historical buildings have been built using different types of sedimentary rocks.
In the Prague region, this involves particularly opuka stone and different types of sandstone gathered at various locations in the area.In many of the most famous Prague monuments including the Charles Bridge, sandstones of the following types were used for both architectural and decorative purposes: (1.) Petřín sandstone -hard sandstone with ferruginous matrix from Petřín quarry is a dark brown quartz-rich stone with claystone fragments as addendum (see Figure 1).The average value of bulk density is 1930 kg/m 3 and porosity is 29.2 % [3].
( .)Nehvizdy sandstone -middle grained porous sandstone from Nehvizdy quarry has horizontal layering conditioned by alternation of middle grained laminas and fine grained laminas (see Figure 2).Main component of both clasts and matrix is SiO 2 .The average value of bulk density is 1940 kg/m 3 and porosity is 24.33 % [4]. (3.) Sandstone from the Charles Bridge -light pinkishgrey hard porous arkose (type of sandstone with more than 25 % of the feldspar grains) is coarse grained sandstone from Žehrovice (see Figure 3.
The average value of bulk density is 2030 kg/m 3 and porosity is 25.16 % [3].

Methods
From the microstructural point of view, all of these materials can be described as multi-phase heterogeneous systems (composites) suitable for analysis using   statistical signal processing methods [5,6].The constituent phases are in this case harder particles, binder (matrix) and pores.In order to utilize grayscale-based statistical signal processing algorithms for RVE dimensions assessment, acquired images have to be properly segmented to set of logical arrays, where every array contains structural information related to only one of constituent phases.

Image acquisition
There are numerous methods and devices suitable for acquisition of images in sufficient quality and reso-lution for precise structural information extraction.
According to application of two different segmentation techniques presented in this paper two different data sources were selected: (1.) SEM -MIRA II LMU (Tescan, CZ) SEM device equipped with the back-scattered electron (BSE) detector.The scanning was performed with identical operating parameters to guarantee the same acquisition conditions.Samples were polished and coated by a thin layer of carbon (thickness approx.15 nm) prior to scanning to increase surface conductivity for SEM observations.The device produces 64 dpi 8-bit grayscale images (see Figure 4).
( .)High resolution flatbed scanner -to compare results from SEM imaging with data produced with commonly available device, images of other samples were captured using high resolution flatbed scanner EPSON Perfection V350 (Seiko Epson Corp., Japan) with resolution 6400 dpi, 3.75 µm pixel size and 16bit sRGB output images (Figures 1, 2, and 3).

Image processing
Several techniques have been developed for image segmentation ranging from general purpose algorithms to highly specialized single-purpose routines for specific applications.However, for successful and effective image segmentation, proper technique has to be always combined with domain's specific knowledge.Generally, image segmentation procedure encounters: i) thresholding, ii) clustering, iii) compression-based methods, iv) histogram-based methods, v) edge detection, vi) region-growing methods, vii) watershed transformation, and viii) multi-scale segmentation.Selection of segmentation algorithm or their combination depends not only on domain's properties, but also on character of the input data.

Segmentation of grayscale image data
It has been shown that for cellular metals thresholding based segmentation on grey-scale image is sufficient enough for reliable predictions of RVE dimensions and numerical homogenization routines [6].However for natural stones results of simple thresholding may not be satisfactory and more complex routines have to be applied (see Figure 5).In this paper the following briefly characterized methods have been used on grayscale image data captured in SEM: (1.) Gradient magnitude calculation -here multidimensional filter with Sobel operator is used for intensity gradient magnitude calculation to emphasise borders of objects in the image.
(2.) Morphological techniques -these operations are based on dependence of a value of given pixel on values of pixels in defined neighbourhood characterized by considered structuring element.There are two general types of morphological procedures -opening and closing.Opening is erosion (setting the value of each pixel to the minimum value of the pixels in the neighbourhood) followed by dilation (setting the value of each pixel to the maximum value of the pixels in the neighbourhood), whereas closing is dilation followed by erosion [7].More precise morphological technique is the morphological reconstruction.Here, first steps in both opening and closing of the image are followed by image reconstruction, which is conceptually repeated dilation between marker and mask image.Processing of images using reconstruction is based on connectivity concept rather than a structuring element and the routine repeats until object stability is reached, i.e. mask image does not change with increasing iterations.
(3.) Thresholding -the simplest segmentation method based on a threshold value to convert a greyscale image into a binary image [7].It is based on the premise that distinct objects in the image have different levels of brightness and that the value of brightness is the same/similar in pixels of a continuous object.In this method, determination of threshold value depends on domain's character and has significant influence on segmentation results.When threshold value is defined globally from the whole image, its initial estimation can be performed using Otsu's method [8], which is histogram shape-based image thresholding algorithm.Possible problems arising from illumination inhomogeneity may be overcome by local thresholding.Instead of having a single global threshold value, the algorithm allows threshold itself to be smoothly varied across the image.The initial value can be determined by various approaches (median value of selected neighbourhood, Bernsen method, Mid-gray method, Niblack's, Sauvola's method, etc.). ( .)Edge detection -segmentation of the image can be also performed using edge detection to identify individual elements in the image (in this case individual particles of stones) with consequent application of methods mentioned hereinbefore.Many of edge detection methods (e.g.Canny [11]) are based on intensity gradient magnitude calculation, when edges are detected at locations, where the gradient is maximal.Gradient calculation is followed by non-maximum suppression.Image is searched to determine if the gradient magnitude assumes a local maximum in the gradient direction producing logical array of edges referred to as thin edges.Then a two-level thresholding with hysteresis has to be applied in order to filter isolated pixels representing isolated local maxima.

Segmentation of colour image data
Particularly when SEM imaging device is not available for image acquisition, one can be interested in possibility of segmentation of images captured by general purpose devices that are commonly available.This approach is in this paper shown on colour image segmentation of images captured by high resolution flatbed scanner.All image segmentation methods can be applied, when full colour image is simply converted into monochromatic form at the disadvantage of loss of information during the conversion.However, on the colour images, clustering approach is promising even though computational requirements may be high according to size of input data.Here, K-means clustering is a common method of cluster analysis.To separate objects in analysed space and aggregate them into groups (clusters), K-means clustering treats each object as having a unique location in space [12].Partitions are found such that objects within each cluster are as close to each other as possible, and as far from objects in other clusters as possible.This process can be noted as minimizing within-cluster sum of squares (objective function) according to where µ i is the mean of points in S i (cluster centre).
In the demonstrated application, input parameter for the calculation is number of phases constituting the material that has to be a priori known.This number of phases then corresponds to number of clusters to be generated.According to algorithmization used in this work, image representation of the investigated material has to be transformed into CIELAB colour space derived from CIE XYZ tristimulus values.This space consists of the luminosity layer L, the chromaticity-layer A, indicating, where colour falls along the red-green axis, and the chromaticity-layer B indicating, where the colour falls along the blue-yellow axis (similarly to the human vision).All of the colour information is contained in the A and B layers and the difference between two colours can be measured using the Euclidean distance metric.Since the colour information exists only in the a * b space, objects for clustering are then pixels with A and B values.

RVE assessment
Utilization of statistical signal processing techniques on determination of typical structural periods in cellular metals was shown in [13], whereas application on natural stones was presented in [6].The method is based on spectral analysis of a segmented binary image, where large peaks in the power spectrum density (PSD) function indicate typical structural periods of the material's microstructure and hence the RVE dimensions.All calculations are created on the domain of spatial path variable ϕ (s) for the material's phase function leading to the following formulation of autocorrelation function R * xx (σ) and its corresponding PSD function where L s is the one-dimensional path length and f s is the structural frequency in m −1 , which is reciprocal of the searched structural period.Spectral estimation itself has been performed using periodogram denoted by the equation where ξ (f s , L s ) is the discrete Fourier transform (DFT) of the phase function ϕ (s) given by

Results
Hereinbefore characterized methods and tools have been applied on SEM BSE and optical scanner images in order to develop image processing procedures for phase volume fraction identification in natural stone materials as well as to show their applicability on pre-processing of image data for spectral analysis.
Concerning the SEM grayscale data, edge detection, watershed transformation and pure gradient magnitude algorithms proved unsuitable for analysis of materials with such a kind of microstructure.Edge detection approach is in this case significantly computationally demanding and fails due to indistinct grain boundaries and in some cases due to low contrast between the neighbouring elements (grains).Watershed transformation was applied in the form of Meyer implementation.When the captured images are seen as a topographic relief, it is evident that watershed algorithm is not in this case applicable.From topographic point of view, the images are in major part constituted by plateaus rather than valleys.This implies that watershed may be used only to identify pores, but still, due to complex and stochastic connectivity between grains of individual phases, the watershed algorithm fails with almost no watershed lines created, substantial oversegmentation and loss of information as a result of numerical errors.Consequently, marker-controlled watershed was tested with set of markers pre-defined by morphological operations.However, even this effort led to the same results.As a demonstration, pure gradient magnitude calculation was performed resulting in boundaries between grains generated together with noise and spurious elements, where grains are constituted of material with mixtures.Any algorithm applied on these results fails in oversegmentation showing that on such a data additional processing has to be carried out.
Hence, the following proposed image processing method for quantification of individual phase's volume fractions was developed and implemented in Matlab environment.Visual inspection of images captured in SEM shows at high magnification that considerable noise is present already during acquisition.For this reason, the images are firstly manipulated to remove noise.Among available tools, adaptive filtering using Wiener filter [14] was applied to preserve edges and possible high frequency parts of microstructure.Then, local variances in intensities (inside grains) were corrected by morphological reconstruction procedure comprising both opening-and closing-by-reconstruction processing.In the next step, custom created thresholding routine to extract information about the grey phase (quartz particles) was applied.In this moment, visual inspection is necessary to set upper and lower intensity limit of the thresholding routine.These parameters cannot be determined automatically due to two reasons: the images exhibit inhomogeneous intensity represented by different intensity of selected grains of the same material (that should be the same thanks to BSE detector) caused by the morphological reconstruction and it is necessary to extract information about only one phase implying that two level threshold is necessary.To segment into black phase (pores filled with epoxy resin) simple global threshold can be used again preferably with manually selected threshold value after visual inspection.Resulting binary images are repeatedly dilated and eroded with various structural elements to smooth the microstructure and to avoid isolated elements.Third phase (feldspar) was identified by subtraction of quartz and porous areas from the original image.In the last step, volume fractions were evaluated by summations of logical arrays of relevant phases.Segmentation to quartz particles is shown in Figure 6.
The same effort was made to identify the constituent phases in the colour image data.Images that were cropped to include only homogeneous area without oblique borders were in the first step converted from sRGB space to the LAB space suitable for cluster analysis.K-means clustering was performed with algorithm pre-set to find 3 clusters corresponding to 3 phases present in the samples.Resulting 3 images of clusters were then re-converted to RGB space and transformed to binary ones for volume fractions' computation.Moreover when the clusters were visually inspected it has been found out that such an analysis is an applicable approach for identification of dominating structural directions in the material in- dicating overall planes of layer formations originated during material's sedimentation and compaction (see Figure 7).Volume fractions determined in all the calculations were compared to results from mercury porosimetry measurements.It has been ascertained that volume fractions from image computations are generally lower due to inability of imaging devices to capture micropores in the material that significantly contribute to overall porosity.SEM BSE images proved to be preferable for image processing and phases identification as the image data are derived from atomic composition of the phases rather than their appearance in optical spectrum.As the developed image analysis routines are superior to commonly used methods, the signal processing approach to RVE dimensions calculation then yields more reliable results.Furthermore, it enables RVE dimensions to be calculated not only with respect to porosity (or its distribution), but also according to distribution of individual phases present in the materials.

Conclusion
Microstructures of heterogeneous materials extracted from historic buildings have been analysed to identify volume fractions of individual phases present in the materials, to calculate porosity and to demonstrate utilization of signal processing routines for RVE dimensions calculation.Advanced image processing methods have been applied in order to obtain high quality data for consequent signal processing analysis.Input images were captured using SEM equipped with BSE detector producing grayscale images and using high resolution flatbed scanner producing full colour sRGB images.According to colour space, different image segmentation algorithms have been developed.Obtained results have been compared to experimentally determined porosities assessed using mercury porosimetry [3,4].It has been found out that optical methods are suitable for volume fractions calculation of constituting phases.In contrary, optical porosity determination gives lower values as optical methods are not able to distinguish micropores that are significantly contributing to overall porosity.K-means clustering method applied on colour data is a promising approach for identification of dominating structural directions.Employment of advanced image segmentation routines yields more reliable data in consequent spectral analysis calculating RVE dimensions.This maximizes probability that specimen with such dimensions extracted from historical building represents the material at the level of constructional element and leads minimal intervention to the built heritage structures.However, given the stochastic nature of their microstructure, large number of samples has to be investigated for every type of stone to obtain reasonable statistics of results.

Figure 4 .
Figure 4. Arkose from the Charles Bridge (left) and Petřín sandstone (right) captured using BSE detector in SEM.

( 4 .
) Watershed transformation -for the purpose of watershed transformation, grayscale image is considered as a topographic relief defined by intensities of individual pixels.Principle of this transformation is directly derived from behaviour of water drops falling on a topographic relief.Water drop follows (flows along) gradient of a path to reach a local minimum (minimum altitude in an area enclosed by a topographical border).Then, watershed of a relief corresponds to the limits of the adjacent catchment basins of the drops of water[9,10].

Figure 5 .
Figure 5. Result of improper segmentation in grey intensity space leading to substantial loss of information with upscaled detail of over-segmented structure.

Figure 6 .
Figure 6.Output from segmentation procedure showing image suitable for structural analysis with highlighted detail of microstructure.

Figure 7 .
Figure 7. Microstructure subjected to cluster analysis with depicted dominating planes (yellow lines).