3 DMADMAC | SPECTRAL : HARDWARE AND SOFTWARE SOLUTION FOR INTEGRATED DIGITIZATION OF 3 D SHAPE , MULTISPECTRAL COLOR AND BRDF FOR CULTURAL HERITAGE DOCUMENTATION

In this article a new 3D measurement system along with the study on 3D printing technology is presented from the perspective of quality of reproduction. In the first part of the paper the 3DMADMAC|SPECTRAL system which integrates 3D shape with additional color and angular reflectance measurement capabilities is presented (see Figure 1). The shape measurement system is based on structured light projection with the use of a DLP projector. The 3D shape measurement method is based on sinusoidal fringes and Gray codes projection. Color is being measured using multispectral images with a set of interference filters to separate spectral channels. Additionally the set up includes an array of compact light sources for measuring angular reflectance based on image analysis and 3D data processing. All three components of the integrated system use the same grayscale camera as a detector. The purpose of the system is to obtain complete information about shape, color and reflectance characteristic of measured surface, especially for cultural heritage objects in order to create high quality 3D documentation. In the second part of the paper the 3D printing technology will be tested on real measured cultural heritage objects. Tests allow to assess measurement and color accuracy of reproduction by selected 3D printing technology and shed some light on how current 3D printing technology can be applied into cultural heritage.


INTRODUCTION
Preservation of cultural heritage is an important task for every modern society.Recently tools which employ new technology for scanning and digitization emerge.They facilitate precise, contactless measurement of artifacts' features such as shape, color or reflectance distribution.There are already several solutions which address this problem, for example Simon et.al. uses separate laser 3D scanner and custom made multispectral camera and develop a method of manual merging of shape and color data [1], whereas Mansouri et.al. propose an integrated system which uses structure light projection for shape measurement and multispectral camera for color acquisition [2].None of these solutions address a problem of reflectivity measurement.
An approach presented in this work is based on an integrated measurement system which is capable of automatic measurement of shape using structured light projection technique, color with the aid of multispectral acquisition and angular reflectance distribution.The presented measurement system utilizes a single CCD (charge-coupled device) detector for all above kinds of measurement, which makes it insensitive to alignment of data which come from different phases of the integrated measurement.
The measurement device along with data processing path is presented in the first part of this paper, whereas the second part is dedicated to a 3D printing technology which gives completely new opportunities in this field.It allows for creation of real copies of the previously digitized objects for promotional and educational purposes.We conclude our work with examples of cultural heritage objects from Central Office of Measures in Warsaw, Poland which were scanned in order to create their digital models and then printed as a test of copying technology and the whole data processing path.

MEASUREMENT SYSTEM
The measurement system used in this work consists of a developed 3D scanner based on structured light projection, multispectral camera and a specially designed device for angular reflectance measurement.All these components are described in more details below, along with data processing procedures.Additional subsection approaches a 3D printing procedure which allows copying of registered objects.

3D shape measurement
The 3D shape measurement method uses the (3D) Measurement with Algorithms of Directional Merging And Conversion (3DMADMAC) system [3].This method of measurement is based on a structured light technique with digital sine patterns and Gray codes projection [4].Collected images allow for calculation of absolute phase which corresponds to surface coordinates in the direction perpendicular to image plane, that is along the depth of the measurement volume.On the other hand transverse surface coordinates are possible to calculate, because in the previous process of geometrical calibration each pixel of the detector has assigned a ray which intersect the image plane in specific point.The single measurement results in a cloud of points which represents shape of the scanned surface, where every point maps to a single pixel of the detector.
The system consists of a Digital Light Projector (DLP) and a matrix detector (an industrial charged-coupled device (CCD) camera or a digital camera).Its components along with a measurement geometrical setup is shown in Figure 1.The 3DMADMAC system can be customized depending on end user requirements regarding size of measurement volume, amount of measurement points and duration of a single measurement.It also consists of a set of Software Development Kit tools which extend its functionality and automate all required measurement and data analysis algorithms.

Multispectral color measurement
Color of a measured surface is captured with a multi-spectral approach.The aim of this solution was to estimate reflectance spectrum of the measured surface in every point, so that its color can be calculated independently of illumination conditions and appearance under different light sources can be simulated.This approach is especially important in case of cultural heritage objects, because it gives meaningful information for art conservators regarding storage and display of artefacts.
The custom built camera was built to register images in 10 spectral bands with the aid of interference filters.The filter wheel is placed between the camera matrix and the lens.It has 11 slots, because additional empty window without a filter is necessary for performing shape measurement which uses the same detector (Figure 2).The lens mount is located outside the case which allows for simple lens replacement according to required measurement conditions.The multispectral capture system uses analytic calibration procedure described in previous work [5].It is based on capturing images of white reference plate for light source spectrum compensation and images of uniform background to compensate for illumination distribution and spectral filters' angular characteristic.Additionally transverse shifts of images due to positioning errors can be eliminated.

Figure 2
The concept of multispectral color measurement system and directional illuminators for BRDF measurement.

Angular reflectance measurement
Another device is employed to measure angular reflectance distribution of surface.The result is a bidirectional reflectance distribution function (BRDF) modelled with Phong parameters [6].The device comprises of a set of light sources distributed on a grid pattern and illuminating measurement volume.Each illuminator estimates lambertian source and allows directional illumination of the investigated surface (Figure 2).Pictures are captured with all illuminators turned on sequentially which gives a collection of reflectance values in the function of illumination angle and serves as a BRDF estimation which leads to Phong parameters calculation [7].

DATA PROCESSING
Raw data coming from the measurement are simply black-and-white images and a set of operations is needed to extract meaningful data from them.For clearer description they can be divided in a similar way as measurement stages because the data coming from them are for the most of the processing path independent and they are combined into concise digital model at the end.

3D shape calculation
The result of the shape measurement is a cloud of points which is a set of vertices distributed in a Cartesian space which resemble the location of surface of the measured object.In order to calculate it from measurement images a calibration data is necessary.It consists of straight lines' equations associated with every pixel of the detector, calculated in real units and a phase distribution interpolated over whole measurement volume, calculated from phase shifted sine fringes which are projected during the measurement procedure [3].A cloud of points is a result of measurement from a single direction, so in order to scan the whole surface of arbitrary shape the measurement procedure should be repeated for multiple directions.
When clouds of points are calculated the mask which can be applied to measurement images is known.It indicates which pixels from these images belong to the scanned surface.In other data processing procedures only these points are being calculated, which means that only the data associated with points in the cloud are further processed.
In the next step the clouds of points from several directions are merged in order to estimate the whole measured surface.Because the object is placed in the measurement volume manually the individual clouds have different local coordinate systems and they need to be fitted together.An automatic algorithms in 3DMADMAC system are used to transform the clouds into a single, global coordinate system.They use two steps.In the first one a coarse fitting is achieved based on local curvature or texture distribution of the overlapping parts of the clouds.The second stage uses ICP algorithms for precise fit [8].
Once clouds are fit they are filtered and smoothed to eliminate noise and ill-calculated coordinates.Afterwards they are simplified to reduce the amount of data and finally a triangulation procedure [9] is applied to create a mesh which is possible to export in VRML format and display in standard 3D imaging and manipulation software.

Color calculation.
Images acquired during measurement correspond to different narrow spectral ranges and they estimate the spectral response of the measured surface combined with the characteristic of the light source.Proposed calibration with the use of white reference plate and uniform background helps to extract reflectance spectrum [5] in 10 intervals corresponding to interference filters used in the experiment.After that data are extrapolated over whole visible spectrum by the means of spline fitting between measurement points.The spectral reflectance calculated this way for every point in the cloud is used for color estimation in XYZ coordinates according to CIE rules [10], with previously chosen illuminant D65.Further processing employs color calculation in sRGB color space.Finally overlapping data from different directions are used to equalize lightness variations which exist because of non uniform illumination distribution during measurement.
After above procedure it is possible to create texture for independently obtained triangle mesh.First the surface of the object is divided into separate areas with small variations in normal vectors directions and averaged normals are treated as independent texture mapping directions.This allows to divide color data into patches which are later mapped to different parts of the triangle mesh.This approach allows for precise texture mapping regardless of the measured shape.

BRDF calculation.
The set of images acquired with illumination from different directions allows estimation of bi-directional reflectance distribution function in every point of the surface.The fast algorithm [7] which calculates coefficients of the Phong model is used [6].It assumes that the BRDF is symmetrical and does not depend on light wavelength, but nevertheless it gives interesting results for surfaces made from different materials, as expected.
Three Phong parameters for diffuse, specular and shininess coefficients are stored as an additional texture which uses the same mapping as the color texture described previously and can be used as a material information in 3D rendering software.Data obtained this way are used for displaying in virtual environment only and does not take part in copying procedure which exploit shape and color.

3D PRINTING
There are several 3D printing methods currently available on the market.Among many properties they differ in material used to create models, precision and speed.An interesting solution is 3D Printing (3DP) technology which is the only one which offers possibility to print in color.This gives a good opportunity for copying models of cultural heritage objects, because color reproduction is an important aspect of their appearance.Therefore this method was applied and evaluated in the presented research.
3D Printing is the fastest known method of producing 3D models in the rapid prototyping technology [11].The use of the method is incremental (additive).The printing process is carried out in several stages.Initially, the created computer model is uploaded by a specialist into print management software.Then it is subdivided into hundreds of horizontal layers.The printer then applies a thin layer of powder on the ground in the working chamber with a roller.Then a layer of a color adhesive (glue) is spread on the powder surface by the printing head.A place where the cartridge is leaving the adhesive corresponds to the contours of the model in a given cross-section (layer).The procedure is shown in Figure 3.After applying all layers of the powder and glue, the printed model needs to be cleaned of non-glued plaster and to be hardened.
Color on the surface is obtained thanks to separate heads spreading glue (resin) in different base colors: cyan, magenta, yellow, black and colorless which is mixed to create desired color similarly as in traditional ink-jet printing technology.

Shape reproduction
In this study a Zcorporation ZPrinter 650 machine was used.According to the manufacturer, it can print models in 390,000 colors with a resolution of 600×540dpi.The size of the working chamber is 254mm × 381mm × 203mm.Single layer thickness lies between 0.089 and 0.102mm.Printed wall thickness may amount to 0.1mm.Many factors resulting from applied technology lead to spread in dimensions of created models.These include movement of the shaft scattering successive layers of plaster; positioning and movement of the printing head and platform in the print chamber; spreading of the glue; gravitational compaction of powder layers (a phenomenon of squash) as well as temperature and humidity.
Conducted evaluation established several guidelines for more precise printing which makes a full use of the printer's capabilities.Among the most important ones; the longest dimension of the model should be placed in the horizontal plane.For the vertical planes projection is weaker.On the other hand long elements that extend beyond the main block of the model should be placed parallel to the acting forces of gravity which may be contradictory to the previous assumption.Alternatively long items that can not be placed as suggested shall be printed with so called supports.This shows that individual decisions should be made according to specific printed shapes.The greatest distortions caused by the phenomenon of squash arise in the middle of the workspace.One way of minimizing this effect is to put the most massive elements of the model on the bottom so that the loose layer of plaster between the platform and the model was as small as possible.Other recommendations suggest that humidity and temperature in the room where the printer is placed should be appropriate (according to the manufacturer's recommendations) and fixed.Additionally visible deterioration in surface reproduction is present due to wear of printing heads.Used heads cause weaker representation of the flatness of the surface.The models are rough; one can see places where each layer was stuck.In the following application these suggestions were taken into consideration with good results.

Color reproduction
Separate aspect of printing considers color reproduction.The manufacturer does not provide any automatic profiling tool for translating color into limited color space of the 3D printer.The only guideline is a set of 729 color samples uniformly distributed in RGB color space which are prepared to print as a reference map of colors possible to reproduce by the printer.The user is supposed to compare color patches and manually choose the one that best resembles the color which he intends to reproduce.Obviously this approach will be impossible to apply when printing an object with complex texture where every point may have different color.Therefore a profiling method for the printer was proposed.It uses mentioned color patches which were measured by the spectrophotometer to establish their CIE LAB coordinates.The acquired data constitute a 3D look up table (LUT) which provides transformation between RGB values sent to printer and device independent color space.The problem to solve is to transform an arbitrary LAB coordinates, not necessarily existing in the LUT into the printer color space (red ball in Figure 4).The proposed solution first finds the nearest LAB coordinates which belong to the LUT and its immediate neighbours (blue balls).All of found coordinate sets have their representations in printers RGB color space which are used to calculate unknown RGB coordinates as their weighted average.The weights are established as inverse of Cartesian distances between the required out of LUT LAB coordinates and its nearest neighbours in LAB color space.

Figure 4
Construction of 3D printer color profile.Blue balls represent adjacent points in look up table between LAB and RGB color spaces, whereas red ball stands for out of LUT point which RGB coordinates are to be established.
Performed analysis shows that suggested gamut mapping compresses the input color gamut which is usually larger to fit into the printer gamut by shifting out of gamut colors to the boundaries of the smaller gamut.
Additionally conducted tests show that final color appearance is very much influenced by after printing treatment which include hardening the surface with resin and drying up the model.The acceptable repeatability is very hard to achieve because the process is made by hand and it would require very skilful operator.This shows that differences in achieved color can be considerable and even an application of a color profile may not produce good results in the face of significant technological inaccuracy.

DIGITIZATION RESULTS
Courtesy of Central Office of Measures in Warsaw three test objects were investigated in order to create their digital models and copies with the aid of 3D printing technology.The original objects are shown in Figure 5, followed by Table 1 where their overall characteristics are provided.They were made from different materials and have different surface finish.The wooden volume reference cup is lacquered, the occasional medal is polished and the iron weight reference has worn, rusty surface.Figure 6 shows virtual models of the digitized objects which were rendered in custom made software.The program allows for displaying color texture and reflectance properties as well as changing illumination and object's location with real time refreshing of its appearance.The last step presented step concerns copying the measured objects.Results of this procedure are shown in Figure 7. However shape in all the cases is preserved well, the color reproduction is poor because of limited gamut of the 3D printer and complicated hand made post printing treatment of copies.These results show that improvements can be made in color processing procedures as well as in printing technology.

CONCLUSIONS
The integrated measurement system presented in this work facilitates digitization of cultural heritage objects.It performs automatic measurement of 3D shape, multispectral color and surface reflectivity with the aid of a single detector, therefore no manual data alignment is necessary.It can be applied to create precise digital models for the purpose of display or storing information about precious artefacts.Moreover it utilizes a 3D printing technology to build copies of the measured objects for promotion or education.As a result of conducted research sample measurements outcome is provided to show the possibilities of presented technology.However shape and color measurement techniques are already well developed there are still issues concerning unbiased merging data from different directions, precision and repeatability of color measurement procedure and printing technology which allows for objects reproduction with limited precision.Particularly 3D printer's color mapping is questionable.This gives a potential for further research, but even know it is very promising technology.

Figure 1
Figure 1 3D shape measurement system concept.

Figure 3 A
Figure 3 A procedure of 3D printing.

Figure 5
Figure 5 Measured objects: a) wooden cup serving as an old volume reference; b) occasional medal; c) old weight reference.

Figure 6
Figure 6 Models of measured objects: a) volume reference cup; b) occasional medal; c) weight reference.

Figure 7
Figure 7 Printed copies of the measured objects: a) volume reference cup; b) occasional medal; c) weight reference.

Table 1
Measured objects properties.