Acta Polytechnica

The paper deals with the possibilities and limitations of the image processing of digitalized terrestrial photographs from analogue metric cameras. By the end of the 20th century, analogue metric cameras such as Zeiss UMK were used for documentation purposes in the fields of industrial and investment construction, nature and landscape preservation, heritage preservation, etc. Currently, the collections of photographs are stored at specialized archives of many different institutions, such as libraries, museums, universities, etc. These sets of photographs provide a material background for a 3D digital reconstruction of a subject of interest at the time of taking the photographs. The digital image processing of old photographs may be more difficult due to unknown parameters of the used camera, such as the focal length, image coordinates of the fiducial marks and distortion parameters of the lens system, etc. In this case, it may be difficult to process these photographs in a photogrammetric software. The paper presents a methodology for the digital photogrammetric processing of analogue terrestrial photographs. The data processing is based on the parameters of the used metric cameras, which are described in their calibration reports. The image processing was tested in two commercial photogrammetric software tools that utilize the technology of Structure-from-Motion (SfM) or multiimage intersection photogrammetry to process image datasets.


Introduction
At present, collections of historical photographs are stored in many specialized archives of many institutions, such as libraries, museums, universities, heritage institutions or private collections. These analogue photographs represent valuable sources of informationspatial and temporal data. The current trend is the digitisation of archival funds in order to protect them and make them available to the professional and lay public, so digitised analogue photographs can be reevaluated in photogrammetric software. This metric survey documentation is able to provide a digital reconstruction of an object of interest at the time of taking the photographs. Consequently, it can be reinterpreted in the context of current knowledge and the subject of a research.
Historical photographs can be divided into two basic groups according to the type and manner of acquisition: (1.) photographs from metric cameras: (a) aerial survey photographs, (b) terrestrial photographs, (2.) photographs from non-metric cameras.
Aerial survey photographs were often used for military mapping purposes. Therefore, they are valuable sources of information for the study of a landscape development. In such cases, they can be used as map bases for orthophotos and digital terrain models (DTM) at the time of the aerial photography [1][2][3][4][5][6]. These outputs can be used as separate data layers in Geographic Information Systems (GIS).
Terrestrial photographs were often used for the documentation in the field of industrial and investment construction, nature and landscape preservation, heritage preservation, etc. In such cases, they can be used as bases for a 3D digital reconstruction of a historic building at the time of taking the photographs. Historical objects may be e.g. buildings [7][8][9], sculptures, paintings [10], archaeological sites [11,12], natural sites [13], etc.
Photographs from non-metric cameras were often taken for photo-documentation in the field of heritage preservation or as art-photography. Mostly, these are separate, lower-quality-photographs. Nevertheless, they can be a valuable data source [4,8,9,11].
The digital processing of historical photographs may be more difficult due to unknown elements of vol.
no. / 3D Digital Reconstruction and Archived Terrestrial Photographs the internal orientation 1 of the used camera, since a camera calibration report is unavailable in most cases. Furthermore, the image processing may have other limitations, such as poor image quality, low resolution, inappropriate configuration of photographs, "high noise" 2 , etc. This issue is sometimes referred to as historical photogrammetry [6,8,[10][11][12]. This paper is focused on the processing of digitized terrestrial photographs from metric cameras, such as the Zeiss Universal Metric Cameras (UMK) and the TAN photo-theodolite, which is based on the calibration reports of the used cameras. Furthermore, this paper presents a methodology according to the internal parameters of the metric camera that can be used in the image processing in the current photogrammetric systems. Currently, it is common practice that the issue of identifying the elements of the internal orientation is neglected in the processing of digitized photographs and the issue is left to solve by the self-calibration process in the used photogrammetric system [1][2][3][4][5][6][9][10][11][12].
The current photogrammetric systems are often based on computer vision techniques, which work with algorithms like SfM (Structure from Motion), SGM (Semi-Global Matching), SIFT (Scale Invariant Feature Transform) or RANSAC (Random Sample Consensus). The aim of this paper is not to describe these algorithms, as the principles of the automatic scene reconstruction from images are well described in other literature, e.g. [15,16].
In this case, the image processing of digitized terrestrial photographs was tested in commercial photogrammetric software -Agisoft PhotoScan Professional, v. 1.3.3 [17] and PhotoModeler UAS, build 2017.0.2 [18]. These software tools have been chosen because authors have a long experience with their use, and both software solutions use Brown's distortion model [17,19,20] to process the image data.
The paper describes an evaluation of collections of terrestrial photographs that document the mainfacade of the "Post Office Palace", currently the seat of the Ministry of Transport and Construction of the Slovak Republic, in Liberty Square in Bratislava, Slovakia, see Fig. 1

Datasets and methodology
The workflow can be divided into several separate steps: (1.) Data acquisition: (a) archival research and digitization of archived documents (b) detailed survey (in situ) -measurement of Ground Control Points (GCPs) (2.) Pre-processing of image datasets: (a) selection of image datasets (b) transformation of image datasets into an image coordinate system (c) editing and unification of parameters of imagedatasets (3.) Processing of image datasets [16]: (a) Key-points detection in the surface texture of the object (b) Matching -pairing of corresponding key-points in different images (c) Relative orientation of the cameras within the scene (d) Bundle adjustment -adjusting the parameters of interior and relative orientation (resulting in a sparse point cloud of tie points) (e) Georeferencing -adding information about GCPs or scales -solving absolute orientation with possible adjustment of interior and exterior orientation parameters based on GCPs (f) Dense point cloud computation -based on MVS (multi-view stereo) or pair-wise stereo image matching techniques   The subjects of digitisation were black and white negatives and positives of terrestrial photographs on glass plates and film foils, as well as associated documents -calibration reports, sketches, technical reports, etc. The terrestrial photographs were taken   The scanning was carried out to a high-quality JPEG format, i.e. Q-factor = 100 %. The JPEG format was chosen to save space in a data repository. The average saving between high-quality JPEG and TIFF with LZW compression is over 50 MB per photo. The use of high-quality JPEG files should have a no significant impact upon the quality of the Digital Surface Model (DSM), following [5,23].
The main facade of the "Post Office Palace" was documented using Zeiss UMK and TAN within the pedagogical activities of the Department of Surveying during the 1970s, see Tab. 1. Due to the fact that the appearance of the "Post Office Palace" had not changed significantly, it was possible to distinguish ground control points (GCPs) on its main facade, see Fig. 2. The GCPs were surveyed by a total station, Leica TCR 407 Power [24], in a local coordinate system. In contrast to [8], it would be inappropriate to use terrestrial laser scanning. In this case, the surroundings in front of the "Post Office Palace" had changed significantly, compare Fig. 1 and Fig. 2. Currently, the front view is blocked by tall trees. By the total station, it was possible to measure the GCPs selectively from a one survey station.
The GCP-locations, ie. X, Y, Z, were calculated by the spatial polar method. The a priori precision of the GCP-locations was estimated by knowing the propagation of errors in the measurement and the accuracy of survey [24,25] and its value was estimated to 0.01 m.

Pre-processing of image-datasets
Preprocessing of the data was used to select images into sets to be processed, see Tab. 1. In the case of terrestrial photogrammetry, it was customary to have a single image taken several times, most often three times with different exposure times. Photographing in stereo pairs was also usual.
Digitised terrestrial photographs can be transformed into an image coordinate system, which is defined in the calibration report of a metric camera by means of coordinates of fiducial marks, see Fig. 3 and Fig. 4. It is appropriate to use the plane affine transformation. For a point P in the source system, the XY coordinates in the target system are given: (1) where the parameters X 0 , Y 0 define the displacement of the origin, α is the rotation angle, β is the shearing angle between the axes and m X , m Y are the scaling factors for x, y [19].
Transforming terrestrial photographs into an image coordinate system can become a complex task in two cases [3]: (1.) The image coordinates of the fiducial marks are unavailable or the calibration protocol of the metric camera is missing.
(2.) Fiducial marks are not displayed on the photoframes.
In the first case, the image coordinates can be read directly from the photographs at a known DPI-value. The effective area of terrestrial photographs is approximately 11 cm × 16 cm [26], the coordinates of fiducial marks are half the values, the origin of the image coordinate system is at the centre of a photograph.
In the second case, any point on the photo-frame can be used in the same way as identical point. These "identical" points need to be stable when switching from photo to photo and well-identifiable on all photoframes. This case may be with Zeiss UMKs when . Interior orientation in a metric camera [19].
the illumination of the fiducial marks has not been switched on at the time of photography, see Fig. 3.
Digitized photographs taken with the same metric camera should have a unified resolution and dimensions. The digitized photographs may have a different size that matches the direction of the scanner-head movement. A unified image resolution can be achieved by cropping, scaling, or both. This step is necessary for an automated processing.
It may be advantageous to use GIS software to preprocess image datasets. The GIS software support raster-data-processing and has tools for coordinate transformations, such as plane affine transformation and editing, such as cropping, scaling, mosaicking of image datasets etc. In this case, the ArcMap, v. 10.6, software was used because it contains these tools [27].

Processing of image datasets
As already mentioned in the Introduction, Chap. 1, PhotoScan and PhotoModeler software were used for the image-processing simultaneously.
The PhotoScan [17] software is one of the first user friendly solutions for an automated 3D image modeling, which has enabled the creation of professional results also to non-photogrammetric educated users [16]. Currently, a software version 1.4.4 is available to support a detection of fiducial marks on photos. At the beginning of our project, September -November 2017, the software version 1.4.0 (preview release) was available. We tested this software version, but its support for the detection of fiducial marks was unsatisfactory for us at the time. Therefore, the software version 1.3.3 was used and the workflow from the Chap. 2.2 was applied during the preprocessing of the imagedata.
The PhotoModeler [18] software has a long history of various professional applications. It supports not only the automated production of dense point clouds, but also multi-image intersection photogrammetry and videogrammetry with the support of coded targets. Also, it supports many transformations and various settings [16]. Currently, PhotoModeler Standard and PhotoModeler Premium are available. At the beginning of our project, September -November 2017, the older software version of PhotoModeler Standard was available. This software version could be extended with a module for a dense point cloud creation, so-called PhotoModeler Scanner, and then a module supporting a data processing from unmanned aerial vehicles (UAV), so-called PhotoModeler UAS. The software version of the PhotoModeler UAS 3 was used because it was the most recent version of Pho-toModeler and we wanted the SfM-algorithms and detailed point cloud generation to be as up-to-date as possible.

Camera calibration
A camera can be modelled as a spatial system that consists of a planar imaging area (film or electronic sensor) and a lens with its perspective centre. The parameters of the interior orientation are [19], see The parameters of interior orientation are determined by the camera calibration and should be written in a camera calibration report. If these parameters are given, the imaging vector x' can be defined: where x p , y p are the measured coordinates of the image point P , x 0 , y 0 are the coordinates of the principal point H and ∆x , ∆y are the correction values for errors in the image plane. At present, the application of close-range photogrammetry uses Brown's distortion model [20]: where K 1 , K 2 , K 3 are coefficients of the radial lens distortion, P 1 , P 2 , P 3 are the coefficients of the tangential distortion andx In the case of metric cameras, the dominant influence of radial lens distortion is assumed. The effects of tangential distortion are neglected because metric cameras have high-quality lenses. Furthermore, the affinity is compensated by plane affine transformation to fiducial marks, see Eq. 1. Therefore, Eq. 3 can be edited: In the case of the Zeiss metric cameras, the values of the radial lens distortion were determined by the coordinate axes of the image coordinate system available from the calibration report. The coefficients

Radius Distortion at axes [mm]
−x +x −y +y  of the radial lens distortion, i.e. K 1 , K 2 , K 3 , can be estimated by the Least Squares Method (LSM). This article deals in detail with the UMK 10/1319, see Tab. 2. LMS can be described as follows: whereX is the vector of adjusted unknowns, L is the vector of observations, A is the Jacobian matrix,v is the vector of residuals andL is the vector of adjusted observations. The other results of LSM are graphs showing the course of the radial lens distortion, see Fig. 5.

Aligning photos
Twelve variants of calculations were tested for the data-processing, see Tab TAN 19/1318. For the variants no. 5, 10 and 12, a two-step calibration process was applied in relation to the previous versions, as in [28].
The two above mentioned software applications were used as the georeferencing method -manually marking of GCPs in the photos. For image-processing, different settings were used: (1.) in PhotoScan software; for a detailed description see [17]: • Accuracy: High • Pair preselection: Generic • Key point limit: 1 000 000 4 • Tie point limit: 0 (2.) in PhotoModeler software; for a detailed description see "Help" menu: • Point density: High • Good Overlap Count 5 : 9 • Match quality threshold: 0.5

3D Modeling
The PhotoScan and PhotoModeler software allow creating and visualizing dense point clouds. Dense point clouds can be edited in their environments or exported to external software for further analysis.
The PhotoScan software showed better results when calculating dense point clouds -the point cloud was denser and more compact than PhotoModeler showed. Therefore, the digital surface model and orthophoto mosaic were processed in the PhotoScan software. An advanced processing of the dense point cloud, i.e. editing, such as filtering and removing noise, analysing and building of a mesh, was made externally in the CloudCompare, v. 2.6.3, software [29]. An advanced editing of the mesh was made externally in the Geomagic Studio, v. 2014.3.0.0, software.

Results
The outputs can be divided into several categories, which may vary according to the used software: (1.) Elements of exterior and interior orientation, 4 The default value is set to 40 000. PhotoScan uses a 35 mm "full-frame" sensor as a standard. While maintaining the default value of the density of points per photo, the value of 1 000 000 corresponds approximately to the image format of Zeiss metric cameras, i.e. 13 cm × 18 cm. 5 Point count threshold to consider a pair of photos as "overlapping" and hence good candidates for full matching.

Elements of exterior and interior orientation
The results of the photo alignment are the elements of the exterior orientation 6 of the terrestrial photographs, see Fig. 6 and Fig. 7 [19]:  Table 3 summarizes the results of the bundle adjustment and the subsequent optimization. Firstly, it was found that the Level of Accuracy (LoA) is influenced by the count and configuration of the terrestrial photographs. Several photos from one camera station do not usually bring new spatial information, so variants No. 2, 4, 7 and 9 mostly achieved better LoAs than variants No. 1, 3, 6 ans 8. Secondly, it was found that the transformation of the terrestrial photographs into the UMK image coordinate system, see Chap. 2.2, increased the LoA of the outputs, cf. variants No. 2 and 4 with No. 7 and 9. Thirdly, it was found that the use of the self-calibration process was strongly demonstrated in the PhotoScan software, where inappropriate camera parameters, especially the values of the radial lens distortion, were estimated, see Tab. 5. In the PhotoModeler software, the self-calibration process usually collapsed, so simplifications were applied for variants No. 1, 2 and No. 6, 7 -the principal point is at the centre of the photo (M ≈ H ); the radial lens distortion is negligible (K 1 = K 2 = K 3 = 0). Finally, it was found that the best results are achieved by variants No. 7, No. 9, No. 10 and No. 12 in both software tools. Table 4 compares the final elements of the exterior orientation, which are derived from the bundle adjustment and the subsequent optimization. From Tab. 4, the systematic shift of the camera location is evident, on average 0.35 m. From the comparison of Tab. 4 and Tab. 3, is evident that one set of elements of the exterior orientation 7 showed significant differences in most of the calculation variants, and therefore, it was excluded from the evaluation as an outlier. Exceptions are variants No. 5 and No. 12, in which this situation has not occurred. Different outputs are probably due to the use of various computational algorithms in the PhotoScan and PhotoModeler software, see Chap. 1. Table 5   the interior orientation 8 of the UMK are mistakenly estimated when the bundle adjustment was optimizing, unlike [13]. The smallest differences between the calibration protocol and the self-calibration process are achieved in variants No. 10 and No. 12, where the two-step calibration process was applied.

3D outputs
The basic 3D output was a dense point cloud of the main facade of the "Post Office Palace". Since the point cloud can be deformed [7], its analysis was performed. The quality of point clouds can be estimated by a comparison with a reference base. A mesh from ground laser scanning [30] can be used, for example, as a reference base. In this case, it was not possible to apply ground laser scanning, see Chap. 2.1, and reference planes were used -vertical and general, which were approximated by GCPs. From Figure 8, it is evident that the main facade of the "Post Office Palace" does not deviate from the vertical flat significantly.
In the CloudCompare software, the dense point clouds were cleaned from noise and then compared to the vertical reference flat. According to this analysis, it was found that the dense point cloud of the variant No. 12 shows the highest density and the smallest deviations from the vertical reference flat than the dense point clouds of of the variants No. 7, No. 9 and No. 10. Since the variant No. 12 shows better results, other outputs follow it. Furthermore, it was possible to determine the spatial arrangement of the facade, see Fig. 9, and to detect deviations of the dense point cloud from the reference flat, see Fig. 10.
From Figure 10, it is seen that the point cloud has the highest deviations at the outer thirds of the main facade of the "Post Office Palace". This is probably due to the configuration of the cameras, see Fig. 6 and Fig. 7, when the most of them are focused on the centre of the main facade. Therefore, the corners of the       building are displayed at the edges of terrestrial photographs, where their projection is more affected by the radial lens distortion and where it is not possible to distinguish GPCs, see Chap. 2.1. Table 6 summarizes the results of the filtering of the point cloud, depending on the distance from the vertical reference flat. From Table 6, it is clear that the dense point cloud shows the greatest density at a ±0.20 m interval from the reference flat. At an interval of ±0.10 m, the dense point cloud has a higher density than the density at the ±1.50 m interval. At an interval of ±0.05 m, the dense point cloud has a higher density than the density of the starting dense point cloud. Since the average reference scale 9 of the captured terrestrial photographs is 1 : 723.6, this output will be best used at reference scales of 1 : 1000 and 1 : 500, in which ±0.10 m and ±0.05 m intervals correspond to the line thickness of 0.2 mm 10 . That is why the deviations from the reference flat can be neglected, while the appropriate Level of Detail (LoD) of the metric survey documentation is maintained, see [31,32].
As another 3D output, a textured Digital Surface Model of the main facade of the "Post Office Palace" was processed, see Fig. 11.

Orthophoto mosaic
The last type of outputs are image data, i.e. the Orthophoto Mosaic with a resolution of 1 cm and the Depth Map with a resolution of 3 cm, see Fig. 12 and Fig. 13. Their final geometric and image quality are dependent on the underlying data, i.e. terrestrial 9 Reference scale was estimated based on the pixel size by the scanning and Ground Sample Distance (GSD), which was found in the processing of image-datasets.  Table 7. Analysis of geometric accuracy of the Orthophoto mosaic.

GSD
photographs and DSM. As expected, in relation to Figure 2 and 12, the image outputs show poorer geometric and image quality in the vicinity of GCPs -PST08 and PST09. The geometric accuracy could be estimated using check distances that were measured between GCPs, see Tab. 7. For a reference scale of 1 : 500, the final RMSE and the maximum distance-difference did not exceed a line thickness of 0.2 mm.

Conclusions
This article deals with the processing of digitised terrestrial photographs from Zeiss metric camerasthe UMKs and the TAN photo-theodolite. The data processing was based on the parameters of the used metric cameras, which are described in their calibration reports. Subsequently, the processing was tested in the commercial photogrammetric software -Photo-Scan [17] and PhotoModeler [18].
On the basis of the literature research, the methodology for the processing of digitised terrestrial photographs was developed and presented. This methodology was subsequently tested in the processing of archival terrestrial photographs of the "Post Office Palace" in Liberty Square in Bratislava, Slovakia, see Fig. 1 and [21].
Several variants of calculations with different types of input data and calculation settings were tested for the photo alignment. The results were evaluated according to RMSEs of GCPs, see Tab. 3, and elements of the exterior orientation, see Tab. 4. It turned out that the plane affine transformation of the photographs of the fiducial marks increases the accuracy of the photo alignment. Furthermore, it has been shown that using the camera calibration report, if available, increases the accuracy of photo alignment. Further, the accuracy of photo alignment can be increased by applying the two-step calibration process. The image processing was tested in two commercial photogrammetric software tools that use different computational algorithms. The testing has confirmed that their results may differ. In this case, the deviations  This study case compared the results of the selfcalibration process with the calibration report of UMK 10/1318, see Tab. 5. Unlike [13], in this case, the elements of the interior orientation of the camera have been erroneously estimated in the optimization of the photo alignment, i.e. in PhotoScan, or the self-calibration process has often collapsed, i.e. in PhotoModeler. Therefore, in cases such as this -few photos with similar view axes, it is not preferable to use the self-calibration process. Instead, it may be preferable to apply simplifications based on generally known parameters of metric cameras, see e.g. [19,26]. Then, a self-calibration process, a so-called two-step calibration, can be applied, resulting in more accurate results [28].
The basic 3D output was a dense point cloud. The analysis of the dense point cloud was carried out in connection with the literature research. The object of the analysis was the distance of the point cloud from the vertical reference flat, see Tab. 6. According to this analysis, it was possible to determine the spatial arrangement of the main facade of the "Post Office Palace", see Fig. 9, and to detect the deviations of the point cloud from the reference flat, see Fig. 10. The LoD of the analysed point cloud is adequate for reference scales of 1 : 1000 and 1 : 500. The LoD and LoA of raster outputs -the Orthophoto Mosaic and the Depth Map -are adequate for a reference scale of 1 : 500.
The final outputs are a dense point cloud, textured DSM, Orthophoto Mosaic and Depth Map. These outputs can be valuable and useful bases for further advanced studies. Raster outputs can also be used as separate data layers in GIS. The dense point cloud and textured DSM can also be used as bases for Computer-Aided Design (CAD) or Building Information Modeling (BIM).