REAL-TIME MONITORING DEFORMATION OF BUILDING USING PHOTOGRAPHY DYNAMIC MONITORING SYSTEM

The spatial structure building is a type of building system; it is necessary to monitor deformation to determine its stability and robustness. Under the dynamic deformation of structures, it is challenging to determine appropriate zero image (the reference image) if we use the PST-IMMP (photograph scale transformation-image matching-motion parallax) method to obtain the deformation of structures. This paper offers the Z-MP (zero-centered motion parallax) method to solve these problems and offers PDMS (Photography Dynamic Monitoring System) based on the digital photography system to monitor the dynamic deformation of the tennis stadium located in Jinan Olympic Sports Center. The results showed that the spatial structures of the tennis stadium were robust, and the deformations were elastic and within the permissible value. Compared with the PST-IM-MP method, the Z-MP method is more suitable for deformation monitoring structures under real-time deformation. This paper indicates PDMS has advantages of the simplicity of operations, automation, and the ability of non-contact dynamic deformation monitoring for multiple points in a short period. In the future, it will have broader application prospects.


INTRODUCTION PHOTOGRAPHY DYNAMIC MONITORING SYSTEM
PDMS consists of two parts: information acquisition and information processing. The information acquisition consists of a digital camera, a distance finder and two tripods. The information processing is a computer with the photography dynamic monitoring software installed. The software can process images and output of deformation results. Figure 1 shows the view of PDMS. The view and parameters of the distance finder are shown in Figure 2 and Table 1, respectively. Bushnell PRIME 1200 range 7-1200Y accuracy ±1Y

Camera and its distortion
In this paper, we used a Sony-α350. The view and parameters of the camera are shown in Figure 3 and Table 2, respectively. Because the sensitive element of the non-metric digital camera used in this paper is CCD, the camera parameters are unknown and unstable, which leads to the instability of the imaging process, so there is a large lens distortion in the digital image. In order to reduce the interference caused by lens distortion, it is necessary to correct the lens distortion. The correction method used in this paper is a grid-based method [19]. Figure 4 (a) is the nine × nine grid used in this paper, which is printed on ISO-A0 matte paper. Figure 4 (b) is the analysis diagram of the distortion difference obtained after the analysis of the distortion difference of the grid. Through the analysis and comparison of several groups of distortion errors, the change rule of distortion difference is explored to correct the image distortion. The specific implementation process is as follows: Step 1: fix the grid on the vertical plane, adjust the digital camera mode to "manual" to ensure constant exposure in the subsequent shooting process, and set it up at a specific distance S from the grid.
Step 2: control the digital camera to shoot the grid many times and record the distance at the same time; Step 3: take the image with excellent image and the actual grid for careful comparison, through the observation of image feature points, analysis of the digital camera image distortion law; Step 4: change the distance S before and after without changing the direction, repeat steps 2 and 3.
Step 5: obtain the mathematical relation of image distortion and complete the distortion correction.

-Camera calibration
Photography dynamic monitoring software Figure 5 shows the software screenshot. The software can obtain the deformation value of multiple monitoring points through the PST-IM-MP method, the Z-MP method, and the MP method. It can also provide sufficient and reliable deformation data, finally forming user-friendly visualization results. Import the data from the information acquisition part into the computer of the information processing part, and batch process the pictures: Import the images into software and carry out the unified preprocessing of the images as the case may be, such as cutting and drawing. Image processing operations such as binarization, sharpening, and improving contrast or clarity. Input scale deformation values, and then get the pixel position and pixel correction of monitoring points and reference points in the corresponding images, pixel displacement and all other images relative to the reference image the actual displacement of the monitoring point in mm.

The zero-centered motion parallax method
In Figure 6, suppose there are three planes, image plane, reference plane, and object plane. Image plane is the image of the camera, reference plane is formed by a pair of tripods, object plane is the side of the tennis stadium. And Figure 5 shows the number and distribution of points on a reference plane and a object plane. There are six control points distributed on the reference plane, namely C0-C6, twelve monitoring points on the object plane namely U0-U12. If monitoring point on the object plane is moved from to , its deformation and on the reference plane are: Where is the principal distance of photo, where is the distance of the image plane and the reference plane. and are the horizontal and vertical deformations of monitoring point on the image plane. and are the horizontal and vertical deformations of monitoring point on the reference plane.

Correction of system errors in the Z-MP method on the reference plane
Strictly speaking, the photos before and after deformation are always taken under different elements of interior and exterior orientation, the interior orientation elements in a photo are the elements for restoring the shape of the photographic beam, the exterior orientation in a photo are the elements for determining the position and orientation of the photographic beam in the space coordinate system. In the test, we keep the camera as stable as possible and set the digital camera mode to ' manual '. Therefore, the error caused by the inconsistency of elements of interior and exterior orientation will have a slight impact on the monitoring results; we weaken the theoretical error to a certain extent by setting control points. To be exact, we weakened 0 ; the specific process is as follows.
On the reference plane, if corresponding monitoring points in the zero image and successive image are ( 1 , 1 ) and ( 2 , 2 ), compared with the ideal image which without errors of camera external and internal parameters, systematic errors of corresponding monitoring point are (ⅆ 1 , ⅆ 1 ) and (ⅆ 2 , ⅆ 2 ), respectively. The equations can be expressed as Equation (2): If there are errors of camera external and internal parameters between the zero image and successive images, the control points located on the reference plane will generate parallax and as Equation (3): The parallax and of the control point must be caused by the errors of camera external and internal parameters in successive zero images, we take parallax for example, it can be expressed as Equation (4), the detailed derivation process is shown in the reference [21]: ) ⅆ 2 + 2 2 ⅆ 2 − 2 ⅆ 2 + 2 ⅆ 2 + ⅆ 0 2 ] Where (ⅆ 2 , ⅆ 2 , ⅆ 2 , ⋯ ) and (ⅆ 1 , ⅆ 1 , ⅆ 1 , ⋯ ) are errors of zero image and successive images. From Equation (4), we notice 2 = 1 + , assume the difference between the errors of zero image and successive images as Equation (5) shows: Then, Equation (4) can be expressed as Equation (6) After sorting out Equation (6), it can be expressed as Equation (7): Article no. 18

THE CIVIL ENGINEERING JOURNAL 1-2021
Because motion 0 is caused by the change of camera external and internal parameters ( , , , , , , 0 ) in the successive and zero images, we can correct 0 with a sufficient number of control points. Differently, motion is caused by the interaction of , , and camera external and internal parameters (ⅆ 2 , ⅆ 2 , ⅆ 2 , ⅆ 2 , ⅆ 2 ) in the successive image. However, and are different at each point, control points cannot be used to correct , so we only discuss 0 as follows: We can express Equation (8) as Equation (9): 0 = + + + ⅆ 2 + ⅇ (9) If there are more than five control points, each unknown coefficient ( , , , ⅆ, ⅇ) can be obtained according to their 0 . We assume the correction of is , so the error equation is: = + + + ⅆ 2 + ⅇ − (10) For convenience, we selected the linear part of the Equation (10) for processing, as Equation (11) shows: In this case, we only need three or more reference points to obtain ( , , ) and ( ′ , ′ , ′ ).Take 0 as an example, when 0 contains only occasional errors, Equation (11) can express as Equation (12): The equation of the composition method is: Calculate barycentric coordinates by control points on the reference plane.
Because coordinates of control points are barycentric coordinates, ∑ ∑ = 0 and the parallax coefficient in the direction as Equation (16) shows: Article no. 18

THE CIVIL ENGINEERING JOURNAL 1-2021
Similarly, we can obtain the parallax coefficient ′ and ′ in the direction. Then, we can obtain 0 and 0 . Finally, we figure out the value of and , as Equation (17) shows: Obtain the deformation of monitoring points on the object plane

Zero-centered motion parallax of monitoring points on the object plane
If the structure is in a state of elastic deformation, the deformation of the monitoring structure points in all directions repeatedly changes around their respective equilibrium positions. That is, the deformation of each point is independent. If we take the first image as the zero image (the reference image), the deformation of the monitoring points is obtained based on it, independence of monitoring points will be ignored. Although obtained by monitoring the relative deformation in X and Z direction is available. However, the visual result is not user-friendly. If we calculate the 2-D deformation value of monitoring points on the X-Z plane by using the data directly, it will exaggerate 2-D deformation value monitoring points in different degrees in the X-Z plane. In other words, for the structure is in a state of elastic deformation all the time, the result of the PST-IM-MP method obtained is incorrect. Therefore, it is necessary to select an independent reference position for each monitoring point and improve the PST-IM-MP method. That is, the deformation centralization: Where ′ and ′ are the horizontal and vertical final deformations of monitoring point on the object plane. Because we obtained ′ and ′ their mean are zero, named zero-centered.

Test process
The monitoring building tennis stadium is located in Jinan, Shandong Province. Tennis stadium has a building area of about 31400m 2 . Reinforced concrete frame shear walls are used for the stands and functional rooms. The wall adopts the folded plate structure in the form of willow leaves, combined with the curtain wall steel columns of the building, to form a diamond lattice column. As Figure 8 shows, the valley depth of the folded plate structure of the outer wall is 3m and combined with the steel columns of the curtain wall to form diamond composite columns. The structure contributes to improving the stability of the folded plate structure out of plane. The height of the steel structure decreases from south to north, with the highest elevation is 31.3m, the lowest point is 21.3m, and the max span is 38.5m [22]. In the test, we used the PDMS to monitor the dynamic deformation of the southeast side of the tennis stadium.  Figure 9 (a) shows we set up a camera at a suitable position on the test field, set up a reference plane by two tripods right in front of the camera. We selected 12 points as monitoring points, as Figure 9 (b) shows. Then, we control the camera to shoot them continuously with 12 times as a group, total four groups are shot. In the process, use rangefinder obtained the relative positions of camera and object plane, use steel ruler obtained the relative positions of camera and the reference plane, and the length of reference baselines. Figure 9 (b) shows the reference points and monitoring points used in this test.

Results and analysis
Use the PST-IM-MP method to obtain the deformation of the monitoring points U0-U11, and the deformations of these monitoring points in the X direction and the Z direction, as shown in Table 3 and 4. For the convenience of the display, only two decimal places were reserved here. Use the Z-MP method to obtain the deformation of monitoring points U0-U11, as shown in Table 5 Figures 10 (a)-(c) show the deformation curve of U0 -U11 point in X and Z direction generated by the PST-IM-MP method and the Z-MP method. The Z-MP method can shift the deformation curve to the X-axis without changing the relative deformation, and shape variable of the monitoring point fluctuates up and down around the X-axis. Figure 10 (d) shows the distribution of the data generated by the two algorithms. We can see that the Z-MP method not only facilitates the intra-group comparison of the monitoring points but also makes the deformation curve more reasonable and makes preparations for the next step to obtain the 2-D deformation of the monitoring points on the X-Z plane.  Table 7 and 8 shows.   The observation of three deformation curves generated by the Z-MP method shows that the three groups of points are undergoing elastic deformation, and the monitoring structure is in a stable equilibrium state. The observation of three deformation curves generated by the Z-MP method shows that the three groups of points are undergoing elastic deformation, and the monitoring structure is in a stable equilibrium state. If U0-U3, U4-U7, and U8-U11 group the data, they correspond to the upper, lower, and middle parts of the monitoring building. By comparing the deformation values of these three groups of points, it can be found that: U0-U3 group is the largest, U8-U11 group is the second, and U4-U7 group is the smallest. Therefore, it can be determined: for the whole structure, the height of the monitoring structure is correlated positively with the deformation range on the whole. However, according to the analysis above, the deformation value and deformation interval of U9 point are the smallest, indicating that there may be abnormal deformation near U9 point, which hinders the normal elastic deformation of this point. If the deformation value and deformation interval of U1 and U3 are the largest, the nearby parts of these points are undergoing relatively large and rapid elastic deformation, prone to damage accumulation and resistance attenuation. Relevant safety maintenance personnel should find out the deformation's reasons and further refine the deformation observation of these parts to ensure the healthy and safe operation of tennis courts.

CONCLUSION
(1) If the spatial structure in a state of elastic deformation, the deformation value using the PST-IM-MP method obtained is available. However, the deformation curve by using these data generated is not user-friendly. If we obtain the 2-D deformation value of monitoring points in the X-Z plane by directly using the data, it will exaggerate the 2-D deformation in the X-Z plane of points at different levels. This paper offered the Z-MP method to solve these problems effectively. (2) This paper proposed the PDMS to use for non-contact dynamic deformation monitoring of complex structures. Moreover, PDMS is robust, easy realizing, economical, and highly automatic, can apply to amounts of complex scenarios.