Recognition of One Class of Surfaces From Structured Point Cloud

This paper presents a method for recognition of surfaces represented by elliptical segments in structured three dimensional (3D) point clouds. The method is based on direct least squares fitting of ellipses in scanned lines. By recognizing elliptical segments in both directions of structured cloud it is possible to efficiently allocate G1 (and higher) continuous regions which represent a certain class of surfaces. The proposed method is primarily developed for recognition of elliptical cylinders and ellipsoids, including cylinders and spheres. Depending on scanning mode, the method can be employed for recognition of other second degree surfaces like cones. Besides, as presented in the paper, the method can be utilized for recognition of certain class of higher degree surfaces such as elliptical tori. The proposed method is experimentally verified using several synthesized point clouds as well as using a real world case study.


INTRODUCTION
Three dimensional (3D) scanning devices and reverse engineering (RE) of geometric models are traditionally utilized during design phase of product life cycle.However, recent developments in the field and performances of the state of the art 3D scanning devices, and especially optical ones, paved the way for their implementation in on-line manufacturing processes control and inspection.3D data acquisition systems are employed for example in 3D navigation of mobile robots [1], automation of welding processes [2], inspection [3] etc.
Generally, RE of geometric models is carried out through the following phases [4]: 1) object scanning, 2) preprocessing of scanned data, 3) recognition of basic geometric primitives, and 4) reconstruction of scanned object in CAD system using obtained information (creation of scanned object CAD model).State of the art 3D scanning devices are characterized by high speed, high resolution and accuracy and provide dense point clouds which contain abundant information about scanned object surfaces [5].There exist fast and reliable algorithms for raw 3D data preprocessing (registration, integration, and meshing) and generation of 3D triangular mesh from point cloud is a standard feature of CAD systems.However, although with aesthetically appropriate shape, triangular meshes do not contain all necessary information about scanned surfaces' shape and structure, especially information about geometric primitives which build surfaces.On the other hand, recognition of geometric primitives may be very important in scanned object remanufacturing especially when remanufacturing is carried out by machining.In addition, when 3D scanning devices are used for feedback in control tasks, adequate recognition of geometric primitives from point cloud is crucial for the performances of control systems.
The process of geometric primitives' recognition has the following steps [4]: 1) segmentation of point cloud into the areas (clusters) which can be represented by basic geometric primitives, and 2) recognition of geometric primitives and estimation of their parameters -surface fitting.The recognition of geometric primitives in standard CAD software is as a rule carried out interactively.The user has to recognize primitives in the cloud and annotate some points from the cloud that belong to them.Afterwards, software can estimate primitives' parameters.However, implementation of 3D scanning devices in on-line process control requires automatic retrieval of geometry information.Unfortunately, efficient real-time applicable algorithms for fully automatic recognition of geometric primitives from point cloud are still an open research area.
Certainly, one of the most challenging issues in recognition of objects' structures is point clouds segmentation.Segmentation implies detection and segregation of points from point cloud.These isolated points represent some patches which are (according to their geometrical characteristics) different from the rest of scanned surface.After segmentation process, a number of different techniques for object surfaces fitting can be used [6], [7].Surfaces reconstruction techniques enable creation of mathematical and computer model of scanned object's surfaces based on distinguished group of points.
In the mechanical engineering the most of objects are built from planes, second degree surfaces -so-called quadrics (cylinder, sphere, cone, ellipsoid, hyperboloid etc.) and tori.In the focus of this paper is segmentation of G1 continuous surfaces that are represented in scanned lines by elliptical segments.This group of surfaces includes quadrics -cylinder and sphere (natural quadrics) and elliptical cylinder and ellipsoid (general quadrics) which are not frequently met in practice.Besides, depending on the scanning procedure, some other quadrics (e.g., cones) can be represented by elliptical segments in scanned lines.Also, the procedure can be employed for recognition of elliptic tori from structured point cloud.
It is important to emphasize that segmentation of surfaces with G0 continuity is less demanding than segmentation of G1 continuous surfaces (boundaries of G0 continuous surfaces are easily distinguishable).On the other hand, when the segmentation of smooth surfaces (with G1 or higher continuity) is needed, problem becomes more difficult.
According to [8] there are two basic groups of techniques for segmentation of quadrics from point clouds: 1) techniques based on edge detection and 2) techniques based on regions.In the edge-based techniques, the cloud's points which represent surface boundaries are first detected.Afterwards, using fitting procedures it's straightforward to recognize point cloud segments (patches) which are bounded by detected edges.These methods are efficient for segmentation of G0 continuous surfaces, but in the case of G1 (or higher) continuity they do not provide appropriate results (the boundaries are not represented by abrupt changes in the signal and it is hard to detect them).It should be noted that a number of objects in mechanical engineering have surfaces which are built of G1 continuous segments usually as a result of cutting tool radius.
Region based segmentation of point clouds is carried out using split and merge or region growing approach.Region growing based algorithms start from one characteristic point (seed point) from which the region expands based on some geometric criteria.Criteria for region growing can be different, but usually they are based on differential geometry parameters.For example, Hou-Chuan Lai et al. in [9] use features of differential geometry of surfaces for evaluating curvature values in each point of triangular mesh.These curvature values represent criterion for clusters creation.In [10] authors use smoothness constraint which finds smoothly connected areas in point clouds.They use only local surface normal vectors and point connectivity.Vančo, M. et al. in [11] showed that points' normal vectors do not provide enough information for segmentation and recognize some of quadrics like cylinders and cones especially when transitions between surfaces are C1continuous.In their segmentation algorithm they use top-down region growing method based on principal curvatures criterion.
The results of region growing based segmentation are in the most cases highly dependant on the choice of seed point.As a rule seed point is selected manually.Alternatively, seed point can be selected randomly or near estimated edges [12].In addition, instead of using single seed point, some authors [13] start region growing utilizing seed region.In that case, region growing algorithm is based on bicubic Bézier surface properties.
An interesting region growing approach is shown in [14].An automatically chosen segment of one of structured point cloud's lines is used for seed.Region growing method is employed for merging appropriate scanned lines' segments into 3D clusters which represent initial (seed) surfaces (planes and quadrics).For region growing criterion properties of co-planarity are utilized.
This paper presents a method for recognition of one class of quadrics (elliptic cylinder and ellipsoid) and elliptic tori from structured point clouds.Considered surfaces can have G1 or higher continuity.The technique is based on segmentation of elliptical regions from scanned lines using region growing method which is based on direct least squares fitting of ellipses [15].Extrapolation of recognition of elliptical segments from scanned lines to two dimensions [16] leads to the method for recognition of class of surfaces that are considered in this paper.The proposed method is independent of seed selection, and the seed point can be selected randomly.
The rest of the paper is organized as follows.In Section 2 we provide basic background for segmentation of elliptical segments from scanned lines which is essential for the proposed method for recognition of surfaces.Section 3 presents the proposed method and its implementation in recognition of elliptical cylinders, ellipsoids and tori from synthesized point clouds.In the Section 4 the method is tested using a real world example, while in Section 5 we provide concluding remarks and future work guidelines.

SEGMENTATION OF ELLIPTICAL REGIONS FROM SCANNED LINES
In the essence of the proposed method for recognition of considered surfaces from structured 3D point cloud is the segmentation of elliptical regions from scanned lines.In this section we will present the employed method for ellipse segmentation as well as its implementation on a synthesized signal.

The method for segmentation of elliptical regions from scanned lines
Segmentation of elliptical regions from scanned lines [15] which is used in this paper is based on the direct least squares fitting of ellipses [7].
The implicit equation of ellipse can be represented by general equation of conic sections [17]: where for ellipse the following condition should be fulfilled: In ( 1) and (2) (x, y) represents the point on the conic section, and a i , i=1,…,6 are the parameters of the curve.Equation ( 1) can be represented in the matrix form: where A=[a 1 a 2 a 3 a 4 a 5 a 6 ] T is the matrix of coefficients and x=[x 2 xy y 2 x y 1].To find ellipse which represents the best approximation of a given set of points (x n , y n ), n=1,…,N, it is necessary to estimate parameters a i , i=1,…,6 (the members of a matrix A).Using direct least squares method, the problem of the parameters' values estimation can be reduced to the following minimization [18]: where D represents the design matrix: ( In order to customize the least squares method for direct fitting of ellipse, Fitzgibbon et al. in [7] introduced the following constraint: which can be represented in the matrix form: where C is 6×6 matrix with the following values: Introducing constraint (7), the minimization problem (4) can be represented by the following system of equations [7]: where T S=D D is scatter matrix [18], and λ represents Lagrange multiplier.In [7] Fitzgibbon et al. have shown that the system of equations ( 8) has single positive eigenvalue and that value provides the solution for A.
When the input set of points (x n , y n ), n =1, …, N are sampled from an exact ellipse without any noise, i.e. when points satisfy elliptical equation, scatter matrix S will be singular.In this case it will not be possible to determine coefficients of equation ( 1) using direct least squares method.
The employed method for recognition of elliptic regions from scanned lines is based on the singularity of scatter matrix S [18].If the input set of points ((x n , y n ), n =1,…,N) approximately represents certain ellipse (which is the case with scanned lines' points) than matrix S will be close to singular.In that case two conditions will be satisfied: 1) reciprocal condition number of matrix S will be close to zero, 2) the eigenvalue of matrix S -1 C can be close to zero or even less than zero [6].
The presented method for segmentation of elliptical regions from scanned lines is the region growing method, and it is carried out as follows.The points from scanned line are added to the region one by one starting from the first point in the line.In each step scatter matrix is calculated for all points in the region and its properties are checked.If the reciprocal condition number of matrix S and eigenvalues of matrix S -1 C are within the preset segmentation thresholds the point that was last added to the region belongs to the considered ellipse.When in an iteration reciprocal condition number of matrix S or eigenvalue of matrix S -1 C pass predefined thresholds, the point that was last added does not belong to the ellipse.That point indicates transition (border) between two elliptical regions and becomes the first point of the next region.
The described segmentation procedure is repeated from the beginning, until all points from the scanned line are processed.The pseudo code for application of the presented methodology for segmentation of elliptical regions using scatter matrix is given in Figure 1 OUTPUT: fin_coef -point on the merged segment's boundary During testing, we have noticed that the algorithm from Figure 1 is prone to over-segmentation and that additional procedure for merging of over-segmented regions should be applied.Since in the scanned lines noise is inherently present, parameters a i can be estimated for each segment using procedure from [7].The estimated parameters are in the sequel exploited for merging of adjacent over-segmented regions as presented in pseudo code from Figure 2.This algorithm requires a new threshold for merging.

Recognition of ellipses from synthesized scan line
To illustrate the properties of the presented method for recognition of ellipses we have synthesized contour (scanned line) consisting of three G1 continuous elliptical segments (Figure 3).It should be noted that points on the contour are not equidistant along x axis.
The parameters of synthesized segments and boundaries between segments are presented in Table 1.When the algorithm was employed to noiseless synthesized signal, it gave excellent results, i.e. it identified exact boundaries between segments.To further test the performances of the algorithm, we have contaminated the synthesized signal with white Gaussian noise with signal to noise ratio (SNR) of 100dB and 120dB.The results of segmentation of noised signals are presented in Table 2. From Tables 1 and 2 it can be observed that obtained segments' boundaries deviate slightly from ideal.The maximum deviation of 7 samples (that correspond to 0.6mm along x axis) is for the first transition in the case of signal with SNR of 100dB.The graphical representation of recognized elliptical segments is presented in Figure 3.

SEGMENTATION OF ELLIPTIC CYLINDERS, ELIPSOIDS, AND ELLIPTICAL TORI FROM STRUCTURED POINT CLOUDS
By extrapolation of the procedure for recognition of elliptical segments from scanned lines to two dimensions, we create the method for segmentation of surfaces from structured point clouds.
Structured point cloud can be observed as two series of intersected scanned lines -each series along one scanning direction.In our method for recognition of surfaces ellipse recognition procedure is applied to every scanned line of point cloud.Elliptical segments are first recognized in scanned lines along one direction, and afterwards in scanned lines along the otherperpendicular direction.In this way it is possible to create surface segmentation along both scanned object axes, separately.By simple intersection of the segmentation results in two directions, we can make final segmentation -group of points which represent different patches of the object's surface.
The proposed method can be utilized for segmentation of surfaces which are presented by ellipses in scanned lines.Elliptical cylinders, ellipsoids and elliptical tori belong to this class of surfaces.Namely, in structured point cloud created by 3D scanning of an ellipsoid, every scanned line will be ellipse.On the other hand, when elliptical cylinders are scanned, the shape of scanned lines in the point cloud will depend on the scan mode.There are two possible ways to scan elliptic cylinders: 1) scanned lines in one direction are straight lines, and in the other direction are ellipses (Figure 4 For illustration of the presented segmentation procedure we utilize three synthesized point clouds -three case studies.First case study considers elliptic cylinders, second ellipsoids, while in the third case segmentation of elliptic tori is presented.
In all three case studies, the surface is created using CAD software, and point cloud is generated from STL format of modeled object.For generation of point cloud, modified z-buffer algorithm [16] is applied to the surface representation in the form of triangular mesh.The rays are set vertically along z axis using uniform raster along x and y axis.The points of cloud represent the intersection points between rays and triangles of STL model.

Case study 1: Segmentation of elliptic cylinders
In the first case study we consider a point cloud representing the surface which is created by extrusion of profile from Figure 3   The results of segmentation of point clouds from  3 presents recognized cylinder's parameters that are estimated using least squares fitting method.Point cloud from Figure 5.b. has straight scanned lines along y axis and recognition algorithm did not detect any boundaries in scanned lines, i.e. in each scanned line along y axis only one segment is detected.It should be emphasized that the proposed method can segment straight lines since (1) for a 1 =a 2 =a 3 =0 represents linear equation.By comparing the estimated parameters of surface from Table 3 for the case presented in Figure 6.a, and parameters of elliptical segments from Table 1 it can be concluded that proposed segmentation method gave excellent results for all segments.The deviation is less than 1% with the exception of coefficient a 7 in segment 2. In the case of surface from Figure 6.b. estimated parameters are in accordance with scanning directions.Namely, it can be observed from Table 3 that: 1) values of parameters a 1 and a 2 are equal (angle of scanning is 45°), 2) estimated parameters a 1 and a 2 for cloud from Figure 6.b. are about two times less than corresponding parameters for cloud from Figure 6.a.((sin(45°)) 2 =1/2), 3) estimated values of parameter a 3 in both cases are equal.

Case study 2: Segmentation of ellipsoids
The second case study presents the results of algorithm utilization for recognition of ellipsoids.General ellipsoids are not commonly met quadrics in mechanical engineering.On the other hand, spheroids (special case of ellipsoids) are more frequent.Therefore, in this section we present the segmentation of surface which is created of three spheroids.The surface is obtained by rotation of chosen elliptical contours around x axis.Generated surface and synthesized structured point cloud (contaminated by noise with SNR of 120dB) are shown in Figure 7, while first section of Table 4 presents the parameters of synthesized segments.It should be noted that surface has G1 continuity.In this case study, the presented algorithm is used for processing point cloud in two directions -along x and along y axis.The segmentation results are given in Figure 8.As expected, implementation of algorithm along x axis detected three clusters of points (segments) which represent three spheroids.On the other hand, segmentation process along y axis did not detect boundary points.Recognized spheroids' parameters are presented in the second section of Table 4.
By comparing data from Table 4 it can be observed that applied segmentation algorithm gave excellent outcome -estimated parameters of recognized spheroids are quite similar to synthesized spheroids' parameters.Therefore, the proposed method adequately segmented parts of the general elliptic tori.General elliptic tori are 4 th degree surfaces and their parameters can be estimated using direct least squares method as well.

SEGMENTATION OF CYLINDERS FROM REAL-WORLD STRUCTURED POINT CLOUD
Experimental real-world verification of the proposed algorithm is carried out using a scanned surface of a part presented in Figure 12.a.This object represents a typical benchmark test part used in a number of research works referring to reverse engineering [8], [19], [20], [21].The part is made on 3-axes machining center and has very poor surface finish.3D scanning of object is carried out using ATOS Compact Scan by GOM mbH [22].The device provides very dense unstructured point cloud at output.Using z-buffer based procedure described in Section 3, we transformed unstructured into the structured point cloud.
The presented part contains several planar and quadric surfaces.In this paper, we have considered the surface which is marked red in Figure 12.a.The surface consists of a set of G1 continuous cylindrical patches.Results of segmentation of the surface along x axis using proposed method is shown in Figure 12.b.Estimated values of segmented surfaces' parameters are presented in Table 5. Parameters were estimated using least squares method.The values of parameters a 2 , a 4 , a 5 and a 8 are close to zero indicating that the profiles are extruded along y axis.Figure 13 shows histograms of boundary points between individual recognized segments along x axis.

CONCLUSION
In this paper we have proposed a method for recognition of surfaces represented by elliptical segments in scanned lines of structured point clouds.The method is primarily intended for segmentation of elliptic cylinders, ellipsoids, and elliptic tori.Besides, depending on the scanning direction the method can be used for segmentation of other quadrics such as cones.
As we have shown in the first case study, the method can be utilized for recognition of planar surfaces as well.However other more effective algorithms for recognition of planes from point clouds are available.The proposed segmentation method is based on recognition of elliptical segments in the both directions of structured point clouds.Recognition of elliptical segments is based on scatter matrix singularity during direct least squares fitting of ellipses.Described recognition algorithm belongs to the class of region growing segmentation algorithms and it can be used for segmentation of G1 and higher continuous surfaces.The proposed method is experimentally verified using three synthesized point clouds and one real world example.The first synthesized point cloud represents surface that is made of parts of elliptic cylinders, the second represents the surface made of ellipsoids, while the third case study refers to the surface made of elliptic tori.Real point cloud presents surface which contains five elliptic cylinders.
Future research will consider the segmentation of unstructured point clouds, as well as the segmentation of surfaces that are represented by other conic sections (e.g.hyperbolae) in point clouds.

Figure 2 .
Figure 2. Merging of over-segmented elliptical regions using parameters' estimate

Figure 3 .
Figure 3. Segmented contour which contain three elliptical segments

Figure 4 .
Figure 4. Cylinder and tori scanning principles: a) Scanning of cylinder along and perpendicular on the axis; b) Scanning of cylinder in directions angular to the axis; c) Scanning of elliptical tori

Figure 5 .
Figure 5. Surface made of parts of elliptical cylinders: a) 3D model; b) point cloud obtained by longitudinal scanning; c) point cloud obtained by angular scanning Figures 5.b. and 5.c.along x axis are shown in Figures 6.a. and 6.b, respectively, while Table

Figure 6 .
Figure 6.Results of segmentation of the surface made of elliptical cylinders along x axis: a) point cloud from figure 5.b.; b) point cloud from figure 5.c.

Figure 7 .
Figure 7. Surface made of parts of spheroids: a) 3D model; b) point cloud

Figure 8 .
Figure 8. Results of segmentation of the surface made of spheroids: a) segmentation along x axis; b) segmentation along y axis Table 4. Synthesized and estimated parameters of recognized spheroids Equation of quadric:

a 1 x 2 3 . 3 Case study 3 :
+a 2 y 2 +a 3 z 2 +a 4 xy+a 5 yz+a 6 xz+a 7 x+a 8 y+a 9 z+a 10 =0 Parameters [a 1 ×10 2 ], i=1,…10 2504 0.11 0.11 0.0 -0.0 0.0 0.007 0.0 0.001 -100.0]Seg.2 [0.0179 0.0625 0.0627 -0.0 0.0 -0.0 -1.6517 -0.0 -0.0042 -61.7384] 0281 0.0625 0.0625 0.0 0.0 -0.0 -2.5 -0.0 0.0 -43.752]Segmentation of elliptic tori Third case study illustrates the application of the proposed algorithm in segmentation of elliptic tori.The point cloud synthesized for this case study represents object's surface which is made of six G1 continuous parts of general elliptic tori.The surface is created by swiping the profile made from two elliptical segments along the path that consists of two elliptical segments.The object's 3D model with marked boundaries and the synthesized point cloud (with 120dB SNR) are presented in Figure 9. Figure 10 presents the results of segmentation using the proposed method.From Figures 9 and 10 it can be observed that detected boundary points in scanned lines along the same direction vary from ideal positions.This deviation in recognized boundary points comes from the presence of noise.The histograms of the segmentation results for each boundary are presented in Figure 11.

Figure 9 .Figure 10 .
Figure 9. Surface made of parts of elliptical tori: a) 3D model; b) point cloud

Figure 11 .
Figure 11.Histograms of detected boundary points from Figure 10.: a) between segments along x axis; b) between segments along y axis

Figure 12 .
Figure 12. Results of segmentation of the real point cloud: a) object's surface of interest; b) results of point cloud segmentation along x axis

Figure 13 .
Figure 13.Histograms of detected boundary points along x axis from figure 12.b.: a) between segments 1 and 2; b) between segments 2 and 3; c) between segments 3 and 4; d) between segments 4 and 5

Figure 1. Segmentation of elliptical regions using scatter matrix INPUT: x, y -
. The thresholds for reciprocal condition number and eigenvalue can be tuned depending on the noise level in scanned line.