WO2008000055A1 - Methods for simultaneous multi-set point matching - Google Patents

Abstract

Methods are provided for aligning multiple images of a three dimensional object that are obtained from different perspectives. The three dimensional object is labelled with markers located on the surface before the images are obtained or a reference object with markers on it is placed in close proximity to the object. Matching triangles are formed from markers on respective pairs of the images. For each of the respective pairs of images, a transformation function is determined to transform an orientation of a triangle of one of the images from the matching triangle pair to an orientation of a triangle of the other image of the matching triangle pair. Based on the grouping of these transformations a particular transformation function is selected to transform an orientation of the first image to an orientation of the second image. Based on the orientations of each of the multiple images with respect to one another, the markers in each of the multiple images are simultaneously assigned an identifier that indicates the same marker in all images. Using the assigned identifiers in each image, the multiple images can be aligned to form the three dimensional model of the object.

Description

Methods for Simultaneous Multi-set Point Matching

Field of the Invention

The invention relates to aligning two or more three dimensional images each having a respective three dimensional coordinate system in such a manner that the two or more three dimensional images share a common three dimensional coordinate system.

Background of the Invention

One manner to generate a 360 degree digital model of an object is to combine multiple images obtained from different angles using a three dimensional (3D) imaging device. A difficulty in such digital model generation is properly aligning the multiple images.

A conventional method of aligning multiple images is using photogrammetry to accurately register the different positions from which the object is scanned so that the relative transformation between the images is known.

Another method is by matching shapes that are part of the images in the overlapping areas, assuming that the object has distinctive features that can be matched.

The first method introduces additional complexity to the alignment problem due to additional scanner position information required to solve the problem. The second method is susceptible to errors if distinct features cannot be easily matched.

A new method of aligning multiple images is needed that is less reliant on knowing the position of a scanner used to take measurements taken and that is more robust to errors incurred by inadequate shape recognition. Summary of the Invention

According to a first aspect of the invention, there is provided a method for aligning a plurality of three dimensional images of an object, each image having a respective three dimensional coordinate system and comprising a set of points related to a surface of the object, such that when aligned within an acceptable error threshold the plurality of three dimensional images share a common three dimensional coordinate system, the method comprising the steps of: i) for each set of points, forming at least one triangle using vertices in the respective set of points; ii) for each respective pair of the plurality of sets of points; a) for each triangle in each set of points of the respective pair,- matching the triangle of a first set of points of the respective pair with a triangle of a second set of points of the respective pair to form at least one matching triangle pair; determining a transformation function to transform an orientation of a first triangle of the matching triangle pair to an orientation of a second triangle of the matching triangle pair; b) grouping together determined transformation functions that are substantially similar to form a respective cluster transformation function, each cluster transformation function representing a transform function for transforming an orientation of the first set of points to an orientation of the second set of points; iii) orienting all the sets of points with respect to an orientation of a single set of points based on the cluster transformation functions of the respective pairs of images; iv) simultaneously assigning to points in at least two sets a global point identifier that indicates the same point in the at least two sets.

According to a second aspect of the invention, there is provided a computer usable medium having computer readable program code means embodied therein for aligning a plurality of three dimensional images of an object, each image having a respective three dimensional coordinate system and comprising a set of points related to a surface of the object, such that when aligned within an acceptable error threshold the plurality of three dimensional images share a common three dimensional coordinate system the computer readable code means comprising: i) for each set of points, computer readable code means for forming at least one triangle in which three of the points in the set of points form vertices of the at least one triangle; ii) for each respective pair of the plurality of sets of points; a) for each triangle in each set of points of the respective pair; computer readable code means for matching the triangle from a first set of points of the respective pair with a triangle of a second set of points of the respective pair to form at least one matching triangle pair; computer readable code means for determining a transformation function to transform an orientation of a first triangle of the matching triangle pair to an orientation of a second triangle of the matching triangle pair; b) computer readable code means for grouping together determined transformation functions having a substantially similar transformation function to form a respective cluster transformation function, each cluster transformation function representing a transform function for transforming an orientation of the first set of point to an orientation of the second set of points; iii) computer readable code means for orienting all the sets of points with respect to an orientation of a single set of points based on the cluster transformation functions of the respective pairs of sets of points; iv) computer readable code means for simultaneously assigning to points in at least two sets a global point identifier that indicates the same point in the at least two sets . Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.

Brief Description of the Drawings

Preferred embodiments of the invention will now be described with reference to the attached drawings in which:

Figure 1 is a flow chart for a method of multi-set point matching according to an embodiment of the invention;

Figure 2 is a schematic diagram of three sets of points that are to be aligned;

Figure 3 is a schematic diagram of three sets of points, each set containing multiple triangles formed from the points;

Figure 4A and 4B are schematic diagrams of a two dimensional transform space demonstrating clustering of transformations ;

Figures 5A, 5B, 5C, 5D, 5E and 5F are schematic diagrams of a sequence of steps involved in setting orientations for matching set pairs according to an embodiment of the invention;

Figure 6A, 6B and 6C are schematic diagrams of different possibilities for alignment of three sets of example points;

Figure 7 is a schematic diagram of a two-dimensional transform space showing how outliers are eliminated from cluster groups in accordance with an embodiment of the invention; Figures 8A, 8B, 8C and 8D are schematic diagrams of an example of how global identifiers are assigned for points in accordance with an embodiment of the invention;

Figures 9A and 9B are schematic diagrams of three sets of points and how the sets are aligned, merged and given a common global identifier in accordance with an embodiment of the invention;

Figure 10 is a schematic diagram of the three sets of points shown in Figure 2 in which points have been matched in the three sets;

Figure 11 is a flow chart for a method of multi-set point matching according to another embodiment of the invention; and

Figures 12A and 12B are schematic diagrams of two examples of three overlapping sets that can be operated on according to embodiments of the invention.

Detailed Description of the Preferred Embodiments

In some embodiments the invention includes a method for aligning two or more 3D images of an object taken by a 3D imaging device that can capture both geometry of a surface of the object and positions of markers on the object's surface. After aligning the images and locating them in a common coordinate system, the images are combined together to generate the 3D model of the object.

In a first embodiment of the invention, a large number of small markers are placed over the entire surface of the object, and 3D images are taken of the object so that every part of the object is recorded by at least one image and there are at least three non-collinear matching points for each pair of overlapping images. These markers are referred to as tie points as they help tie the images together. Tie points in all images are matched simultaneously and matching points are assigned with the same global identifier (ID) . The images are then aligned by rigid transformation so that the tie points with the same global ID become nearly coincident within an acceptable margin of error.

In a second embodiment of the invention, we have a ground control point (GCP) object, which is an object (usually in a standard shape, for example a cube or plate) with many markers, called GCP points, located on the object's surface. The positions of the GCP points are known, either by precise machining of the points or by measuring them using Coordinate Measuring Machine (CMM) or imaging equipment. The object to be modelled, which has no markers located on it's surface is placed close to the GCP object, and the two objects are imaged together at different angles so as to fully image the object. Each image is to have at least three non-collinear GCP points. For each image two sets of points are analyzed: a first set, S, includes the known positions of the markers of the GCP object and a second set, T, includes the GCP points visible in the image. The positions of GCP points in S are exactly known as the spacing of the points is known apriori . The GCP points in T are matched with the GCP points in S, so that the rigid transformation that transforms the image towards the coordinate system of the GCP object can be found. After repeating this process for other images, the images are aligned in a common coordinate system.

In some embodiments the global ID described above is a unique identification integer (normally starting from 0) assigned to all matching points in different sets. Therefore, two points in the same set cannot have the same global ID. Unmatched points may be given a global ID of -1, or more generally some other distinct value. Two points in different sets are said to be matching if they have the same global ID, that is they coincide within a margin of error after appropriate rigid 3D transformations on the sets. The margin of error is an acceptable misalignment distance between points with the same global ID.

The expression "rigid 3D transformation" is used to describe the transformation of a set of points. In some embodiments the transformation includes one or both of a rotation of the points and a translation of the points in each of three axes.

Two sets of points are considered to be overlapping or connected if they have three or more matching points that are not collinear. A family of three or more sets of points are said to be all connected if for any two sets, for example A and B in the family, we can find a sequence ("path") of sets in the family, say C, D and E, such that A and C are overlapping, C and D are overlapping, D and E are overlapping and E and B are overlapping.

In some embodiments, the invention is capable of handling one or more of the following cases: a) positions of the points may not be accurate; b) some points in a set may be singular, that is they cannot be matched with a point in other sets; c) symmetry is allowed, that is three matching points in two sets may form an isosceles (two sides equal) or equilateral (all sides equal) triangle; d) the same pattern of points may repeat within a set and may repeat in another set; e) there can be a large number of sets, in which each set has a large number of points; and f) the same global ID can be shared by points from a large number of overlapping sets.

In some implementations the invention is not as effective when a given family of sets are not all connected, that is there exists two sets A and B in the family such that a path of overlapping sets joining A and B cannot be found. In some embodiments of the invention at least three non-collinear matching points between sets are used, so that no sets are left unpaired. Situations in which there are only one or two common points between every pair of sets, can be overcome in some embodiments of the invention as described below in the section entitled "Further embodiments of the Invention" .

Also, if there are many symmetries and/or repetitions of patterns in the object being modelled, then there are a large number of possible combinations of matching points. In some embodiments, even though the most likely combinations of matching points are selected first, it may take several iterations to get to the combination that is a correct match. In some embodiments apriori knowledge can be used to help reduce the large number of possible combinations.

The methods described herein are considered to be robust since the situation of many symmetries or pattern repetitions are unlikely or can be easily avoided, for example by simply randomly positioning the markers on the surface of the object or having more than three matching points per each pair of overlapping sets.

Conventional methods have limited robustness. Some can only do point matching for two sets of points. Some assume that the positions of the points are accurate without error. Some do not allow symmetry or pattern repetition. Some assume that all points in each set can match with some other point in another set .

A method according to an embodiment of the invention will now be described with respect to the flow chart of Figure 1. A first step 100 is inputting multiple sets of tie points on which the method will operate. A second step 105 includes forming candidate edges, where an edge is a line between two points in a set. A third step 110 is to form matching triangle pairs making use of the candidate edges of step 105. Following the third step 110 is a decision step 115 in which it is to be determined if all the sets are connected by matching triangle pairs. If any of the sets are not connected (no path) then the method is complete (END) . If the sets are connected (yes path) then a further step 120 is to determine transformation functions for each matching triangle pair and from these determined transformation functions form transformation function cluster combinations. Following the forming of the transformation cluster combinations at step 120, a next step 125 is to sort the transformation function cluster combinations in a desirable manner. Once the transformation function cluster combinations are sorted at step 125, one of the transformation function cluster combinations is selected 130 to be operated on. A next step, step 140 involves finding set orientations of the sets for the selected transformation function cluster combinations. Once the set orientations are determined, a next step, step 145 involves removing outlying transformations for each set pair. After outlying transformations are removed, a further step, step 150 is assigning common global IDs to points in each set. Following step 150, points that are in close proximity, for example within a particular distance threshold range, are merged and given a common global ID. Another decision point occurs at 160. It must be determined if a result output from steps 140-155 is acceptable. If the result is not acceptable (no path) , a subsequent transformation function c Luster combinations of the sorted cluster combinations from step 130 is selected at step 165 and steps 140, 145, 150, 155 and 160 are repeated. If the result is acceptable (yes path of step 160) , the global IDs are output at step 170 allowing the aLignment of the multiple images to form the 3D model. The individual steps of Figure 1 will now be described in further detail .

Input multiple sets of points (step 100)

The number of sets is provided and for each set, the number of points. Then for each point in each set, the local identifier (IDs) and the 3D position in the coordinate system of each respective set are also provided. The local IDs are used to identify the points within each respective set.

Figure 2 shows three sets of points, Set 0, Set 1 and Set 2. Set 0 includes points having local IDs A, B, C, D, E and F. Set 1 includes points having local IDs G, H, I, J and K. Set 2 includes points having local IDs L, M, N and O. The sets have differently shaped contours such that there is no simple manner to determine which points in the three sets may be matching points. The sets are considered to represent 3D images and therefore the contours and points in Figure 2 are understood to have a 3D representation as well.

Form candidate edges (step 105)

After information regarding the number of sets, the number of points in each set, and the local IDs for points in the respective sets is provided, a maximum positional error ε is determined for the points. In some embodiments the error is at least in part due to the quality of the imaging equipment and therefore the error is estimated from the accuracy and/or resolution of the 3D imaging system. As each point has a maximum positional error of ε, the maximum error in the distance between any two points in a set is equal to 2*ε. Consequently, a maximum error in the difference between distances between two points in a first set and two points in a second set is 4*ε. An edge is an ordered pair of points in the same set.

So for two points in a set, say A and B of Set 0, two edges can be formed: AB and BA. The length of an edge is the distance between the two points .

In a given set, a candidate edge is an edge that has a corresponding or matching edge in another set whose length differs from that of the candidate edge by less than or equal to a magnitude of 4*ε.

To find candidate edges, the edges in each set are first defined. A set with n points should have n* (n-1) edges. The edges in each set are sorted in a particular order. In some embodiments this includes ascending order according to the edge lengths, that is shortest length at the top of the list and longest length at the bottom of the list. Edges that are longer than the potential candidate edge in a different set, but whose length do not exceed the length of the potential candidate edge by more than 4* ε are recorded in a list of corresponding edges. For each potential candidate edge, when searching other sets for matching edges of a corresponding length it is possible to take advantage of the fact that each set has a list of all edges in that set that is sorted in order of length. Therefore, entire list need not be searched when looking for edges of a particular length. If there exists at least one edge within the allowable maximum error, then the potential candidate edge is a candidate edge, and the list of corresponding edges in other sets is maintained for the candidate edge. The end result is that each candidate edge in each set has an associated list of edges in other sets that may correspond to the candidate edge in those respective sets. Arranging the edges in ascending order is one example of how the edges can be arranged. Those skilled in the art would realize that the edges can be arranged in other ways, for example descending order of edge length. Edges of other sets that are shorter than the candidate edge by 4* ε are also included in the associated list of edges in other sets that may be the same edge in those respective sets.

Figure 3 shows the sets from Figure 2 in which solid lines between points in each set indicate candidate edges and dashed lines indicate non-candidate edges. An example of a scale representation of the maximum error in distance difference (4*ε) is also indicated in the figure.

Form triangles in sets and matching triangle pairs in pairs of sets (step 110)

In addition to the maximum positional error value, another variable that is considered when aligning points between sets is a maximum root -mean-square (rms) residual p. When a first set of n points Pi, P₂,-, P_n is aligned with n matching points Qi, Q₂,-/Q_n in a second set, a rigid transformation T can be applied to the first set so that T(P_x) is close to Qi, T(P₂) is close to Q₂, etc. The rms residual of this alignment transformation is defined as the square-root of the arithmetic mean of the squares of distances between T(Pi) and Qi, for i = l,2,...,n. The rms residual between the first and second sets of points is the minimum value of the rms residuals between the respective matched points over different transformations. There are minimization techniques that can be used for finding the transformation that gives the rms residual between two sets of points. One example of a minimization technique is the use of eigenvalues.

A triangle is an ordered triple of points in the same set that are not substantially collinear. The triangle is denoted by the three points or vertices of the triangle in a particular order, for example ΔXYZ . Six labelled triangles can be formed by the same three points. A triangle pair is made up of a triangle, for example ΔXYZ in a first set and a triangle ΔPQR in a second set . The triangle pair of the example triangles is denoted by (ΔXYZ, ΔPQR) . Although triangles are ordered by definition, triangle pairs are invariant under transposition, that is triangle pairs (ΔXYZ, ΔPQR) , (ΔYZX, ΔQRP) , (ΔZXY, ΔRPQ) , (ΔXZY, ΔPRQ) , (ΔYXZ, ΔQPR) , (ΔZYX, ΔRQP) are all regarded as equivalent.

A triangle pair, for example (ΔXYZ, ΔPQR) is considered to be a matching triangle pair if one or both of the following conditions are true:

a) the differences in lengths between XY and PQ, between YZ and QR and between ZX and RP, each respectively do not exceed 4*ε,- and

b) the rms residual between ΔXYZ and ΔPQR, (i.e. the minimum rms residual of all rigid transformations from ΔXYZ towards ΔPQR) does not exceed p.

These two conditions are not independent. The first one can imply the second if ε is sufficiently smaller than p, and vice versa. A reason for using two conditions is to allow a choice of using either one of the conditions or both of the conditions together.

In some embodiments, one more condition is imposed on the triangle pair (ΔXYZ, ΔPQR) for it to be considered a matching triangle pair. The additional condition is that the two triangles have normal vectors for two edges of each respective triangle that point in a same direction with respect to a surface of the object. By way of example, ΔXYZ and ΔPQR are only considered to be matching triangle pairs if the normal vector for ΔXYZ satisfying the right-hand rule (parallel to cross product XY X XZ) and the normal vector of ΔPQR satisfying the right-hand rule (parallel to cross product PQ X PR) are either both pointing away from the object or both pointing toward the object.

A candidate triangle is a triangle in a set that has a chance of forming a matching triangle pair with a triangle in another set.

To form all matching triangle pairs, all candidate triangles are determined for each set. For each set, all combinations of three candidate edges are determined in the set such that the edges join together tail -to-head to form a triangle in which the edges are not substantially collinear. The edges can be confirmed as not being substantially collinear by checking that the difference between a longest edge and a sum of the other two edges is not less than 4*ε.

For each potential candidate triangle in each set, a search is performed for triangles in other sets such that the potential candidate triangle and any triangles in other sets satisfy at least one of the first two conditions of matching triangle pairs described above. The first condition is satisfied by determining that the lengths of each corresponding edge in the triangles do not differ by more than 4*ε. The second condition is satisfied by finding the rigid transformation to transform an orientation of a first triangle of the matching triangle pair to an orientation of a second triangle of the matching triangle pair that minimizes the rms residual. The rigid transformation that minimizes the rms residual can be found, for instance, based on determining eigenvalues. Furthermore, in some embodiments the third condition may also be used to establish whether the potential candidate triangle and other triangle are matching triangle pairs.

To make the search more efficient, the list of corresponding edges in other sets for each candidate edge maintained in previous step 105 can be used when forming triangle pairs. To further increase the search efficiency, a data structure can be used to associate each edge to the list of triangles containing that edge as the shortest edge.

When searching for matching triangles in each pair of sets a scalene triangle is considered to be a triangle with all sides having different length, such that the difference between each pair of sides is greater than 4*ε. An isosceles triangle is considered to be a triangle with two of three sides equal in length such that the difference in length between the two equal sides is less than 4*ε. An equilateral triangle is considered to be a triangle with three equal sides such that the difference in length between the three equal sides is less than 4*ε. Therefore, due to the ordered triple nature of labelling triangles, a pair of potentially matching scalene triangles in different sets can only form one possible match based on the different edge lengths. A pair of potentially matching isosceles triangles in different sets can form two matching pairs having different ordered vertices when the two conditions described above are applied. A pair of potentially matching equilateral triangles in different sets can form six matching pairs having different ordered vertices when the two conditions described above are applied.

Referring to Figure 3, when searching for scalene triangles in Set 0 and Set 1, three different matching pairs are found in the form of (ΔBEF,ΔlHK), (ΔBDE,ΔKJH), and (ΔDBA,ΔIJG) . When searching for isosceles triangles, two matching pairs are found in the form of (ΔCBD,ΔIKJ) and

(ΔCDB,ΔKIJ) . When searching for equilateral triangles six matching triangle pairs are determined (ΔECD,ΔHIJ), (AECD, ΔHJI) , (ΔECD,ΔIHJ) , (ΔECD,ΔIJH), (ΔECD,ΔJHI), and

(ΔECD,ΔJIH) .

If the third condition is imposed on the groups of isosceles and equilateral triangles, only one matching triangle pair is determined for the group of isosceles triangles (ΔCBD, ΔIKJ) and three matching pairs for the group of equilateral triangles (ΔECD, ΔHIJ) , (ΔECD, ΔIJH) , and (ΔECD, ΔJHI) .

Sets all connected? (step 115)

To check that the sets are all connected, or overlapping as defined above, each pair of sets is examined. Two sets are identified as overlapping if they have at least one matching triangle pair between them.

Every combination of pairs of sets is examined. The examination process is started by selecting a first set and labelling it with an identifier. The selected set is compared to a second set. If the two sets are overlapping, and the second set is unlabelled, the second set is labelled with a new identifier. If the second set already has an identifier, then another set is selected for comparison with the first set. When no more sets can be compared or labelled with respect to the first set, a set that is connected to the first set is selected and the labelling process repeated. This is continued until no more new sets can be labelled.

At a point when no more new sets can be labelled, if all of the sets are labelled, it is concluded that the sets are all connected and the method proceeds to the next step (yes path from step 115) . Otherwise, the method is stopped at this point (no path from step 115) .

If the method is stopped at this point the method can be reinitiated using the same number of sets in which the points in the sets are changed, or the preceding steps can be repeated for only the sets that were identified with labels and are connected. If the method is to be reinitiated with the same number of sets one manner of changing the sets is to add more points to some or all sets so as to improve the potential for sets to be connected.

Determine transformation function for each matching triangle pair and form transformation cluster combinations (step 120)

Step 120 is performed for each pair of overlapping sets in turn, however the description below applies to an example of a single pair only. Such a process is to be repeated for all overlapping pairs. To avoid ambiguity and repetition, a convention is adopted that in a pair of sets, the index of the first set i is less than that of the second set j . Each set is given an incremental index as they are input at step 100 e.g. 0, 1, 2,..JsT. Also, in some embodiments a particular convention is used such that transformations between pairs of sets transform a first set towards a second set.

For a three dimensional space, each rigid transformation between two overlapping sets can be expressed uniquely by six transformation parameters including three rotations about x, y and z axes and three translations along x, y and z axes. The uniqueness of angles is guaranteed by restricting values to principal values of inverse trigonometric functions. The transformation between triangles of the matching tiriangle pair can be represented by a single point in a six- dimensional "transformation space" formed by the six transformation parameters. As a six-dimensional transformation space is difficult to portray, Figure 4A represents a two- dimensional transform space for illustration purposes. Figure 4A represents a transform space where the coordinate grid of the transform space represents movement for a particular type of transform, such as a rotation about an axis or a translation along an axis. Locations indicating transformation functions to transform an orientation of a first triangle of the matching triangle pair to an orientation of a second triangle of the matching triangle pair in Figure 4A are indicated by lowercase letters a-k.

For comparison purposes, each square in the two- dimensional transform space of Figure 4A would represent a six dimensional "hyper-cube" in the six-dimensional transformation space .

If a transformation function transforms most points in the first set towards corresponding points in the second set, then that same transformation function should transform each triangle formed by the points in the first set towards the corresponding triangle in the second set . Points representing each matching triangle pair transformation for different matching triangle pairs across a pair of sets form a grouping or a cluster around a common location in the transformation space if the transformation function is consistent within an error threshold for all matching triangle pairs in the set pair.

Since there may be more than one way to transform the points in the first set toward the second set, there may be several clusters that form in the transformation space. Each cluster of transformation functions represents triangle matching pairs having transformation functions that are substantially similar. It should be noted that the transformation functions determined to represent the two different potential matching triangle pairs for matching isosceles triangles described above may correspond to different groupings or clusters in the transformation space. The same can be said for the six potential matching triangle pairs for matching equilateral triangles .

If there are multiple clusters in the transform space a decision is made which cluster takes preference. There are many cluster location determining techniques. However, the techniques usually assume that the total number of clusters is known before hand. In some embodiments of the invention a special clustering method is used which does not need the apriori knowledge of the number of clusters. Furthermore, a single transformation location can be treated as a cluster. This technique is hereinafter referred to as accumulation and is based on weighting of the transformations in the transform space. In some embodiments the cluster with the largest total weighting value of the multiple clusters is selected as the most probable transform function for the pair of sets of points .

The entire six-dimensional transformation space is divided into equally sized cells. The resolution of each cell for the three rotational -axes and three translational-axes are defined so that the cells are not so small that they can only contain one transform point and not so large that adjacent clusters of multiple transformations merge in single cell. In some embodiments the resolution that is selected is a function of maximum positional error ε and maximum rms residual p. To facilitate dealing with the cells in the six dimensional transform space, a coordinate system is used in which each cell is identified by six integers, in which each one of the six integers represents one cell in a row of consecutively numbered cells in that respective dimension.

Each point in the transform space corresponds to a particular transformation and is represented by six real numbers .

Each cell in the transform space corresponds to a collection of points within a hypercube and is represented by six integers. For each cell, each point in the six dimensional transform space representing a transform is identified and a weight is assigned to that point. In some embodiments the weight is equal to the kth-power of the reciprocal of the rms residual p for the matching triangle pair represented by that point, where k is a parameter between 0 and 1 (inclusive) that depends on the amount of overlap between images and the accuracy of the tie points in the overlap region. In cases where the tie points overlap well and are accurate, k can be 1 (i.e. the weight = reciprocal of residual) .

In a case in which a thin object having two sides of substantially the same shape is imaged, tie points on the physical edges of the two sides are utilized as this is where the two sides overlap. In this case the images must be taken at an angle so as to include the edges . The edges are narrow and have high curvature, so the common tie points there are less accurate and the triangles are thin. In order to make use of these tie points, the contribution of accuracy (i.e. reciprocal of residual) to the weight is reduced by setting k to a small number, for example 1/2 (i.e. weight = square root of reciprocal), 1/4, or 0 (i.e. weight = 1 always) .

In some embodiments the accumulation of the weights for each transformation point in each cell also takes into account transformation points in the neighbourhood of that respective cell. Two cells are considered to be adjacent if any of the integers in the two cells' six integer labels differ by a value equal to one. Therefore, an order of adjacency is herein defined as the number of integers in the labels that differ by one. For example, a six integer label (5,2,2,3,4,1) is adjacent to (5,2,2,4,4,1) with an order of one because the fourth integers in the two labels differ by one, "3" in the first label and "4" in the second label. A six integer label (5,2,2,3,4,1) is adjacent to (4,1,1,2,5,0) with order six as all six integers differ by one. A six integer label

(5,2,0,3,4,1) is not considered to be adjacent to the six integer label (5,2,2,3,4,1) because the third integer value in the two labels differs by a value of two, "0" in the first label and "2" in the second label. Care has to be taken in identifying adjacent cells along the rotation-axes so as to properly handle wrapping of angles. For example, an angle of -L79 degrees is substantially equal to an angle of 178 degrees even though their numerical values are vastly different.

When adjacent cells are used in determining the overall weighting of the cell a new total weight is calculated for each cell that is equal to the sum of the cell's original total weight, plus the original total weight of each of its adjacent cells multiplied by 0.5^k, where k is the order of adjacency. Hence, adjacent cells that are closer to the cell contribute more to the new total weight.

An issue that may arise when dividing the six dimensional space into cells is that a cluster of transform locations may be spread over more than one cell. However, if the boundaries of the cells in the transformation space are shifted the transform locations of the cluster may substantially fit within the confines of a single cell . In order to mitigate such an issue, in some embodiments the boundaries of the cells in the transformation space are shifted by an amount equal to half a cell width in all six directions and the above weighting procedure is repeated for the shifted boundaries. Either one of the original six dimensional space or the shifted boundary six dimensional space is selected depending on which one has cells having clusters with a largest number of transform locations. Figure 4B shows the same two dimensional transformation space with the same transformation points as Figure 4A, where the grid of cells as been shifted by half a cell with in both dimensions. Where the cluster of transformation locations a, c, i and j is spread over four cells in Figure 4A, the same four transformation locations are substantially in a single cell in Figure 4B. Figure 4B also indicates the accumulated weight of the cells by different cross hatching patterns in the cells. The cell substantially encompassing transformation locations a, c, i and j is shown to a particular weighting value which is the largest weighting value. Cells that contain a single transformation location or are in close proximity to a single transformation location have different cross hatching patterns, indicating their weighting value is less. Cells that contain no transformation location are a white, indicating there is no weighting value for those cells .

Using the selected grid with the new total weights for each cell, cells are identified whose new total weights are at local maximum, i.e. greater than or equal to the new total weights of all its neighbours. There can be one or more local - maximum cells, and the local -maximum cell with the greatest weight value is called the global -maximum cell. These local- or global-maximum cells will be referred to herein as "clusters" or "cluster cells" .

Sort cluster combinations (step 125) In an example having n images to be aligned, there are n sets of points and m pairs of overlapping sets (m <= n*(n-l)/2) . As this can be a substantially large number, it is desirable to arrange the sets in some convenient order. For each of the m pairs of overlapping sets, a list of clusters of transformation functions is determined, that is sorted in descending order of the new total accumulated cell weights from greatest weighted value to least weighted value. In the list, the cluster at the top (which is in fact a global -maximum cell) is given an identifier 0, the one below it is given an identifier 1 and so on.

A cluster combination, or transform function combination, is an ordered m-tuple of identifiers, where the ith component in the m-tuple is the identifier of the cluster used for the ith pair of overlapping sets. So, the m-tuple of aLl zeros (0, 0,....,0) is a cluster combination formed by c Lusters that each have an identifier equal to 0, that is the global -maximum cluster for each pair of overlapping sets.

If there are N₁ clusters for the ith pair of overlapping sets of the m pairs of overlapping sets, then there are altogether a total number of cluster combinations equalling N] *N₂*...*N_m. In theory, each cluster combination can be a solution to the alignment of the sets, though the combinations formed by clusters with the largest number of zeros is a more reasonable choice of selection to be the solution. As Ni*N₂*...*N_m can be a large number, in some embodiments a number of cluster combinations to be chosen as possible solutions to try is limited to a maximum number L.

In some embodiments the cluster combinations are sorted in a list in descending order of preference, so that the more preferred ones are tried first. In some embodiments the preference is measured by a "ratio-product" , which is obtained by multiplying the ratio of the total weight of each cluster by the total weight of the global -maximum cluster of all the M clusters for the m pairs of overlapping triangles. For example, for the cluster combination (0,0,...,0), the ratio-product is equal to one. For a cluster combination (3, 1, 0,...,0), the ratio-product is equal to (a3/A) * (bl/B) . The value "a3" is the new total weight of the first cluster, the cluster having the ordered identifier equal to "3" in the cluster combination and "A" is that of the global -maximum cluster for the first pair of overlapping sets. The value "bl" is the new total weight of the second cluster, the cluster having the ordered identifier equal to "1" in the cluster combination and "B" is that of the global -maximum cluster for the second pair. The remainder of the clusters with a "0" identifier have a ratio equal to one. Hence the ratio-product for each cluster combination is less than or equal to one .

Select best cluster combination (step 130)

Once the cluster combinations are sorted, the cluster combination with the largest ratio-product is selected. In some cases the top cluster combination is (0, 0,...,0) . In this case the cluster combination is the global -maximum cluster of each o E the m pairs of overlapping sets.

Find set orientations (step 140)

The term "orientation" is used to describe the rigid transformation that brings the set of points from an original coordinate frame to another coordinate frame . The inverse of the rigid transformation is used to bring the object back to the original frame. A set with an orientation of I, the identity transformation, is considered to be in a reference coordinate frame . To find the respective orientation of each set, a first step is to determine the transformations between all pairs of overlapping sets, using the cluster combination selected in step 130. For each pair of overlapping sets, say Set i and Set j (i < j) , a "centre" location of the selected cluster is determined in the six-dimensional transformation space, as the weighted average of the positions of the points in the cluster cell and its adjacent cells. The weights for the points in the cluster cell are the reciprocal of the minimum rms residual for the matching triangle pair represented by that point. The weights of the points in its adjacent cells are obtained by multiplying the reciprocal of the minimum rms residual by a power of 0.5 as described above. The six- dimensional coordinates of this centre location, which are the three rotation and three translation parameters, are then synthesized into a transformation function Ti_j , which represents the transformation for points from Set i to Set j in matrix form. The inverse transformation function T_ji for points from Set j to Set i is also determined.

The set-to-set transformations are applied to the sets to determine the set orientations. The transformations may not all be consistent. To resolve the inconsistency, the list of pairs of overlapping sets is sorted in descending order of the new total weight of the cluster cells of the transformation between the pair, and the transformations are applied in that order. In this way, the more reliable transformations will take preference .

Initially, none of the sets have a particular group assignment . The overlapping sets are assigned in order of the sorted list, say (Set i, Set j). The transformation and inverse transformation between Set i and Set j are Ti_j and T_ji, respectively. In an alternative representation, transformation Tij is denoted by α and inverse transformation T_j1 is denoted by α^"1 for convenience. A further convention is that concatenation of multiple transforms describing orientation of a coordinate frame through multiple re-orientations to the reference coordinate frame is expressed from left to right.

There are five cases for assigning the orientation:

1) If both sets do not belong to any group, then Set i is assigned the orientation I and Set j is assigned the orientation α. Then both sets are put into a new group with the next available group number.

2) If Set i belongs to Group g and has orientation σ, but Set j does not, then Set j is put into Group g and is given an orientation of σα.

3) If Set j belongs to Group h and has orientation τ, but Set i does not, then Set i is put into Group h and is given an orientation of τα^"1.

4) If Set i and Set j belong to the same group, having orientations σ and τ respectively, then nothing need be done to Set i and Set j . However, in a further step it is determined whether their orientations are "roughly" consistent, i.e. σα is roughly equal toτ. This can be done by checking whether the three rotational and three translational parameters of the concatenation σα and those of τ differ by some prescribed threshold, with due care about the wrapping of angles. If their orientations are not "roughly" consistent, the link between Set i and Set j is severed, so that the sets will be treated as non-overlapping. Therefore, in the subsequent step of global ID assignment, these sets will not be used as an overlapping pair of sets. Breaking this pair of sets is sensible when it is lower in the sorted list. Transformations of pairs higher in the list are more reliable and should dictate the set orientations .

5) If Set i and Set j belong to different Groups g and h, having respective orientations σ and τ, then these two groups are merged together into one group by putting all sets of Group h into Group g, and adjusting the orientations of all sets of Group h using the link α between the two groups.

Each pair of overlapping sets in the sorted list is processed in this manner. Since it has previously been determined that the sets are all connected, at step 115 of Figure 1, eventually all sets belong to one group only, and only one set has the orientation of reference coordinate frame I. That set becomes the reference set, placed in its own coordinate system, with all other sets placed relative to it.

The above-described assignment process will be described with respect to the example of Figures 5A-5F. Figure 5A shows a group of eight cluster generally indicated by 1-8, in which the order of the numbering indicates the descending order of the weight of the clusters. Transformations between set-pairs of clusters are indicated by symbols α, β, χ, δ, ε, φ, γ, η, i with an arrow indicating the direction of the transformation. For example, transformation α is the transformation used to orient cluster 1 in the same coordinate frame as cluster 5. Ultimately, the goal is to obtain transformations for each cluster with respect to the reference coordinate frame I .

In Figure 5B each one of clusters 1, 3 and 4 uses its own coordinate frame as the reference coordinate frame I . Transformations α, β and χ are applied to clusters 1, 3 and 4, respectively to orient clusters 1, 3 and 4 with respect to clusters 5, 6 and 7. In Figure 5C transforms δ and ε are applied to cluster 2 and cluster 7, respectively to orient clusters 2 and cluster 7 to cluster 6 and cluster 8. The transformation from cluster 4 to cluster 2, via cluster 6 is represented by β*δ^~\ where δ^"1 is the inverse transformation function of δ. The inverse transformation is used because δ represents the transformation in the direction from 2 to 6, which is opposite to the desired orientation. The transformation from cluster 3 to cluster 8, via cluster 7 is represented by χ*ε.

In Figure 5D transformation φ is applied from cluster 2 to cluster 6. Since cluster 3 and cluster 4 are now coupled via transformation β and newly applied transformation φ, a single reference coordinate frame I must be selected as either c Luster 3 or cluster 4. In this example the reference coordinate frame I of cluster 3 is maintained. The transformation from cluster 3 to cluster 4 via cluster 6 now becomes φ*β^"1.

In Figure 5E, transformation γ from cluster 2 to cluster 5 is applied. Since cluster 3 and cluster 1 are coupled via transformations α, δ and φ and newly applied transformation γ, a single reference coordinate frame I must be selected as either cluster 3 and cluster 1. In this example the reference coordinate frame I of cluster 3 is maintained. The transformation from cluster 3 to cluster 1 via clusters 6, 2 and 5 now becomes φ*δ^"1*γ*α^"1. The transformation from cluster 3 to cluster 5, via clusters 6 and 2 is represented by φ*δ^"1*γ.

In Figure 5F, transformations η and i are applied from cluster 6 and cluster 4, respectively to orient clusters 6 and 4 with respect to clusters 8 and 5. In this particular example two assumptions are made. A first assumption is that φ*η≤χ*ε and a second assumption is that φ*β^"1≠ φ*δ^"1*γ. Therefore, transformation η between cluster 6 and 8 is maintained and transformation i between cluster 4 and 5 is considered to be inconsistent with the other transformations and is eliminated.

The set-to-set transformations are applied to the sets to determine the set orientations. These transformations may not be all consistent, as shown in Figure 5F. Figures 6A, 6B and 6C illustrate a further example of set alignment and how transformations are considered.

Figure 6A shows an example of three individual sets of points, Set 0, Set 1 and Set 2 to be aligned. Set 1 includes a group of three points that form a single triangle. Set 2 includes a group of four points forming multiple triangles. Set 3 includes a group of three points that form a single triangle and a group of four points forming multiple triangles. Figure 613 shows respective alignments for pairs of the three sets, in the form of Set Pair (1,2), Set Pair (1,3) and Set Pair (2,3). Figure 6C shows three different alignment solutions for the combination of all three sets based on weighting factors for the three set pairs. In a first alignment solution, the weighting of Set Pair (1,2) is larger than the weighting of Set Pair (1,3) and the weighting of set pair (1,3) is larger than the weighting of Set Pair (2,3) . Therefore, the first alignment solution uses Set Pair (1,2) and (1,3) and ignores the Set Pair (2,3) . In a second solution, the weighting of Set Pair (1,2) is larger than the weighting of Set Pair (2,3) and the weighting of Set Pair (2,3) is larger than the weighting of Set Pair (1,3). Therefore, the second alignment solution uses Set Pair (1,2) and (2,3) and ignores the Set Pair (1,3). In a third alignment solution, the weighting of Set Pair (1,3) is larger than the weighting of Set Pair (2,3) and the weighting of Set Pair (2,3) is larger than the weighting of Set Pair (1,2) . Therefore, the third alignment solution uses Set Pair (1,3) and (2,3) and ignores the Set Pair (1,2).

Remove outlying transformations (step 145)

In this step, each pair of overlapping sets is processed in turn. Note that some pairs of originally overlapping sets may become non-overlapping during the step 140 to ensure consistency of orientations, and these pairs are not dealt with here.

In the six-dimensional transformation space, each transformation location represents the transformation between matching triangle pairs. Let C be a centre of a selected duster in the cluster combination being processed. A six- dimensional "bounding hyper-ellipsoid" is formed around C. A hyper-ellipsoid is an ellipsoid of three or more dimensions. For each of the six parameters, the standard deviation of the points inside the bounding hyper-ellipsoid about C are determined, and a multiple of this standard deviation is used as the semi-axis-length of an "inlying hyper-ellipsoid" . In some embodiments a constant multiple of the standard deviation for each dimension is used. Since the standard deviation for the 6 dimensions are all different, the 6 semi-axis-lengths are all different. All points outside the inlying hyper-ellipsoid are treated as outliers and ignored in the next step.

Figure 7 shows an example of the two-dimensional transform space of Figure 4B with the centre of the cluster C indicated with an "X" . The bounding hyper-ellipsoid is indicated by a dashed line and the inlying hyper-ellipsoid is indicated by a solid line.

In some embodiments, a more rigorous approach to eliminating outliers is to use a method that finds eigenvalues of a covariance matrix. However, this method only works when there are six or more points in the cluster, and is generally found not to be significantly better than the method described above .

Assign global IDs to points in sets (step 150)

This step aims to assign all points in all sets global IDs so that matching points in different sets share the same global ID. In some implementations non-negative integers, for example 0, 1, 2, ... are assigned for global IDs of matching points, and a negative integer for example -1 as assigned as a rogue global ID for all points with no match.

Step 145 finds inlying points in the six-dimensional transform space in the selected cluster for each pair of overlapping sets. Each inlying point corresponds to a matching triangle pair for that pair of sets . The triangle pairs are used to generate the global IDs.

A list of all matching triangle pairs for all pairs of overlapping sets is collected, and the list is sorted in ascending order of rms residual values. The rms residual of a pair of triangles is the minimum rms residual of the three respective pairs of vertices as one triangle is transformed towards the other. In some embodiments, matching triangle pairs higher in the list have greater merit and dominate over the lower pairs in determining the global IDs.

In some embodiments triangle pairs in the sorted list are processed in the listed order. As an example, let a particular triangle pair be (ΔP1P2P₃, AQiQ₂Q₃) where the first triangle ΔP1P2P3 is in Set i and the second triangle ΔQ₁Q₂Q₃ is in Set j . A first step is to test whether the triangle pair (-\P₁P₂P₃, ΔQ₁Q₂Q₃) should be rejected. It will be rejected if one of the following cases is true:

1. There exists a number k such that vertex P_k is assigned global ID m, the corresponding vertex Q_k is not assigned global ID m, but there is another point in Set j with global ID m. Q_k cannot be assigned global ID m, else there will be two distinct points in Set j with the same global ID, which is not permissible;

2. There exists a number k such that vertex Qk is assigned global ID n, the corresponding vertex P_k is not assigned global ID n, and there is another point in Set i with global ID n. P_k cannot be assigned global ID n, else there will be two distinct points in Set i with the same global ID, which is not permissible;

3. There exists a number k such that vertex P_k is assigned global ID and the corresponding vertex Qk is assigned a different global ID. Since these vertices have been assigned global IDs from previous iterations using the triangle pairs higher in the sorted list, we should respect these global IDs, and disregard the current triangle pair.

If none of the above is true, the triangle pair is accepted as valid and the global ID assignment for each of the three corresponding vertex pairs (i.e. for each k = 1,2,3) is performed according to the following cases:

i) If both vertices P_k and Q_k are not assigned any global ID, then give them a new global ID, for example the next available global ID;

ii) If vertex Pk is assigned global ID m, and the corresponding vertex Q_k is not, then give Q_k global ID m; iii) If vertex Q_k is assigned global ID n, and the corresponding vertex P_k is not, then give P_k global ID n;

iv) If vertices P_k and Qk are not assigned the same global ID, then do nothing.

In the situation of case 3 above, for each pair of vertices Pk and Q_k that have been assigned different global IDs m and n, it has to be determined whether the global IDs should be merged together or not. A pair of different global IDs that are being considered for merger are referred to as a "merge- pending global ID pair" . Three statistical values will be determined for each merge-pending global ID pair: the total number N of vertex pairs with these global IDs that belong to a matching triangle pair (i.e. the number of occurrences of case 3 for these global IDs) , the average A of the rms residuals of a Ll the matching triangle pairs that contain these vertex pairs, and a demerit measure B defined by A/ (square-root of N) .

All the merge-pending global ID pairs are gathered together, each having the three statistical values, and are sorted in ascending order of the value B. The merge-pending global ID pairs that belong to a greater number of well -aligned triangle pairs are given higher priority. Each global ID pair is processed in turn, and if its values of N, A and B are within a specified threshold region, and if merging does not cause any set to have different points with the same global ID (as in cases 1 and 2) , then these two global IDs are merged together by assigning global ID n to all points with global ID m. Otherwise the pairs are not merged and the points with global IDs m or n are left unchanged.

An example of assigning global IDs will be described with respect to Figures 8A-8D. Figures 8A- 8D shows a group of points labelled as A, B, C, D, E, F, G, H, I, J, K, L, M and N. For these points it is considered that pairs of the points have been arranged in descending order of rank according to the rms residual of corresponding triangle pairs associated with the pairs of points. The arrangement of pairs of points in descending order is as follows: AB, CD, EF, LM, KL BG, AH, DI, FJ, DE, BI, FI, JL and JM. It is also known that B and H are in the same set and E and K are in the same set as generally- indicated by the dashed lines around these points. In Figure 8B, based on being top ranked, pair AB is given a global ID equal to non-negative integer 0 (case i above) . Based on being second ranked, pair CD is given a global ID equal to non- negative integer 1 (case i above) . Based on being third ranked, pair EF is given a global ID equal to non-negative integer 2 (case i above) . Based on being fourth ranked, pair LM is given a global ID equal to non-negative integer 3 (case i above) . The next ranked pair KL includes point L, which was previously- given a global ID equal to 3 as part of pair LM. As there is no indication that L, K, and/or M are in the same set, K is assigned a global ID of 3 along with L and M (case ii above) . Similarly, G is assigned a global ID of 0. In the case of the pair AH, H cannot truly be a match with A, because A forms a matching pair with B, and B and H are in the same set. Therefore, the AH match is ignored and point H is given a global ID equal to -1. Point N has no match and it is also given a global ID of -1. Point I is given a global ID of 1 based on the matching pair of DI and Point J is given a global ID of 2 based on the matching pair of FJ.

In Figure 8C, the pairs DE, BI, FI are dealt with. Pair DE includes point D that has a global ID of 1 and point E that has a global ID of 2. Therefore, global ID 2 and global ID 1 become a merge-pending global ID pair. As none of points C, D and I, which have a global ID equal to 1 and none of points E, F and J, which have a global ID equal to 2 are in the same set, and their values of N, A, and B as defined above are determined to be within the specified threshold region, the points are merged and points E, F and J are assigned a new global ID equal to 1. Pair BI includes point B that has a global ID of 0 and point I that has a global ID of 1. Therefore, global ID 0 and global ID 1 become a merge-pending global ID pair (0, 1) . Points A, B and G, which have a global ID equal to 0 and points C, D, I, E, F and J, which have a global ID equal to 1 do not have any points in the same set . However, the values of N, A, and B are determined not to be within the specified threshold region and therefore the global IDs of 0 and 1 are not merged. As global IDs 1 and 2 are now merged, matching pair FI is irrelevant.

In Figure 8D, merge-pending global ID pair (1, 3) , resulting from pairs JL and JM are not merged because points E and K are in the same set, which means they cannot be assigned the same global ID.

After the global ID merge step is completed, those points that cannot be matched will have no global ID, or have a rogue global ID assigned to them, for example a value equal -1. In some embodiments, the points in each set are relabelled so that the global IDs increases in an orderly manner, e.g. the first point in the first set has global ID 0, the second point in the first set has global ID 1, and so on.

Align & merge close points and assign global IDs (step 155)

In step 155 each point in each set that matches with points in other sets are assigned global IDs. For each pair of overlapping sets, pairs of matching points are assigned the same global ID. All the sets are simultaneously aligned using these matching point pairs in such a way that the overall rms residual of the point pairs is minimized. In some implementations the transformations of the sets are stored for future use . Simultaneous point alignment is performed for all the sets even though their orientations have already been found in a previous step. This is because the orientations found previously were only approximate due to the fact that the respective alignment transform functions were determined based on an average location that is a center of the respective clusters, and because the alignment transforms for each set pair were found sequentially, not simultaneously. When performed simultaneously over multiple iterations, all the sets are adjusted at the same time until the alignment solution converges .

After the simultaneous alignment of all the sets, it is determined whether there are points in different sets that are not assigned a global ID or whether there are points in different sets that are assigned different global IDs, but in either situation are now within a particular distance threshold such that they can be assigned the same global ID. This will be described in further detail with respect to Figures 9A and 9B.

Figure 9A shows an example of three individual sets of points, Set 0, Set 1 and Set 2 to be aligned. Set 0 includes a triangle consisting of three points and a single point labelled as A. Set 1 includes a triangle consisting of three points and a single point labelled as B. Set 2 includes two groups, each including three points. Figure 9B shows an alignment solution for Set 0, Set 1 and Set 2. Each of the triangles of Set 0 and Set 1 are found to align with one of the respective triangles of Set 2. Points A and B are not exactly aligned. However, if they are within an acceptable distance threshold, they are merged and assigned the same global ID.

If neither point currently has a global ID, the two points are assigned a next available global ID. If one of the points has global ID, the other one is assigned that same global ID. If both points have global IDs that are different, all points in each set with these two global IDs are assigned the same global ID. In some embodiments, following the merging of points in close proximity, step 155 can be repeated to re- align the sets, making use of new matches.

Accept this result? (step 160)

Step 160 is a decision point used to determine whether the assignment of global IDs is acceptable. If so, the transformations for the sets found in step 155 are output. If not, steps 140,145,150,155 are repeated for a next cluster combination in the list. In some embodiments the method is used to match and align the same tie points on multiple images. After this is done, the surface are checked to determine whether of the surfaces of the object in the images are also aligned. In some embodiments this is done qualitatively by a user visually inspecting the surface after alignment is completed. In some embodiments the surface alignment with the object is determined by computing the "surface rms residual". If the surfaces align well, the result of global ID assignment is accepted. In some situations there are cases where the tie points align well but the surfaces do not, for example when there are three tie points on an image forming an equilateral triangle. In such situations a user performing a visual inspection may be more appropriate for determining whether the assignment of global IDs is acceptable.

If it is not a first iteration of steps

140,145,150,155, it is determined whether the result of global IDs obtained based on the current iteration is identical to that of previous iterations. If so, then the result has possibly been rejected previously and the method automatically goes to the next cluster combination on the list. If it is the first iteration or if the result of the current iteration is different than the previous iteration, a user will be asked whether the result should be accepted or not. It may be difficult for the user to judge from a myriad of global IDs whether the result should be accepted or not, unless the sets of points are actually contained in the surface images as tie points or GCPs.

When the method is implemented as a computer algorithm that is run on a computer processor including a display, in some embodiments the user can transform all the images by the transformations found in step 155 and display all the transformed images together (with or without stitching) in a same computer window on the display, seeing visually how well they align. In some embodiments, the user arranges for the computer processor to determine the overall surface rms residual found by the simultaneous alignment of all the images and check whether it is below some threshold value.

Select next cluster combination (step 165)

Following step 160, if the result is not acceptable, a next cluster combination is selected from the sorted list generated in step 125 and steps 140-160 are repeated. This process continues until a suitable result is determined.

Output global IDs (step 170)

In the final step, the global IDs of all matching points in all sets are output. From the output, it can be seen which point in a set matches which point in another set, based on the shared global IDs. All non-matching points may be given a rogue global ID of -1.

The algorithm may also output the rigid transformations on each set that align the sets towards the same reference coordinate system. These transformations have already been found by the simultaneous point alignment algorithm during the step "Align and merge close global IDs" 155. The inverses of these transformations are orientations of the sets with respect to the reference coordinate frame.

Figure 10 is a solution of the example of Figure 2, in which the global IDs for each point are in brackets by the local IDs.

Figure 1 is an example of a method for implementing the invention. More generally, in some embodiments of the invention some steps in Figure 1 can be omitted and the resulting method is still within the scope of the invention. Figure 11 shows another embodiment of the invention that has some steps in common with Figure 1.

At step 100 multiple sets of points on which the method will operate are input. At step 110 matching triangle pairs are formed directly from the points in the multiple sets, as opposed to using candidate edges as in Figure 1. Steps 115 and 120 are similar to Figure 1. Once the matching triangle pair transformation clusters are formed at 120, a particular cluster combination is selected 130 to be operated on, but where the clusters are not sorted according to any particular criterion, the selection may be made randomly or based on some other factor. At step 152 global IDs are assigned to points in each set. In some embodiments step 152 encompasses steps 140, 145, 150 and 155 as described above relating to Figure 1. In some embodiments steps other than those specifically described with regard to Figure 1 are used to assign global IDs to points in the multiple sets. Following step 152, steps 160 and 170 are the same as Figure 1. In step 166, another cluster is selected either randomly or based on some other factor. For example, the next best cluster may be based on sorted clusters as described above with regard to Figure 1.

In some embodiments of the invention the method may only be lacking some of the missing steps from Figure 1, but not as many as detailed in Figure 11. In other embodiments additional steps to that of Figure 1 may be included in the method to provide additional improvement to the multiple image alignment method that would be known to one skilled in the art, that are not specifically detailed here, and still be considered within the scope of the invention.

Further embodiments of the Invention

Two of the ways in which the method can be extended are to allow user intervention to reduce the search size and to make the method capable of handling cases with very few points.

For the first way of extending the operation of the method, the user informs the algorithm which pairs of sets are disjoint (i.e. with no matching points) so that the algorithm does not have to look for matching triangle pairs for such pairs of sets. A more convenient manner of providing the information is that a subfamily A of m sets is separate from another subfamily B of n sets, which implies m times n pairs of disjoint sets.

Also, the user may provide the algorithm with the information that a particular point in one set matches with a particular point in another set so that it can enforce such a fact, or at least discard any matching that is inconsistent with such a fact .

Another approach is that the algorithm can be provided with information that particular sets of points should be matched first. If the result is acceptable, the result is saved and the algorithm is called again to match points for all the sets, giving the saved matches greater weight so that they have a higher chance of appearing in a final match. In some embodiments the saved matches are not given ultimate priority over all matches as newly added sets may add, change or even iivalidate the saved matches.

The second way of extending the method is handling the two situations shown in Figures 12A and 12B. Figure 12A includes three overlapping sets Set 0, Set 1 and Set 2. Sets 0 and 1 have one matching point in common. Sets 0 and 2 have two matching points in common. Sets 1 and 2 have three matching points in common. As Sets 1 and 2 have three matching points in common, these two sets can be aligned. However, with less than three matching points between Sets 0 and 1, and Sets 0 and 2, the points in Set 0 cannot be matched and Set 0 cannot be aLigned. This is because no matching triangle pairs can be formed. One manner in which to solve this problem is to align Set 1 and Set 2 first and combine their points together to form a new set, Set 3. Then Set 0 and Set 3 will have three matching points between them and Set 0 can be aligned with respect to Set 3.

Figure 12B is more difficult to solve, since no pair o E sets have three matching points. Sets 0 and 1 have two matching points in common. Sets 0 and 2 have two matching points in common. Sets 1 and 2 have two matching points in common. In the six-dimensional transformation space, the transformation of aligning a pair of triangles corresponds to a point. The transformation to align a pair of edges corresponds to a curve with one degree of freedom as two sets can be rotated about the edge . When two such curves are concatenated in the respective six-dimensional spaces for Set Pair (0,1) and Set Pair (1,2), then a curved surface with two degrees of freedom in a new six-dimensional space is formed. This surface intersects with the curve for Set Pair (0,2) at a unique point, which gives the transformations between the three sets.

In practice, such a case can be easily avoided by adding more tie points on the object before being scanned and making sure that the images have good overlap.

In some embodiments of the invention, implementation of the invention takes the form of a computer algorithm. The algorithm provides simultaneous solution capability, as opposed to dealing with the sets in a pair-wise fashion or a sequential manner.

In some embodiments of the invention the computer algorithm is expressed as computer implemented program code that is stored on a computer readable medium. The computer implemented program code can be run from the computer readable medium by a processing engine adapted to run such computer implemented program code. Examples of types of computer readable media are portable computer readable media such as 3.5" floppy disks, compact disc ROM media, or a more fixed location computer readable media such as a hard disk media storage unit in a desktop computer, laptop computer or central server providing memory storage or a workstation.

The multiple 3D images for which embodiments of the invention will operate on are stored in a digital format on a computer readable memory.

Numerous modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the invention may be practised otherwise than as specifically described herein.

Claims

CLAIMS :

1. A method for aligning a plurality of three dimensional images of an object, each image having a respective three dimensional coordinate system and comprising a set of points related to a surface of the object, such that when aligned within an acceptable error threshold the plurality of three dimensional images share a common three dimensional coordinate system, the method comprising the steps of:

i) for each set of points, forming at least one triangle using vertices in the respective sets of points;

ii) for each respective pair of the plurality of sets of points;

a) for each triangle in each set of points of the respective pair,-

matching the triangle of a first set of points of the respective pair with a triangle of a second set of points of the respective pair to form at least one matching triangle pair;

determining a transformation function to transform an orientation of a first triangle of the matching triangle pair to an orientation of a second triangle of the matching triangle pair;

b) grouping together determined transformation functions that are substantially similar to form a respective cluster transformation function, each cluster transformation function representing a transform function for transforming an orientation of the first set of points to an orientation of the second set of points; iii) orienting all the sets of points with respect to an orientation of a single set of points based on the cluster transformation functions of the respective pairs of images;

iv) simultaneously assigning to points in at least two sets a global point identifier that indicates the same point in the at least two sets.

2. The method of claim 1 further comprising aligning the plurality of images using the global point identifiers to form a three dimensional model of the object in the common three dimensional coordinate system.

3. The method of claim 1 further comprising subsequent to determining a transformation function for each triangle in each respective pair, determining for each cluster transformation function a weighted value.

4. The method of claim 3 further comprising sorting the weighted values for the cluster transformation functions in order of greatest weighted value to least weighted value.

5. The method of claim 1 further comprising forming a plurality of transformation function cluster combinations, each transformation function cluster combination comprising a coefficient for each respective pair of sets, wherein the coefficient is a function of a weighted value of a cluster transformation function of the respective pair of sets.

6. The method of claim 5 wherein the plurality of transformation function cluster combinations are sorted in a particular order based on a product of a plurality of ratios, a ratio for each coefficient, each ratio being the ratio of the weighted value of the cluster transformation function of the pair of sets for a respective coefficient to that of a maximum weighted value for all the cluster transformation functions of the same pair of sets.

7. The method of claim 6 further comprising selecting a transformation function cluster combination from the plurality of sorted transformation function cluster combinations for use in orienting all the sets with respect to the orientation of the single set.

8. The method of claim 7 wherein orienting all the sets with respect to the orientation of the single set comprises using the respective cluster transform functions identified by the coefficients of the transformation function cluster combination for each pair of sets to determine the orientation between the first set and the second set of the pair of sets and using the orientation between each respective pair of sets to determine the orientation between all sets and the single set .

9. The method of claim 7, wherein if selecting a transformation function cluster combination from the plurality of transformation function cluster combinations for use in orienting all the sets with respect to the orientation of the single set does not result in an acceptable alignment of the plurality of three dimensional images within a common three dimensional coordinate system, selecting another transformation function cluster combination from the plurality of sorted transformation function cluster combinations for use in orienting all the sets with respect to the orientation of the single set.

10. The method of claim 1, further comprising:

prior to matching the triangle from a first set of points with a triangle of a second set of points for each set, matching edges in different respective pairs of sets, an edge defining a line between two points in the set.

11. The method of claim 10, wherein forming at least one triangle comprises using matching edges from different respective pairs of sets.

12. The method of claim 1 further comprising subsequent to matching the triangle of a first set of points with a triangle of a second set of points, determining if all sets of points are connected.

13. The method of claim 1 further comprising subsequent to simultaneously assigning a global point identifier to points in each set :

determining if there are points in sets that have not been assigned a global point identifier, but that are within a distance threshold;

aligning and merging points in different sets that are within the distance threshold;

assigning the merged points a new global point identifier.

14. The method of claim 1 wherein simultaneously assigning a global point identifier to points in each set further comprises:

assigning points with different global point identifiers a common global point identifier if:

no points having the same global point identifier are within the same set; and

quantitative parameters for the points are within a particular threshold range.

15. The method of claim 14, wherein the quantitative parameters are one or more of a group consisting of: a total number N of vertex pairs with global identifiers that belong to a matching triangle pair, an average A of root mean square (rms) residuals of all the matching triangle pairs that contain the vertex pairs, and a demerit measure B defined by A/ (square root of N) .

16. The method of claim 1 wherein selecting a transformation function comprises selecting a transform function that is represented in a six dimensional transform space comprising a rotational transform for each axis of a three dimensional coordinate system and a translational transform for each axis of the three dimensional coordinate system.

17. The method of claim 1 wherein a set of points related to a surface of the object is a set of marker locations on the object .

18. The method of claim 1 wherein a set of points related to a surface of the object is a set of marker points on a three dimensional reference object that is in close proximity to the object .

19. The method of claim 1 wherein matching the triangle from a first set of points with a triangle of a second set of points is performed according to at least one of the following conditions being met:

i) differences in lengths between corresponding edges of the triangles to be matched do not exceed four times a maximum error value ; and

ii) a root mean square (rms) residual of triangles to be matched does not exceed a predefined threshold.

20. The method of claim 19, wherein matching the triangle of a first set of points with a triangle of a second set of points is performed according to an additional condition being met as follows:

two edges of the two respective triangles are represented by normal vectors that point in a same direction with respect to a surface of the object.

21. The method of claim 1, wherein when orienting all the sets with respect to an orientation of a single set, orienting a second set with respect to a first set is performed according to at least one of the following conditions:

i) if the first set and the second set do not belong to a group, the first set is selected to have a reference orientation I, the second set is assigned an orientation α and both sets are assigned to a new group;

ii) if the first set belongs to a first group and has orientation σ, but the second set does not have orientation σ, the second set is assigned to the first group and is given an orientation of σα;

iii) if the second set belongs to a second group and has orientation τ, but the first set does not have orientation τ, then the first set is put into the second group and is assigned an orientation of τα^"1;

iv) if the first and second sets belong to the same group, having respective orientations σ and τ, then nothing is done to the first and second sets;

v) if the first and second sets belong to a first group and a second group, respectively that are different, having respective orientations σ and τ, then the two groups are merged together into one group by assigning all sets of the second group to the first group, and adjusting orientations of all sets of the second group using the orientation α between the first and second groups.

22. A computer usable medium having computer readable program code means embodied therein for aligning a plurality of three dimensional images of an object, each image having a respective three dimensional coordinate system and comprising a set of points related to a surface of the object, such that when aligned within an acceptable error threshold the plurality of three dimensional images share a common three dimensional coordinate system the computer readable code means comprising:

i) for each set of points, computer readable code means for forming at least one triangle in which three of the points in the set of points form vertices of the at least one triangle;

ii) for each respective pair of the plurality of sets of points;

a) for each triangle in each set of points of the respective pair,-

computer readable code means for matching the triangle from a first set of points of the respective pair with a triangle of a second set of points of the respective pair to form at least one matching triangle pair,-

computer readable code means for determining a transformation function to transform an orientation of a first triangle of the matching triangle pair to an orientation of a second triangle of the matching triangle pair; b) computer readable code means for grouping together determined transformation functions having a substantially similar transformation function to form a respective cluster transformation function, each cluster transformation function representing a transform function for transforming an orientation of the first set of point to an orientation of the second set of points;

iii) computer readable code means for orienting all the sets of points with respect to an orientation of a single set of points based on the cluster transformation functions of the respective pairs of sets of points;

iv) computer readable code means for simultaneously assigning to points in at least two sets a global point identifier that indicates the same point in the at least two sets.

23. The computer usable medium of claim 22 further comprising computer readable code means for aligning the plurality of set of points using the global point identifiers to form a three dimensional model of the object in the common three dimensional coordinate system.

24. The computer usable medium of claim 22 further comprising computer readable code means for determining for each cluster transformation function a weighted value.

25. The computer usable medium of claim 24 further comprising computer readable code means for sorting the weighted values for the cluster transformation functions in order of greatest weighted value to least weighted value.

26. The computer usable medium of claim 22 further comprising computer readable code means for forming a plurality of transformation function cluster combinations, each transformation function cluster combination comprising a coefficient for each respective pair of sets of points, wherein the coefficient is a function of a weighted value of a cluster transformation function of the respective pair of sets of points.

27. The computer usable medium of claim 26 further comprising computer readable code means for sorting the plurality of transformation function cluster combinations in a particular order based on a product of a plurality of ratios, a ratio for each coefficient, each ratio being the ratio of the weighted value of the cluster transformation function of the pair of sets of points for a respective coefficient to that of a maximum weighted value for all the cluster transformation functions of the same pair of sets of points.

28. The computer usable medium of claim 27 further comprising computer readable code means for selecting a transformation function cluster combination from the plurality of sorted transformation function cluster combinations for use in orienting all the sets of points with respect to the orientation of the single set of points.

29. The computer usable medium of claim 22 further comprising computer readable code means for matching edges in different respective pairs of sets of points, an edge defining a line between two points in the set.

30. The computer usable medium of claim 22 further comprising computer readable code means for determining if all sets of points are connected such that each set of points is overlapping with at least one other set of points by at least one matching triangle pair.

31. The computer usable medium of claim 22 further comprising computer readable code means for: determining if there are points in the respective sets of points that have not been assigned a global point identifier, but that are within a distance threshold;

aligning and merging points in different sets of points that are within the distance threshold;

assigning the merged points a new global point identifier.

32. The computer usable medium of claim 22, wherein computer readable code means for simultaneously assigning a global point identifier to points in each set of points further comprises computer readable code means for:

assigning points with different global point identifiers a common global point identifier if:

no points having the same global point identifier are within the same sets of points; and

quantitative parameters for the points are within a particular threshold range.