WO2003030738A1

WO2003030738A1 - 2-d and 3-d pose estimation of articles from 2-d images

Info

Publication number: WO2003030738A1
Application number: PCT/AU2002/001368
Authority: WO
Inventors: Jeremy Graham; John Stewien; Alexander Anderson
Original assignee: Integra Medical Imaging (Aust) Pty Ltd
Priority date: 2001-10-09
Filing date: 2002-10-09
Publication date: 2003-04-17
Also published as: AUPR812601A0

Abstract

A prosthesis imaging and migration tracking method and apparatus that analyses two-dimensional images, typically X-rays, to determine the position and orientation of a three-dimensional object such as prosthesis. A three-dimensional representation of the object of the implanted three-dimensional object is matched to a silhouette, so as to estimate its pose. A scaling factor can also be determined to indicate reasonably accurate distances. Once the pose of the known three-dimensional object is determined, the surrounding bone can also be identified in the two-dimensional image using object determination techniques or matching with historical or model information of the other object. Knowledge of the three-dimensional pose of the objects is used for determining the relative complex movement of the objects by comparing their poses over time. Pictorial and graphical records can than be generated to assist clinical assessment of the bone and prosthesis union. The method and apparatus of the invention can also be used on historical two-dimensional images and the larger the number of images, even those having different aspects of the same limb, the better the assessment of the migration of the objects identified.

Description

2-D AND 3-D POSE ESTIMATION OF ARTICLES FROM 2-D IMAGES

This invention relates to the automatic analysis of two-dimensional images to determine the spatial orientation of a three-dimensional object within the two- dimensional image. In particular this invention can be applied to the analysis of the bonding of medically implanted orthopedic prosthesis in human bone. Furthermore, over time the degree of loosening and migration of the implant relative to the bone can be quantified. It is also possible using the apparatus and method of the invention to assist in the classification and assessment of bone fractures.

BACKGROUND

This invention is applicable to image analysis the image itself being of any subject matter. However, for illustrative purposes only, this specification will describe image analysis in the medical field and in particular, the analysis of radiographic images of the human skeleton involving an implanted prosthesis. The medical procedures developed by surgeons for orthopaedic prosthesis implantation are now well developed and many bones and joints of the body damaged by disease and injury can now be replaced or supplemented.

The surgeon however, needs to maintain a constant watch over the development of the bonds that form between the implanted prosthesis and the natural bone. The practitioner is ever mindful of the possibility of the bonds not being formed correctly or sufficiently. The result of a less than acceptable bonding of the bone to the prosthesis or an avoidable misalignment, can range from a failure of the supplemented joint or bone to support the patient's weight and consequent collapse of the patient, to minor patient discomfort.

Clearly, the importance of reliable analysis of the bonding between bone and prosthesis requires the use of a combination of many clinical tools. The primary and cheapest tool is two-dimensional X-rays of the patient. These however can not be easily interpreted to determine with certainty the actual orientation of the implant with respect to the bone and more particularly their relative orientations from one X-ray to the next. Furthermore there can be a lack of consistency in the interpretation of these factors by different clinicians. More difficult again is the assessment of the relative movement of the prosthesis and bone over time. Nonetheless, bone growth about the prosthesis can make image comparison difficult if not totally subjective in some circumstances.

The next, but much more expensive tool is three-dimensional tomography of the patient. CAT scans being the typical apparatus used for this task will readily identify the three-dimensional form of both the implant and the bone. Not surprisingly this tool is used less than would be desired because it is so expensive.

Experienced clinicians skilled in the interpretation of X-rays and three- dimensional scans of the implant and associated bone can use these tools to improve the accuracy and usefulness of their assessment of the bone to implanted prosthesis bonding process and determine which of numerous levels of intervention are required. The earlier abnormalities can be assessed the more likely it is that remedial work can be done to rectify the developing problem.

Some levels of non-clinician assisted analysis is conducted by scan operators manually identifying consistently recognizable points or regions (landmarks) on the implant and bone after which it is possible to measure relative movements if they have occurred. Unfortunately, this approach is liable to also repeat the same errors committed by clinicians.

It is known that early intervention if properly assessed has benefits for patients as well as the community because more expensive and less debilitating medical procedures are avoided. In the worst case, the implant can be the cause of further damage to the bone. If the bone is usable again there may be a need for a further operation/s to extract the implant. A new implant is inserted or pinned on to the bone as the case may be. Importantly, frequent and reliable assessment can greatly lessen the chance that this situation will arise.

In the best case, the clinician may recommend more strenuous physiotherapy or advise the patient they can resume normal activity with the affected joint or limb.

Of importance also is the assessment of bone fracture in good bone of the affected limb or in the bone surrounding an implant caused in part by the lack of or incorrect bonding between bone and prosthesis.

Fractures are more difficult to identify. Hence, the variability of the assessed degree of fracture by the same clinician can be inconsistent. While there is also a likelihood that the same patient when assessed by different clinicians will have different fracture assessments even when the same tools are used. Unfortunately, human error and interpretation will always be present when two-dimensional and even three-dimensional imaging are the only tools available.

The migration of the prosthesis overtime longitudinally, transversally and rotationally is particularly difficult to assess when only a two-dimensional X-ray view is available. Such X-ray views are sometimes referred to as an Anterior Posterior (AP) radiograph and even with the use of filters and software, accurate migration analysis due to distortions caused by patient positioning can still occur.

An Ein Bild Roentgen Analyse (EBRA) method has been developed to reject non-comparable AP radiographs and provide some independence from the skill of the clinician to provide repeatable and reliable analysis. A further arrangement called Roentgen Stereophotgr Ammetry (RSA) involves the use of two radiographs to create a stereographic representation of the implant and limb one or both of which have X- ray opaque landmarks fitted or implanted during surgery. The analysis apparatus and the surgical elements are very expensive to purchase, to maintain and still requires accurate patient orientation during the recording process. The apparatus does not allow for the analysis of historical X-ray images.

It is an aim of the invention to provide a tool that overcomes or lessens some at least of the problems described above.

Preferably the method and apparatus of the invention will in part at least automate and make more consistent the assessment of two-dimensional images of implants in bone structures of bonding and fracture characteristics.

It is also preferable that assessment information is provided to the clinician in a format that shows trends and highlights features for further consideration. The system should be able to measure distances so that threshold distances, determined from an expert system environment, if exceeded can identify to clinicians, areas of the image deserving of further investigation and provide a way to display and thus analyze trends of migration over time.

BRIEF DESCRIPTION OF THE INVENTION

In a broad aspect of the invention A method for analysis of a two-dimensional image to determine the position of a three-dimensional object therein consisting of the steps; a) determining the form of said three-dimensional object in the two-dimensional image; b) determining the pose of a three-dimensional representation of said object that conforms to the form of the said three-dimensional object in the two-dimensional image; c) determining a first landmark with reference to said three-dimensional object; d) determining a second landmark on an other object in said two-dimensional image; and e) determining the distance between said landmarks. In a yet further aspect of the invention a method of analysis of a two- dimensional image to determine the position and orientation of two three- dimensional objects therein consisting of the further steps of: f) determining a third landmark on either said three-dimensional object or on said other object; and g) further determining the pose of said three-dimensional object with respect to said first, second and third landmarks.

In a yet further aspect of the invention a method of analysis of a two- dimensional image to determine the position and orientation of two three- dimensional objects therein consisting of the further steps of: h) repeating steps a to g at another time; and i) comparing poses obtained in step g) to determine the relative movement of said three-dimensional object.

In a broad aspect of the invention an apparatus for the analysis of a two-dimensional image to determine the position and orientation of a three-dimensional object therein consists of; a shape determining means for creating a silhouette of said three- dimensional object in said two-dimensional image; and a three-dimensional pose estimation means which matches a three-dimensional representation of said object to the two-dimensional silhouette of said object in said two-dimensional image.

In a further broad aspect of the invention an apparatus for analysis of a two- dimensional image comprises; a two-dimensional prosthesis pose estimation means for identifying a predetermined prosthesis in said two-dimensional image; a two- dimensional bone pose estimation means for identifying landmarks on said bone; and a two-dimensional measure determination means for determining a scale for said two-dimensional image and a measure of the relative distance of one or more of said landmarks from predetermined points on said prosthesis.

In a further aspect of the invention said apparatus for analysis further comprises; a three-dimensional prosthesis pose estimation means for identifying a predetermined prosthesis in said three-dimensional image; a three-dimensional bone pose estimation means for identifying a bone pose; and a three-dimensional measures determination means for determining the relative pose of said bone to said prosthesis.

In a yet further aspect of the invention wherein over time the three- dimensional measures determination means provides the components of axial rotation, rotational and /or distal migration of the complex movement of said prosthesis relative to said bone.

Specific embodiments of the invention will now be described in some further detail with reference to and as illustrated in the accompanying figures. These embodiments are illustrative, and not meant to be restrictive of the scope of the invention. Suggestions and descriptions of other embodiments may be included within the scope of the invention but they may not be illustrated in the accompanying figures or alternatively features of the invention may be shown in the figures but not described in the specification.

BRIEF DESCRIPTION OF THE FIGURES

Fig. 1 depicts a functional block diagram of an apparatus that is in accord with a method of analysis of a two-dimensional image to determine the relative two- dimensional position between a bone and an implanted prosthesis within the image;

Fig. 2 depicts a functional block diagram of the apparatus and method of Fig 1 supplemented with an apparatus that automates the analysis of a two-dimensional image to determine the relative two-dimensional position between a bone and an implanted prosthesis;

Fig. 3 depicts a functional block diagram of the apparatus of Figs 1 and 2 supplemented so that relative distances between the bone and prosthesis can be determined and outputs data that provides the basis for a three-dimensional presentation of the relationship between the position of a bone and an implanted prosthesis;

Fig. 4 depicts a graph showing a migration history of the anterior posterior distal movement of a prosthesis relative to a bone;

Fig. 5 depicts a representation of a methodology for improving the relative measurement accuracy of the apparatus.

Fig. 6 depicts the nature of the possible migration and rotation of a prosthesis;

Fig. 7 depicts a bone and prosthesis example and in particular identifies specific features of the said prosthesis; and

Fig. 8 depicts examples of prosthesis shoulder position determination comparing prior methods and the method of the invention.

DETAILED DESCRIPTION OF AN EMBODIMENT OF THE INVENTION

It should be understood that although the embodiment conveniently compartmentalizes the apparatus and proscribes a complementary method, it does not limit the ability for the invention to be provided by a different arrangement of the same or different functions using alternative methods.

Neither is the invention limited in its application to the analysis of the bonding of medically implanted orthopaedic prosthesis to human bone as is disclosed in detail herein.

The inventors have identified and describe by way of example, an apparatus and method for the determination of the relative orientation of bone and implant, and over time track the effectiveness of bonding between the two. In this specification and in particular in the description of the embodiment, the two- dimensional and three-dimensional position and orientation of the bone and implant will be referred to as its "pose".

In one aspect of the invention as disclosed in this embodiment, there has been identified a need to determine the three-dimensional position and orientation of an implant with respect to the surrounding bone using a single 2-dimensional view of those objects.

A patient having bone or joint problems presents to a clinician, X-rays and three-dimensional representations of the relevant body portion (including the affected bone) are obtained almost as a matter of course as an aid to the diagnosis of the problem.

An analysis of the medical condition of the patient may conclude the need for surgery and their suitability for prosthesis implant is undertaken and a suitable prosthesis is chosen. As part of the preparation for use of this embodiment of the invention, a three-dimensional digital representation of the chosen prosthesis may also be created but this is not always done.

The prosthesis placement operation is performed and preferably during the following months X-rays are taken at convenient times. The taking of a X-ray immediately following the operation greatly improves the detection of early failure of the bonding between bone and implant but X-rays become less frequent if no problems to the patient present themselves.

It is those 2-d X-rays taken over time and the three-dimensional prosthesis and bone data that can be used by the present invention to provide automatic and semiautomatic analysis of the bone/prosthesis union and other characteristics. The consequent presentation by the apparatus of the invention of prosthesis migration information is useful for the clinician in assessing the condition over time of the union between bone and the prosthesis.

The inventors have determined that at least in this embodiment, it is preferable to break down the analysis process into steps.

As stated previously, an X-ray image of the bone and implant is cheap relative to other types of medical imaging. Taking an X-ray of the patient following surgery will ideally provide a well-contrasted shape of the outline of a solid three- dimensional object, such as an implant against the bone it has been surgically placed on or into. This X-ray can form the basis for a particular type of analysis by the apparatus of the invention that will firstly identify the outline (silhouette) of the implant.

The identification of the bone in the X-ray is also a necessary task but will be described later in the specification.

The X-ray machine may produce a film which needs to be digitized or it may produce a digital image which itself can be used to produce a film image. Having an X-ray image provides a view of the prosthesis from only one perspective. One of the main tasks identified by the inventors in providing the invention is the determination of the actual outline of the solid object (typically the prosthesis) solely from the typically gray scale image information contained in a digitized X-ray image.

There exist a number of processes for isolating /identifying a feature boundary in a two-dimensional image.

In one embodiment, any line shape (circle, ellipses, lines) in a two-dimensional image can be identified using a Hough transform. This is merely a preferred method for achieving this aim, as there are a number of other processes that could also be used. In the example of hip joint replacement prosthesis, shapes such as the ball and edges of the stem of the implant can be identified most easily. The remainder of the implant can then be identified using other methods and known geometric characteristics of the implant itself.

Having determined the outline of the implant, the next step is to determine the three-dimensional spatial orientation of the implant that would give that outline.

There also exist a number of different processes that can provide this determination but in this embodiment, the approach is to use a modified Expectation Maximization algorithm and an alternative method is the use of the Hausdorff distance. An Expectation Maximization (EM) algorithm and the Hausdorff distance method are iterative processes for estimating truth from limited data.

The EM algorithm is statistical in nature and deals with probability density functions for unknown variables, whereas the Hausdorff distance is geometrical in nature and compares the measured distances between a plurality of predetermined points on both the image and the actual three-dimensional shape of the implant which has been previously digitized.

In this embodiment, an initial estimate of the pose of the implant is made. Once made, the apparatus assesses if a change in the pose will "improve" the fit of the silhouette of a three-dimensional implant that is evident in the digital X-ray image. The apparatus then selects a change and assesses whether the new pose created by the change improves the fit. The method then iterates these steps until convergence occurs.

The actual implementation and modification to an Expectation Maximization algorithm may include modifications to both the Generalized Expectation Maximization algorithm and the Expectation-Conditional Maximization algorithm such as the inclusion of nuisance variables (which have to be integrated out) related to the prosthesis modeling mesh as well as constraining pose changes to real rotations in three-dimensional space. These modifications are merely preferable and it is with in the skill of one in the art to implement both the EM algorithm and any modifications that are deemed useful.

A yet further method for determining the pose of the three-dimensional object (prosthesis) in a digitized two-dimensional image is to use an area-matching algorithm. The area occupied by the three-dimensional object in the two-dimensional image can be equated to, for example, the number of picture element used to store and display it, while the number of picture elements used to display the representation of the actual three-dimensional object at different poses, can also be determined. The area-matching algorithm can be implemented in a number of ways and preferably a clipped region minimization process is used to align the silhouette of the three-dimensional object in the two-dimensional image with pose of the representations of the prosthesis. Again this process is implement in steps that iterate until convergence occurs.

Once the pose of the implant has been determined from the two-dimensional X-ray image, it is then possible to use the same processes to determine the pose of the bone and thus determine the relative orientation in three dimensions of the implant to the bone. This is one way to establish a base line for future comparisons of the bone to implant union.

Alternatively, (either with the assistance of a clinician or by automatic image analysis techniques) a reference point (landmark /s) can be identified on the bone and determination can then be made in two dimensions of the position of the implant relative to the landmark. Likewise this is another way to establish a base line.

Most importantly, it does not matter whether the reference points are located on or simply related to the respective objects. That is, it is possible for reference points to be any point in the co-ordinate system of either object or an actual point on the object, which of course, in any event, is a point in a co-ordinate system of choice. Distance measurements can be made between co-ordinate systems using one of them as a reference or a common origin of the two co-ordinate systems can be created. There are many ways in which the relative movement of the objects can be measured, that is, their orientation as well for the future their relative displacement over time.

Absolute measurements (with some error) can be determined by knowing the actual dimensions of the prosthesis and the use of a scaling factor.

The invention provides much promise that clinicians will be encouraged to take relatively cheap X-rays of their patient's implants at appropriate intervals following surgery, to benefit from the increased bone /prosthesis bonding analysis that is possible using the invention. The two-dimensional information from an X-ray will form the basis for an automatically generated bone to implant bonding assessment which will include the ability to flag the need for more detailed analysis of potential problem areas. The apparatus and method of the invention can provide automatic, repeatable and consistent pose estimation of the bone and implant as well as a measure of the relative movement (lateral as well as rotational) between them.

Fig 1 depicts the first three main parts of the apparatus that is common to all other parts.

A means 10 is used to input a two-dimensional digital image to the apparatus. The image of the bone and implant of a patient is recorded. One way in which this is done, is to X-ray the relevant part of the patient. The X-ray image is digitized at a predetermined resolution direct so as to preserve as many details as possible of the image. The preferred manner is to generate at the X-ray machine a digital X-ray image of the patent having the highest manageable resolution. Some post digitizing filtering can be performed to clean up the image to better define the outlines of solid objects most particularly the prosthesis. Elimination of extraneous printed information on the image and any standard imperfections created by the imaging process are just some of the filtering that can be done. Such filters are well known in the art.

The second of the three elements of the apparatus shown in Fig. 1 is a means 12 to retrieve from digital storage, either or both of a three-dimensional digital model of the bone of the same patient that was digitized prior to the surgery and/or a three- dimensional model of the prosthesis, the later of the two being the most useful.

The three-dimensional image of the patient's bone or even a suitably manipulated contra-limb is preferably obtained from a CAT scan but it may also be possible to use a standardised bone model. The later would obviate taking the CAT scan. The raw scan is used to create a surface model in a digital-modeling format that is consistent within the remainder of the apparatus and method, and such techniques are within the skill of those in the art.

The three-dimensional model of the implant is created from the implant manufacturing files or recreated from the actual object and is typically in the form of a "net" of control points located on a virtual surface. The file representing those control points is provided in a digital modeling format that will be consistent within the remainder of the apparatus and method.

The two data files are stored in the historical data block 12 and are available for acceptance and manipulation by the two-dimensional pose estimator 14 which is the third of the elements of the first part of the apparatus disclosed in Fig. 1.

The two-dimensional pose estimator 14 may use a variety of techniques for determining pose, some of which have been described previously. Preferably edge detection algorithms are used to identify boundaries in the image having particular characteristics. For example, a Hough transform can be used and the simplest shapes to identify in most prosthesis are their straight lines. The next simplest shape to be identified depends on what type of prosthesis it is. As an example only, a hip joint replacement prosthesis has a spherical shape at one end that takes the place of the head of the femur in the hip joint. Thus, a sphere or circular shape will be readily identifiable. Generally, prosthesis shape identification is simple because the prosthesis has regular and consistent shape.

Where appropriate it is also possible to use a subtractive clipped region process as the primary or even as a secondary confirmatory process or even in combination with other processes to align the prosthesis silhouette so as to determine its pose.

As will be described later, it is also possible to advantageously improve the speed of determining a three-dimensional pose of an object by using as a starting point, the results of the two-dimensional pose 14 estimation process 22, as is depicted in Fig. 3 by the dotted line 15. However, a random starting point would also suffice however convergence will likely not be as quick as a prior process based estimation.

Furthermore, it is possible to further improve the two-dimensional measurement process 16 for determining relative position (distance) and orientation by using the results of the three-dimensional pose estimation of the prosthesis 22, as is depicted by the connection created by the dashed line 23 in Fig. 3.

Fig 1 depicts a two-dimensional measure means 16 that receives the spatial data available from the two-dimensional pose estimation means and calculates a scaling factor with the available information. The means 16 knows the actual dimensions of the prosthesis and can compare it with the two-dimensional image dimensions and thereby provide a scale for absolute and relative measurements on the two-dimensional image. Such measurements can be used when combining the method and apparatus of this invention with known EBRA methods and independently generated guidelines for comparability of two-dimensional images. Fig 2 depicts a manual two-dimensional pose estimation means 20 which allows a clinician or other skilled person to identify landmarks on the two- dimensional representation of the bone surrounding the implanted prosthesis (in this example, a femur). Preferably, more than one and likely 10-20 landmarks are identified. The use of easily identifiable landmarks is important, otherwise, as is the problem with prior arrangements, there will be a lack of comparability of images over time. As will be described later, it is also possible to use the three-dimensional representation of the bone before surgery to assists in the determination of relative position and orientation of the bone and prosthesis. No one landmark should be relied upon as clinical variance can be the cause of inconsistent results. Not unsurprisingly this situation can arise from a number of circumstances, for example, the x-rays separated in time will have slightly different perspectives of the same bone and land marks may become obscured and misidentification can occur; bone will grow and some land marks may move as a result; the accuracy associated with digital image manipulation will impinge upon a humans ability to readily identify a specific point in the 2-dimensional plane which is consistent over time.

Once an adequate number of landmarks have been identified, the form of the bone surrounding the prosthesis can be defined by that set of landmarks. Thus in the two-dimensional measure means 16 (Figs 1 and 2), the relative distance and known landmarks on the prosthesis from one or more of the landmarks can be calculated. If more than one X-ray image is available and the manual pose estimation step performed in 20 can be done for each, it is possible to more accurately determine the relative distances involved. This is in preparation for the time that the current determination is to be compared with historical image data determinations. The distances between landmarks on the femur and the prosthesis that are likely to have changed can be compared recognising that all measurement systems have error characteristics unique to that system.

Clearly, changes in relative distances are an indication of the migration of the prosthesis and these differences, if they exist or not, are illustrated using the 2-d diagnosis presentation process in 18. Typically, quantification of migration overtime is of greatest interest to clinicians and a graphical presentation not unlike that of Fig 4 is most useful. It is of course a clinical decision as to whether the quantification in light of any margins for accuracy error are indicative of acceptable or unacceptable migration for that particular patient.

It is however, advantageous to automate these processes using historical data 12 of the bone prior to or following surgery to provide the bases for the two- dimensional pose estimation of the bone. Fitting the three-dimensional bone form information into the two-dimensional image containing some portion of the bone is the first step. Account can be taken for changes to the bone that results from the surgery, such as the absence of the bone or additional growths following surgery.

Fig 3 depicts the third part of the apparatus.

Again using historical data the three-dimensional form of the prosthesis can facilitate advantageous use of the estimation methodologies described above in matching the projected silhouette of the three-dimensional prosthesis model with the silhouette of the extracted two-dimensional image of the prosthesis. The process conducted in 22 is a three-dimensional pose estimation of the prosthesis.

Again, it is also necessary to initially estimate the three-dimensional pose of the bone (femur in this example), using the same techniques discussed in relation to the two-dimensional pose estimation conducted in means 14 as alluded to earlier and indicated by dotted line 15. As an example only, the two-dimensional pose estimation can be used as a starting point for the three-dimensional pose estimation. The two-dimensional pose estimation is a likely orientation of the prosthesis, which is unlikely to be any more than 90° out along any axis. Therefore the use of the two- dimensional pose estimation as a starting point for the three-dimensional pose estimation can save a great deal of calculation. Choice of a 3-dimensional co-ordinate system for this process will likely have coincidence with the 2-dimensional co- ordinate system of the 2-d pose estimation and 2-d measures processes but need not necessarily be so. When comparisons are required suitable translations can be provided.

Not unlike the steps taken with element 20, historical data can be used along with the pose estimation methodologies described above to create a 3-d pose estimation of the bone in 24. Once the three-dimensional pose estimation of the bone (in this embodiment the femur) is complete, it is possible to calculate distances between one or more landmarks or the two objects, one being the bone and the other being the prosthesis using the 3-d measures processes in 26 for presentations using the 3-diagnosis process 28.

Therefore, it will be possible to not only accurately determine the orientation of one element (the prosthesis) relative to another element (bone /femur), it is possible to calculate migration in all the axes of possible motion and thus much better represent for the clinician the relative migration of the elements over time.

As depicted in Fig. 6 there are two types of migration (lateral with respect to the longitudinal axis of the implant and distal typically along the longitudinal axis of the implant) and one type of rotation (axial). It is the manipulation of the above calculation results into graphs or animations when presented to clinicians that will provide the greatest assistance to them. They will be better able to assess the bone prosthesis union characteristics and the result can then be relied upon for consideration as to whether an expensive CAT scan should be done and then based on all the information at hand whether further surgery should be contemplated.

Fig 4 is one example of a graph that discloses the migration of an implanted prosthesis over time in the distal direction of an AP view. The vertical axis displays migration in millimetres while the horizontal axis displays years following the operation involving the implantation of the prosthesis into the bone. The intervals between X-rays are not regular as is indicated by the crosses that denote the value of migration in millimetres at particular times following the operation. As would be expected the majority of movement occurs soon after the operation, while the degree of movement falls within a relatively narrow distance boundary as time progresses. Marked fluctuations outside an acceptable clinical variation will likely confirm patient experience and alert the clinician, sometimes though even without patient indications, that the bone prosthesis union is deteriorating. Fig 4 is but one example of a way to depict the results of 2-d and 3-d measurements taken from images collected overtime. Error bars could be added and there could be use of representative images and measurements as well as expert systems information in the form of clinically acceptable migration boundaries overtime. Features of relevance to a clinician can be identified and highlighted by using look up tables and rules.

Fig 5 depicts a proposed approach for improving the two-dimensional and three-dimensional pose estimation processes by taking more than one X-ray record of the same limb containing the prosthesis from different angles about the prosthesis. Using the two-dimensional and three-dimensional pose estimation techniques described above for both the prosthesis and the surrounding bone allows multiple calculations to occur and subsequently the two-dimensional measures as well as the three-dimensional relative distance and orientation measures can help to improve in accuracy.

There exist a number of techniques for analyzing these results but the simplest is to use an arithmetic mean of results to provide what will become clinically acceptable measures, distances and orientations.

Prior art two-dimensional techniques determine the migration of a prosthesis implant using the change in position, of a femoral landmark, along the axis of the stem of the prosthesis, relative to a fixed point on the stem. The point regularly chosen is the shoulder of the stem - in other words, the edge of the stem, as viewed in the X-ray, where its axis crosses the stem edge (refer Fig. 7). The tip and shoulder are defined with respect to the intersection points of the outer edge of the stem of the prosthesis silhouette and lie along the bisector of the prosthesis stem. The shoulder is a useful reference point on the prosthesis for use in migration determinations, as long as in prior art determinations the view of the prosthesis is taken from a consistent angle about the stem's axis.

However, as the view of the stem rotates about its axis (i.e. X-rays taken from a more lateral than AP view) the shoulder point that a two-dimensional determination alone provides, actually moves up towards the ball. In particular, when the X-ray is taken from a lateral view, the shoulder cannot be seen at all, as the ball of the implant obscures it. In this case, the images cannot be used for tracking. The difference between a prior art determination of the location of a shoulder is compared with the actual position of the shoulder and is depicted in Fig. 8.

The method and apparatus of the invention determines the rotation of the view of the prosthesis about its axis, which, amongst other determinations, enables determination of an exact three-dimensional view of the prosthesis. By mapping the prosthesis model back on to the image, it is thus possible to determine the exact position of the axis and shoulder point of the prosthesis in the image. Therefore it is possible to more accurately determine the actual migration of the shoulder with respect to another landmark on the bone of the patient. Also, with the determinations made using the method and apparatus of the invention, all X-rays, whether AP or lateral, can be used to calculate any amount migration that may have occurred.

As is described as a preferable approach, a single, manually marked reference point on the femur can be used to calculate migration relative to the shoulder point on the prosthesis. The point is either derived from the two-dimensional techniques of the invention or makes use of the three-dimensional pose determination to give a better 2-dimensional result. The error due to manual point marking is approximately in the range of ±0.4mm. This has been calculated using the fact that the marker can be placed 2 to 3 picture elements away from the desired landmark, and that the images are generally processed at a resolution of around 0.1 to 0.2mm per picture elements.

It will be appreciated by those skilled in the art, that invention is not restricted in its use to the particular application described. Neither is the present invention restricted as preferred embodiment with regard to the particular elements and /or features described or depicted herein. It will be appreciated that various modifications can be made without departing from the principles of the invention. Therefore, the invention should be understood to include all such modifications within its scope.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:

1. A method for analysis of a two-dimensional image to determine the position of a three-dimensional object therein consisting of the steps; a) determining the form of said three-dimensional object in the two- dimensional image; b) determining the pose of a three-dimensional representation of said object that conforms to the form of the said three-dimensional object in the two-dimensional image; c) determining a first landmark with reference to said three-dimensional object; d) determining a second landmark on an other object in said two- dimensional image; and e) determining the distance between said landmarks.

2. A method according to claim 1 consisting of the further steps of; f) determining a third landmark on either said three-dimensional object or on said other object; and g) further determining the pose of said three-dimensional object with respect to said first, second and third landmarks.

3. A method according to claim 2 consisting of the further steps of; h) repeating steps a to g at another time; and i) comparing poses obtained in step g) to determine the relative movement of said three-dimensional object.

4. A method according to claim 3 where in step a) the form is determined using an edge detection algorithm to identify the silhouette of predetermined shapes of said three-dimensional object in the two-dimensional image.

5. A method according to claim 3 where in step b) the pose is determined by iterative matching of the area of the two-dimensional image of the three-dimensional object in the two-dimensional image against the area of the three-dimensional representation of the three-dimensional object in different poses.

6. A method of analysis of a two-dimensional image to determine the position and orientation of two three-dimensional objects therein consisting of the steps of: a) determining the form of both three-dimensional objects in the two dimensional image; b) determining the pose of both three-dimensional objects; c) determining the relative pose of one said object to the other.

7. A method according to claim 6 consisting of the further steps of; d) repeat steps a to c at another time; and e) comparing the relative pose obtained in step d) with the relative pose obtained in step c) to determine the relative movement of said three- dimensional objects.

8. An apparatus for the analysis of a two-dimensional image to determine the position and orientation of a three-dimensional object therein consists of; a shape determining means for creating a silhouette of said three-dimensional object in said two-dimensional image; and a three-dimensional pose estimation means which matches a three-dimensional representation of said object to the two-dimensional silhouette of said object in said two-dimensional image.

9. An apparatus for analysis of a two-dimensional image according to claim 8 further consists of; a two-dimensional prosthesis pose estimation means for identifying a predetermined prosthesis in said two-dimensional image; a two-dimensional bone pose estimation means for identifying landmarks on said bone; and a two-dimensional measure determination means for determining a scale for said two-dimensional image and a measure of the relative distance of one or more of said landmarks from predetermined points on said prosthesis.

10. An apparatus for analysis of a two-dimensional image according to claim 9 further consists of: a three-dimensional prosthesis pose estimation means for identifying a predetermined prosthesis in said three-dimensional image; a three-dimensional bone pose estimation means for identifying a bone pose; and a three-dimensional measures determination means for determining the relative pose of said bone to said prosthesis.

11. An apparatus for analysis of a two-dimensional image according to claim 10 wherein over time the three-dimensional measures determination means provides the components of axial rotation, rotational and /or distal migration of the complex movement of said prosthesis relative to said bone.