US20100177946A1 - Semi-automatic contour detection - Google Patents

Semi-automatic contour detection Download PDF

Info

Publication number
US20100177946A1
US20100177946A1 US12/451,510 US45151008A US2010177946A1 US 20100177946 A1 US20100177946 A1 US 20100177946A1 US 45151008 A US45151008 A US 45151008A US 2010177946 A1 US2010177946 A1 US 2010177946A1
Authority
US
United States
Prior art keywords
contour
vertebrae
image
conditional
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/451,510
Inventor
Marleen De Bruijne
Juan Eugenio Iglesias
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bioclinica Inc
Original Assignee
Nordic Bioscience Imaging AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from GB0709599A external-priority patent/GB0709599D0/en
Priority claimed from GB0710540A external-priority patent/GB0710540D0/en
Priority claimed from GB0718369A external-priority patent/GB0718369D0/en
Application filed by Nordic Bioscience Imaging AS filed Critical Nordic Bioscience Imaging AS
Assigned to Nordic Bioscience Imaging A/S reassignment Nordic Bioscience Imaging A/S ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: IGLESIAS, JUAN E., DE BRUIJNE, MARLEEN
Publication of US20100177946A1 publication Critical patent/US20100177946A1/en
Assigned to SYNARC INC. reassignment SYNARC INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NORDIC BIOSCIENCE IMAGING A/S, A.K.A. SYNARC IMAGING TECHNOLOGIES A/S
Assigned to BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT reassignment BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT SECURITY AGREEMENT Assignors: SYNARC INC.
Assigned to BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT reassignment BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT NOTICE OF SECURITY INTEREST IN PATENTS Assignors: SYNARC INC.
Assigned to SYNARC INC. reassignment SYNARC INC. TERMINATION AND RELEASE OF SECURITY INTEREST IN U.S. PATENTS (REEL 027282, FRAME 0959) Assignors: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Assigned to SYNARC INC. reassignment SYNARC INC. TERMINATION AND RELEASE OF SECURITY INTEREST IN UNITED STATES PATENTS (RELEASES REEL 030157 FRAME 0547) Assignors: BANK OF AMERICA, N.A.
Assigned to BIOCLINICA, INC. reassignment BIOCLINICA, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SYNARC INC.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/143Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/149Segmentation; Edge detection involving deformable models, e.g. active contour models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/755Deformable models or variational models, e.g. snakes or active contours
    • G06V10/7553Deformable models or variational models, e.g. snakes or active contours based on shape, e.g. active shape models [ASM]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10116X-ray image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30008Bone
    • G06T2207/30012Spine; Backbone

Definitions

  • the present invention relates to a method of locating the contour of a structure in an image.
  • Osteoporosis is a bone disease in which the bone mineral density (BMD) is reduced, bone micro-architecture is disrupted, and the amount and variety of non-collagenous proteins in bone is altered. Bones affected by the disease are more likely to fracture. Osteoporosis is defined by the World Health Organisation (WHO) as either a bone mineral density 2.5 standard deviations below peak bone mass (20-year-old sex-matched healthy person average) as measured by dual X-ray absorptiometry (DXA), or any fragility fracture. Due to its hormonal component, more women, particularly after menopause, suffer from this disease than men.
  • WHO World Health Organization
  • Osteoporotic fractures are those that occur under slight amounts of stress that would not normally lead to fractures in non-osteoporotic people. Typical fractures occur in the vertebral column, hip and wrist. Vertebral fractures are the most common ones. Occurring in younger patients, they are a good indicator of the risk of future spine and hip fractures. These two are the most serious cases, leading to limited mobility and possibly disability. Hip fractures, in particular, usually require prompt surgery, which has other important risks associated, such as deep vein thrombosis and pulmonary embolism. Although osteoporosis patients have an increased mortality rate due to the complications of fractures, most patients die with the disease rather than of it.
  • Vertebral fractures are conventionally detected and graded on lateral X-rays. Apart from the subjective judgement of the image by a radiologist, a standard six-point morphometry is commonly used. In this technique, six landmarks are placed on the corners and in the middle point of both vertebra endplates, defining the anterior, middle and posterior heights. These measurements can be used to calculate a fracture grade. In current clinical trials, a fractured vertebra is defined as the one for which one of the three heights is at least 20% larger than any other.
  • the six-point representation captures most of the important information in the image, but it is unable to capture certain structures (such as osteophytes) or subtle shape variations.
  • Use of a full contour would overcome this problem.
  • manually annotating the full contour of the vertebrae represents a huge amount of work. Accordingly, many methods have been proposed to segment vertebrae automatically or semi-automatically.
  • Gardner et al discloses a semi-automated system, based on active contour (snake) modelling of vertebrae. Points on the vertebral boundary are specified by a user on a digital image of a vertebra. The selected boundary points then act as physical constraints on an active contour that is automatically fitted to the vertebra boundary. Various parameters are monitored and adjusted to provide an appropriate degree of freedom for the snake. Too much freedom can result in an improbable shape as the snake may be misled by an unclear edge of the vertebra and too little freedom can result in sharp turns in the contour, for example the corners of the vertebra or an osteophyte, being missed.
  • Active shape models have the advantage that they make use of prior knowledge of vertebral shape and appearance, and therefore do not need to rely as much on the information in the images.
  • Zamora et al (Medical Imaging: Image Processing, Vol 5032 of Proceedings of SPIE, SPIE Press, 2003, pp 631-642) used an active shape model (ASM) including grey level edge information, initialising it with a customised implementation of the generalised Hough transform. They applied the method on x-ray images and achieved errors lower than 6.4 mm in 50% of the lumbar images.
  • Smyth et al also used ASM methods to segment vertebrae in dual energy x-ray absorptiometry (DXA). They require the user to annotate the mid-points of the bottom of L4, top of T12 and top of T7. They achieved a root mean square (RMS) point-to-line error lower than 1.23 mm in 95% of the cases in healthy vertebrae, and lower than 2.24 mm in 92% of the fractures.
  • DXA dual energy x-ray absorptiometry
  • Roberts et al (Volume 2750 of LNCS, Springer, 2005, pp 733-840) incorporated an active appearance model (AAM) and a dynamic ordering algorithm to segment the vertebrae in DXA images. Requiring the user to mark the same three points as Smyth et al., they achieved a point-to-line error of 0.79 mm. Better results have been achieved by Roberts et al (Proceedings of Medical Image Understanding and Analysis, 2006, Vol.
  • a method of locating a contour of a structure in an image by processing said image including the structure comprising the steps of:
  • the invention is generally applicable to any structure in an image, for example, an x-ray, computer tomography or magnetic resonance image, which has a generally predictable shape and where there is some difference in light intensity between the structure and the background of the image.
  • Previous images that include similar structures can then be used in the training of a statistical model that is then used to locate the contour of a structure in a new image.
  • the method may be used for locating body parts and specifically bones.
  • bones generally have a predictable shape such that a statistical model may be used with assistance from manual annotations on an image of a bone, to locate the contour of a bone.
  • the structure is a vertebra and the image is of part of a spine.
  • the method is described, hereinafter, with reference to its application in locating the contour of a vertebra.
  • the method is equally applicable to extracting a contour of any structure in an image for which a general expectation of its shape is deducible by statistical analysis of the contours of similar structures in a set of such images.
  • the described method may also be used to locate the contour of bones in x-ray images of a hand or wrist, or to locate the contours of chambers of a heart or heart walls in MR or CT scans.
  • the vertebra has annotated on it four to eight, for example six, landmark positions, one on each of the corners and one in the middle point of each vertebra endplate.
  • Annotating vertebrae with six landmarks as described above is current standard practice. As a result, compared to standard practice, no additional manual work is required.
  • the method comprises training the statistical model of the vertebra using information from points approximating respective contours of a set of other vertebrae. It will be appreciated that any number of points may be used that provides an approximation of the contour of the vertebrae. However, in a preferred embodiment, 20 or more points are used to approximate the contour, for example, more than 40, more than 50 or more than 60 points.
  • the method further comprises training the statistical model using information from three to ten, for example, four to eight, landmark positions annotated on vertebrae in said set of other vertebrae.
  • landmark positions are annotated on the set of vertebrae.
  • the set of vertebrae used in training the statistical model includes unfractured and fractured vertebrae.
  • the statistical model is enabled to locate the edges of fractured vertebrae as well as unfractured vertebrae. Vertebrae with osteophytes may also be included in the training stage to increase the likelihood of finding osteophytes in new images with this method.
  • the statistical model is a conditional point distribution model.
  • other statistical shape models may alternatively be used, for example, statistical models based on spherical harmonics, Fourier descriptors, distance maps, image warps and volumetric or medial representations.
  • conditional point distribution model is constructed from information approximating the respective contours of a set of vertebrae and information of three to ten, four to eight, or six landmarks annotated on each member of said set of vertebrae.
  • conditional point distribution model is constructed from a first point distribution model constructed from information approximating the respective contours of said set of vertebrae and a second point distribution model constructed from information of three to ten, four to eight, or six landmarks annotated on each member of said set of vertebrae.
  • the initial estimate of the contour is the mean of the conditional point distribution model fitted to the landmark positions.
  • the iterative movement of the estimated contour is constrained by its proximity to the current estimate of the contour. Additionally and/or alternatively, the iterative movement is constrained by the conditional covariance. These constraints reduce the search space around the initial estimate and result in plausible shapes.
  • the movement of the contour boundary is constrained by restricting divergence of grey level information derived from points adjacent the estimated contour with equivalent information derived from said statistical model.
  • the iterative movement of the contour boundary is continued until the difference between the estimated contours at two consecutive iterations is smaller than a preset limit. For example, when the distance between consecutive contour estimations is less than 2 pixels or less than 1 pixel, the iterative process stops.
  • a grey level profile is built by sampling grey level information in the image along the normal to the contour across each contour point.
  • the invention has principally been defined as a method of deriving information from a digital image. However, it is of course equally applicable as an instruction set for a computer carrying out a said method or as a suitably programmed computer.
  • FIG. 1 shows an example of a vertebra with six initial landmark positions and the contour annotated
  • FIG. 2 shows an example of an initial estimate of the contour of a vertebra of an embodiment of the present invention
  • FIG. 3 illustrates the influence of the maximum allowed Mahalanobis distance on a result
  • FIG. 4 shows the distance of a contour as determined by the present invention from the real contour in the form of a histogram and cumulative distribution function
  • FIG. 5 shows some examples from leave-one-out experiments
  • FIG. 6 shows a graph of mean point-to-line error against point index that illustrates the error depending on the point number
  • FIG. 8 shows a comparison of segmentation according to the present invention, illustrating the difference between the use of standard PCA and ⁇ -PCA.
  • the present invention will hereinafter be described with particular reference to the analysis of x-ray images of vertebrae of a spine. It will, however, be appreciated that the described method could be applied to other medical images of a spine for example, DXA, Computer Tomography (CT) or Magnetic Resonance (MR).
  • CT Computer Tomography
  • MR Magnetic Resonance
  • the steps described below are equally applicable to extracting a contour of any structure in an image for which a general expectation of its shape is deducible by statistical analysis of the contours of similar structures in a set of such images.
  • the method may be applied to extracting the contour of bones in x-ray images of a hand or wrist, or in extracting the contour of chambers of the heart and the heart walls in MR of CT scans.
  • the preferred method of locating a contour of a vertebra of the present invention consists of two main steps.
  • the first step is to construct a conditional point distribution model (PDM).
  • PDM conditional point distribution model
  • two point distribution models (PDM) are constructed, a first derived from a training set of vertebrae annotated with the traditional six landmark points, and a second derived from the same training set of vertebrae annotated with a large number of points, for example, 20 or over, that approximate the actual contour outline.
  • the relationship between the two PDMs is then modelled to make it possible, for a new case, to construct a conditional PDM for the full contour depending on the position of six points that are manually annotated by a clinician.
  • the second step is then to apply this conditional PDM to a new image of a vertebra with the traditional six landmark points annotated and to approximate an initial contour of the vertebra.
  • Active shape modelling is then used to manipulate the initial contour to find the actual contour of the vertebra subject to the constraints of the conditional PDM covariance.
  • the training set used in this specific example consisted of information of full contours of vertebrae and sets of six landmark positions of vertebrae from 142 patients. Where, as a result of imaging and projection of a vertebra in the image, a vertebra was shown to have two contours, the lower one was always chosen. Vertebrae L1, L2, L3 and L4 were analysed, so 568 vertebrae (including 64 fractured vertebrae) were included in the study.
  • the images were 12-bit deep and their resolution was 570 DPI. All images were stored in DICOM format. As the application did not require such a high resolution, the images were smoothed with a Gaussian kernel and down-sampled by a factor of five.
  • the six landmark positions and the contours were marked for training purposes by three different radiologists.
  • the radiologist who marked the landmark positions on an image would always annotate the contour too.
  • the corners are marked first and then the perpendicular bisector of the segment joining the upper corners is displayed. It serves as a guide for the radiologist, who is supposed to place a landmark on the point of minimum height, and if it is unclear, as close to the bisector as possible. The process is then repeated for the lower plate.
  • the displayed bisectors help the radiologists be consistent throughout the annotation process, minimizing the impact of interobserver variability in the PDM.
  • the radiologists In order to annotate the full contour, the radiologists drew a polygonal line with as many vertices as they wanted. This contour was used as the ground truth for the study. As can be seen in FIG. 1 , the six landmarks and the contours were annotated in different passes without showing the earlier annotation, so they do not necessarily overlap.
  • the contour points must be placed on the vertebrae of the training image at equivalent points of the vertebrae.
  • the number of points can be arbitrarily chosen by the radiologist, however, it has been shown that 20 or more points is sufficient to mark the outline of the contour.
  • the maximum number of points annotated by a radiologist was 53.
  • the contour model would consist of 67 points. This allows for an equal number of points between landmarks. It will, however, be appreciated that a different, fixed, number of points could be used, provided that the number of points allows a sufficient degree of accuracy for the outline of the contour. Accordingly, to arrive at 67 points, the contour was completed, using the points marked by the radiologist, and then the 67 points were assigned to the contour. In the described embodiment, the points of the contour closest to the initial six landmark positions were chosen to be points 1 , 13 , 25 , 43 , 55 and 67 . The rest of the contour points were equidistantly placed between these six. The third segment has 50% more points because it is on average (approximately) 50% longer than the other four. A sample image is shown in FIG. 1 . Here the six initial landmark positions are marked as stars and the contour with selected points are marked as asterisks.
  • each ⁇ i p is the full Procrustes fit of ⁇ i onto ⁇ circumflex over ( ⁇ ) ⁇ .
  • the full Procrustes estimate of the mean shape [ ⁇ ] can be found as the eigenvector corresponding to the largest eigenvalue of the complex sum of squares and products matrix:
  • x [x 1 , x 2 , . . . , x N , y 1 , y 2 , . . . , y N ] t
  • PCA Principal Component Analysis
  • Each shape can then be approximated by
  • b is a column vector of k components, representing the projection of the shape onto the space of the model.
  • the mean of this vector will be zero, and the covariance C will be a diagonal matrix including the k eigenvalues.
  • PCA is an orthogonal linear transformation that spans a subspace which approximates the data optimally in a least-squares sense. This is accomplished by maximising the variance of the transformed coordinates.
  • C is the cost
  • N is the number of training cases
  • x i are the centred data vectors to approximate.
  • V i is the set of binary variables.
  • is a twice differentiable function such that ⁇ (x 2 ) is convex.
  • is twice differentiable makes it possible to use Hessian-based methods in the optimisation, providing quadratic convergence.
  • the convexity requirement ensures the existence of just one minimum for C.
  • the ⁇ -PCA which consists of searching the basis vectors P that minimise the cost function in the original equation (??), can be performed. Numberical methods will be required in both minimising C and C ⁇ , as there is no closed-form expression for ⁇ ⁇ or P.
  • ⁇ n+1 ⁇ ⁇ n ⁇ ⁇ [HC ⁇ ⁇ ( ⁇ n ⁇ )] ⁇ 1 ⁇ C ⁇ ⁇ ( ⁇ n ⁇ ),
  • Line search can then be used with a normal PCA as initialisation in order to quickly keep the optimal P.
  • different step sizes are probed at each iteration, keeping the one that leads to the minimum value of the cost function C. It is important to mention that the orthonormality condition, which would simplify the expressions of the cost and the gradient, cannot be assumed throughout the process, as the P matrix is being modified unconstrainedly (even though it converges to an orthonormal matrix).
  • a set of landmarks is defined on a set of previously aligned shapes.
  • One data vector s i is built per shape by stacking of the x and y coordinates of the landmarks.
  • the mean is subtracted from them and PCA performed on the resulting x i data vectors, aiming at representing the shapes with a lower dimensionality and with a higher specificity than the explicit Cartesian coordinates, at the expense of a certain approximation error.
  • the shapes are aligned with the Procrustes method and their mean calculated. Rotation, translation and scaling are allowed for aligning the shapes.
  • the alignment parameters and the mean are optimised simultaneously, minimising:
  • T i (z i , ⁇ i ) represents the aligned s i shape according to the set of parameters ⁇ i .
  • the constraint ⁇ t ⁇ 1 prevents the shapes from shrinking towards zero.
  • the iterative algorithm described in Cootes et al. was used for solving the problem: 1. Normalise the size of the first shape and use it as a first estimate of the mean. 2. Align all the shapes to the current estimate of the mean. 3. Update the estimate of the mean by finding the mean of the aligned shapes. 4. Normalise the size of the new estimate of the mean. 5. Go to step 2 until convergence.
  • the mean in the third step must be found by numerically minimising the cost.
  • the alignments in the second step can be easily calculated by minimising the sum of squared distances in the standard way.
  • Another consequence of this property is that the PCA coordinates of b i of a shape can still be calculated in the same way as in the normal PCA:
  • the distribution of the principal components of the full-contour model F can be modelled as a conditional Gaussian, dependent on the principal components of the landmark positions L. If ⁇ F , ⁇ L , ⁇ F , ⁇ L are the means and covariances of the principal components for the two models across the training data, and ⁇ FL , ⁇ LF represent their cross-covariances, it is then possible to write:
  • is the conditional mean and ⁇ the conditional covariance matrix for the principal component coordinates of the full contour given the principal component coordinates L of the six points.
  • the mean of the conditional model can be used as initialisation and the covariance is useful when fitting the model to the images.
  • the conditional covariance is in general much “smaller” than the unconditional covariance of C: the differential entropy of the distribution decreases almost 10 logarithmic units from the unconditional to the conditional model (from ⁇ 13.88 to ⁇ 23.85). It is thus possible to look for the solution around the conditional mean, in a region limited by a certain value of Mahalanobis distance defined according to the new conditional covariance. The search space will hence be reduced, making it easier to fit the model and making shapes relatively far away from the six landmark positions unlikely.
  • these landmark positions will be used in the same way as in the described embodiment when calculating the conditional covariance.
  • the six landmarks in the training data are not constrained to stay on their corresponding points on the contour.
  • the lack of this constraint allows the 11-D full contour conditional shape model to represent exactly the same shapes as the non-conditional one, although with higher Mahalanobis distances and hence smaller likelihood. This is important because it allows the conditional covariance matrix to remain full-rank, and thus invertible.
  • the active shape model is an iterative algorithm that tries to fit the shape model to the contours of the vertebrae in the image.
  • the first step is to find the translation (t x ,t y ), rotation ( ⁇ ) and scale (s) parameters that best fit the corners of the six given landmark positions to the mean of the shape model.
  • These pose parameters define the transform that allows to switch between the positions of the points in the image X (in “physical” coordinates), and their positions in the shape model “normalised” coordinates x.
  • the pose parameters will be kept constant throughout the process.
  • a translation along the normal to the contour is proposed for every contour point in the model at each iteration.
  • a set of candidate positions t are selected along the normal to the contour at each contour point.
  • a grey value profile is then built for each t by sampling the grey levels in the image along the normal to the contour and around the point t.
  • the derivative is calculated for the points in the profile, and then scaled so that the sum of the absolute values of the derivative profile is one. This makes the algorithm robust against contrast variations.
  • the resulting profiles p i (t) are then compared to the ones of the training cases on the contour at the same contour position.
  • ⁇ i (t) can be calculated for each t by comparing p i (t) to the model built from the training examples (with semilength T t ):
  • ⁇ i ( t ) ( p i ( t ) ⁇ p i ) t S pi ⁇ 1 ( p i ( t ) ⁇ p i )
  • This new desired shape X+dX is translated into the normalised coordinates, becoming x+dx.
  • the shape model parameters b are then updated to fit x+dx as well as possible:
  • W s is a diagonal matrix with weights that measure the importance of each point in the fitting.
  • the weights depend on the magnitude of the displacement and on the goodness of the fit:
  • ⁇ i ′ 1 2 + ⁇ d ⁇ ⁇ X i ⁇ 2 ⁇ 1 f i ⁇ ( t m )
  • ⁇ i ⁇ i ′ ⁇ ⁇ j ′
  • b i + 1 ⁇ b i + 1 ′ , if ⁇ ⁇ d ⁇ d max ⁇ + ( b i + 1 ′ - ⁇ ) ⁇ ( d max / d ) , if ⁇ ⁇ d > d max
  • d max is the parameter that controls how free the algorithm is to fit the contour to the edges in the image.
  • a large value allows the result to move around the principal component space, which can lead to implausible solutions if the edges are not clear in the image.
  • a small value makes the algorithm rely mostly on the model, leading to more conservative solutions, closer to the mean of the distribution. This can prevent the algorithm from finding the correct solution, especially in abnormal cases with fractures or osteophytes, in which the solution is relatively far from the initialisation in the principal component space, as shown in FIG. 3 .
  • FIG. 3 shows the influence of maximum allowed by Mahalanobis distance on the result.
  • the shape model is unable to fit the contour to the osteophyte.
  • the threshold has been increased by 1.5 and the contour approximates the osteophyte better.
  • the new coordinates are easily calculated using:
  • T p represents how far from the current solution one tries to find the contour. Making this parameter too large would make the search profile too long and hence make it more likely that the algorithm captures a wrong edge, especially if this edge does not represent an implausible shape. This typically happens in the type of “double contour” cases shown in FIG. 5( b ).
  • FIG. 4 shows the distance to real contour in the form of a histogram and cumulative distribution function.
  • the RMS error was equal to 0.68 mm, while the mean error was 0.48 mm and its standard deviation was 0.48 mm. 89% of the points were located within 1 mm of the manually annotated contour, 96% within 1.5 mm and 98% within 2 mm. The average of the maximum errors in each vertebra was 1.53 mm.
  • the cut-off Mahalanobis distance is the one that affects the results the most, representing a trade-off between freedom (better approximation to outliers) and safety (lower likelihood of implausible shapes).
  • the profile lengths affect mostly the convergence speed.
  • FIG. 6 shows an illustration of the error depending on the point number.
  • the points corresponding to the six landmarks are marked with a star and the distances from the manually placed landmarks to the true contour are marked with crosses.
  • the curve has local minima around the points corresponding to the landmarks, except for the middle point of the lower endplate. This is possible because the six landmarks are not constrained to be on the contour.
  • the mean distances from these points to the contour are marked with crosses in the same figure.
  • further training cases including osteophytes could be used, or a higher weighting could be given to such cases, as with the fractures.
  • more flexibility could be allowed for the ASM around points 27 and 41 (the typical locations for osteophytes), for example by modifying the contour with some kind of smooth curve to fit the points.
  • a more complex solution could be to improve the correspondence of the points around the osteophytes.
  • Minimum description length (MDL) in which the best hypothesis for a given set of data is the one that leads to a largest compression, can be used for choosing corresponding points.
  • Results are also provided of an alternative dataset based on the alternative weighting applied to outliers using ⁇ -PCA.
  • the study is based on a dataset which consists of lateral X-rays from the spine of 141 patients. Vertebrae L1 through L4 were outlined by three different expert radiologists, providing the ground truth of the study. 65 landmarks were extracted for each vertebra using the MDL algorithm, described in Thodberg “Description Length Shape and Appearance Models”, proceedings of Information Processing in Medical Imaging (2003) Springer. The same radiologists also provided information regarding the fracture type (wedge, biconcave, crush) and grade (mild, medium, severe) for the vertebrae.
  • PCA and ⁇ -PCA were tested in an active shape model for segmenting the L1-L4 vertebrae in the images.
  • Two shape models were built, one for the six landmarks and the other for the full contour, and the relationship between the ( ⁇ -) PCA coordinates of both models fitted to a conditional Gaussian distribution.
  • a higher number of principle components was utilised, seven for the six landmarks and eleven for the complete contour, keeping approximately 98% of the total variance in both cases.
  • the mean of the conditional distribution was used as initialisation for the segmentation of the full contour.
  • the gray level information along a profile perpendicular to the contour was used to calculate a desired position for each point at the following looping.
  • the new contour can then be calculated by fitting the model to the new points.
  • the conditional covariance was used to measure the Mahalanobis distance from the new ( ⁇ -) PCA coordinates b to the conditional mean.
  • FIGS. 7( a ) and 7 ( b ) show the dependence on ⁇ of the sum of squared errors when fitting the model to labelled points.
  • the maximum error decreases with ⁇ as expected doing it faster for the fractures.
  • the mean error shows how values of ⁇ lower than one tend to increase the error in fractures, as they are no longer important in the model, and decrease it in unfractured vertebrae, even if not by much.
  • Unfractured vertebrae are in general quite well modelled already. Values larger than one initially improve the results in fractures, at the expense of making them slightly worse in unfractured vertebrae. Finally, if ⁇ increases too much, the model tends to fit merely the most unlikely cases, making the average results worse both for unfractured vertebrae and mild fractures.
  • FIG. 7 ( c ) shows that ⁇ -PCA is especially useful, clearly outperforming the normal PCA, when the number of fractures in the training set is relatively small.

Abstract

A method of locating a contour of a structure in an image by processing said image including the structure is provided. A starting set of digital data representative of the image including the structure is taken, the structure in said image having annotated on it from three to ten landmark positions. A statistical model of said structure to the landmark positions annotated on the image is fitted and an initial estimate of the contour of the structure made. Using grey level information derived from points adjacent the estimated contour the contour boundary is iteratively moved to produce a final estimate of the contour of the structure.

Description

  • The present invention relates to a method of locating the contour of a structure in an image.
  • Osteoporosis is a bone disease in which the bone mineral density (BMD) is reduced, bone micro-architecture is disrupted, and the amount and variety of non-collagenous proteins in bone is altered. Bones affected by the disease are more likely to fracture. Osteoporosis is defined by the World Health Organisation (WHO) as either a bone mineral density 2.5 standard deviations below peak bone mass (20-year-old sex-matched healthy person average) as measured by dual X-ray absorptiometry (DXA), or any fragility fracture. Due to its hormonal component, more women, particularly after menopause, suffer from this disease than men.
  • Osteoporotic fractures are those that occur under slight amounts of stress that would not normally lead to fractures in non-osteoporotic people. Typical fractures occur in the vertebral column, hip and wrist. Vertebral fractures are the most common ones. Occurring in younger patients, they are a good indicator of the risk of future spine and hip fractures. These two are the most serious cases, leading to limited mobility and possibly disability. Hip fractures, in particular, usually require prompt surgery, which has other important risks associated, such as deep vein thrombosis and pulmonary embolism. Although osteoporosis patients have an increased mortality rate due to the complications of fractures, most patients die with the disease rather than of it.
  • Vertebral fractures are conventionally detected and graded on lateral X-rays. Apart from the subjective judgement of the image by a radiologist, a standard six-point morphometry is commonly used. In this technique, six landmarks are placed on the corners and in the middle point of both vertebra endplates, defining the anterior, middle and posterior heights. These measurements can be used to calculate a fracture grade. In current clinical trials, a fractured vertebra is defined as the one for which one of the three heights is at least 20% larger than any other.
  • The six-point representation captures most of the important information in the image, but it is unable to capture certain structures (such as osteophytes) or subtle shape variations. Use of a full contour would overcome this problem. However, manually annotating the full contour of the vertebrae represents a huge amount of work. Accordingly, many methods have been proposed to segment vertebrae automatically or semi-automatically.
  • Gardner et al (SPIE Vol. 2710, February 1996, pp 996-1008) discloses a semi-automated system, based on active contour (snake) modelling of vertebrae. Points on the vertebral boundary are specified by a user on a digital image of a vertebra. The selected boundary points then act as physical constraints on an active contour that is automatically fitted to the vertebra boundary. Various parameters are monitored and adjusted to provide an appropriate degree of freedom for the snake. Too much freedom can result in an improbable shape as the snake may be misled by an unclear edge of the vertebra and too little freedom can result in sharp turns in the contour, for example the corners of the vertebra or an osteophyte, being missed.
  • Semi-automatic and automatic segmentation of the vertebrae in lateral x-ray images is a difficult task because of the nature of the images. Lateral x-ray images show the superimposition of many layers, making it difficult to distinguish the region of interest—a sagittal plane along the spine. This is the reason why many classical segmentation approaches, such as those based on region growing, fail.
  • Active shape models have the advantage that they make use of prior knowledge of vertebral shape and appearance, and therefore do not need to rely as much on the information in the images.
  • In Smyth et al (Radiology, May 1999, pp 571-578), a point distribution model (PDM) was used to represent the full contour of vertebrae, and a classifier to separate normal and fractured vertebrae. An improvement was observed in the performance of the full contour representation compared to that of the six landmarks. A similar conclusion was drawn in de Bruijne et al (“Vertebral Fracture Classification,” in Medical Imaging: Image Processing, J. P. Pluim and J. M. Reinhardt, eds., Proceedings of SPIE 6512, SPIE Press, 2007).
  • Zamora et al (Medical Imaging: Image Processing, Vol 5032 of Proceedings of SPIE, SPIE Press, 2003, pp 631-642) used an active shape model (ASM) including grey level edge information, initialising it with a customised implementation of the generalised Hough transform. They applied the method on x-ray images and achieved errors lower than 6.4 mm in 50% of the lumbar images. Smyth et al also used ASM methods to segment vertebrae in dual energy x-ray absorptiometry (DXA). They require the user to annotate the mid-points of the bottom of L4, top of T12 and top of T7. They achieved a root mean square (RMS) point-to-line error lower than 1.23 mm in 95% of the cases in healthy vertebrae, and lower than 2.24 mm in 92% of the fractures.
  • De Bruijne et al applied a fully automated method based on shape particle filtering, lowering the average point-to-line error to 1.mm. Roberts et al (Volume 2750 of LNCS, Springer, 2005, pp 733-840) incorporated an active appearance model (AAM) and a dynamic ordering algorithm to segment the vertebrae in DXA images. Requiring the user to mark the same three points as Smyth et al., they achieved a point-to-line error of 0.79 mm. Better results have been achieved by Roberts et al (Proceedings of Medical Image Understanding and Analysis, 2006, Vol. 1, pp 120-124), who recently reported a 0.64 mm mean point-to-line error (95% of the points 2 mm within the contour) in radiographs of healthy vertebrae. The user is required to approximately mark the center of each vertebra. The algorithm was extended to DXA images by Roberts et al (Proceeding of MICCAI Conference Workshop on joint and bone disease, 2006, Vol. 1, pp 1-8). They achieved 0.69 mm mean point-to-line errors in normal vertebrae and 0.96 in fractures.
  • According to a first aspect of the present invention, there is provided a method of locating a contour of a structure in an image by processing said image including the structure, comprising the steps of:
  • taking a starting set of digital data representative of the image including the structure, the structure in said image having annotated on it from three to ten landmark positions;
  • fitting a statistical model of said structure to the landmark positions annotated on the image, and making an initial estimate of the contour of the structure; and
  • using grey level information derived from points adjacent the estimated contour iteratively to move the contour boundary to produce a final estimate of the contour of the structure.
  • The invention is generally applicable to any structure in an image, for example, an x-ray, computer tomography or magnetic resonance image, which has a generally predictable shape and where there is some difference in light intensity between the structure and the background of the image. Previous images that include similar structures can then be used in the training of a statistical model that is then used to locate the contour of a structure in a new image.
  • The method may be used for locating body parts and specifically bones. In this respect, bones generally have a predictable shape such that a statistical model may be used with assistance from manual annotations on an image of a bone, to locate the contour of a bone.
  • In a preferred embodiment, the structure is a vertebra and the image is of part of a spine. The method is described, hereinafter, with reference to its application in locating the contour of a vertebra. However, it will be appreciated that the method is equally applicable to extracting a contour of any structure in an image for which a general expectation of its shape is deducible by statistical analysis of the contours of similar structures in a set of such images.
  • For example, the described method may also be used to locate the contour of bones in x-ray images of a hand or wrist, or to locate the contours of chambers of a heart or heart walls in MR or CT scans.
  • Providing multiple landmark positions on a vertebra to which a statistical model may then be fitted provides a good starting point for the iterative process of locating the edge of the vertebra. Preferably, the vertebra has annotated on it four to eight, for example six, landmark positions, one on each of the corners and one in the middle point of each vertebra endplate. Annotating vertebrae with six landmarks as described above is current standard practice. As a result, compared to standard practice, no additional manual work is required.
  • In an embodiment, the method comprises training the statistical model of the vertebra using information from points approximating respective contours of a set of other vertebrae. It will be appreciated that any number of points may be used that provides an approximation of the contour of the vertebrae. However, in a preferred embodiment, 20 or more points are used to approximate the contour, for example, more than 40, more than 50 or more than 60 points.
  • Preferably, the method further comprises training the statistical model using information from three to ten, for example, four to eight, landmark positions annotated on vertebrae in said set of other vertebrae. In a preferred embodiment six landmark positions are annotated on the set of vertebrae.
  • In a preferred embodiment, the set of vertebrae used in training the statistical model includes unfractured and fractured vertebrae. By including fractured vertebrae in the training, the statistical model is enabled to locate the edges of fractured vertebrae as well as unfractured vertebrae. Vertebrae with osteophytes may also be included in the training stage to increase the likelihood of finding osteophytes in new images with this method.
  • In an embodiment, the statistical model is a conditional point distribution model. However, it will be appreciated that if the mathematical process is appropriately adapted, other statistical shape models may alternatively be used, for example, statistical models based on spherical harmonics, Fourier descriptors, distance maps, image warps and volumetric or medial representations.
  • Preferably, the conditional point distribution model is constructed from information approximating the respective contours of a set of vertebrae and information of three to ten, four to eight, or six landmarks annotated on each member of said set of vertebrae.
  • Additionally, and/or alternatively, the conditional point distribution model is constructed from a first point distribution model constructed from information approximating the respective contours of said set of vertebrae and a second point distribution model constructed from information of three to ten, four to eight, or six landmarks annotated on each member of said set of vertebrae.
  • In a preferred embodiment, the initial estimate of the contour is the mean of the conditional point distribution model fitted to the landmark positions.
  • Preferably, the iterative movement of the estimated contour is constrained by its proximity to the current estimate of the contour. Additionally and/or alternatively, the iterative movement is constrained by the conditional covariance. These constraints reduce the search space around the initial estimate and result in plausible shapes.
  • Additionally and/or alternatively, the movement of the contour boundary is constrained by restricting divergence of grey level information derived from points adjacent the estimated contour with equivalent information derived from said statistical model.
  • Preferably, the iterative movement of the contour boundary is continued until the difference between the estimated contours at two consecutive iterations is smaller than a preset limit. For example, when the distance between consecutive contour estimations is less than 2 pixels or less than 1 pixel, the iterative process stops.
  • In an embodiment, a grey level profile is built by sampling grey level information in the image along the normal to the contour across each contour point.
  • The invention has principally been defined as a method of deriving information from a digital image. However, it is of course equally applicable as an instruction set for a computer carrying out a said method or as a suitably programmed computer.
  • Embodiments of the present invention will hereinafter be described, with reference to the accompanying diagrams, in which:
  • FIG. 1 shows an example of a vertebra with six initial landmark positions and the contour annotated;
  • FIG. 2 shows an example of an initial estimate of the contour of a vertebra of an embodiment of the present invention;
  • FIG. 3 illustrates the influence of the maximum allowed Mahalanobis distance on a result;
  • FIG. 4 shows the distance of a contour as determined by the present invention from the real contour in the form of a histogram and cumulative distribution function;
  • FIG. 5 shows some examples from leave-one-out experiments;
  • FIG. 6 shows a graph of mean point-to-line error against point index that illustrates the error depending on the point number;
  • FIG. 7 shows a graph of sum of squared errors against α, where a) shows the dependence of the mean sum of squared errors with α, b) shows the dependence of the maximum sum of squared errors with α and c) shows the dependence of the mean sum of squared errors with the number of fractures present in the training set for α=1.75; and
  • FIG. 8 shows a comparison of segmentation according to the present invention, illustrating the difference between the use of standard PCA and α-PCA.
  • The present invention will hereinafter be described with particular reference to the analysis of x-ray images of vertebrae of a spine. It will, however, be appreciated that the described method could be applied to other medical images of a spine for example, DXA, Computer Tomography (CT) or Magnetic Resonance (MR).
  • All of the steps described below are equally applicable to extracting a contour of any structure in an image for which a general expectation of its shape is deducible by statistical analysis of the contours of similar structures in a set of such images. For example, the method may be applied to extracting the contour of bones in x-ray images of a hand or wrist, or in extracting the contour of chambers of the heart and the heart walls in MR of CT scans.
  • The preferred method of locating a contour of a vertebra of the present invention consists of two main steps.
  • The first step is to construct a conditional point distribution model (PDM). For this, initially two point distribution models (PDM) are constructed, a first derived from a training set of vertebrae annotated with the traditional six landmark points, and a second derived from the same training set of vertebrae annotated with a large number of points, for example, 20 or over, that approximate the actual contour outline. The relationship between the two PDMs is then modelled to make it possible, for a new case, to construct a conditional PDM for the full contour depending on the position of six points that are manually annotated by a clinician.
  • The second step is then to apply this conditional PDM to a new image of a vertebra with the traditional six landmark points annotated and to approximate an initial contour of the vertebra. Active shape modelling is then used to manipulate the initial contour to find the actual contour of the vertebra subject to the constraints of the conditional PDM covariance.
  • The two steps above will now be described in more detail with reference to a specific example.
  • The training set used in this specific example consisted of information of full contours of vertebrae and sets of six landmark positions of vertebrae from 142 patients. Where, as a result of imaging and projection of a vertebra in the image, a vertebra was shown to have two contours, the lower one was always chosen. Vertebrae L1, L2, L3 and L4 were analysed, so 568 vertebrae (including 64 fractured vertebrae) were included in the study.
  • The images were 12-bit deep and their resolution was 570 DPI. All images were stored in DICOM format. As the application did not require such a high resolution, the images were smoothed with a Gaussian kernel and down-sampled by a factor of five.
  • The six landmark positions and the contours were marked for training purposes by three different radiologists. The radiologist who marked the landmark positions on an image would always annotate the contour too. In the annotation of the six landmarks, the corners are marked first and then the perpendicular bisector of the segment joining the upper corners is displayed. It serves as a guide for the radiologist, who is supposed to place a landmark on the point of minimum height, and if it is unclear, as close to the bisector as possible. The process is then repeated for the lower plate. The displayed bisectors help the radiologists be consistent throughout the annotation process, minimizing the impact of interobserver variability in the PDM.
  • In order to annotate the full contour, the radiologists drew a polygonal line with as many vertices as they wanted. This contour was used as the ground truth for the study. As can be seen in FIG. 1, the six landmarks and the contours were annotated in different passes without showing the earlier annotation, so they do not necessarily overlap.
  • A method of a preferred embodiment is based on the following steps:
  • Training:
      • six landmarks and an approximation of the contours of vertebrae are annotated on the training images by an expert radiologist;
      • the vertebrae in the data set are aligned
      • two PDMs are constructed, a first using information of an approximation of the respective contours of a set of vertebrae, and a second using information of landmark positions annotated on each of the same set of vertebrae.
    Vertebral Edge Location:
      • six landmarks are marked on the actual image of a vertebra by a radiologist
      • a conditional model based on the position of the six landmarks on the image and on the two PDMs derived in the training stage is built for analysis
      • an initial solution is estimated, using the mean of the conditional model
      • active shape modelling is used to fit the contour to the vertebra in the image, using the conditional covariance to constrain the solution.
  • These steps will now be described in further detail.
  • An embodiment of the method is described with reference to use of point distribution models. However, it will be appreciated that the mathematic processes may be adapted to enable use of other statistical models, for example, statistical shape models based on spherical harmonics, Fourier descriptors, distance maps, image warps and volumetric or medial representations.
  • As a point distribution model is to be built, the contour points must be placed on the vertebrae of the training image at equivalent points of the vertebrae. The number of points can be arbitrarily chosen by the radiologist, however, it has been shown that 20 or more points is sufficient to mark the outline of the contour. When building the point distribution model, it is necessary for the number of contour points to be the same for every case. Accordingly, resampling is necessary in the contours, as the number of points marked by radiologists is likely to be inconsistent. In the training set described in this example, the maximum number of points annotated by a radiologist was 53.
  • For convenience, in the described embodiment, it was decided that the contour model would consist of 67 points. This allows for an equal number of points between landmarks. It will, however, be appreciated that a different, fixed, number of points could be used, provided that the number of points allows a sufficient degree of accuracy for the outline of the contour. Accordingly, to arrive at 67 points, the contour was completed, using the points marked by the radiologist, and then the 67 points were assigned to the contour. In the described embodiment, the points of the contour closest to the initial six landmark positions were chosen to be points 1, 13, 25, 43, 55 and 67. The rest of the contour points were equidistantly placed between these six. The third segment has 50% more points because it is on average (approximately) 50% longer than the other four. A sample image is shown in FIG. 1. Here the six initial landmark positions are marked as stars and the contour with selected points are marked as asterisks.
  • This specific example has been described with reference to the use of six landmark positions. However, it will be appreciated that a range of between three and ten landmarks may be positioned that are still, in general, representative of the shape of the vertebra.
  • In the following step, the 6-point representations of the vertebrae were aligned in order to eliminate translation, scale and rotation deviations between point sets. The Procrustes method was used, using only the information from the corners. The complex representation of an annotation shape can be defined as:

  • w i=(x 1 +i*y 1 , x 2 +i*y 2 , . . . , x n +i*y n)T
  • and generalised Procrustes alignment can be expressed as

  • ωi pi*{circumflex over (μ)}ωiii, i=1, 2, . . . , n
  • where each ωi p is the full Procrustes fit of ωi onto {circumflex over (μ)}. The full Procrustes estimate of the mean shape [μ] can be found as the eigenvector corresponding to the largest eigenvalue of the complex sum of squares and products matrix:
  • S = i = 1 n w i w i * / w i * w i i = 1 , 2 , n .
  • The transform parameters for each vertebra were then applied on the full contours, so that both representations were aligned. Each shape was then represented by a vector, where N=6 or N=67, depending on the model:

  • x=[x1, x2, . . . , xN, y1, y2, . . . , yN]t
  • Once the training shapes were aligned, a point distribution model was built for both the six landmark positions and the full contour representations, mixing vertebrae L1, L2, L3 and L4 into a single model. In order to build the model, a Principal Component Analysis (PCA) was performed on the aligned data vectors. PCA is a technique that can be used to reduce the dimensionality of a dataset. It is a linear transformation towards a new coordinate system such that the greatest variance by any projection of the data lies on the first coordinate (known as the first principal component), the second greatest variance on the second coordinate, and so on. It is hence possible to simplify the dataset by keeping only the first principal components in the new representation. It can be shown that an orthonormal base of the new space is given by the eigenvectors of the covariance matrix of the dataset.
  • The shape model is then characterised by a mean shape χ, the kept eigenvalues λi, i=1, . . . , k and the corresponding eigenvectors, which are grouped columnwise in the P matrix, which accomplishes that PtP=I. Each shape can then be approximated by

  • x≈ x+Pb
  • where b can be calculated for a certain shape:

  • b=P t(x− x )
  • and the error vector is:

  • =x− x−Pb=x− x−P(P t(x− x ))
  • where b is a column vector of k components, representing the projection of the shape onto the space of the model. Along the training set, the mean of this vector will be zero, and the covariance C will be a diagonal matrix including the k eigenvalues.
  • In order to verify whether a certain vector b corresponds to a plausible shape, it must be checked that it is not too far away from the mean of the model, that is, the zero vector. At the same time, plausible shapes can be generated just by taking b vectors close to the zero vector. The valid region is defined by limiting the Mahalanobis distance of b. The limit dmax can be chosen using the chi-square distribution.
  • d = b t C - 1 b = i = 1 k ( b i 2 λ i )
  • If the condition d<dmax is not true, the b vector can be modified:

  • b′=b(d max /d)
  • As the number of fractures in the data set is low, compared to the number of healthy ones, their influence on the model was increased by giving them a higher weight when building it. Two different weights were given to normal and fractured vertebrae when calculating the mean and the variance of the shapes, so that their total contributions were equal. As 504 healthy and 64 fractured vertebrae were available, the weights were
  • ω h = 0.5 504 and ω f = 0.5 64 ,
  • respectively.
  • As an alternative to this method of weighting, a variation on PCA (described above) can be implemented that deals with the importance of outliers. There are cases in which a small subset of samples may appear as outliers and still correspond to plausible data. This is true where some of the data obtained of vertebrae include fractured vertebrae. These may appear as outliers but the information they provide is still important. In this case, the modelling of outliers is important and rather than disregarding them, they may be used to enhance the results.
  • PCA is an orthogonal linear transformation that spans a subspace which approximates the data optimally in a least-squares sense. This is accomplished by maximising the variance of the transformed coordinates.
  • If the dimensionality of the data is to be reduced to N, an equivalent formulation of PCA is to find the N-set of orthonormal vectors, grouped in the P matrix, which minimises the error made when reconstructing the original data points in the data set. The error is measured in a L2 norm fashion:
  • C = PP t x i - x i 2
  • where C is the cost, N is the number of training cases and xi are the centred data vectors to approximate.
  • Least-squares is not robust when outliers are present in the dataset, as they can skew the result from the desired solution, leading to inflated error rates and distortions in statistical estimates (Hampel et al.: “Robust statistics: the approach based on influence functions” Wiley 1986). It has been attempted to reduce the impact of outliers on PCA by modifying the cost in the above equation, for example, by introducing a binary variable that is zero when a data sample is considered to be an outlier:
  • C X u = i = 1 N [ V i PP t x i - x i 2 + η ( 1 - V i ) ]
  • where Vi is the set of binary variables. The term η(1−V) prevents the optimisation from converging to the trivial solution Vi=0,∀i.
  • In contrast to this and in contrast to directly minimising the squared data reconstruction error as in normal PCA, the presented Φ-PCA algorithm minimises:
  • C = Φ [ PP t x i - x i 2 ]
  • where Φ is a twice differentiable function such that Φ(x2) is convex. The fact that Φ is twice differentiable makes it possible to use Hessian-based methods in the optimisation, providing quadratic convergence. The convexity requirement ensures the existence of just one minimum for C.
  • A simple and at the same time powerful form of the function is Φ(x)=xα, with α>0.5 in order to accomplish the convexity condition. This special case will be called α-PCA. Large values for α (α>1, in general), will enhance the outliers, as they become more expensive compared to normal cases. In particular, α=∞ would lead to minimising the L norm, and hence the maximum reconstruction error over shapes measured in a L2 norm fashion. On the other hand, smaller values (0.5<α<1) will have the opposite effect, leading to a more robust PCA. The case α=0.5 minimises the L1 norm. Finally α=1 amounts to standard PCA.
  • The data points xi must be centred, which means that their means must be substracted from them: xi=si−μ, where si represents the original, non-zero mean data samples. In the proposed algorithm, the “mean” is no longer the component-wise arithmetic mean of the data points as in the standard PCA, but the vector which minimises (assuming M dimensions for the data points):
  • C μ Φ = i = 1 N Φ [ μ Φ - s i 2 ] = i = 1 N Φ [ t = 1 M ( μ t Φ - s i t ) 2 ]
  • Once the xi vectors have been calculated the Φ-PCA which consists of searching the basis vectors P that minimise the cost function in the original equation (??), can be performed. Numberical methods will be required in both minimising C and Cμ, as there is no closed-form expression for μφ or P.
  • An important difference between standard and φ-PCA is that, in the latter, the principal components have to be recalculated if the desired dimensionality changes. In standard PCA, on the other hand, the first N1 principal components are common for analyses with N1 and N2 assuming that N2>N1.
  • The expressions for the gradient and the Hessian of the cost Cμ φ are quite simple and fast to calculate. Using the component-wise arithmetic mean as initialisation, Newton's method converges rapidly to the solution:

  • μn+1 Φn Φ −[HC μ Φ n Φ)]−1 ∇C μ Φ n Φ),
  • where the gradient ∇Cμ φ is a column vector consisting of the first-order derivatives:
  • C μ Φ μ k Φ = 2 i = 1 N Φ [ l = 1 M ( μ l Φ - s i l ) 2 ] ( μ k Φ - s i k )
  • and the Hessian matrix H consists of the second-order derivatives:
  • H kk = 2 C μ Φ μ k Φ2 = 2 i = 1 N { 2 Φ [ l = 1 M ( μ l Φ - s i l ) 2 ] ( μ k Φ - s i k ) + Φ [ m = 1 M ( μ m Φ - s i m ) 2 ] } H bu = H uk = 2 C μ Φ μ k Φ μ u Φ = 4 i = 1 N Φ [ l = 1 M ( μ l Φ - s i l ) 2 ] ( μ k Φ - s i k ) ( μ u Φ - s i u )
  • Once the mean has been subtracted from the data points, the original cost C must be minimised. The function has the interesting property that it reaches its global minimum for an orthonormal P matrix such that PtP=I. This makes it possible not to have to constrain P to accomplish this condition during the optimisation, even it that implies that in general PPt will not represent a projection matrix, and hence PPtxi−xi does not express the reconstruction error any longer.
  • In the minimisation problem, only the expression for the gradient is implemented, as the one for the Hessian matrix is too complex and its computation too expensive. Using matrix calculus, all the partial derivatives can be calculated simultaneously:
  • C P = P i = 1 N Φ [ PP t x i - x i 2 ] = i = 1 N P Φ [ ( PP t x i - x i ) t ( PP t x i - x i ) ] = i = 1 N Φ [ x i t PP t PP t x i - x i t x i - 2 x i t PP t x i ] [ - 4 x i x i t P + 2 [ x i x i t PP t + PP t x i x i t ] P ]
  • Once the gradient is known, different standard techniques can be used to update P. In a simple gradient descent scheme, for example:
  • P n + 1 = P n - k C P
  • where k is the step size.
  • Line search can then be used with a normal PCA as initialisation in order to quickly keep the optimal P. In this algorithm, different step sizes are probed at each iteration, keeping the one that leads to the minimum value of the cost function C. It is important to mention that the orthonormality condition, which would simplify the expressions of the cost and the gradient, cannot be assumed throughout the process, as the P matrix is being modified unconstrainedly (even though it converges to an orthonormal matrix).
  • In shape models (Cootes et al. 1995), a set of landmarks is defined on a set of previously aligned shapes. One data vector si is built per shape by stacking of the x and y coordinates of the landmarks. Next, the mean is subtracted from them and PCA performed on the resulting xi data vectors, aiming at representing the shapes with a lower dimensionality and with a higher specificity than the explicit Cartesian coordinates, at the expense of a certain approximation error. The differences between shape models based on standard and φ-PCA will be described.
  • First, the shapes are aligned with the Procrustes method and their mean calculated. Rotation, translation and scaling are allowed for aligning the shapes. The alignment parameters and the mean are optimised simultaneously, minimising:
  • C align = i = 1 N Φ [ T i ( z i , θ i ) - μ Φ 2 ] = i = 1 N Φ [ s i - μ Φ 2 ] = i = 1 N Φ [ x i 2 ]
  • where Ti(zii) represents the aligned si shape according to the set of parameters θi. The constraint μtμ=1 prevents the shapes from shrinking towards zero. The iterative algorithm described in Cootes et al. was used for solving the problem:
    1. Normalise the size of the first shape and use it as a first estimate of the mean.
    2. Align all the shapes to the current estimate of the mean.
    3. Update the estimate of the mean by finding the mean of the aligned shapes.
    4. Normalise the size of the new estimate of the mean.
    5. Go to step 2 until convergence.
  • The mean in the third step must be found by numerically minimising the cost. However, as minimising φ(t2) is equivalent to minimising t2=∥PPtx−x∥2, the alignments in the second step can be easily calculated by minimising the sum of squared distances in the standard way. Another consequence of this property is that the PCA coordinates of bi of a shape can still be calculated in the same way as in the normal PCA:

  • s i≈μφ +Pb i

  • b i =P t(s i−μφ)
  • The distribution of the principal components of the full-contour model F can be modelled as a conditional Gaussian, dependent on the principal components of the landmark positions L. If μF, μL, ΣF, ΣL are the means and covariances of the principal components for the two models across the training data, and ΣFL, ΣLF represent their cross-covariances, it is then possible to write:

  • P(F|L)=N(μ,Σ)

  • μ=μFFLΣLL −1(L−μ L)=ΣFLΣLL −1 L

  • Σ=ΣFF−ΣFLΣLL −1ΣLF
  • where μ is the conditional mean and Σ the conditional covariance matrix for the principal component coordinates of the full contour given the principal component coordinates L of the six points.
  • It is also possible to model the position of the points and landmarks instead of the principal components. The latter has an advantage as the size of the covariance matrices is smaller. If the coordinates of the points are used directly, ΣLL might become non-invertible due to multi-collinearity in the positions, and regularisation would be required.
  • As shown in FIG. 2, the mean of the conditional model can be used as initialisation and the covariance is useful when fitting the model to the images. The conditional covariance is in general much “smaller” than the unconditional covariance of C: the differential entropy of the distribution decreases almost 10 logarithmic units from the unconditional to the conditional model (from −13.88 to −23.85). It is thus possible to look for the solution around the conditional mean, in a region limited by a certain value of Mahalanobis distance defined according to the new conditional covariance. The search space will hence be reduced, making it easier to fit the model and making shapes relatively far away from the six landmark positions unlikely.
  • In an embodiment having fewer or more than six landmarks, these landmark positions will be used in the same way as in the described embodiment when calculating the conditional covariance.
  • In the described example, the six landmarks in the training data are not constrained to stay on their corresponding points on the contour. The lack of this constraint allows the 11-D full contour conditional shape model to represent exactly the same shapes as the non-conditional one, although with higher Mahalanobis distances and hence smaller likelihood. This is important because it allows the conditional covariance matrix to remain full-rank, and thus invertible.
  • The active shape model (ASM) is an iterative algorithm that tries to fit the shape model to the contours of the vertebrae in the image. The first step is to find the translation (tx,ty), rotation (Θ) and scale (s) parameters that best fit the corners of the six given landmark positions to the mean of the shape model. These pose parameters define the transform that allows to switch between the positions of the points in the image X (in “physical” coordinates), and their positions in the shape model “normalised” coordinates x. The pose parameters will be kept constant throughout the process.
  • Starting from an initial solution, which is calculated in the “normalised” coordinates with the six landmark positions, a translation along the normal to the contour is proposed for every contour point in the model at each iteration. In order to calculate it, a set of candidate positions t are selected along the normal to the contour at each contour point. A grey value profile is then built for each t by sampling the grey levels in the image along the normal to the contour and around the point t. The derivative is calculated for the points in the profile, and then scaled so that the sum of the absolute values of the derivative profile is one. This makes the algorithm robust against contrast variations. The resulting profiles pi(t) are then compared to the ones of the training cases on the contour at the same contour position.
  • If pi(t) represents the vector of normalised derivatives around point t in the profile around contour point i (let us say in the interval [t−Tt,t+Tt]), a fitness function ƒi(t) can be calculated for each t by comparing pi(t) to the model built from the training examples (with semilength Tt):

  • ƒi(t)=(p i(t)− p i)t S pi −1(p i(t)− p i)
  • where p i is the average of the profiles of length 2Tt+1 around point i in the training cases, and Spi; is a diagonal matrix including the (independent) variances of each element in the profile. The function ƒi(t) will just be defined in the interval [−Tp,Tp]. Hence, Tp defines how far from the current point location the search for the contour is performed. If tm minimises ƒ(t), the shift
  • 2 3 t m
  • is chosen to make the algorithm evolve in a smoother way.
  • This new desired shape X+dX is translated into the normalised coordinates, becoming x+dx. The shape model parameters b are then updated to fit x+dx as well as possible:

  • (P t W S)dx=(P t W s P)db
  • where Ws is a diagonal matrix with weights that measure the importance of each point in the fitting. The weights depend on the magnitude of the displacement and on the goodness of the fit:
  • ω i = 1 2 + d X i 2 1 f i ( t m )
  • ω i = ω i ω j
  • Before updating the contour, it is important to check that db leads to a plausible shape. It is at this point that the information in the covariance of the conditional model becomes useful.
  • d = ( b i + 1 - μ ) t - 1 ( b i + 1 - μ ) b i + 1 = b i + db and then b i + 1 = { b i + 1 , if d d max μ + ( b i + 1 - μ ) ( d max / d ) , if d > d max
  • Hence, dmax is the parameter that controls how free the algorithm is to fit the contour to the edges in the image. A large value allows the result to move around the principal component space, which can lead to implausible solutions if the edges are not clear in the image. A small value makes the algorithm rely mostly on the model, leading to more conservative solutions, closer to the mean of the distribution. This can prevent the algorithm from finding the correct solution, especially in abnormal cases with fractures or osteophytes, in which the solution is relatively far from the initialisation in the principal component space, as shown in FIG. 3. Specifically, FIG. 3 shows the influence of maximum allowed by Mahalanobis distance on the result. In FIG. 3( a) the shape model is unable to fit the contour to the osteophyte. In FIG. 3( b) the threshold has been increased by 1.5 and the contour approximates the osteophyte better.
  • The new coordinates are easily calculated using:

  • x≈ x+Pb
  • and then transformed back to physical coordinates by the transform defined by −tx, −ty, −Θ, s−1. A new translation is then proposed for each point once more, starting a new iteration. When the difference between the shapes at two consecutive iterations is smaller than a certain limit, the process stops. For example, when the distance between consecutive estimated contours is less than 2 pixels or less than 1 pixel, the iterative process stops.
  • First of all, several preliminary experiments were run in order to find suitable values for the different parameters in the algorithm. Regarding the selection of the number of principal components to be kept, it depends on the proportion of the variance to be preserved. Seven and eleven components, for the six landmarks and the full contour respectively, were enough to keep approximately 97% of the total variance. From that point on, raising the proportion becomes very expensive in terms of the number of principal components.
  • Regarding the profile lengths, it was found that using a grey value model profile semilength Tt=12 pixels (2.7 mm) and making Tp=8 pixels (1.8 mm) leads to good results. Tp represents how far from the current solution one tries to find the contour. Making this parameter too large would make the search profile too long and hence make it more likely that the algorithm captures a wrong edge, especially if this edge does not represent an implausible shape. This typically happens in the type of “double contour” cases shown in FIG. 5( b).
  • Finally, the choice of the maximum Mahalanobis distance dmax was made with the help of the χ2 distribution for eleven degrees of freedom. Arbitrarily fixing a 10% tail probability, a compromise limit equal to 4.1 was picked. All the parameters were kept fixed for all the further experiments, unless additional description is provided.
  • A leave one out experiment was carried out. For each vertebra, a complete model (six landmarks, full contour, profiles) was built from the other images and used for the contour location. The distance between each point of the physician-annotated contour and the closest point in the detected contour (point-to-line distance) was calculated for each vertebra. The distribution of the error is represented in FIG. 4. Specifically, FIG. 4 shows the distance to real contour in the form of a histogram and cumulative distribution function.
  • The RMS error was equal to 0.68 mm, while the mean error was 0.48 mm and its standard deviation was 0.48 mm. 89% of the points were located within 1 mm of the manually annotated contour, 96% within 1.5 mm and 98% within 2 mm. The average of the maximum errors in each vertebra was 1.53 mm.
  • The fact that fractures were given a higher weight when building the model prevents the performance of the model from decreasing too much when fitting shapes corresponding to fractured vertebrae. The mean error for the fractures was 0.54 mm. 86% of the points were 1 mm within the real contour in fractures, and 94% within 1.5 mm.
  • The results were also judged visually, especially in cases with osteophytes and fractures, in which the contour may be oversmoothed. Some sample images are shown in FIG. 5. As can be seen, the model has difficulties approximating the ground truth when the ground truth takes an abrupt turn (like around osteophytes), but otherwise works quite well, as the quantitative results suggest.
  • It is also important to note that the results are not very sensitive to changes in the different tuned parameters. The increments in the mean error when changing the main parameters are:
  • For the profile length used for building the model: 4 μm in a 4 pixels wide region around the chosen value;
  • For the profile length used in the contour search: 6 μm in a 4 pixels wide region around the chosen value; and
  • For the cut-off Mahalanobis distance: 25 μm in a 1 unit wide region around the chosen value.
  • These values suggest that the method is robust against non-optimal choices of the parameters. The cut-off Mahalanobis distance is the one that affects the results the most, representing a trade-off between freedom (better approximation to outliers) and safety (lower likelihood of implausible shapes). The profile lengths affect mostly the convergence speed.
  • In the preferred embodiment, higher weights are given when building the model to fractures, lowering the overall mean performance but increasing the accuracy in the most difficult cases and hence lowering the largest errors, too. In spite of that, an average point-to-line error lower than 0.5 mm was achieved. 95% of the errors were lower than 1.36 mm. The performance is higher than those of the most recent systems described.
  • By annotating the six points and the full contour in different passes, the solution is not constrained to pass through the six landmarks. This adds flexibility to the shape model, making it possible for it to fit a wider range of different shapes. The fact that different radiologists made the annotations also adds some interesting variability to the model.
  • FIG. 6 shows an illustration of the error depending on the point number. The points corresponding to the six landmarks are marked with a star and the distances from the manually placed landmarks to the true contour are marked with crosses. It is clear here that the mean position error peaks clearly just after the third landmark and just before the fourth, which are the typical locations of osteophytes. The curve has local minima around the points corresponding to the landmarks, except for the middle point of the lower endplate. This is possible because the six landmarks are not constrained to be on the contour. The mean distances from these points to the contour are marked with crosses in the same figure.
  • In an embodiment, further training cases including osteophytes could be used, or a higher weighting could be given to such cases, as with the fractures. Alternatively, more flexibility could be allowed for the ASM around points 27 and 41 (the typical locations for osteophytes), for example by modifying the contour with some kind of smooth curve to fit the points. A more complex solution could be to improve the correspondence of the points around the osteophytes. Minimum description length (MDL), in which the best hypothesis for a given set of data is the one that leads to a largest compression, can be used for choosing corresponding points.
  • Results are also provided of an alternative dataset based on the alternative weighting applied to outliers using α-PCA. The study is based on a dataset which consists of lateral X-rays from the spine of 141 patients. Vertebrae L1 through L4 were outlined by three different expert radiologists, providing the ground truth of the study. 65 landmarks were extracted for each vertebra using the MDL algorithm, described in Thodberg “Description Length Shape and Appearance Models”, proceedings of Information Processing in Medical Imaging (2003) Springer. The same radiologists also provided information regarding the fracture type (wedge, biconcave, crush) and grade (mild, medium, severe) for the vertebrae. In addition, they also annotated the six landmarks used in the standard six-point morphometry, located on the corners and in the middle point of both vertebra endplates. These points define the anterior, middle and posterior heights, which are used to estimate the fracture grade and type.
  • Both normal PCA and α-PCA (for different values of α) were applied on the dataset keeping 7 (α-) PCA coordinates, capable of preserving approximately 95% of the toal variance in the data in all the cases. For both algorithms the mean and maximum squared reconstruction errors were calculated. The dependence of the error on the number of fractures in the training set was also studied. It should be noted that a higher number of components would achieve better precision and still provide a good trade-off with respect to the specificity of the model, but a smaller amount was kept in this experiment in order to better illustrate the difference between PCA and α-PCA.
  • Finally, PCA and α-PCA were tested in an active shape model for segmenting the L1-L4 vertebrae in the images. Two shape models were built, one for the six landmarks and the other for the full contour, and the relationship between the (α-) PCA coordinates of both models fitted to a conditional Gaussian distribution. In order to allow for more flexibility in the model, a higher number of principle components was utilised, seven for the six landmarks and eleven for the complete contour, keeping approximately 98% of the total variance in both cases.
  • The mean of the conditional distribution was used as initialisation for the segmentation of the full contour. At each iteration, the gray level information along a profile perpendicular to the contour was used to calculate a desired position for each point at the following looping. The new contour can then be calculated by fitting the model to the new points. The conditional covariance was used to measure the Mahalanobis distance from the new (α-) PCA coordinates b to the conditional mean. In case of it being larger than a certain threshold Dmax, the vector is scaled down b′=b(Dmax/D(b) to ensure D(b)≦Dmax. This way, the solution is constrained to stay close to the six landmarks. The process is repeated until convergence.
  • FIGS. 7( a) and 7(b) show the dependence on α of the sum of squared errors when fitting the model to labelled points. The maximum error decreases with α as expected doing it faster for the fractures. The mean error shows how values of α lower than one tend to increase the error in fractures, as they are no longer important in the model, and decrease it in unfractured vertebrae, even if not by much. Unfractured vertebrae are in general quite well modelled already. Values larger than one initially improve the results in fractures, at the expense of making them slightly worse in unfractured vertebrae. Finally, if α increases too much, the model tends to fit merely the most unlikely cases, making the average results worse both for unfractured vertebrae and mild fractures.
  • In this experiment, the model was built with all the unfractured vertebrae and different fractions of the total amount of available fractures: from 12.5% to 100% in 12.5% increments. α was set to 1.75, providing a good trade-off between the maximum and mean errors in fractured and unfractured vertebrae, according to the results presented above. FIG. 7 (c) shows that α-PCA is especially useful, clearly outperforming the normal PCA, when the number of fractures in the training set is relatively small.
  • Vertebrae L1 through L4 were segmented from the available images using a shape model conditioned on the six landmarks annotated by the radiologists, using both standard and α-PCA (α=1.75). The experiments were performed in a leave-one-out fashion: the model used for segmenting a certain image is built upon all the other ones.
  • The point-to-line errors from the true contour to the output of the algorithm are displayed in the table below, along with the p-values resulting from a paired, double-sided t-test and a paired, double-sided Wilcoxon signed rank test. The results show that the standard PCA leads to a lower mean error in unfractured vertebrae, but α-PCA provides more uniform results along the different grades of fractures severity, at the expense of a slight increase in the total mean error. Moreover, α-PCA significantly outperforms the standard PCA in fractures, especially in the severe ones. It also has the property of assigning different importance to each case in a continuous manner without requiring fracture information for the training data. If this information was available, it would be possible to build two different models, but then a large number of training fractures would be required. Regarding the p-values, both tests indicate that the difference in the means between the two setups is significant.
  • Error Error
    No. PCA α-PCA p signed
    shapes (mm) (mm) p t-test rank
    Unfractured 500 0.44 0.47 1.79 · 10−4 6.05 · 10−4
    Mild fractures 15 0.62 0.56 1.14 · 10−2 1.25 · 10−2
    Medium fractures 38 0.66 0.57 7.87 · 10−3 1.40 · 10−2
    Severe fractures 11 0.97 0.60 4.78 · 10−3 9.77 · 10−3
    All fractures 64 0.70 0.57 3.10 · 10−11 3.53 · 10−12
  • Finally, two radiographs which have been segmented with a standard and α-PCA (α=1.75) are displayed along with the contour provided by the radiologists in FIG. 8. They both correspond to sever fractures. α-PCA provides a better approximation of the real shape, especially around the points in which it changes its direction rapidly.
  • It will be appreciated that modifications of, and alterations to, the embodiments as described and illustrated may be made within the scope of this application.

Claims (16)

1. A method of locating a contour of a structure in an image by processing said image including the structure, comprising the steps of:
taking a starting set of digital data representative of the image including the structure, the structure in said image having annotated on it from three to ten landmark positions;
fitting a statistical model of said structure to the landmark positions annotated on the image, and making an initial estimate of the contour of the structure; and
using grey level information derived from points adjacent the estimated contour iteratively to move the contour boundary to produce a final estimate of the contour of the structure.
2. A method as claimed in claim 1, wherein the structure is a bone.
3. A method as claimed in claim 1, wherein the structure is a vertebra and the image is of part of a spine including said vertebra.
4. A method as claimed in claim 3, further comprising training the statistical model of the vertebra using information from points approximating respective contours of a set of other vertebrae.
5. A method as claimed in claim 4, further comprising training the statistical model using information from three to ten landmark positions annotated on vertebrae in said set of other vertebrae.
6. A method as claimed in claim 4, wherein the set of vertebrae used in training the statistical model includes unfractured and fractured vertebrae.
7. A method as claimed in claim 1, wherein the statistical model is a conditional point distribution model.
8. A method as claimed in claim 7, wherein the conditional point distribution model is constructed from information approximating the respective contours of a set of vertebrae and from information of three to ten landmarks annotated on said set of vertebrae.
9. A method as claimed in claim 7, wherein the conditional point distribution model is constructed from a first point distribution model constructed from information approximating the respective contours of a set of vertebrae and a second point distribution model constructed from information of three to ten landmarks annotated on said set of vertebrae.
10. A method as claimed in claim 7, wherein the conditional point distribution model is a conditional Gaussian dependent on the positions of the landmark positions in the image being processed.
11. A method as claimed in claim 10, wherein the conditional point distribution model is a conditional Gaussian modelling the principal components of the point distribution model constructed from information approximating the respective contours of a set of vertebrae, dependent on the principal component coordinates of the six landmark positions in the image being processed.
12. A method as claimed in claim 7, wherein the initial estimate of the contour is the mean of the conditional point distribution model fitted to the landmark positions.
13. A method as claimed in claim 1, wherein the iterative movement of the estimated contour is constrained by the conditional covariance and the proximity of the conditional mean to the current estimate of the contour.
14. A method as claimed in claim 1, wherein the movement of the contour boundary is constrained by restricting divergence of grey level information derived from points adjacent the estimated contour with equivalent information derived from said statistical model.
15. A method as claimed in claim 1, wherein the iterative movement of the contour boundary is continued until the difference between the estimated contours at two consecutive iterations is smaller than a preset limit.
16. A method as claimed in claim 1, wherein a grey level profile is built by sampling grey level information in the image along the normal to the contour across each contour point.
US12/451,510 2007-05-18 2008-05-15 Semi-automatic contour detection Abandoned US20100177946A1 (en)

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
GB0709599A GB0709599D0 (en) 2007-05-18 2007-05-18 Semi-automatic contour detection
GB0709599.5 2007-05-18
GB0710540A GB0710540D0 (en) 2007-06-01 2007-06-01 Semi-automatic contour detection
GB0710540.6 2007-06-01
GB0718369.2 2007-09-20
GB0718369A GB0718369D0 (en) 2007-09-20 2007-09-20 Semi-automatic contour detection
PCT/EP2008/055956 WO2008141996A2 (en) 2007-05-18 2008-05-15 Semi-automatic contour detection

Publications (1)

Publication Number Publication Date
US20100177946A1 true US20100177946A1 (en) 2010-07-15

Family

ID=39681013

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/451,510 Abandoned US20100177946A1 (en) 2007-05-18 2008-05-15 Semi-automatic contour detection

Country Status (4)

Country Link
US (1) US20100177946A1 (en)
EP (1) EP2147398A2 (en)
JP (1) JP5417321B2 (en)
WO (1) WO2008141996A2 (en)

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120281895A1 (en) * 2010-01-07 2012-11-08 Hitachi Medical Corporation Medical image diagnostic apparatus and medical image contour extraction processing method
JP2014004359A (en) * 2012-06-22 2014-01-16 General Electric Co <Ge> Method and apparatus for iterative reconstruction
US20150043809A1 (en) * 2013-08-09 2015-02-12 Siemens Medical Solutions Usa, Inc. Automatic Segmentation of Articulated Structures
US9002105B2 (en) 2013-03-06 2015-04-07 Xerox Corporation Automated contour detection methods, systems and processor-readable media
CN104905766A (en) * 2014-03-11 2015-09-16 Ge医疗系统环球技术有限公司 Medical scanning system and method for determining scanning parameters based on scout views
US20160015347A1 (en) * 2014-07-21 2016-01-21 Zebra Medical Vision Ltd. Systems and methods for emulating dexa scores based on ct images
US9636181B2 (en) 2008-04-04 2017-05-02 Nuvasive, Inc. Systems, devices, and methods for designing and forming a surgical implant
US20170366703A1 (en) * 2010-10-19 2017-12-21 Apple Inc. Systems, methods, and computer-readable media for placing a representation of the captured signature in a document
US9848922B2 (en) 2013-10-09 2017-12-26 Nuvasive, Inc. Systems and methods for performing spine surgery
US9913669B1 (en) 2014-10-17 2018-03-13 Nuvasive, Inc. Systems and methods for performing spine surgery
US9940537B2 (en) 2014-11-26 2018-04-10 Toshiba Medical Systems Corporation Medical image processing apparatus, medical imaging apparatus and medical image processing method
US10140543B2 (en) 2015-04-03 2018-11-27 Toshiba Medical Systems Corporation Medical image processing apparatus, medical image processing method, and medical imaging device
CN108921171A (en) * 2018-06-22 2018-11-30 宁波工程学院 A kind of Bones and joints X-ray film automatic identification stage division
US10318841B2 (en) 2016-02-26 2019-06-11 Toshiba Medical Systems Corporation Medical-image processing apparatus, ultrasonic diagnostic apparatus, and medical-image processing method
US10325068B2 (en) * 2012-11-20 2019-06-18 General Electronic Company Methods and apparatus to label radiology images
CN109979572A (en) * 2019-03-25 2019-07-05 合肥工业大学 The section acquisition methods and device of pedicle of vertebral arch in a kind of three dimensional spine model
US10497127B2 (en) 2015-04-23 2019-12-03 Koninklijke Philips N.V. Model-based segmentation of an anatomical structure
US10588589B2 (en) 2014-07-21 2020-03-17 Zebra Medical Vision Ltd. Systems and methods for prediction of osteoporotic fracture risk
US11207132B2 (en) 2012-03-12 2021-12-28 Nuvasive, Inc. Systems and methods for performing spinal surgery

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8126249B2 (en) * 2008-05-30 2012-02-28 Optasia Medical Limited Methods of and system for detection and tracking of osteoporosis
EP2441045A1 (en) * 2009-06-11 2012-04-18 Synarc Inc. Improved alignment of shapes of body parts from images
US9390502B2 (en) 2013-04-22 2016-07-12 Kabushiki Kaisha Toshiba Positioning anatomical landmarks in volume data sets
JP2017029324A (en) * 2015-07-30 2017-02-09 国立大学法人 千葉大学 Method and program for creating blood vessel highlighted image data
KR102044528B1 (en) * 2018-03-15 2019-11-14 울산대학교 산학협력단 Apparatus and method for modeling bone
CN111598949B (en) * 2020-04-10 2023-03-24 上海中医药大学 Automatic positioning method for head and neck acupoints of human body

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050123189A1 (en) * 2002-03-14 2005-06-09 Dieter Bayer Method and device for reconstructing and representing multidimensional objects from one-dimensional or two-dimensional image data
US20070047790A1 (en) * 2005-08-30 2007-03-01 Agfa-Gevaert N.V. Method of Segmenting Anatomic Entities in Digital Medical Images
US20080216845A1 (en) * 2005-02-16 2008-09-11 Nordic Bioscience A/S Vertebral Fracture Quantification

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040101186A1 (en) * 2002-11-27 2004-05-27 Xin Tong Initializing model-based interpretations of digital radiographs
JP2004188201A (en) * 2002-12-10 2004-07-08 Eastman Kodak Co Method to automatically construct two-dimensional statistical form model for lung area

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050123189A1 (en) * 2002-03-14 2005-06-09 Dieter Bayer Method and device for reconstructing and representing multidimensional objects from one-dimensional or two-dimensional image data
US20080216845A1 (en) * 2005-02-16 2008-09-11 Nordic Bioscience A/S Vertebral Fracture Quantification
US20070047790A1 (en) * 2005-08-30 2007-03-01 Agfa-Gevaert N.V. Method of Segmenting Anatomic Entities in Digital Medical Images

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11453041B2 (en) 2008-04-04 2022-09-27 Nuvasive, Inc Systems, devices, and methods for designing and forming a surgical implant
US10500630B2 (en) 2008-04-04 2019-12-10 Nuvasive, Inc. Systems, devices, and methods for designing and forming a surgical implant
US9636181B2 (en) 2008-04-04 2017-05-02 Nuvasive, Inc. Systems, devices, and methods for designing and forming a surgical implant
US20120281895A1 (en) * 2010-01-07 2012-11-08 Hitachi Medical Corporation Medical image diagnostic apparatus and medical image contour extraction processing method
US9072489B2 (en) * 2010-01-07 2015-07-07 Hitachi Medical Corporation Medical image diagnostic apparatus and medical image contour extraction processing method
US20170366703A1 (en) * 2010-10-19 2017-12-21 Apple Inc. Systems, methods, and computer-readable media for placing a representation of the captured signature in a document
US11207132B2 (en) 2012-03-12 2021-12-28 Nuvasive, Inc. Systems and methods for performing spinal surgery
JP2014004359A (en) * 2012-06-22 2014-01-16 General Electric Co <Ge> Method and apparatus for iterative reconstruction
US10325068B2 (en) * 2012-11-20 2019-06-18 General Electronic Company Methods and apparatus to label radiology images
US9002105B2 (en) 2013-03-06 2015-04-07 Xerox Corporation Automated contour detection methods, systems and processor-readable media
US9286688B2 (en) * 2013-08-09 2016-03-15 Siemens Medical Solutions Usa, Inc. Automatic segmentation of articulated structures
US20150043809A1 (en) * 2013-08-09 2015-02-12 Siemens Medical Solutions Usa, Inc. Automatic Segmentation of Articulated Structures
US9848922B2 (en) 2013-10-09 2017-12-26 Nuvasive, Inc. Systems and methods for performing spine surgery
CN104905766A (en) * 2014-03-11 2015-09-16 Ge医疗系统环球技术有限公司 Medical scanning system and method for determining scanning parameters based on scout views
US10111637B2 (en) 2014-07-21 2018-10-30 Zebra Medical Vision Ltd. Systems and methods for emulating DEXA scores based on CT images
US10588589B2 (en) 2014-07-21 2020-03-17 Zebra Medical Vision Ltd. Systems and methods for prediction of osteoporotic fracture risk
US20160015347A1 (en) * 2014-07-21 2016-01-21 Zebra Medical Vision Ltd. Systems and methods for emulating dexa scores based on ct images
US10716529B2 (en) 2014-07-21 2020-07-21 Zebra Medical Vision Ltd. Systems and methods for prediction of osteoporotic fracture risk
US10039513B2 (en) * 2014-07-21 2018-08-07 Zebra Medical Vision Ltd. Systems and methods for emulating DEXA scores based on CT images
US10327725B2 (en) 2014-07-21 2019-06-25 Zebra Medical Vision Ltd. Systems and methods for emulating DEXA scores based on CT images
US10433893B1 (en) 2014-10-17 2019-10-08 Nuvasive, Inc. Systems and methods for performing spine surgery
US10485589B2 (en) 2014-10-17 2019-11-26 Nuvasive, Inc. Systems and methods for performing spine surgery
US9913669B1 (en) 2014-10-17 2018-03-13 Nuvasive, Inc. Systems and methods for performing spine surgery
US11213326B2 (en) 2014-10-17 2022-01-04 Nuvasive, Inc. Systems and methods for performing spine surgery
US9940537B2 (en) 2014-11-26 2018-04-10 Toshiba Medical Systems Corporation Medical image processing apparatus, medical imaging apparatus and medical image processing method
US10140543B2 (en) 2015-04-03 2018-11-27 Toshiba Medical Systems Corporation Medical image processing apparatus, medical image processing method, and medical imaging device
US10497127B2 (en) 2015-04-23 2019-12-03 Koninklijke Philips N.V. Model-based segmentation of an anatomical structure
US10318841B2 (en) 2016-02-26 2019-06-11 Toshiba Medical Systems Corporation Medical-image processing apparatus, ultrasonic diagnostic apparatus, and medical-image processing method
CN108921171A (en) * 2018-06-22 2018-11-30 宁波工程学院 A kind of Bones and joints X-ray film automatic identification stage division
CN109979572A (en) * 2019-03-25 2019-07-05 合肥工业大学 The section acquisition methods and device of pedicle of vertebral arch in a kind of three dimensional spine model

Also Published As

Publication number Publication date
WO2008141996A2 (en) 2008-11-27
WO2008141996A3 (en) 2009-03-19
JP5417321B2 (en) 2014-02-12
JP2010527647A (en) 2010-08-19
EP2147398A2 (en) 2010-01-27

Similar Documents

Publication Publication Date Title
US20100177946A1 (en) Semi-automatic contour detection
US11657518B2 (en) Method for deformable 3D-2D registration using multiple locally rigid registrations
Mattes et al. Nonrigid multimodality image registration
Korez et al. A framework for automated spine and vertebrae interpolation-based detection and model-based segmentation
US8170306B2 (en) Automatic partitioning and recognition of human body regions from an arbitrary scan coverage image
US7382907B2 (en) Segmenting occluded anatomical structures in medical images
US7916919B2 (en) System and method for segmenting chambers of a heart in a three dimensional image
JP5603859B2 (en) Method for controlling an analysis system that automatically analyzes a digitized image of a side view of a target spine
EP1551296B1 (en) Method, code, and system for assaying joint deformity
Pereañez et al. Accurate segmentation of vertebral bodies and processes using statistical shape decomposition and conditional models
Kim et al. Automation of spine curve assessment in frontal radiographs using deep learning of vertebral-tilt vector
Benameur et al. Three-dimensional biplanar reconstruction of scoliotic rib cage using the estimation of a mixture of probabilistic prior models
US20120249551A1 (en) alignment of shapes of body parts from images
US7983464B2 (en) System and method for corpus callosum segmentation in magnetic resonance images
Steger et al. Articulated atlas for segmentation of the skeleton from head & neck CT datasets
Logithasan et al. Using machine learning to automatically measure axial vertebral rotation on radiographs in adolescents with idiopathic scoliosis
EP1760659B1 (en) Method of segmenting anatomic entities in digital medical images
Schramm et al. Toward fully automatic object detection and segmentation
CN114093462A (en) Medical image processing method, computer device, and storage medium
US20220414869A1 (en) Detecting and segmenting regions of interest in biomedical images using neural networks
Iglesias et al. Semiautomatic segmentation of vertebrae in lateral x-rays using a conditional shape model
WO2022271838A1 (en) Detecting and segmenting regions of interest in biomedical images using neural networks
Lecron et al. Points of interest detection in cervical spine radiographs by polygonal approximation
Pilgram et al. Knowledge-based femur detection in conventional radiographs of the pelvis
Benjelloun et al. X-ray image segmentation for vertebral mobility analysis

Legal Events

Date Code Title Description
AS Assignment

Owner name: NORDIC BIOSCIENCE IMAGING A/S, DENMARK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DE BRUIJNE, MARLEEN;IGLESIAS, JUAN E.;SIGNING DATES FROM 20091111 TO 20091112;REEL/FRAME:023605/0101

AS Assignment

Owner name: SYNARC INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NORDIC BIOSCIENCE IMAGING A/S, A.K.A. SYNARC IMAGING TECHNOLOGIES A/S;REEL/FRAME:026693/0479

Effective date: 20110401

AS Assignment

Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, WA

Free format text: SECURITY AGREEMENT;ASSIGNOR:SYNARC INC.;REEL/FRAME:027282/0959

Effective date: 20111123

AS Assignment

Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, CA

Free format text: NOTICE OF SECURITY INTEREST IN PATENTS;ASSIGNOR:SYNARC INC.;REEL/FRAME:030157/0547

Effective date: 20130405

AS Assignment

Owner name: SYNARC INC., CALIFORNIA

Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN U.S. PATENTS (REEL 027282, FRAME 0959);ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:030173/0902

Effective date: 20120405

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: SYNARC INC., CALIFORNIA

Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN UNITED STATES PATENTS (RELEASES REEL 030157 FRAME 0547);ASSIGNOR:BANK OF AMERICA, N.A.;REEL/FRAME:032776/0402

Effective date: 20140310

AS Assignment

Owner name: BIOCLINICA, INC., NEW JERSEY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SYNARC INC.;REEL/FRAME:048468/0743

Effective date: 20190221