WO1991011959A1

WO1991011959A1 - Analysing screen-displayed images

Info

Publication number: WO1991011959A1
Application number: PCT/EP1991/000203
Authority: WO
Inventors: Douglas Benn
Original assignee: Benn Computer Consultants
Priority date: 1990-02-13
Filing date: 1991-02-04
Publication date: 1991-08-22
Also published as: GB9003172D0

Abstract

A method of comparing an earlier- and a later-acquired screen-displayed image of a part or structure and determining any significant changes in at least one selected area of interest therein in which the method of selecting includes selecting on the screen the area of interest while a corresponding previous selection is displayed elsewhere on the screen to facilitate more accurate selection.

Description

Λ-

ANALYSING SCREEN-DISPLAYED IMAGES

This invention relates to a method of analysing a screen-displayed image of a part or structure, e.g. of a human or animal body part, for locating any changes in the part or structure, e.g. diagnosing disease in a body part. In particular, but not exclusively, the invention has application in dentistry for diagnosing caries and/or periodontal diseases from images derived from an X-ray image of a person's teeth and jaws.

However the invention may find application in other fields involving the interpretation of either radiographically produced records of other body parts of a human or animal, or of non-biological imaging systems where subjective identification of reference points for making measurements is required, e.g. for detecting or monitoring the progression of cracks in a metal structure. The invention also relates to a system of image analysis. Conventionally dentists use radiography to diagnose and monitor on X-ray film any abnormalities in a patient's teeth or jaw bone. Such radiographic methods are particularly helpful in enabling dentists to locate teeth, gum and bone disorders which are not normally readily visible. By way of example, intraoral X-ray films are typically used for the location of unerupted or impacted teeth, the detection of retained broken tooth roots, the detection of small carious lesions at approximal surfaces, the detection of resorption of the bone from the necks of teeth and the detection of subgingival deposits of salivary calculus. However the interpretation of an X-ray film and the resultant diagnosis of any associated dental problems are skilled arts which can only be undertaken by highly trained medical personnel such as dentists or doctors. The object of the present invention is to provide a more accurate method of analysing and measuring screen-displayed images of a part or structure. When applied to the analysis and measurement of screen-displayed images of human or animal body parts the invention assists in the diagnosis of disease and the monitoring of its progress in the body part by reducing intra- and inter-examiner variability. Accordingly the present invention comprises a method of comparing an earlier- and a later-acquired screen-displayed image of a part or structure and determining any significant changes in at least one selected area of interest therein in which the method of selecting includes selecting on the screen the area of interest while a corresponding previous selection is displayed elsewhere on the screen to facilitate more accurate selection.

In a preferred embodiment, the invention comprises identifying at least one area of interest on the earlier screen-displayed image, selecting on the screen-displayed image at least two reference points associated with the or each area of interest, recording information to enable subsequent display of the or each area of interest and the said selected reference points associated therewith, and comparing, for the or each area of interest, at least one reference measurement, which is a function of at least some of the said selected reference points for the area of interest in question, with either at least one preset value or at least one equivalent reference measurement, the or each of which is a function of further selected similarly positioned reference points created from the later- acquired, screen-displayed image of the part or structure for the area of interest in question, any significant differences in these comparison(s) indicating a change in the said part or structure.

Preferably the invention is concerned with the analysis of screen-displayed images of anatomical body parts and the reference points selected are anatomical reference points.

Preferably the selection of reference points for the or each area of interest comprises positioning under human control a first set of reference points on the earlier screen-displayed image defining an area of interest and storing their positions, distance(s) between the points and the associated image window, reproducing on the screen from the stored information a display window showing the area of interest with the first set of reference points displayed thereon superimposed on the first display but not covering the area of interest, positioning under human control a second set of reference points on the first display in positions corresponding as closely as possible to the positions of the first set of reference points and averaging the distances between the corresponding points of said first and second sets of reference points to derive an average distance in each case.

In the case where one or more equivalent reference measurements obtained from a later-produced screen- displayed image are compared with the originally obtained reference measurement(s), the later selected reference points for the or each area of interest are each chosen so as to be positioned as closely as possible to the corresponding originally selected reference point but with regard to the particular region or boundary of the part or structure under investigation for example, in diagnosis of dental problems particular original anatomical reference points would typically be positioned at least on the alveolar bone crest margin and the cemento-enamel junction. In that case, the corresponding further selected anatomical reference points must also be selected to be positioned on the parts of the alveolar bone crest margin and the cemento-enamel junction which are positioned nearest to the positions of the original anatomical reference points even if the outline of the alveolar bone crest margin has changed (e.g. as a result of recession of the bone). Ideally the later selected reference points are averaged, e.g. as described with regard to the originally selected reference points. Furthermore, an error estimation is conveniently also obtained with regard to the accuracy with which the reference points for the later-produced screen-displayed image are positioned under human control.

Preferably the method is designed to be put into effect in a computerised diagnostic system comprising a computer, e.g. an IBM PC-AT, with a screen, means for transferring the image to the computer, eg a video camera and frame store or any other image acquisition device or stored image, and means for displaying the image(s) to be viewed, e.g. an X-ray film, on the screen, storage means, e.g, disc storage, for storing information regarding the screen-displayed image and said selected reference points and a computer program controlling operation of the various features of the method. Preferably the computer has a "mouse" or the like for enabling a cursor to be moved on the screen to effect selection of the said reference points.

In certain fields of application, e.g. dental diagnosis of a screen displayed image derived from an X-ray film, it may be desirable when analysing different images of the same area of interest of a person, to select at least two anatomical reference points associated with a part of the image showing overlapping regions of the body part in order to check that the different images have been taken from the same angle. For example, in an intraoral x-ray film, adjacent three dimensional teeth may show up as overlapping approximal surface shadow regions on the two-dimensional X-ray film and the dimensions of these shadow regions would change with alteration of the angle and position of the X-ray beam from the X-ray generator relative to the patient. Accordingly if the corresponding shadow regions for two differently produced images have substantially the same dimensions, it can be assumed that the angle and position of the X- ray beams from the generator or generators in the two instances are substantially the same and thus the anatomical reference measurements for the two images can be validly compared. Alternatively, if the dimensions of the two shadow regions are substantially different, then a warning can be produced alerting a human operator that measured differences between separate films are likely to suffer from artificial changes.

An embodiment of the invention will now be described, by way of example, with reference to the accompanying drawings, in which

Figure 1 is a schematic view of an exposed intraoral X-ray film depicting a person's teeth,

Figure 2 is a schematic view of an image of part of the X-ray film shown in Figure 1 displayed on a computer screen, Figure 3 is a schematic view on an enlarged scale of part of the image shown in Figure 2 with selected anatomical reference points located thereon, and

Figure 4 is a schematic view of part of a screen displayed image of an X-ray film depicting two overlapping teeth.

Figure 5 is a schematic diagram showing the relationship between distance measurements made on a primary image, the generation of stored areas of interest and their use in selecting further reference points for the same and subsequent images.

Figure 1 illustrates a conventional exposed intraoral radiograph or X-ray film 1 (typically having a size of 4cm by 3cm) showing a person's upper and lower posterior teeth biting upon a bite part 2. There are shown four upper teeth 3a - 3d projecting between alveolar bone, parts 4a-4e and four lower teeth 5a - 5d projecting between alveolar bone parts 6a - 6e. In order to diagnose the X-ray film 1 it has been necessary for a skilled dental practitioner to visually inspect the X-ray film to identify any abnormalities present in the X-ray images of the teeth and alveolar bone parts.

In accordance with the present invention, it is proposed to at least partially automate the analysis of one or more anatomical X-ray images in order to provide a degree of objectivity to subsequent diagnosis of disease or decay to the teeth, alveolar bones or gums. In particular, this aim is achieved with the aid of a computerised diagnostic system comprising a computer * -

(not shown) having a screen 10 (e.g. forming part of a monitor), a video camera (not shown) for displaying at least part of an X-ray film on the screen 10, a memory (not shown), e.g. in the form of disk memory associated with the computer and a computer program. The screen 10 may be of any size but will typically comprise a high resolution monitor having a diagonal dimension of at least 25cm.

Initially the X-ray film is displayed on the screen 10 and various areas of interest 11 are then selected for detailed analysis. In Figure 2, only one such area of interest 11 (shown as a rectangular block bordered by chain lines) is depicted although in practice there would be a number of such areas of interest 11 covering all parts of the screen-displayed image where abnormalities to the teeth, alveolar bone or gums might occur.

In Figure 2, the area of interest 11 chosen shows the crest margin of the alveolar bone part 4b and the facing surfaces of adjacent teeth 3a and 3b. Each of these teeth surfaces incorporates the cemento-enamel junction. Other "types" of area of interest could be chosen, e.g. areas of interest showing occlusal teeth surfaces. The present invention is concerned with diagnosing problems not readily apparent by means of a direct visual inspection but which are present on X-ray films.

Under the control of the computer program, a computer operator is prompted to identify anatomical reference points associated with each area of interest. Typically the operator will use a "mouse" or similar device to position a cursor at the selected location, but other forms of cursor control, e.g. keyboard keys or joystick, could be employed. A number of selected anatomical reference points 12 are shown within the 9

area of interest 11 in Figure 2. However Figure 3 illustrates typical preferred reference points which may be selected. For instance reference points 12a and 12b correspond to the location of the cemento-enamel junction on the teeth 3a and 3b, respectively; reference point 12c corresponds to the lowest point (as viewed in Figure 3) on the alveolar bone crest margin 13; reference points 12d and 12e correspond to opposite ends of the generally flat lower edge of the crest margin 13; reference joints 12f and 12g correspond to points on the dentine surface of the teeth 3a and 3b, respectively, opposite the reference points 12d and 12e, respectively; and reference 12h represents where a cut-away corner portion (indicating alveolar bone recession, base of vertical bone loss where periodontal ligament space is of "normal" width) of the crest margin 13 meets a flank of the alveolar bone. It is the responsibility of the computer operator to select the various reference points 12 for each area of interest. ^' Normally these reference points will be the same for similar areas of interest and it is a relatively easy task even for dentally untrained operators to identify and select the various reference points. Once all the reference points 12 have been positioned for each area of interest, information regarding these areas of interest and the positions of the reference points 12 therefor are stored in digital form in the memory. Thus information on only part of the screen-displayed image is stored in memory - I.e. only information regarding the various "blocks" or areas of interest 11 which contain the most relevant dental information. Much of the less relevant information on the original screen displayed image is thus discarded at this stage. 5 „

The next stage involves assessing the degree of operator error associated with the placement of the anatomical reference points 12. This error assessment can be performed at any period of time after the initial placement and storage of the reference points 12 - e.g. it could be performed immediately after the initial placement and storage or several days or even weeks later. The error assessment involves firstly accessing the memory in order to display on the screen 10 a first display and a second display 15. The first display is either an exact digital copy of the original image or the original intraoral radiograph re-entered into the system and comprises at least one area of interest 11 (see Figure 2). The second display 15, positioned, for example, immediately below the first display, consists of the same area or areas of interest but containing the initially positioned reference points 12. The computer operator is then prompted to enter on the area of interest 11 of the first display, e.g. with the aid of a mouse-controlled cursor, ref rence points 12' cprresponding as closely as possible in position to the initially positioned reference points 12 shown on the second display 15. This procedure is repeated for each of the stored areas of interest.

The procedure for using a previously stored image window to assist in the selection of the equivalent area in a subsequent selection is shown schematically in Fig 5 wherein a method involving two images is illustrated. R represents a reading, AOI represents a stored image window of the area of interest, Image 1 is the earlier image and Image 2 is the later image. In the making of the second reading R2 the image window associated with the first reading R^ is displayed to assist in selecting the equivalent reference points. Λo-

In making readings R3 and R4 the stored image windows from readings R2 and R3 respectively are used. This greatly improves the accuracy with which the operator can select the same area at each reading. At the end of the error assessment procedure, the computer is programmed to calculate the operator reproducibility error involved in placing reference points at known locations. Furthermore, it is possible to average the position of each reference point, or to average a distance between two reference points or reference lines to provide an averaged reference measurement. For example, with reference to Figure 3, there is shown a first reference measurement Dl which is the shortest distance between the reference point 12c and a line passing through reference points 12a,

12b and a second reference measurement dl which is the distance between reference points 12d and 12f. The program controls the computer to obtain an average first reference measurement Dlav which is the average of the initial first reference measurement Dl^

(obtained from the initially placed reference points 12) and the final first reference measurement DI2

(obtained from the later placed reference points 12'). In other words,

Dl_av - (Dl! + Dl₂)/2

Similarly,

dl_av - (dlχ + dl₂)/2

The program also enables the computer to calculate the operator error involved in the initial and final reference measurements. For example, the operator measurement errors Dl^ff and D2,*jiff in the first and second reference measurements are given by ^D1diff ^{* D1}l " ^D12 ^and

^dldiff " ^dll " ^dl2.

The computerised diagnostic system is primarily concerned with building up a time-displaced series of records of one or more person's teeth - e.g. a series of records taken at 6-monthly or yearly intervals. In this case, the procedure at a subsequent dental inspection is to take a similar intraoral X-ray of the patient's teeth to produce a similarly sized X-ray film of the same teeth as previously on record. Each area of interest originally recorded, after error assessment, in the memory, is recalled on the screen 10 with the last recorded reference points positioned thereon. The newly produced X-ray film (or a part or parts corresponding to the area of interest(s) displayed) is also displayed on the screen 10 using a video camera/frame store or other image acquisition device and ensuring that the new screen-displayed image is of the same size and scale as the recalled areas of interest. The computer operator is prompted by the program to enter new reference points on the new screen-displayed image corresponding to those of the originally recorded reference points. The aim when the operator positions the new reference points is to identify similar zones or intersections as in the reference image block 15. For example, the aim is to identify the cemento-enamel junctions corresponding to reference points 12a and 12b the lowest point 12c of the crest margin 13 etc.. As before a procedure for assessing the degree of operator error associated with reference point location is initiated in precisely the ML

same manner as previously described. As a result of this procedure new values are obtained for the new reference measurements as follows:

where D2^ and D2₂ are the initial and final first reference measurements for the new screen-displayed image and d2^ and d2 are the initial and final second reference measurements for the new screen-displayed image. Under the control of the program, the computer is able to determine changes in reference measurements, namely Dl_av - D2_av and dl_av - d2_av for the first and second reference measurements, and changes in operator error measurement, namely Dl^^ff, D2₍jiff and dl^iff, ^d2<aiff for the first and second reference measurements, in respect of the two time-displaced recorded images.

The computer under program control is able to determine if distance change has been accurately measured with regard to error change., For example, if Change in Reference Measurement >= (3.2 x Change in Error Measurement), then it is 90% certain that the Change in Reference Measurement recorded is a true measure of the change between the time displaced images and not due to human operator error. If a distance change is thus considered to be valid, the value can be compared with stored pre-set values to determine the significance or otherwise of the change in reference measurement.

It will be appreciated that a "library" of "block" records of areas of interest of a patient's teeth and M

jaws can be built up over a period of time. In this manner a particular area of interest of a patient's teeth can be compared over a period of time from the recorded information on that area of interest compiled over the period of time. In particular the manner in which the positions of the reference points change or the manner in which the reference measurements change are recorded.

In order for valid reference point measurements to be made, it is important that the X-ray irradiation geometry is constant. Whether or not irradiation geometry changes have occurred can be detected by identifying overlapping teeth (approximal surface shadow overlapping) or measuring periodontal ligament space widths. In Figure 4, two overlapping teeth 20 and 21 are shown with an area of overlap 22. Normally the degree of overlap 23 between such teeth will remain substantially constant over a period of time. If this degree of overlap 23 is shown as being substantially the same for two screen-displayed images, it can be assumed that the irradiation geometries were the same. If the degree of overlap changes, a warning can be issued automatically by the computer that even though reference changes may have been measured reliably by the system, the image distortion produced by the irradiation geometry changes could invalidate the reference distance changes. A pre-set value can be stored in the computer regarding approximal surface shadow overlap or periodontal ligament space distance changes which, if exceeded, could initiate the irradiation geometry change warning. The degree of overlap 23 is normally calculated by the computer by obtaining two reference points at the two extremes of the overlap. AH -

It will be realised that the program may be able to automatically detect or diagnose a problem from an averaged reference measurement if that measurement is greater than a pre-set stored measurement distance by a significant amount. For example a problem may be diagnosed if Dl_av significantly exceeds a pre-set value for this distance in a "normal" tooth.

Although the invention has been described with particular reference to dental radiography, it has application in other radiographic fields, e.g. for diagnosing arthritis from X-ray photographs of bone joints, metal welds etc.

Claims

/■ 5 .CLAIMS :

1. A method of comparing an earlier- and a later- acquired screen-displayed image of a part or structure and determining any significant changes in at least one selected area of interest therein in which the method of selecting includes selecting on the screen the area of interest while a corresponding previous selection is displayed elsewhere on the screen to facilitate more accurate selection.

2. A method as claimed in claim 1 which comprises identifying at least one area of interest on the earlier screen-displayed image, selecting on the screen-displayed image at least two reference points associated with the or each area of interest, recording information to enable subsequent display of the or each area of interest and the said selected reference points associated therewith, and automatically comparing, for the or each area of interest, at least one reference measurement, which is a function of at least some of the said selected reference points for the area of interest in question, with either at least one preset value or at least one equivalent reference measurement, the or each of which is a function of further selected similarly positioned reference points created from the later-acquired, screen-displayed image of the part or structure for the area of interest in question, any significant differences in these comparisons) indicating a change in the said part or structure.

3. A method as claimed in claim 1 or 2 in which the screen-displayed image is of anatomical body parts and the reference points selected are anatomical reference points.

4. A method as claimed in any of claims 1 - 3 in which the selection of reference points for the or each area of interest comprises positioning under human control a first set of reference points on the earlier screen-displayed image defining an area of interest and storing their positions, distance(s) between the.points and the associated image window, reproducing on the screen from the stored information a display window showing the area of interest with the first set of reference points displayed thereon superimposed on the first display but not covering the area of interest, positioning under human control a second set of reference points on the first display in positions corresponding as closely as possible to the positions of the first set of reference points and averaging the distances between the corresponding points of said first and second sets of reference points to derive an average distance in each case.

5. A method as claimed in claim 4 in which a similar procedure is employed to acquire data from the later- acquired image wherein the stored image window from the earlier screen-displayed image is superimposed on the later-acquired image to act as a guide in selecting the first set of reference points.

6. A method as claimed in any of claims 1 - 5 in which the significance or otherwise of the said comparison(s) is determined on the basis of an error estimation calculated on the accuracy with which the said positioning under human control of the second set of reference points is achieved relative to the first set.

7. A method as claimed in any of claims 3 to 4 in which the anatomical body parts are teeth and associated bone.

8. A method according to claim 7 in which one or more equivalent reference measurements obtained from the later-acquired screen-displayed image are compared -

with the originally obtained reference measureraent(s), the original anatomical reference points being positioned at least on the alveolar bone crest margin and the cemento-enamel junction and the corresponding further selected anatomical reference points being selected to be positioned on the parts of the alveolar bone crest margin and the cemento-enamel junction which are positioned nearest to the positions of the original anatomical reference points even if the outline of the alveolar bone crest margin has changed (e.g. as a result of recession of the bone).