WO2006085068A1

WO2006085068A1 - Apparatus and method for image processing of specimen images for use in computer analysis thereof

Info

Publication number: WO2006085068A1
Application number: PCT/GB2006/000439
Authority: WO
Inventors: John R. Maddison
Original assignee: Medical Solutions Plc
Priority date: 2005-02-08
Filing date: 2006-02-08
Publication date: 2006-08-17
Also published as: GB0602446D0; GB2422899A; GB0502511D0

Abstract

The present invention provides a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

Description

APPARATUS AND METHOD FOR IMAGE PROCESSING OF SPECIMEN IMAGES FOR USE IN COMPUTER ANALYSIS THEREOF

Background

The present invention relates to image processing, and more particularly to the processing of digital images of a microscope specimen to identify paxts of the image of the specimen for computer analysis.

The computer analysis of biological specimens performed on digital images of a specimen acquired from scanning the specimen on a microscope slide has advanced the accuracy of diagnostic and prognostic techniques, and is increasingly used for research puiposes to identify the effect of drugs on biological tissue. One example of such computer image analysis techniques is field fraction measurements, which are widely used to determine a quantity of stain taken up by in a stained biological specimen, the quantity of stain being associated with an attribute or characteristic of the specimen.. Another example involves counting of the number of features (e.g. cell nuclei) within a specimen. Such computer measurements are vastly superior to the subjective and potentially less accurate corresponding manual analysis previously performed by laboratory technicians or pathologists in analysing a specimen directly using a microscope.

In certain circumstances, the computer analysis of a biological specimen concerns only the analysis of certain tissue types, which may correspond to only a part of the specimen. For example, the analysis may only be concerned with epithelial tissue. In particular, a typical biological specimen on a microscope slide may contain not only the tissue type to be analysed, but also other types of tissue such as connective tissue, which, if included within the computer analysis might lead to spurious or misleading results. Thus, conventionally, a laboratory technician will review the digital image of a specimen on a computer monitor prior to computer analysis thereof, and identify areas of the image showing the type of tissue of interest for image analysis. Thus, conventional software applications allow a user to manually draw lines around areas of the specimen image displayed on a computer monitor using a mouse or equivalent.

Unfortunately, the manual identification or delineation of parts of the image of a biological specimen to undergo computer analysis is generally imprecise, subjective and extremely time consuming. The accuracy of identifying parts of the image containing the relevant tissue type, to undergo analysis, is dependent upon the user's skill, such as dexterity in using the mouse, and recognition of the types of cell or tissue to be analysed, as well as the time available to the user. Moreover, a technician may be required to perform delineation on several hundred specimen images at a time, which may lead poor delineation by the user as he or she becomes tired or bored.

It would therefore be desirable to provide a method for automatically identifying parts of a biological specimen image to undergo computer analysis by image processing, thereby avoiding the problems associated with performing the function manually.

Summary of the present invention

Accordingly, the present invention provides a method which automatically performs the above described delineation on an image of a biological specimen, by, for example, distinguishing parts of the image corresponding to tissue having a certain morphological structure (such as epithelial tissue), from parts of the image showing other biological and non-biological material.

According to a first aspect, the present invention provides a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

The image masks may specifically represent the stained areas of the biological specimen or may specifically represent the non-stained areas of the biological image i.e. the image mask may be a positive or negative image. One or more of the image masks may or may not be displayed during the method.

The step of identifying and adapting isolated areas of the image may comprise blurring regions of the image mask representing stained areas of the biological specimen by an amount to increase interconnection of clumped areas of the image mask.

The step of identifying and adapting isolated areas of the image may comprise measuring a two dimensional area of regions of the image mask representing stained areas of the biological specimen, and adapting the image mask such that areas of the image mask below a threshold size are represented as non-stained areas of the biological specimen.

The method may comprise adapting the image mask to remove noise from the image mask prior to the identifying and adapting process.

The method may comprise adapting the image mask to remove areas of the image mask corresponding to isolated cells below a predetermined size prior to the identifying and adapting process. The method may comprisr using a morphological opening operator prior to the identifying and adapting process.

The method may comprise using a morphological opening operator prior to the identifying and adapting process to adapt regions of the image mask representing stained areas of the biological specimen to remove noise.

The method may comprise using a morphological opening operator prior to the identifying and adapting process to adapt regions of the image mask representing stained areas of the biological specimen to remove regions of the image mask representing cells of a size below a predetermined threshold.

The method may comprise smoothing the boundaries of the image mask representing stained areas of the biological specimen subsequent to the identifying and adapting process.

The method may comprise using a closing morphological operator on regions of the image mask representing stained areas of the biological specimen subsequent to the identifying and adapting process.

The method may comprise increasing the size of regions of the image mask corresponding to stained areas of the biological specimen subsequent to the identifying and adapting process by an amount to encompass the edges of the epithelium.

^■ The method may comprise using a dilating morphological operator subsequent to the identifying and adapting process to encompass the edges of the epithelium.

The image mask may be a binary image mask. In a second aspect, the present invention provides an image mask produced by a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

In a third aspect, the present invention provides computer code arranged to perform a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

In a fourth aspect, the present invention provides apparatus arranged to perform a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

Brief Description of the drawings

Embodiments of the present invention will now be described, by way of example, with reference to the accompanying drawings in which:

Figure 1 depicts a typical specimen image in which parts of the image corresponding to a certain tissue type have been identified by manual delineation.;

Figure 2 is a flow diagram illustrating the method of an embodiment of the present invention; Figure 3 is a flow diagram illustrating the method of identifying parts of a specimen image corresponding to epithelial tissue in accordance with a preferred embodiment of the present invention;

Figure 4 is a black and white image resulting from thresholding a microscope image of a specimen of a human colon using step 120 of the method of Figure 3;

Figure 5 is a black and white image of the specimen image of Figure 4 following step 130 of the method of Figure 3;

Figure 6 is a black and white image of the specimen image of Figure 4 following step 140 of the method of Figure 3;

Figure 7 is a black and white image of the specimen image of Figure 4 following step 150 of the method of Figure 3; Figure 8 is a black and white image of the specimen image of Figure 4 following step 160 of the method of Figure 3, and

Figure 9 shows the specimen of Figure 1 in which the areas of the specimen have been delineated in accordance with the method of Figure 3;

Figure 10 compares manual and automatic delineation of a human colon specimen.

Specific Embodiments

Figure 1 shows an image of a typical biological specimen, which has been prepared from a sample of human tissue taken from a patient for pathology analysis. In particular, the specimen is prepared using conventional pathology techniques and is stained with a stain that is taken up by the cell nuclei of epithelial tissue. The illustrated example is a breast tissue section stained with H and E (Hematoxylin and Eosin), which stains cell nuclei deeply blue. The breast tissue section contains structured epithelial tissue, connective tissue and fat cells. Computer analysis is to be performed on areas of epithelial tissue.

The skilled person will appreciate that whilst the following description relates particularly to the analysis of particular types of epithelial tissue, it may be used during the analysis of a wide variety of different tissue types having an identifiable morphological structure.

As shown in Figure 1, a human user has manually identified areas that correspond to epithelial tissue, by drawing a black line, in accordance with conventional techniques. Computer analysis of the portions of the image within the boundaries of the hand drawn lines is then performed. As can be seen from Figure 1, the boundary drawn by the human user is very rough and includes parts of the image that do not contain the epithelial cells of interest. Thus, the computer analysis is performed on parts of the specimen image which should not, ideally, he considered in the analysis. This may lead to poor results.

Accordingly, the present invention aims to automatically identify and delineate parts of an image of a biological specimen for computer analysis.

Figure 2 shows the steps performed in accordance with a method of the present invention. The method is typically performed by a computer program rumαing on a computer receiving digital specimen images acquired from scanning a microscope slide containing the specimen, in accordance with conventional techniques.

At step 10, the method receives high resolution image data for the specimen which is to be subject to analysis. The analysis technique may be any form of computer analysis used for the purposes of prognosis or diagnosis, or other pathology related puiposes. The high resolution image data received at step 10 may be the image data for a complete specimen obtained from the scanning of the microscope slide, or, more preferably, it may be a user or computer determined selected large part of the specimen.

At step 20, the method divides the image data received at step 10 into multiple, smaller image portions. Typically, the image is divided into image squares, which may each be more conveniently processed separately.

At step 30, the method processes, in rum, each image portion to identify areas of the image contained therein which should undergo computer analysis. In a preferred embodiment, the image processing performed at step 30 identifies areas of epithelial tissue, such as contained in colon crypts, using the method of a preferred embodiment of the present invention illustrated in Figure 3 and described below.

At step 40, the method performs the conventional computer analysis technique only on the identified areas of the image from step 30.

Thus, the method of Figure 2 performs image analysis only on computer determined areas of the specimen image which are relevant for the computer analysis technique. Thus, the results of computer analysis are optimised, and the results are consequently more useful to the pathologist.

Figure 3 illustrates a method for image processing, to identify image areas of epithelial tissue to undergo analysis, in accordance with a preferred embodiment of the present invention. The method is typically performed by a computer program running on a computer, which may form part of, or be separated from the program performing the method illustrated in Figure 2, and may run on the same or a different computer. The program is typically stored in a computer readable medium such as magnetic or optical disk, which may be loaded into the memory of the computer on which it is run. The skilled person will appreciate, from the following description, that not all of the illustrated method steps are essential to the present invention, although optimal results may be obtained for certain tissue types using all of the steps of the preferred embodiment illustrated in Figure 3.

For the purposes of illustrating the method of Figure 3, reference will be made to Figures 4 to S, which each depicts a processed binarised (black and white) version of a stained specimen image (not shown), to be used as a mask to identify and delineate the epithelial tissue of tubular glands called "crypts" of a human colon. It is within the colon crypts that the intestinal epithelium is renewed and thus, this part of the specimen is of particular interest in pathology analysis. At step 110, the method receives a first portion of a high resolution image of the biological specimen. Typically, the image portion is one of multiple, relatively small image portions from a complete specimen, or large part thereof, as provided by step 20 of the method of Figure 2.

At step 120, the method performs image thresholding to provide a black and white (i.e. binarised) image from the original colour or greys cale image. The automatic thresholding of images is well known in the art and will be familiar to the skilled person. A description of the thresholding technique is described in a paper by N Otsu entitled "A Threshold Selection Method From Grey-Level Histograms", IEEE Trans. Systems, Man and Cybernetics, VoI 9, No 1, pages 377-393, 1979.

The thresholding of step 120 results in an image mask, such as that shown in Figure 4, whereby the white areas of the mask represent the most highly stained portions of the original specimen image from Figure 1. As can be seen from Figure 4, as well as clearly identifying the tubular-shaped epithelial tissue of the colon crypts, it also includes other randomly positioned stained cells and other material, as well as noise resulting from image processing (such as the thresholding step). Thus, use of this mask to delineate the areas of the specimen image to undergo computer analysis would not be satisfactory.

Accordingly, in accordance with the preferred embodiment of Figure 3, at step 130, the method filters out or removes (white) objects which are considered to be small, in order to remove from the image objects corresponding to small cells and nuclei, as well as noise resulting from the thresholding process. This filtering technique may be performed using the "morphological opening operator" well known in the field of image processing.

The filtering of step 130 results in the black and white image mask shown in Figure 5. Notably, the noise lines within the boundaries of the generally circular colon crypts have been removed, as well as a number of smaller cells surrounding the colon crypts, as is evident from comparing Figure 5 with Figure 4.

Next, at step 140, the method filters out or removes from the image, objects which are not part of a regular tissue configuration comprising groups of interconnected cells (referred to generally herein as "clumped cells") typical of epithelial tissue. In particular, epithelial tissue typically comprises cells which are clumped or joined together. It will be noted that the image of Figure 5, resulting from step 130, includes isolated cells and/or cell nuclei which are not part of epithelial tissue.

It will be noted in Figure 5, that clumped cells, although in close proximity to one another, will not necessarily all be touching one another in the image mask. To identify clumped cells, white regions are blurred in the x and y directions by a particular amount to expand their size. Thus, neighbouring cells forming part of the clumped structure, which were previously not touching in the image mask, will now be touching one other. However, although the blurring process will also increase the size of regions not considered to be part of the clumped structure (i.e. isolated regions), they will, in most (if not all) cases lead to these regions remaining as "non- clumped" (i.e. they will still be isolated regions not forming a clumped structure). This is done by ensuring that the blurring process, which is part of the filtering process 140, does not over-expand the size of white regions/objects to produce such errors.

Then, the filtering process of step 140 measures the two-dimensional area of each (white) object within the image, and removes such (white) areas which are below a predefined threshold area. Typically, the threshold area is predetermined based on the type of specimen, particularly the tissue type of interest, the magnification of the objective lens used to acquire the specimen image, and other known factors. The threshold is chosen so that the process removes objects of a size below the normal size for the tissue of interest. The measurement of the area of each object within the image is achieved by first identifying each individual object within the image, and then determining the number of pixels (which is proportional to the area, and thus the size, of the object). It will be appreciated that individual objects comprise all white areas where adjacent- pixels are touching. Thus, the connected loops typical of colon crypts illustrated in Figure 5 are treated as single objects, whilst the spurious cells or cell nuclei between the colon crypts are treated individually, since they are disconnected (isolated). This leads to the black and white image illustrated in Figure 6, whereby all disconnected cells and cell nuclei between the colon crypts have been removed, as can be seen from comparing Figure 6 with Figure 5. The image of Figure 6 ma}' be used as the mask for delineating the original specimen image (not shown). However, the boundaries of the mask are intricate such that their use may not encompass the edges of the epithelium of interest. Thus, in the preferred embodiment, further image processing is performed to ensure that all the relevant epithelial areas of the specimen are contained within the delineated part of the image.

Thus, at step 150, the method first removes the fine detail from the boundaries of the objects left within the image, in order to provide a smooth outline for the mask. This removes the fine black lines which extend into the epithelial tissue of the colon crypts, as can be seen in Figure 6. Step 150 may be performed using the "closing morphological operator" well known in image processing. This leads to the black and white image of Figure 7.

Then, at step 160, the image mask may be further enhanced by expanding the boundaries outwardly using the "dilating morphological operator" well known in image processing techniques. This step ensures that the edges of the epithelium are contained within the boundaries for image processing. Step 160 results in the final image of Figure 8, which is to be subsequently used as a mask for identifying the areas of the image corresponding to the epithelial colon crypts, for use in computer analysis. In particular, the boundaries between the black and white parts of the image/mask of Figure 8 are superimposed on the original specimen image to delineate the areas to undergo computer analysis. This is analogous to the human user performing delineation as illustrated in Figure 1.

The image resulting from step 160 is stored prior to continuing with step 170.

At step 170, the method considers whether there is another image portion, of the multiple image portions representing the complete specimen image, to be processed. If there is another portion to be processed, the method returns to step 110.

Eventually, all of the image portions have been processed, and the method ends at step 180.

Computer analysis may then be performed on the delineated portions of the complete specimen, or large part thereof, received at step 10 of the method of Figure 2, using conventional techniques as described above in relation to step 40 of Figure 2.

Figure 9 shows the specimen of Figure 1 on which automatic delineation has been performed in accordance with the preferred embodiment of the present invention. As can be seen by comparing Figure 9 with Figure I₅ the method of the present invention identifies areas of epithelium more precisely and, as in this case, may identify epithelia areas of the specimen that were not evident from the manual inspection of the image.

Various modifications and changes may be made to the described embodiments. It is intended to include all such variations, modifications and equivalents which fall within the scope of the present invention. The present invention encompasses one or more aspects or embodiments of the present invention in all various combinations whether or not specifically claimed or described in the present specification in that combination.

Claims

1. A method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

2. A method according to claim 1, wherein the step of identifying and adapting isolated areas of the image comprises blurring regions of the image mask representing stained areas of the biological specimen by an amount to increase interconnection of clumped areas of the image mask.

3. A method according to claim 1, wherein the step of identifying and adapting isolated areas of the image comprises measuring a two dimensional area of regions of the image mask representing stained areas of the biological specimen, and adapting the image mask such that areas of the image mask below a threshold size are represented as non-stained areas of the biological specimen.

4. A method according to claim 1, comprising adapting the image mask to remove noise from the image mask prior to the identifying and adapting process.

5. A method according to claim 1, comprising adapting the image mask to remove areas of the image mask corresponding to isolated cells below a predetermined size prior to the identifying and adapting process.

6. A method according to claim 1, comprising using a morphological opening operator prior to the identifying and adapting process.

7. A method according to claim 1, comprising using a morphological opening operator prior to the identifying and adapting process to adapt regions of the image mask representing stained areas of the biological specimen to remove noise.

8. A method according to claim I₅ comprising using a morphological opening operator prior to the identifying and adapting process to adapt regions of the image mask representing stained areas of the biological specimen to remove regions of the image mask representing cells of a size below a predetermined threshold.

9. A method according to claim 1, comprising smoothing the boundaries of the image mask representing stained areas of the biological specimen subsequent to the identifying and adapting process.

10. A method according to claim 1, comprising using a closing morphological operator on regions of the image mask representing stained areas of the biological specimen subsequent to the identifying and adapting process.

11. A method according to claim I₅ comprising increasing the size of regions of the image mask corresponding to stained areas of the biological specimen subsequent to the identifying and adapting process by an amount to encompass the edges of the epithelium.

12. A method according to claim 1, comprising using a dilating morphological operator subsequent to the identifying and adapting process to encompass the edges of the epithelium.

13. A method according to claim 1, wherein the image mask is a binary image mask.

14. An image mask produced by a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.

15. Computer code arranged to perform a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated area^'s represent non-stained areas of the biological specimen.

16. Apparatus arranged to perform a method for automatically distinguishing between epithelial tissue from other regions of a biological specimen ^"using computer image analysis, epithelial tissue being characterised by clumped cells, the method comprising : thresholding a stained biological specimen to provide an image mask of the biological specimen, the image mask distinguishing between stained and non-stained areas of the biological specimen, the stained areas comprising epithelial tissue; and identifying isolated areas of the image mask representing stained areas of the biological specimen in the image mask and adapting the image mask such that the isolated areas represent non-stained areas of the biological specimen.