« PreviousContinue »
SYSTEMS AND METHODS FOR CREATING
AND VIEWING THREE DIMENSIONAL
RELATED APPLICATION 5
The present application claims the benefit of U.S. provisional patent application Ser. No. 60/574,991 filed on May 27, 2004, which is incorporated herein by reference in its entirety. 10
1. Field of the Invention
The present invention generally relates to creating and 15 viewing virtual microscope slide images and more particularly relates to the creating and viewing of three dimensional virtual microscope slide images.
2. Related Art
Conventional microscopes, using mechanical focusing 20 mechanisms of sufficient quality, are easily focused by an observer. The feedback loop between the eye/brain and the fingers is extremely rapid and accurate. If any part of the feedback loop is impaired, however, the process of obtaining perfect focus becomes extremely difficult, time consuming 25 and even inaccurate. These impairments may arise from poor optical quality, insufficient or incorrect illumination, inaccuracies in the focusing mechanism, or even due to poor visual acuity of the operator.
More recently, conventional computer implemented 30 microscope systems have provided the ability to remotely access high resolution images of microscope slides for a number of years. Devices capable of creating such images fall into two broad categories: (1) remote controlled (or robotic) microscopes; and (2) imagers, which create some type of 35 electronic or photographic image of some, or all, of the specimen. Each type has its advantages and disadvantages, many of which are unique to the particular application.
Remote controlled microscopes, which essentially replace the operator's eye with an electronic camera, allow the opera- 40 tor to control the focus plane of the camera. This capability is particularly advantageous when viewing thick specimens. However, the optical field of view at moderate or high resolution is extremely small due to a limited number of camera pixels. This makes it very difficult and time consuming to 45 view an entire specimen on a microscope slide even at moderate magnification.
Conventional imaging systems are capable of creating large enough images to capture an entire specimen at moderate or high magnification. These images are called virtual 50 microscope slides or virtual slides. Because of their extremely large size, virtual slides are usually limited to a single focus level ("Z-plane"). Many types of microscope specimens are very thin so a single Z-plane virtual slide is sufficient. Other types of microscope specimens, however, 55 are thicker than the depth of field of the objective lens.
Remote controlled microscopes additionally suffer from an even more degrading effect—time lag. Because a remotely controlled microscope is merely a microscope with motorized stages for positioning and focusing the specimen, and a 60 camera to transmit images to a remote user, if the image being displayed to the user is delayed by even a small fraction of a second, the feedback loop will be frustrating, if not nearly impossible to use. The user, viewing the electronic image on a remote monitor, will attempt to converge on the optimally 65 focused image, only to find that the image continues to zoom after he has attempted to stop at the best focus. The process
must then be repeated in smaller and smaller iterations, at slower and slower speeds until finally, a satisfactory focus is obtained. Time lag in conventional systems, in particular as it applies to a networked attached device making use of an internet connection, has several contributing causes. One cause can be traced to the design of the device itself. Poor system design, and inefficient data processing methods can add significantly to the perceived sluggishness of conventional systems.
Two other factors are more serious in that they cannot always be controlled by the designer. The first is bandwidth. Bandwidth refers to the capacity of a network to transmit data, or in the case of a robotic microscope, the imagery being displayed to the operator, and the control signals returning to the device to affect its position and focus. Low network bandwidth limits the update rate of the image on the video display. Fortunately, increased bandwidth may be purchased at greater expense, effectively solving this problem. The second problem, called latency, is not so easily solved. Latency is simply the delay time between network nodes. Even fast networks have some degree of latency. Delays, often on the order of seconds, are not uncommon over the Internet. Latency is the Achilles heel of real-time feedback control systems.
What is needed is a remote device that appears to the user as an extremely responsive, high bandwidth, low latency, remote controlled microscope that does not suffer from the limitations of the conventional systems described above.
Accordingly, certain embodiments are described herein that provide systems and methods that allow users to locally or remotely interact with a microscope slide image acquisition device or image data server device for responsive, high bandwidth, low latency acquisition and viewing of virtual slide images in multiple Z-planes. The systems and methods described herein combine the advantages of remote controlled microscopes and microscope imaging systems and provide the ability to scan large regions of a specimen on a microscope slide at high resolution, as well as the ability for an operator to quickly view selected areas of interest and control the focus depth within a thick specimen.
One aspect of the invention is a computer implemented method for creating and viewing three dimensional virtual microscope slides. A microscope slide is positioned in an image acquisition device that is capable of scanning the slide to create a digital image of the specimen on the slide. The specimen is then scanned into a two dimensional image at a medium or high resolution. This two dimensional image is provided to an image viewing workstation where it can be reviewed by an operator. The operator can pan and zoom the two dimensional image at the image viewing workstation and select an area of interest for scanning at multiple depth levels (Z-planes). The image acquisition device receives a set of Z-stack parameters including a location and a depth and then scans a series of Z-plane images, where each Z-plane image corresponds to a portion of the specimen within the depth parameter.
BRIEF DESCRIPTION OF THE DRAWINGS
The details of the present invention, both as to its structure and operation, may be gleaned in part by study of the accompanying drawings, in which like reference numerals refer to like parts, and in which:
FIG. 1 is a block diagram illustrating an example networked system according to an embodiment of the present invention;
FIG. 2 is a side view of an example microscope slide with a specimen according to an embodiment of the present inven- 5 tion;
FIG. 3 is a block diagram illustrating an example three dimensional Z-stack according to an embodiment of the present invention;
FIG. 4 is flow diagram illustrating an example high level 10 process for creating a three dimensional virtual microscope slide according to an embodiment of the present invention;
FIG. 5 is a flow diagram illustrating an example process for creating a three dimensional virtual microscope slide according to an embodiment of the present invention; 15
FIG. 6 is a block diagram illustrating an example user interface for creating a three dimensional virtual slide according to an embodiment of the present invention;
FIG. 7 is a block diagram illustrating an example scan path for an image acquisition device over a specimen having vary- 20 ing thickness according to an embodiment of the present invention;
FIG. 8 is a block diagram illustrating an example three dimensional virtual slide that results from a three dimensional scan of the specimen shown in FIG. 7 according to an embodi- 25 ment of the present invention;
FIG. 9 is a flow diagram illustrating an example process for viewing three dimensional virtual slides according to an embodiment of the present invention; and
FIG. 10 is a block diagram illustrating an exemplary com- 30 puter system as may be used in connection with various embodiments described herein.
Certain embodiments as disclosed herein provide for systems and methods for creating and viewing three dimensional virtual slides. For example, one method as disclosed herein allows for one or more microscope slides to be positioned in an image acquisition device that scans the specimens on the 40 slides and makes two dimensional images at a medium or high resolution. Alternatively, the entire specimens can be captured at once in a low resolution "macro" image. These high or low resolution two dimensional images are provided to an image viewing workstation where they are viewed by an 45 operator who pans and zooms the two dimensional image and selects an area of interest for scanning at multiple depth levels (Z-planes). The image acquisition device receives a set of parameters for the multiple depth level scan, including a location and a depth. The image acquisition device then scans 50 the specimen at the location in a series of Z-plane images, where each Z-plane image corresponds to a depth level portion of the specimen within the depth parameter. After reading this description it will become apparent to one skilled in the art how to implement the invention in various alternative 55 embodiments and alternative applications. However, although various embodiments of the present invention will be described herein, it is understood that these embodiments are presented by way of example only, and not limitation. As such, this detailed description of various alternative embodi- 60 ments should not be construed to limit the scope or breadth of the present invention as set forth in the appended claims.
FIG. 1 is a block diagram illustrating an example networked system 10 according to an embodiment of the present invention. In the illustrated embodiment, the system 10 com- 65 prises an image acquisition device ("LAD") 20, an image data server ("IDS") 30, and an image viewing workstation
("IVW") 40. Each of these devices is associated with a respective data storage area 25, 35, and 45, and each of the devices is communicatively coupled to the others via a network 50. Alternatively, the devices may be directly connected to each other or connected to each other via separate networks . In one embodiment, two or more of the devices may be modularly integrated into the same physical device, in which case the communication between functional modules may take place through inter-process communication, shared memory, common files or the like.
The IAD 20 comprises hardware and software which can acquire two-dimensional images frommicroscope slides. The IAD 20 may acquire images from the slide at any location within the slide (X,Y) and at any focal depth (Z).
FIG. 1 is a block diagram illustrating an example networked system 10 according to an embodiment of the present invention. In the illustrated embodiment, the system 10 comprises an image acquisition device ("IAD") 20, an image data server ("IDS") 30, and an image viewing workstation ("IVW") 40. Each of these devices is associated with a respective data storage area 25, 35, and 45, and each of the devices is communicatively coupled to the others via a network 50. Alternatively, the devices may be directly connected to each other or connected to each other via separate networks . In one embodiment, two or more of the devices may be modularly integrated into the same physical device, in which case the communication between functional modules may take place through inter-process communication, shared memory, common files or the like.
Specimens on a microscope slide usually have an overall dimension of between 10 and 30 mm per side. For example, a typical sample could cover a region of 20x15 mm. Such a sample, when imaged fully at a resolution of 0.25 microns per pixel (i.e., 40x), yields an image that has 80,000x60,000 pixels. Accurately obtaining full slide images is technically difficult, but may be accomplished with instruments such as the Aperio ScanScope® that is described in detail in U.S. Pat. No. 6,711,283 (Soenksen), which is incorporated herein by reference in its entirety.
The IDS 30 comprises a server computer and software which stores and enables access to two- and three-dimensional images acquired by the IAD. The IDS can be located near the IAD 20 on a local-area network. In an alternative embodiment it may be desirable for the IDS 30 to be remote from the IAD 20 and connected via a wide-area network.
The IVW 40 comprises a personal computer and software which can display two and three dimensional images acquired by the IAD 20 and retrieved from the IDS 30. The IVW 40 may be located across a local area network or wide area network from the IAD 20 and IDS 30, using standard computer network technology such as TCP/IP. The user at an IVW 40 directs the IAD 20 to capture image data of the specimen on a microscope slide, and views it interactively. The captured image data are stored on the IDS 30, from which they may be later viewed using an IVW 40.
The network 50 may be a local area network, a wide area network, a wired or wireless network, or any combination of networks. On such combination of networks is commonly referred to as the Internet. Network 50 may be the Internet or any sub-combination of the networks that comprise the Internet. Network 50 may also be a private network or a public network or any combination of networks.
FIG. 2 is a side view of an example microscope slide with a specimen according to an embodiment of the present invention. In the illustrated embodiment, the slide 110 supports a sample of tissue 100 (also referred to herein as a specimen).