WO1995014252A1

WO1995014252A1 - 3-d video microscope

Info

Publication number: WO1995014252A1
Application number: PCT/US1994/013204
Authority: WO
Inventors: Carlton L. Frederick
Original assignee: Q-Lamp, Inc.
Priority date: 1993-11-15
Filing date: 1994-11-15
Publication date: 1995-05-26
Also published as: AU1256195A

Abstract

A stereo microscope having a three-dimensional video display. A conventional stereo microscope is fitted with a video camera in place of the left and right eyepieces. Each camera's video is fed to a separate image capture board in a single computer, which controls the left and right boards and adds the capability for image storage, titling, etc. The video output from each capture board is displayed alternately, left and right image, frame by frame, on a single video monitor. The switching between the two capture boards is done by a video switching unit which also provides synchronization signals for glasses, so that the user's eyes each see the appropriate left-right alternating frames, creating a high quality, real-time three-dimensional image on the monitor.

Description

3-D VIDEO MICROSCOPE

FDELD OF THE INVENTION

The invention pertains to the field of stereo microscopy. More particularly, the invention pertains to stereo microscopy using three dimensional video display techniques.

BACKGROUND OF THE INVENTION

The stereo microscope has long been known to the art. It comprises essentially two microscopes in tandem, with the objectives converged on a single specimen, and the ocular lenses spaced such that a user can view each microscope with one eye, resulting in a three dimensional image of the specimen.

This is very useful in research, since it is much easier to visualize structure from a three dimensional image than from the flat image of a single microscope. For teaching, however, the requirement that the user must physically put his eyes to the ocular lenses 3f the microscope limits the instrument to one student at a time.

Even for a single researcher, there are ph. ical difficulties in spending long periods in a fixed position peering into the lenses of a conventional stereo microscope. The user must remove his eyes from the scope in order to take notes, and it is tiring to sit hunched over a scope for long periods.

This limitation has long been avoided in single-optic microscopes by fitting

(or replacing) the ocular with a television camera, so that a large number of students can look at the same image c~ one or more video monitors. Of course, these video-equipped microscopes could only produce a simple flat (two dimensional) image.

It is therefore an object of the invention to provide a microscope with a video display which is capable of stereo (three dimensional) imaging.

It would also be useful for researchers and teachers with a need for stereo microscopes to be able to take notes and/or annotate the images they see, and to store the images themselves for later use. To date, no stereo microscope known to the inventor is capable of this.

It is thus a further object of the invention to provide a microscope capable of three dimensional imaging, in which the images can be stored, annotated and retrieved for later use.

Three dimensional television images have in the past been created using the same "red/green glasses" technology familiar to movie-goers in the 1950' s. The viewer must wear glasses with complementary colors in each eye (i.e. red on the left, green on the right). The two images are simply combined and shown simultaneously, with (at best) each image in a single color) and the colored filters separate the image so that each eye theoretically only sees one image. In practice, this system is not very effective. There is inevitably some degree of double- imaging, even in monochrome situations where left is only shades of red and right only shades of green. Full color images are, at best, marginal. The colored filters cut the brightness of the image too far to be really practical for long term use, and the resolution of the image is poor.

In recent years a number of companies have begun to produce special glasses for three dimensional television images, either for movies or, especially, for video games. These glasses are provided with liquid crystal display (LCD) shutters, which can be selectively opaqued by the application of an electrical voltage. SEGA and Nintendo both have video games out on the commercial toy market using this technology. Lenny Lipton of the StereoGraphics Corporation, of San Rafael, California has a number of patents using such glasses for 3-D imaging of movies or television programming. See, for example, U.S. Patent numbers 4,523,226 (1985); 4,562,463 (1985) and 4,583,117 (1986). Later Lipton patents concerned improvements in the glasses themselves. For example, U.S. Patent numbers 4,967,268 (1990) and 5,181,133 (1992) show an infrared (IR) transmission system to synchronize the glasses to the TV images.

To inventor's knowledge, however, until the present invention no one has attempted to apply this imaging technology to the microscope field.

Through the use of the LCD glasses technique, and the apparatus of the invention described below, a researcher can sit at his desk and view a high resolution, flicker free, three dimensional image on a large television monitor. This eliminates the need to sit at the microscope itself, and allows the freedom to change position, take notes, etc., without losing sight of the stereo image. Using the IR drive technique pioneered by Lipton, and used in the more expensive commercially available LCD glasses, an entire classroom can simultaneously see the same full color three dimensional image that the teacher can see.

SUMMARY OF THE INVENTION

The invention presents a stereo microscope having a three-dimensional video display. A conventional stereo microscope is fitted with a video camera in place of the left and right eyepieces. Each camera's video is fed to a separate image capture board in a single computer, which controls the left and right boards and adds the capability for image storage, titling, etc. The video output from each capture board is displayed alternately, left and right image, frame by frame, on a single video monitor. The switching between the two capture boards is done by a video switching unit which also provides synchronization signals for glasses, so that the user's eyes each see the appropriate left-right alternating frames, creating a high quality, real-time three dimensional image on the monitor.

BRIEF DESCRIPTION OF THE DRAWING

Fig. 1 shows the microscope of the invention in use.

Fig. 2 shows a block diagram of the overall circuitry of the invention.

Fig. 3 shows a diagram of the video switch / synchronizer unit

Fig. 4 shows a timing diagram of the invention

DESCRIPTION OF THE PREFERRED EMBODIMENT

Figure 1 shows the invention in use, in its embodiment as a retrofit to existing stereo microscopes (6). The microscope (6) is a conventional optical stereo (3 dimensional) microscope, which can be of any of the many kinds available on the market from many manufacturers. The microscope has the usual dual eyepieces, which are removed and two television cameras (7) and (8) are put in their place through the use of conventional adapter tubes. The television cameras are preferably small color units, of which many are available. The Javelin Electronics model JE3662HR cameras have been found to be especially useful in the preferred embodiment of the invention, because of their capability of high resolution, high-scan rate images.

The video output (5) from the right (7) and left (8) video cameras is fed into video image capture circuits. Preferably this circuitry is in the form of image capture boards in a microcomputer, which allows the computer to supply various timing and synchronization signals and to superimpose text, store images, etc., as will be discussed below.

The image output from the two video capture boards is routed through an appropriate cable (4) to a video switch / synchronization unit (3). The video switch unit is described in detail below.

The switched output from the video switch is then sent to an appropriate video monitor (10), on which the image is displayed. Any video monitor capable of displaying the image would work, although a high-resolution monitor capable of operation at high vertical scan frequencies (i.e. 120Hz) and non-interlaced operation with small dot sizes is preferred. These are sometimes called "multisync SVGA" monitors, and are available commercially from many sources. The Mag- Innovision model C-SV1448 has been successfully used with the preferred embodiment of the invention.

The video switch / synchronization unit (3) also supplies a synchronization signal on a cable (2) or IR link to one or more sets of LCD switching glasses (1), which are worn by the user. These glasses are commercially available from video¬ game manufacturers such as SEGA or Nintendo, or from 3DTV, inc. , and allow the left and right lenses to be selectively rendered opaque and clear through the application of the appropriate voltages to the LCD shutters in each lens. Through the use of the video switch / synchronization unit, the left lens is rendered opaque when the right eye image is on the monitor, and vice-versa. The result in the preferred embodiment is the appearance of a high-quality, three dimensional microscope image. If desired, the system can be set up to be viewed by multiple viewers wearing parallel-triggered glasses, either by simply paralleling more than one output cord or through the use of glasses which are capable of being triggered by infra-red (IR) synchronization signals. These glasses are more expensive than the cable-operated video-game glasses, but they allow, for example, an entire class of students to view the 3-D image from a microscope operated by their professor.

Referring now to the block diagram of figure 2, the operation of the preferred embodiment of the invention will be described in more detail. The conventional stereo microscope is simply shown as the two optical paths (31) viewing sample (32). The two TV cameras for left (29) and right (30) images are mounted to the microscope so as to produce images of the left and right eyepiece views, respectively.

The video output (33) from the left camera (29) is fed to the video input of left image capture board (22). Similarly, the video output (34) from the right camera (30) is fed to the video input of right image capture board (20). The video capture boards, under the control of the microcomputer (39), accept video images from the cameras and store them in memory which is addressable by the microcomputer. Each video capture board has a video output (28) and (27) for right and left, respectively. The video output might be an NTSC composite signal, for low resolution applications. Preferably, the video outputs are separate reg- green-blue (RGB) drive, vertical synchronization (V Sync) and horizontal synchronization (H Sync) signals as are used in the SVGA video standards.

The video capture boards which have been found to be especially useful in the preferred embodiment of the invention are the Omnicomp model M&M pro boards, which allow capture of high resolution images, and are addressable to allow combination of the captured images with text or images from the computer through the Video board (21), which is attached to the image capture boards through feature bus (23). In addition, the image capture boards derive master clocking information, and vertical and horizontal synchronization signals from the video board.

The video board (21) is preferably of high resolution, and must have sufficient output to drive the two image capture boards. Preferred, in the technology current at the present time, are SVGA boards which allow selectable frequency clocking and high resolution imaging. The Diamond Speedster Pro model SVGA board has been found to be usable in the preferred embodiment of the invention. All of the above-mentioned cards (20), (21) and (22) plug into the microcomputer's master bus (24), which connects the cards to the CPU with the usual associated RAM (25) for control and text presentation. The application is not very demanding, so any of the commonly available microprocessor systems would serve. The preferred embodiment of the invention is currently running on a

486DX processor with 8 MB of RAM, but it is certain that other processors will in the future be developed which will be equally useful. Also, the CPU may be software programmed to allow storage of images from the capture boards on a disk (26) or other mass storage as may in the future be developed. As the images are quite large (in excess of one megabyte per 3-D color image, even compressed) a very large disk is preferred. This storage of images also allows annotation of the images, either by text or verbally through digitization of sound, which can be useful in research or teaching applications. The images can later be recalled and displayed with the annotations by retrieving them from the disk and placing them in the memory of the capture boards, to be displayed according to the teachings of the invention.

The video outputs (27) and (28) of the two image capture boards (20) and (22) are the inputs to the Video switch / Glasses synchronizer unit (35), which is described in more detail below. The switch / synch unit (35) drives the monitor (36) alternately with the left / right video signals, and at the same time drives the glasses (37) through wire or IR / RF link (38) to alternately opaque the left and right lenses as appropriate to the image on the screen.

In the preferred embodiment, the entire system is driven at twice the normal speed (120 Hz), or 60 complete left and right images per second, as opposed to the conventional 60Hz scanning. Preferably, the images are non¬ interlaced, which means that the full resolution of the screen is used for each image, instead of the prior-art standard of using the alternating lines of the interlaced image for left and right. As a result, the preferred embodiment is capable of producing a flicker-free high resolution image in full three dimensions.

Figure 3 shows the details of the video switch / synchronizer unit. The two parts of the unit, shown in dotted lines, are the video switch circuit (56) and the LCD glasses driver circuit (57). The various blocks may be made of discrete components, or, preferably by commercially available integrated circuits (IC's). Typical IC's currently available will be identified by standard part number in this description, but it will be understood by one skilled in the art that substitutions can be easily made within the teachings of the invention.

Video inputs (40) and (41) correspond to the RGB video portion of the image capture board outputs (27) and (28) on figure 2.

The other inputs to the unit are the horizontal synchronization (Hsync) (42) and vertical synchronization (Vsync)(43) signals. These signals can come from either right (28) or left (27) board video outputs. Since the two are synchronized by the video board (21), the synchronization outputs are identical. Both Vsync (43) and Hsync signals (42) are passed unmodified through the unit and become Vsync (55) and Hsync (50) outputs, part of the video signal sent to monitor (36).

The Vsync signal (43) becomes the source of the synchronization signals used to switch the video signals and glasses from left to right, as it is detected by synch detector (46) and becomes the trigger input to flip-flop (45). The synch detector can be an Op-amp circuit, such as one-half of an LM158 integrated circuit, or any other convenient synch detector circuit. The flip flop can be a

CD4013 IC, or any other commonly available chip as might be appropriate for the voltages and power drain requirements set by the designer.

The two outputs of a flip-flop are Q and not-Q (conventionally shown as a Q with a line over it). Q and not-Q are complements which switch logical values as the trigger input of the flip-flop is actuated. That is, if Q is true initially, then not-Q is false. When triggered, Q will switch to false, and not-Q to true, and so on.

The flip-flop serves to change the momentary vertical synch pulse into a signal which changes state each time a new frame is sent. That is, the output of the flip-flop is high (for example) throughout the entire left image frame, then switches to low for the entire right frame.

One output of the flip-flop (shown as "Q" in figure 3) acts as the switching input to a video switch IC, such as an LM2044, a specialized IC designed for the purpose. The LM2044 simultaneously switches the red, green and blue lines from two inputs to a single output based on the state of the input. As the flip-flop output switches from high to low and back, the video output (49) is switched from following the left input (40) to the right input (41), and back.

A flip-flop output (shown as not-Q, although it will be recognized by one skilled in the art that, with appropriate choice of components, the Q output could just as easily be used) drives the glasses-synchronization circuit (57). A circuit called a synch-integrator (47), which could be the other half of the LM158 dual op-amp, detects the presence of horizontal synch (i.e. an image is being received). The output (51) of the synch integrator (47) becomes active if there is an Hsync signal on input (42). By feeding this output to one input of AND circuit (48), and the flip-flop output to the other input, the designer can ensure that the glasses are switched only if there is a video image on the monitor. This keeps the glasses from flickering annoyingly (or, worse, from freezing with one eye opaque) when no image is present.

The output of the AND (52) becomes the glasses drive signal to the glasses driver circuit, which is conventional and available from a number of standard sources. The glasses drive signal (52) and video-active signal (51) switches a transistor and a number of interconnected gates in a quad KNEAD circuit such as a CD4030. The output of the circuit is the glasses drive signal (56), comprising left, right and common lines. If desired, these lines could be used to drive a conventional IR transmitter circuit to operate the glasses remotely, although the most economical approach is simply to supply the glasses drive signals to a conventional "mini-stereo^H jack, which is standard for the LCD glasses used in video games.

The operation of the invention is illustrated by the timing diagram of figure

In this diagram, the rectangular blocks on lines (60)-(66) and (69) represent individual video "frames" (a "frame" is a complete screen of video information). The frames from the right camera (61) are shaded in horizontal stripes and numbered from RO through R4, the frames from the left camera (60) are shaded vertically and numbered L0 through L4. As the frames are propagated through the system, they retain their shading and numbering. It is assumed for this diagram that both cameras have been running before the diagram begins, and that, as the diagram begins, the capture boards have been instructed by the cpu to begin storing and transmitting video frames.

It should be noted that for the purposes of this diagram, "true" or "on" is considered to be a positive pulse (i.e. above the baseline). This is done for clarity of explanation, only, and it will be understood by one skilled in the art that for a given set of components a given pulse might be either positive or negative, or a normally positive or negative line switching to zero or the other polarity, as might be defined by the components or devices used. This does not affect the teachings of the invention, but it was felt to be less confusing visually if "up" always meant "on".

Lines (60) and (61) represent the video data from the left and right cameras, respectively, this corresponds to (33) and (34) on figure 2. Each stream of data comprises a sequence of video frames. As the frames are received by the capture boards, they are stored in memory - shown in lines (62) and (63) for the data in the left and right capture boards, respectively.

Once the boards have stored the frames, they are instructed to write them out to their video image outputs (64) and (65) for the left and right image outputs, respectively, which corresponds to (27) and (28) on Fig. 2. As the capture boards begin to write out the images, the horizontal synchronization (66) and vertical synchronization (67) portions of the video outputs also begin. As noted above, because the two capture boards are synchronized together by clock signals from the computer video board, their Vsync and Hsync lines are identical. Thus, only one set of synch lines (67) and (68) are shown in the diagram.

The Vsync (67) pulses, detected by synch detector (46), cause the flip flop

(45) to switch alternately, which in turn causes the video switch (44) to alternately select frames from the left and right video image outputs (64) and (65). The output of the video switch is shown as line (69), which shows the alternating frames (RO), (LI), (R2), etc.

The presence of Hsync (67), detected by the synch integrator (47) in figure

2, causes the video active line (68) to become "true". As noted above, this activates the glasses synchronization circuit (57) in figure 2. The glasses synchronization circuit (57) switches the lines which opaque the left (71) or right (72) lenses. It is assumed for the purpose of this diagram that the lenses are normally clear, and opaque when voltage is applied, which corresponds to the operation of all of the glasses currently in use. It will be understood that this can be inverted if a given set of glasses should operate in an "apply voltage to clear" mode.

Accordingly, it is to be understood that the embodiments of the invention herein described are merely illustrative of the application of the principles of the invention. Reference herein to details of the illustrated embodiments are not intended to limit the scope of the claims, which themselves recite those features regarded as essential to the invention.

Claims

I claim:

1. An improved stereo microscope of the kind having a left and a right optical path, one for each of a user's eyes, capable of creating a three-dimensional image by simultaneous viewing of the two optical paths, the improvement comprising:

a) a right and a left video camera, each having an optical input for accepting the image from one of the optical paths, and a video output, each video output comprising a sequence of video frames and at least a vertical synchronization signal representing the beginning of a frame;

b) two image storage means for capturing the images from the right and left video cameras, each having a video input connected to the video output of a video camera, memory means for storing the image from the video camera, and an image output for outputting a video signal representing the image stored in the memory means, each image output comprising a sequence of video frames and at least a vertical synchronization signal representing the beginning of a frame;

c) video switch means having two video inputs connected to the image outputs of the two image storage means, a control input, and a video output capable of being switched between either of the two video inputs under the control of the control input, such that the image output on the video output will be either the image from the right or left video camera depending on the state of the control input;

d) frame synchronization means for creating an output signal, having a vertical synchronization input connected to the vertical synchronization signal from the image output of at least one of the image storage means, and a control output which changes state upon receipt of a vertical synchronization signal indicating the beginning of a new frame, the control output being connected to the control input of the video switch means, such that the video output of the video switch means will be alternately switched between the right and left video camera images for each frame in the video output stream;

e) a video display means for displaying a video image, connected to the video output of the video switch means;

f) glasses to be worn by the user of the microscope while viewing the video image on the video display, having left and right lenses capable of being selectively rendered opaque by a signal on a control input, the control input being connected to the control output of the frame synchronization means, such that the left lens is rendered opaque when the video display means is displaying the image from the right video camera, and the right lens is rendered opaque when the video display means is displaying the image from the left video camera.

2. The microscope of claim 1 in which the control input of the glasses is connected to the control output of the frame synchronization means by a wire.

3. The microscope of claim 1 in which the control input of the glasses is connected to the control output of the frame synchronization means by an infrared link comprising infrared transmission means for creating an encoded IR signal connected to the control output of the frame synchronization means, and an infrared receiver means for detecting an encoded IR signal, connected to the control input of the glasses, whereby the lenses of the glasses are selectively rendered opaque by the encoded IR signals transmitted on the infrared signal.

4. The microscope of claim 1 in which there are a plurality of glasses driven by the control output of the frame synchronization means, whereby a plurality of users can view the image on the video display means simultaneously.

5. The microscope of claim 1 in which the image storage means are image capture boards in a programmed microcomputer.

6. The microscope of claim 5 further comprising data storage means for storing images connected to the image capture boards, such that images from the video cameras captured by the image storage means may be stored by the data storage means.

7. The microscope of claim 1 in which the video camera means create high resolution video images, and the video display means is capable of displaying high resolution video images.

8. The microscope of claim 7 in which the video display means displays images in a non-interlaced mode.

9. A three dimensional video imaging system for stereo microscopes, comprising:

a) a right and a left video camera, each having an optical input for accepting the image from one of the optical paths of the stereo microscope, and a video output, each video output comprising a sequence of video frames and at least a vertical synchronization signal representing the beginning of a frame;

c) video switch means having two video inputs connected to the image outputs of the two image storage means, a control input, and a video output capable of being switched between either of the two video inputs under the control of the control input, such that the image output on the video output will be either the image from the right or left video camera depending on the state of the control input; d) frame synchronization means for creating an output signal, having a vertical synchronization input connected to the vertical synchronization signal from the image output of at least one of the image storage means, and a control output which changes state upon receipt of a vertical synchronization signal indicating the beginning of a new frame, the control output being connected to the control input of the video switch means, such that the video output of the video switch means will be alternately switched between the right and left video camera images for each frame in the video output stream;

10. The video display system of claim 9, in which the optical input of each of the video cameras replaces an ocular lens of a stereo microscope.

11. The method of creating three dimensional video images from a stereo microscope, comprising the steps of:

a) installing a right and a left video camera in the right and left optical paths of the stereo microscope;

b) storing the images from the right and left video cameras in right and left image storage means for capturing the images from the right and left video cameras; c) alternately switching between the frames stored in the right and left image storage means and displaying the images on a video display means for displaying a video image;

d) viewing the images on the video display means through glasses having left and right lenses capable of being selectively rendered opaque;

e) controlling the lenses of the glasses such that the left lens is rendered opaque when the video display means is displaying the image from the right video camera, and the right lens is rendered opaque when the video display means is displaying the image from the left video camera, whereby the user sees a three dimensional video image from the microscope on the video display means.