US20070142705A1 - Endoscope device - Google Patents
Endoscope device Download PDFInfo
- Publication number
- US20070142705A1 US20070142705A1 US11/702,337 US70233707A US2007142705A1 US 20070142705 A1 US20070142705 A1 US 20070142705A1 US 70233707 A US70233707 A US 70233707A US 2007142705 A1 US2007142705 A1 US 2007142705A1
- Authority
- US
- United States
- Prior art keywords
- image
- subject
- image data
- vbs
- displayed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00147—Holding or positioning arrangements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00045—Display arrangement
- A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/042—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by a proximal camera, e.g. a CCD camera
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/267—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the respiratory tract, e.g. laryngoscopes, bronchoscopes
- A61B1/2676—Bronchoscopes
Definitions
- the present invention relates to an endoscope device, or more particularly, an endoscope device capable of navigating an endoscope to help insert the endoscope into an intracorporeal lumen, for example, the bronchi.
- an X-ray computed tomography (CT) device is used to produce tomographic images of a subject so as to construct three-dimensional image data representing the interior of the subject.
- the three-dimensional image data is used to diagnose a lesion.
- One of such three-dimensional images is a three-dimensional image representing the bronchi in the lungs.
- the three-dimensional image of the bronchi is utilized in order to three-dimensionally grasp the position of an abnormality that is suspected to be, for example, a pulmonary carcinoma.
- a bronchoscope is inserted in order to collect a sample of a tissue using a biopsy needle located in the distal section of the endoscope.
- FIG. 31 shows a bronchi 500 .
- an intracorporeal lumen that branches out in multiple stages when the position of an abnormality is close to the terminal of a branch, it is hard to lead the distal end of an endoscope to a target region accurately for a short period of time.
- Japanese Unexamined Patent Application Publication No. 2000-135215 has proposed a system for navigating a bronchoscope to a target region.
- a three-dimensional image of an intracorporeal lumen of a subject is constructed based on image data acquired from a three-dimensional field of the subject.
- the course to a target point along the lumen is determined using the three-dimensional image.
- Virtual endoscopic images representing regions of the lumen that lie along the course are constructed based on the image data, and displayed in order to navigate a bronchoscope.
- the navigation to a target region to be performed in the system described in the Japanese Unexamined Patent Application Publication No. 2000-135215 includes display of a live endoscopic image produced by the bronchoscope.
- virtual endoscopic images representing branch points of bronchi are displayed for the purpose of guiding the bronchoscope into a destination of insertion.
- the bronchi branch out in multiple stages.
- images constructed with the bronchoscope located in the respective bronchi are images representing a plurality of branch destinations and resembling one another.
- An object of the present invention is to provide an endoscope device capable of reliably navigating an endoscope to a target region using guide images that represent actual branch positions.
- An endoscope device in accordance with the present invention comprises: three-dimensional image constructing means for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of images representing a three-dimensional field of the subject; an endoscope that picks up images of the intracorporeal lumen of the subject; and navigation image constructing means that constructs a navigation image which includes the endoscopic image of the intracorporeal lumen of the subject that is produced by the endoscope and the three-dimensional image. While presenting the course of the endoscope to be steered in order to insert the endoscope into the intracorporeal lumen of the subject, the endoscope device helps observe or treat the subject.
- the navigation image constructing means constructs the navigation image adding reduced images of three-dimensional images, which represent all branch points at which the intracorporeal lumen of the subject branches out, added thereto. Owing to these constituent features, the endoscope can be reliably navigated to reach a target region using guide images representing the actual branch positions.
- FIG. 1 shows the configuration of an endoscope device in accordance with an embodiment of the present invention
- FIG. 2 shows the configuration of an input unit shown in FIG. 1 ;
- FIG. 3 is a flowchart describing the flow of constructing navigation data to be executed by a bronchoscope navigation device shown in FIG. 1 ;
- FIG. 4 is a first diagram showing a route determination screen in which the execution of the process described in FIG. 3 is performed;
- FIG. 5 is a second diagram showing the route determination screen in which the execution of the process described in FIG. 3 is performed;
- FIG. 6 is a flowchart describing the flow of determining a route
- FIG. 7 is a first diagram showing a route determination screen in which the execution of the process described in FIG. 6 is performed;
- FIG. 8 is a second diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed;
- FIG. 9 is a third diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 10 is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown in FIG. 1 ;
- FIG. 11 is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 ;
- FIG. 12 is a first diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 13 is a second diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 14 is a third diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 15 is a fourth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 16 is a fifth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 17 is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 ;
- FIG. 18 is a first diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed;
- FIG. 19 is an explanatory diagram concerning a motion picture to be displayed by executing the process described in FIG. 17 ;
- FIG. 20 is a second diagram showing the navigation screen in which the execution of the process described in FIG. 17 is performed;
- FIG. 21 is a third diagram showing the navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 22 is a third diagram showing the navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 23 shows a first variant of the navigation screen shown in FIG. 14 ;
- FIG. 24 is a second variant of the navigation screen shown in FIG. 14 ;
- FIG. 25 is a third variant of the navigation screen shown in FIG. 14 ;
- FIG. 26 is a fourth variant of the navigation screen shown in FIG. 14 ;
- FIG. 27 is a fifth variant of the navigation screen shown in FIG. 14 ;
- FIG. 28 is a sixth variant of the navigation screen shown in FIG. 14 ;
- FIG. 29 is a seventh variant of the navigation screen shown in FIG. 14 ;
- FIG. 30 is an eighth variant of the navigation screen shown in FIG. 14 ;
- FIG. 31 shows the structure of bronchi.
- FIG. 1 to FIG. 30 are concerned with an embodiment of the present invention.
- FIG. 1 shows the configuration of an endoscope device.
- FIG. 2 shows the configuration of an input unit shown in FIG. 1 .
- FIG. 3 is a flowchart describing the flow of constructing navigation data to be performed by a bronchoscope navigation device shown in FIG. 1 .
- FIG. 4 is a first diagram showing a route determination screen in which the execution of the process described in FIG. 3 is performed.
- FIG. 5 is a second diagram showing the route determination screen in which the execution of the process described in FIG. 3 is performed.
- FIG. 6 is a flowchart describing the flow of route determination to be executed at a step in the process described in FIG. 3 .
- FIG. 1 shows the configuration of an endoscope device.
- FIG. 2 shows the configuration of an input unit shown in FIG. 1 .
- FIG. 3 is a flowchart describing the flow of constructing navigation data to be performed by a bronchoscope navigation device shown in
- FIG. 7 is a first diagram showing a route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 8 is a second diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 9 is a third diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 10 is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown in FIG. 1 .
- FIG. 11 is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 .
- FIG. 12 is a first diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 13 is a second diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 14 is a third diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 15 is a fourth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 16 is a fifth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 17 is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 .
- FIG. 18 is a first diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 19 is an explanatory diagram concerning a motion picture to be displayed by executing the process described in FIG. 17 .
- FIG. 20 is a second diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 21 is a third diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 22 is a third diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 23 shows a first variant of the navigation screen shown in FIG. 14 .
- FIG. 24 shows a second variant of the navigation screen shown in FIG. 14 .
- FIG. 25 is a third variant of the navigation screen shown in FIG. 14 .
- FIG. 26 shows a fourth variant of the navigation screen shown in FIG. 14 .
- FIG. 20 is a second diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 21 is a third diagram showing a navigation screen in which the execution of the
- FIG. 27 is a fifth variant of the navigation screen shown in FIG. 14 .
- FIG. 28 is a sixth variant of the navigation screen shown in FIG. 14 .
- FIG. 29 is a seventh variant of the navigation screen shown in FIG. 14 .
- FIG. 30 is an eighth variant of the navigation screen shown in FIG. 14 .
- an endoscope device 1 in accordance with the present embodiment comprises: a bronchoscope 2 that is inserted into the patient's bronchi in order to pick up images of the interiors of the bronchi or to biopsy a tissue of a lesion at the terminal of a bronchus; a degree-of-insertion detection unit 3 that detects a degree of insertion of the insertion unit of the bronchoscope 2 into the patient's body and that includes, for example, an encoder; an input unit 4 formed with a flexible sheet that realizes a plurality of switches in the form of a film and that is disposed on a proximal end of the bronchoscope 2 ; and a bronchoscope navigation device 6 that constructs virtual endoscopic images (hereinafter, VBS images) representing the interiors of bronchi on the basis of CT image data, synthesizes an endoscopic image (hereinafter, a live image), which is produced by the bronchoscope 2 , with the VBS images according to an
- the input unit 4 is formed on the proximal end of the bronchoscope 2 .
- the input unit 4 may be formed with a footswitch.
- the bronchoscope navigation device 6 comprises: a CT image data fetching unit 11 that fetches three-dimensional image data, which is constructed in a known CT system that is not shown and that produces X-ray tomographic images of a patient, via a transportable storage medium, for example, a magneto-optical (MO) disk or a digital versatile disk (DVD); a CT image data storage unit 12 in which three-dimensional image data fetched by the CT image data fetching unit 11 is stored; an MPR image construction unit 13 that constructs an MPR image on the basis of the three-dimensional image data stored in the CT image data storage unit 12 ; a route determination unit 14 that constructs a route determination screen that will be described later and that contains an MPR image constructed by the MPR image constructing unit, and determines a navigation route (hereinafter, simply, a route) along which the bronchoscope 2 is navigated through bronchi; a VBS image construction unit 15 that constructs VBS images, which represent successive regions along the route determined by the route determining unit
- the CT image data storage unit 12 and VBS image storage unit 16 may be realized with one hard disk, and the MPR image construction unit 13 , route determination unit 14 , VBS image construction unit 15 , and image processing unit 17 may be realized with one arithmetic circuit.
- the CT image data fetching unit 11 fetches CT image data from a transportable storage medium such as a MO disk or a DVD.
- the CT image data fetching unit 11 may be realized with an interface circuit that can be connected on the in-house LAN. In this case, the CT image data can be fetched over the in-house LAN.
- the input unit 4 has: as shown in FIG. 2 , a Next VBS switch SW 1 for use in directing update of a VBS image that is contained in a navigation screen and that represents each branch point of bronchi; an Auto Image Rotation switch SW 2 for use in directing automatic rotation of a VBS image so that the direction of the VBS image will correspond to the direction of a live image; a Manual Image Rotation switch SW 3 for use in directing rotation of a VBS image in units of a predetermined angle of rotation so that the direction of the VBS image will correspond to the direction of a live image; and a Freeze switch SW 4 for use in directing fetching of a still image of a live image.
- a Next VBS switch SW 1 for use in directing update of a VBS image that is contained in a navigation screen and that represents each branch point of bronchi
- an Auto Image Rotation switch SW 2 for use in directing automatic rotation of a VBS image so that the direction of the VBS image will correspond to the direction of a live image
- the bronchoscope navigation device 6 is activated prior to observation or treatment performed using the bronchoscope 2 .
- the CT image data fetching unit 11 fetches three-dimensional image data of a patient, which is constructed by a CT system.
- the fetched three-dimensional image data is stored in the CT image data storage unit 12 .
- the route determination unit 14 displays the route determination screen 21 like the one shown in FIG. 4 on the monitor 5 .
- Patient information is selected through a patient information tag screen 22 contained in the route determination screen 21 .
- an MPR image is constructed from multiple different planar images, for example, three different multi-sectional images of a patient selected at step S 4 .
- the MPR image 23 is contained in the route determination screen 21 .
- patient information is selected through the patient information tag screen 22 by entering a patient identifier (ID), with which a patient is identified, using the setting information input unit 19 .
- ID patient identifier
- step S 6 the setting information input unit 19 is used to select a route determination tag 24 (see FIG. 4 ) in the route determination screen 21 . Consequently, a route determination tag screen 25 like the one shown in FIG. 5 appears in the route determination screen 21 . Route determination that will be described later is then executed in order to determine a route along which the bronchoscope 2 is navigated through bronchi in order to help insert the bronchoscope 2 .
- the VBS image construction unit 15 constructs successive VBS images, which represent regions along the entire route, in units of a frame.
- the constructed VBS images are stored in the VBS image storage unit 16 .
- the bronchoscope navigation device 6 employed in the present embodiment can navigate the bronchoscope 2 through bronchi so as to help insert the bronchoscope 2 according to, for example, two navigation methods to be described later.
- the methods shall be implemented in mode 1 and mode 2 respectively.
- the setting information input unit 19 is used to determine mode (navigation mode) for the route determination unit 14 .
- the information of the navigation mode is transmitted to the image processing unit 17 via the route determination unit 14 , and stored in both the image processing unit 17 and route determination unit 14 .
- step S 1 to step S 9 preparations are made for navigation to be performed by the bronchoscope navigation device 6 for observation or treatment by the bronchoscope 2 .
- a start point entry direction window 31 that prompts a user to enter a start point of a route like the one shown in FIG. 7 is displayed in the route determination screen 21 at step S 11 .
- a start point is defined in one of the tomographic images constituting the MPR image 23 by moving a cursor 30 in the route determination screen 21 . Once the start point is defined, the start point is indicated at corresponding positions in the other two tomographic images included in the MPR image 23 .
- An end point entry direction window 32 that prompts a user to enter an end point of the route like the one shown in FIG. 8 is displayed in the route determination screen 21 .
- an end point is defined in one of the tomographic images constituting the MPR image 23 by moving the cursor 30 in the route determination screen 21 . Once the end point is defined, the end point is indicated at corresponding positions in the other two tomographic images included in the MPR image 23 .
- the route determination unit 14 searches a route that links the start point and end point and that runs through bronchi.
- the bronchi have complex paths.
- the route that links the start point and end point and that runs through bronchi cannot therefore always be determined unconditionally.
- the route determination unit 14 therefore searches a first candidate for the route that links the start point and end point and that runs through bronchi.
- the route determination unit 14 then displays the route searched at step S 14 by superimposing the route on the MPR image 23 contained in the route determination screen 21 as shown in FIG. 9 . Moreover, a route finalization window 33 that prompts a user to finalize a route is displayed.
- the route finalization window 33 comprises: a Finalize Route button 41 for use in directing finalization of a searched route; a Search Next Candidate button 42 for use in directing search of the next candidate for a route; a Re-determine Route button 43 for use in redefining a start point and an end point; and a Cancel button 44 for use in canceling route search.
- step S 15 it is judged whether the Search Next Candidate button 42 is clicked. If the Search Next Candidate button 42 is clicked, the next candidate for a route is automatically searched at step S 16 . Control is then passed to step S 17 . If the Search Next Candidate button 42 is not clicked, control is passed to step S 18 . At step S 17 , it is judged whether the next candidate is found as a result of search. If the next candidate is unfound, the warning that the next candidate for a route is unfound is displayed, and control is returned to step S 13 . If the next candidate is found, control is returned to step S 14 .
- step S 18 it is judged whether the Re-determine Route button 43 is clicked. If the Re-determine Route button 43 is clicked, control is returned to step S 11 . If the Re-determine Route button 43 is not clicked, control is passed to step S 19 .
- step S 19 it is judged whether the Finalize Route button 41 is clicked. If the Finalize Route button 41 is not clicked, control is returned to step S 15 . If the Finalize Route button 41 is clicked, control is passed to step S 20 . At step S 20 , a route and positions of branch points on the route are determined, and control is returned to step S 7 in FIG. 6 .
- the order numbers of the branch points lying on the course to be steered in order to insert the bronchoscope from a start point 101 to an end point 102 are appended to the VBS images representing the branch points.
- the VBS images are then stored in the VBS image storage unit 16 at step S 8 . Consequently, the VBS image data representing each branch point includes virtual image data that represents the branch portion, three-dimensional position information concerning the branch portion, and information of an order number assigned to the branch portion lying on the selected course to be steered in order to insert the bronchoscope.
- An image 100 constructed using a volume rendering technique as shown in FIG. 10 may be used to designate the start point 101 and end point 102 for the purpose of route search.
- a navigation screen 51 like the one shown in FIG. 12 is displayed on the monitor 5 at step S 21 .
- the navigation screen 51 comprises: an endoscopic live image display area 52 in which a live image captured by the bronchoscope 2 is displayed; a VBS image display area 53 in which a VBS image is displayed; and a thumbnail VBS image area 54 in which images formed reducing VBS images representing branch points on a route are displayed as branch thumbnail VBS images.
- step S 21 at which the bronchoscope 2 has not yet been inserted, no live image is displayed in the endoscopic live image display area 52 .
- a VBS image 53 a depicting the first branch point on the route is displayed in the VBS image display area 53 .
- Thumbnail VBS images 54 ( a ) to 54 ( j ) representing all the branch points are displayed in the thumbnail VBS image area 54 .
- a marker 55 is superimposed on the VBS image 53 a in order to indicate a lumen of a path that advances along the route.
- the frame of the thumbnail VBS image representing the same branch point as the VBS image 53 a displayed in the VBS image display area 53 is displayed in boldface or in color so that the thumbnail VBS image can be discriminated from the others. An operator can easily discern which of the branch points is represented by a VBS image displayed in the VBS image display area 53 .
- the frame of the thumbnail VBS image 54 ( a ) is displayed in boldface or in color.
- step S 22 inserting the bronchoscope 2 into bronchi is started.
- a live image 52 a captured by the bronchoscope 2 is displayed in the endoscopic live image display area 52 within the navigation screen 51 .
- the VBS image 53 a representing the first branch point on the route is displayed in the VBS image display area 53
- the thumbnail VBS images 52 ( a ) to 54 ( j ) representing all the branch points are displayed in the thumbnail VBS image area 54 .
- the frame of the thumbnail VBS image 54 ( a ) is displayed in boldface or in color.
- step S 24 it is judged at step S 24 whether navigation mode is set to mode 1 in the image processing unit 17 that has received a live image.
- an operator judges whether or not to press the Freeze switch SW 4 . If the operator can discern the image of the lumen of the path indicated with the marker 55 on the live image 52 a by merely viewing the VBS image 53 a in the VBS image display area 53 and the live image 52 a , the operator keeps inserting the bronchoscope. If it is hard to discern the lumen of the path indicated with the marker 55 on the live image 52 a , the operator presses the Freeze switch SW 4 .
- step S 25 it is judged whether the Freeze switch SW 4 is turned on. If the Freeze switch SW 4 is turned on, a still image acquired from the live image 52 a is fetched at step S 26 . The still image and the VBS image 53 a representing the first branch point are compared with each other in order to detect the degree of similarity between them according to a known image processing technique. If the Freeze switch SW 4 is not turned on, control is passed to step S 27 .
- step S 26 If it is found at step S 26 that the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than the determined route (or, the still image is compared with a VBS image representing a different branch point). Consequently, a warning is displayed (not shown).
- step S 26 If it is judged at step S 26 that the still image of the live image 52 a and the VBS image 53 a resemble each other at a degree of similarity exceeding the predetermined value, both the images are regarded to represent the same branch point.
- the operator then handles the Auto Image Rotation switch SW 2 or Manual Image Rotation switch SW 3 so as to rotate the VBS image 53 a so that the still image of the live image 52 a and the VBS image 53 a will be matched with each other. Thereafter, control is passed to step S 27 .
- the Auto Image Rotation switch SW 2 when an operator presses the Auto Image Rotation switch SW 2 , the still image of the live image 52 a and the VBS image 53 a are automatically matched with each other through, for example, pattern recognition processing. Moreover, when the operator presses the Manual Image Rotation switch SW 3 , the VBS image 53 a is rotated by a predetermined angle. The operator repeatedly presses the switch SW 3 so as to match the live image 52 a and VBS image 53 a with each other.
- the operator can now discern the lumen of the path that is indicated with the marker 55 on the live image 52 a .
- the operator therefore keeps inserting the bronchoscope according to the directive by the marker 55 .
- step S 27 it is judged whether the operator has pressed the Next VBR switch SW 1 . If the Next VBR switch SWl is not pressed, control is returned to step S 25 . If the operator has pressed the Next VBR switch SW 1 , the VBS image 53 a corresponding to a thumbnail VBS image (b) representing the next (second) branch point is displayed in the VBS image display area 53 . At this time, the frame of the thumbnail VBS image 54 ( b ) representing the second branch point is displayed in boldface or in color. The frame of the thumbnail VBS image 54 ( a ) becomes the same as the frames of the other thumbnail VBS images.
- step S 29 it is judged whether the bronchoscope has reached a lesion (that is, reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S 25 . The process from step S 25 to step S 29 is repeated until the bronchoscope reaches the lesion.
- step S 25 to step S 29 A concrete example of the process from step S 25 to step S 29 will be described in conjunction with the navigation screen 51 .
- the VBS image 53 a corresponding to a thumbnail VBS image 54 ( h ) representing the eighth branch point is displayed in the VBS image display area 53 .
- the live image 52 a representing a region near the eighth branch point is displayed in the endoscopic live image display area 52 .
- only the frame of the thumbnail VBS image 54 ( h ) representing the eighth branch point is displayed in boldface or in color. This allows an operator to recognize that navigation is performed at the eighth branch point.
- step S 25 if the operator presses the Freeze switch SW 4 (step S 25 ), a still image acquired from the live image is fetched. If the operator presses the Auto Image Rotation switch SW 2 , the VBS image 53 a is rotated as shown in FIG. 15 so that the still image of the live image 52 and the VBS image 53 a will be matched with each other (step S 26 ).
- step S 27 the VBS image 53 a corresponding to a thumbnail VBS image 54 ( i ) representing the ninth branch point is displayed in the VBS image display area 53 . Only the frame of the thumbnail VBS image 54 ( i ) representing the ninth branch point is displayed in boldface or in color, whereby the operator recognizes that the next position of navigation is the ninth branch point (step S 28 ). Navigation is repeated in the same manner until the bronchoscope reaches the lesion (that is, the end point of navigation) (step S 29 ).
- step S 24 described in FIG. 11 If it is judged at step S 24 described in FIG. 11 that the navigation mode is not set to mode 1 in the image processing unit 17 having received a live image but set to mode 2 , control is passed to step S 41 in FIG. 17 .
- the navigation mode 2 is mode to be utilized by an operator who has the expertise in inserting a bronchoscope into bronchi. When an operator expects navigation but has a thorough knowledge of lumens of paths that are destinations inserted at almost all branch points, the operator designates the mode 2 instead of the mode 1 because he/she does not need the navigation to a predetermined position at each branch point.
- the Next VBS switch SW 1 is pressed.
- the position of a branch point represented by a live image can therefore be matched with the position thereof represented by a VBS image.
- navigation is not needed at each branch point until a bronchoscope reaches a predetermined position, insertion is continued, and navigation is expected at a certain branch point. In this case, it is hard to find a thumbnail VBS image representing the branch point at which navigation is expected.
- a thumbnail VBS image representing a branch point at which navigation is expected is searched based on a degree of insertion to which the insertion unit of a bronchoscope is inserted.
- the thumbnail VBS image 53 a is then displayed in the VBS image display area 53 , whereby navigation is enabled.
- navigation mode 2 if an operator does not need navigation, the operator continues insertion of the bronchoscope 2 while monitoring a live image.
- the operator presses the Freeze switch SW 4 .
- Step S 41 it is waited until the Freeze switch SW 4 is pressed. If the switch SW 4 is pressed, a reduced image 60 of a still image acquired with the press of the Freeze switch SW 4 is superimposed on the live image 52 a displayed in the endoscopic live image display area 52 .
- the Freeze switch SW 4 when the Freeze switch SW 4 is pressed, the bronchoscope lies at the eighth branch point. Navigation has not been needed by this time.
- the Freeze switch SW 4 is pressed for the first time.
- the VBS image 53 a displayed in the VBS image display area 53 is a VBS image corresponding to a thumbnail VBS image 54 ( a ) representing the first branch point.
- the degree-of-insertion detection unit 3 detects a degree of insertion to which the insertion unit of the bronchoscope 2 has been inserted into a patient body.
- the position of the distal end of the bronchoscope 2 and the position of a nearby branch point are calculated based on the detected degree of insertion.
- VBS images representing regions started with a branch point immediately preceding the calculated branch point and ended with an immediately succeeding branch point are displayed in the form of a motion picture in the VBS image display area 53 .
- the frame of only a thumbnail VBS image representing the calculated branch point is displayed in boldface or in color.
- FIG. 19 shows display of a motion picture composed of VBS images representing regions that start with the seventh branch point immediately preceding the eighth branch point, which is shown in FIG. 18 and serves as a reference point, and that end with the ninth branch point immediately succeeding the eighth branch point.
- FIG. 20 shows the navigation screen 51 with the VBS image 53 a being displayed, which represents the seventh branch point and with which display of the motion picture is started, displayed in the VBS image display area 53 thereof.
- FIG. 21 shows the navigation screen 51 with the VBS image 53 a being displayed, which represents the ninth branch point and with which the display of the motion picture is terminated, displayed in the VBS image display area 53 thereof.
- a motion picture domain bar 71 that indicates a domain to be displayed in the form of a motion picture in the VBS image display area 53 is displayed below the thumbnail VBS images displayed in the thumbnail VBS image area 54 .
- FIG. 20 and FIG. 21 show examples of display in which the motion picture domain bar 71 is displayed below the thumbnail VBS images that start with the thumbnail VBS image 54 ( g ) representing the seventh branch point and that end with the thumbnail VBS image 54 ( i ) representing the ninth branch point.
- the frame of only the thumbnail VBS image 54 ( h ) representing the calculated branch point is displayed in boldface or in color.
- an operator can recognize at what branch point the distal end of the insertion unit is currently located.
- the operator can therefore advance or withdraw the inserted bronchoscope so as to find out a live image representing a nearby branch point.
- the live image 52 a representing the branch point is displayed in the endoscopic live image display area 52 .
- step S 46 Once the live image 52 a representing a position at which navigation is needed is thus displayed, it is waited at step S 46 until the Freeze switch SW 4 is turned on. If the switch SW 4 is turned on, a still image of the live image 52 a is fetched at step S 47 . The VBS image 53 a representing the calculated branch point is displayed in the VBS image display area 53 . At step S 48 , the still image of the live image 52 a and the VBS image 53 a representing the calculated branch point are compared with each other in order to detect the degree of similarity between the images according to the known image processing technique. A process similar to the one executed in mode 1 is then executed.
- the VBS image 53 a corresponding to the thumbnail VBS image 54 ( h ) representing the calculated branch point is, as shown in FIG. 22 , displayed in the VBS image display area 53 .
- the frame of only the thumbnail VBS image 54 ( h ) representing the calculated branch point is displayed in boldface or in color.
- step S 48 similarly to the one in mode 1 , if the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than a determined route (or the still image is compared with a VBS image representing a different branch point). A warning is then displayed (not shown).
- step S 48 similarly to the one in mode 1 , if it is judged that the degree of similarity exceeds the predetermined value and the still image of the live image 52 a and the VBS image 53 a resemble each other, both the images are regarded to represent the same branch point. An operator then handles the Auto Image Rotation switch SW 2 or Manual Image Rotation switch SW 3 so as to rotate the VBS image 53 a . The still image of the live image 52 a and the VBS image 53 a are thus matched with each other. Thereafter, control is passed to step S 49 .
- step S 49 it is judged whether the bronchoscope has reached a lesion (that is, whether the bronchoscope has reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S 41 . The process from step S 41 to step S 49 will be repeated until the bronchoscope reaches the lesion.
- VBS images representing regions that start with a branch point immediately preceding a calculated branch point serving as a reference point and that end with an immediately succeeding branch point are displayed in the form of a motion picture in the VBS image display area 53 .
- the present invention is not limited to this mode.
- VBS images representing regions that start with a branch point of two points preceding a calculated branch point serving as a reference point and that end with a two point succeeding branch point may be displayed in the form of a motion picture in the VBS image display area 53 .
- thumbnail VBS images representing all branch points are displayed in the thumbnail VBS image area 54 .
- the frame of a thumbnail VBS image corresponding to the VBS image 53 a which is compared with the live image 52 a , is displayed in boldface or in color. Consequently, which of branch points is represented by the live image 52 a can be easily discerned.
- the VBS image 53 a representing a branch point at which navigation is needed can be retrieved easily.
- the VBS images 53 a representing regions near a region represented by the live image 52 a are displayed in the form of a motion picture. Consequently, what branch point is represented by the live image 52 a can be easily discerned.
- the live image 52 a and VBS image 53 a are compared with each other, and the degree of similarity between the images is calculated according to an image processing technique. If the degree of similarity is equal to or smaller than a predetermined value, a warning is given. An operator suspends insertion at a branch point which the operator has misidentified. The operator can easily dispose the distal end of the bronchoscope 2 at an appropriate branch point.
- the frame of the thumbnail VBS image that corresponds to the VBS image 53 a and that is displayed in the thumbnail VBS image area 54 is displayed in boldface or in color.
- the thumbnail VBS image corresponding to the VBS image 53 a may be enlarged, and the frame of the enlarged thumbnail VBS image may be displayed in boldface or in color.
- the number of branch points is nine, and the VBS image 53 a corresponding to the thumbnail VBS image representing the seventh branch point is displayed.
- thumbnail VBS images representing branch points are displayed in a row. If the number of branch points is large, thumbnail VBS images representing branch points may be, as shown in FIG. 24 , displayed in two or more rows. Moreover, as shown in FIG. 25 , if the frame of the thumbnail VBS image area 54 is structured such that thumbnail VBS images can be scrolled, a plurality of thumbnail VBS images can be scrolled and thus displayed.
- the marker 55 is superimposed on the VBS image 53 a in the navigation screen 51 in order to indicate a lumen of a path extending along a route.
- the present invention is not limited to this mode.
- the contour of the periphery of a lumen of a path extending along a route which is depicted in the VBS image 53 a may be displayed while being enhanced. Otherwise, the interior of a lumen of a path may be depicted in color.
- upward and downward directions (directions of gravity) depending on a patient's posture may be indicated in the VBS image 53 a within the navigation screen 51 .
- the upward and downward directions may also be, as shown in FIG. 28 , indicated in the live image 52 a.
- a marker 81 may be superimposed on the VBS image 53 a in order to indicate the position of a tissue that is an object of biopsy and that is located at a destination of insertion. The distance from a current position to the position of the tissue that is an object of biopsy and the diameter of the tissue may be indicated in the VBS image 53 a .
- a tissue image 82 that is patterned after the diameter or shape of a tissue may be displayed on the VBS image 53 a.
- an endoscope can be reliably navigated to reach a target region using guide images representing actual branch positions.
- an endoscope device in accordance with the present invention will prove effective when used as: an endoscope device that navigates an endoscope so as to help insert the endoscope into an intracorporeal lumen; an endoscope device that navigates a bronchoscope so as to help insert the bronchoscope into an intracorporeal lumen such as bronchi; or an endoscope device for industrial use that navigates an endoscope so as to help insert the endoscope into a lumen that branches out at points in multiple stages.
- an endoscope device in accordance with the present invention is the most suitable for reliably navigating the distal end of a bronchoscope.
Abstract
In an endoscope device 1 in accordance with the present invention, a navigation screen 51 is displayed. The navigation screen 51 includes: an endoscopic live image display area 52 in which a live image produced by a bronchoscope 2 is displayed; a VBS image display area 53 in which a VBS image is displayed; and a thumbnail VBS image area 54 in which images constructed by reducing VBS images that represent all branch points on a route are displayed as thumbnail VBS images representing branch points. Consequently, an endoscope can be reliably navigated to reach a target region using guide images that represent actual branch positions.
Description
- This application is a divisional application of U.S. patent application Ser. No. 10/498,155, filed Jun. 8, 2004, the content of which is incorporated herein by reference.
- The present invention relates to an endoscope device, or more particularly, an endoscope device capable of navigating an endoscope to help insert the endoscope into an intracorporeal lumen, for example, the bronchi.
- In recent years, diagnosis based on images has been widely adopted. For example, an X-ray computed tomography (CT) device is used to produce tomographic images of a subject so as to construct three-dimensional image data representing the interior of the subject. The three-dimensional image data is used to diagnose a lesion.
- As far as the CT system is concerned, while an X-ray irradiation and detection unit is continuously rotated, a subject is continuously moved in the direction of the body axis. Thus, helical scanning is performed on the three-dimensional field of the subject, and a three-dimensional image is produced based on tomographic images that represent successive sections of the subject in the three-dimensional field.
- One of such three-dimensional images is a three-dimensional image representing the bronchi in the lungs. The three-dimensional image of the bronchi is utilized in order to three-dimensionally grasp the position of an abnormality that is suspected to be, for example, a pulmonary carcinoma. In order to check the abnormality through biopsy, a bronchoscope is inserted in order to collect a sample of a tissue using a biopsy needle located in the distal section of the endoscope.
-
FIG. 31 shows abronchi 500. As far as an intracorporeal lumen that branches out in multiple stages is concerned, when the position of an abnormality is close to the terminal of a branch, it is hard to lead the distal end of an endoscope to a target region accurately for a short period of time. For example, Japanese Unexamined Patent Application Publication No. 2000-135215 has proposed a system for navigating a bronchoscope to a target region. Herein, a three-dimensional image of an intracorporeal lumen of a subject is constructed based on image data acquired from a three-dimensional field of the subject. The course to a target point along the lumen is determined using the three-dimensional image. Virtual endoscopic images representing regions of the lumen that lie along the course are constructed based on the image data, and displayed in order to navigate a bronchoscope. - However, the navigation to a target region to be performed in the system described in the Japanese Unexamined Patent Application Publication No. 2000-135215 includes display of a live endoscopic image produced by the bronchoscope. In addition, virtual endoscopic images representing branch points of bronchi are displayed for the purpose of guiding the bronchoscope into a destination of insertion. As mentioned above, the bronchi branch out in multiple stages. Moreover, images constructed with the bronchoscope located in the respective bronchi are images representing a plurality of branch destinations and resembling one another. When virtual endoscopic images are merely displayed, even if the virtual endoscopic images represent branch positions different from actual branch positions represented by a live endoscopic image, an operator may misidentify the virtual endoscopic images as images representing the correct branch positions. The misidentification may become a serious obstacle to navigation of the bronchoscope to a target region.
- The present invention attempts to break through the foregoing situation. An object of the present invention is to provide an endoscope device capable of reliably navigating an endoscope to a target region using guide images that represent actual branch positions.
- An endoscope device in accordance with the present invention comprises: three-dimensional image constructing means for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of images representing a three-dimensional field of the subject; an endoscope that picks up images of the intracorporeal lumen of the subject; and navigation image constructing means that constructs a navigation image which includes the endoscopic image of the intracorporeal lumen of the subject that is produced by the endoscope and the three-dimensional image. While presenting the course of the endoscope to be steered in order to insert the endoscope into the intracorporeal lumen of the subject, the endoscope device helps observe or treat the subject. Herein, the navigation image constructing means constructs the navigation image adding reduced images of three-dimensional images, which represent all branch points at which the intracorporeal lumen of the subject branches out, added thereto. Owing to these constituent features, the endoscope can be reliably navigated to reach a target region using guide images representing the actual branch positions.
-
FIG. 1 shows the configuration of an endoscope device in accordance with an embodiment of the present invention; -
FIG. 2 shows the configuration of an input unit shown inFIG. 1 ; -
FIG. 3 is a flowchart describing the flow of constructing navigation data to be executed by a bronchoscope navigation device shown inFIG. 1 ; -
FIG. 4 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 3 is performed; -
FIG. 5 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 3 is performed; -
FIG. 6 is a flowchart describing the flow of determining a route; -
FIG. 7 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 6 is performed; -
FIG. 8 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed; -
FIG. 9 is a third diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed; -
FIG. 10 is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown inFIG. 1 ; -
FIG. 11 is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 ; -
FIG. 12 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed; -
FIG. 13 is a second diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed; -
FIG. 14 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed; -
FIG. 15 is a fourth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed; -
FIG. 16 is a fifth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed; -
FIG. 17 is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 ; -
FIG. 18 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed; -
FIG. 19 is an explanatory diagram concerning a motion picture to be displayed by executing the process described inFIG. 17 ; -
FIG. 20 is a second diagram showing the navigation screen in which the execution of the process described inFIG. 17 is performed; -
FIG. 21 is a third diagram showing the navigation screen in which the execution of the process described inFIG. 17 is performed; -
FIG. 22 is a third diagram showing the navigation screen in which the execution of the process described inFIG. 17 is performed; -
FIG. 23 shows a first variant of the navigation screen shown inFIG. 14 ; -
FIG. 24 is a second variant of the navigation screen shown inFIG. 14 ; -
FIG. 25 is a third variant of the navigation screen shown inFIG. 14 ; -
FIG. 26 is a fourth variant of the navigation screen shown inFIG. 14 ; -
FIG. 27 is a fifth variant of the navigation screen shown inFIG. 14 ; -
FIG. 28 is a sixth variant of the navigation screen shown inFIG. 14 ; -
FIG. 29 is a seventh variant of the navigation screen shown inFIG. 14 ; -
FIG. 30 is an eighth variant of the navigation screen shown inFIG. 14 ; and -
FIG. 31 shows the structure of bronchi. - Referring to the drawings, an embodiment of the present invention will be described below.
-
FIG. 1 toFIG. 30 are concerned with an embodiment of the present invention.FIG. 1 shows the configuration of an endoscope device.FIG. 2 shows the configuration of an input unit shown inFIG. 1 .FIG. 3 is a flowchart describing the flow of constructing navigation data to be performed by a bronchoscope navigation device shown inFIG. 1 .FIG. 4 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 3 is performed.FIG. 5 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 3 is performed.FIG. 6 is a flowchart describing the flow of route determination to be executed at a step in the process described inFIG. 3 .FIG. 7 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 6 is performed.FIG. 8 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed.FIG. 9 is a third diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed.FIG. 10 is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown inFIG. 1 .FIG. 11 is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 .FIG. 12 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 13 is a second diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 14 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 15 is a fourth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 16 is a fifth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 17 is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 .FIG. 18 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 19 is an explanatory diagram concerning a motion picture to be displayed by executing the process described inFIG. 17 .FIG. 20 is a second diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 21 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 22 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 23 shows a first variant of the navigation screen shown inFIG. 14 .FIG. 24 shows a second variant of the navigation screen shown inFIG. 14 .FIG. 25 is a third variant of the navigation screen shown inFIG. 14 .FIG. 26 shows a fourth variant of the navigation screen shown inFIG. 14 .FIG. 27 is a fifth variant of the navigation screen shown inFIG. 14 .FIG. 28 is a sixth variant of the navigation screen shown inFIG. 14 .FIG. 29 is a seventh variant of the navigation screen shown inFIG. 14 .FIG. 30 is an eighth variant of the navigation screen shown inFIG. 14 . - As shown in
FIG. 1 , anendoscope device 1 in accordance with the present embodiment comprises: abronchoscope 2 that is inserted into the patient's bronchi in order to pick up images of the interiors of the bronchi or to biopsy a tissue of a lesion at the terminal of a bronchus; a degree-of-insertion detection unit 3 that detects a degree of insertion of the insertion unit of thebronchoscope 2 into the patient's body and that includes, for example, an encoder; aninput unit 4 formed with a flexible sheet that realizes a plurality of switches in the form of a film and that is disposed on a proximal end of thebronchoscope 2; and abronchoscope navigation device 6 that constructs virtual endoscopic images (hereinafter, VBS images) representing the interiors of bronchi on the basis of CT image data, synthesizes an endoscopic image (hereinafter, a live image), which is produced by thebronchoscope 2, with the VBS images according to an image pickup signal sent from thebronchoscope 2, a detection signal sent from the degree-of-insertion detection unit 3, and an input signal sent from theinput unit 4, displays the resultant synthetic image on amonitor 5, and navigates thebronchoscope 2 through the bronchi. - Incidentally, the
input unit 4 is formed on the proximal end of thebronchoscope 2. Alternatively, theinput unit 4 may be formed with a footswitch. - The bronchoscope navigation device 6 comprises: a CT image data fetching unit 11 that fetches three-dimensional image data, which is constructed in a known CT system that is not shown and that produces X-ray tomographic images of a patient, via a transportable storage medium, for example, a magneto-optical (MO) disk or a digital versatile disk (DVD); a CT image data storage unit 12 in which three-dimensional image data fetched by the CT image data fetching unit 11 is stored; an MPR image construction unit 13 that constructs an MPR image on the basis of the three-dimensional image data stored in the CT image data storage unit 12; a route determination unit 14 that constructs a route determination screen that will be described later and that contains an MPR image constructed by the MPR image constructing unit, and determines a navigation route (hereinafter, simply, a route) along which the bronchoscope 2 is navigated through bronchi; a VBS image construction unit 15 that constructs VBS images, which represent successive regions along the route determined by the route determining unit 14, in units of a frame on the basis of the three-dimensional image data saved in the CT image data storage unit 12; a VBS image storage unit 16 in which VBS images constructed by the VBS image constructing unit 15 are stored; an image processing unit 17 that receives an image pickup signal sent from the bronchoscope 2, a detection signal sent from the degree-of-insertion detection unit 3, and an input signal sent from the input unit 4, and constructs a navigation screen that will be described later and that comprises a live image, a VBS image, and a plurality of thumbnail VBS images; an image display control unit 18 that displays the route determination screen, which is constructed by the route determination unit 14, and the navigation screen, which is constructed by the image processing unit 17, on the monitor 5; and a setting information input unit 19 including a keyboard and a pointing device to be used to enter setting information in the route determination unit 14.
- Incidentally, the CT image
data storage unit 12 and VBSimage storage unit 16 may be realized with one hard disk, and the MPRimage construction unit 13,route determination unit 14, VBSimage construction unit 15, andimage processing unit 17 may be realized with one arithmetic circuit. Moreover, the CT imagedata fetching unit 11 fetches CT image data from a transportable storage medium such as a MO disk or a DVD. Alternatively, when a CT system or an in-house server in which CT image data is preserved is connected on a in-house LAN, the CT imagedata fetching unit 11 may be realized with an interface circuit that can be connected on the in-house LAN. In this case, the CT image data can be fetched over the in-house LAN. - The
input unit 4 has: as shown inFIG. 2 , a Next VBS switch SW1 for use in directing update of a VBS image that is contained in a navigation screen and that represents each branch point of bronchi; an Auto Image Rotation switch SW2 for use in directing automatic rotation of a VBS image so that the direction of the VBS image will correspond to the direction of a live image; a Manual Image Rotation switch SW3 for use in directing rotation of a VBS image in units of a predetermined angle of rotation so that the direction of the VBS image will correspond to the direction of a live image; and a Freeze switch SW4 for use in directing fetching of a still image of a live image. - The operation of the present embodiment having the foregoing components will be described below.
- As shown in
FIG. 3 , prior to observation or treatment performed using thebronchoscope 2, thebronchoscope navigation device 6 is activated. In thebronchoscope navigation device 6, at step S1, the CT imagedata fetching unit 11 fetches three-dimensional image data of a patient, which is constructed by a CT system. At step S2, the fetched three-dimensional image data is stored in the CT imagedata storage unit 12. - At step S3, the
route determination unit 14 displays theroute determination screen 21 like the one shown inFIG. 4 on themonitor 5. Patient information is selected through a patientinformation tag screen 22 contained in theroute determination screen 21. With the selection, an MPR image is constructed from multiple different planar images, for example, three different multi-sectional images of a patient selected at step S4. At step S5, theMPR image 23 is contained in theroute determination screen 21. - Incidentally, patient information is selected through the patient
information tag screen 22 by entering a patient identifier (ID), with which a patient is identified, using the settinginformation input unit 19. - Next, at step S6, the setting
information input unit 19 is used to select a route determination tag 24 (seeFIG. 4 ) in theroute determination screen 21. Consequently, a routedetermination tag screen 25 like the one shown inFIG. 5 appears in theroute determination screen 21. Route determination that will be described later is then executed in order to determine a route along which thebronchoscope 2 is navigated through bronchi in order to help insert thebronchoscope 2. - After a route along which the bronchoscope is navigated in order to help insert the bronchoscope is determined, the VBS
image construction unit 15 constructs successive VBS images, which represent regions along the entire route, in units of a frame. At step S8, the constructed VBS images are stored in the VBSimage storage unit 16. - The
bronchoscope navigation device 6 employed in the present embodiment can navigate thebronchoscope 2 through bronchi so as to help insert thebronchoscope 2 according to, for example, two navigation methods to be described later. The methods shall be implemented inmode 1 andmode 2 respectively. At step S9, the settinginformation input unit 19 is used to determine mode (navigation mode) for theroute determination unit 14. The information of the navigation mode is transmitted to theimage processing unit 17 via theroute determination unit 14, and stored in both theimage processing unit 17 androute determination unit 14. - By executing step S1 to step S9, preparations are made for navigation to be performed by the
bronchoscope navigation device 6 for observation or treatment by thebronchoscope 2. - Now, route determination to be executed at step S6 will be described in conjunction with
FIG. 6 . - As shown in
FIG. 6 , during the route determination of step S6, the settinginformation input unit 19 is handled in order to click a Route Search button in the routesetting tag screen 25 shown inFIG. 5 . A start pointentry direction window 31 that prompts a user to enter a start point of a route like the one shown inFIG. 7 is displayed in theroute determination screen 21 at step S11. A start point is defined in one of the tomographic images constituting theMPR image 23 by moving acursor 30 in theroute determination screen 21. Once the start point is defined, the start point is indicated at corresponding positions in the other two tomographic images included in theMPR image 23. An end pointentry direction window 32 that prompts a user to enter an end point of the route like the one shown inFIG. 8 is displayed in theroute determination screen 21. - At step S12, similarly to the definition of the start point, an end point is defined in one of the tomographic images constituting the
MPR image 23 by moving thecursor 30 in theroute determination screen 21. Once the end point is defined, the end point is indicated at corresponding positions in the other two tomographic images included in theMPR image 23. - After the start point and end point are defined, the
route determination unit 14 searches a route that links the start point and end point and that runs through bronchi. The bronchi have complex paths. The route that links the start point and end point and that runs through bronchi cannot therefore always be determined unconditionally. At step S13, theroute determination unit 14 therefore searches a first candidate for the route that links the start point and end point and that runs through bronchi. - The
route determination unit 14 then displays the route searched at step S14 by superimposing the route on theMPR image 23 contained in theroute determination screen 21 as shown inFIG. 9 . Moreover, aroute finalization window 33 that prompts a user to finalize a route is displayed. - The
route finalization window 33 comprises: a FinalizeRoute button 41 for use in directing finalization of a searched route; a SearchNext Candidate button 42 for use in directing search of the next candidate for a route; aRe-determine Route button 43 for use in redefining a start point and an end point; and a Cancelbutton 44 for use in canceling route search. - At step S15, it is judged whether the Search
Next Candidate button 42 is clicked. If the SearchNext Candidate button 42 is clicked, the next candidate for a route is automatically searched at step S16. Control is then passed to step S17. If the SearchNext Candidate button 42 is not clicked, control is passed to step S18. At step S17, it is judged whether the next candidate is found as a result of search. If the next candidate is unfound, the warning that the next candidate for a route is unfound is displayed, and control is returned to step S13. If the next candidate is found, control is returned to step S14. - At step S18, it is judged whether the
Re-determine Route button 43 is clicked. If theRe-determine Route button 43 is clicked, control is returned to step S11. If theRe-determine Route button 43 is not clicked, control is passed to step S19. - At step S19, it is judged whether the Finalize
Route button 41 is clicked. If the FinalizeRoute button 41 is not clicked, control is returned to step S15. If the FinalizeRoute button 41 is clicked, control is passed to step S20. At step S20, a route and positions of branch points on the route are determined, and control is returned to step S7 inFIG. 6 . - After the three-dimensional position information of the branch points on the route is determined at step S20, the order numbers of the branch points lying on the course to be steered in order to insert the bronchoscope from a
start point 101 to anend point 102 are appended to the VBS images representing the branch points. The VBS images are then stored in the VBSimage storage unit 16 at step S8. Consequently, the VBS image data representing each branch point includes virtual image data that represents the branch portion, three-dimensional position information concerning the branch portion, and information of an order number assigned to the branch portion lying on the selected course to be steered in order to insert the bronchoscope. - Incidentally, route search to be performed using an MPR image has been described. An
image 100 constructed using a volume rendering technique as shown inFIG. 10 may be used to designate thestart point 101 andend point 102 for the purpose of route search. - A description will be made of navigation of the
bronchoscope 2 which thebronchoscope navigation device 6 performs in order to help insert thebronchoscope 2 for observation or treatment after a route is determined as mentioned above. Incidentally, the description will be made by taking for instance a case where the number of branch points on a route is ten. - As described in
FIG. 11 , when thebronchoscope navigation device 6 starts navigation, anavigation screen 51 like the one shown inFIG. 12 is displayed on themonitor 5 at step S21. - The
navigation screen 51 comprises: an endoscopic liveimage display area 52 in which a live image captured by thebronchoscope 2 is displayed; a VBSimage display area 53 in which a VBS image is displayed; and a thumbnailVBS image area 54 in which images formed reducing VBS images representing branch points on a route are displayed as branch thumbnail VBS images. At step S21 at which thebronchoscope 2 has not yet been inserted, no live image is displayed in the endoscopic liveimage display area 52. However, aVBS image 53 a depicting the first branch point on the route is displayed in the VBSimage display area 53. Thumbnail VBS images 54(a) to 54(j) representing all the branch points are displayed in the thumbnailVBS image area 54. - A
marker 55 is superimposed on theVBS image 53 a in order to indicate a lumen of a path that advances along the route. Moreover, the frame of the thumbnail VBS image representing the same branch point as theVBS image 53 a displayed in the VBSimage display area 53 is displayed in boldface or in color so that the thumbnail VBS image can be discriminated from the others. An operator can easily discern which of the branch points is represented by a VBS image displayed in the VBSimage display area 53. In the stage of step S21, the frame of the thumbnail VBS image 54(a) is displayed in boldface or in color. - At step S22, inserting the
bronchoscope 2 into bronchi is started. As shown inFIG. 13 , at step S23, alive image 52 a captured by thebronchoscope 2 is displayed in the endoscopic liveimage display area 52 within thenavigation screen 51. Even inFIG. 13 , theVBS image 53 a representing the first branch point on the route is displayed in the VBSimage display area 53, and the thumbnail VBS images 52(a) to 54(j) representing all the branch points are displayed in the thumbnailVBS image area 54. The frame of the thumbnail VBS image 54(a) is displayed in boldface or in color. - Next, it is judged at step S24 whether navigation mode is set to
mode 1 in theimage processing unit 17 that has received a live image. - To begin with, navigation to be performed by the
bronchoscope navigation device 6 will be described on the assumption that the navigation mode is set tomode 1. - First, when the distal end of the
bronchoscope 2 reaches a first branch point, an operator judges whether or not to press the Freeze switch SW4. If the operator can discern the image of the lumen of the path indicated with themarker 55 on thelive image 52 a by merely viewing theVBS image 53 a in the VBSimage display area 53 and thelive image 52 a, the operator keeps inserting the bronchoscope. If it is hard to discern the lumen of the path indicated with themarker 55 on thelive image 52 a, the operator presses the Freeze switch SW4. - At step S25, it is judged whether the Freeze switch SW4 is turned on. If the Freeze switch SW4 is turned on, a still image acquired from the
live image 52 a is fetched at step S26. The still image and theVBS image 53 a representing the first branch point are compared with each other in order to detect the degree of similarity between them according to a known image processing technique. If the Freeze switch SW4 is not turned on, control is passed to step S27. - If it is found at step S26 that the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than the determined route (or, the still image is compared with a VBS image representing a different branch point). Consequently, a warning is displayed (not shown).
- If it is judged at step S26 that the still image of the
live image 52 a and theVBS image 53 a resemble each other at a degree of similarity exceeding the predetermined value, both the images are regarded to represent the same branch point. The operator then handles the Auto Image Rotation switch SW2 or Manual Image Rotation switch SW3 so as to rotate theVBS image 53 a so that the still image of thelive image 52 a and theVBS image 53 a will be matched with each other. Thereafter, control is passed to step S27. - Incidentally, when an operator presses the Auto Image Rotation switch SW2, the still image of the
live image 52 a and theVBS image 53 a are automatically matched with each other through, for example, pattern recognition processing. Moreover, when the operator presses the Manual Image Rotation switch SW3, theVBS image 53 a is rotated by a predetermined angle. The operator repeatedly presses the switch SW3 so as to match thelive image 52 a andVBS image 53 a with each other. - After the still image of the
live image 52 a and theVBS image 53 a are thus matched with each other, the operator can now discern the lumen of the path that is indicated with themarker 55 on thelive image 52 a. The operator therefore keeps inserting the bronchoscope according to the directive by themarker 55. - At step S27, it is judged whether the operator has pressed the Next VBR switch SW1. If the Next VBR switch SWl is not pressed, control is returned to step S25. If the operator has pressed the Next VBR switch SW1, the
VBS image 53 a corresponding to a thumbnail VBS image (b) representing the next (second) branch point is displayed in the VBSimage display area 53. At this time, the frame of the thumbnail VBS image 54(b) representing the second branch point is displayed in boldface or in color. The frame of the thumbnail VBS image 54(a) becomes the same as the frames of the other thumbnail VBS images. - At step S29, it is judged whether the bronchoscope has reached a lesion (that is, reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S25. The process from step S25 to step S29 is repeated until the bronchoscope reaches the lesion.
- A concrete example of the process from step S25 to step S29 will be described in conjunction with the
navigation screen 51. As shown inFIG. 14 , assuming that the bronchoscope has been navigated to reach the eighth branch point, theVBS image 53 a corresponding to a thumbnail VBS image 54(h) representing the eighth branch point is displayed in the VBSimage display area 53. Thelive image 52 a representing a region near the eighth branch point is displayed in the endoscopic liveimage display area 52. At this time, only the frame of the thumbnail VBS image 54(h) representing the eighth branch point is displayed in boldface or in color. This allows an operator to recognize that navigation is performed at the eighth branch point. - In this state, if the operator presses the Freeze switch SW4 (step S25), a still image acquired from the live image is fetched. If the operator presses the Auto Image Rotation switch SW2, the
VBS image 53 a is rotated as shown inFIG. 15 so that the still image of thelive image 52 and theVBS image 53 a will be matched with each other (step S26). - Thereafter, after the operator recognizes the lumen of the path that is a destination of insertion, if the operator presses the Next VBS switch SWl (step S27), the
VBS image 53 a corresponding to a thumbnail VBS image 54(i) representing the ninth branch point is displayed in the VBSimage display area 53. Only the frame of the thumbnail VBS image 54(i) representing the ninth branch point is displayed in boldface or in color, whereby the operator recognizes that the next position of navigation is the ninth branch point (step S28). Navigation is repeated in the same manner until the bronchoscope reaches the lesion (that is, the end point of navigation) (step S29). - Next, a description will be made of navigation to be performed by the
bronchoscope navigation device 6 in a case where the navigation mode is set tomode 2. - If it is judged at step S24 described in
FIG. 11 that the navigation mode is not set tomode 1 in theimage processing unit 17 having received a live image but set tomode 2, control is passed to step S41 inFIG. 17 . - The
navigation mode 2 is mode to be utilized by an operator who has the expertise in inserting a bronchoscope into bronchi. When an operator expects navigation but has a thorough knowledge of lumens of paths that are destinations inserted at almost all branch points, the operator designates themode 2 instead of themode 1 because he/she does not need the navigation to a predetermined position at each branch point. - However, in
mode 1, every time the bronchoscope reaches a branch point, the Next VBS switch SW1 is pressed. The position of a branch point represented by a live image can therefore be matched with the position thereof represented by a VBS image. However, assume that navigation is not needed at each branch point until a bronchoscope reaches a predetermined position, insertion is continued, and navigation is expected at a certain branch point. In this case, it is hard to find a thumbnail VBS image representing the branch point at which navigation is expected. - In the
navigation mode 2, a thumbnail VBS image representing a branch point at which navigation is expected is searched based on a degree of insertion to which the insertion unit of a bronchoscope is inserted. Thethumbnail VBS image 53 a is then displayed in the VBSimage display area 53, whereby navigation is enabled. - To be more specific, in
navigation mode 2, if an operator does not need navigation, the operator continues insertion of thebronchoscope 2 while monitoring a live image. When the distal end of thebronchoscope 2 reaches a branch point at which the operator needs navigation, the operator presses the Freeze switch SW4. - Navigation to be performed by the
bronchoscope navigation device 6 inmode 2 is described inFIG. 17 . Namely, at step S41, it is waited until the Freeze switch SW4 is pressed. If the switch SW4 is pressed, a reducedimage 60 of a still image acquired with the press of the Freeze switch SW4 is superimposed on thelive image 52 a displayed in the endoscopic liveimage display area 52. - Referring to
FIG. 18 , when the Freeze switch SW4 is pressed, the bronchoscope lies at the eighth branch point. Navigation has not been needed by this time. The Freeze switch SW4 is pressed for the first time. TheVBS image 53 a displayed in the VBSimage display area 53 is a VBS image corresponding to a thumbnail VBS image 54(a) representing the first branch point. - At step S43, the degree-of-
insertion detection unit 3 detects a degree of insertion to which the insertion unit of thebronchoscope 2 has been inserted into a patient body. At step S44, the position of the distal end of thebronchoscope 2 and the position of a nearby branch point are calculated based on the detected degree of insertion. - At step S45, VBS images representing regions started with a branch point immediately preceding the calculated branch point and ended with an immediately succeeding branch point are displayed in the form of a motion picture in the VBS
image display area 53. Moreover, the frame of only a thumbnail VBS image representing the calculated branch point is displayed in boldface or in color. -
FIG. 19 shows display of a motion picture composed of VBS images representing regions that start with the seventh branch point immediately preceding the eighth branch point, which is shown inFIG. 18 and serves as a reference point, and that end with the ninth branch point immediately succeeding the eighth branch point.FIG. 20 shows thenavigation screen 51 with theVBS image 53 a being displayed, which represents the seventh branch point and with which display of the motion picture is started, displayed in the VBSimage display area 53 thereof.FIG. 21 shows thenavigation screen 51 with theVBS image 53 a being displayed, which represents the ninth branch point and with which the display of the motion picture is terminated, displayed in the VBSimage display area 53 thereof. - As shown in
FIG. 20 andFIG. 21 , a motion picture domain bar 71 that indicates a domain to be displayed in the form of a motion picture in the VBSimage display area 53 is displayed below the thumbnail VBS images displayed in the thumbnailVBS image area 54.FIG. 20 andFIG. 21 show examples of display in which the motion picture domain bar 71 is displayed below the thumbnail VBS images that start with the thumbnail VBS image 54(g) representing the seventh branch point and that end with the thumbnail VBS image 54(i) representing the ninth branch point. At this time, the frame of only the thumbnail VBS image 54(h) representing the calculated branch point is displayed in boldface or in color. - Consequently, an operator can recognize at what branch point the distal end of the insertion unit is currently located. The operator can therefore advance or withdraw the inserted bronchoscope so as to find out a live image representing a nearby branch point. Eventually, the
live image 52 a representing the branch point is displayed in the endoscopic liveimage display area 52. - Once the
live image 52 a representing a position at which navigation is needed is thus displayed, it is waited at step S46 until the Freeze switch SW4 is turned on. If the switch SW4 is turned on, a still image of thelive image 52 a is fetched at step S47. TheVBS image 53 a representing the calculated branch point is displayed in the VBSimage display area 53. At step S48, the still image of thelive image 52 a and theVBS image 53 a representing the calculated branch point are compared with each other in order to detect the degree of similarity between the images according to the known image processing technique. A process similar to the one executed inmode 1 is then executed. - In the state shown in
FIG. 21 , if the Freeze switch SW4 is turned on, theVBS image 53 a corresponding to the thumbnail VBS image 54(h) representing the calculated branch point is, as shown inFIG. 22 , displayed in the VBSimage display area 53. Moreover, the frame of only the thumbnail VBS image 54(h) representing the calculated branch point is displayed in boldface or in color. - At step S48, similarly to the one in
mode 1, if the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than a determined route (or the still image is compared with a VBS image representing a different branch point). A warning is then displayed (not shown). - Moreover, at step S48, similarly to the one in
mode 1, if it is judged that the degree of similarity exceeds the predetermined value and the still image of thelive image 52 a and theVBS image 53 a resemble each other, both the images are regarded to represent the same branch point. An operator then handles the Auto Image Rotation switch SW2 or Manual Image Rotation switch SW3 so as to rotate theVBS image 53 a. The still image of thelive image 52 a and theVBS image 53 a are thus matched with each other. Thereafter, control is passed to step S49. - Once the still image of the
live image 52 a and theVBS image 53 a are matched with each other, an operator can easily discern a lumen of a path that is indicated with themarker 55 on thelive image 52 a . The operator continues insertion according to the directive given with themarker 55. - At step S49, it is judged whether the bronchoscope has reached a lesion (that is, whether the bronchoscope has reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S41. The process from step S41 to step S49 will be repeated until the bronchoscope reaches the lesion.
- At step S45, VBS images representing regions that start with a branch point immediately preceding a calculated branch point serving as a reference point and that end with an immediately succeeding branch point are displayed in the form of a motion picture in the VBS
image display area 53. The present invention is not limited to this mode. Alternatively, VBS images representing regions that start with a branch point of two points preceding a calculated branch point serving as a reference point and that end with a two point succeeding branch point may be displayed in the form of a motion picture in the VBSimage display area 53. - According to the present embodiment, when navigation is performed in
mode 1, thumbnail VBS images representing all branch points are displayed in the thumbnailVBS image area 54. Moreover, the frame of a thumbnail VBS image corresponding to theVBS image 53 a , which is compared with thelive image 52 a , is displayed in boldface or in color. Consequently, which of branch points is represented by thelive image 52 a can be easily discerned. - Moreover, when navigation is performed in
mode 2, theVBS image 53 a representing a branch point at which navigation is needed can be retrieved easily. TheVBS images 53 a representing regions near a region represented by thelive image 52 a are displayed in the form of a motion picture. Consequently, what branch point is represented by thelive image 52 a can be easily discerned. - Furthermore, when navigation is performed in
mode live image 52 a andVBS image 53 a are compared with each other, and the degree of similarity between the images is calculated according to an image processing technique. If the degree of similarity is equal to or smaller than a predetermined value, a warning is given. An operator suspends insertion at a branch point which the operator has misidentified. The operator can easily dispose the distal end of thebronchoscope 2 at an appropriate branch point. - Incidentally, in the
navigation screen 51, the frame of the thumbnail VBS image that corresponds to theVBS image 53 a and that is displayed in the thumbnailVBS image area 54 is displayed in boldface or in color. As shown inFIG. 23 , the thumbnail VBS image corresponding to theVBS image 53 a may be enlarged, and the frame of the enlarged thumbnail VBS image may be displayed in boldface or in color. Referring toFIG. 23 , the number of branch points is nine, and theVBS image 53 a corresponding to the thumbnail VBS image representing the seventh branch point is displayed. - In the
navigation screen 51, thumbnail VBS images representing branch points are displayed in a row. If the number of branch points is large, thumbnail VBS images representing branch points may be, as shown inFIG. 24 , displayed in two or more rows. Moreover, as shown inFIG. 25 , if the frame of the thumbnailVBS image area 54 is structured such that thumbnail VBS images can be scrolled, a plurality of thumbnail VBS images can be scrolled and thus displayed. - Furthermore, the
marker 55 is superimposed on theVBS image 53 a in thenavigation screen 51 in order to indicate a lumen of a path extending along a route. The present invention is not limited to this mode. For example, as shown inFIG. 26 , the contour of the periphery of a lumen of a path extending along a route which is depicted in theVBS image 53 a may be displayed while being enhanced. Otherwise, the interior of a lumen of a path may be depicted in color. - Furthermore, as shown in
FIG. 27 , upward and downward directions (directions of gravity) depending on a patient's posture may be indicated in theVBS image 53 a within thenavigation screen 51. The upward and downward directions may also be, as shown inFIG. 28 , indicated in thelive image 52 a. - Moreover, as shown in
FIG. 29 , amarker 81 may be superimposed on theVBS image 53 a in order to indicate the position of a tissue that is an object of biopsy and that is located at a destination of insertion. The distance from a current position to the position of the tissue that is an object of biopsy and the diameter of the tissue may be indicated in theVBS image 53 a . Instead of themarker 81, atissue image 82 that is patterned after the diameter or shape of a tissue may be displayed on theVBS image 53 a. - Therefore, according to the present embodiment, an endoscope can be reliably navigated to reach a target region using guide images representing actual branch positions.
- The present invention is not limited to the foregoing embodiment. Various modifications and variations can be made within the gist of the present invention.
- As mentioned above, an endoscope device in accordance with the present invention will prove effective when used as: an endoscope device that navigates an endoscope so as to help insert the endoscope into an intracorporeal lumen; an endoscope device that navigates a bronchoscope so as to help insert the bronchoscope into an intracorporeal lumen such as bronchi; or an endoscope device for industrial use that navigates an endoscope so as to help insert the endoscope into a lumen that branches out at points in multiple stages. In particular, when a target region is located at a terminal of an intracorporeal lumen that branches out at points in multiple stages, such as, bronchi, an endoscope device in accordance with the present invention is the most suitable for reliably navigating the distal end of a bronchoscope.
Claims (11)
1. An endoscopic image display method, comprising:
an endoscopic image data reception step of receiving image data that represents a lumen of a subject and that is picked up by an endoscope;
a first subject image data construction step of constructing first subject image data, which represents a lumen, on the basis of three-dimensional data acquired from the subject;
a second subject image data construction step of constructing second subject image data, which represents a lumen, on the basis of the three-dimensional data representing the interior of the subject;
an endoscopic image display step of displaying the endoscopic image on the basis of the endoscopic image data;
a first subject image display step of displaying the first subject image on the basis of the first subject image data; and
a second subject image display step of displaying the second subject image on the basis of the second subject image data.
2. The endoscopic image display method according to claim 1 , wherein the three-dimensional data acquired from the subject is data representing bronchi.
3. The endoscopic image display method according to claim 2 , wherein an image representing a branch point of the bronchi is constructed at the first subject image data construction step.
4. The endoscopic image display method according to claim 2 , wherein: images representing a plurality of different branch points of the bronchi are constructed at the second subject image data construction step; and the images representing the plurality of different branch points of the bronchi are displayed at the second subject image display step.
5. The endoscopic image display method according to claim 4 , wherein at least part of the images representing the branch points of the bronchi displayed at the second subject image display step are displayed in the order of their being juxtaposed along the course of the endoscope to be steered in order to insert the endoscope.
6. An endoscopic image display program comprising:
an endoscopic image data reception step of receiving image data that represents a lumen of a subject and that is picked up by an endoscope;
a first subject image data construction step of constructing first subject image data, which represents a lumen, on the basis of three-dimensional data acquired from the subject;
a second subject image data construction step of constructing second subject image data, which represents a lumen, on the basis of three-dimensional data representing the interior of the subject;
an endoscopic image display step of displaying an endoscopic image on the basis of the endoscopic image data;
a first subject image display step of displaying the first subject image on the basis of the first subject image data; and
a second subject image display step of displaying a second subject image on the basis of the second subject image data.
7. The endoscopic image display program according to claim 6 , wherein the three-dimensional data acquired from the subject is data representing bronchi.
8. The endoscopic image display program according to claim 7 , wherein an image representing a branch point of the bronchi is constructed at the first subject image data construction step.
9. The endoscopic image display program according to claim 7 , wherein: images representing a plurality of different branch points of the bronchi are constructed at the second subject image data construction step; and the images representing the plurality of different branch points of the bronchi are displayed at the second subject image display step.
10. The endoscopic image display program according to claim 9 , wherein at least part of the images representing the branch points of the bronchi displayed at the second subject image display step are displayed in the order of their being juxtaposed along the course of the endoscope to be steered in order to insert the endoscope.
11. An endoscopic image display method comprising:
an endoscopic image data reception step of receiving image data that represents a lumen of a subject and that is picked up by an endoscope;
a first subject image data construction step of constructing first subject image data, which represents a lumen, on the basis of three-dimensional data acquired from the subject;
a second subject image data construction step of constructing second subject image data, which represents a lumen, on the basis of the three-dimensional data representing the interior of the subject; and
a display step of displaying an endoscopic image that is based on the endoscopic image data received at the endoscopic image data reception step, a first subject image that is constructed at the first subject image data construction step, and a second subject image that is constructed at the second subject image data construction step.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/702,337 US20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2002223618 | 2002-07-31 | ||
JP2002-223618 | 2002-07-31 | ||
PCT/JP2003/009733 WO2004010857A1 (en) | 2002-07-31 | 2003-07-31 | Endoscope |
US10/498,155 US7641609B2 (en) | 2002-07-31 | 2003-07-31 | Endoscope device and navigation method for endoscope device |
US11/702,337 US20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2003/009733 Division WO2004010857A1 (en) | 2002-07-31 | 2003-07-31 | Endoscope |
US10/498,155 Division US7641609B2 (en) | 2002-07-31 | 2003-07-31 | Endoscope device and navigation method for endoscope device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20070142705A1 true US20070142705A1 (en) | 2007-06-21 |
Family
ID=31184975
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/498,155 Active 2024-12-05 US7641609B2 (en) | 2002-07-31 | 2003-07-31 | Endoscope device and navigation method for endoscope device |
US11/702,337 Abandoned US20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/498,155 Active 2024-12-05 US7641609B2 (en) | 2002-07-31 | 2003-07-31 | Endoscope device and navigation method for endoscope device |
Country Status (7)
Country | Link |
---|---|
US (2) | US7641609B2 (en) |
EP (1) | EP1466552B1 (en) |
JP (1) | JP4009639B2 (en) |
CN (1) | CN100364479C (en) |
AT (1) | ATE349944T1 (en) |
DE (1) | DE60310877T8 (en) |
WO (1) | WO2004010857A1 (en) |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080207997A1 (en) * | 2007-01-31 | 2008-08-28 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
US20080300458A1 (en) * | 2004-06-21 | 2008-12-04 | Byung Kyu Kim | Capsule Type Endoscope Control System |
US20090227837A1 (en) * | 2008-03-10 | 2009-09-10 | Fujifilm Corporation | Endoscopy system and method therefor |
US20100128116A1 (en) * | 2007-10-30 | 2010-05-27 | Olympus Corporation | Endoscope apparatus |
US20100287154A1 (en) * | 2009-05-07 | 2010-11-11 | Creative Technology Ltd. | Methods for searching digital files on a user interface |
US20110065982A1 (en) * | 2009-09-17 | 2011-03-17 | Broncus Technologies, Inc. | System and method for determining airway diameter using endoscope |
US20110184238A1 (en) * | 2010-01-28 | 2011-07-28 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US20110245607A1 (en) * | 2010-03-31 | 2011-10-06 | Kentaro Hayashi | Endoscopic gaseous material feed system |
EP2377457A1 (en) * | 2010-02-22 | 2011-10-19 | Olympus Medical Systems Corp. | Medical device |
US20130158352A1 (en) * | 2011-05-17 | 2013-06-20 | Olympus Medical Systems Corp. | Medical apparatus, method for controlling marker display in medical image and medical processor |
EP2641561A1 (en) * | 2012-03-21 | 2013-09-25 | Covidien LP | System and method for determining camera angles by using virtual planes derived from actual images |
US8591401B2 (en) * | 2010-08-18 | 2013-11-26 | Olympus Corporation | Endoscope apparatus displaying information indicating gravity direction on screen |
US9020229B2 (en) | 2011-05-13 | 2015-04-28 | Broncus Medical, Inc. | Surgical assistance planning method using lung motion analysis |
US9875544B2 (en) | 2013-08-09 | 2018-01-23 | Broncus Medical Inc. | Registration of fluoroscopic images of the chest and corresponding 3D image data based on the ribs and spine |
US10694929B2 (en) | 2014-12-15 | 2020-06-30 | Olympus Corporation | Medical equipment system and operation method of medical equipment system |
Families Citing this family (117)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7555333B2 (en) * | 2000-06-19 | 2009-06-30 | University Of Washington | Integrated optical scanning image acquisition and display |
US7998062B2 (en) | 2004-03-29 | 2011-08-16 | Superdimension, Ltd. | Endoscope structures and techniques for navigating to a target in branched structure |
EP2316328B1 (en) | 2003-09-15 | 2012-05-09 | Super Dimension Ltd. | Wrap-around holding device for use with bronchoscopes |
WO2005025635A2 (en) * | 2003-09-15 | 2005-03-24 | Super Dimension Ltd. | System of accessories for use with bronchoscopes |
JP4573517B2 (en) * | 2003-10-29 | 2010-11-04 | オリンパス株式会社 | Insertion support system |
EP2245983A3 (en) | 2003-10-31 | 2013-12-25 | Olympus Corporation | Insertion support system |
JP3847744B2 (en) | 2003-11-04 | 2006-11-22 | オリンパス株式会社 | Insertion support system |
US7901348B2 (en) * | 2003-12-12 | 2011-03-08 | University Of Washington | Catheterscope 3D guidance and interface system |
US8764725B2 (en) * | 2004-02-09 | 2014-07-01 | Covidien Lp | Directional anchoring mechanism, method and applications thereof |
WO2005079492A2 (en) | 2004-02-17 | 2005-09-01 | Traxtal Technologies Inc. | Method and apparatus for registration, verification, and referencing of internal organs |
US7517314B2 (en) | 2004-10-14 | 2009-04-14 | Karl Storz Development Corp. | Endoscopic imaging with indication of gravity direction |
CA2586560A1 (en) | 2004-11-05 | 2006-06-01 | The Government Of The United States Of America, As Represented By The Se Cretary, Department Of Health And Human Services | Access system |
US7811224B2 (en) * | 2004-11-09 | 2010-10-12 | Karl Storz Development Corp. | Method for dealing with singularities in gravity referenced endoscopic imaging |
US7751868B2 (en) | 2004-11-12 | 2010-07-06 | Philips Electronics Ltd | Integrated skin-mounted multifunction device for use in image-guided surgery |
US7805269B2 (en) | 2004-11-12 | 2010-09-28 | Philips Electronics Ltd | Device and method for ensuring the accuracy of a tracking device in a volume |
EP1838378B1 (en) * | 2005-01-18 | 2017-03-22 | Philips Electronics LTD | Apparatus for guiding an instrument to a target in the lung |
WO2006076789A1 (en) * | 2005-01-24 | 2006-07-27 | Claron Technology Inc. | A bronchoscopy navigation system and method |
US7221522B2 (en) | 2005-01-28 | 2007-05-22 | Karl Storz Development Corp. | Optical system for variable direction of view instrument |
US7662090B2 (en) * | 2005-02-07 | 2010-02-16 | Olympus Corporation | Endoscope system |
US7967742B2 (en) * | 2005-02-14 | 2011-06-28 | Karl Storz Imaging, Inc. | Method for using variable direction of view endoscopy in conjunction with image guided surgical systems |
US7530948B2 (en) * | 2005-02-28 | 2009-05-12 | University Of Washington | Tethered capsule endoscope for Barrett's Esophagus screening |
JP4776954B2 (en) * | 2005-03-17 | 2011-09-21 | オリンパス株式会社 | Endoscope insertion support device |
US9289267B2 (en) * | 2005-06-14 | 2016-03-22 | Siemens Medical Solutions Usa, Inc. | Method and apparatus for minimally invasive surgery using endoscopes |
WO2007002079A2 (en) | 2005-06-21 | 2007-01-04 | Traxtal Inc. | System, method and apparatus for navigated therapy and diagnosis |
CA2612603C (en) | 2005-06-21 | 2015-05-19 | Traxtal Inc. | Device and method for a trackable ultrasound |
WO2007011968A2 (en) | 2005-07-18 | 2007-01-25 | Seattle Genetics, Inc. | Beta-glucuronide-linker drug conjugates |
EP3492008B1 (en) | 2005-09-13 | 2021-06-02 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
US20070066881A1 (en) | 2005-09-13 | 2007-03-22 | Edwards Jerome R | Apparatus and method for image guided accuracy verification |
DE102005045602B4 (en) | 2005-09-23 | 2017-07-13 | Siemens Healthcare Gmbh | A method of supporting interventional medical intervention |
WO2007067163A1 (en) | 2005-11-23 | 2007-06-14 | University Of Washington | Scanning beam with variable sequential framing using interrupted scanning resonance |
US7547277B2 (en) * | 2005-12-15 | 2009-06-16 | Microvision, Inc. | Method and apparatus for calibrating an endoscope system |
JP2009528128A (en) * | 2006-03-03 | 2009-08-06 | ユニヴァーシティ オブ ワシントン | Multi-clad optical fiber scanner |
DE602006015913D1 (en) * | 2006-04-10 | 2010-09-16 | Storz Karl Dev Corp | Endoscopic imaging with indication of the direction of gravity |
WO2007129493A1 (en) | 2006-05-02 | 2007-11-15 | National University Corporation Nagoya University | Medical image observation support device |
US20080058629A1 (en) * | 2006-08-21 | 2008-03-06 | University Of Washington | Optical fiber scope with both non-resonant illumination and resonant collection/imaging for multiple modes of operation |
US20080086051A1 (en) * | 2006-09-20 | 2008-04-10 | Ethicon Endo-Surgery, Inc. | System, storage medium for a computer program, and method for displaying medical images |
US20080132834A1 (en) * | 2006-12-04 | 2008-06-05 | University Of Washington | Flexible endoscope tip bending mechanism using optical fibers as tension members |
US20080319307A1 (en) * | 2007-06-19 | 2008-12-25 | Ethicon Endo-Surgery, Inc. | Method for medical imaging using fluorescent nanoparticles |
US8457718B2 (en) * | 2007-03-21 | 2013-06-04 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in a patient image data |
US8155728B2 (en) * | 2007-08-22 | 2012-04-10 | Ethicon Endo-Surgery, Inc. | Medical system, method, and storage medium concerning a natural orifice transluminal medical procedure |
US20080221388A1 (en) * | 2007-03-09 | 2008-09-11 | University Of Washington | Side viewing optical fiber endoscope |
US20080221434A1 (en) * | 2007-03-09 | 2008-09-11 | Voegele James W | Displaying an internal image of a body lumen of a patient |
US20080234544A1 (en) * | 2007-03-20 | 2008-09-25 | Ethicon Endo-Sugery, Inc. | Displaying images interior and exterior to a body lumen of a patient |
US8081810B2 (en) * | 2007-03-22 | 2011-12-20 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in image data of a patient |
US8840566B2 (en) | 2007-04-02 | 2014-09-23 | University Of Washington | Catheter with imaging capability acts as guidewire for cannula tools |
US20080243030A1 (en) * | 2007-04-02 | 2008-10-02 | University Of Washington | Multifunction cannula tools |
WO2008137710A1 (en) * | 2007-05-03 | 2008-11-13 | University Of Washington | High resolution optical coherence tomography based imaging for intraluminal and interstitial use implemented with a reduced form factor |
JP2008301968A (en) * | 2007-06-06 | 2008-12-18 | Olympus Medical Systems Corp | Endoscopic image processing apparatus |
US8905920B2 (en) | 2007-09-27 | 2014-12-09 | Covidien Lp | Bronchoscope adapter and method |
US20090137893A1 (en) * | 2007-11-27 | 2009-05-28 | University Of Washington | Adding imaging capability to distal tips of medical tools, catheters, and conduits |
WO2009069394A1 (en) * | 2007-11-29 | 2009-06-04 | Olympus Medical Systems Corp. | Endoscope system |
EP2633810B1 (en) | 2008-02-12 | 2017-08-30 | Covidien LP | Controlled perspective guidance method |
US9575140B2 (en) * | 2008-04-03 | 2017-02-21 | Covidien Lp | Magnetic interference detection system and method |
EP2297673B1 (en) | 2008-06-03 | 2020-04-22 | Covidien LP | Feature-based registration method |
US8218847B2 (en) | 2008-06-06 | 2012-07-10 | Superdimension, Ltd. | Hybrid registration method |
US8932207B2 (en) | 2008-07-10 | 2015-01-13 | Covidien Lp | Integrated multi-functional endoscopic tool |
US20100121139A1 (en) | 2008-11-12 | 2010-05-13 | Ouyang Xiaolong | Minimally Invasive Imaging Systems |
US20110009694A1 (en) * | 2009-07-10 | 2011-01-13 | Schultz Eric E | Hand-held minimally dimensioned diagnostic device having integrated distal end visualization |
WO2010055817A1 (en) * | 2008-11-13 | 2010-05-20 | 株式会社 日立メディコ | Image processing device and image processing method |
DE102009010263B4 (en) * | 2009-02-24 | 2011-01-20 | Reiner Kunz | Method for navigating an endoscopic instrument during technical endoscopy and associated device |
US8337397B2 (en) | 2009-03-26 | 2012-12-25 | Intuitive Surgical Operations, Inc. | Method and system for providing visual guidance to an operator for steering a tip of an endoscopic device toward one or more landmarks in a patient |
US10004387B2 (en) | 2009-03-26 | 2018-06-26 | Intuitive Surgical Operations, Inc. | Method and system for assisting an operator in endoscopic navigation |
EP2413777B1 (en) * | 2009-04-03 | 2014-12-31 | Koninklijke Philips N.V. | Associating a sensor position with an image position |
US8611984B2 (en) | 2009-04-08 | 2013-12-17 | Covidien Lp | Locatable catheter |
EP2427867A1 (en) * | 2009-05-08 | 2012-03-14 | Koninklijke Philips Electronics N.V. | Real-time scope tracking and branch labeling without electro-magnetic tracking and pre-operative scan roadmaps |
JP5322794B2 (en) * | 2009-06-16 | 2013-10-23 | 株式会社東芝 | Endoscopy support system |
JP6200152B2 (en) * | 2009-06-29 | 2017-09-20 | コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. | Tracking method and apparatus in medical procedure |
US9962229B2 (en) * | 2009-10-12 | 2018-05-08 | Corindus, Inc. | System and method for navigating a guide wire |
JP5457841B2 (en) | 2010-01-07 | 2014-04-02 | 株式会社東芝 | Medical image processing apparatus and medical image processing program |
WO2011159834A1 (en) | 2010-06-15 | 2011-12-22 | Superdimension, Ltd. | Locatable expandable working channel and method |
US20130303887A1 (en) | 2010-08-20 | 2013-11-14 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation |
US9560953B2 (en) | 2010-09-20 | 2017-02-07 | Endochoice, Inc. | Operational interface in a multi-viewing element endoscope |
US20120130171A1 (en) * | 2010-11-18 | 2012-05-24 | C2Cure Inc. | Endoscope guidance based on image matching |
EP2581029B1 (en) * | 2011-01-24 | 2014-12-31 | Olympus Medical Systems Corp. | Medical device |
US9757021B2 (en) * | 2011-02-04 | 2017-09-12 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
JP2014209930A (en) * | 2011-08-31 | 2014-11-13 | テルモ株式会社 | Navigation system for respiration area |
JP5977497B2 (en) * | 2011-09-22 | 2016-08-24 | オリンパス株式会社 | Endoscope apparatus, operation method and program |
US9138165B2 (en) | 2012-02-22 | 2015-09-22 | Veran Medical Technologies, Inc. | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
WO2013132880A1 (en) * | 2012-03-06 | 2013-09-12 | オリンパスメディカルシステムズ株式会社 | Endoscopic system |
US10039473B2 (en) | 2012-05-14 | 2018-08-07 | Intuitive Surgical Operations, Inc. | Systems and methods for navigation based on ordered sensor records |
EP3524184B1 (en) * | 2012-05-14 | 2021-02-24 | Intuitive Surgical Operations Inc. | Systems for registration of a medical device using a reduced search space |
WO2014024995A1 (en) * | 2012-08-08 | 2014-02-13 | 株式会社東芝 | Medical image diagnosis device, image processing device and image processing method |
JP6233913B2 (en) * | 2013-01-07 | 2017-11-22 | 晃 竹林 | Endoscope navigation device |
US10499794B2 (en) | 2013-05-09 | 2019-12-10 | Endochoice, Inc. | Operational interface in a multi-viewing element endoscope |
EP4324382A2 (en) * | 2013-05-09 | 2024-02-21 | EndoChoice, Inc. | Operational interface in a multi-viewing elements endoscope |
US11547446B2 (en) | 2014-01-13 | 2023-01-10 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
US9370295B2 (en) | 2014-01-13 | 2016-06-21 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
US10342579B2 (en) | 2014-01-13 | 2019-07-09 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
US20150305650A1 (en) | 2014-04-23 | 2015-10-29 | Mark Hunter | Apparatuses and methods for endobronchial navigation to and confirmation of the location of a target tissue and percutaneous interception of the target tissue |
US20150305612A1 (en) | 2014-04-23 | 2015-10-29 | Mark Hunter | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
US10952593B2 (en) | 2014-06-10 | 2021-03-23 | Covidien Lp | Bronchoscope adapter |
US11188285B2 (en) * | 2014-07-02 | 2021-11-30 | Covidien Lp | Intelligent display |
CN107205781B (en) * | 2014-12-05 | 2020-03-13 | 科林达斯公司 | System and method for guiding a wire |
US10401611B2 (en) | 2015-04-27 | 2019-09-03 | Endochoice, Inc. | Endoscope with integrated measurement of distance to objects of interest |
US10426555B2 (en) | 2015-06-03 | 2019-10-01 | Covidien Lp | Medical instrument with sensor for use in a system and method for electromagnetic navigation |
CN108024695B (en) | 2015-08-11 | 2021-05-04 | 特里斯医疗有限公司 | Fully integrated disposable tissue visualization device |
JP6594133B2 (en) * | 2015-09-16 | 2019-10-23 | 富士フイルム株式会社 | Endoscope position specifying device, operation method of endoscope position specifying device, and endoscope position specifying program |
EP3359075B1 (en) * | 2015-10-09 | 2021-08-18 | Covidien LP | Apparatus comprising an angled endoscope and means for rotating the endoscope and for generating a panoramic view |
US10478254B2 (en) | 2016-05-16 | 2019-11-19 | Covidien Lp | System and method to access lung tissue |
JP6608111B2 (en) * | 2016-09-28 | 2019-11-20 | 富士フイルム株式会社 | MEDICAL IMAGE STORAGE / REPRODUCING DEVICE AND METHOD, AND PROGRAM |
US10638952B2 (en) | 2016-10-28 | 2020-05-05 | Covidien Lp | Methods, systems, and computer-readable media for calibrating an electromagnetic navigation system |
US10418705B2 (en) | 2016-10-28 | 2019-09-17 | Covidien Lp | Electromagnetic navigation antenna assembly and electromagnetic navigation system including the same |
US10792106B2 (en) | 2016-10-28 | 2020-10-06 | Covidien Lp | System for calibrating an electromagnetic navigation system |
US10722311B2 (en) | 2016-10-28 | 2020-07-28 | Covidien Lp | System and method for identifying a location and/or an orientation of an electromagnetic sensor based on a map |
US10446931B2 (en) | 2016-10-28 | 2019-10-15 | Covidien Lp | Electromagnetic navigation antenna assembly and electromagnetic navigation system including the same |
US10517505B2 (en) | 2016-10-28 | 2019-12-31 | Covidien Lp | Systems, methods, and computer-readable media for optimizing an electromagnetic navigation system |
US10615500B2 (en) | 2016-10-28 | 2020-04-07 | Covidien Lp | System and method for designing electromagnetic navigation antenna assemblies |
US10751126B2 (en) | 2016-10-28 | 2020-08-25 | Covidien Lp | System and method for generating a map for electromagnetic navigation |
CN110461209B (en) * | 2017-03-30 | 2021-10-29 | 富士胶片株式会社 | Endoscope system and processor device |
JPWO2019078237A1 (en) * | 2017-10-18 | 2020-10-22 | 富士フイルム株式会社 | Medical image processing equipment, endoscopy system, diagnostic support equipment, and medical business support equipment |
US11219489B2 (en) | 2017-10-31 | 2022-01-11 | Covidien Lp | Devices and systems for providing sensors in parallel with medical tools |
US11622753B2 (en) | 2018-03-29 | 2023-04-11 | Trice Medical, Inc. | Fully integrated endoscope with biopsy capabilities and methods of use |
CN110742690A (en) * | 2019-09-12 | 2020-02-04 | 东南大学苏州医疗器械研究院 | Method for configuring endoscope and terminal equipment |
CN111415564B (en) * | 2020-03-02 | 2022-03-18 | 武汉大学 | Pancreatic ultrasonic endoscopy navigation method and system based on artificial intelligence |
US20210378759A1 (en) * | 2020-06-03 | 2021-12-09 | Covidien Lp | Surgical tool navigation using sensor fusion |
JP2023167710A (en) | 2022-05-12 | 2023-11-24 | 富士フイルム株式会社 | Information processing device, bronchus endoscope device, information processing method, and program |
CN117221177B (en) * | 2023-11-08 | 2024-01-09 | 湖南省华芯医疗器械有限公司 | Image transmission delay monitoring method and system |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6346940B1 (en) * | 1997-02-27 | 2002-02-12 | Kabushiki Kaisha Toshiba | Virtualized endoscope system |
US6602185B1 (en) * | 1999-02-18 | 2003-08-05 | Olympus Optical Co., Ltd. | Remote surgery support system |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5417210A (en) * | 1992-05-27 | 1995-05-23 | International Business Machines Corporation | System and method for augmentation of endoscopic surgery |
US5526812A (en) * | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
US5803089A (en) * | 1994-09-15 | 1998-09-08 | Visualization Technology, Inc. | Position tracking and imaging system for use in medical applications |
US5765561A (en) * | 1994-10-07 | 1998-06-16 | Medical Media Systems | Video-based surgical targeting system |
US6151404A (en) * | 1995-06-01 | 2000-11-21 | Medical Media Systems | Anatomical visualization system |
US6702736B2 (en) * | 1995-07-24 | 2004-03-09 | David T. Chen | Anatomical visualization system |
US7954056B2 (en) * | 1997-12-22 | 2011-05-31 | Ricoh Company, Ltd. | Television-based visualization and navigation interface |
JP2000135215A (en) * | 1998-10-30 | 2000-05-16 | Ge Yokogawa Medical Systems Ltd | Conduit guiding method and device thereof and radiation tomographic equipment |
JP3850217B2 (en) * | 2000-12-27 | 2006-11-29 | オリンパス株式会社 | Endoscope position detector for bronchi |
US6961908B2 (en) * | 2001-12-05 | 2005-11-01 | International Business Machines Corporation | System and method for navigating graphical images |
-
2003
- 2003-07-31 DE DE60310877T patent/DE60310877T8/en active Active
- 2003-07-31 JP JP2004524322A patent/JP4009639B2/en not_active Expired - Lifetime
- 2003-07-31 US US10/498,155 patent/US7641609B2/en active Active
- 2003-07-31 EP EP03771449A patent/EP1466552B1/en not_active Expired - Lifetime
- 2003-07-31 CN CNB038020335A patent/CN100364479C/en not_active Expired - Lifetime
- 2003-07-31 AT AT03771449T patent/ATE349944T1/en not_active IP Right Cessation
- 2003-07-31 WO PCT/JP2003/009733 patent/WO2004010857A1/en active IP Right Grant
-
2007
- 2007-02-05 US US11/702,337 patent/US20070142705A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6346940B1 (en) * | 1997-02-27 | 2002-02-12 | Kabushiki Kaisha Toshiba | Virtualized endoscope system |
US6602185B1 (en) * | 1999-02-18 | 2003-08-05 | Olympus Optical Co., Ltd. | Remote surgery support system |
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080300458A1 (en) * | 2004-06-21 | 2008-12-04 | Byung Kyu Kim | Capsule Type Endoscope Control System |
US8672836B2 (en) * | 2007-01-31 | 2014-03-18 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
US20080207997A1 (en) * | 2007-01-31 | 2008-08-28 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
US20100201795A1 (en) * | 2007-10-30 | 2010-08-12 | Olympus Corporation | Endoscope apparatus |
US8004560B2 (en) | 2007-10-30 | 2011-08-23 | Olympus Corporation | Endoscope apparatus |
US20100128116A1 (en) * | 2007-10-30 | 2010-05-27 | Olympus Corporation | Endoscope apparatus |
US8353816B2 (en) * | 2008-03-10 | 2013-01-15 | Fujifilm Corporation | Endoscopy system and method therefor |
US20090227837A1 (en) * | 2008-03-10 | 2009-09-10 | Fujifilm Corporation | Endoscopy system and method therefor |
US20100287154A1 (en) * | 2009-05-07 | 2010-11-11 | Creative Technology Ltd. | Methods for searching digital files on a user interface |
WO2011035144A2 (en) * | 2009-09-17 | 2011-03-24 | Broncus Technologies, Inc. | System and method for determining airway diameter using endoscope |
WO2011035144A3 (en) * | 2009-09-17 | 2011-06-16 | Broncus Technologies, Inc. | System and method for determining airway diameter using endoscope |
CN102596003A (en) * | 2009-09-17 | 2012-07-18 | 布朗卡斯科技有限公司 | System and method for determining airway diameter using endoscope |
US20110065982A1 (en) * | 2009-09-17 | 2011-03-17 | Broncus Technologies, Inc. | System and method for determining airway diameter using endoscope |
US8696547B2 (en) * | 2009-09-17 | 2014-04-15 | Broncus Medical, Inc. | System and method for determining airway diameter using endoscope |
US20110184238A1 (en) * | 2010-01-28 | 2011-07-28 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US10667679B2 (en) * | 2010-01-28 | 2020-06-02 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
US20180220883A1 (en) * | 2010-01-28 | 2018-08-09 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
EP2377457A1 (en) * | 2010-02-22 | 2011-10-19 | Olympus Medical Systems Corp. | Medical device |
EP2377457A4 (en) * | 2010-02-22 | 2013-01-09 | Olympus Medical Systems Corp | Medical device |
US20110245607A1 (en) * | 2010-03-31 | 2011-10-06 | Kentaro Hayashi | Endoscopic gaseous material feed system |
US8591401B2 (en) * | 2010-08-18 | 2013-11-26 | Olympus Corporation | Endoscope apparatus displaying information indicating gravity direction on screen |
US9020229B2 (en) | 2011-05-13 | 2015-04-28 | Broncus Medical, Inc. | Surgical assistance planning method using lung motion analysis |
US20130158352A1 (en) * | 2011-05-17 | 2013-06-20 | Olympus Medical Systems Corp. | Medical apparatus, method for controlling marker display in medical image and medical processor |
US8876700B2 (en) * | 2011-05-17 | 2014-11-04 | Olympus Medical Systems Corp. | Medical apparatus, method for controlling marker display in medical image and medical processor |
EP2641561A1 (en) * | 2012-03-21 | 2013-09-25 | Covidien LP | System and method for determining camera angles by using virtual planes derived from actual images |
US9875544B2 (en) | 2013-08-09 | 2018-01-23 | Broncus Medical Inc. | Registration of fluoroscopic images of the chest and corresponding 3D image data based on the ribs and spine |
US10694929B2 (en) | 2014-12-15 | 2020-06-30 | Olympus Corporation | Medical equipment system and operation method of medical equipment system |
Also Published As
Publication number | Publication date |
---|---|
ATE349944T1 (en) | 2007-01-15 |
EP1466552A1 (en) | 2004-10-13 |
US20050020878A1 (en) | 2005-01-27 |
EP1466552B1 (en) | 2007-01-03 |
DE60310877T8 (en) | 2007-09-13 |
DE60310877T2 (en) | 2007-05-24 |
JPWO2004010857A1 (en) | 2005-11-24 |
US7641609B2 (en) | 2010-01-05 |
WO2004010857A1 (en) | 2004-02-05 |
DE60310877D1 (en) | 2007-02-15 |
CN1612708A (en) | 2005-05-04 |
CN100364479C (en) | 2008-01-30 |
JP4009639B2 (en) | 2007-11-21 |
EP1466552A4 (en) | 2005-10-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7641609B2 (en) | Endoscope device and navigation method for endoscope device | |
US7659912B2 (en) | Insertion support system for producing imaginary endoscopic image and supporting insertion of bronchoscope | |
US9613418B2 (en) | Method and device for examining or imaging an interior surface of a cavity | |
JP3847744B2 (en) | Insertion support system | |
US8509877B2 (en) | Endoscope insertion support system and endoscope insertion support method | |
JP3930423B2 (en) | Endoscope device | |
US20060195033A1 (en) | Insertion support system for specifying a location of interest as an arbitrary region and also appropriately setting a navigation leading to the specified region | |
US20050261550A1 (en) | System, apparatus, and method for supporting insertion of endoscope | |
WO2012068194A2 (en) | Endoscope guidance based on image matching | |
JP2005131319A (en) | Insertion support system | |
JP2004089484A (en) | Endoscope | |
JP4245880B2 (en) | Endoscope device | |
JP4022192B2 (en) | Insertion support system | |
CN112967276A (en) | Object detection method, object detection device, endoscope system, electronic device, and storage medium | |
JP4445792B2 (en) | Insertion support system | |
JP4160487B2 (en) | Insertion support system | |
JP4546064B2 (en) | Insertion simulation device | |
JP4354353B2 (en) | Insertion support system | |
US20240062471A1 (en) | Image processing apparatus, endoscope apparatus, and image processing method | |
JP4190454B2 (en) | Insertion support device | |
JP2006181110A (en) | Insertion support system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |