|Publication number||WO1995001757 A1|
|Publication date||19 Jan 1995|
|Filing date||5 Jul 1994|
|Priority date||7 Jul 1993|
|Publication number||PCT/1994/156, PCT/NL/1994/000156, PCT/NL/1994/00156, PCT/NL/94/000156, PCT/NL/94/00156, PCT/NL1994/000156, PCT/NL1994/00156, PCT/NL1994000156, PCT/NL199400156, PCT/NL94/000156, PCT/NL94/00156, PCT/NL94000156, PCT/NL9400156, WO 1995/001757 A1, WO 1995001757 A1, WO 1995001757A1, WO 9501757 A1, WO 9501757A1, WO-A1-1995001757, WO-A1-9501757, WO1995/001757A1, WO1995001757 A1, WO1995001757A1, WO9501757 A1, WO9501757A1|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (2), Referenced by (159), Classifications (23), Legal Events (6)|
|External Links: Patentscope, Espacenet|
ROBOTIC SYSTEM FOR CLOSE INSPECTION AND REMOTE TREATMENT OF MOVING PARTS
Background of the Invention
This invention relates to a robotic system for observing and remote treatment of moving parts. In a preferred embodiment the invention relates to a minimally invasive robotic surgical system that integrates automated target tracking of a moving body part by robotic surgical tools with stereoscopic video-image guided control of these tools by the surgeon. Closed chest coronary artery bypass graft surgery is presented as an example in which such a system is utilized to attach the distal end of a bypass graft to a coronary artery without the need to arrest the heart.
In the Western countries, coronary artery disease (narrowing of the arteries which supply blood to the muscle cells of the heart) is the leading cause of death and a major cause of morbid¬ ity.
The following details about coronary artery bypass graft surgery are largely derived from: Kirklin JW, Barratt-Boyes BG. Cardiac Surgery - Morphology, Diagnostic Criteria, Natural History, Techniques, Results, and Indications. New York: Churchill Living¬ stone, second edition, 1993 (for bypass surgery pp. 73-112; 143- 147; 175-177; 299-316).
If drug treatment of the consequences of coronary artery disease fails, the patient is treated by percutaneous transluminal coronary angioplasty (PTCA) or by coronary artery bypass graft (CABG) surgery. In 1991, in the US about 300,000 CABG operations were performed and approximately the same number of PTCAs. In 1991, the CABG operation and the subsequent 7-10 days in hospital was in the US associated with an average cost of $40,000. Thus, the cur¬ rent US expenditure for CABG exceeds 12 billion dollar.
The current CABG operation consists of the following sequence of steps: opening the chest by median sternotomy and spreading left and right rib cage; mobilization from chest wall of the left (and right) internal mammary artery (arterial grafts) and removal of greater saphenous vein from the leg (venous grafts); opening peri- card; arterial and venous cannulation for cardiopulmonary bypass (CPB); start extracorporeal circulation by the heart-lung machine which oxygenates the blood and produces the perfusion pressure for the body; cooling body to 32°C; aortic cross clamping and cold (4 degrees) cardioplegic perfusion of the coronary arteries via the aortic root to arrest and cool the heart; attachment of on average one arterial and three venous distal anastomoses (the distal bypass graft end is connected to the coronary artery beyond the obstruc¬ tion; the proximal bypass graft end is attached to the root of the aorta, unless the graft is the internal mammary artery, in that case the bypass blood supply originates from the subclavian artery, a major branch of the aortic arch); removal of aortic crossclamp and warming up; attachments of proximal anastomoses to ascending aorta after applying side biting clamp on root aorta and punching holes in the aortic wall; weaning from CPB and decannulation; closing chest. The operation usually requires about 3.5 hours. Cardiac arrest lasts about 1 hour. The bypass surgery itself requires some 80 minutes. Thus, conditioning the patient for CABG surgery and the subsequent deconditioning takes most of the operat¬ ing room time.
To appreciate the fundamental gains that potentially result from the present invention, the problems involved in these prepara¬ tory steps are listed in some detail.
Cardiopulmonary bypass (CPB) for cardiac surgery is concept- ually simple, and equipment is now available to accomplish it with ease. To enable arresting the heart, all the patient's systemic blood, which normally returns to the right atrium, is diverted into a device (oxygenator) in which oxygen is supplied to the blood and carbondioxide is removed. The newly arterialized blood is pumped from the oxygenator into the patient's aorta at adequate blood pressure for further distribution to the body. The extracorporeal circuit includes, in addition to the oxygenator and several pumps, a reservoir, a temperature exchange system to lower or increase the temperature of the blood, several filters and pressure, flow and temperature sensors.
The problems of CPB result from the facts that blood is unac¬ customed to travelling through nonendothelially lined channels, to receiving gaseous and particulate emboli, and to experiencing non- physiologic shear stresses. In particular, exposure of blood to foreign surfaces results in the activation of virtually all the humoral and cellular components of the inflammatory response, and some of the more slowly reacting specific immune responses are initially activated as well.
Details of this generalized response to extra-corporeal cir¬ culation include: neutrophil activation; platelet activation with a 40% drop in circulating platelets; depression of platelet function with sometimes strong bleeding tendency; complement activation; kallikein activation and bradykinin release; activation of the fibrinolytic cascade which may contribute to postoperative bleed¬ ing; enhanced production of interleukin-1 and Tissue Necrosis Fac¬ tor.
Additional complications of CPB include: loss of red blood cells (haemolysis) due to shear stress damage in the heart-lung machine; defective coagulation postoperatively because protamine only partly restores the coagulation cascade which was inhibited by heparin prior to CPB; in rare cases, (fatal) anaphylactic shock (low blood pressure) due to complement activation by protamine. The metabolic response to the stress of CPB is reflected in the large rise of plasma catecholamines during CPB. Finally, establishing CPB adds 15 minutes to the operating room time and it adds to the cost of CABG by the required assist¬ ance of a perfusionist during the entire procedure and by the expenses for both the standard equipment (roller pumps, etc) and a number of expensive disposables (oxygenator, etc).
With respect to cross-clamping the aorta the following remarks may be made. Prior to putting the patient on full CPB in order to start cold cardioplegic cardiac arrest, the aorta is clamped proximal to the insertion site of the arterial CPB cannula. The application of the aortic cross-clamp carries more than the usual hazard in patients undergoing CABG, because of the frequent presence of arteriosclerosis in the ascending aorta in these patients and the tendency of the cross-clamping to produce arterio- sclerotic emboli to the brain (about 0.5% embolic complications).
In spite of deep cooling of the heart, cold cardioplegic cardiac arrest, in combination with the CPB induced generalized inflammatory response, may result in temporary dysfunction of car- diac muscle cells (stunned or hibernating myocardium) or in perma¬ nent damage of the often already jeopardized heart. Insufficient capacity of the heart to pump an adequate amount of blood into the aorta is a life threatening hypotensive condition which requires intensive treatment. It is the major cause of hospital mortality (1-3%, depending on risk factors).
The principal disadvantage of open chest surgery is to the patient: almost 1% chance of sternal infection which severely pro¬ longs hospital stay and adds $42,000 to the hospital bill, and post-operative pain due to spreading of the ribcage. The post-oper¬ ative pain may lead to inadequate coughing and results in a 10-20% risk of pneumonia which also extends the hospital stay and requires expensive antibiotic treatment.
In addition to cardiopulmonary bypass, the patient undergoing cardiac surgery experiences all the stress responses characteristic of major operations and trauma.
Most patients survive cardiopulmonary bypass and the oper¬ ation, and convalesce in a reasonably normal manner. Even then, however, ^every patient retains for a time a few demonstrable stig¬ mata from the procedure, some have major morbidity, and a few die from their response to CPB and the surgery. The prevalence of these unfavourable outcome events in a group of patients is in part determined by the prevalence of identified risk factors, but the determinants of their occurrence and severity in an individual patient have not been completely described.
After isolated CABG operations, in a heterogeneous group of patients the one month death rate is 3.5%. The major cause of death and complications are failure of the heart muscle to recover ade¬ quately from the cold arrest in combination with the generalized inflammation response and deregulation of haemostasis. The low output state is associated with intensive drug treatment, aortic balloon pumping and implantation of temporary cardiac assist devices, all measures which add to the cost of the procedure.
During the first 6-8 weeks of convalescence from the CABG operation, patients commonly have a poor appetite, insomnia, emo¬ tional depression; visual, memory, or intellectual deficits; loss of sexual ability; lack of desire to return to work; and other potentially disabling manifestations of the postoperative state. Studies have documented the transient nature of most of these phe¬ nomena. Except in unusual circumstances, patients who were active and gainfully employed before surgery can return to full activity and employment within 2 to 3 months after surgery.
In the past two decades, to overcome the problems mentioned above, some cardiac surgeons have operated directly on the beating heart without CPB and without cardiac arrest, as is disclosed in: - Kluge TH, Kerth WJ, Gerbode F. Aorto-coronary bypass in experimental animals without the use of extracorporeal circulation. Scand J Thoracic Cardiovasc Surg 1972:6:257-61;
- Trapp WG, Bisarya R. Placement of coronary artery bypass graft without pump oxygenator. Ann Thorac Surg 1975;19:1-9;
- Akins CW, Boucher CA, Pohost GM. Preservation of interven- tricular septal function in patients having coronary artery bypass grafts without cardiopulmonary bypass. Am Heart J 1984;107:304-9;
- Benetti FJ. Direct coronary surgery with saphenous vein bypass without either cardiopulmonary bypass or cardiac arrest. Cardiovasc Surg 1985;26:217-20; - Buffolo E, Andrade JCS, Branco JNR, Aguiar LF, Ribeiro EE, Jatene AD. Myocardial revascularization without extracorporeal circulation: seven years experience in 593 cases. Eur J Cardio- thorac Surg 1990;4:504-8;
- Pfister AJ, Zaki MS, Garcia JM et al. Coronary artery bypass without cardiopulmonary bypass. Ann Thorac Surg 1992;5-
The distal anastomosis is made after immobilization of the target artery with traction sutures. However, the coronary artery has to be clamped to do the arterioto y and suture the anastomosis. As a result, this approach is limited to a subset of patients which are selected on the presence of occluded or almost occluded, well collateralized and easily accessible left anterior descending and right coronary arteries. This rare approach followed by the authors mentioned above is controversial because of poor results by some groups which is attributed to the trauma of immobilizing and clamp¬ ing the coronary artery.
An image-directed robotic system for precise robotic surgery has recently been described by Glasmann et al. (US Patent 5,086,401). It involves the presurgical planning and shaping of a bone implant for hip joint replacement based on computer tomography data and the subsequent drilling during surgery of the properly dimensioned hole in the leg bone by means of a programmed robotic drill. The robotic surgery is limited to programmed drilling on an immobilized bone. It does not involve surgery on a moving object and it does not include the combination of automated and superim¬ posed surgical movements of the robotic instruments guided by stereoscopic video input to the surgeon of the surgical target area.
Summary of the invention
A primary object of the present invention is to provide a system for close inspection in real time of moving parts, e.g., moving elements of a machine.
A further object of the present invention is to provide a medical system for close inspection in real time of a moving object within a chest cavity without the need to open the chest.
A further cbject of the present invention is to provide a system for precise manipulation of the moving object within the chest cavity, e.g. robotic precise surgery on the moving object, without the need to open the chest.
Still, a further object of the present invention is to pro¬ vide a medical system for closed chest robotic coronary artery bypass grafting on a beating heart.
To that purpose, the present invention provides an inspection system comprising:
- at least a first and a second camera to be mounted in a fixed position relative to each other and to direct their respect¬ ive optical ranges to a moving target; - an image processor receiving output signals from both the first and second cameras, the output signals containing video images of the moving target, the image processor being programmed to
* detect reference locations of beacons on the moving target area in a reference video image, as well as locations of the bea¬ cons in each subsequent video image; * compute a set of distances between the locations of the beacons in each subsequent video image and the corresponding refer¬ ence locations, the set of distances being used to translate and rotate the locations of the beacons in each subsequent video image in order to match them with the reference locations and to obtain a substantially arrested image of the moving target area (virtual target image arrest);
- display means connected to the image processor to receive and display the arrested image of the moving target area.
The main advantage of such a system is that it provides means to inspect accurately a moving part, e.g.," of a machine without the need to stop the movement of that part.
In another embodiment the invention provides an inspection and robotic tracking system comprising:
- at least a first and a second camera to be mounted in a fixed position relative to each other and to direct their respect¬ ive optical ranges to a moving target; - an image processor receiving output signals from both the first and second cameras, the output signals containing video images of the moving target, the image processor being programmed to
* detect reference locations of beacons on the moving target area in a reference video image, as well as locations of the bea¬ cons in each subsequent video image;
* compute a set of distances between the locations of the beacons in each subsequent video image and the corresponding refer¬ ence locations, the set of distances being used to translate and rotate the locations of the beacons in each subsequent video image in order to match them with the reference locations and to obtain a substantially arrested image of the moving target area (virtual target image arrest) ;
- display means connected to the image processor to receive and display the arrested image of the moving target area.
- at least one robot arm to be mounted in a fixed position relative to the first and second cameras;
- at least one manipulation instrument to be connected to the robot arm and to a robotic computer system to control its actions;
- a tracking control unit connected to the image processor and to the robot arm in order to supply the robot arm with control signals determined by output signals of the image processor;
- at least one control robotic instrument to be manually operated and connected to the robotic computer system to supply the robotic computer system with control signals.
By applying these measures, one may, e.g., repair a moving part of a machine without interrupting that machine. There may be situations in which interrupting a machine will cost a lot of money, which may, therefore, now be saved.
In the inspection and robotic tracking system defined above the set of distances being used to translate and rotate the loca¬ tions of the beacons in the subsequent video images in order to match them with the reference locations and to obtain a substan- tially arrested image of the moving target area is converted by the image processor into control signals for the robot arm in such a way that, when no human operator operates the control robotic instrument, the manipulation instrument connected to the robot arm substantially tracks the moving target area, i.e. the position of the manipulation instrument relative to the moving target area remains substantially the same.
In a preferred embodiment the present invention provides a medical system comprising: - at least a first and a second endoscopic video camera mounted on a first and a second mount, respectively, to be posi¬ tioned in a chest cavity in such a way that they both observe a predetermined moving target area within the chest cavity;
- an image processor receiving output signals from both first and second video cameras, the output signals containing video images of the moving target, the image processor being programmed to
* detect reference locations of beacons on the moving target area in a reference video image, as well as locations of the bea¬ cons in each subsequent video image; * compute a set of distances between the locations of the beacons in each subsequent video image and the corresponding refer¬ ence locations, the set of distances being used to translate and rotate the locations of the beacons in each subsequent video image in order to match them with the reference locations and to obtain a substantially arrested image of the moving target area (virtual target image arrest) ;
- display means and being connected to the image processor to receive and display the arrested image of the moving target area.
By applying these measures, a rapid close stereoscopic in¬ spection or diagnostics in real time of moving body parts within a chest cavity are possible without the need to arrest the moving body part in reality. These measures constitute important precondi¬ tions for remotely operating on a moving body part.
The beacons may be anatomic locations within the moving tar¬ get area. However, also artificial beacons may be installed on the moving target area.
In a preferred embodiment of the invention, the medical sys¬ tem defined above further comprises a main camera mounted on an adjustable ball bearing to be inserted into the chest between neighbouring ribs and to supply video image information of any area within the chest to at least one video monitor. These measures provide the operating team with a real time unmodified overview of the target area and its surroundings to allow the team to see what body parts are moving or standing still in reality.
The image processor may be connected to a voice activator receiving voice signals from a microphone, the voice activator, during operation, generating zooming-in or zooming-out commands for zooming-in or zooming-out the video image of the first and second cameras. By the application of these measures the primary surgeon does not need his hands for zooming-in on a desired target area.
In a further preferred embodiment the medical system further comprises:
- at least one robot arm to be mounted in a fixed position relative to the first and second cameras;
- at least one robotic surgical instrument to be connected to the robot arm and to a robotic computer system to control its actions;
- a tracking control unit connected to the image processor and to the robot arm in order to supply the robot arm with control signals determined by output signals of the image processor; - at least one control robotic instrument to be manually operated and connected to the robotic computer system to supply the robotic computer system with control signals.
In such a preferred embodiment the set of distances being used to translate and rotate the locations of the beacons in the subsequent video images in order to match them with the reference locations and to obtain a substantially arrested image of the mov¬ ing target area is converted by the image processor into control signals for the robot arm in such a way that, when no human oper- ator operates the control robotic instrument, the surgical in¬ strument connected to the robot arm substantially tracks the moving target area, i.e. the position of the surgical instrument relative to the moving target area remains substantially the same.
Preferably, the medical system also comprises ultrasound means to locate and evaluate a coronary artery on a beating heart, and at least two electromotor driven, rotating ultrasound trans¬ ducers (20-60 MHz), the ultrasound means being provided with suc¬ tion means to fix the ultrasound means to the beating heart.
The ultrasound means may have a hole to receive a robotic stapler/suturing device with one end of a bypass graft.
The robotic stapler/suturing device may comprise an adjust¬ able expansion ring and a thick walled cylinder provided with an aperture to receive the bypass graft to bypass an obstructed part of the coronary artery, wherein an end part of the bypass graft is to be expanded in diameter by the expansion ring and folded back around the thick walled cylinder by the expansion ring, and wherein the thick walled cylinder, during operation, accommodates bonding means to bond the end part of the bypass graft to the coronary artery (distal anastomosis) or to an aorta (proximal anastomosis).
Preferably such a medical system comprises hole making means to be inserted in the bypass graft through a temporary side branch after the bypass graft is bonded to the coronary artery.
By way of example the hole making means comprises a circular cutter within a cutter housing, which allows substantially resis¬ tance free rotation of the circular cutter within the bypass artery.
Moreover, the medical .system may comprise other hole making means to make a hole in an aortic wall, the other hole making means comprising a first circular rotary cutter driven by a flexible cable and a contra cutter with an exceedingly sharp screw front, the contra cutter being housed within the first circular cutter, being driven by another flexible cable housed within the first flexible cable, and to be screwed into and through the aortic wall to catch a circular piece of aortic wall cut out by the first cir- cular cutter.
Summarizing, among other things, the invention discloses an entirely novel method of performing coronary artery bypass graft
(CABG) surgery by means of a minimally invasive (thoracoscopic) approach (MICABG) without the need for cardiopulmonary bypass
(CPB), cardiac arrest and opening and spreading the ribcage. All procedures are performed with standard thoracoscopic techniques, but for clearing of the distal anastomosis sites if necessary and positioning an ultrasound guided anastomosis device (USGAD) con¬ taining a robotic stapler/suturing device and effecting the anasto- mosis bonding.
The method employs robotic tracking of cardiac surface motion to enable attaching the graft (donor vessel) onto the moving reci¬ pient coronary artery distal to its obstruction. The end-to-side anastomoses are made with a robotic, ultrasound guided sutur¬ ing/stapling device using an approach first described (and applied) on static arteries by Tulleken using current manual suturing tech¬ niques in:
- Tulleken CAF, Dieren Avan, Verdaasdonk RM: A new type of end to side anastomosis between arteries using the Neodynium-YAG laser with a sapphire tip (abstr). Lasers Med Sci 1998;3:335;
- Tulleken CAF, Verdaasdonk RM, Dieren Avan: End-to-side anastomosis without occlusion of the recipient artery, using the Neodynium YAG-laser with a sapphire tip (abstr). Lasers Med Sci 1991;6:88-9;
- Tulleken CAF, Van Dieren A, Verdaasdonk RM, Berendsen W. End-to-side anastomosis of small vessels using an Nd:YAG laser with a hemispherical contact probe (technical note) . J Neurosurgery 1992;76:546-9; - Tulleken CAF, Verdaasdonk RM: Use of the excimer laser in the end-to-side anastomosis between brain vessels. In: Anderson RR (ed), Laser Surgery: Advanced Characterization, Therapeutics and Systems III. Bellingham: SPIE vol. 1643, 1992, pp. 205-12;
- Tulleken CAF, Verdaasdonk RM: Use of excimer laser in high flow bypass surgery of the brain (abstr). Laser Medizin 1992;8:138;
- Tulleken CAF, Verdaasdonk RM, Berendsen W, Mali WPTM: Use of the excimer laser in high flow bypass surgery of the brain. J Neurosurg 1993;78:477-80.
The Tulleken approach obviates the need to interrupt the blood flow in the recipient artery. By sizing all instrumentation to the intercostal rib space, the procedure can be performed thoracoscopically. The present surgical approach is modified from the novel end-to-side anastomosis technique described by Tulleken et al. and relates to end-to-side anastomoses but is, in principle, applicable to side-to-side anastomoses (jump grafts) as well.
Benefits of the robotic Minimally Invasive Coronary Artery Bypass Graft (MICABG) system are:
I . no extracorporal circulation; 2. no cold cardioplegic cardiac arrest;
3. no need for perfusionist;
4. no need for expensive disposables associated with CPB;
5. no need to cross-clamp or side-clamp the aorta;
6. shorter grafts on the left side because the proximal anastomosis is on the aorta descendens rather than on the aorta ascendens;
7. reduced mortality of CABG surgery;
8. reduced morbidity;
9. reduced need for intensive treatment of haemostatic disorders or cardiac failure postoperatively; 10. reduced hospital stay (from 7-10 days to 4 days);
I I . reduced reconvalescence period and earlier return to normal activities;
12. reduced overall costs of the operation and hospitalization.
13. no large mid line thorax wound from neck to abdomen and subsequently, no large scar.
Brief description of the drawings
The above set forth and other features of the invention are made more apparent in the ensuing Detailed Description of the Invention when read in conjunction with the attached Drawings, wherein:
Figure 1 is a block diagram of the presently preferred em- bodiment of endoscopic stereoscopic (3-D) video imaging of a surgi¬ cal target, together with the possibility to view an entire oper- ation field on a video monitor or to view an operating room.
Figure 2A schematically depicts a beating heart and immobile other structures in a chest cavity.
Figure 2B schematically depicts effects of virtual target image arrest with in the image a virtually arrested heart and 'beating' structures in a 'beating' chest cavity.
Figure 3 schematically depicts the surgical target area on the beating heart with traction sutures to reduce coronary" artery motions and motions of beacons in the vicinity of the surgical target area.
Figure 4 is a block diagram showing arbitrary positions of beacons in a reference video image and in subsequent images; by matching the position of the beacons in the latter images to the beacons in the reference image, virtual target image arrest is obtained.
Figure 5 is a block diagram of virtual target image arrest combining figs. 1, 3 and 4, illustrating EKG triggering for the reference image.
Figure 6 is a schematic diagram of operator controlled (scaled) motions of surgical instruments which are superimposed on automated target tracking movements of robotic arms with surgical instruments.
Figure 7 illustrates attachment of an expandable ring to an end of a bypass graft.
Figure 8 is a schematic diagram of a robotic stapler/suturing device around the end of the graft.
Figure 9 is a schematic cross-section of the end of the robotic stapler/suturing device and the everted end of a graft.
Figure 10A is a schematic drawing (transparent top view) of an ultrasound guidance device with two electromotor driven rotating ultrasound transducers adjacent to a central hole which can accom¬ modate a robotic stapler/suturing device with an end of the bypass graft.
Figure 10B provides a longitudinal section of the ultrasound guidance device of Figure 10A.
Figure 10C provides a cross-section of the ultrasound guid¬ ance device at the level indicated by arrow C in figure 10B.
Figure 10D provides a cross-section of the ultrasound guid¬ ance device at the level indicated by arrow D in figure 10B.
Figure 10E provides a cross-section of the ultrasound guid¬ ance device at the level indicated by arrow E in figure 10B.
Figure 11 is a schematic longitudinal section of the ultra¬ sound guided anastomosis device (USGAD) with the bypass graft (with temporary side branch), its end folded back over the robotic sta¬ pler/suturing device which fits into the ultrasound guidance device.
Figure 12A is a magnification of the robotic stapler end from fig. 11 after ejection of staples.
Figure 12B gives a schematic top view of circumferential staple bonding.
Figure 12C illustrates a preformed staple which has penetrated the graft wall at both ends and which remains within the wall of a recipient artery. Figure 12D illustrates a preformed staple which has penetrated the graft wall at one end and folds back on itself.
Figure 13 shows a schematic drawing of a circular cutter introduced through the temporary side branch of the bypass graft to establish continuity between graft lumen and recipient coronary artery lumen by punching a hole in the coronary artery wall.
Figure 1 shows, similar to Figure 13, a schematic drawing of a circular (or ellipsoid) spark erosion electrode to ablate a ring of coronary artery wall tissue to punch a hole.
Figure 15 shows a schematic drawing of a cutter and contra- cutter with sharp screw-end of an aortic wall punch.
Detailed description of the invention
Endoscopic stereoscopic (3-D) video imaging viewed through operating spectacles is shown in Figure 1. The preferred thoraco- scopic surgical technique uses two intrathoracic CCD cameras 1 , 2.
To allow the surgeon to see depth, stereoscopic video imaging is implemented to the separate eyes in operating spectacles 111.
The two directional endoscopic CCD cameras 1, 2 are posi- tioned on ball bearings 3, 4 at the end of mounts 5, 6, respective¬ ly. The mounts 5, 6 are inserted through the chest between two neighbouring ribs, one of which 110 is shown in Figure 1. The mounts 5, 6 are mounted on a cross bar 7 at variable distance and right angles. The bearings 3, 4 are externally controlled like eyes. First, ball bearing 3 is adjusted by the surgeon to direct CCD camera 1 to get the target segment 22 of the coronary artery 23 in the middle of its video image. Next, ball bearing 4 is adjusted to direct CCD camera 2 to get the target segment 22 of the coronary artery 23 in the middle of its video image. The angles α, β are defined by the normal to the cross bar 7 and the normals to the input planes of CCD cameras 1 , 2, respectively. The angle γ is defined by the plane through mounts 5, 6 and the plane through the normals to the input planes of CCD cameras 1, 2. The angles α, β, γ are monitored and fed to a digital image processor 8. The adjust¬ able distance between the CCD camera mounts 5, 6 on the cross bar 7 is monitored and fed to the digital image processor 8. From angles α, β, γ and the distance between CCD cameras 1 and 2 (mounts 5, 6), the distance calibration in the stereoscopic video image is com¬ puted by image processor 8. This embodiment will optimize stereo¬ scopic vision of the target over a sufficient space angle and com- ply with anatomical constraints on making access holes in the chest wall in between ribs. Illumination of an object within the chest is provided separately in standard thoracoscopic fashion by means of light sources 9 and 10.
Two synchronized video systems 11 simultaneously produce the image of the first CCD camera 1 and the image of the second CCD camera 2. Both images are fed to the digital image processor 8 which creates simultaneously the digital output to "operating spec¬ tacles" 111 (Figure 1). In these "spectacles" a usual optical lens system (magnification about 3x) has been replaced by a "left" and a "right" video image, respectively. Video image display is by color liquid crystal display and optical means or by other means which provide adequate resolution of a color image in about 2-4 cm2.
A primary surgeon 101 has a microphone 12 which allows voice activation of the digital zoom capability of the images from the CCD cameras 1 , 2 by a voice control unit 13 through the image pro¬ cessor 8. The digital zoom of the images from the CCD cameras 1, 2 does not need to be centred at the centre of the image. By voice activation, the centre of the zoom field is called to be displayed as cursor in the center of the image and, next, is moved by voice command to a point whose coordinates are read at the edges of the video image. Zooming-in will scale the original coordinates. These coordinates are derived from the angles α, β and γ involved in the CCD camera 1 , 2 positions as well as from the distance between CCD cameras 1 and 2. Of course, the zooming may be provided by other means than by voice activation, e.g. by a foot switch.
By providing other members 102, 103, 104 of the operating team and any visitors 105, 106 with such operating spectacles 111, the endoscopic operating procedure can be closely followed by all in 3-D.
The left and right (zoom) images may be recorded on two syn¬ chronized video recorders 14, 15 coupled to the image processor 8, which are also preferably activated by voice command through voice control unit 13. These recorders 14, 15 may monitor an audio chan¬ nel as well.
In addition to the stereoscopic video image in the operating spectacles system, the entire surgical area of interest is moni¬ tored by a standard thoracoscope 107 which has a CCD camera 19 mounted on a ball bearing 18 to allow vision in all directions. General illumination of the object region is provided by a main light source 108, which supplies light in the standard thoraco- scopic fashion. The output of CCD camera 19 is fed to the video system 17, which has e.g. 4 monitors connected: a first monitor 16 at the head of the operating table (not shown), a second monitor 16a at the right side of the operating table, a third monitor 16b at the left side of the operating table, and a fourth monitor 16c for the cardio-anaesthesiologist. In this way all members 101-104 of the surgical team have both an overview of the entire surgical field within the thorax and a magnified stereoscopic view of the grafting area.
The major advantage of incorporating the left and right video images in operating spectacles 111 is that a cardiac surgeon is used to perform CABG surgery with operating spectacles which mag¬ nify the target region about 3 times, but which leave peripheral vision undisturbed. As a result, in the new approach described here the surgeon can concentrate on the target, use zooming in if neces¬ sary, and at the same time, keep visual contact with a monitor 16 displaying, for instance, the general view of the heart and chest cavity, with the monitors displaying the EKG and haemodynamic para¬ meters of the patient, with his hands and the instruments outside the body, and with the other people in the operating room.
Figures 2 to 5 relate to virtual cardiac target image arrest which will now be explained. The surface of e.g. a beating heart 20 (Fig. 2A) shows (regular) motion with respect to the chest cavity 21 and other objects which are stationary with respect to the oper- ating table (not shown). Hence, the potential site of the distal anastomosis of the bypass graft, the target area 22, moves with respect to the chest wall, the thoracoscope 107, CCD cameras 1, 2 and thoracoscopic instruments. It is preferred to partly immobilize the coronary artery segment (Fig. 3) which is to be grafted 23. To that purpose, by current endoscopic techniques several traction sutures 25 are placed around the target area 22, which is distal to a (partially) obstructed part of a coronary artery 23.
In the vicinity of the target area 22, beacons 24 are ident- ified. The beacons 24 may be clearly identifiable anatomic struc¬ tures or clips placed on the surface of the heart 20 or e.g. tiny LED's which are temporarily attached to the surface of the heart 20.
The patient is under beta-blockade to reduce heart rate and reduce the electrical irritability of the cardiac muscle to mechan¬ ical stimulation. If in spite of the anaesthesia, the heart rate or rhythm fluctuates considerably, a temporary right heart pacing electrode may be attached to the right auricle to pace the heart slightly above its own highest rate.
At end-diastole one video image (left and right) is frozen, preferably, by voice command. End-diastolic freezing of the video image is accomplished by triggering on the QRS complex of the elec- trocardiogram (EKG) with an appropriate delay. Beacons 24 are defined interactively (mouse or joy stick controlled cursor in video image) on the surface of the heart near the edge of the tar¬ get area (Fig. 3). Their 3-D coordinates are computed relative to the coordinate system of the two video cameras 1 , 2. This coordinate system is defined by the plane through cross bar 7 and the mounts 5, 6, with the origin in the middle of cross bar 7. In each following stereoscopic image the beacons are recognized by fast computer image analysis algorithm (Fig. 4).
Figure 4 shows the procedure how the image processor 8 pro- cesses the stereoscopic video images received from CCD cameras 1 , 2. First, an image is chosen as reference image 115 with beacons 24, as explained below. In all subsequent images, e.g. image 116, the beacons 24' are displaced relative to the positions of the beacons 24 shown in video image 115. Since the actual displacement of the heart is three dimensional, the displacements of the beacons 24' in video image 116 relative to their original positions in video image 115 are also three dimensional.
Video image 117 in Figure 4 shows a coinciding stereoscopic image of video images 115, 116. The arrows P denote the 3D dis¬ placements of beacons 24' in video image 116 to the original bea¬ cons 24 in video image 115 . Video image 118 shows beacons 24' matching beacons 24 when beacons 24' are virtually shifted back along arrows P (and rotated) to their original positions in video image 115. Why beacons 24' are virtually shifted back to their original positions will be explained below.
Figure 5 shows a flow chart of virtual target arrest by image processing by image processor 8. In step 120, video images of the moving surgical target 22 on the beating heart 20 are received by video processor 8. In step 121, an end-diastolic image, for instance video image 115 (Figure 4), is frozen using EKG triggering 122 input to the image processor 8 (not shown). After unfreezing the video display, in the original video image 115 the beacons 24 are identified. In step 124, in each subsequent video image, e.g. image 116 (Figure 4), the positions of the beacons 24' are recog- nized and determined. Next, the 3-D distance between each of the beacons 24 in video image 115 and the corresponding beacons 24' in video image 116 are computed in step 125. Then, in step 126 the video image 116 is translated and rotated in such a way by appro- priate computations that video images 115 and 116 match each other, as shown in video image 118 in Figure . This process is repeated with each subsequent video image resulting in virtual image arrest (cardiac target arrest) in step 127. As a consequence, non-cardiac structures will appear to be beating (Figure 2B), when the target arrested video image 127 is transmitted 128 to the spectacles 111 of the surgeon 101 (Figure 1).
An absolute condition for virtual target arrest is that the target area and the nearby beacons never disappear from the orig- inal video image during the cardiac cycle and the respiratory cycle.
If robotic tracking of the target (vide infra) is not fast enough due to a relatively long interval between video-images (40 ms), the standard frame rate (25 Hz) is increased, or detection and tracking of beacons by other means is employed.
Figure 6 shows how the system shown in Figures 1-5 may be used during an operation of a body part within the chest, for instance, a coronary artery 23. The Figure shows the two mounts 5, 6 for the CCD cameras 1 and 2, respectively, which transmit their video signals to the image processor 8. For the sake of clarity, the light sources 9, 10 and the synchronized video systems 11 as depicted in Figure 1 have been omitted in Figure 6. By using the method of Figure 5, image processor 8 generates a video image of the virtually arrested object of interest, e.g. the target segment 22 of coronary artery 23. The arrested video image is outputted to the spectacles 111 of at least the surgeon 101. Then, the computa¬ tional algorithm to obtain the arrested target image is employed in real time to translate and rotate robot arms 32, 33 with robotic surgical instruments 34, 35, the robot arms 32, 33 being inserted between two neighbouring ribs like 110 in the same way as the two mounts 5, 6 of CCD cameras 1, 2. The robot ...rms 32, 33 with their robotical surgical instruments 34, 35 receive tracking signals from tracking control 31 in order to translate and rotate them with respect to the coordinate system in precisely the opposite way as does the video image 116 (Figure 4) to match video image 115. As a result, the robotic tracking arms 32, 33 with their robotic surgi¬ cal instruments 34, 35 track the target 22 in real time. In other words, if the surgeon 101 does not interfere, the (small) distance between the surgical instruments 34, 35 and the target 23 remains constant, whatever the movements of the target area 23. The track¬ ing of each of the two arms 32, 33 may preferably be switched on/off by voice control unit 13.
To insert the robot arms 32, 33 with surgical instruments 34, 35 through appropriate holes between neighbouring ribs, they will either have to be inserted before fixing them to crossbar 7 or will have to be connected to crossbar 7 by suitable fixing means (not shown) allowing the freedom of insertion through the holes without being limited by the crossbar 7, which is afterwards connected to the operating table in a stable fashion (not shown) . To exchange a surgical instrument 34, 35/ the instrument is disconnected either within or outside the chest from robotic arm 32, 33 and exchanged.
Preferably, during operating a patient the patient is laid on his back or on one of his sides and one lung is collapsed in order to create suitable operating space for CCD cameras 1, 2, 19, the robot arms 32, 33 and the instruments 34, 35.
The surgeon 101 manually handles control robotic instruments 36a, 36b (with left hand and right hand, respectively) (e.g. tweez¬ ers) which control a robotic computer system 37, which in turn steers by means of independent controls 38, 38' the output surgical instruments 34, 35 (robotic telesurgery) which are mounted on the tracking robot arms 32, 33. Any movement of the surgeon 101 with the control robotic instruments 36a, 36b is translated - with or without voice command controlled scaling down of movements - to the robotic surgical instruments 34, 35. The combined motion of robotic surgical instrument 34 resulting from the control signals generated by tracking control 31 and the control robotic instrument 36a results in operation on the moving target 22 by robotic surgical instrument 34. The same applies to surgical instrument 35. The surgeon 101, in contrast, experiences the procedure in his operat¬ ing spectacles 111 as operating on the arrested target, whereas one look over the rim of the operating spectacles 111 at the video monitor 16 will tell him that he is working on the moving target 22.
Now, a circular or ellipsoid robotic stapler/suturing device (ultrasound guided anastomosis device USGAD) will be described, referring to Figures 7-12. An expandable ring 41 (like key ring) is attached to the end of a bypass graft 40 by a running suture 42, and expanded (Figure 7). The end of the graft 40 now looks like a trumpet. The opening of this "trumpet" may be circular, however, also an ellipsoid form is possible and even preferred.
The graft 40 is then inserted into a hollow robotic stapler/suturing device 44 with staple/suturing control cable 131 by squeezing the vessel gently in between an opening slit 45 in the stapler/suturing device 44 (Figure 8). Once the graft is inside the robotic stapler/suturing device 44, the expanded ring 41 is now retracted over the stapler device 44 while the latter is gently advanced until the ring 41 passes a ridge 47. With a suture 49 proximal to the ridge 47, the end 39 of graft 40 is fixed to the stapler/suturing device 44 and the tension of the suture closes slit 45. The slit is designed to be opened after establishing the anastomosis bonding as exit for the bypass graft. Next, the running suture 42 around ring 41 is removed and the ring is withdrawn. As a result, the everted graft wall 46 now fits the outside of the sta¬ pler/suturing device 44, as illustrated in Figure 9 which shows a cross section of the stapler device 44 and the everted graft 46. Staples 48 are present in the end part of the hollow cylindrical stapler device 44 to be ejected from its end part, as will be explained below. The robotic stapler/suturing device may also and preferably end in a bevelled, ellipsoid configuration (not shown) to allow creating an anastomosis corresponding more to current CABG practice, which avoids kinking of the graft and which maximizes the area of the anastomosis opening.
A high frequency (20-60 MHz) ultrasound analysis of the sur¬ face of the heart at the designated target area 22 to find the optimal anastomosis site on the coronary artery to be operated is performed as follows (Figure 10 A-E and Figure 11). High frequency (30 MHz) ultrasound analysis of the surface of the heart and the underlying coronary artery is performed with methods similar to intravascular ultrasound imaging described in e.g. Borst C, Savalle LH, Smits PC, Post MJ, Gussenhoven WJ, Bom N., "Imaging of post¬ mortem coronary arteries by 30 MHz intravascular ultrasound", Int J Cardiac Imag 1991;6:239-46.
An ultrasound diagnostic device 150 is made of light weight materials. It consists of a rectangular block less than about 25 x
12 x 5 mm) at the end of a flexible tube 151, which fits through a
15 mm I.D. trocart (not shown). In the middle, an opening 152 can accommodate the stapler/suturing device as will be explained below
(Figure 10 A, B and E) . There are various sizes holes in different ultrasound devices to accommodate grafts on different diameter coronary arteries. At e.g. eight positions, suction openings 153 connected through suction channel 154 to the lumen 155 of tube 151 allows temporary attachment by subatmospheric pressure of the ultrasound device 150 to the moving surface of the heart, on top of coronary artery 23 underneath.
The ultrasound device (Figure 10) contains proximally one ultrasound unit consisting of an electromotor 156 (diameter less than 3-4 mm, length less than about 8 mm) which is powered by the electrical cable 157. The electromotor 156 runs at about 4000 RPM, but through 4:1 gears 158, the speed is brought down to 1000 RPM. In this way a plane circular 30 MHz (20-60 MHz) ultrasound trans¬ ducer 159 (diameter 1 mm) is rotated at 1000 RPM, the rotation speed of commercially available intravascular ultrasound devices. The ultrasound transducer is excited by high frequency pulses con- ducted by electrical cable 160 through sliding contacts 161. The ultrasound system 162 is basically the same as the commercially available intravascular ultrasound system employed in Borst et al (1991) mentioned earlier. An electromotor driven rotary ultrasound element has been described by N. Bom and C.T. Lancee in a European Patent Application 423,895 with a miniature motor (1 mm diameter, 3 mm long) to be incorporated in a coronary catheter.
The ultrasound device (Figure 10)' contains a second ultra¬ sound unit distally at the end which consists identically of an electromotor 166, electric supply cable 167, 4:1 gears 168, an ultrasound transducer 169, ultrasound cable 170, and sliding con¬ tacts 171. The 4:1 gears 158 also allow positioning the ultrasound transducer 159, 169 closer to the ultrasound transparent TPX window 163 (Figure 10B).
Preferably, the ultrasound transducer 169, 179 plane makes an angle of 15° with respect to the rotation shaft to avoid too large a reflection from the TPX window 163 through which the ultrasound beam is transmitted to the superficial layers of the heart 20 (Fig- ure 10 D) . The chamber 172 is filled with distilled water.
The ultrasound device 150 is introduced in the chest cavity through an appropriate opening between two neighbouring ribs and handed over to a suitable surgical instrument 34 or 35 on one robotic tracking arm 32 or 33. The tracking control 37 is switched on and the ultrasound device 150 is lowered onto the prospective anastomosis site (the site has been determined previously on the basis of the coronary angiogram) . Once the ultrasound device 150 touches the surface of the heart, the eight suction channels 153 are activated to help securing perfect tracking of the target area 23 by the ultrasound device 150 (Figure 10C). Once the ultrasound device 150 is properly attached to the surface of the beating heart, the proximal 159 and distal 169 ultrasound transducers will display cross sections of the recipient coronary artery 23 and its surroundings on both sides of the target anastomosis site 22. Both ultrasound transducers are connected to the ultrasound generating and analysing system 162. The ultrasound pulse frequency of 1000 Hz drives the transducers 159 and 169 each at 500 Hz by sending (and receiving), for example, the odd pulses to transducer 159 and the even pulses to transducer 169. The corre- spending ultrasound cross-sections 181, 182, respectively, are displayed on videomonitor 180. If the ultrasound device 150 appears to be improperly positioned (not on top of the artery or if the place 22 shows too much plaque in the recipient artery), its posi¬ tion is altered by stopping the suction and repositioning. The ultrasound device 150 allows optimizing the distal anastomosis site, because the ultrasound cross-sections will reveal the actual plaque load 29 of the artery 23 (Figure 10D). Angiography, in con¬ trast, only provides a projection of the lumen.
Once the ultrasound device 150 is attached to the surface of the heart over the coronary artery 23, it is left attached to the surface of the heart by means of the eight suction channels 153. Now the stapler/suturing device 44 having graft 40 retracted around its end part, as shown in Figure 9, is introduced into the chest cavity, again through an appropriate opening between two neighbour¬ ing ribs, handed to a surgical instrument 34 or 35 attached to a robotic tracking arm 32 or 33, lowered to the ultrasound device and positioned in the centre hole 152 of the ultrasound device. The ultrasound diagnostic device 150 (Figure 10) and the robotic sta- pier/suturing device 44 (Figures 8 and 9) combined form the ultra¬ sound guided anastomosis device, USGAD 190 (Figure 11). The ultra¬ sound cross-section of the artery 23 (during light pressure by the catheter) allows determination of the thickness 29 of the arterial wall of the artery 23 (Figure 10 D).
The proper stapler/suturing device 44 has been chosen as to its diameter and its stapling depth. Note that the device 44 has been attached to the end of the graft outside the chest. This is also possible with the internal mammary artery (arterial bypass graft) after mobilization, because this artery is long enough to bring the distal end outside the chest. Besides, staples 48 have been inserted into the end part of the stapler device 44 outside the chest. After inserting the staples 48, staple ejecting means 130 to be operated by press gas through the hollow stapler device 44 are laid upon the staples 48, controlled from outside the chest by stapler/suturing control cable 131.
The stapler/suturing device 44 is advanced until the ridge 47 is blocked by the ultrasound device 150. Preferably, by manual control the staples 48 are ejected sequentially at end-diastole (EKG triggered) one at the time by supplying press gas through a supply hose 131. Ejecting one staple simultaneously prepares for ejecting the adjacent staple. The pressing gas . is designated by arrow Q in Figure 11. The staples 48 should eject sufficiently fast in order that the coronary artery wall 23 does not move. Figure 12A shows the end result (the staples 48 are drawn schematically). In Figure 12B the staples 48 are viewed from above.
In order to obtain a secure leakfree contact between the graft 40 and the artery 23, preferably eight staples 48 are posi- tioned according to fig. 12B. The staple 48 is about 100 micron thick and has a pre-bent form which causes it to behave like a paper staple (Figure 12C). Alternatively, staple 48 may have a pre- bent form which causes it to turn back on itself, as shown in Fig¬ ure 12D. The staple 48 should pass through a good deal of recipient wall, i.e. the wall of artery 23, but preferably not enter the recipient lumen. The material for staples 48 may be titanium as is being used for stents. The staples 48 are non ferromagnetic. Instead of staples, other bonding means known by persons skilled in the art may be used.
Once the bypass graft 44 has been anastomosed to the coronary artery 23, the stapler/suturing device 44 is lifted from the ultra¬ sound device, suture 42 is cut and slit 45 is widened sufficiently to exit graft 44 through slit 45. Finally, suction on openings 153 is released and both the stapling/suturing device and the ultra- sound device are brought outside the chest.
The ultrasound guidance provided by the USGAD 190 will help to avoid highly calcified anastomosis sites, because calcific deposits might hamper the staple's course.
Below, punching a hole in the wall of a coronary artery (dis¬ tal anastomosis) will be described, referring to figures 13 and 14.
The current Tulleken method to punch a hole in the coronary artery wall is described in:
- Tulleken CAF, Verdaasdonk RM, Berendsen W, Mali WPTM: Use of the excimer laser in high flow bypass surgery of the brain. J Neurosurg 1993;78:477-80.
In the technique according to the prior art for ablating the recipient arterial wall 61 (Figure 12A) to establish continuity between the graft (donor) lumen 63 and the recipient coronary artery lumen 62 use is made of an excimer laser (308 run) and a densily packed multifiber catheter which ablates the tissue of the recipient wall 61. To avoid creating large water vapor bubbles (3 mm diameter within 100 microseconds, lifetime 200 microseconds), a multifiber catheter may be used which has only a single row of fibers at the perimeter of the catheter (not shown) . By suction through its lumen a cut out disk of tissue is retrieved.
Alternatively, a circular cutter 73 (Fig. 13) or electrical spark erosion (Fig. 14) may be employed because these methods are cheeper. The latter can be made in a bevelled ellipsoid configur- ation (not shown) if the graft and recipient artery meet under an angle much less than 90°. The circular cutter 73 shown in Figure 13 is fabricated like a Simpson atherectomy cutter, as known from: US patent 4,781,186 by Simpson JB et al. and from e.g. Johnson DE, Hinohara T, Selmon MR, Braden LJ, Simpson JB. Primary peripheral arterial stenoses and restenoses excised by transluminal atherectomy: a histopathologic study. J Am Coll Cardiol 1990;15:419-25. The circular knife 73 in various diameters (1.5-3.5 mm) is driven by a straight shaft 72 which is connected to a flexible cable 70, preferably, made from strands of wire which are for 50% counter wound to have maximum flexibility. In a preferred embodiment, this cable 70 is connected to a small battery powered, hand held motor (not shown) which is switched on/off with a simple switch.
The cutter housing 71 is advanced through a thin walled, flexible, atraumatic catheter 67 which has a carefully polished, rounded end. The thin walled catheter 67 has been advanced first to minimize injury to the graft vessel wall by the much stiffer circu¬ lar cutter.
A light suction 69 in the thin walled catheter 67 is switched on to grab the recipient wall 61. The circular cutter 73 is rotated at about 1000 RPM and advanced like the Simpson cutter, but its progress is limited (adjustable) to slightly over the thickness of the recipient wall as previously determined by the USGAD 190. When the circular cutter 73 is retracted, the cut tissue should remain in the cutter 73 due to the suction (the cutter housing 71 has a few holes, not shown, at its proximal end) .
When electrical spark erosion (Fig. 14) is used, a circular ring or ellipsoid monopolar electrode 80 isolated laterally by an isolator 81 and connected to a catheter lead 82, which is connected to spark erosion ablation means 83, is applied. The spark erosion method may be accomplished according to the method originally described by Slager CJ, Essed CE, Schuurbiers JHC et al., in "Vaporization of atherosclerotic plaques by spark erosion", J Am Coll Cardiol 1985;5:1382-6, and by Slager CJ, Bom N, Serruys PW, Schuurbiers JCH, Vandenbroucke WVA, Lancee CT, in "Spark erosion and its combination with sensing devices for ablation of vascular lesions". In: Vogel JHK, King SB, "Interventional Cardiology: Future Directions". St. Louis: The C.V. Mosby Company, 1989, pp. 157-169. Again, suction in the thin walled catheter 67 is employed to keep the punched out disk from embolizing into the lumen 62 of the recipient artery 23.
As an alternative to the methods described in Figures 13 and 14, in the aortic wall 135 (proximal anastomosis) one can simply punch a hole, as will be described referring to Figure 15.
A catheter aortic punch has a sharp screw 84 on top of a retraction circular knife 85 which enables screwing the screw 84 through the aortic wall 135. By retraction of the central shaft 86 of the distal circular knife 85 against the advance movement of a motor driven proximal circular knife 83 with a fractionally larger diameter than the diameter of retraction circular knife 85, a disk of for example 3.5 mm is punched out of the aortic wall 135 and retrieved.
Again, the thin walled flexible catheter 67 is inserted first, through the temporary side branch 132 (Figure 11) of the graft 40 to protect the graft wall from mechanical trauma by the stiff cutting device (see also Figure 13).
The robotic surgical system described above can also be used for open chest CABG surgery on the beating heart. In that case, the constraints on seize and positioning of the instruments do not hold anymore (everything has to pass through 15 mm I.D. trocarts). Fur¬ thermore, the minimally invasive robotic surgical system can also be used for other operations, e.g. to remove thoracoscopically a myocardial bridge over a coronary artery. Subepicardially located accessory atrioventricular pathways may similarly be mapped with robotic motion compensated electrodes and interrupted surgically by this minimally invasive system. In general, virtual object arrest of a moving object within the video image frame can be employed as a 2D or 3D diagnostic system for close inspection of e.g. a moving part of a machine when the machine is preferably not to be stopped. In conjunction with the diagnostic system, the robotic tracking with repair instruments will allow robotic manipulation and repair of the moving part of the machine.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5054045 *||14 Nov 1990||1 Oct 1991||Cedars-Sinai Medical Center||Coronary tracking display|
|US5279309 *||27 Jul 1992||18 Jan 1994||International Business Machines Corporation||Signaling device and method for monitoring positions in a surgical operation|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|WO1998016164A1 *||15 Oct 1997||23 Apr 1998||Mayer Paul W||Relative motion canceling platform for surgery|
|WO1999050721A1||18 Sep 1998||7 Oct 1999||Massachusetts Institute Of Technology||Robotic apparatus|
|WO2000015119A2||3 Sep 1999||23 Mar 2000||Medtronic, Inc.||Method and apparatus for temporarily immobilizing a local area of tissue|
|WO2000033723A2||18 Nov 1999||15 Jun 2000||Intuitive Surgical, Inc.||Performing cardiac surgery without cardioplegia|
|WO2004002351A2 *||26 Jun 2003||8 Jan 2004||Georges Bogaerts||Guiding member for surgical instruments|
|WO2004002351A3 *||26 Jun 2003||8 Jul 2004||Georges Bogaerts||Guiding member for surgical instruments|
|WO2004029786A1 *||25 Sep 2003||8 Apr 2004||Imperial College Innovations Limited||Control of robotic manipulation|
|WO2007136769A3 *||18 May 2007||21 Feb 2008||Hyosig Kang||Method and apparatus for controlling a haptic device|
|WO2008086434A2 *||9 Jan 2008||17 Jul 2008||Cyberheart, Inc.||Depositing radiation in heart muscle under ultrasound guidance|
|WO2008086434A3 *||9 Jan 2008||13 Nov 2008||Cyberheart Inc||Depositing radiation in heart muscle under ultrasound guidance|
|WO2008086493A2 *||10 Jan 2008||17 Jul 2008||Hansen Medical, Inc.||Robotic catheter system|
|WO2008086493A3 *||10 Jan 2008||4 Sep 2008||Hansen Medical Inc||Robotic catheter system|
|DE10354642A1 *||22 Nov 2003||16 Jun 2005||Bayerische Motoren Werke Ag||Vorrichtung und Verfahren zum Programmieren eines Industrieroboters|
|EP1015944A1 *||18 Sep 1998||5 Jul 2000||Massachusetts Institute Of Technology||Robotic apparatus|
|EP1015944A4 *||18 Sep 1998||31 Dec 2008||Massachusetts Inst Technology||Robotic apparatus|
|EP1125557A2 *||16 Feb 1996||22 Aug 2001||Hitachi, Ltd.||Remote surgery support system|
|EP1125557A3 *||16 Feb 1996||2 Jan 2004||Hitachi, Ltd.||Remote surgery support system|
|EP1150601A2 *||18 Nov 1999||7 Nov 2001||Intuitive Surgical, Inc.||Performing cardiac surgery without cardioplegia|
|EP1150601A4 *||18 Nov 1999||15 Aug 2007||Intuitive Surgical Inc||Performing cardiac surgery without cardioplegia|
|EP2106764A3 *||18 Nov 1999||23 Dec 2009||Intuitive Surgical, Inc.||System for performing cardiac surgery without cardioplegia|
|EP2362283A3 *||18 Sep 1998||5 Mar 2014||Massachusetts Institute Of Technology||Robotic apparatus|
|EP2362284A3 *||18 Sep 1998||22 May 2013||Massachusetts Institute Of Technology||Robotic apparatus|
|EP2866722A4 *||1 Jul 2013||2 Nov 2016||Childrens Nat Medical Ct||Automated surgical and interventional procedures|
|EP3042625A1 *||18 Nov 1999||13 Jul 2016||Intuitive Surgical Operations, Inc.||Cooperative minimally invasive telesurgical system|
|US5749892 *||24 Jul 1996||12 May 1998||Heartport, Inc.||Device for isolating a surgical site|
|US5799661 *||7 Jun 1995||1 Sep 1998||Heartport, Inc.||Devices and methods for port-access multivessel coronary artery bypass surgery|
|US5807243 *||20 Feb 1996||15 Sep 1998||Heartport, Inc.||Method for isolating a surgical site|
|US5836311 *||20 Sep 1995||17 Nov 1998||Medtronic, Inc.||Method and apparatus for temporarily immobilizing a local area of tissue|
|US5855210 *||6 Nov 1996||5 Jan 1999||Heartport, Inc.||Methods for performing heart surgery|
|US5871017 *||15 Oct 1997||16 Feb 1999||Mayer; Paul W.||Relative motion cancelling platform for surgery|
|US5927284 *||21 Aug 1997||27 Jul 1999||Medtronic, Inc||Method and apparatus for temporarily immobilizing a local area of tissue|
|US5961481 *||12 Mar 1996||5 Oct 1999||Heartport, Inc.||Systems for coronary bypass procedures|
|US6015378 *||3 Oct 1996||18 Jan 2000||Medtronic, Inc.||Method and apparatus for temporarily immobilizing a local area tissue|
|US6017304 *||20 Apr 1998||25 Jan 2000||Vierra; Mark A.||Device and method for isolating a surgical site|
|US6027476 *||19 Sep 1997||22 Feb 2000||Heartport, Inc.||Methods and systems for performing thoracoscopic coronary bypass and other procedures|
|US6071295 *||20 Nov 1997||6 Jun 2000||Medivas Opcab, Inc.||Device to hold an anastomotic site of coronary artery motionless and bloodless for the bypass operation|
|US6106511 *||8 Sep 1998||22 Aug 2000||Sri International||Methods and devices for positioning a surgical instrument at a surgical site|
|US6139492 *||8 Apr 1999||31 Oct 2000||Heartport, Inc.||Device and method for isolating a surgical site|
|US6149583 *||6 May 1999||21 Nov 2000||Heartport, Inc.||Device and method for isolating a surgical site|
|US6201984||26 Jan 1995||13 Mar 2001||International Business Machines Corporation||System and method for augmentation of endoscopic surgery|
|US6231585||9 Jun 1999||15 May 2001||Medivas, Llc||Device for stabilizing a treatment site and method of use|
|US6246200||3 Aug 1999||12 Jun 2001||Intuitive Surgical, Inc.||Manipulator positioning linkage for robotic surgery|
|US6259806||15 Oct 1998||10 Jul 2001||Sri International||Method and apparatus for transforming coordinate systems in a telemanipulation system|
|US6311693||4 Oct 1999||6 Nov 2001||Wesley D. Sterman||Method and systems for performing thoracoscopic cardiac bypass and other procedures|
|US6332468||7 Sep 1999||25 Dec 2001||Cardiothoracic Systems, Inc.||Method for coronary artery bypass|
|US6338710||16 Apr 1999||15 Jan 2002||Medivas, Llc||Device for stabilizing a treatment site and method of use|
|US6346077||27 Jan 1997||12 Feb 2002||Cardiothoracic Systems, Inc.||Surgical instrument for stabilizing the beating heart during coronary artery bypass graft surgery|
|US6381499||17 May 2000||30 Apr 2002||Cardiothoracic Systems, Inc.||Method and apparatus for using vagus nerve stimulation in surgery|
|US6406472||9 May 2000||18 Jun 2002||Sri International, Inc.||Remote center positioner|
|US6424885||13 Aug 1999||23 Jul 2002||Intuitive Surgical, Inc.||Camera referenced control in a minimally invasive surgical apparatus|
|US6468265||9 Nov 1999||22 Oct 2002||Intuitive Surgical, Inc.||Performing cardiac surgery without cardioplegia|
|US6478029||5 Aug 2000||12 Nov 2002||Hearport, Inc.||Devices and methods for port-access multivessel coronary artery bypass surgery|
|US6482151||3 Oct 2000||19 Nov 2002||Heartport, Inc.||Method of performing a procedure on a coronary artery|
|US6494211||19 Jan 2000||17 Dec 2002||Hearport, Inc.||Device and methods for port-access multivessel coronary artery bypass surgery|
|US6574355||21 Mar 2001||3 Jun 2003||Intuitive Surigical, Inc.||Method and apparatus for transforming coordinate systems in a telemanipulation system|
|US6592573||27 Jun 2001||15 Jul 2003||Popcab, Llc||Through-port heart stabilization system|
|US6594552||6 Apr 2000||15 Jul 2003||Intuitive Surgical, Inc.||Grip strength with tactile feedback for robotic surgery|
|US6659939||3 Nov 1999||9 Dec 2003||Intuitive Surgical, Inc.||Cooperative minimally invasive telesurgical system|
|US6684129||11 Apr 2002||27 Jan 2004||Intuitive Surgical, Inc.||Master having redundant degrees of freedom|
|US6714839||17 Sep 1999||30 Mar 2004||Intuitive Surgical, Inc.||Master having redundant degrees of freedom|
|US6758843||26 Apr 2002||6 Jul 2004||Sri International, Inc.||Remote center positioner|
|US6764445||14 Dec 2001||20 Jul 2004||Intuitive Surgical, Inc.||Stabilizer for robotic beating-heart surgery|
|US6785593||24 Apr 2003||31 Aug 2004||Computer Motion, Inc.||Modularity system for computer assisted surgery|
|US6786896||18 Sep 1998||7 Sep 2004||Massachusetts Institute Of Technology||Robotic apparatus|
|US6788018||20 Dec 2001||7 Sep 2004||Intuitive Surgical, Inc.||Ceiling and floor mounted surgical robot set-up arms|
|US6799088||24 Apr 2003||28 Sep 2004||Computer Motion, Inc.||Modularity system for computer assisted surgery|
|US6821247||27 Sep 2002||23 Nov 2004||Heartport, Inc.||Device and method for isolating a surgical site|
|US6837883||5 Oct 2001||4 Jan 2005||Intuitive Surgical, Inc.||Arm cart for telerobotic surgical system|
|US6860877||29 Sep 2000||1 Mar 2005||Computer Motion, Inc.||Heart stabilizer support arm|
|US6879880||13 May 2003||12 Apr 2005||Intuitive Surgical, Inc.||Grip strength with tactile feedback for robotic surgery|
|US6933695||21 Apr 2004||23 Aug 2005||Intuitive Surgical||Ceiling and floor mounted surgical robot set-up arms|
|US6999852||26 Oct 2004||14 Feb 2006||Sri International||Flexible robotic surgery system and method|
|US7006895||26 Oct 2004||28 Feb 2006||Sri International||Computed pivotal center surgical robotic system and method|
|US7025064||6 Nov 2002||11 Apr 2006||Intuitive Surgical Inc||Method and apparatus for performing minimally invasive cardiac procedures|
|US7025722||30 Apr 2004||11 Apr 2006||Heartport, Inc.||Device and method for isolating a surgical site|
|US7027892||4 Dec 2002||11 Apr 2006||Intuitive Surgical||Method and apparatus for performing minimally invasive cardiac procedures|
|US7028692||19 Sep 2001||18 Apr 2006||Heartport, Inc.||Methods and systems for performing thoracoscopic coronary bypass and other procedures|
|US7131447||26 Apr 2004||7 Nov 2006||Heartport, Inc.||Methods and systems for performing thoracoscopic coronary bypass and other procedures|
|US7250028||28 Nov 2001||31 Jul 2007||Intuitive Surgical Inc||Endoscopic beating-heart stabilizer and vessel occlusion fastener|
|US7333642||15 Apr 2003||19 Feb 2008||Sri International, Inc.||Method and apparatus for transforming coordinate systems in a telemanipulation system|
|US7373219||7 Mar 2005||13 May 2008||Intuitive Surgical, Inc.||Grip strength with tactile feedback for robotic surgery|
|US7390325||17 Feb 2006||24 Jun 2008||Intuitive Surgical, Inc.||Apparatus for performing minimally invasive cardiac procedures with a robotic arm that has a passive joint and system which can decouple the robotic arm from the input device|
|US7395249 *||16 Sep 2004||1 Jul 2008||Intuitive Surgical, Inc.||Speech interface for an automated endoscope system|
|US7403835||19 May 2006||22 Jul 2008||Bayerische Motoren Werke Aktiengesellschaft||Device and method for programming an industrial robot|
|US7594912||30 Sep 2004||29 Sep 2009||Intuitive Surgical, Inc.||Offset remote center manipulator for robotic surgery|
|US7769427 *||15 Jul 2003||3 Aug 2010||Magnetics, Inc.||Apparatus and method for catheter guidance control and imaging|
|US7778733||18 Jan 2008||17 Aug 2010||Intuitive Surgical Operations, Inc.||Grip strength with tactile feedback for robotic surgery|
|US7785320||20 Mar 2008||31 Aug 2010||Intuitive Surgical Operations, Inc.||Method for operating a medical robotic system by stopping movement of a surgical instrument about a pivot point or issuing a warning if the pivot point moves beyond a threshold value|
|US7865266||24 Apr 2003||4 Jan 2011||Intuitive Surgical Operations, Inc.||Cooperative minimally invasive telesurgical system|
|US7869854||23 Feb 2006||11 Jan 2011||Magnetecs, Inc.||Apparatus for magnetically deployable catheter with MOSFET sensor and method for mapping and ablation|
|US7873401||13 Jan 2006||18 Jan 2011||Magnetecs, Inc.||System and method for a magnetic catheter tip|
|US7873402||9 Oct 2007||18 Jan 2011||Magnetecs, Inc.||System and method for radar-assisted catheter guidance and control|
|US7920909||25 Apr 2006||5 Apr 2011||Veran Medical Technologies, Inc.||Apparatus and method for automatic image guided accuracy verification|
|US8027714||27 May 2005||27 Sep 2011||Magnetecs, Inc.||Apparatus and method for shaped magnetic field control for catheter, guidance, control, and imaging|
|US8062288||8 Jul 2009||22 Nov 2011||Intuitive Surgical Operations, Inc.||Offset remote center manipulator for robotic surgery|
|US8068649||10 Jan 2008||29 Nov 2011||Sri International, Inc.||Method and apparatus for transforming coordinate systems in a telemanipulation system|
|US8105235||8 Jun 2004||31 Jan 2012||Intuitive Surgical Operations, Inc.||Stabilizer for robotic beating-heart surgery|
|US8108069||10 Jan 2008||31 Jan 2012||Hansen Medical, Inc.||Robotic catheter system and methods|
|US8123740 *||15 Jul 2004||28 Feb 2012||Massachusetts Institute Of Technology||Robotic apparatus|
|US8150495||7 Nov 2008||3 Apr 2012||Veran Medical Technologies, Inc.||Bodily sealants and methods and apparatus for image-guided delivery of same|
|US8256319||3 Oct 2011||4 Sep 2012||Intuitive Surgical Operations, Inc.||Offset remote center manipulator for robotic surgery|
|US8345821||8 Oct 2010||1 Jan 2013||Cyberheart, Inc.||Radiation treatment planning and delivery for moving targets in the heart|
|US8457714||25 Nov 2008||4 Jun 2013||Magnetecs, Inc.||System and method for a catheter impedance seeking device|
|US8483801||8 Nov 2010||9 Jul 2013||Veran Medical Technologies, Inc.||Methods, apparatuses, and systems useful in conducting image guided interventions|
|US8526737||25 Oct 2011||3 Sep 2013||Sri International||Method and apparatus for transforming coordinate systems in a telemanipulation system|
|US8562594||31 Jul 2012||22 Oct 2013||Intuitive Surgical Operations, Inc.||Offset remote center manipulator for robotic surgery|
|US8600551||7 Apr 2010||3 Dec 2013||Intuitive Surgical Operations, Inc.||Medical robotic system with operatively couplable simulator unit for surgeon training|
|US8690908||3 Dec 2012||8 Apr 2014||Intuitive Surgical Operations, Inc.||Flexible wrist for surgical tool|
|US8784290||16 Jul 2010||22 Jul 2014||Cyberheart, Inc.||Heart treatment kit, system, and method for radiosurgically alleviating arrhythmia|
|US8840628||10 Jul 2013||23 Sep 2014||Intuitive Surgical Operations, Inc.||Surgical manipulator for a telerobotic system|
|US8870900||21 Mar 2007||28 Oct 2014||Intuitive Surgical Operations, Inc.||Endoscopic beating-heart stabilizer and vessel occlusion fastener|
|US8911428||7 Mar 2012||16 Dec 2014||Intuitive Surgical Operations, Inc.||Apparatus for pitch and yaw rotation|
|US8911499||23 Jun 2008||16 Dec 2014||Mako Surgical Corp.||Haptic guidance method|
|US8914150||24 Jan 2014||16 Dec 2014||Intuitive Surgical Operations, Inc.||Cooperative minimally invasive telesurgical system|
|US8944070||30 Mar 2005||3 Feb 2015||Intuitive Surgical Operations, Inc.||Non-force reflecting method for providing tool force information to a user of a telesurgical system|
|US9002426||23 Jun 2008||7 Apr 2015||Mako Surgical Corp.||Haptic guidance system and method|
|US9005112||29 Mar 2012||14 Apr 2015||Intuitive Surgical Operations, Inc.||Articulate and swapable endoscope for a surgical robot|
|US9022998||26 Feb 2010||5 May 2015||Maquet Cardiovascular Llc||Blower instrument, apparatus and methods of using|
|US9039681||16 Jul 2008||26 May 2015||Intuitive Surgical Operations, Inc.||Minimally invasive surgical training using robotics and telecollaboration|
|US9068628||8 May 2013||30 Jun 2015||Intuitive Surgical Operations, Inc.||Robotic arms with strap drive trains|
|US9085083||8 May 2014||21 Jul 2015||Intuitive Surgical Operations, Inc.||Tool grip calibration for robotic surgery|
|US9095317||2 Jul 2012||4 Aug 2015||Intuitive Surgical Operations, Inc.||Flexible wrist for surgical tool|
|US9101397||21 Aug 2013||11 Aug 2015||Intuitive Surgical Operations, Inc.||Real-time generation of three-dimensional ultrasound image using a two-dimensional ultrasound transducer in a robotic system|
|US9119654||13 Jan 2012||1 Sep 2015||Intuitive Surgical Operations, Inc.||Stabilizer for robotic beating-heart surgery|
|US9138129||11 Nov 2011||22 Sep 2015||Intuitive Surgical Operations, Inc.||Method and system for moving a plurality of articulated instruments in tandem back towards an entry guide|
|US9138165||22 Feb 2013||22 Sep 2015||Veran Medical Technologies, Inc.||Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation|
|US9149339 *||12 Jun 2014||6 Oct 2015||Titan Medical Inc.||Methods, systems and devices for three dimensional input and control methods and systems based thereon|
|US9218663||26 Feb 2011||22 Dec 2015||Veran Medical Technologies, Inc.||Apparatus and method for automatic image guided accuracy verification|
|US9218664||9 May 2011||22 Dec 2015||Veran Medical Technologies, Inc.||Apparatus and method for image guided accuracy verification|
|US9232984||10 Nov 2010||12 Jan 2016||Intuitive Surgical Operations, Inc.||Real-time generation of three-dimensional ultrasound image using a two-dimensional ultrasound transducer in a robotic system|
|US9261172||15 Dec 2006||16 Feb 2016||Intuitive Surgical Operations, Inc.||Multi-ply strap drive trains for surgical robotic arms|
|US9271798||13 Aug 2013||1 Mar 2016||Intuitive Surgical Operations, Inc.||Multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures|
|US9317651||11 Jan 2008||19 Apr 2016||Intuitive Surgical Operations, Inc.||Tool grip calibration for robotic surgery|
|US9320916||14 Sep 2012||26 Apr 2016||Cyberheart, Inc.||Heart treatment kit, system, and method for radiosurgically alleviating arrhythmia|
|US9333042||3 Dec 2013||10 May 2016||Intuitive Surgical Operations, Inc.||Medical robotic system with coupled control modes|
|US9345387||13 Jun 2007||24 May 2016||Intuitive Surgical Operations, Inc.||Preventing instrument/tissue collisions|
|US9402608||3 Feb 2014||2 Aug 2016||Maquet Cardiovascular Llc||Organ manipulator apparatus|
|US9402619||29 Sep 2014||2 Aug 2016||Intuitive Surgical Operation, Inc.||Rigidly-linked articulating wrist with decoupled motion transmission|
|US9469034||7 Mar 2013||18 Oct 2016||Intuitive Surgical Operations, Inc.||Method and system for switching modes of a robotic system|
|US9492927||9 Nov 2011||15 Nov 2016||Intuitive Surgical Operations, Inc.||Application of force feedback on an input device to urge its operator to command an articulated instrument to a preferred pose|
|US9498198||27 Apr 2010||22 Nov 2016||Maquet Cardiovascular, Llc||Surgical instruments for accessing and stabilizing a localized portion of a beating heart|
|US9498231||15 Mar 2013||22 Nov 2016||Board Of Regents Of The University Of Nebraska||On-board tool tracking system and methods of computer assisted surgery|
|US9516996||17 Sep 2014||13 Dec 2016||Intuitive Surgical Operations, Inc.||Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the position and orienting of its tip|
|US9572481||13 May 2011||21 Feb 2017||Intuitive Surgical Operations, Inc.||Medical system with multiple operating modes for steering a medical instrument through linked body passages|
|US9585641||6 Jul 2015||7 Mar 2017||Intuitive Surgical Operations, Inc.||Flexible wrist for surgical tool|
|US9622826||24 Nov 2014||18 Apr 2017||Intuitive Surgical Operations, Inc.||Medical robotic system providing sensory feedback indicating a difference between a commanded state and a preferred pose of an articulated instrument|
|US9623563||24 Jun 2015||18 Apr 2017||Intuitive Surgical Operations, Inc.||Tool grip calibration for robotic surgery|
|US9629520||24 Aug 2015||25 Apr 2017||Intuitive Surgical Operations, Inc.||Method and system for moving an articulated instrument back towards an entry guide while automatically reconfiguring the articulated instrument for retraction into the entry guide|
|US9636185||2 Jun 2016||2 May 2017||Mako Surgical Corp.||System and method for performing surgical procedure using drill guide and robotic device operable in multiple modes|
|US9636186||26 Jan 2016||2 May 2017||Intuitive Surgical Operations, Inc.||Multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures|
|US9655539||19 Apr 2012||23 May 2017||Magnetecs, Inc.||System and method for targeting catheter electrodes|
|US9662434||28 Feb 2011||30 May 2017||Maquet Cardiovascular Llc||Blower instrument, apparatus and methods of using|
|US9666101||26 Jan 2016||30 May 2017||Intuitive Surgical Operations, Inc.||Multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures|
|US9717486||10 Nov 2014||1 Aug 2017||Intuitive Surgical Operations, Inc.||Apparatus for pitch and yaw rotation|
|US9717563||24 Jun 2015||1 Aug 2017||Intuitive Surgical Operations, Inc.||Medical robotic system providing an auxilary view including range of motion limitations for articulatable instruments extending out of a distal end of an entry guide|
|US9718190||29 Jun 2006||1 Aug 2017||Intuitive Surgical Operations, Inc.||Tool position and identification indicator displayed in a boundary area of a computer display screen|
|US9724165||18 May 2007||8 Aug 2017||Mako Surgical Corp.||System and method for verifying calibration of a surgical device|
|US9730572||20 Mar 2015||15 Aug 2017||Intuitive Surgical Operations, Inc.||Articulate and swappable endoscope for a surgical robot|
|US20140316435 *||12 Jun 2014||23 Oct 2014||Titan Medical Inc.||Methods, systems and devices for three dimensional input and control methods and systems based thereon|
|International Classification||A61B17/02, A61B19/00, G06T7/20, A61B17/00, H04N13/00|
|Cooperative Classification||G06T7/246, A61B34/30, A61B2090/502, A61B34/77, A61B2017/00699, A61B2017/00694, A61B2017/0243, H04N13/0239, H04N2013/0092, G06T2207/10012, H04N13/0003, A61B2017/00703, A61B2017/00243, H04N2013/0085, A61B2017/00203|
|European Classification||H04N13/02A2, A61B19/22, G06T7/20C|
|19 Jan 1995||AK||Designated states|
Kind code of ref document: A1
Designated state(s): AM AT AU BB BG BR BY CA CH CN CZ DE DK ES FI GB GE HU JP KE KG KP KR KZ LK LT LU LV MD MG MN MW NL NO NZ PL PT RO RU SD SE SI SK TJ TT UA US UZ VN
|19 Jan 1995||AL||Designated countries for regional patents|
Kind code of ref document: A1
Designated state(s): KE MW SD AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG
|26 Apr 1995||121||Ep: the epo has been informed by wipo that ep was designated in this application|
|10 Jan 1996||122||Ep: pct application non-entry in european phase|
|25 Apr 1996||REG||Reference to national code|
Ref country code: DE
Ref legal event code: 8642
|7 May 1996||NENP||Non-entry into the national phase in:|
Ref country code: CA