IMPROVED COMPUTER-BASED MINIMAL- INVASIVE SURGERY
SIMULATION SYSTEM
TECHNICAL FIELD OF THE INVENTION
This invention relates to a computer-based minimal- invasive surgery simulation system.
BACKGROUND OF THE INVENTION
In modern surgery, minimal invasive techniques are used in more and more applications. The development of technology within this relatively new field advances quickly, which results in great training requirements for surgeons. One way of rendering the training more effective is to use computer simulations. Known techniques for providing a credible simulation are very complicated and expensive with respect to computer utility in the form of processor and memory. Moreover, the result is not sufficient to provide a realistic simulation environment. The visual properties that the anatomy exhibits in reality are difficult and time- consuming to recreate in a simulation.
Within the field of laparoscopy, a camera is used to supply picture information from the patient to the surgeon. The display screen shows the picture that the camera catches of the inside of, for example, the abdominal cavity. All the instruments and the anatomy with which the surgeon works are reproduced by means of the camera and the display screen. The surgeon uses the information on the display screen to control and operate his or her instruments and carry out the procedures which are required to perform the surgical operation. Since the minimal invasive techniques supply information to the surgeon by means of a display screen, the three- dimensional reality is reduced to two dimensions on the display screen. The picture therefore lacks, among other things, the information as to depth that exists in reality. The surgeon has to make up for this loss of
information by studying lighting conditions, colours, etc.
By means of modern computer engineering, it is possible to provide realistic training situations in a virtual environment created by a computer program. In the computer program, a three-dimensional model of the object which the simulation concerns is provided. The user is provided with a projection thereof which should correspond to the picture information which in a real situation is caught by a camera. This means that all visual information, such as instrument and anatomy, is drawn by the computer. However, there are still considerable differences between the picture information that the computer tries to recreate and the actual picture that a camera catches. A large part of the picture information that the surgeon uses in real life, for example light effects and anatomic structures, factors such as breathing and beat of the pulse, which are present in the real situation are difficult to recreate realistically in the computer.
Many complicated processes within the field of surgery are expensive, if not impossible, to fully simulate in a computer. This means that the simulation can only consist of short procedures which lack the continuity that is usually present when the process is carried out. For instance, in US 5,791,907, Ramshaw et al disclose a system for simulation of an entire operation. This solution is founded on a database of recorded video sequences which visualise different procedures in an operation. The actual simulation then consists of the user being introduced to a video sequence which shows a separate procedure. Subsequently, the user is asked a number of questions . Then a video sequence is played showing the continuation of the operation which is determined by the answers to the questions. However, even if the displayed images are fully realistic, this solution has its limitation regarding the quality of the training, since all possible situations cannot be
simulated and the user has a limited possibility of affecting the simulation result. Moreover, the users skill of using the laparoscopic tool is not improved in a satisfactory way, since only the knowledge of basic biologic surgery information is tested by system.
An improved surgical simulation system is described in the patent document WO-2002059859, providing a more realistic simulation environment. However, this document is mainly focused on providing a realistic anatomic environment for performing simulation. Hence, a more realistic training environment regarding the complete surgical simulation is desired.
SUMMARY OF THE INVENTION Hence, an object of the present invention is to provide a system which create a realistic simulation environment, and which wholly or partly solve the above- mentioned problems. Yet an object of this invention is to provide an improved simulation system in which a users skill regarding the handling of a laparoscopic surgical tool may be trained.
These and other objects are at least partly achieved by a computer-based minimal-invasive surgery simulation system, comprising: a virtual anatomic environment simulator, for simulating a virtual anatomic environment; a surgical tool simulator, for simulating a surgical tool within said virtual anatomic environment; a surgical tool input device, being arranged to be manipulated by a user of the system, whereby said manipulation results in a corresponding manipulation of the tool simulated by said surgical tool simulator; a surgical tool parameter input device, for user controlled input of parameters for controlling the simulated surgical tool; an interaction simulation device, for simulating the interaction between the virtual anatomic environment and the simulated surgical tool, whereby said interaction is adapted to the effects of said inputted parameters to the actions of the surgical tool; and a display device, for displaying the
simulated interaction between the simulated tool and the simulated virtual anatomic environment. Thereby, the surgical simulation may be improved regarding the handling of the surgical tool, since the surgical tool parameter input device provides for a realistic simulated surgical environment, enabling improved testing of the user skills of handling the surgical tool.
According to a first alternative of the invention, the surgical tool parameter input device comprises a detection device positioned on a handle of said surgical tool input device for detection of a gripping force applied on said handle by a user, whereby this information is transmitted as a parameter to the interaction simulation device. Thereby, the effects of the gripping force may be brought into the simulation, providing a more realistic simulation.
According to a second alternative of the invention, the surgical tool parameter input device comprises a user input device, for inputting information regarding what type of instrument that is to be simulated by said surgical tool simulator, whereby this information is transmitted as a parameter to the interaction simulation device. By type as used in this document should be included the form, technical function and size of a surgical tool. This enables improved testing of the user's skill regarding appropriate choice of instrument for a certain procedure.
According to a third alternative of the invention, the surgical tool parameter input device comprises a user input device, for inputting information regarding what type of energy is to be applied by said simulated surgical tool, whereby this information is transmitted as a parameter to the interaction simulation device. This also enables improved testing of a user's skill in choosing an appropriate type of tool for a specific surgical procedure. Preferably, said energy is set to be one of thermal energy (generated by means of an electrical current or the like) , ultrasonic energy, or
radiation energy, such as laser energy, providing the possibility of choosing the most common types used in surgical procedures. Moreover, the user input device further suitably comprises means for setting a desired energy level to be applied by said simulated surgical tool, whereby this information is transmitted as a parameter to the interaction simulation device. This enables testing of a user's ability to use an appropriate energy level for a certain procedure, and also to display for the user what happens if he chooses a less suitable alternative .
According to a fourth alternative of the invention, the user input device further comprises means for manually setting a desired sharpness of the simulated surgical tool, being yet a factor that effects the result of the surgical procedure .
According to a fifth alternative, the tool parameter input device further comprises a device for detecting a simulated applied pressure by a simulated surgical tool on a tissue of the virtual anatomic environment, whereby this information is transmitted as a parameter to the interaction simulation device. This enables testing of the user's ability to apply an appropriate pressure to a tissue by using a 2 -dimensional picture, and also displaying the result of the applied pressure visually.
According to a sixth alternative, the tool parameter input device further comprises a device for measuring a time during which the simulated surgical tool is applied to a tissue of the virtual anatomic environment, whereby this information is transmitted as a parameter to the interaction simulation device. This enables realistic simulation of the effects of tool application for a certain time, which is much valuable for example, when a tissue is to be cut by laser, when an excess application time may result in damage of underlying damage, and a too short application time may result in insufficient cutting.
Suitably the computer-based minimal-invasive surgery simulation device further comprises a surgical tool parameter consequence library, for providing the interaction simulator with consequence information for a variety of values or states for each parameter. This enables realistic information regarding the impact of the parameter value on the total simulation. Moreover, said library suitable includes a video library for providing visual consequence information for a variety of values or states for each parameter, to be displayed on said display device. This enables realistic visualisation of the effects of the parameter values in question, to be viewed and analysed by the user of the system.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will hereinafter be described in closer detail by means of presently preferred embodiments thereof, with reference to the accompanying drawings.
Fig 1 is a schematic view of the basic components of a simulation system in which the present invention may be implemented.
Fig 2 is a block diagram of a simulation system according to an embodiment of the invention.
Fig 3 is a schematic view of a computer interface of an embodiment of a surgical tool input device according to the invention.
DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION
Hereinafter, reference will be made in detail to a preferred embodiment of this invention, which is illustrated in the accompanying drawings. Generally, we will refer to the person using the inventive system as a "user" but it shall be understood that the term "user" includes everything from a student, about to learn the basic skills for this surgical method, to an experienced surgeon, preparing for a specific surgical procedure.
As stated above this invention relates to the field of computer-based minimal-invasive surgery simulation
systems, and a schematic drawing of such a system is disclosed in fig 1. Such a system essentially comprises a processing unit 12 for simulating a minimal-invasive surgery situation, a surgical tool input device 4, for inputting information regarding movements etc of a surgical tool to be manoeuvred by a user to the processing unit 12, and a display device 7 for displaying the simulated surgery procedure in order to visualise the simulated surgery situation for the user. The processing unit may, for example, comprise a personal computer which is preferably equipped with a graphics card having 3D- circuits . Moreover, the surgical tool input device 4 may or may not be provided with a system for tactile feedback to the user of the system. A first basic embodiment of this invention will hereinafter be described with reference to fig 2. According to this embodiment, the system for computer- based minimal-invasive surgery simulation comprises a surgical tool input device 4, i.e. an instrument to be hold and manoeuvred by the user, and that is arranged to imitate a real surgical tool. The movements and the position of the surgical tool input device 4 is arranged to be registered by a position tracker device (not shown) and the registered information regarding the movements and position of the surgical tool input device 4 is arranged to be transmitted to the processing unit 12 through a first input terminal 13. Moreover, according to the invention, the system comprises a surgical tool parameter input device 5, for user-controlled input of one or more parameters relating to the action of the surgical tool . Information regarding the inputted parameters is arranged to be transmitted to the processing unit 12 through a second input terminal 14. Alternatively, the surgical tool parameter input device 5 may be arranged in connection with the surgical tool input device 4, whereby said information regarding the inputted parameters may be arranged to be transmitted to the processing unit by said first input terminal 13.
Further, the surgical tool parameter input device 5 may be subdivided so that information is transmitted to the processing unit 12 through both terminals 13, 14. According to the present embodiment of the invention, the processing unit mainly comprises a surgical tool simulator 3, for simulating the surgical tool to be manoeuvred by the user, an anatomic environment simulator 2, for simulating the anatomic environment in which simulated surgery is to be performed, and an interaction simulator 6, being arranged to simulate the interaction between the simulated tool and the simulated anatomic environment, in order to provide total simulation of a procedure performed by a user. The total simulation provided by the interaction simulator 6 is arranged to be outputted from the processing unit 12 by a first output terminal 15, and displayed on a screen of a display device 7, to be viewed by the user of said simulation system. The surgical tool simulator 3 is connected with the first and second input terminals 13, 14 of the processing unit 12, as indicated above being arranged to receive information from said surgical tool input device 4 and said surgical tool parameter input device 5.
Hereinafter, the inventive surgical tool parameter input device 5 will be closer described. According to the invention, the surgical tool parameter input device 5 comprises one or more parameter input components, so that the user may input parameters affecting the desired use and effect of the tool to be simulated. In the below, a plurality of different possible components of the surgical tool parameter input device 5 will be described. However, it shall be realised that the system according to the invention may comprise any one of the below components, or any combination of the below components, in order to generate a simulation system having a suitable level of difficulty for the intended user. For example, a system primarily designed for an inexperienced user may comprise fewer of the below components, in order
to facilitate complicity, while a system primarily designed for an experienced user may comprise many or all of the below components, in order to provide a simulation in which the handling of the surgical tool is simulated in a realistic way. Moreover, it is also possible to generate a system comprising a plurality or all of the below components, but whose functionality may be turned on or off, in order to provide a simulation environment having a suitable level of difficulty regarding the handling of the surgical tool, being customised for a current user. Furthermore, other components than the ones described below may also be included in the surgical tool parameter input device 5.
As a first example, the surgical tool parameter input device 5 may comprise a gripping force detection device (not shown), positioned on a handle of the surgical tool input device 4, so that the gripping force applied on said handle by a user may be detected. In real laparoscopic surgery, the gripping force of the surgical tool affects the surgical effect of the tool in a way that may be predicted and simulated. The applied gripping force may affect the surgery result in several ways. A too low gripping force results in sliding of the instrument on the tissue, making the procedure or specific task impossible to perform. This will also possibly damage the tissue, by causing bleeding or leakage. A too high gripping force will deform and possibly damage the tissue with the above-mentioned consequences. The gripping force may be detected discrete or continuous, i.e. either the magnitude of the force may be detected as being on a certain level on a grade scale, for example a three grade scale (low, medium, high) , or the magnitude may be detected by measuring the actual value. The detected gripping force is thereafter transmitted to the surgical tool simulator, in which the effect of the certain gripping force (or the gripping force level) is stored in a memory of a surgical tool
parameter consequence library or the like, whereby the effect of the applied gripping force is taken into consideration when generating the simulated surgical tool in the surgical tool simulator 3. Moreover, the surgical tool parameter consequence library comprises information, such as video information or digital information for providing information regarding visual consequences for the measured parameter value, which information is transmitted to the display device for display. Hence, the consequence of a certain gripping force on the surgical procedure will be displayed on the display.
As a second example, the surgical tool parameter input device 5 may comprise a user input device (not shown) , by which a user actively may select settings to be used for the simulated surgical tool . Such a user input device may either be realised as selection controls on the handle of said surgical tool input device 4 or on a separate selection control unit, but may also be provided by means of a computer device or the like (not shown) providing computerised selection controls for the parameters in question. For example, the user input device may comprise input selection controls for one or more of the following parameters; type of tool to be simulated; type of energy to be applied by the tool to be simulated; level of energy to be applied by the tool to be simulated, sharpness of tool to be simulated and so on.
In the below example, the user input device will be described in the form of a computer device, but it shall be understood that the corresponding functionality may be achieved by a plurality of switches or the like, being arranged on the handle of the surgical tool input device 4 or alternatively in a separated unit. An example of a- computer interface for the user input device is disclosed in fig 3. In fig 3, three menus are disclosed. Selection within each of the menus may be performed by the user by means of a selection device, such as a computer mouse or
a key board. Alternatively, the display screen of the user input device may be provided as a touch screen.
A first menu 16 is used to select the type of tool that is to be used in the simulated procedure, i.e. what tool is to be simulated. Examples of tools that may be used m the simulation and hence may be chosen in said first menu are scissors, ultrasound scissors, forceps, diathermy blade diathermy hook, suction device, stapler machine, laser probe, ports or a needle driver (for illustrative purposes, only a few of the above instruments are displayed in fig 3) . This selection gives the user the possibility to actively select what tool is to be used for a special surgical procedure, hence testing the user's ability of choosing an appropriate tool for the procedure in question. The tool choice is thereafter transmitted to the surgical tool simulator, in which the effect of the certain tool is stored in a memory of a surgical tool parameter consequence library or the like, whereby the effect of the chosen tool is taken into consideration when generating the simulated surgical tool in the surgical tool simulator 3. Moreover, the surgical tool parameter consequence library may comprise information, such as video information or digital information for providing information regarding visual consequences for using the chosen tool, which information is transmitted to the display device for display.
A second menu 17 may be used to select what kind of energy is to be applied. Preferably, the available choices of this second menu is connected to the user choice made in the above-described first menu 16. If, for example, the user has chosen a blade tool in the first menu 16, the second menu 17 may be used what type of energy is to be applied by the blade tool. Examples of appropriate types of energies are; thermal energy; ultrasonic energy; or radiation energy, such as laser energy. It is known in the art of surgery, that application of different kinds of energy to tissue has
various different effects. By the above inventive possibility, this property is incorporated into the simulation. In the same way as above, the energy type choice is transmitted to the surgical tool simulator, in which the effect of the certain energy type on tissue is stored in a memory of a surgical tool parameter consequence library or the like, whereby the effect of the chosen energy type is taken into consideration when generating the simulated surgical tool in the surgical tool simulator 3. Moreover, the surgical tool parameter consequence library may comprise information, such as video information or digital information for providing information regarding visual consequences for using the chosen energy type, which information is transmitted to the display device for display. The effect of the above mentioned energy types will hereinafter be closer described. Thermal energy, which is generated by electricity, is used for haemostasis (stop bleeding) from small vessels and dissections of tissues. Ultrasonic energy is often used in a special type of scissors for coagulation of bigger blood vessels (having a diameter of < 5 mm) , i.e. stop bleeding there from and in the same movement cut the vessel. This is also used for dissection of tissue. The advantage is that one may use the same instrument for haemostasis and division of structures and tissues. Laser Energy is used for dissection and to some extent also haemostasis of small vessels. Visually, what happens with tissue as different types of energy is applied to it, is that, within a central application area, the tissue gets charred (it turns black) and becomes shrivelled and, in a more peripheral area surrounding the central area, the tissue o through a colour change and assumes a grey or white colour. Outside this peripheral area, the tissue is visually unchanged by the energy application. The size of the respective areas, as well as the time period during which a certain energy level may be applied before the above effect takes place, is different for different energy application tools,
applying different energies, and information regarding the visual result of every simulated tool in question is stored in the visual consequence library. Hence the visual result of each tool may be incorporated into the picture of the simulation displayed on the display device 7. Other features that may be incorporated with the present invention is that different types of tissue react differently to different kinds of energy, and hence different visual effects are achieved depending on tissue type. Moreover, the contact area between the tool and the tissue affects the result of energy application, and this aspect may also be included with the present invention.
Furthermore, a third menu 18 may be provided, which in the present case is used to select the level of energy that is to be applied by the chosen tool. The available choices of this third menu is connected to the user choice made in the above-described first menu 16 and second menu 17. According to an embodiment, the choices of this menu may be; low energy level; medium energy level; and high energy level. However, a continuous scale or a discrete scale comprising more or less steps may be used. Different levels of applied energy result in different effect when applied to tissue, and hence it is of great importance to learn what energy level is to be applied for a desired surgical result. The present invention provides a system for training this ability. In the same way as above, the chosen energy level to be applied is transmitted to the surgical tool simulator, in which the effect of the certain energy level is stored in a memory of a surgical tool parameter consequence library or the like, whereby the effect of the chosen energy level is taken into consideration when generating the simulated surgical tool in the surgical tool simulator 3. Also, the surgical tool parameter consequence library comprises information, such as video information or digital information for providing information regarding visual consequences for using the chosen energy level,
which information is transmitted to the display device for display.
By using the above three parameters, a more realistic simulation of the handling of surgical tools may be provided. As configured above, the inventive system tests the user's ability to chose the correct tool for a certain procedure, to chose the correct kind of energy for a certain desired result (such as cutting or coagulation) and to chose a correct level of energy to be applied (for example in order to cut through a tissue without harming underlying tissues) .
Other parameters may also be taken into account by said user input device, such as if a surgical knife to be used is to have a sharp or blunt mode.
As a third example, the surgical tool parameter input device 5 may comprise a device 10 for detecting a simulated applied pressure by a simulated surgical tool on a tissue of the virtual anatomic environment. This is suitably realised by feeding back a signal comprising information regarding contact and degree of contact between the simulated surgical tool and the tissue in question from the interaction simulation device, whereafter the simulated applied pressure is obtained. Thereafter, the simulated applied pressure is transmitted to the surgical tool simulator, in which the effect of the simulated applied pressure is stored in a memory of a surgical tool parameter consequence library or the like, whereby the effect of the applied pressure is transmitted to the interaction simulation device and taken into consideration when generating the total simulation. Also, the surgical tool parameter consequence library comprises information, such as video information or digital information for providing information regarding visual consequences for the applied pressure, which information is transmitted to the display device for display.
Finally, as a fourth example, the surgical tool parameter input device 5 may comprise a device 11 for measuring a time during which a simulated surgical tool is applied to a tissue in the virtual anatomic environment . This is suitably realised by feeding back a signal comprising information regarding the contact time between the simulated surgical tool and the tissue in question from the interaction simulation device. Thereafter, the contact time is transmitted to the surgical tool simulator, in which the effect of the contact time is stored in a memory of a surgical tool parameter consequence library or the like, whereby the effect of the contact time is transmitted to the interaction simulation device and taken into consideration when generating the total simulation. Also, the surgical tool parameter consequence library comprises information, such as video information or digital information for providing information regarding visual consequences for the contact time, which information is transmitted to the display device for display.
Regarding the above described third and fourth examples, the feed-back signal from the interaction simulation device may be feed directly to the surgical tool parameter consequence library, and in this case, the devices 10 and 11 may be said to form part of the tool parameter device 5 (not shown) .
The correspondence between a movement of the surgical input device 4 and the corresponding movement of the tool simulated by the surgical tool simulator 3 may be linear or non-linear. According to this invention, the correspondence between the surgical input device 4 and the tool simulated by the surgical tool simulator 3 may be varied, in order to further improve the realism of the simulation. In a normal state, the position and movement of the simulated instrument is linearly dependant upon the position and movement of the input device 4. Hence, one may say that the input device and the simulated
instrument are linearly synchronised, i.e. the motion of an instrument in the simulation is proportional to the motion of the input device. This state may be used for all interaction in the simulated environment. However, in a second state, for example in situations in which a part the simulated instrument is in virtual contact with an item of the simulated anatomic environment, such as another part of the simulated instrument or tissue, the dependency between the simulated instrument and the input device 4 may be varied according to set rules. Such a change may be used when two instrument are about to collide in the virtual environment, or if an instrument in the virtual environment is about to push or pull elastic tissue, or otherwise soft tissue. By changing the dependency, i.e. the proportionality factor between simulated instrument movement and the input device 4 , the system may make visual simulation of for instance tissue, that start out soft, and get harder when it is stretched, in that case changing the dependency gradually as the tissue is stretched. Thereby, the natural behaviour of for instance tissue may be simulated in an improved way. Hence, a movement of the surgical input device 4 a distance d will result in a larger movement of the tool simulated by the surgical tool simulator 3 if the tool is used to pull soft tissue, than if it is used to pull harder tissue (such as tissue already being stretched to a certain extent) and hence the user's ability to recognize and handle different structures in the simulated environment. Many modifications, improvements and variations of the above described embodiments are obvious for those skilled in the art, and such modifications, improvements and variations are intended to form a part of the disclosure, and are also intended to fall within the scope and spirit of this invention as defined by the appended claims. Hence, the above description of preferred embodiments is only given by way of example, and the invention is only to be limited to what is stated
in the following claims, taking any equivalents into account. For example, as indicated above, a system according to the invention may include any on of, or any combination of components as defined above. It shall also be realised that the effects to be displayed due to different tool actions are obvious for a man skilled in the art, but for example, bleeding may be displayed as a result of excess pressure of the tool, coagulating and cutting of tissue may be displayed as the result of energy application and so on.