WO2007096269A1 - Device and method for the real-time control of an effector - Google Patents

Abstract

The invention relates to a device for the real-time control of an effector, which device comprises a bleeder electrode for determining input signals from the neuronal activity of the brain of a patient and a data processing system having an evaluation unit for calculating control signals for the effector based on the input signals. The evaluation unit is adapted to the training for determining a prediction model and the calculation of the control signals from the input signals taking into consideration the prediction model. The neuronal activity is produced by natural neuronal information processing processes of the brain.

Description

 February 12, 2007 Albert-Ludwigs-Universität Freiburg U50875PC GS

Device and method for real-time control of an effector

The invention relates to a device and a method for real-time control of an effector.

Medical technology has in the past been of very limited help to patients with severe paralysis. Recent motor neuro-prosthetics are aimed at patients with paralysis where organic healing is no longer possible, although the cortex, or at least the motor cortex, is at least partially intact as the area essential for the control of voluntary movements Nerve connections to the musculature are interrupted - or the musculature / limbs are no longer present. The most important and very common cause of such paralyzes is ischemic brain infarction or intracerebral hemorrhage ("stroke").

A particularly serious case is the so-called locked-in patients who have been deprived of any voluntary movement due to complete paralysis of skeletal muscle (such as amyotrophic lateral sclerosis (ALS) or muscle soreness) or a stroke in the brainstem area. Such patients, with full awareness of their situation, are condemned to pure passivity. Similarly, one can also think of patients who have lost a limb or are paralyzed. Paralysis or disability prevents these patients from performing intended movements. Movements here and in the following should be understood to mean not only a movement in the sense of running, but any movement of muscles such as an arm movement, but also facial expressions or speech. Therefore, an attempt is made to restore or improve the ability to move by deliberately controlling a prosthesis by means of its own brain signals. The core of such a neuroprosthesis is a so-called Brain Machine Interface (BMI). Their function is to translate the neural activity of the patient into commands to the prosthesis.

A conventional BMI system is based on electrodes that are placed upside down and record an electroencephalogram (EEG). Then the patient is taught to intensify or mitigate certain aspects of recorded brain activity by volitional effort.

Here are three different systems known, each of which exploit certain neural activities:

A) Slow Cortical Potentials (SCPs)

SPCs are slow voltage changes over the entire cortex at intervals of one-half to ten seconds. Negative SCPs are typically associated with movement or other causes of cortical activation, while positive SCPs are associated with decreased cortical activity. A patient can learn to consciously raise and lower the SCPs in a training that typically spans several weeks and months. This information, which can also be discriminated from the outside via the EEG, can then be used for a simple cursor control after completion of the training phase The disadvantages are a long and strenuous training with subsequently only a very low transmission bandwidth: in the duration of an SPC in the In the order of several seconds, just one bit can be transmitted, and this type of communication is tiring and tedious for the patient.

B) P300 Rare or particularly significant stimuli interspersed with common stimuli, which may be auditory, visual, or somatosensory, induce an activity peak (typically in the EEG above the parietal cortex) with a time delay of about 300 ms versus the stimulus. If you show the patient a selection of letters, for example, and let them flash one after the other, the P300 is strongest for the desired letter. The process can be speeded up a bit by showing letters in rows and columns, with the letter resulting from the strongest column and the strongest row. Disadvantage of this method is that you can only select from predetermined possibilities, and that a whole series of presentations with high attention to the selection of only one letter is used.

C) Mu and Beta Rhythm: The 8-12 Hz Mu and 13-30 Hz beta activity is found more frequently over the somatosensory and motor cortex. Movement or exercise preparation is usually associated with a reduction of mu and beta rhythms, while after a movement or relaxation usually a rhythm enhancement occurs. Once again, after several weeks of training, subjects were shown to be able to control the strength of these rhythms, most recently independently for Mu and Beta, allowing two-dimensional cursor control. This is an improvement over the SPCs achieved, but this is only gradual and does not eliminate the disadvantages. In addition, movements can affect the mu rhythms and thus reduce the accuracy of the control. All these methods have in common that a voluntary achievement of the corresponding brain activities in training as application requires a high concentration and is very tiring for the patient. It should be noted that the patient often has in addition to his paralysis with serious other health problems to fight. From a healing in the sense of a casual dealing with other things using a prosthesis, which replaces the lost mobility without effort, so there can be no question. In addition, the detection of brain activity requires a relatively large amount of time and can only transmit little information (typically between 1-20 bits per minute). This creates a BMI with a very low information transmission bandwidth. Complex and even simple movements can not be controlled in this way. Everyday activities like writing a simple letter cost hours, if not days of extreme concentration.

It is also known to drive a prosthesis on the activation of still arbitrarily controllable muscles. The disadvantage of this is firstly that the patient can not express himself in the usual way, for example, he must get used to communicate by blinking eyes. On the other hand, such a solution has a limited field of use and obviously can not help a completely locked-in patient.

It is therefore an object of the invention to provide a possibility of communication and mobility in which an effector with a high information transmission rate is driven from its brain activity in a manner which is harmless and simple for the patient.

This object is achieved by a device according to claim 1 and a method according to claim 13 and 21, respectively. Advantageous developments of the invention düng are defined in the dependent claims.

The inventive solution is initially based on the knowledge that conventional BMI systems use any or arbitrarily trained signal that has nothing to do with the original biological signal for the function to be performed. As a result, training often takes a long time and control of complex natural movements is prevented or prevented is at least very difficult. Therefore, the invention is based on the principle of exploiting the patient's natural patterns of activity, as he has already learned before losing his own ability to move. For the understanding of the inventive solution, it should be added that "natural neural information processing processes" are to be understood as meaning those which directly represent the central nervous processing of movement-relevant information, as is also the case in healthy people, and which therefore do not require any training. Delimitations, on the other hand, are "artificial neuronal processes" that need to be learned specifically for the control of the effector, or that may correspond to a natural brain activity, but this activity was originally unrelated to the movement to be learned before training.

An advantage of the invention is to enable the translation of the natural signals of a movement into a corresponding movement of the prosthesis by means of a direct, easily trainable access to the desired control possibilities in their entire natural complexity and diversity. Training is directly related to the control of movement in the usual way, and is therefore motivating, purposeful, not quickly tiring and generally easier, as the activity patterns learned from childhood can be used. In the application, the patient is therefore not remembered with every information to be transmitted to the artificial and laboriously learned.

This facilitates the use and acceptance of a prosthesis to such an extent that it is possible to speak of a substantially improved therapy in the sense of a real step towards a cure. Overall, the patient, apart from the very personal advantages of a regained possibility of expression, also improves his independence, social integration and even earning capacity in economic terms and reduces his dependency on social benefits and care. Advantageously, the data processing system has a receiving unit for the input signals and a drive unit for supplying the drive signals to the effector. The data processing system can then communicate with the components for information input and output via these additional units.

Advantageously, the Ableitelektrode for the determination of the electromagnetic input signals from neuronal population activity is formed, in particular an LFP (local field potential) -, ECoG (electrocorticography) - or EEG (electroencephalography) electrode. This allows for good access to the brain areas to be derived measurements of high signal stability such as vibrations or other changes in long-term derivation and accurate movement prediction.

Preferably, the Ableitelektrode is non-invasive, the Ableitelektrode is placed in operation outside on the head surface, under the skin, under the skull on the brain tissue or inserted within a sulcus without injuring brain tissue. Although invasive procedures can reach single neurons (SUA) or neuronal groups (LFP). This, however, is paid for by an injury, the consequences of which are often unforeseeable. Because of this and because of the risks of infection and surgery, the use of invasive electrodes is limited.

Advantageously, the discharge electrode determines during operation movement-related input signals from brain areas responsible for movements, in particular information processes during the presentation, planning, execution or control of a movement in the motor cortex. Although other input signals can usefully control an effector, such as visual signals. However, motion-related input signals, particularly those of the motor-controlled brain, provide the most suitable input for motion control. Preferably, in the training, one pass of a standard movement forms one

Training step, and during each training step, the input signals are recorded as training data. A standard movement (presented, presented or executed) can thus form the basis of a movement database for which the neural activity is stored. This can be used not only to directly improve the determination of the predictive model, but also much later for recalibration, such as when new, ie at the time of training still unknown, prediction developed or existing methods are further developed.

More preferably, a training step is a demonstration, introduction, and / or execution of the standard movement. These three steps, as explained in more detail below, allow an accurate and targeted training. It also exploits the plasticity of the brain, because not only the evaluation unit improves its prediction accuracy, but also the patient improves his idea of the intended movement by means of feedback from the executed movement of his prosthesis. As a result, intentional and actual movement rapidly converge on both sides.

Advantageously, the condition of the patient is categorized and stored during each training step, and this condition influences the weighting of the respective training data in the determination of the predictive model, in particular health, attention or fatigue. The inventors were able to show that the relevant activity is dependent on such state parameters. Therefore, if taken into account by the predictive model, it may become more accurate and correctly classify systemic deviations depending on the condition of the patient.

Advantageously, external circumstances are categorized and stored during each training step, and this condition affects the weighting of the training respective training data in the determination of the predictive model, in particular the position of the patient - lying, sitting, standing, the lighting conditions or the environment. These external circumstances also affect the relevant activity, so that the predictive model becomes more accurate if it takes it into account.

Preferably, a portion of the training data remains disregarded or erased in response to a non-corrective signal, and this non-corrective signal is, in particular, a predetermined standard movement. If the patient sees that his intention has been misapplied, he must have a chance to correct the appropriate training data. Particularly easy he succeeds in this, if it is possible on a - certainly learned - standard movement, because he is then not dependent on a distracting and thus disturbing other input method during training, which are in extreme cases because of his paralysis not available like.

Advantageously, the predictive model uses the standard motions as interpolation points in the composition of new motions. It will rarely be possible to train all possible movements. By means of an interpolation, the entire range of possible movements can be exploited with as few standard movements as possible to be trained.

Advantageously, the standard motion is converted into a representation of a frequency and phase modulation of the time-frequency resolved bands of the input signals. These parameters have proven themselves for fast and reliable prediction.

Preferably, the effector is a prosthesis, a patient's own body part or a data processing system. Each of these effectors has its fields of application and advantages. Thus, a separate body part is only in question, if it is still intact, and a computer allows complex behavior with little need for information, such as the triggering of an emergency signal. Advantageously, the drive signals have a destination, which are converted by the effector into an action or movement including intermediate destinations not contained in the drive signals. The patient then does not have to drive all intermediate stations during an intended movement, which would also make the correct prediction difficult. Simple physical laws often determine the more accurate motion, and these details can be determined by the evaluation unit or electronics in the effector, independent of the prediction.

Preferably, an amplifier between the Ableitelektrode and the receiving unit is provided, which filters the input signals and / or amplified. The neural signals which derive the electrodes often require treatment before they can be started with their evaluation. This in turn increases the prediction accuracy.

The inventive method has similar and further features and advantages as are described by way of example but not limitation in the appended subclaims.

The invention will be described in detail below with respect to further features and advantages with reference to the accompanying drawings. The drawing shows in:

Fig. 1 is a schematic overview of the invention; Figure 2 is a plan view of the carrier surface with a plurality of individual electrodes of a preferred embodiment of the discharge electrode.

Fig. 3 is a schematic representation of the signal transmission interface for an embodiment of the invention; FIG. 4 shows neuronal activity signals for a movement in three different states of the subject; FIG.

Fig. 5 decoding probabilities for three different variants to take into account the state of the subject in the prediction model;

6 shows an exemplary schematic representation for the conversion of neuronal signals / data into control signals for an effector with the aid of a predictive model;

7 shows an illustration of an arm prosthesis as an example of an effector, which can be actuated by an embodiment of the invention;

8 is a flow chart of the method according to the invention in use;

9 is a flowchart of the method according to the invention for constructing the database; and

10 is an overview of the system according to the invention and

Method.

1 shows an overview of the structure of a neuroprosthesis according to the invention. The particular details of the components will be described in more detail below. A lead-out electrode 1 for deriving the neuronal activity of the cerebral cortex (cerebral cortex) 2 of a human brain is inserted under the skull of a patient or, in alternative embodiments, placed under the scalp or on the head surface.

The lead-out electrode 1 measures the neuronal activity and forwards it via an Si ¹ Sg ¹ nal schnittsteile 3 as electromagnetic input signals to an amplifier 4, which is preferably designed as a multi-channel amplifier. The amplifier amplifies and filters the electromagnetic input signals of the lead-out electrode 1 with high temporal resolution and forwards the thus preprocessed signals in real time to an evaluation chip, a computer or the like system 5 Signal processing continues. There, the signals are classified in real time using a predictive model and generated corresponding control signals. At the functional descriptive level, the action intention from the activity signals is determined here for certain, trained arbitrary movements of the patient, which the patient can no longer perform himself.

With the aid of the control signals, an effector 6 is activated. This can be a classic prosthesis, ie a mechanical replica of a body part, but also its own body parts. It is also conceivable a virtual command such as the to a computer cursor or a menu selection. Conversely, the effector 6 may return effector condition signals to the system 5 to allow feedback of the effector control.

There is a whole series of derivation methods with which the activity of neurons is electromagnetically measured by a deflection electrode 1. In this case, the discharge electrode 1 is formed in the practical application as a multi-electrode.

Usually, the electrical voltage is recorded. A distinction should be made between invasive and non-invasive methods, depending on whether the lead-off electrode 1 penetrates the body or even the brain tissue, thereby necessarily injuring it. Invasive procedures include single unit activity (SUA), where the lead-in electrode 1 is brought into or even enters the vicinity of a single neuron, as well as the local field potential (LFP) determination, where the lead-off electrostatic field - de 1 measures the electric potential field from its environment that is determined by the neighboring neurons. The invasive methods allow to reach deeper layers of the brain and to obtain more accurate measurement data (spatially resolved). However, this is achieved by partially destroying the brain tissue with often unpredictable consequences for the patient. In addition, it is difficult to stably hold the lead-out electrode 1 in position to constantly obtain data of the same neurons. Among the non-invasive procedures is the electroencephalography (EEG), which allows for attaching the lead-off electrode 1 outside the skull a particularly simple application, but can provide only (spatially) low-resolution data. Although all of the mentioned methods can be used for the invention, in a preferred embodiment epicortical field potentials are measured (by means of ECoG, electrocorticography). In this case, the lead-out electrode 1 is a thin foil electrode, as shown in FIG. 2, which is implanted subdurally or epidurally over one or more selected areas of the cerebrum 2. It is alternatively conceivable to accommodate the discharge electrode 1 outside the skull. When used over multiple areas, a corresponding plurality of diverting electrodes 1 designed as described herein are used.

Experimental results of the inventors show that such thin films would be advantageous for BMI use over higher density electrodes used by default in clinical diagnosis. In a special embodiment, the lead-out electrode 1 is inserted in a groove (sulcus) of the brain. This also allows areas of the brain to be reached that are not on the outer surface without injuring brain tissue. This arrangement is explained in detail in the introductory said sister application.

The shape of the preferred foil electrode as a deflection electrode 1 will now be explained in more detail with reference to FIG. 2. The deflection electrode 1 has a carrier Ia made of flexible or elastic material. As a material offer themselves polyimide or silicone because of their compatibility or biocompatibility, easy processability and insensitivity. Equally, however, any other material is suitable which has the required flexibility and biocompatibility, ie which does not affect the brain tissue even with long-term use. Of course, the material should not be conductive. It should allow in a simple way to give the wearer its individual shape, so for example easy to cut to size. Finally, the wearer must be elastic and thin enough, usually with a thickness. In order not to injure tissue, the wearer has rounded edges.

On the carrier are a number of electrodes Ic, which are each connected individually to a cable Ib, can be passed through the signals to the outside. The carrier Ia is shown in a rough approximation rectangular. In the application, it will often be advantageous to adapt it to the area to be derived.

The electrodes Ic are arranged in matrix form as contact points. Conductor tracks Ie inside the carrier Ib connect each electrode Ic individually and without overlapping their respective interconnects Ie with the cable Ib for the signal exchange. The production possibilities of such printed conductors Ie and possibilities of their arrangement are known to the person skilled in the art.

Also electrodes can be made of different materials, in particular gold, platinum, a metallic alloy or also of conductive plastics and semiconductor materials. The support 1a may take on a size of less than one to more than ten centimeters. The electrode contacts are designed with a typical density of 1 to about 1000 electrode contacts per cm ² . A higher density of electrode contacts improves the signal resolution, but of course increases the expense not only of the production of the electrodes Ic, but also of the gain and the computational complexity of the drive. Note also the increased energy requirements for wireless transmission of many channels of a large number of electrodes. The energy supply can definitely become the limiting factor.

Of course, the arrangement may vary as needed from the illustrated here with staggered rows. Here practically every arrangement of a point share on a (top) surface is possible. Advantageously, this carrier does not injure the brain tissue, in contrast to penetrating electrodes. Thus, a better long-term stability of the signal acquisition is achieved because the penetrating into the brain tissue electrodes can lead to a local tissue destruction and thus the extinction of local neuronal activity. Depending on the application, the carrier Ia with the electrodes Ic can also be very small ("lern"). In this case, the operation with which the carrier Ia is used for the patient, with little effort and very little impairment of the patient is possible.

This operation, with which the lead-out electrode 1 is used, requires specific pre-surgical diagnostics and surgical planning. One of the most important aspects is to determine the exact target area for implantation, which can not be determined a priori due to the strong inter-individual neuroanatomical variability of the human brain. Only in exceptional cases would it be desirable to insert it at a location that was not previously determined individually. Although one knows general mappings of the brain and therefore knows, where certain functional areas are roughly to be found. In the more specific example of the motor and somatosensory cortex, even the anatomy of humans is replicated locally and individual body parts are assigned to spatially distinct areas of the cortex. For the individual patient, however, this prior knowledge is mostly not accurate enough.

An exact localization of the target areas for an individual patient is therefore performed preoperatively by using functional magnetic resonance imaging (fMRI). Here, site-specific activation of the brain is measured while the patient is trying to imagine or observe the control of the effector and thus the implantation site can be determined with high spatial accuracy. In addition, to improve the localization, an EEG measurement with subsequent source reconstruction during the same mo- toric paradigms (trying, imagining or observing the effector control).

FIG. 3 shows a preferred embodiment of the signal interface 3. Alternatively, as a solution, data transmission by cable can take place, as has hitherto been used by default in neurosurgical diagnostics. A permanent cable connection through the body surface, however, carries an increased risk of infection and is also less attractive from a cosmetic and practical point of view. In the preferred embodiment described here, the signal transmission between the electrode and the amplifier takes place by inductive energy transmission without a transcutaneous cable connection.

The wireless signal transmission system 3 is divided into two parts each above and below the skin surface 3a. From the outer transmitter / receiver unit outside the body, that is to say here above the skin surface 3a, only a coil 3b is shown as representative. In one embodiment, this external transceiver unit can only transmit data to the amplifier 4 or the computer system 5 wirelessly or by direct cable connection. Also conceivable is an alternative embodiment in which amplifier 4 and / or computer system 5 is partially or completely contained in a chip which is accommodated on the skull surface or another suitable location on the body. Which embodiment is preferred in each case or is feasible at all depends on the complexity of the application. At present, at least one compact transmitting / receiving unit to an external amplifier 4 or a computer system 5 over almost arbitrary distances (mobile radio, Bluetooth, WLAN) is technically possible without further ado.

One of the mentioned transmission paths can also be used for the data exchange with the effector 6. In the case of an activation of a paralyzed natural body part, a further two-part signal transmission interface similar to that described here can be used in the respective body part. Since the external transceiver unit is easily accessible, it can also be adapted or replaced according to the advancing technology, without the need for a renewed surgical intervention.

Below the main surface 3a, a multi-functional chip 3c is used as the inner transmitting / receiving unit as a counterpart to the outer transceiver unit. This multi-functional chip 3c has a receiver unit 3c 1, a transmitter unit 3c2 and optionally a battery unit 3c3. Via the cable Ib, the signals from the electrodes Ic of the carrier Ia of the transmitter unit 3c2 and the receiving unit 3c 1 are supplied.

In operation, the coil 3b of the outer transceiver transmits power and any control signals for the Ableitelektrode 1 inductively via radio frequency signals to the receiving unit 3c 1. Such control signals can be on and off commands or about a query on the power state of the battery 3c3. The interface is basically also suitable for the transmission of stimulation signals to the deflection electrode 1.

The multifunction chip 3c determines the modulated control or the said stimulation signals in a manner known from telecommunications engineering. The energy for the required arithmetic operations of control units in the multi-functional chip 3c are obtained from the high-frequency signals. Alternatively, battery 3c3 or an accumulator can be inductively charged via the high-frequency signals, so that the energy supply is decoupled in time from the transmission at the interface. Of course, it is necessary to distinguish between charging and stimulation signals, for example by time windows or by separate frequency bands.

Conversely, signals from the measuring electrodes Ic are transmitted via the cable Ib to the transmitter unit 3c2 and there, preferably in the 402-405 MHz signal band of the MICS (Medical Implantable Service Band), on the coil 3b or one for the Reception designed counterpart of only representatively shown coil 3b transmitted.

So far, the transmission interface has been described so that the transmission power of the transmitting unit 3c2 extends only to the coil 3b of the external transmitting / receiving unit. Alternatively, the transmitting unit 3c2 could also send directly to the amplifier 4, which may not even be seated on the skull surface. In this case, the power supply of the multi-functional chip 3c either by long-lasting batteries (currently technically unsatisfactory) or a Aufla- possibility to ensure approximately in the manner described by induction or by exploitation of the body's own energy sources.

The input signals of the deflection electrode 1 are now accessible to the amplifier 4 outside the skull. The amplifier 4 operates on the input signals in a known manner. In addition to a gain, high-pass, low-pass or band-pass filters may be used (for example Savitzky-Golay, Buttersworth or Chebychev filters). A high temporal resolution for real-time transmission is advantageous, ideally the sampling rate is more than 200 Hz, but lower values are not excluded. The thus preprocessed input signals are then forwarded to the system 5 for evaluation.

The function of the system 5 is to calculate control signals for the effector 6 from the input signals. In this case, two phases can be distinguished, namely a training and an application phase. The application can be interrupted again and again by training to improve or to learn further controls. The training is used to determine a predictive model, with the help of which the system 5 then detects during the application, the movement intention of the patient and calculates corresponding control signals. The aim of the training is to enter into the system 5 the required data for the classification of standard movements in order to determine the predictive model. For this purpose, a standard movement is selected for the individual training step, which the patient then gets presented in one of three scenarios, introduces himself or attempts to control the effector 6. It has been proven that this produces similar input signals as a natural, self-made controller. If the paralysis still allows a (residual) movement, of course, the actual movement can be used. Among other things, this is conceivable in the case of non-sudden paralysis causes such as prior to surgical amputations or slowly progressive diseases such as muscle wasting.

In the training paradigm is an important aspect of the invention. The patient does not have to produce a state of neuronal activity dictated by the evaluation procedure, which under natural conditions does not serve to control the function to be controlled, as conventionally described in the introduction, under tiring, highly concentrated concentration. Instead, he naturally resorts to his patterns of activity, which he had long since learned throughout his life. Subjectively, the patient does nothing else during training or during the application, as if he wanted to address the healthy muscles just as before the illness.

During each repetition of the training step, the system 5 stores the incoming brain signals and can thus later allocate brain signals to specific movements.

This will be explained in more detail with reference to an arm / hand prosthesis. A standard exercise to be trained is determined by the patient, either by selection from a computer menu or by communication with a helper. Examples of standard movements with the aforementioned arm prosthetic hand can to be: open hand - close hand, move arm in different directions and lead back, or close the hand with different strength.

Now the three different training datasets are generated. First, the subject is repeatedly shown the chosen standard natural motion. The derived brain signals in a time interval from about one second before until the completion of the standard motion yields a record for the first training data set of the matrix. Repeated recording of the selected and the remaining standard movements to be learned results in the further data sets which together form the complete first training data set.

Thereafter, a computer program repeatedly dictates to the subject sequentially the standard movements, and the brain signals derived therefrom yield the second training data set. In the third step, a computer program tells the subject to try the standard moves. The derived during this time brain signals result in the third training data set.

The training is now optimized by providing feedback on the accuracy of the predictive model used to correctly correlate the brain signals derived during each experiment to the displayed standard motion. So the test person notices directly how well he has already learned a movement, and his brain has the ability to adapt to the effector 6. In other words, not only the system 5 improves through the training, but also the patient due to the neuronal plasticity. It is important that this does not happen due to strenuous activity patterns to be generated artificially. The subjective feeling of the patient is much more in line with the natural process of learning a new motor skill, such as swimming or cycling. Because of the efficiency of feedback learning, the third training set's data will be weighted progressively more heavily than the other two training sets. The last part of the training is the repetition of the third part, but under different subjective conditions (eg prepared and unprepared, awake and sleepy) and objective conditions (eg lighting conditions, posture like sitting, standing, lying). The result is an enhanced third training set that further improves training performance and prediction accuracy, ensuring a more robust association of brain signals under different circumstances. The subjective conditions are also called states, the objective conditions as circumstances.

Fig. 4 shows the measured activity of a particular neuron over time for one and the same movement in three different states 1, 2, 3 of the experimental animal. As can be seen from the three diagrams, the neural activities or signal patterns differ considerably, in particular also during the execution of the movement. The measured signal patterns are therefore classified and stored with respect to the trained motions and states, e.g. in a database.

In use, the measured neural activities of the patient are compared with the stored signal patterns to determine the movement that the patient is intending to perform with the effector. Here, if possible, the conditions - external and internal - are recorded.

If one chooses a general probabilistic approach according to Bayes, this means that the model for the motion prediction B from neural activity N, P (BIN) = P (B) * P (N) ^"1 * P (NIB) for a optimal motion prediction needs to be extended to state Z:

P (BIN ₅ Z) = P (B) * P (ZIB) * P (Z) ^"1 * P (NIZ) ^{" 1} * P (NIB ₅ Z). FIG. 5 shows how advantageous it is to train the system 5 under many different conditions and to have available signal patterns with reference to the conditions and the movements presented. There are, for two living beings, "subject 1" and "subject 2", the probabilities of correct decoding of intended movements depending on the number of neurons used (signal pattern) for different trained predictive models specified. Black bars indicate the decoding probability if the application takes place under known, ie trained conditions; Mid-gray bars apply in the event that the states are not known, and all three trained states are combined, as it were, "into a large trained condition", while light gray bars denote the case of conditions other than training.

As you can see, the case that the states are known and associated training data exists (ie motion training data exists) consistently gives the best results. If many neurons (signal patterns) are measured at the same time, the medium gray bar is almost as good. That is, the combination of the training data of the three different states can compensate for the loss of specificity of the available training data.

The case that only training data is available that has been obtained in another state (light gray bars), consistently delivers significantly worse results. For example, in subject 1, even with 100 neurons, there is an accuracy of almost 70%, compared to 97% and 96%, respectively, when training data of the same state is present or when the training data of all states are combined.

10 summarizes the mode of action of the system 5 during the application, that is to say in "normal operation." The neuronal activity in the brain is determined under the condition or condition X measured. If training data (ie training signal patterns) are present in the system for the condition X or the state X, they are used for determining the movement prediction or the movement activation signals for the effector 6. If, on the other hand, there are no training data for condition X in the system, existing training data of similar conditions are used to determine the movement prediction. If condition X is unknown, for example because it is not captured, all training data that exists in the system will be used to determine the motion prediction.

As stated above, the more conditions and associated training data have been trained in the system 5, the better the movement prediction.

In addition, the patient can independently start a new training program for new standard movements or for ones already learned under a new circumstance. Finally, the patient can train a 'correct / incorrect signal' that will enable a workout restart. This gives the patient control to tell the system 5 that the prediction accuracy does not meet the requirements and that a re-training is required. Of course, this restart could also be commanded independently of standard movements, for example via a computer menu. However, if the effector 6 is the patient's only communication capability, he should have this robust capability through the specially-trained "non-correct" default motion to communicate to the system 5. Alternative non-correct signals are original / natural brain error signals that may occur, for example if an assignment is wrong.

6 shows an exemplary schematic representation of the conversion of input signals into effector control signals with the aid of the training data. Three voltage curves of three electrodes Ic are shown by way of example on the left side. These voltage signals are first amplified and filtered as input signals in the amplifier 4. The filter functionality can also be localized in system 5. be siert. As an exemplary filter method - others are mentioned above in connection with the amplifier 4 - the voltage signals are filtered in a bandpass, then averaged over small time windows and divided into short time windows. The activity is then evaluated by means of mathematical methods. The predictive model is therefore determined on the one hand by selection of the mathematical method, on the other hand by calibration by means of the training data. Thus, the prediction of the intended movement by the system 5 is enabled ("intention prediction").

Typical mathematical methods are: (1) preprocessing of the signals, for example a) filtering (eg low-pass or bandpass), b) time-frequency analysis (e.g., Fourier transform or multi-tapering) and / or c) binning and averaging in the time domain; (2) decoding of the preprocessed signals, for example discriminant analysis (linear, quadratic or regularized) or support vector machine (linear or radial basis function). The methods mentioned are not exhaustive. In particular, for the decoding of continuous movements, other than the mentioned discriminant analysis and support vector machine are to be used, for example linear filters or Kalman filters.

The result is the Effektorsteuerungssignale shown on the right, in which case two effector devices, such as two motors, and their applied power are shown as an example according to their rotational speed.

It should be emphasized that the predictive model is not limited to recognizing the limited number of standard movements. Further movements can be detected by interpolation and extrapolation of the trained and recorded neuronal correlates during standard movements.

The effector 6 translates the effector control signals provided by the system 5 into movements. It should be repeated again that movements in understand a comprehensive sense, so may include a voice control or the mimic.

For the effector 6, the three above-mentioned groups come into consideration: a mechanical device such as a robot, robotic arm or prosthesis, a separate body part or an electrical device controlled by a virtual command of a computer, such as a computer, a mobile device, a household appliance or like.

The first case of a prosthesis, ie an artificial replica of a body part, will be explained in more detail below with reference to a hand prosthesis shown schematically in FIG. Of course, any type of prosthesis can be controlled.

Via an effector input line 6a 1, the effector control signals are transmitted from the system 5 to the effector 6.

The prosthesis has a rotation system 6b 1 for rotating the hand. Control of a motor of the rotation system 6b 1 rotates the prosthesis in accordance with the effector control signals. In addition, the prosthesis has a gripping system 6b2 with motor and control that performs the effector control signals corresponding opening and closing movements of a finger part of the hand. It should be noted that it is not expedient to attempt to determine all control details from the neural data. Instead, system 5 could also only predict the type of movement intended and then autonomously determine the required individual steps.

For the generation of a feedback pressure sensors 6c are attached to the finger part. Their detected effector state signals are returned to the system 5 via an effector output line 6a2. These data could be returned to the brain as stimulation data in an extension of the invention. ben. In addition, the system 5 should also be able to query the actual state of the prosthesis, since the internal representation in the system 5 does not necessarily coincide with this. This is not necessarily due to inaccuracies of the system 5, but could also be caused by an external disturbance, such. As a bumping of the prosthesis, caused. By means of the queried actual state, the system then knows again the starting position for a correct motion specification.

Finally, the prosthesis is still surrounded by a covering which expediently imitates the appearance of a human hand. It should be noted that a hand prosthesis is not limited to the opening and closing, but with technically advanced prostheses within the scope of the invention, the implementation of more complex movements is possible.

In the second group of possible effectors 6 with special medical relevance, own body parts are activated via functional electrostimulation as effector 6, if only the neuronal connection between the brain and the body part is interrupted. Either intact nerve cells of the body part or directly the muscle fibers are stimulated. Any feedback may also be provided either via still intact endogenous pressure, strain, etc. Receptors or by means of supporting sensors, as described above for the case of the prosthesis control. Likewise, even in the case of incomplete paralysis, where there is still a (weak) residual mobility, it is conceivable to support these residual movements by motor-driven mechanical devices.

The third group of "virtual" effectors 6 is particularly large. Thus, a computer cursor or a menu selection, but also the switching on of the light, the sending of an emergency call, etc. are controlled. Of particular interest is the control of a virtual prosthesis. In this case, a body part is displayed three-dimensionally on a screen and controlled by neuronal activity of the patient or subject. This considerably facilitates the adaptation and selection of a suitable prosthesis.

Once you have such control and stimulation data available in computer-processable form, of course, thanks to the Internet, you no longer depend on physical proximity. The effectors to be controlled therefore do not have to be in direct spatial proximity / connection to the individual controlling the effector. Thus, a prosthesis or a robot could be virtually represented and controlled, which are actually in another place. Medical applications in which a surgeon operates remotely, in the military, which can control robots with high precision, without endangering people, or in contaminated or otherwise inaccessible areas such as nuclear power plants, the deep sea, the universe are conceivable ,

FIG. 8 summarizes the method according to the invention for the movement control of an effector 6 associated with a living being. The method comprises the following steps: a step 810 of receiving input signals representative of neural activities of the brain of the animal; A step 850 of obtaining motion drive signals for the effector 6 based on the input signals, wherein signal patterns are detected from the input signals, step 820, and from the signal patterns, the motion drive signals are calculated based on a predictive model, in steps 830 and 840; in which, according to a predetermined predictive model, a comparison process of the detected signal patterns is performed with stored training signal patterns representative of neural generated by the brain of the animal

Activities related to trained standard movements are and each corresponding stepwise movements of the effector 6 are assigned, step 830, and wherein those stepwise movements are used for obtaining the movement drive signals associated with those training signal patterns, which according to the

Comparison process are the most similar to the detected signal patterns, step 840.

9 illustrates the method of the invention for constructing a database of data from a predictive model for use in motion control of an effector 6 associated with an animal. The method comprises the following steps:

A step 910 of receiving input signals representative of neural activities produced by the brain of the subject with reference to standard trained movements of the subject

Are living thing;

A step 920 of storing the input signals in the database as a training signal pattern; and

A step 930 of assigning stepwise motion control signals to the training signal patterns according to the respective standard movements.

Used reference signs

1 discharge electrode Ia carrier

Ib cable

Ic (single) electrodes

Ie tracks

2 cerebrum / cortex 3 signal interface

3 a skin surface

3b coil of the outer transceiver

3c multi-functional chip

3c 1 receiving unit 3c2 transmitting unit

3c3 battery / accumulator

4 amplifiers

5 system for evaluation and central control

6 Effector 6a 1 Effect input line

6a2 effector output line

6b 1 rotation system

6b2 gripping system

6c pressure sensors 6d panel

Claims

February 12, 2007 Albert-Ludwigs-Universität Freiburg U50875PC GS claims

A device for motion control of an effector (6) associated with an animal, comprising: a lead-out unit (1) for receiving input signals representative of neuronal activities of the brain (2) of the animal and a data processing system (5) an evaluation unit for obtaining motion control signals for the effector (6) on the basis of the input signals, wherein the evaluation unit is designed to determine signal patterns from the input signals, to calculate the motion control signals for the effector (6) from the detected signal patterns and to perform according to a predetermined predictive model, a comparison process of the determined signal patterns with stored training signal patterns representative of neural activities generated by the brain of the animal with respect to trained

Are standard movements and to which respective respective stepwise movements of the effector (6) are associated, and wherein those stepwise movements are used to obtain the movement activation signals associated with the training signal patterns determined in the comparison process.

2. Device according to claim 1, wherein the data processing system comprises a receiving unit for the input signals and a drive unit for supplying the drive signals to the effector (6).

3. Device according to claim 1 or 2, wherein the discharge unit has at least one Ableitelektrode (1), which is adapted to determine in operation movement-related input signals from responsible brain areas, in particular of information processing in the presentation, planning, execution or control of a Movement in the engine cortex.

4. Device according to claim 3, wherein the lead-out electrode (1) is designed for the determination of electromagnetic input signals from neuronal population activity, in particular a local field potential, electrocorticography or electroencephalography electrode.

5. Device according to one of the preceding claims, wherein the state of the living being is categorized and stored during each training step and the state influences the weighting of the respective training data in the determination of the predictive model, in particular body posture, health, attention or fatigue.

6. Device according to one of the preceding claims, wherein the evaluation unit is configured so that external circumstances are categorized and stored during each training step and the state influences the weighting of the respective training data in the determination of the predictive model.

7. Device according to one of the preceding claims, wherein the evaluation unit is configured such that a detected signal pattern in the calculation of a movement drive signal to a non-correct signal is disregarded, wherein the non-correct signal corresponds to a natural error signal of the brain.

8. Device according to one of the preceding claims, wherein the evaluation unit is configured to, and that a detected signal pattern is disregarded in the calculation of a movement drive signal to a non-correct signal, the non-correct signal corresponds to a natural error signal of the brain.

The apparatus of any preceding claim, wherein the predictive model uses the standard motions as interpolation points in composing new motion drive signals.

10. Device according to one of the preceding claims, wherein the standard movement is converted into a representation of a frequency and phase modulation of time-frequency-resolved bands of the input signals.

11. Device according to one of the preceding claims, wherein the effector (6) is a prosthesis, a separate body part of the patient or a data processing system.

12. Device according to one of the preceding claims, wherein the movement drive signals have a target which can be converted by the effector (6) into an action or movement including intermediate destinations not included in the drive signals.

13. Device according to one of the preceding claims, wherein an amplifier (4) between the Ableitelektrode (1) and the receiving unit (5) is provided, which filters the input signals and / or amplified.

14. A method of motion control of an effector (6) associated with a subject, the method comprising the following steps having:

Receiving input signals representative of neuronal activities of the subject's brain, and obtaining motion actuation signals for the effector (6) based on the input signals, determining signal patterns from the input signals, and calculating from the signal patterns the motion drive signals based on a predictive model wherein, according to a predetermined predictive model, a comparison process of the detected signal patterns is performed on stored training signal patterns representative of neural activities generated by the subject's brain with respect to standard trained movements and corresponding stepwise movements of the effector (6) respectively. and those stepwise movements are used to obtain the motion drive signals associated with those training signal patterns that determined the one according to the comparison process n most similar to signal patterns.

15. The method of claim 14, wherein the method is performed in real time.

16. The method of claim 14 or 15, wherein for calculating a state of the living being is detected and wherein according to the predictive model predetermined states of the living being are detected and related to the neural activities and the training signals.

17. The method according to any one of claims 14 to 16, wherein according to the predictive model, external circumstances of the living being are detected and be related to the neural activities and the training signals.

18. The method of claim 14, wherein the input signals are motion related and representatively neural activities of for

Movement responsible brain areas, wherein the neural activities in particular information processing processes in the presentation, planning, execution or control of a movement in the motor cortex of the brain correspond.

19. The method according to claim 14, wherein a detected signal pattern is disregarded in the calculation of a movement activation signal in response to a non-correct signal, wherein the non-correct signal corresponds in particular to a defined standard movement.

20. The method of claim 14, wherein a detected signal pattern is disregarded in calculating a motion drive signal in response to a non-corrective signal, the non-corrective signal corresponding to a natural error signal of the brain.

21. The method of claim 14, wherein the prediction model uses standard movements as interpolation points in the composition of motion drive signals.

22. A method of constructing a database of data of a predictive model for use in a motion control of an effector (6) associated with a subject, the method comprising the steps of:

Receiving input signals representative of neural activities produced by the brain of the animal related to trained standard movements of the animal, Storing the input signals in the database as training signal patterns, and

Assigning stepwise motion drive signals to the training signal patterns according to the respective standard movements.

23. The method of claim 22, wherein in a training, a sweep of a standard movement of the animal forms a training step, and during each training step, input signals are recorded as a training signal pattern.

The method of claim 22 or 23, wherein a training step includes presenting, imagining, and / or performing a standard move.

25. The method according to claim 22, wherein the state of the living being is categorized and stored in the database during each training step, and the state influences the weighting of the respective training signal patterns in the determination of the predictive model, the state in particular posture, health,

Attention and / or fatigue includes.

26. The method according to any one of claims 22 to 25, wherein a training series standard movements in various states is performed.

27. The method according to any one of claims 22 to 26, wherein a detected signal pattern is discarded in response to a received non-correct signal, without being stored.

28. The method of claim 22, wherein a stored training signal pattern is responsive to a received non-correct signal out of Database is deleted.

29. The method of claim 27, wherein the non-correct signal is representative of the neural activity of a predetermined standard motion or natural error signal.

30. The method of claim 13, wherein the input signals are electromagnetic input signals of neuronal population activity, in particular from a local field potential, electrocorticography or electroencephalography derivative.

The method of any one of claims 14 to 30, wherein the predictive model uses the standard motions as interpolation points in composing new motion drive signals.

The method of any one of claims 14 to 30, wherein the neural activity corresponding to a standard motion is converted into a representation of a frequency and phase modulation of the time-frequency resolved bands of the input signals.

33. The method according to any one of claims 14 to 32, wherein the to be controlled effector (6) is a prosthesis, a separate part of the body or a data processing system.

The method of any one of claims 14 to 33, wherein the motion actuation signals comprise a target which is converted by the effector (6) into an action or movement including intermediate goals not included in the actuation signals.

35. The method according to any one of claims 14 to 34, wherein the Predictive model is capable of learning.