WO2002082421A2

WO2002082421A2 - Method for touch simulation on a keyboard

Info

Publication number: WO2002082421A2
Application number: PCT/FR2002/001225
Authority: WO
Inventors: Laurent Collot; Bertrand Reulet
Original assignee: Laurent Collot; Bertrand Reulet
Priority date: 2001-04-09
Filing date: 2002-04-09
Publication date: 2002-10-17
Also published as: FR2823328B1; FR2823328A1; WO2002082421A3

Abstract

The invention concerns a method for touch simulation on a keyboard (1) which consists in performing the following actions iteratively: measuring at least a movement parameter of each key (2) of the keyboard (1) using a sensor (11); calculating a force exerted on the finger of the instrumentalist based on the measured parameters and deducing therefrom a control value; driving an actuator (3) which applies the calculated force to the key (2). The invention is characterised in that it consists in calculating said force on the basis of a modelling of the dynamic mechanical behaviour of the different elements of a mechanism of a keyboard instrument which is to be simulated, the force calculated by means of said modelling corresponding to the reaction force which would be exerted by the mechanism on the key (2) taking into account the prior dynamic trajectory thereof.

Description

METHOD OF SIMULATING THE TOUCH ON A KEYBOARD

The present invention relates to the simulation of the touch of instruments, and in particular the touch of keyboard instruments. When an instrumentalist plays a keyboard instrument, he uses the keys of the keyboard as well as the possible pedal games and couplings to produce a melody. The pitch of the notes played depends on the key pressed. The tone of the sound produced, in particular the timbre or the loudness, depends on the force it exerted on the key during its movement.

For example, for the piano, the sound intensity of a note depends on the speed of the hammer at the time of the impact on the string. The decay of the note is determined by the position of the dampers after the sound has been emitted. The final speed of the hammer depends on the kinematics of the key pressed by the pianist.

When an instrumentalist presses a key, he feels a reaction force under his finger due to the action of the mechanism linked to this key. The evolution of this reaction force during the movement of the key constitutes the "touch" of a keyboard instrument. Touch is essential for an instrumentalist because it allows him to feel the impulses he gives to the keys. Thus, depending on its sensitivity, it can control the sound of the notes.

In electronic keyboards, sounds are generated electronically and not by the triggering of a mechanism. The keys of these keyboards do not have the same mechanism as that of a traditional instrument but generally a simplified mechanism which aims to offer a dynamic touch to the instrumentalist. The result is a very mediocre feel, incomparable to that of the traditional instrument. This is why, efforts have been made to artificially reconstruct on electronic keyboards a touch similar to the touch of certain existing instruments. For example, document US 4,899,631 describes a keyboard in which the reaction force is exerted by electric motors connected to the keys by cables. A microprocessor makes it possible to control each motor according to the speed of the keys, their acceleration, the force exerted by the instrumentalist, inertia and balance parameters. To this end, the motors are controlled by an electrical parameter, for example the intensity. The measurement of another electrical parameter, for example the voltage, makes it possible to know the speed of the motor and therefore the speed of the associated key. The system further includes an interface for modulating certain response sensitivity parameters.

Document US 5,922,983 proposes a device comprising a keyboard provided with sensors, actuators and a microprocessor. The position, speed and acceleration of a key are measured by the associated sensors. The force applied to this key by the actuator is the sum of three components read from data tables.

The disadvantage of these various devices is that they do not allow the touch of a real instrument to be faithfully reproduced because they generate a reaction in response to a measurement of instantaneous parameters of the movement of the key. To overcome this drawback, the invention proposes a method of touch simulation making it possible to take into account not only the instantaneous kinematic parameters of each key pressed but the whole of its movement since the instant of start of actuation. This is essential to simulate the touch of instruments with a complex mechanism for which touch depends not only on the instantaneous kinematic parameters of the keys but also on those of the moving parts driven by the keys.

To this end, the invention proposes a method of simulating touch on a keyboard according to which: - at least one movement parameter of each key of the keyboard is measured by means of a sensor, - a force to be exerted is calculated as a function of the parameters measured and a control quantity is deduced therefrom,

- an actuator is controlled which applies the calculated force to the key, and consequently to the finger of the instrumentalist characterized in that said force is calculated according to a modeling of the dynamic mechanical behavior of the various elements of a mechanism of a keyboard instrument that we want to simulate, the force calculated using this modeling corresponding to the reaction force that would be exerted by the mechanism on the key. This process allows to faithfully reproduce the touch of an existing instrument insofar as it takes into account the dynamic behavior of each of the elements which constitute the real mechanism connecting a key to an element producing the sound.

In particular, according to the method of the invention, during the actuation of a key, information is stored on the past of the trajectory of the key as well as on the moving parts of the mechanics to be simulated, and we take take this information into account to calculate the force to be exerted on the key.

The parameters of the calculation algorithm being determined for a given type of instrument, it is advantageously possible to modify them in order to modulate the restored touch. For example, the player can lighten or harden the response of the keys.

The simulation process also makes it possible to simulate a completely new instrument, whose touch is not found among traditional instruments.

Since the algorithm constantly calculates the position of moving parts in the mechanics it simulates, it is able to provide with great precision the moment when for example a hammer of a piano touches a string, and what speed the shock takes place. It is therefore possible thanks to this algorithm to determine the exact moment when the sound is produced and with what intensity. Thanks to the method of the invention, it is possible to generate, transmit or record game orders. These game orders can also be sent to an electronic synthesizer or a motorized instrument.

The method also has the advantage of exclusively using an algorithm for determining dynamic forces, which lends itself to the use of naturally digital sensors and actuators, and to the use of digital streams in binary form, as for example example encoding by pulse width modulation.

Other characteristics and advantages of the invention will emerge from the description which follows, which is purely illustrative and not limiting and should be read with reference to the appended figure which illustrates an example of a device for implementing the method of invention.

In the example of the figure, the device comprises a keyboard 1. Each key 2 of the keyboard 1 is connected to an actuator 3 capable of exerting a force on the key. Key 2 is provided with a sensor 11 making it possible to measure a movement parameter, for example its position or its speed. The output of the sensor 11 is connected to a multiplexer 10. The multiplexer 10 collects the signals coming from a set of sensors linked to the keys of the keyboard 1. The multiplex signal is converted by an analog / digital converter 9 before being transmitted to a computer 7.

The computer 7 comprises an interface 8, memory means and calculation means. It receives the digital signal from the converter 9 and sends a digital setpoint signal to a digital / analog converter 6. The setpoint signal is demultiplexed by the multiplexer 5. The demultiplexed signal is transmitted to a control module 4 of a actuator 3. The actuator 3 thus controlled acts on the corresponding key 2. The use of multiplexers 5 and 10 makes it possible to reduce the number of signals to be converted.

When the instrumentalist actuates the key 2, the movement which it imposes on it is measured by the sensor 11, coded and transmitted to the computer 7. For each key, the computer 7 calculates in real time its position, its instantaneous speed and acceleration , and deduces based on a algorithm the force to be restored to the key 2. It sends a setpoint signal back to the actuator 3 which then applies to the key 2 the force thus calculated.

To simulate the touch of a real keyboard instrument, the force to be restored is the force that the finger of an instrumentalist playing on a real instrument would undergo. The algorithm used to calculate the force contains routines which model the behavior of certain elements of the mechanism of the real instrument. We associate with this instrument an ActionMechanics routine which calculates at each time step the force to be applied to a key considered. The ActionMechanics routine receives as input a pointer to a DynMsg structure which contains the number of the key, its position, its speed and its acceleration:

typedef struct {int channel; float position, velocity, acceleration; } DynMsg;

Information on the past of the trajectory of a key as well as on the moving parts of the mechanics to simulate is stored in local variables which keep the necessary information, allowing the routine ActionMechanics to calculate the force which must feel the instrumentalist. This force is the output parameter of the routine.

The interface 7 allows a user to modify certain parameters of the simulated mechanics during the game. These parameters are stored in a kbm_aparameter type structure, which is accessed by the PARAMETER (int) macro.

The mechanics of most instruments can be modeled by a series of static weights, springs, friction, inertia, rebounds and masses launched. To faithfully reproduce the simulated touch, these different behaviors are modeled separately. Weight modeling

Weight is a constant force. We simulate a weight affecting the force to determine a stored parameter by executing:

force = PARAMETER (n_weight);

where njpoids is the number of the parameter containing the resting weight to apply.

Modeling of an elastic force

Simulating a spring consists in applying a force proportional to the difference between the position of the key and its position corresponding to the spring at rest. We simulate a spring by executing:

ch = dynmsg-> channel;

Y = dynmsg-> position-HLG [ch] .lowpos; force = PARAMETER (n_raideur) * (Y-PARAMETER (n_repos));

where n_raideur is the number of the parameter describing the stiffness of the spring and n_repos that of the parameter giving its rest position. The HLGQ structure table contains the positions of the keys at the top and bottom as well as a calibration value of the effector (force per bit of the converter):

struct hi_lo_gain {value_t hi_pos, low_pos; float gain;

};

Friction To simulate a viscous friction, apply a force proportional to the speed of the key, in the opposite direction. We execute: V = dynmsg->velocity; force = -V * PARAMETER (n_visco);

where n_visco is the number of the parameter describing the viscosity of friction. Any other form of friction can be envisaged according to the same principle, such as for example a friction force proportional to the cube of the speed of the key.

Inertia Similarly, to simulate an inertial mass, a force proportional to the acceleration of the key is applied in the opposite direction. We execute:

A-dynmsg-> acceleraϋon; force = -A * PARAMETER (n_inertia);

where n nertia is the number of the parameter describing the mass in motion.

Mass launched

The modeling of a launched mass requires the calculation of the kinematics of the mass in question. The mass launched is described by its mass_pos position and its mass_velo speed. The force felt under the finger depends on the presence or not of the mass on the rod; the stuck flag reflects this presence. It is necessary to describe the coupling member between the rod and the mass. A simple model is to use a spring in parallel with a viscous friction. The choice of the stiffness of the spring and the viscosity of the friction allows the control of the feeling of the rebound. We execute as follows:

if (stuck) force = (Y-mass_βos) * PARAMETER (n_raideur) - (V-mass_velo) ΨARAMETER (n_visco); else force = 0; The mass trajectory also depends on whether it rests on the rod or not:

if (stuck) {mass_pos = Y; mass_velo = V; } else {mass_velo- = PARAMETER (n_gravity); mass_pos + = mass_velo; }

where n_gravity describes the effect of the force of gravity on the mass when the latter is free.

It remains to describe the bonding / detachment test of the mass. The. detachment takes place if the force undergone by the mass is greater than gravity, while bonding can be arbitrarily chosen when the mass passes under the rod:

if (force> gravity) stuck≈O; if (mass_pos <Y) {mass_pos = Y; stuck = 1; ;

Thus, if the mass / rod coupling is stiff, the mass will rebound. As it will have lost energy due to the viscous friction of the coupling member, it will go back up lower than during the previous rebound, to end up sticking on the rod.

The damper_action.h module reproduces weight, spring, viscous friction and inertia. The mass_action.h module simulates the behavior of a ball which can be thrown by pressing the key and which rebounds on a felt located at the end of the arm of a key.

From these elementary routines, we build an algorithm allowing to simulate a mechanism. The clavichord_action.h, harpsijaction.h, clavinova_action.h, organ2_action.h, and piano_action.h modules simulate the touches of a clavichord, a harpsichord, a typical commercial digital piano keyboard, a organ and piano.

Simulation of a clavichord

The mechanism of a clavichord key consists of a plectrum (integral with the key) which strikes a string. Three forces are therefore present: the weight of the fingerboard, its inertia, and the stiffness of the string if it is in contact with the plectrum. Four parameters are involved in this modeling:

- PARAMETER (O) is the weight of the key,

- PARAMETER (I) is the stiffness of the rope,

- PARAMETER (2) is the position of the string,

- PARAMETER (3) is the inertia of the mechanism. An additional parameter PARAMETER (4) is introduced in order to alleviate a tendency of the key to oscillate because of its flexion modes: instead of simulating the inertia as presented above, it is preferable to couple the key to a mass virtual via a dissipative element. The mass undergoes the force of inertia and transmits its movement to the key via a force proportional to the cube of the difference in their speeds, the proportionality factor being PARAMETER (4). The force calculation is therefore written:

// Action of the virtual mass on the force key = PARAMETER (4) * pow (mass_velo, 3.); // Kinematics of the virtual mass mass_velo + = force / PARAMETER (3); // Force weight action + = PARAMETER (0); // Action of the string bump = Y-PARAMETER (2); if (bump <0) force + = bump * PARAMETER (1);

where bump measures the distance between the plectrum and the string. The model allows you to know exactly when the plectrum strikes the string, and can then give the order to a synthesizer to generate a sound of the pitch corresponding to the key and the sound intensity being related to the kinetic energy of the plectrum (ie of the touch at the moment of shock). The ok flag is used to send this order only on each shock and not each time the plectrum and the string are in contact:

if ((bump <0) && (ok)) {

// order the synthesizer to play ReplySyntheOrder (ch, V * V); // Prevents sound bounce ok = 0; ;

// Reset the flag if the key is down enough if (bump> 100) ok = 1;

Simulation of a harpsichord The mechanism of the harpsichord is similar to that of the clavichord: the string is pinched by a point of feather held by a wooden blade (the grasshopper) secured to the key. The instrumentalist only feels the weight and inertia of the fingerboard as long as the feather does not come into contact with the string. Once this contact is established, the grasshopper bends until the feather emerges from the rope, releasing the key. On the return the same phenomenon occurs but the grasshopper folds more easily and the feather - beveled - is released faster. Eight parameters are required to model this:

- PARAMETER (O) is the weight of the key,

- PARAMETER (I) is the stiffness of the grasshopper when the button is lowered,

- PARAMETER (2) is the stiffness of the grasshopper when the button is raised, - PARAMETER (3) is the position of the rope,

- PARAMETER (4) is the maximum force before unhooking the grasshopper when the button is lowered,

- PARAMETER (δ) is the maximum force before the toggle hangs up when the button is raised, - PARAMETER (β) is the inertia of the mechanism,

- PARAMETER (7) is the viscosity of the virtual key / ground coupling.

The action of weight and inertia, modeled by the interaction with a virtual mass, is calculated as in the case of the clavichord. A single force is not associated with a given position of the key: it must be taken into account whether the grasshopper may be in contact with the rope or not. This requires the use of a state machine (note_info_ring [ch] .machinestate) which goes to 1 when the grasshopper has crossed the rope and returns to 0 when it is ironed below. The algorithm corresponding to the key / string interaction is therefore written:

// 1st case: grasshopper under the string if (note_info_ring [ch] .machinestate == 0) {if (Y <PARAMETER (3)) {// the grasshopper folds fcompress = (PARAMETER (3) -Y) * PARAMETER ( 1); if (fcompress> PARAMETER (4)) {// limit force reached note_info_ring [ch] .machinestate = 1; fcompress = 0 .;

// order the synthesizer to foil ReplySyntheOrder (ch, 1); ; force + = fcompress;

}} else {// 2nd case: grasshopper above the string if (Y> PARAMETER (3)) {

// the grasshopper folds fcompress = (Y-PARAMETER (3)) * PARAMETER (2); if (fcompress> PARAMETER (5)) {note_info_ring [ch] .machinestate = 0; fcompress = 0 .;

} force-≈fcompress;

};

The sound is emitted when the grasshopper releases the rope. Similarly when the feather comes back into contact with the string it is possible to insert an instruction ordering the synthesizer to muffle the sound with the change of timbre which accompanies it.

Simulation of a digital piano keyboard

The typical mechanics of a "digital" piano consists of a simple combination of weight, inertia, spring and viscous friction. It is therefore deduced from the examples previously described. In particular, no "hidden" variable describing the dynamics of an internal mechanism is necessary.

Simulation of an organ

The organ keyboard mechanics can be modeled as follows: the key, assigned a weight and inertia, is connected to a valve by through a spring with play and subjected to solid friction. The valve, for its part, has an inertia and is subjected to the action of the mechanism putting it in motion and of the suction caused by the air which penetrates into the pipe which it obstructs at rest. As in the case of the launched mass, it is necessary to calculate the kinematics of the valve and to know if it is pressed against its seat or not.

Piano simulation

The mechanics of the piano, although complex, can also be broken down into simple elements modeled by the algorithms presented above. Each key is subject to its weight, its inertia, friction on the exhaust felts and the reaction of the hammer it causes. It is obviously necessary to calculate the trajectory of the hammer, which will also make it possible to know when and with what vigor the sound must be emitted. The use of state machines is necessary to know if the hammer is in contact with the button, free or in the catch, and if it has been rearmed or not by the double escapement mechanism.

Special considerations related to enslavement problems

The above examples of instrument models do not take into account possible measurement imperfections of the sensors 11 and the response limits of the actuators 3, as well as mechanical imperfections of the keys 2. It is therefore necessary to carry out a treatment digital on the raw data coming from the sensors (filtering, linearization ...) before injecting them into the model. Likewise, in order to apply the exact force calculated by the model to the player's finger, it is necessary to take into account all of the disturbing effects of the electromechanical assembly (mass and inertia of key 2, friction , bending of the key 2, non-linearity of the actuator 3 including its control electronics ...). Due to the variety of mechanical models that can be used for touch simulation, it is not possible to guarantee a priori the stability of the entire mechanical system including key 2 and the finger interacting with it under the effect of the force exerted by the actuator 3. Indeed, even if the filtering strategies mentioned above could have forced the isolated system to stability, the complexity of the reaction of the player's finger on the touch (dynamic crushing of the pulp, stiffening of the tendons ...) can cause the appearance of parasitic oscillatory phenomena. The use of more sophisticated signal processing methods, such as adaptive filtering (detection and elimination by dynamic equalization of resonances, etc.) may prove necessary.

We understand that thanks to the method of the invention, we can simulate the touch of any existing instrument from the moment we know how to model its mechanical functioning. But we can also simulate keyboards with a radically new touch by programming a new algorithm for calculating touch.

According to a first application of the invention, the interface 8 allows the user to modify the parameters of the algorithms. It is for example possible to modulate the touch of the instrument. The user can request that the keyboard be heavier, lighter, faster. He can even request that each key has its own touch. For example, it can reduce the treble or harden the response of the keys to increase the effectiveness of an exercise. It can also command the computer 7 to modify the touch of certain keys at certain moments of the song only, for example to facilitate the execution of a perilous passage.

According to another application of the invention, the keyboard simulates the touch of a real instrument, for example a piano. The calculator calculates the movements of the various components of the mechanism. It calculates the exact moment when a hammer hits a string and with what energy the impact is made. The calculator is able to give play orders to a synthesizer to which it is connected. The issued playing orders can be stored in a memory or on a physical medium (hard disk, CD-ROM, etc.) so that the player can record his playing.

According to another application of the invention, the playing orders can also be sent to a traditional instrument whose mechanics would have been motorized. The instrumentalist plays on the keyboard, the orders are transmitted to another instrument which reproduces the instrumentalist's playing. Game orders can be sent as far as desired by the calculator via an information transmission network, which allows an instrumentalist playing in one place in the world to be listened to at the same instant, anywhere else and on any instrument.

Finally, if the computer is connected to an information transmission network, it is possible to download to the computer memory touch algorithms such as those described above. These algorithms can also be stored in media such as floppy disks or CD-ROMs.

Claims

1. Method for simulating touching on a keyboard (1) according to which:

- at least one movement parameter of each key (2) of the keyboard (1) is measured by means of a sensor (11),

- an effort to be exerted on the finger of the instrumentalist is calculated as a function of the parameters measured and a control quantity is deduced therefrom,

- an actuator (3) is controlled which applies the calculated force to the key (2), characterized in that said force is calculated according to a modeling of the dynamic mechanical behavior of the various elements of a mechanism of a keyboard instrument that we want to simulate, the force calculated using this modeling corresponding to the reaction force which would be exerted by the mechanism on the key (2).

2. A method of touch simulation on a keyboard (1) according to claim 1, characterized in that the position of the moving parts is continuously calculated in the mechanism which it simulates.

3. A method of simulating touch on a keyboard (1) according to claim 2, characterized in that one stores, during the actuation of a key (2), information on the past of the trajectory of the key ( 2) as well as on the moving parts of the mechanics to be simulated in local variables, and this information is taken into account to calculate the force to be exerted on the key (2).

4. A method of simulating touch on a keyboard (1) according to claim 2, characterized in that one determines at what instant the sound must be generated and in that one transmits playing orders to a synthesizer.

5. A method of simulating touch on a keyboard (1) according to claim 2, characterized in that one determines the position of the moving parts in the simulated mechanism and in that one transmits playing orders to an instrument whose the sound generation mechanism is triggered by actuators.

6. Touch simulation device on a keyboard (1) comprising

- a keyboard (1) with keys, - means (11) for measuring a movement parameter of a key

(2)

means (7) able to calculate by an algorithm an effort to be exerted as a function of the parameters measured and able to deduce therefrom a control value,

- an actuator (3) capable of exerting the force calculated on the key (2),

- Means (4) for controlling the actuator (3) to apply the force calculated on the key (2), characterized in that the calculation means (7) comprise storage means in which one or more are stored function (s) modeling dynamic mechanical behavior of the various elements of a mechanism of a keyboard instrument that we want to simulate, as well as means capable of calculating, using this modeling, a force value corresponding to the reaction force which would be exerted by the mechanism on the key (2).

7. Touch simulation device on a keyboard (1) according to claim 6 characterized in that the calculation means (7) include an interface (8) for programming one or more modeling function (s) or modifying the parameters of functions already programmed.

8. A touch simulation device on a keyboard (1) according to claims 6 and 7, characterized in that it further comprises an information storage device (hard disk, floppy drive, CD-ROM, DVD, tape, etc., local or networked) and in that this device makes it possible to store or read the algorithms for modeling touch or parameters relating to the functions already programmed and / or to store the parameters of the game of a instrumentalist measured by the sensors (11) as a function of time (in order to record his playing).

9. Touch simulation device on a keyboard (1) according to one of claims 6 to 8, characterized in that the calculation means (7) are connected to an information transmission network by which are transmitted or downloaded modeling functions and / or parameters relating to the functions already programmed, as well as possibly the parameters of the playing of an instrumentalist measured by the sensors (11) as a function of time in order, for example, to transmit his playing over time to a remote means of sound reproduction.

10. Touch simulation device on a keyboard (1) according to claim 6 to 9, characterized in that it comprises multiplexing means (10) which receive the measurements from measurement means (11) associated with a plurality of keys (2) and transmit them to an input of the calculation means (7) by managing the distribution over time of said measurements on said input.

11. Touch simulation device on a keyboard (1) according to one of claims 6 to 10, characterized in that it comprises demultiplexing means (5) which receive on an input a plurality of setpoint signals and distribute these setpoint signals on different actuators (3).