WO2002041129A2

WO2002041129A2 - 3d sensitive plate

Info

Publication number: WO2002041129A2
Application number: PCT/EP2001/013239
Authority: WO
Inventors: Janez Stare; Branislav Popovic; Franc Novak; Karel Rankel
Original assignee: Aventec Handels-, Produktions- Und Beratungsgesellschaft Mbh
Priority date: 2000-11-20
Filing date: 2001-11-15
Publication date: 2002-05-23
Also published as: SI20774A; WO2002041129A3; AU2002220709A1

Abstract

A 3D sensitive plate as an element of a universal controller for the devices controlled by means of a visible feedback is used for a three-dimensional data entry. The 3D sensitive plate detects the medium touch location in co-ordinates and the force of the medium pressure. The universal controller with a 3D sensitive plate according to the invention combines and upgrades the functions of a conventional keyboard, a pointing device, e.g. a mouse, graphic tablets, TV remote controls and remote controls for manipulators, i.e. robots, as an input, i.e. entry, unit for computers. A basis of the device is a basic sensitive plate (1) with scanning sensors (7, 8, 9) and a control unit (6). The scanning sensors (7, 8, 9) are arranged between the basic sensitive plate (1) and a pad (63). The device detects and recognises the medium touch location and the force of the medium pressure and thus guarantees the execution of various activities on the basic sensitive plate (1): commands with hand movements, three-dimensional positioning, typing, freehand drawing, writing and shorthand writing. At the same time, it provides feedback through the shuttle (5) on the screen (3).

Description

3D Sensitive Plate

This invention relates to a 3D sensitive plate as an element of a universal controller of devices controlled by a visible feedback. It is used for three- dimensional data entry. The 3D sensitive plate detects the location of a touch and the force of medium pressure in three co-ordinates. The universal controller with the 3D sensitive plate according to the invention combines and upgrades functions of a conventional keyboard, a pointing device, e.g. a mouse, graphic tablets, and remote controls for TV sets and remote controls for manipulators, i.e. robots, as an entry unit for computers.

From the known solutions partly or completely pursuing the same aim, the embodiment of the sensitive plate differs in its technology and in some cases also in its functions. The majority of the known solutions disclosed hereinafter in the description of the related art is limited to two-dimensional detection of the location and does not measure the pressure at the location of medium pressure.

Description of the Related Art

The known keyboard is based on the principle of a typing machine from the end of the 19th century, which has been supplemented to suit teletypewriters and, later, computers. Keyboards have been improved only in their physical appearance due to their ergonomic shortcomings. Labels indicated on keys very often do not correspond to the characters they are to indicate, particularly in the case of different languages or fonts. The keyboard is not suitable for direct entry of ideograms since the latter can be entered only by means of Latin syllables which are then translated into suitable ideograms. An increased need for multilingual communications and usage of different symbols reveales a shortcoming of a keyboard as far as fixed key labels are concerned.

Although pointing devices, e.g. touch-sensitive plates, trackpoints, trackballs and ^"similar, are physically mounted into keyboards of, for example, notepad computers, they substitute only for the mouse and offer no additional functions to users.

With the use of the mouse and the keyboard, it takes approximately one fourth of the total time required for pointing, typing and return of the hands into their initial position. The majority of users does not master ten-finger touch-typing and, consequently, constantly jump with their eyes from the keyboard to the screen.

The mouse permits only relative pointing. Consequently, a comparatively great number of co-ordinate movements of hands/fingers are required when moving the pointing device across the document, from the document to command buttons and back, and when choosing a command from a menu. The common relative pointing devices, e.g. a mouse, trackpads, are designed for two- dimensional relative positioning on the screen and fulfil requirements of work with complex graphic objects only to a minor extent. In the latter case special devices such as comparatively costly graphic tablets of different embodiments are required. They, however, permit neither sensing of forces and, consequently, depth pointing nor on-line changes of object parameters (graphic or textual) during entry; therefore, they require certain commands, e.g. those concerning line thickness, colours, work in layers etc. For the on-line entry of hand-written calligraphic texts, e.g. with Asian ideograms, Arabic characters, no suitable on-line input device is known.

Also in near future none of the following alternative technologies will prevail over the keyboard fitness for purpose. Some specific solutions are given hereinafter.

US 5,995,084 (Chan) discloses a sensitive pad sensing a movement of a handle across its surface. The patent focuses on the issue of processing of the signals containing information on the location, i.e. movement, of the handle. The sensitive pad is composed of several resistance layers. The location of a touch is determined by measurement of current and voltage respectively at the contact established between the resistance layers at the point of pressure.

US 5,917,476 (Czerniecki) discloses a combination of a sensitive plate and an alphanumeric field permitting a user the choice of characters wanted. The technology of the sensitive plate is not determined in detail. General solutions known from practice are adopted.

US 5,008,497 (Asher) discloses a sensor control of pressure applying a resistance membrane and an electronic circuit to accurately measure the location of pressure and the force of pressure applied to the sensor. The patent focuses on the electronic circuits of the controller combined with known devices for measurement of the location and the pressure at a touch point.

US 5,159,159 (Asher) discloses a pressure sensor sensing a two-dimensional location and measuring the pressure at a touch point. The sensor consists of two insulating substrates enclosing a resistance layer locally changing resistance in dependence of the pressure. The measurement of a voltage gradient in X and Y directions determines the point of pressure. The device sensor provides three output signals, i.e. location X, location Y, and the pressure at point (X, Y).

US 5,283,558 (Chan) discloses an application of a field of discrete switches and a resistance mesh for determination of location (X, Y). Upon pressure, two resistance segments proportional to the actual point of pressure are selected through switch contacts.

US 5,038,142 (Flower and others) discloses a device for determination of X and Y co-ordinates and a pressure in direction Z. The object to which pressure is applied is a display with springs clamped into a housing. Pressure (or elongation) sensors are mounted directly at the springs and connected to a sensor heatstone bridge.

US 4,587,378 (Moore) discloses a method of determination of X and Y coordinates by means of a device of which an essential part are two resistance layers and thereto appertaining conductive layers. Between the two extreme edges of the resistance layers, a power supply is connected. Pressure at a certain point produces a contact between the resistance layer and the appertaining conductive layer. X- and Y co-ordinates are determined by measuring voltage along both orthogonal conductive layers.

US 5,262,778 (Saunders) discloses a method and an embodiment for determination of the location of a touch (X and Y co-ordinates) and the pressure (Z co-ordinate) at point (X, Y). The location is determined by means of two conductive layers separated by an intermediate layer. Between a pair of contacts, current of a known intensity flows. Pressure produces a contact between the conductive layers. Based on voltage measured at the contacts, X, Y and Z co-ordinates will be calculated.

US 4,484,026 (Thomburg) discloses a multilayer pad consisting of two resistance layers separated by an intermediate layer. Voltage is connected to each resistance layer at two opposite edges in orthogonal directions. Pressure at a given point produces a contact between the two conductive layers. X and Y co-ordinates of the point of pressure are determined on the basis of measured voltage potentials. US 4,475,008 (Doi and others) discloses a combination of a resistance layer to which direct currents are applied alternately in orthogonal directions and of an intermediate insulating layer. Pressure of a probe applied at a certain point produces an electric contact with the resistance layer. Position X, Y is determined by measurement of co-ordinate voltages at the point of pressure (in a given time interval).

US 4,739,299 (Eventoff and others) discloses a device containing at least two pairs of contacts. Between the contacts in a pair, there is a resistance layer with a specified gradient profile. The output contact of the pair connects the electric conductive layer with the point of pressure of the resistance layer. A power supply is connected in series to the pairs of contacts and measures voltage at a relevant output contact. Based on measurements, co-ordinates of the point of pressure are determined.

US 4,897, 511 (Itaya) discloses an application of resistance layers to substrates separated by a gap and to which the power supply is connected. The point of pressure is indicated by the contact between the resistance layers. It is determined by measurement of a voltage drop.

According to the known solutions of entry devices, a simple and reliable way of the three-dimensional data entry is not solved in a satisfactory manner.

It is an object of the present invention to provide a design of a universal entry device ensuring a simple and reliable three-dimensional data entry. By means of the 3D sensitive plate according to the invention, a user positions, chooses and moves objects in three co-ordinates, wherein the objects are parts of a text, graphic elements, menus, and commands at a computer or TV screen. This makes it possible to enter (multilingual) texts by an on-line and simple setting of their parameters. In addition, with this device a user can draw, write and write in shorthand with a free hand and by on-line changes of line thickness. All operations with the device are carried out with fingers, but usual pencils or sticks of any stiff material can be used too. In case of a manipulator, a user controls the manipulator by means of the device in all three space dimensions.

This problem is solved according to the invention with the 3D sensitive plate used with devices controlled with a visible feedback _. in accordance with independent patent claims.

Summary of the invention

The 3D sensitive plate according to the invention for the devices controlled by means of a visible feedback (referred to as a "the controller" from here on) combines and upgrades the functions of a conventional keyboard, pointing device, e.g. a mouse, graphic tablets, TV remote controls and remote controls for manipulators i.e. robots. A basic element of the device is a sensitive plate 18 with control units in the plate 6 and in the controlled device 16. The device detects the location of medium contact and the force of the medium pressure at all three co-ordinates (x, y, z). The medium can be fingers, normal pencils or sticks made of any type of material, e.g. mouth sticks for handicapped users. With the device we can simultaneously make entries of a text composed of different character sets of different languages, including ideographic languages.

The said invention retains and thoroughly upgrades the existing input devices. It supports handwriting (including calligraphy and shorthand) and thus personalises communication between people in spite of greater and greater mechanisation of it.

The invention is explained in detail hereinafter with reference to preferred embodiments illustrated in the drawings, in which: Fig. 1 is a block scheme of the controller according to the invention,

Fig. 2 is an embodiment of the basic sensitive plate,

Fig. 3 is an embodiment of the controller of electronic circuits according to the invention,

Fig. 4 is a geometry for the calculation of the location of touch point.

Fig. 1 shows a block scheme of the controller 100 according to the invention, consisting of the sensitive plate 18, the control unit 16 in the controlled device, e.g. PC, TV, manipulator or robot, and a screen 3. The sensitive plate 18 consists of a basic sensitive plate 1 , scanning sensors 7, 8, 9, and the plate control unit 6. The control unit 16 in the controlled device consists of application interfaces 13 and a program for the analysis of the data provided by the sensitive plate 12. On the screen 3 there is a shuttle 5 reflecting the command semantics, and one or more pointers 4 showing the current position on the screen 3. The control unit 16 in the controlled device communicates with the operating system and applications 17 not making a part of the invention.

By pressing the medium to the basic sensitive plate 1 , the forces distribute among the scanning sensors 7, 8, 9. The scanning sensors 7, 8, 9 detect the size of the force. These scanning sensors, along with the sensitive plate they carry, are called the sensor trio. Theoretically, there can be as many sensor trios in the device as there are simultaneous controlling commands.

The plate control unit 6 scans all scanning sensors 7, 8, 9, processes their data, so that it can send, according to the current state of the device, the necessary information to the program for the analysis of the data provided by the sensitive plate 12 in the control unit 16. The programs for the analysis of the data provided by the sensitive plate 12, based on the data from the scanning sensors 7, 8, 9, determine the location and size of the force exerted by the medium touch. Thus ascertained data are then sent to the application interfaces 13, which then, according to the state of the device and the ascertained data, prepare and send the data obtained in standard logical and physical form to the applications 17.

The states of the device 100 are consistent with regard to the currently executed application 17 and the desired operations within this application. The state of the device 100 will be controlled and altered only by the program for the analysis of the data provided by the sensitive plate 12. The plate control unit 6 is also informed of the state of the device 100, so that it can optimally perform its functions. The transitions between these states are made by the program for the analysis of the data provided by the sensitive plate 12 upon the demand of the applications 17 or the user.

The application interfaces 13 control the shuttle 5, i.e. the manipulator, on the screen 3, i.e. in the visual field, because of the assurance of the visible feedback from the device to the user. The main function of the application interfaces 13 is to send the input data to the applications 17 in the form the applications 17 require. Thus it is also possible to use the device 100 for the given operating systems and applications.

The device according to the invention can have many different embodiments.

Fig. 2 shows the sensitive plate 18 without the control unit 6. The dimensions of the basic sensitive plate 1 can vary. The basic sensitive plates 1 can be moulded into arbitrary ergonomic forms. The sensitive plate 18 consists of the basic sensitive plate 1 with three noses 66 for force transmission, a pad 63 of the sensitive plate-18, a spring 64, two buses 65 and three sensors 7, 8, 9. The spring 64 presses the basic sensitive plate 1 against the sensors 7, 8, 9 and thus causes idle forces at the sensors 7, 8, 9. At the same time, it holds together the pad 63 and the sensitive plate 1. The pressure of the spring 64 and the idle operating point of the sensors can be adjusted with the screw 67. The elastic connection between the basic sensitive plate 1 and the pad 63 of the sensitive plate 18 can also be made of rubber or other elastic materials. It can be fixed at one point or at multiple points. The buses 65 take care of the exact position of the sensitive plate on the pad 63 and thus on the sensors 7, 8, 9. The idle forces at the sensors 7, 8, 9 put the sensors in the centre of the linear area of the sensor 7, 8, 9 operation, so the non-linearity of sensors 7, 8, 9 in the area when there is no pressure on the sensors can be avoided.

A basis for a quality sensitive plate is a quality pressure sensor. On each side of the sensor there are two resistors. The resistors are connected through a bridge connection. The bridge supplies the voltage proportional to the force exerted on the sensor. The voltage is obtained because of the difference in resistance at the top and bottom sides of the sensor.

Fig. 3 shows an embodiment of the electronic circuits of the controller according to the invention. Signals from the sensors 7, 8, 9 are directed to the operational amplifiers. The processor 56 starts up, via the parallel port 59, a time generator generating a steadily increasing voltage of a saw-tooth waveform. Voltage of the time generator is superimposed on the sensor voltages. When the bridge is in balance with the voltage of the time generator, the output polarity of the operational amplifier 50 changes. These signals are directed to the disconnect inputs 55 of the processor 56. The time elapsed between the beginning of the time base and the change of the polarity at the operational amplifier is proportional to the pressure applied to the sensor. The processor uses its program and internal counter to measure this time for each sensor separately. Thus the analogue/digital conversion of the size of the sensor forces is made. After the conversion, this cycle repeats. From the obtained digital value the data obtained in the idle state are subtracted. Thus the force caused by the spring 64 on each sensor can be eliminated and the asymmetry of the sensor 7, 8, 9 bridges compensated. The processor then converts the data obtained into the co-ordinates of the point of pressure and into the force at the point of pressure according to the following method. Fig. 4 gives a basis for the co-ordinate calculation. The sensors are symmetrically arranged around the gravity centre of the sensitive plate. The gravity centre of the sensitive plate is also the starting point of the co-ordinate system. With this geometry, the x and y co-ordinates of the medium touch on the basic sensitive plate are calculated according to the following formulas:

x=r(1/2)(F2+F3-2F1 )/(F1 +F2+F3) y=r(3^(1/2)/2)(F3-F2)/(F1 +F2+F3)

where r is the distance of the sensors from the gravity centre, F1 , F2 and F3 are the forces on the sensors 9, 8 and 7. The formulas are derived from the equations of equilibrium for moments, which are products of arms 80, 82, 87 by forces F1 , F2 and F3. Fig. 4 shows an embodiment of the position of the basic sensitive plate 1 relative to the sensors. The collective force, i.e. the z coordinate, is calculated according to the following formula:

z=F1+F2+F3

The sensitive plate 18 can include sensors made with other technologies, which, like the resistance ones, detect forces.

The sensitive plate 18 can be made of several parts, each of which has its own trio of scanning sensors, while they share the electronic circuits. It can theoretically be made of as many parts as there are simultaneously positioned objects needed. For example, instead of two mice (with some applications) the device with two trios of scanning sensors beneath the sensitive field can be used.

By designing the sensitive plate 18, a two-phase positioning of the pointer according to Fig. 4 is realised. Phase 1 of positioning is absolute positioning, where the position of the medium touch on the sensitive plate 18 is mapped in accordance with the position of the pointer 4 on the screen 3. If the ratio of the sides of the sensitive plate 8 is a1:b1 and that of the screen 3 is a2:b2, then the control unit in the PC 16 executes mapping from the position (x, y) at the sensitive plate 18 onto the position (u=x*a2/a1 , v=y*b2/b1) on the screen 3. Phase 2 of positioning is relative positioning, where the exact location of the pointer is set by dragging the medium across the sensitive plate 18. The direction of dragging is determined with the changing of co-ordinates (x, y) of the medium touch at the basic sensitive plate 1 , which are then mapped onto the changes of co-ordinates (u=x*a2/a1 , v=y*b2/b1) on the screen 3. The length of the pointer 4 movement from the position set during phase 1 of positioning is determined with the speed of dragging the medium across the basic sensitive plate 1 , that is movement=k*speed, where k can be set by the user at will, similar as with the mouse. In spite of the two-phase positioning principle, the relative positioning enables access from any point to any other point of the screen 3. Thus the advantages of the relative positioning have been retained and upgraded with the advantages of the absolute positioning, so that the necessary movement of the medium across the basic sensitive plate 1 is minimised. When the pointer 4 has been positioned at the desired position, it can be activated with a stronger pressure of the medium.

Beside the two-phase positioning, i.e. with co-ordinates (x, y), with the sensitive plate 18 the depth positioning, with which the pointer 4 is positioned onto different layers of objects, is also performed. The depth positioning corresponds to the change of force of the medium perpendicular to the basic sensitive plate 1 , i.e. co-ordinate z, after co-ordinates (x, y) have been determined with the two-phase positioning. The depth positioning enables access to lower layers of graphic objects, menu commands etc.

With a larger number of scanning sensor trios multiple-positioning, i.e. the setting of more simultaneous pointers 4, is realised. The example has been limited only to two-positioning because simultaneous control of more than two pointers is not ergonomically sound. While two pointers are often wished-for, they are difficult to control with two positioning devices, i.e. two mice.

Positioning can thus be absolute, relative, a sequence of absolute and relative, any combination of the two with two-positioning and a combination of any of these methods with the depth positioning. The relative positioning can be executed with tracing (as with touching the track-pad). From here on positioning can mean any of the above mentioned methods.

With the design of the sensitive plate 18 the character sets of not only alphabetic but also ideographic symbols and symbols of various languages are set. Beside national languages and writings, mathematical symbols, key words of a computer language and other symbols are also counted among languages. When a certain working language is chosen, the shuttle 5 displays characters, i.e. primitive characters, of the chosen working language. The shuttle 5 appears only when the key is gently touched. Depending on the user choice, it displays the entire keyboard layout or only the surrounding area of the pressed key, the size of which the user can set. The characters carried can be accessed under the primitive characters. The characters carried can dynamically appear in shuttle 5 in place of the key of the primitive character.

Thus the device does not need key labels, though the users can be provided with key labels appropriate for their primary language. The keys are dynamic, because they can represent any character, depending on the chosen working language, but they are marked only in the shuttle 5. The invention thus provides a dynamic keyboard which enables the users to constantly have at their disposal a keyboard suitable for the chosen working language. A key-word set of a computer language and names of variables in the program, as defined by the programmer, are also counted as a working language; each key of such a dynamic keyboard carries its own key word, a variable, mathematical and logical symbols. The chosen dynamic keyboard has its layout in the shuttle 5 on the screen 3.

As soon as the user presses the medium against the sensitive plate 18, in the case of the positioning state, one or two pointers appear on the screen 3, whereas in the case of the input state, shuttle 5 appears.

The sensitive plate 18 can be made in various sizes. Different size does not narrow its fitness for purpose, only the speed of use diminishes with the smaller size of the device. The user can set the size of shuttle 5 and with that proportionally the size of the characters by arbitrary graphic increasing/reducing the size of the shuttle 5 with a setting button for the shuttle 5 size.

The shuttle 5 and the dynamic keyboard enable the user to only watch the screen and eliminates the need to glance from keyboard to the screen and back. The dynamic keyboard and the shuttle 5 enable a replacement of the sensitive plate 18 and the screen 3 in certain cases with a single device, a touch-sensitive screen with the characteristics of the sensitive plate 18. The dynamic keyboard and the shuttle 5 also enable the use of monitor glasses instead of a computer monitor, because all the keys and reactions of the system are presented on the screen 3 on input, in this case at the monitor glasses. Between the actions on the sensitive plate 18 and the representations on the screen 3 there is a permanent bijective mapping.

Using the sensitive plate 18 does not require simultaneous key strokes as each character or command has its own location in the co-ordinate space (x, y) or (x, y, z) of the size of the character set in the working language. The same goes for the upper case and lower case letters: the upper case Latin letters - the capital letters - can be pressed without simultaneous pressing of the Shift key as with normal keyboards, merely with a stronger pressure on the key of the appropriate character.

The user has a constant control over the operations with the visible feedback and can immediately detect the mistakes made.

The device enables electronic signing and freehand drawing and writing.

The embodiments of the device can be of various sizes depending on the use: for a PC or a workstation, for a laptop, for a PDA or a communicator, for remote control for a PC/TV hybrid or for remote control of a manipulator or robot. The invention is useful in the case of the screen 3 on a CRT or LCD monitor. In the case of robot or manipulator control with the help of the visual or any other sensory feedback, the screen represents the working area of the manipulator or robot, which is in the visual field of the user. From here on the screen will be a generic term for a CRT or LCD monitor or for the visual field of the user. Based on the invention, a touch-sensitive screen can be made by mounting a specific type of the device on an LCD or CRT monitor.

Claims

Patent claims

1. A 3D sensitive plate as an element of a controller of devices controlled by means of visible feedback, characterised by having a sensitive plate (18), comprising a basic sensitive plate (1), at least three scanning sensors (7, 8, 9), and a pad (63), wherein the scanning sensors (7, 8, 9) are arranged between the basic sensitive plate (1) and the pad (63) being elastically connected to the basic sensitive plate (1 ).

2. The 3D sensitive plate according to claim 1 , characterised in that the basic sensitive plate (1) is connected to the pad (63) by a spring (64) so that the pressure is adjustable.

3. The 3D sensitive plate according to claim 1 , characterised in that the pressure location is calculated from the forces acting on the sensors (7, 8, 9).

4. A 3D sensitive plate, characterised in that

• a shuttle (5) appears on the screen (3) automatically when the user touches the 3D sensitive plate, and disappears from the screen (3) when the user-set time control between the touches of the 3D sensitive plate runs out or when the user requests it with a special hand motion,

• the user is optionally able to turn of the shuttle (5) appearance, wherein the appearance and the content of the shuttle (5) are dependent on the user activities on the 3D sensitive plate and on the application 17 currently being executed.