US20110102349A1 - Touch Sensitive Device - Google Patents
Touch Sensitive Device Download PDFInfo
- Publication number
- US20110102349A1 US20110102349A1 US12/921,977 US92197709A US2011102349A1 US 20110102349 A1 US20110102349 A1 US 20110102349A1 US 92197709 A US92197709 A US 92197709A US 2011102349 A1 US2011102349 A1 US 2011102349A1
- Authority
- US
- United States
- Prior art keywords
- vibration
- screen
- stylus
- writing
- paper
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03545—Pens or stylus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0412—Digitisers structurally integrated in a display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/043—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
- G06F3/0436—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
Definitions
- the invention relates to touch sensitive devices including touch sensitive screens or panels, in particular panels for use with a stylus or other writing instrument.
- U.S. Pat. No. 4,885,565, U.S. Pat. No. 5,638,060, U.S. Pat. No. 5,977,867, US2002/0075135 describe touch-operated apparatus having tactile feedback for a user when touched.
- an actuator is provided for imparting motion to the CRT when the actuator is energised to provide tactile feedback.
- U.S. Pat. No. 5,638,060 a voltage is applied to a piezo-electric element which form a switch to vibrate the element to apply a reaction force to a user's finger.
- a tactile feedback unit generates a mechanical vibration sensed by the user when the touch screen is touched with a finger or a pointer.
- the amplitude, vibration frequency and pulse length of the mechanical vibration are controlled, with the pulse width being long enough to be felt but short enough to terminate before the next key touch.
- US2002/0075135 describes the use of a second transducer to provide a pulse in the form of transient spike to simulate a button click.
- tactile feedback is provided in response to a discrete touch, of a user's finger or pointer.
- tactile feedback may also be useful for continuous movements across the touch screen.
- a method of simulating the sensation of a writing implement writing on paper when using a hand-held stylus to write on a touch-sensitive screen comprising arranging the screen to vibrate when contacted by the stylus to provide user feed-back.
- Paper comprises a fibre mat in a binder with individual fibres having a random orientation resulting in a rough surface having local variations in the coefficient of friction (sliding or static) on the level traced by the small contact patch between the tip of a pencil and the paper surface.
- the motion of a pencil over the surface may be described as stick-slip style motion.
- a similar effect is felt when writing with other writing instruments, such fibre tip pen.
- variations on the effect may be felt.
- a fountain pen this may be scratchy if a bad nib but have a smoothish, water lubricated glide if good.
- the type of paper also has an effect and often fountain pen users are selective about finding paper with the right nib feel and which takes ink at the right rate without bleeding. For biros, there is a viscous smooth friction sliding but the paper does have some underlying fibrous texture.
- the screen may be vibrated by applying a signal comprising multiple pulses or a stream of pulses.
- the method may comprise sensing the velocity of movement of the stylus over the screen face, for example, by using a sensor.
- the screen may be arranged to vibrate according to the sensed velocity.
- the signal may have a mean pulse rate comparable to that of the sensed velocity.
- the method may comprise arranging for the vibration to simulate the drag of writing implement moving over paper by period modulating the vibration, for example for a signal having multiple pulses by changing the spacing in time between pulses, i.e. by changing the pulse rate.
- the spacing may be randomised whereby the random spacing of the paper fibres is simulated.
- the spacing of the pulses may in the range of 1 ⁇ 5 to 4 ⁇ 5 of twice the mean inter-fibre spacing.
- the vibration may simulate the axial reciprocating movement of writing implement moving over paper by amplitude modulating the vibration.
- the amplitude may be randomised whereby the random height of the paper fibres is simulated.
- the amplitude may be defined by the scale factor which is the tactile equivalent of the volume control in audio.
- the scale factor may adjusted by the user to give a suitable level of stimulation.
- the amplitude may be in the range of 3 ⁇ 8 to 7 ⁇ 8 of the scale factor.
- Two connected random sequences may thus be applied to generate the vibration, a first sequence to account for inter-fibre spacing and a second to account for the height of the fibres. Together the random sequences may simulate a synthetic paper structure for the touch screen.
- a pencil writing on paper may also have its own resonances which contribute to the feel of the writing sensation. Accordingly, the stylus may be configured so that it is excited into beam resonance in response to vibration of the screen whereby the vibration simulates resonance of writing implement moving over paper.
- the screen may be excited to produce a sound effect simulating that of writing implement writing on paper.
- the vibration may include any type of vibration, including bending wave vibration, more specifically resonant bending wave vibration.
- apparatus comprising a touch sensitive screen having a face adapted to receive and record a user's hand-writing via a hand-held stylus, wherein the screen comprises a vibration exciter exciting the screen to vibrate so as to transmit the vibration to the stylus to simulate the sensation of a writing implement writing on paper as the stylus is moved over the face of the screen.
- the vibration exciter may comprise means for applying a bending wave vibration to the screen face.
- the vibration exciter may be electro-mechanical and may comprise signal generating means for applying an electrical signal to the vibration exciter to cause the exciter to vibrate the screen.
- the signal generating means may comprise means generating a signal comprising multiple pulses, e.g. phase locked loop module generating a stream of pulses having a mean pulse rate.
- the apparatus may comprise means for sensing the velocity of movement of the stylus over the screen face.
- the means for modulating the vibration may be configured to modulate the vibration according to the sensed velocity, for example, the means generating the pulsed signed may be configured to adjust the mean pulse rate to match the sensed velocity.
- the apparatus may comprise means for period modulating and/or means for amplitude modulating the electrical signal.
- the period and/or amplitude modulation may be random and may be applied by a jitter module.
- the signal generating means may generate a signal to cause the screen to radiate an acoustic component simulating that of a writing implement writing on paper, the acoustic signal being modulated by the velocity sensing means.
- the signal generating means may further comprise a filter to reduce high-frequency content. In this way, a realistic feel may be provided with as little noise as possible.
- the vibration exciter may be a moving coil transducer or a piezoelectric bending transducer, for example one comprising a resonant element as described e.g. in WO01/54450, incorporated herein by reference.
- the exciter may be inertial
- the touch screen may be a panel-form member which is a bending wave device, for example, a resonant bending wave device.
- the touch screen may also be a loudspeaker wherein a second vibration exciter excites vibration which produces an acoustic output.
- the touch screen may be a resonant bending wave mode loudspeaker as described in International Patent Application WO97/09842 which is incorporated by reference.
- the invention further provides processor control code to implement the above-described methods, in particular on a data carrier such as a disk, CD- or DVD-ROM, programmed memory such as read-only memory (Firmware), or on a data carrier such as an optical or electrical signal carrier.
- Code (and/or data) to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (Trade Mark) or VHDL (Very high speed integrated circuit Hardware Description Language).
- a data carrier such as a disk, CD- or DVD-ROM, programmed memory such as read-only memory (Firmware), or on a data carrier such as an optical or electrical signal carrier.
- Code (and/or data) to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language
- FIG. 1 a is a plan view of a touch sensitive screen
- FIG. 1 b is a block diagram of the system for use with the touch sensitive screen of FIG. 1 a;
- FIG. 2 is a flow chart showing the interaction of the various components in the system of FIG. 1 b;
- FIG. 3 a shows the impulse responses of four signals which may be generated by the system of FIG. 1 b;
- FIG. 3 b shows the smoothed frequency spectra of the signals of FIG. 3 a
- FIG. 4 shows a 2-D model of pencil on paper
- FIG. 5 a shows the trace history of the reactions at a hand holding the pencil in FIG. 4 during FE simulation
- FIG. 5 b shows the trace history of the pencil tip in FIG. 4 during FE simulation
- FIG. 6 a plots the surface fibre distribution in time (tc) and amplitude (h) for a randomised surface structure
- FIG. 6 b is a smoothed version of the sampling spectrum for the structure of FIG. 6 a;
- FIG. 6 c shows the smoothed acceleration and force spectrum for the structure of FIG. 6 a measured at 44.1 kHz;
- FIG. 7 a plots the modified haptic click signal against time
- FIG. 7 b plots the spectrum of the modified haptic signal of FIG. 7 a and the target spectrum
- FIG. 8 a plots an alternative modified haptic click signal against time
- FIG. 8 b plots the spectrum of the modified haptic signal of FIG. 8 a and the target spectrum
- FIG. 9 a plots the scaled variation in amplitude over time for four types of signals.
- FIG. 9 b plots the frequency spectra of the generated randomised tactile signals corresponding to each of the impulse signals of FIG. 9 a.
- FIG. 1 a shows a touch sensitive device 10 comprising a touch sensitive screen 12 on which a stylus 18 or pencil or similar writing instrument is being used to write text 20 .
- One or more sensors 17 are used to detect a touch or movement of the stylus on the screen and an exciter 16 is provided to generate a signal within the screen.
- the slip stick behaviour of pencil on paper is a key element of the writing sensation. Writing to touch sensitive surfaces with a stylus has improved sensation, appeal and user satisfaction when there is simulation of the pencil on paper writing characteristic.
- FIG. 1 b shows how the touch sensitive device 10 may be adapted to use haptic methods and mechanical feedback technology to create such a simulation.
- the writing surface is mechanically energised under the pre-programmed control of the writing input from the stylus to simulate the feel of pencil on paper.
- a model including the inherent mechanical behaviour of a pencil may be incorporated into the system but this would not be recognised or understood by a user who simply feels the result of the simulation.
- the touch sensitive screen 12 is connected to a sensor 16 which detects the nature of the contact from the stylus.
- the sensor is connected to PLL (phase locked loop) 24 which is one of the key elements used to generate the algorithm to generate the desired sensation of pencil on paper.
- PLL phase locked loop
- the elements of the algorithm may be implemented in hardware or software. The individual elements have the following description:
- PLL module 24 Phase-Locked Provides a stream of pulses having a mean impulse Loop rate locked to the speed of writing as determined by software from the touch screen.
- Jitter module 26 Pulse Randomly modifies the regular pulses into pulses randomiser of varying amplitude and separation according to statistical rules
- Fs 32 Audio sample Generates clock for audio samples (texture) rate
- Impulse FIR filter Impulse response that is triggered by the jittered generator 28 pulses Filter 30 Optional post- Reduces high-frequency content to make quieter filter
- the PLL and Jitter blocks 24 , 26 run at the relatively slow rate set by the resulting pulse train. This should be below 150 Hz average rate, but the resolution of the jitter should be closer to the audio rate.
- the audio rate, Fs is set as appropriate for the signal bandwidth required, but will almost certainly be no more than 11025 Hz (i.e. 1 ⁇ 4 of the standard CD audio rate). Where multiple audio channels are used, these signals may be multiplexed in a single channel. Thus where four audio channels are used, and instead of having four channels at 11025 Hz, it would be possible to have one audio channel of 44100 Hz, which would be time-domain multiplexed (TDM) into four haptic channels.
- TDM time-domain multiplexed
- FIG. 2 is a flow chart showing the steps implemented by each block in the system.
- the sensor which may be implemented as software on the touch sensitive device, is monitoring the touch sensitive screen.
- a “touch” is detected as at step S 10 , it requests a “haptic click” (step S 12 ) from the signal generator or impulse generator 28 and the impulse generator generates a pulse which provides a “click” sensation at step S 28 .
- Screen surfaces are generally softish and quite well damped. Thus the stylus impact is rather quiet.
- the “click” may or may not provide an audible feedback depending on the proposed use for the touch screen.
- the sensor monitors the changes in position and calculates the drag rate, or velocity as at step S 16 .
- This velocity data is then fed to the PLL module 24 which produces a stream of pulses.
- the sensor detects no touch as at step S 30 and sends instructions to the PLL to stop (step S 32 ).
- step S 34 the PLL stops generating any commands.
- the pulses used for the haptic click sensation may be the same as the pulses which form the basis of the writing simulation but are not necessarily the same. For example, in the suggested implementation below, they are different.
- the spectra of the different signals are all chosen to match the sensitivity of the finger-tips to vibration.
- the PLL module 24 The function of the PLL module 24 is described as follows.
- the PLL module On receipt of velocity information from the sensor, the PLL module provides a steady stream of pulses which act as start commands to the Jitter module (step S 20 ).
- the PLL module measures the mean error between the rate of this stream (the actual rate) and the incoming velocity estimates (the target rate) and adjusts the actual rate to match the target.
- the PLL module should provide memory and some filtering, so that in the presence of noisy or missing estimates there is still a regular output.
- Inputs Start/Stop, Velocity (target rate)
- the function of the Jitter module 26 which provides the synthetic paper structure to the texture, is described as follows.
- the Jitter module 26 assigns a random amplitude to the pulse, and then at step S 22 , delays the pulse for a random duration before passing it on to the signal or impulse generator 28 at step S 24 .
- the statistics of the amplitude distribution are unconstrained by the pulse rate, but the average delay should be 1 ⁇ 2 the inter-pulse spacing, which means that some knowledge of the pulse rate is required.
- the Beta distribution may be used for the random signals. This sets the correct mean and variance for both jitter and amplitude data. The values chosen “by inspection” seem close to optimal—other values tested produce less realistic sensations. Generating Beta statistics by software or hardware may be difficult, so pragmatically it is suggested to use a simple uniform distribution (i.e. rectangular distribution) of the same mean and variance.
- the rectangular distribution is by far the simplest to generate and is as good as, or almost as good as the most complicated version
- PRBS Physical Random Bit Sequence
- pseudo-random bit sequence This is produced by a shift-register with feed-back occurring on certain bit patterns, or masks.
- the choice of mask affects the repeat length of the sequence and the “whiteness” of the noise.
- a standard method for generating a specified statistical distribution from the uniform distribution is the so-called “Inverse transform sampling” method (see. It maps noise samples from a uniform distribution on (0, 1] into samples having the specified statistical distribution. For this method, it is necessary to know the inverse of the cumulative density function.
- PDF probability density function
- CDF cumulative density function
- the closest approximation to the Beta distribution is the Kumaraswamy distribution. It is controlled by two variables, a and b (real and positive), and is bound on [0,1] with
- Each of the three distributions highlighted above may be used to generate randomness to simulate texture.
- Each sample requires two sets of random data—the spacing and the amplitude. In principle, these two sets could be completely independent, but testing suggests that they should be generated from the same uniform distribution. (This does make some sense, as a long gap will be associated with a large amplitude and a small gap with a small amplitude, thus making the signal energy more uniform).
- the impulse generator 28 which provides spectral properties of the texture, is described as follows. On receipt of a signal from the jitter module 26 , the impulse generator 28 outputs a signal in the form of a stream of sample values (step S 26 ). If another input is received before the stream is complete, then a new stream begins. The amplitude of the output signal is modified according to data received from the jitter module 26 .
- the impulse generator comprises a FIR (finite impulse response) filter to match the haptic signal output from the impulse generator 28 to the sensitivity of the fingers to touch.
- the filter may reduce the high-frequency content to a reasonable level.
- the optional filter 30 may be provided to reduce any remaining high-frequency breakthrough from the impulse generator. It is envisaged that this will be a very simple recursive, 1 st order stage filter with coefficients chosen to avoid multiplication; e.g. 2 ⁇ ( ⁇ n), 1-2 ⁇ ( ⁇ n). If this proves insufficient, a 2 nd order filter could be used instead.
- the cut-off frequency would be around, say, 500 Hz-600 Hz.
- FIG. 3 a shows the impulse responses of four signals from the impulse generator incorporating a filter, each with a cut-off at or near 300 Hz.
- Filters universally have integer order roll-off; for example the R-C network of electronics has a first-order response, while the L-C-R network may have a second-order response.
- An n-th order roll-off on a bode-plot is represented by a slope of 6 ⁇ n dB per octave or 20 ⁇ n dB per decade.
- the type 1 signal shown in FIG. 3 a is unipolar and it was found that a rate of approximately 2.25 to 2.5 gave the best feel/sound.
- Such a signal may be generated by using a cascaded first-order roll-off filter with unity gain pass-band. Filters of this type have the transfer function
- the impulse response is unipolar, and may be considered as a generalisation of the exponential decay.
- the type 2 signal shown in FIG. 3 a is bipolar and it was found that a rate of approximately 1.75 to 2.0 gave the best feel/sound.
- Such a signal may be generated by using a cascade second-order roll-off filter with unity gain pass-band. Filters of this type have the transfer function
- the roll-off rate is 2p-th order, i.e. 12p dB/octave. Again, we wish to generalise this to the case when p is not an integer.
- J p is a Bessel function of order p.
- the impulse response is bipolar, and may be considered as a generalisation of the damped sinusoid.
- the time-domain response is simply a damped, zeroth order Bessel function.
- the trigonometric approximation may be used (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 9.2.1), i.e.
- type 1 nor type 2 signals have the colouration in sound produced by a pencil.
- the type 6 signal was produced by convolving the type 1 signal with a similar signal of higher frequency. In this case the convolution has a closed form.
- the “Type 8” signal was produced by directly convolving the Type 2 signal with a similar signal of higher frequency, but lower Q.
- the spectra (i.e. sound output against frequency) of the corresponding texture signals for a mean pulse rate of 72.6 Hz are shown in FIG. 3 b .
- the modified spectra diverge from the original spectra from about 800 Hz, and the signals are noticeably quieter in “silent mode”.
- An alternative signal (type 3) may be generated using a filter having cascaded pairs of first-order sections with two, non-equal turning points. Filters of this type have the transfer function
- h ⁇ ( t ) a p ⁇ b p ⁇ ⁇ ( p ) ⁇ ⁇ ⁇ ( t a + b ) p - 1 2 ⁇ ⁇ - a + b 2 ⁇ t ⁇ I p - 1 2 ( a - b 2 ⁇ t )
- I p is a modified Bessel function (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 29.3.50)
- the target pulse frequency of the PLL module is equal to drag velocity/inter-fibre spacing.
- This equation was derived by generating a simple 2-D model of the paper surface and pencil interaction to understand the fundamental process. Paper is typically manufactured using a “web” of cellulose fibres of 2-5 mm length and a fine clay coating (particles ⁇ 0.1 um). The resulting surface has ridges at many distance scales, but the major ones are in the range 0.1 mm to 0.5 mm, depending on the paper.
- Pencils are typically manufactured from a soft-wood surround (Larch or Cedar) around a “lead”.
- the lead is actually a ceramic formed as a co-fired mixture of graphite and clay, which is usually dipped in a polymer.
- the typical “sound” of a pencil is a function of the hardness of the lead and the beam resonances of the shaft.
- a typical series of modes might be, say; 350 Hz, 900 Hz, 2.0 kHz, 3.3 kHz, etc. The values will, of course, depend on the length of the pencil.
- the mechanism may be described as the stick-slip motion of the pencil over the rough paper surface, with the sound being modified by the resonances of the pencil.
- FIG. 4 shows the simple 2-D model of the paper surface and pencil interaction in which the paper is modelled with semi-circular ridges of equal height spaced at a regular 0.15 mm pitch.
- the pencil tip, and a short section of the shaft are modelled explicitly, and the hand-arm system is modelled by lumped parameters.
- the simulation occurs in two phases, each lasting 1 second.
- the pencil In the first phase, the pencil in lowered onto the paper surface, and a writing force on 1 N is applied.
- the pencil In the second phase, the pencil is dragged at 1 mm/s along the paper surface.
- the resulting forces on the hand and motions of the pencil tip may be seen in FIG. 5 a and FIG. 5 b respectively.
- the “cogging” seen in both sets of traces is in part due to the discrete nature of the model—the nodes. In real life, other texture details would be likely to cause similar effects.
- the precise wave shape is set by the degree of control applied to the pencil.
- the velocity at the hand is constant, and the tip follows appropriately.
- the opposite extreme would be to apply a constant force sufficient to overcome the average dynamic friction. In this case, the velocity would be non-uniform.
- FIG. 6 a shows a more realistic model of the paper surface.
- Statistical distributions of fibre distances and heights have been used to generate the depicted randomised surface structure.
- the beta distribution was chosen to generate the model for two main reasons; it is bounded on [0,1] (unlike the boundless normal distribution), and with two independent parameters it is possible to control two of the main statistical parameters (the mean, the mode, and the variance).
- the exact parameters of the statistical distributions are completely arbitrary.
- a cut-off frequency (fc) is calculated.
- a smoothed version of the spectrum generated by the model of FIG. 5 a is seen in FIG. 5 b , where fc is seen to feature strongly.
- the sampling data is convolved with a leaky integrator.
- the time-constant corresponded to 40 rad/s, but this merely controls the amount of very low frequency information in the resulting signal.
- the signal is played over the laptop loudspeaker, it sounds like a finger-nail being dragged over paper.
- FIG. 6 c shows the force spectra for measurements obtained from a custom writing tool.
- the tool has interchangeable tips, and is fitted with a force gauge and an accelerometer, both connected to a charge amplifier (ENDEVCO Model 133). Neither gauges are fully calibrated, but the force gauge sensitivity is known to be close to 1 V/N.
- Data was acquired via a NI PCI-4452 data acquisition card.
- the spectrum of the measured signal is strongly affected by the speed of writing.
- the essentially low-pass spectra has cut-off frequencies that are directly proportional to the speed of writing.
- the bandwidth was roughly established by tracing over graph paper and using a stop watch to be about 50-60 Hz at 1 inch/sec writing rate for normal paper. The bandwidth was different for different surfaces.
- the writing speed is about 2 in/s and the data is sampled at 44.1 kHz.
- the force results measure the forces applied between the pencil tip and the shaft of the writing tool.
- the acceleration results measure the resulting motion of the pencil. From these measurements, it is possible to derive the effective impedance of the system by using the relation
- the effective impedance is like the combination of a 0.4 kg mass and a 100 Ns/m dashpot. This is, in effect, acting as a lossy integrator with a corner frequency of about 40 rad/s.
- the measured force spectra of FIG. 6 c corresponds to the simulated spectra of FIG. 6 b .
- the signal of FIG. 6 b is then filtered to apply boosts at frequencies corresponding to modes in a pencil.
- the new signal spectrum is reminiscent of the accelerometer spectra seen in FIG. 6 c , which has some resonances in the acceleration trace due to modes in the writing tool.
- the signal is played over the laptop loudspeaker, it sounds much more like a pencil being dragged over paper.
- a “click” signal is requested.
- One such signal is a frequency and amplitude modulated cosine function, i.e.
- h ⁇ ( t ) ⁇ ⁇ ⁇ t ⁇ ⁇ ⁇ 1 - ⁇ ⁇ ⁇ t ⁇ cos ⁇ ( ⁇ ⁇ ⁇ ct 1 + ⁇ ⁇ ⁇ t )
- h(t) is the product of g(t)—the envelope function and fm(t)—a frequency modulating function.
- This signal may also be used as the “type 3” signal mentioned above which is used to generate handwriting texture when a drag is detected.
- This cosine function signal was found to be more effective than its sine-based counterpart and further improvements are investigated below by adding a new variable ⁇ was added to the function. This amended function is then optimally fitted to the target haptic spectrum.
- h ⁇ ( t ) ⁇ ⁇ ⁇ t ⁇ ⁇ ⁇ 1 - ⁇ ⁇ ⁇ t ⁇ cos ⁇ ( ⁇ ⁇ ⁇ ct 1 + ⁇ ⁇ ⁇ t - ⁇ )
- h ⁇ ( t ) ⁇ ⁇ ⁇ t ⁇ ⁇ 1 - ⁇ ⁇ ⁇ t ⁇ cos ⁇ ( ⁇ ⁇ ⁇ ct 1 + ⁇ ⁇ ⁇ t - ⁇ ⁇ ⁇ c ⁇ + ⁇ )
- h(t) is the product of g(t)—the envelope function and fm(t)—a frequency modulating function but in this case
- fm ⁇ ( t ) cos ⁇ ( ⁇ ⁇ ⁇ ct 1 + ⁇ ⁇ ⁇ t - ⁇ ⁇ ⁇ c ⁇ + ⁇ ) .
- the parameters are chosen to match a spectral template which shows the relative sensitivity of the finger-tips to vibration as a function of the vibration frequency.
- the aim is to put the most energy in the frequency range at which the fingers are most sensitive.
- FIG. 7 a shows the variation in time for the envelope function g(t) and the frequency modulation function fm(t) of the signal.
- FIG. 7 a also shows how ⁇ is derived. The timing of the first non-zero point of intersection is equivalent to 1/ ⁇ .
- FIG. 7 b shows the target spectrum (dotted line) which gives the desired sensation to a user and the actual spectrum of the modified function detailed above. There is a good match between the two spectra. Other values of the parameters, or even other signals, may be used to achieve similar aims.
- the signal of FIGS. 8 a and 8 b is just such a signal—its parameter values are also chosen to match a target spectrum.
- FIGS. 8 a and 8 b illustrate an alternative signal having its basis starting in the frequency domain and expressed as.
- FIG. 8 a shows that the sensitivity curve (fm(t)) resembles a normal distribution curve. It is also known that this curve (fm(t)) is its own Fourier transform, so the time domain signal should be similar.
- FIG. 8 a also shows how ⁇ and ⁇ c are calculated. ⁇ and ⁇ c are calculated as described above. As shown in FIG. 8 b , this alternative does not provide as good a fit to the target spectrum as the signal of FIG. 7 a but has the main advantage that the high-frequency end of its spectrum falls faster.
- the alternative signal appears to be 25% more energy efficient. However, some testing has shown that 15%-20% higher amplitude is needed to get the same sensation, thereby eliminating the advantage. There is less high-frequency energy in the alternative signal, which may well help with making it more silent. In short, there is not much to choose between them.
- the alternative signal is illustrated in FIGS. 8 a and 8 b as “type 9” signal.
- FIGS. 9 a and 9 b compare four signals for effectiveness as texture waveforms. The signals are adjusted in amplitude to give the same degree of sensation but use different amounts of power to achieve the sensation.
- FIGS. 7a and 7b 6 Unipolar, dual slope generalised impulse, from type 1 - 0.406 see FIG. 3a 8 Bipolar, dual slope generalised impulse, from type 2 - 0.201 see FIG. 3a 9 Alternative haptic click of FIGS. 8a and 8b 0.237
- the type 8 signal is the most energy efficient, with the new haptic click (type 9) coming a close second.
- Type 6 is the least efficient.
- Type 9 is the quietest in “silent mode” in the absence of additional filtering.
- the signal type currently suggested (type 8) is a minor adjustment to the type 2 signal previously chosen. It is still the most efficient, although not by much. It is worth comparing it to the new type 9 signal.
Abstract
Description
- The invention relates to touch sensitive devices including touch sensitive screens or panels, in particular panels for use with a stylus or other writing instrument.
- U.S. Pat. No. 4,885,565, U.S. Pat. No. 5,638,060, U.S. Pat. No. 5,977,867, US2002/0075135 describe touch-operated apparatus having tactile feedback for a user when touched. In U.S. Pat. No. 4,885,565 an actuator is provided for imparting motion to the CRT when the actuator is energised to provide tactile feedback. In U.S. Pat. No. 5,638,060, a voltage is applied to a piezo-electric element which form a switch to vibrate the element to apply a reaction force to a user's finger. In U.S. Pat. No. 5,977,867, a tactile feedback unit generates a mechanical vibration sensed by the user when the touch screen is touched with a finger or a pointer. The amplitude, vibration frequency and pulse length of the mechanical vibration are controlled, with the pulse width being long enough to be felt but short enough to terminate before the next key touch. US2002/0075135 describes the use of a second transducer to provide a pulse in the form of transient spike to simulate a button click.
- In each of the prior art documents described above, tactile feedback is provided in response to a discrete touch, of a user's finger or pointer. However, it is recognised by the applicant that tactile feedback may also be useful for continuous movements across the touch screen.
- According to the invention, there is provided a method of simulating the sensation of a writing implement writing on paper when using a hand-held stylus to write on a touch-sensitive screen, comprising arranging the screen to vibrate when contacted by the stylus to provide user feed-back.
- Paper comprises a fibre mat in a binder with individual fibres having a random orientation resulting in a rough surface having local variations in the coefficient of friction (sliding or static) on the level traced by the small contact patch between the tip of a pencil and the paper surface. The motion of a pencil over the surface may be described as stick-slip style motion. A similar effect is felt when writing with other writing instruments, such fibre tip pen. For other types of writing instruments, variations on the effect may be felt. For example for a fountain pen this may be scratchy if a bad nib but have a smoothish, water lubricated glide if good. The type of paper also has an effect and often fountain pen users are selective about finding paper with the right nib feel and which takes ink at the right rate without bleeding. For biros, there is a viscous smooth friction sliding but the paper does have some underlying fibrous texture.
- In contrast, when writing with a stylus or pointer on a polymer or glass cover of a touch sensitive panel or surface, this slip-stick motion is lost. The slip stick behaviour of pencil on paper is a key element of the writing sensation. Writing to touch sensitive screens with a stylus has improved sensation, appeal and user satisfaction when there is a simulation of the pencil on paper writing characteristic.
- The screen may be vibrated by applying a signal comprising multiple pulses or a stream of pulses.
- The method may comprise sensing the velocity of movement of the stylus over the screen face, for example, by using a sensor. The screen may be arranged to vibrate according to the sensed velocity. Thus for a signal having multiple pulses, the signal may have a mean pulse rate comparable to that of the sensed velocity.
- The method may comprise arranging for the vibration to simulate the drag of writing implement moving over paper by period modulating the vibration, for example for a signal having multiple pulses by changing the spacing in time between pulses, i.e. by changing the pulse rate. The spacing may be randomised whereby the random spacing of the paper fibres is simulated. The spacing of the pulses may in the range of ⅕ to ⅘ of twice the mean inter-fibre spacing.
- The vibration may simulate the axial reciprocating movement of writing implement moving over paper by amplitude modulating the vibration. The amplitude may be randomised whereby the random height of the paper fibres is simulated. The amplitude may be defined by the scale factor which is the tactile equivalent of the volume control in audio. The scale factor may adjusted by the user to give a suitable level of stimulation. The amplitude may be in the range of ⅜ to ⅞ of the scale factor.
- Two connected random sequences may thus be applied to generate the vibration, a first sequence to account for inter-fibre spacing and a second to account for the height of the fibres. Together the random sequences may simulate a synthetic paper structure for the touch screen.
- A pencil writing on paper may also have its own resonances which contribute to the feel of the writing sensation. Accordingly, the stylus may be configured so that it is excited into beam resonance in response to vibration of the screen whereby the vibration simulates resonance of writing implement moving over paper.
- The screen may be excited to produce a sound effect simulating that of writing implement writing on paper.
- The vibration may include any type of vibration, including bending wave vibration, more specifically resonant bending wave vibration.
- According to another aspect of the invention, there is provided apparatus comprising a touch sensitive screen having a face adapted to receive and record a user's hand-writing via a hand-held stylus, wherein the screen comprises a vibration exciter exciting the screen to vibrate so as to transmit the vibration to the stylus to simulate the sensation of a writing implement writing on paper as the stylus is moved over the face of the screen.
- The vibration exciter may comprise means for applying a bending wave vibration to the screen face. The vibration exciter may be electro-mechanical and may comprise signal generating means for applying an electrical signal to the vibration exciter to cause the exciter to vibrate the screen.
- The signal generating means may comprise means generating a signal comprising multiple pulses, e.g. phase locked loop module generating a stream of pulses having a mean pulse rate. The apparatus may comprise means for sensing the velocity of movement of the stylus over the screen face. The means for modulating the vibration may be configured to modulate the vibration according to the sensed velocity, for example, the means generating the pulsed signed may be configured to adjust the mean pulse rate to match the sensed velocity.
- The apparatus may comprise means for period modulating and/or means for amplitude modulating the electrical signal. The period and/or amplitude modulation may be random and may be applied by a jitter module.
- The signal generating means may generate a signal to cause the screen to radiate an acoustic component simulating that of a writing implement writing on paper, the acoustic signal being modulated by the velocity sensing means.
- The signal generating means may further comprise a filter to reduce high-frequency content. In this way, a realistic feel may be provided with as little noise as possible.
- The vibration exciter may be a moving coil transducer or a piezoelectric bending transducer, for example one comprising a resonant element as described e.g. in WO01/54450, incorporated herein by reference. The exciter may be inertial
- The touch screen may be a panel-form member which is a bending wave device, for example, a resonant bending wave device. The touch screen may also be a loudspeaker wherein a second vibration exciter excites vibration which produces an acoustic output. For example, the touch screen may be a resonant bending wave mode loudspeaker as described in International Patent Application WO97/09842 which is incorporated by reference.
- Contact by the stylus on the screen may be detected and/or tracked as described in International patent applications WO 01/48684, WO 03/005292 and/or WO 04/053781 to the present applicant. These International patent applications are hereincorporated by reference. Alternatively, other known methods may be used to receive and record or sense such contacts.
- The invention further provides processor control code to implement the above-described methods, in particular on a data carrier such as a disk, CD- or DVD-ROM, programmed memory such as read-only memory (Firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and/or data) to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (Trade Mark) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate such code and/or data may be distributed between a plurality of coupled components in communication with one another.
- The invention is diagrammatically illustrated, by way of example, in the accompanying drawings in which:—
-
FIG. 1 a is a plan view of a touch sensitive screen; -
FIG. 1 b is a block diagram of the system for use with the touch sensitive screen ofFIG. 1 a; -
FIG. 2 is a flow chart showing the interaction of the various components in the system ofFIG. 1 b; -
FIG. 3 a shows the impulse responses of four signals which may be generated by the system ofFIG. 1 b; -
FIG. 3 b shows the smoothed frequency spectra of the signals ofFIG. 3 a; -
FIG. 4 shows a 2-D model of pencil on paper; -
FIG. 5 a shows the trace history of the reactions at a hand holding the pencil inFIG. 4 during FE simulation; -
FIG. 5 b shows the trace history of the pencil tip inFIG. 4 during FE simulation; -
FIG. 6 a plots the surface fibre distribution in time (tc) and amplitude (h) for a randomised surface structure; -
FIG. 6 b is a smoothed version of the sampling spectrum for the structure ofFIG. 6 a; -
FIG. 6 c shows the smoothed acceleration and force spectrum for the structure ofFIG. 6 a measured at 44.1 kHz; -
FIG. 7 a plots the modified haptic click signal against time; -
FIG. 7 b plots the spectrum of the modified haptic signal ofFIG. 7 a and the target spectrum; -
FIG. 8 a plots an alternative modified haptic click signal against time; -
FIG. 8 b plots the spectrum of the modified haptic signal ofFIG. 8 a and the target spectrum; -
FIG. 9 a plots the scaled variation in amplitude over time for four types of signals, and -
FIG. 9 b plots the frequency spectra of the generated randomised tactile signals corresponding to each of the impulse signals ofFIG. 9 a. -
FIG. 1 a shows a touchsensitive device 10 comprising a touchsensitive screen 12 on which astylus 18 or pencil or similar writing instrument is being used to writetext 20. One ormore sensors 17 are used to detect a touch or movement of the stylus on the screen and anexciter 16 is provided to generate a signal within the screen. The slip stick behaviour of pencil on paper is a key element of the writing sensation. Writing to touch sensitive surfaces with a stylus has improved sensation, appeal and user satisfaction when there is simulation of the pencil on paper writing characteristic. -
FIG. 1 b shows how the touchsensitive device 10 may be adapted to use haptic methods and mechanical feedback technology to create such a simulation. The writing surface is mechanically energised under the pre-programmed control of the writing input from the stylus to simulate the feel of pencil on paper. As explained in more detail below, a model including the inherent mechanical behaviour of a pencil may be incorporated into the system but this would not be recognised or understood by a user who simply feels the result of the simulation. - As shown in
FIG. 1 b, the touchsensitive screen 12 is connected to asensor 16 which detects the nature of the contact from the stylus. The sensor is connected to PLL (phase locked loop) 24 which is one of the key elements used to generate the algorithm to generate the desired sensation of pencil on paper. The elements of the algorithm may be implemented in hardware or software. The individual elements have the following description: -
Short Element description Function PLL module 24 Phase-Locked Provides a stream of pulses having a mean impulse Loop rate locked to the speed of writing as determined by software from the touch screen. Jitter module 26Pulse Randomly modifies the regular pulses into pulses randomiser of varying amplitude and separation according to statistical rules Fs 32 Audio sample Generates clock for audio samples (texture) rate Impulse FIR filter Impulse response that is triggered by the jittered generator 28pulses Filter 30 Optional post- Reduces high-frequency content to make quieter filter - The PLL and Jitter blocks 24,26 run at the relatively slow rate set by the resulting pulse train. This should be below 150 Hz average rate, but the resolution of the jitter should be closer to the audio rate. The audio rate, Fs, is set as appropriate for the signal bandwidth required, but will almost certainly be no more than 11025 Hz (i.e. ¼ of the standard CD audio rate). Where multiple audio channels are used, these signals may be multiplexed in a single channel. Thus where four audio channels are used, and instead of having four channels at 11025 Hz, it would be possible to have one audio channel of 44100 Hz, which would be time-domain multiplexed (TDM) into four haptic channels. The output from the
filter 30 is sent to anexciter 17 which generates the signal in the screen to simulate the desired feel. -
FIG. 2 is a flow chart showing the steps implemented by each block in the system. The sensor, which may be implemented as software on the touch sensitive device, is monitoring the touch sensitive screen. When a “touch” is detected as at step S10, it requests a “haptic click” (step S12) from the signal generator orimpulse generator 28 and the impulse generator generates a pulse which provides a “click” sensation at step S28. Screen surfaces are generally softish and quite well damped. Thus the stylus impact is rather quiet. The “click” may or may not provide an audible feedback depending on the proposed use for the touch screen. When a “drag” is detected as at step S14, the sensor monitors the changes in position and calculates the drag rate, or velocity as at step S16. This velocity data is then fed to thePLL module 24 which produces a stream of pulses. When the stylus lifts, the sensor detects no touch as at step S30 and sends instructions to the PLL to stop (step S32). At step S34, the PLL stops generating any commands. - The pulses used for the haptic click sensation may be the same as the pulses which form the basis of the writing simulation but are not necessarily the same. For example, in the suggested implementation below, they are different. The spectra of the different signals are all chosen to match the sensitivity of the finger-tips to vibration.
- The function of the
PLL module 24 is described as follows. At step S18, on receipt of velocity information from the sensor, the PLL module provides a steady stream of pulses which act as start commands to the Jitter module (step S20). The PLL module measures the mean error between the rate of this stream (the actual rate) and the incoming velocity estimates (the target rate) and adjusts the actual rate to match the target. The PLL module should provide memory and some filtering, so that in the presence of noisy or missing estimates there is still a regular output. In summary, - Inputs: Start/Stop, Velocity (target rate)
Outputs: Pulses at target rate
Function: Measure mean error between target and actual rates, and adjust appropriately.
Notes: Target pulse frequency=drag velocity/inter-fibre spacing - The function of the
Jitter module 26, which provides the synthetic paper structure to the texture, is described as follows. At step S20, theJitter module 26 assigns a random amplitude to the pulse, and then at step S22, delays the pulse for a random duration before passing it on to the signal orimpulse generator 28 at step S24. The statistics of the amplitude distribution are unconstrained by the pulse rate, but the average delay should be ½ the inter-pulse spacing, which means that some knowledge of the pulse rate is required. In summary, - Inputs: Synchronisation (start), mean rate
Output: Amplitude, delayed synch
Function: Effectively, a programmable mono-stable with additional output - The Beta distribution may be used for the random signals. This sets the correct mean and variance for both jitter and amplitude data. The values chosen “by inspection” seem close to optimal—other values tested produce less realistic sensations. Generating Beta statistics by software or hardware may be difficult, so pragmatically it is suggested to use a simple uniform distribution (i.e. rectangular distribution) of the same mean and variance. The rectangular distribution is by far the simplest to generate and is as good as, or almost as good as the most complicated version
- The standard way of generating a uniform distribution of samples by software or hardware is the PRBS, or pseudo-random bit sequence. This is produced by a shift-register with feed-back occurring on certain bit patterns, or masks. The choice of mask affects the repeat length of the sequence and the “whiteness” of the noise.
- A standard method for generating a specified statistical distribution from the uniform distribution is the so-called “Inverse transform sampling” method (see. It maps noise samples from a uniform distribution on (0, 1] into samples having the specified statistical distribution. For this method, it is necessary to know the inverse of the cumulative density function. A simple example follows;
- Assume a target uniform distribution on (a, b]; the probability density function (PDF) is
-
- The cumulative density function (CDF) is obtained by integrating the PDF. The inverse CDF is a function that inverts the CDF, i.e. CDF−1(CDF(x))=x. From the PDF, it is also possible to calculate the mean and standard deviation. Fixing any two different statistical properties allows a and b to be determined.
- Unfortunately, the inverse CDF in closed form is not known for the Beta distribution, hence this method does not help. However, we may look at similar distributions which do have a known inverse CDF, and use these instead. The simplest approximation is the triangular distribution, so called because its PDF is in the shape of a triangle. In its most general form, it is controlled by three variables; a, b and c. In a simplified form, a=0, b=1 and 0<=c<=1 and
-
- The closest approximation to the Beta distribution is the Kumaraswamy distribution. It is controlled by two variables, a and b (real and positive), and is bound on [0,1] with
-
- Each of the three distributions highlighted above may be used to generate randomness to simulate texture. Each sample requires two sets of random data—the spacing and the amplitude. In principle, these two sets could be completely independent, but testing suggests that they should be generated from the same uniform distribution. (This does make some sense, as a long gap will be associated with a large amplitude and a small gap with a small amplitude, thus making the signal energy more uniform).
-
Distribution Parameters for spacing Parameters for amplitude Kumaraswamy a = 3, b = 5 a = 5, b = 3 Triangular [0, 1] a = 0, b = 1, c = 0.5 a = 0, b = 1, c = 0.75 Rectangular a = 0.207, b = 0.793 a = 0.375, b = 0.875 - Plotting the PDF together with the spectra for each distribution suggests that the “odd man out” is the triangular distribution. The main differences are in the 1-5 Hz range, and to a lesser extent at the 50 Hz mean repetition rate. Allowing the triangular distribution to cover the full range is not best. The rectangular distribution is as good as, or almost as good as the most complicated version (Kumaraswamy) and thus is the most logical choice since it is by far the simplest to generate.
- The function of the
impulse generator 28, which provides spectral properties of the texture, is described as follows. On receipt of a signal from thejitter module 26, theimpulse generator 28 outputs a signal in the form of a stream of sample values (step S26). If another input is received before the stream is complete, then a new stream begins. The amplitude of the output signal is modified according to data received from thejitter module 26. In summary, - Inputs: Synchronisation (start), amplitude
Output: Sequence of haptic “audio” samples
Function: Filter the pulse train into an analogue signal - This function may be selected from a number of mathematical models and fed appropriate (adjustable) parameters. Using these functions, arbitrary non-integer roll-off rates are possible as explained below. The impulse generator comprises a FIR (finite impulse response) filter to match the haptic signal output from the
impulse generator 28 to the sensitivity of the fingers to touch. The filter may reduce the high-frequency content to a reasonable level. - The
optional filter 30 may be provided to reduce any remaining high-frequency breakthrough from the impulse generator. It is envisaged that this will be a very simple recursive, 1st order stage filter with coefficients chosen to avoid multiplication; e.g. 2̂(−n), 1-2̂(−n). If this proves insufficient, a 2nd order filter could be used instead. The cut-off frequency would be around, say, 500 Hz-600 Hz. -
FIG. 3 a shows the impulse responses of four signals from the impulse generator incorporating a filter, each with a cut-off at or near 300 Hz. Filters universally have integer order roll-off; for example the R-C network of electronics has a first-order response, while the L-C-R network may have a second-order response. An n-th order roll-off on a bode-plot is represented by a slope of 6×n dB per octave or 20×n dB per decade. - Many natural phenomena have “fractal” characteristics—that is, their dimensionality is non-integer. A simple and well known example is “1/f noise” which has a ½ order roll-off, or a 3 dB per octave slope. In order to produce a signal with the right “feel” and “sound”, it was found useful to have the ability to assign a fully variable roll-off to the signal. That is, its level falls as frequencŷp or its power as frequencŷ2p.
- It is known from Laplace transform theory that there is a direct relationship between the impulse response and its transfer function; and in particular between their rates of decay (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 29.3.7)
-
- All these transfer functions are infinite at DC, so not particularly useful in practice. There are, however, more useful transform pairs that allow synthesis in either domain.
- The
type 1 signal shown inFIG. 3 a is unipolar and it was found that a rate of approximately 2.25 to 2.5 gave the best feel/sound. Such a signal may be generated by using a cascaded first-order roll-off filter with unity gain pass-band. Filters of this type have the transfer function -
- For integer p, it is easy to see how this represents a cascade of p, first-order low-pass filters. Each filter has a cut-off frequency of ωc=a radians/sec. The roll-off rate is p-th order, i.e. 6p dB/octave. We wish to generalise this to the case when p is not an integer.
- From a table of Laplace transforms, or a program that calculates them symbolically, we find (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 29.3.11)
-
- The impulse response is unipolar, and may be considered as a generalisation of the exponential decay.
- The filter may be a classic first order filter with p=1. This should be familiar to anyone involved in simple systems design.
-
- Alternatively, the filter may be a half-order filter with p=½. This filter would turn white noise into pink noise above the cut-off
-
- Notice the symmetrical nature of this pair—in both the time and the frequency domains, the power-law is the reciprocal square root. It is this symmetrical nature which is at the heart of the explanation of 1/f noise (it is a quantum-mechanical phenomenon).
- The
type 2 signal shown inFIG. 3 a is bipolar and it was found that a rate of approximately 1.75 to 2.0 gave the best feel/sound. Such a signal may be generated by using a cascade second-order roll-off filter with unity gain pass-band. Filters of this type have the transfer function -
- For integer p, it is easy to see how this represents a cascade of p, second-order low-pass filters. Each filter has a cut-off frequency of ωc=sqrt(a2+b2) radians/sec, and a Q of ωc/2a. The roll-off rate is 2p-th order, i.e. 12p dB/octave. Again, we wish to generalise this to the case when p is not an integer.
- From a table of Laplace transforms, or a program that calculates them symbolically, we find (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 29.3.57 &29.2.12)
-
- where Jp is a Bessel function of order p.
-
- The impulse response is bipolar, and may be considered as a generalisation of the damped sinusoid.
- The filter may be a classic second-order filter with p=1. This classic filter transforms to the familiar damped sinusoid in the time domain.
-
- Alternatively, the filter may be first-order filter with a Q and p=½.
-
- The time-domain response is simply a damped, zeroth order Bessel function. For large t, the trigonometric approximation may be used (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 9.2.1), i.e.
-
- which shows that this is essentially an amplitude modulated version of the ½-order filter of the half order filter described above.
- Neither
type 1 nortype 2 signals have the colouration in sound produced by a pencil. Thetype 6 signal was produced by convolving thetype 1 signal with a similar signal of higher frequency. In this case the convolution has a closed form. The “Type 8” signal was produced by directly convolving theType 2 signal with a similar signal of higher frequency, but lower Q. - The spectra (i.e. sound output against frequency) of the corresponding texture signals for a mean pulse rate of 72.6 Hz are shown in
FIG. 3 b. The modified spectra diverge from the original spectra from about 800 Hz, and the signals are noticeably quieter in “silent mode”. - An alternative signal (type 3) may be generated using a filter having cascaded pairs of first-order sections with two, non-equal turning points. Filters of this type have the transfer function
-
-
- where Ip is a modified Bessel function (see e.g. Abramowitz & Stegun, “Handbook of mathematical functions”, article 29.3.50)
-
- One example is a cascaded first-order filter with p=1, where
-
- As described above, the target pulse frequency of the PLL module is equal to drag velocity/inter-fibre spacing. This equation was derived by generating a simple 2-D model of the paper surface and pencil interaction to understand the fundamental process. Paper is typically manufactured using a “web” of cellulose fibres of 2-5 mm length and a fine clay coating (particles ˜0.1 um). The resulting surface has ridges at many distance scales, but the major ones are in the range 0.1 mm to 0.5 mm, depending on the paper.
- Pencils are typically manufactured from a soft-wood surround (Larch or Cedar) around a “lead”. The lead is actually a ceramic formed as a co-fired mixture of graphite and clay, which is usually dipped in a polymer. The typical “sound” of a pencil is a function of the hardness of the lead and the beam resonances of the shaft. A typical series of modes might be, say; 350 Hz, 900 Hz, 2.0 kHz, 3.3 kHz, etc. The values will, of course, depend on the length of the pencil.
- The mechanism, then, may be described as the stick-slip motion of the pencil over the rough paper surface, with the sound being modified by the resonances of the pencil.
-
FIG. 4 shows the simple 2-D model of the paper surface and pencil interaction in which the paper is modelled with semi-circular ridges of equal height spaced at a regular 0.15 mm pitch. The pencil tip, and a short section of the shaft are modelled explicitly, and the hand-arm system is modelled by lumped parameters. - The simulation occurs in two phases, each lasting 1 second. In the first phase, the pencil in lowered onto the paper surface, and a writing force on 1 N is applied. In the second phase, the pencil is dragged at 1 mm/s along the paper surface. The resulting forces on the hand and motions of the pencil tip may be seen in
FIG. 5 a andFIG. 5 b respectively. The “cogging” seen in both sets of traces is in part due to the discrete nature of the model—the nodes. In real life, other texture details would be likely to cause similar effects. - It is plain to see that the periodicity of the signal is directly obtained from the drag rate and the inter-fibre spacing, i.e. the frequency=drag rate/inter-fibre spacing.
- The precise wave shape is set by the degree of control applied to the pencil. In this simulation, the velocity at the hand is constant, and the tip follows appropriately. The opposite extreme would be to apply a constant force sufficient to overcome the average dynamic friction. In this case, the velocity would be non-uniform.
-
FIG. 6 a shows a more realistic model of the paper surface. Statistical distributions of fibre distances and heights have been used to generate the depicted randomised surface structure. The beta distribution was chosen to generate the model for two main reasons; it is bounded on [0,1] (unlike the boundless normal distribution), and with two independent parameters it is possible to control two of the main statistical parameters (the mean, the mode, and the variance). At this stage, the exact parameters of the statistical distributions are completely arbitrary. - From the drag rate and the mean fibre separation, a cut-off frequency (fc) is calculated. A smoothed version of the spectrum generated by the model of
FIG. 5 a is seen inFIG. 5 b, where fc is seen to feature strongly. - The sampling data is convolved with a leaky integrator. In the measurements, the time-constant corresponded to 40 rad/s, but this merely controls the amount of very low frequency information in the resulting signal. When the signal is played over the laptop loudspeaker, it sounds like a finger-nail being dragged over paper.
-
FIG. 6 c shows the force spectra for measurements obtained from a custom writing tool. The tool has interchangeable tips, and is fitted with a force gauge and an accelerometer, both connected to a charge amplifier (ENDEVCO Model 133). Neither gauges are fully calibrated, but the force gauge sensitivity is known to be close to 1 V/N. Data was acquired via a NI PCI-4452 data acquisition card. The spectrum of the measured signal is strongly affected by the speed of writing. The essentially low-pass spectra has cut-off frequencies that are directly proportional to the speed of writing. The bandwidth was roughly established by tracing over graph paper and using a stop watch to be about 50-60 Hz at 1 inch/sec writing rate for normal paper. The bandwidth was different for different surfaces. - In the example of
FIG. 6 c, the writing speed is about 2 in/s and the data is sampled at 44.1 kHz. The force results measure the forces applied between the pencil tip and the shaft of the writing tool. The acceleration results measure the resulting motion of the pencil. From these measurements, it is possible to derive the effective impedance of the system by using the relation -
Zm=F/v=j.2.π.f.F/a - where F=force, v=velocity, a=acceleration, f=frequency.
- Bearing in mind that the accelerometer is uncalibrated, the effective impedance is like the combination of a 0.4 kg mass and a 100 Ns/m dashpot. This is, in effect, acting as a lossy integrator with a corner frequency of about 40 rad/s.
- As is clearly seen, the measured force spectra of
FIG. 6 c corresponds to the simulated spectra ofFIG. 6 b. The signal ofFIG. 6 b is then filtered to apply boosts at frequencies corresponding to modes in a pencil. The new signal spectrum is reminiscent of the accelerometer spectra seen inFIG. 6 c, which has some resonances in the acceleration trace due to modes in the writing tool. When the signal is played over the laptop loudspeaker, it sounds much more like a pencil being dragged over paper. - As shown in
FIG. 2 , if the sensor detects a touch, a “click” signal is requested. One such signal is a frequency and amplitude modulated cosine function, i.e. -
- h(t) is the product of g(t)—the envelope function and fm(t)—a frequency modulating function.
- where g(t)=α·t·e1−α·t, which has a maximum value of unity at time t=1/α.,
- α is a decay rate of the envelope
β is a parameter controlling the rate of frequency modulation, and
ωc is the angular frequency at time t=0. - This signal may also be used as the “
type 3” signal mentioned above which is used to generate handwriting texture when a drag is detected. - This cosine function signal was found to be more effective than its sine-based counterpart and further improvements are investigated below by adding a new variable φ was added to the function. This amended function is then optimally fitted to the target haptic spectrum.
-
- It was observed that for the best signals, the peak of the envelope at t=1/α corresponds with a peak in the cosine function. In this case, we can set φ directly. Using elementary calculus confirms that the correct value sets the argument of the cosine to 0 at t=1/α, thus:
-
- as before h(t) is the product of g(t)—the envelope function and fm(t)—a frequency modulating function but in this case
-
- The optimal values for the three variables—α=532.5, β=83.85, ωc=3133—are slightly different from those of the original function. In both cases, the parameters are chosen to match a spectral template which shows the relative sensitivity of the finger-tips to vibration as a function of the vibration frequency. The aim is to put the most energy in the frequency range at which the fingers are most sensitive.
-
FIG. 7 a shows the variation in time for the envelope function g(t) and the frequency modulation function fm(t) of the signal.FIG. 7 a also shows how α is derived. The timing of the first non-zero point of intersection is equivalent to 1/α.FIG. 7 b shows the target spectrum (dotted line) which gives the desired sensation to a user and the actual spectrum of the modified function detailed above. There is a good match between the two spectra. Other values of the parameters, or even other signals, may be used to achieve similar aims. The signal ofFIGS. 8 a and 8 b is just such a signal—its parameter values are also chosen to match a target spectrum. -
FIGS. 8 a and 8 b illustrate an alternative signal having its basis starting in the frequency domain and expressed as. -
h(t)=√{square root over (2α)}t exp(0.5−αt 2)cos(ωct−φ) - Where a=96505=310.72, b=2011, φ=5.181 rad=297°.
-
FIG. 8 a shows that the sensitivity curve (fm(t)) resembles a normal distribution curve. It is also known that this curve (fm(t)) is its own Fourier transform, so the time domain signal should be similar.FIG. 8 a also shows how α and ωc are calculated. α and ωc are calculated as described above. As shown inFIG. 8 b, this alternative does not provide as good a fit to the target spectrum as the signal ofFIG. 7 a but has the main advantage that the high-frequency end of its spectrum falls faster. - Comparing the two signals, for the same peak amplitude, the alternative signal appears to be 25% more energy efficient. However, some testing has shown that 15%-20% higher amplitude is needed to get the same sensation, thereby eliminating the advantage. There is less high-frequency energy in the alternative signal, which may well help with making it more silent. In short, there is not much to choose between them. The alternative signal is illustrated in
FIGS. 8 a and 8 b as “type 9” signal. -
FIGS. 9 a and 9 b compare four signals for effectiveness as texture waveforms. The signals are adjusted in amplitude to give the same degree of sensation but use different amounts of power to achieve the sensation. -
Integrated Type Description rms level 3 Original haptic click with phase optimisation of 0.251 FIGS. 7a and 7b 6 Unipolar, dual slope generalised impulse, from type 1 - 0.406 see FIG. 3a 8 Bipolar, dual slope generalised impulse, from type 2 - 0.201 see FIG. 3a 9 Alternative haptic click of FIGS. 8a and 8b 0.237 - The
type 8 signal is the most energy efficient, with the new haptic click (type 9) coming a close second.Type 6 is the least efficient. - The quicker decay of the
type 9 signal, when compared to the otherwisesimilar type 3 signal, improves the feel considerably.Type 9 is the quietest in “silent mode” in the absence of additional filtering. - The small change to the existing
type 3 click described in relation toFIGS. 7 a and 7 b is worth doing—it costs nothing, and makes an improvement. Whether or not it is worth changing to the alternative signal (type 9) will depend on subjective assessment. - The signal type currently suggested (type 8) is a minor adjustment to the
type 2 signal previously chosen. It is still the most efficient, although not by much. It is worth comparing it to thenew type 9 signal.
Claims (21)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GBGB0814506.2 | 2008-08-08 | ||
GB0814506.2A GB2462465B (en) | 2008-08-08 | 2008-08-08 | Touch sensitive device |
PCT/JP2009/064366 WO2010016627A1 (en) | 2008-08-08 | 2009-08-07 | Touch sensitive device |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2009/064366 A-371-Of-International WO2010016627A1 (en) | 2008-08-08 | 2009-08-07 | Touch sensitive device |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/861,942 Continuation US10191551B2 (en) | 2008-08-08 | 2018-01-04 | Touch sensitive device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20110102349A1 true US20110102349A1 (en) | 2011-05-05 |
Family
ID=39790477
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/921,977 Abandoned US20110102349A1 (en) | 2008-08-08 | 2009-08-07 | Touch Sensitive Device |
US15/861,942 Active US10191551B2 (en) | 2008-08-08 | 2018-01-04 | Touch sensitive device |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/861,942 Active US10191551B2 (en) | 2008-08-08 | 2018-01-04 | Touch sensitive device |
Country Status (9)
Country | Link |
---|---|
US (2) | US20110102349A1 (en) |
EP (1) | EP2684110A4 (en) |
JP (1) | JP4920796B2 (en) |
KR (1) | KR101228512B1 (en) |
CN (2) | CN101978344A (en) |
CA (1) | CA2769930A1 (en) |
GB (1) | GB2462465B (en) |
TW (1) | TWI437470B (en) |
WO (1) | WO2010016627A1 (en) |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110320204A1 (en) * | 2010-06-29 | 2011-12-29 | Lenovo (Singapore) Pte. Ltd. | Systems and methods for input device audio feedback |
US20120038568A1 (en) * | 2009-02-16 | 2012-02-16 | New Transducers Limited | Touch Sensitive Device |
US20120242599A1 (en) * | 2011-02-10 | 2012-09-27 | Samsung Electronics Co., Ltd. | Device including plurality of touch screens and screen change method for the device |
CN103186302A (en) * | 2012-01-01 | 2013-07-03 | 赛普拉斯半导体公司 | Contact identification and tracking on a capacitance sensing array |
US20140176472A1 (en) * | 2012-12-26 | 2014-06-26 | Lenovo (Beijing) Co., Ltd. | Haptic feedback method, haptic feedback apparatus, electronic device and stylus |
US20140281954A1 (en) * | 2013-03-14 | 2014-09-18 | Immersion Corporation | Systems and Methods For Haptic And Gesture-Driven Paper Simulation |
US20140267065A1 (en) * | 2013-03-14 | 2014-09-18 | Immersion Corporation | Contactor-based haptic feedback generation |
WO2014148771A1 (en) * | 2013-03-22 | 2014-09-25 | Samsung Electronics Co., Ltd. | Portable terminal and method for providing haptic effect |
US9292090B2 (en) | 2012-01-31 | 2016-03-22 | Panasonic Intellectual Property Management Co., Ltd. | Haptic feedback device and haptic feedback method |
US9372661B2 (en) | 2012-04-02 | 2016-06-21 | Samsung Electronics Co., Ltd. | Apparatus and method of generating a sound effect in a portable terminal |
US9507454B1 (en) * | 2011-09-19 | 2016-11-29 | Parade Technologies, Ltd. | Enhanced linearity of gestures on a touch-sensitive surface |
US9639188B2 (en) | 2011-09-01 | 2017-05-02 | Lg Innotek Co., Ltd. | Touch panel and liquid crystal display comprising the same |
US9690382B1 (en) * | 2016-09-06 | 2017-06-27 | Apple Inc. | Devices, methods, and graphical user interfaces for haptic mixing |
US9830784B2 (en) | 2014-09-02 | 2017-11-28 | Apple Inc. | Semantic framework for variable haptic output |
US9984539B2 (en) | 2016-06-12 | 2018-05-29 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US9996157B2 (en) | 2016-06-12 | 2018-06-12 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US10175762B2 (en) | 2016-09-06 | 2019-01-08 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US10671186B2 (en) | 2016-06-15 | 2020-06-02 | Microsoft Technology Licensing, Llc | Autonomous haptic stylus |
CN112835456A (en) * | 2021-02-03 | 2021-05-25 | 维沃移动通信有限公司 | Touch control pen and control method |
US11314330B2 (en) | 2017-05-16 | 2022-04-26 | Apple Inc. | Tactile feedback for locked device user interfaces |
US20220283639A1 (en) * | 2019-05-07 | 2022-09-08 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Touch interface offering improved localised vibrotactile feedback |
US11439907B2 (en) | 2019-06-14 | 2022-09-13 | Nintendo Co., Ltd. | Audio feedback that varies based on direction of input stroke |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2482190A (en) | 2010-07-23 | 2012-01-25 | New Transducers Ltd | Methods of generating a desired haptic sensation in a touch sensitive device |
US9639213B2 (en) | 2011-04-26 | 2017-05-02 | Sentons Inc. | Using multiple signals to detect touch input |
US10198097B2 (en) | 2011-04-26 | 2019-02-05 | Sentons Inc. | Detecting touch input force |
US11327599B2 (en) | 2011-04-26 | 2022-05-10 | Sentons Inc. | Identifying a contact type |
US9189109B2 (en) | 2012-07-18 | 2015-11-17 | Sentons Inc. | Detection of type of object used to provide a touch contact input |
US9449476B2 (en) | 2011-11-18 | 2016-09-20 | Sentons Inc. | Localized haptic feedback |
EP2780783B1 (en) | 2011-11-18 | 2022-12-28 | Sentons Inc. | Detecting touch input force |
US11262253B2 (en) | 2017-08-14 | 2022-03-01 | Sentons Inc. | Touch input detection using a piezoresistive sensor |
US9075438B2 (en) * | 2012-07-18 | 2015-07-07 | Htc Corporation | Systems and related methods involving stylus tactile feel |
CN103576903B (en) * | 2012-07-24 | 2017-11-24 | 联想(北京)有限公司 | A kind of writing pencil of information processing method and application this method |
KR102349956B1 (en) | 2014-09-17 | 2022-01-11 | 삼성전자주식회사 | Film for writing and display apparatus comprising the same |
US10908741B2 (en) | 2016-11-10 | 2021-02-02 | Sentons Inc. | Touch input detection along device sidewall |
US10585522B2 (en) | 2017-02-27 | 2020-03-10 | Sentons Inc. | Detection of non-touch inputs using a signature |
US11580829B2 (en) | 2017-08-14 | 2023-02-14 | Sentons Inc. | Dynamic feedback for haptics |
US10620705B2 (en) | 2018-06-01 | 2020-04-14 | Google Llc | Vibrating the surface of an electronic device to raise the perceived height at a depression in the surface |
Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4885565A (en) * | 1988-06-01 | 1989-12-05 | General Motors Corporation | Touchscreen CRT with tactile feedback |
US4980518A (en) * | 1988-10-20 | 1990-12-25 | Canon Kabushiki Kaisha | Coordinate input apparatus |
US5627348A (en) * | 1995-04-07 | 1997-05-06 | A.T. Cross Company | Electronic stylus with writing feel |
US5638060A (en) * | 1992-10-15 | 1997-06-10 | Yazaki Corporation | System switch device |
US5977867A (en) * | 1998-05-29 | 1999-11-02 | Nortel Networks Corporation | Touch pad panel with tactile feedback |
US20010006006A1 (en) * | 1999-12-23 | 2001-07-05 | Hill Nicholas P.R. | Contact sensitive device |
US20020075135A1 (en) * | 2000-12-20 | 2002-06-20 | New Transducers Limited | Multi-functional vibro-acoustic device |
US6429846B2 (en) * | 1998-06-23 | 2002-08-06 | Immersion Corporation | Haptic feedback for touchpads and other touch controls |
US20020190963A1 (en) * | 2001-06-14 | 2002-12-19 | Koninklijke Philips Electronics N.V. | Data input system |
US20030067450A1 (en) * | 2001-09-24 | 2003-04-10 | Thursfield Paul Philip | Interactive system and method of interaction |
US20050030284A1 (en) * | 2000-09-28 | 2005-02-10 | Braun Adam C. | Directional tactile feedback for haptic feedback interface devices |
US20060044287A1 (en) * | 2004-09-01 | 2006-03-02 | Yasunori Ake | Coordinate input system and coordinate input method |
US20060158440A1 (en) * | 2005-01-19 | 2006-07-20 | Motion Computing Inc. | Active dynamic tactile feedback stylus |
US20070080951A1 (en) * | 2002-08-29 | 2007-04-12 | Sony Corporation | Input device and electronic device using the input device |
US7369115B2 (en) * | 2002-04-25 | 2008-05-06 | Immersion Corporation | Haptic devices having multiple operational modes including at least one resonant mode |
US20080204266A1 (en) * | 2004-02-03 | 2008-08-28 | Jussi Malmberg | Method and Device For Implementing Vibration Output Commands in Mobile Terminal Devices |
US20090079703A1 (en) * | 2007-09-20 | 2009-03-26 | Electronics And Telecommunications Research Institute | Device and system for providing user with sensation effect on touch screen |
US20090135164A1 (en) * | 2007-11-26 | 2009-05-28 | Ki Uk Kyung | Pointing apparatus capable of providing haptic feedback, and haptic interaction system and method using the same |
US20100013613A1 (en) * | 2008-07-08 | 2010-01-21 | Jonathan Samuel Weston | Haptic feedback projection system |
US20100090815A1 (en) * | 2006-12-25 | 2010-04-15 | Konica Minolta Holdings, Inc. | Handwriting electronic input system |
US8228299B1 (en) * | 2005-01-27 | 2012-07-24 | Singleton Technology, Llc | Transaction automation and archival system using electronic contract and disclosure units |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2749580B2 (en) | 1988-01-19 | 1998-05-13 | 花王 株式会社 | Perfume method of alkaline powder detergent |
GB9014130D0 (en) * | 1990-06-25 | 1990-08-15 | Hewlett Packard Co | User interface |
DE69318746T2 (en) * | 1992-02-18 | 1999-02-18 | Ncr Int Inc | Data processing device with feedback of user input |
UA51671C2 (en) | 1995-09-02 | 2002-12-16 | Нью Транзд'Юсез Лімітед | Acoustic device |
IL123486A (en) | 1995-09-02 | 2000-10-31 | New Transducers Ltd | Display screens incorporating loudspeakers |
JP4567817B2 (en) * | 1997-09-11 | 2010-10-20 | ソニー株式会社 | Information processing apparatus and control method thereof |
BR0016706A (en) | 1999-12-23 | 2002-09-24 | New Transducers Ltd | Sensitive contact device |
TW511391B (en) * | 2000-01-24 | 2002-11-21 | New Transducers Ltd | Transducer |
JP3949912B2 (en) * | 2000-08-08 | 2007-07-25 | 株式会社エヌ・ティ・ティ・ドコモ | Portable electronic device, electronic device, vibration generator, notification method by vibration and notification control method |
GB0116310D0 (en) | 2001-07-04 | 2001-08-29 | New Transducers Ltd | Contact sensitive device |
JP2005500725A (en) * | 2001-07-20 | 2005-01-06 | ニュー トランスデューサーズ リミテッド | Electronic article comprising a loudspeaker and a touchpad |
JP2003300015A (en) * | 2002-04-05 | 2003-10-21 | Nec Tokin Corp | Method for driving vibration actuator, and portable device |
GB0228512D0 (en) | 2002-12-06 | 2003-01-15 | New Transducers Ltd | Contact sensitive device |
EP1752860B1 (en) | 2005-08-12 | 2015-03-18 | LG Electronics Inc. | Mobile terminal with touch screen providing haptic feedback and corresponding method |
KR100710362B1 (en) | 2005-08-12 | 2007-04-23 | 엘지전자 주식회사 | Mobile communication terminal having touch screen |
CN1996217A (en) * | 2005-12-31 | 2007-07-11 | 北京三星通信技术研究有限公司 | Conversion input device based on handwriting input manner and method therefor |
US8405618B2 (en) * | 2006-03-24 | 2013-03-26 | Northwestern University | Haptic device with indirect haptic feedback |
JP4576537B2 (en) * | 2006-12-13 | 2010-11-10 | 国立大学法人埼玉大学 | Pen tablet interface |
BRPI0804355A2 (en) * | 2008-03-10 | 2009-11-03 | Lg Electronics Inc | terminal and control method |
KR101486343B1 (en) * | 2008-03-10 | 2015-01-26 | 엘지전자 주식회사 | Terminal and method for controlling the same |
-
2008
- 2008-08-08 GB GB0814506.2A patent/GB2462465B/en active Active
-
2009
- 2009-08-06 TW TW098126544A patent/TWI437470B/en active
- 2009-08-07 JP JP2011517324A patent/JP4920796B2/en active Active
- 2009-08-07 WO PCT/JP2009/064366 patent/WO2010016627A1/en active Application Filing
- 2009-08-07 CN CN2009801102828A patent/CN101978344A/en active Pending
- 2009-08-07 CN CN201510218444.7A patent/CN105005406B/en active Active
- 2009-08-07 CA CA2769930A patent/CA2769930A1/en not_active Abandoned
- 2009-08-07 EP EP09805103.0A patent/EP2684110A4/en not_active Ceased
- 2009-08-07 KR KR1020107023344A patent/KR101228512B1/en active IP Right Grant
- 2009-08-07 US US12/921,977 patent/US20110102349A1/en not_active Abandoned
-
2018
- 2018-01-04 US US15/861,942 patent/US10191551B2/en active Active
Patent Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4885565A (en) * | 1988-06-01 | 1989-12-05 | General Motors Corporation | Touchscreen CRT with tactile feedback |
US4980518A (en) * | 1988-10-20 | 1990-12-25 | Canon Kabushiki Kaisha | Coordinate input apparatus |
US5638060A (en) * | 1992-10-15 | 1997-06-10 | Yazaki Corporation | System switch device |
US5627348A (en) * | 1995-04-07 | 1997-05-06 | A.T. Cross Company | Electronic stylus with writing feel |
US5977867A (en) * | 1998-05-29 | 1999-11-02 | Nortel Networks Corporation | Touch pad panel with tactile feedback |
US6429846B2 (en) * | 1998-06-23 | 2002-08-06 | Immersion Corporation | Haptic feedback for touchpads and other touch controls |
US20010006006A1 (en) * | 1999-12-23 | 2001-07-05 | Hill Nicholas P.R. | Contact sensitive device |
US20050030284A1 (en) * | 2000-09-28 | 2005-02-10 | Braun Adam C. | Directional tactile feedback for haptic feedback interface devices |
US20020075135A1 (en) * | 2000-12-20 | 2002-06-20 | New Transducers Limited | Multi-functional vibro-acoustic device |
US20020190963A1 (en) * | 2001-06-14 | 2002-12-19 | Koninklijke Philips Electronics N.V. | Data input system |
US7119796B2 (en) * | 2001-06-14 | 2006-10-10 | Koninklijke Philips Electronics N. V. | Data input system |
US20030067450A1 (en) * | 2001-09-24 | 2003-04-10 | Thursfield Paul Philip | Interactive system and method of interaction |
US7369115B2 (en) * | 2002-04-25 | 2008-05-06 | Immersion Corporation | Haptic devices having multiple operational modes including at least one resonant mode |
US20070080951A1 (en) * | 2002-08-29 | 2007-04-12 | Sony Corporation | Input device and electronic device using the input device |
US20080204266A1 (en) * | 2004-02-03 | 2008-08-28 | Jussi Malmberg | Method and Device For Implementing Vibration Output Commands in Mobile Terminal Devices |
US20060044287A1 (en) * | 2004-09-01 | 2006-03-02 | Yasunori Ake | Coordinate input system and coordinate input method |
US20060158440A1 (en) * | 2005-01-19 | 2006-07-20 | Motion Computing Inc. | Active dynamic tactile feedback stylus |
US8228299B1 (en) * | 2005-01-27 | 2012-07-24 | Singleton Technology, Llc | Transaction automation and archival system using electronic contract and disclosure units |
US20100090815A1 (en) * | 2006-12-25 | 2010-04-15 | Konica Minolta Holdings, Inc. | Handwriting electronic input system |
US20090079703A1 (en) * | 2007-09-20 | 2009-03-26 | Electronics And Telecommunications Research Institute | Device and system for providing user with sensation effect on touch screen |
US20090135164A1 (en) * | 2007-11-26 | 2009-05-28 | Ki Uk Kyung | Pointing apparatus capable of providing haptic feedback, and haptic interaction system and method using the same |
US20100013613A1 (en) * | 2008-07-08 | 2010-01-21 | Jonathan Samuel Weston | Haptic feedback projection system |
Cited By (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120038568A1 (en) * | 2009-02-16 | 2012-02-16 | New Transducers Limited | Touch Sensitive Device |
US9804673B2 (en) * | 2009-02-16 | 2017-10-31 | Nvf Tech Ltd. | Touch sensitive device |
US10296094B2 (en) | 2009-02-16 | 2019-05-21 | Nvf Tech Ltd | Touch sensitive device |
US8595012B2 (en) * | 2010-06-29 | 2013-11-26 | Lenovo (Singapore) Pte. Ltd. | Systems and methods for input device audio feedback |
US20110320204A1 (en) * | 2010-06-29 | 2011-12-29 | Lenovo (Singapore) Pte. Ltd. | Systems and methods for input device audio feedback |
US10635295B2 (en) * | 2011-02-10 | 2020-04-28 | Samsung Electronics Co., Ltd | Device including plurality of touch screens and screen change method for the device |
US20120242599A1 (en) * | 2011-02-10 | 2012-09-27 | Samsung Electronics Co., Ltd. | Device including plurality of touch screens and screen change method for the device |
US9639188B2 (en) | 2011-09-01 | 2017-05-02 | Lg Innotek Co., Ltd. | Touch panel and liquid crystal display comprising the same |
US9507454B1 (en) * | 2011-09-19 | 2016-11-29 | Parade Technologies, Ltd. | Enhanced linearity of gestures on a touch-sensitive surface |
US20130169582A1 (en) * | 2012-01-01 | 2013-07-04 | Cypress Semiconductor Corporation | Contact identification and tracking on a capacitance sensing array |
WO2013101306A1 (en) * | 2012-01-01 | 2013-07-04 | Cypress Semiconductor Corporation | Contact identification and tracking on a capacitance sensing array |
CN103186302A (en) * | 2012-01-01 | 2013-07-03 | 赛普拉斯半导体公司 | Contact identification and tracking on a capacitance sensing array |
US8982090B2 (en) * | 2012-01-01 | 2015-03-17 | Cypress Semiconductor Corporation | Optical stylus synchronization |
US9292090B2 (en) | 2012-01-31 | 2016-03-22 | Panasonic Intellectual Property Management Co., Ltd. | Haptic feedback device and haptic feedback method |
US9372661B2 (en) | 2012-04-02 | 2016-06-21 | Samsung Electronics Co., Ltd. | Apparatus and method of generating a sound effect in a portable terminal |
US20140176472A1 (en) * | 2012-12-26 | 2014-06-26 | Lenovo (Beijing) Co., Ltd. | Haptic feedback method, haptic feedback apparatus, electronic device and stylus |
US9329686B2 (en) * | 2012-12-26 | 2016-05-03 | Lenovo (Beijing) Co., Ltd. | Haptic feedback method, haptic feedback apparatus, electronic device and stylus |
US9436282B2 (en) * | 2013-03-14 | 2016-09-06 | Immersion Corporation | Contactor-based haptic feedback generation |
US9547366B2 (en) * | 2013-03-14 | 2017-01-17 | Immersion Corporation | Systems and methods for haptic and gesture-driven paper simulation |
US20140281954A1 (en) * | 2013-03-14 | 2014-09-18 | Immersion Corporation | Systems and Methods For Haptic And Gesture-Driven Paper Simulation |
US20140267065A1 (en) * | 2013-03-14 | 2014-09-18 | Immersion Corporation | Contactor-based haptic feedback generation |
WO2014148771A1 (en) * | 2013-03-22 | 2014-09-25 | Samsung Electronics Co., Ltd. | Portable terminal and method for providing haptic effect |
US10977911B2 (en) | 2014-09-02 | 2021-04-13 | Apple Inc. | Semantic framework for variable haptic output |
US9928699B2 (en) | 2014-09-02 | 2018-03-27 | Apple Inc. | Semantic framework for variable haptic output |
US9830784B2 (en) | 2014-09-02 | 2017-11-28 | Apple Inc. | Semantic framework for variable haptic output |
US10504340B2 (en) | 2014-09-02 | 2019-12-10 | Apple Inc. | Semantic framework for variable haptic output |
US10089840B2 (en) | 2014-09-02 | 2018-10-02 | Apple Inc. | Semantic framework for variable haptic output |
US10417879B2 (en) | 2014-09-02 | 2019-09-17 | Apple Inc. | Semantic framework for variable haptic output |
US11790739B2 (en) | 2014-09-02 | 2023-10-17 | Apple Inc. | Semantic framework for variable haptic output |
US10156903B2 (en) | 2016-06-12 | 2018-12-18 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US11037413B2 (en) | 2016-06-12 | 2021-06-15 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US10276000B2 (en) | 2016-06-12 | 2019-04-30 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US11735014B2 (en) | 2016-06-12 | 2023-08-22 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US11468749B2 (en) | 2016-06-12 | 2022-10-11 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US10139909B2 (en) | 2016-06-12 | 2018-11-27 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US9996157B2 (en) | 2016-06-12 | 2018-06-12 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US10175759B2 (en) | 2016-06-12 | 2019-01-08 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US11379041B2 (en) | 2016-06-12 | 2022-07-05 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US9984539B2 (en) | 2016-06-12 | 2018-05-29 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US10692333B2 (en) | 2016-06-12 | 2020-06-23 | Apple Inc. | Devices, methods, and graphical user interfaces for providing haptic feedback |
US10671186B2 (en) | 2016-06-15 | 2020-06-02 | Microsoft Technology Licensing, Llc | Autonomous haptic stylus |
US10528139B2 (en) | 2016-09-06 | 2020-01-07 | Apple Inc. | Devices, methods, and graphical user interfaces for haptic mixing |
US10620708B2 (en) | 2016-09-06 | 2020-04-14 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US9864432B1 (en) * | 2016-09-06 | 2018-01-09 | Apple Inc. | Devices, methods, and graphical user interfaces for haptic mixing |
US9690382B1 (en) * | 2016-09-06 | 2017-06-27 | Apple Inc. | Devices, methods, and graphical user interfaces for haptic mixing |
US10901513B2 (en) | 2016-09-06 | 2021-01-26 | Apple Inc. | Devices, methods, and graphical user interfaces for haptic mixing |
US11221679B2 (en) | 2016-09-06 | 2022-01-11 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US10175762B2 (en) | 2016-09-06 | 2019-01-08 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US10901514B2 (en) | 2016-09-06 | 2021-01-26 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US11662824B2 (en) | 2016-09-06 | 2023-05-30 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US10372221B2 (en) | 2016-09-06 | 2019-08-06 | Apple Inc. | Devices, methods, and graphical user interfaces for generating tactile outputs |
US11314330B2 (en) | 2017-05-16 | 2022-04-26 | Apple Inc. | Tactile feedback for locked device user interfaces |
US20220283639A1 (en) * | 2019-05-07 | 2022-09-08 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Touch interface offering improved localised vibrotactile feedback |
US11934580B2 (en) * | 2019-05-07 | 2024-03-19 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Touch interface offering improved localised vibrotactile feedback |
US11439907B2 (en) | 2019-06-14 | 2022-09-13 | Nintendo Co., Ltd. | Audio feedback that varies based on direction of input stroke |
CN112835456A (en) * | 2021-02-03 | 2021-05-25 | 维沃移动通信有限公司 | Touch control pen and control method |
Also Published As
Publication number | Publication date |
---|---|
WO2010016627A8 (en) | 2010-04-22 |
GB0814506D0 (en) | 2008-09-17 |
EP2684110A1 (en) | 2014-01-15 |
US20180129289A1 (en) | 2018-05-10 |
CN105005406B (en) | 2018-12-18 |
JP4920796B2 (en) | 2012-04-18 |
EP2684110A4 (en) | 2016-10-05 |
CN105005406A (en) | 2015-10-28 |
CN101978344A (en) | 2011-02-16 |
TWI437470B (en) | 2014-05-11 |
TW201019185A (en) | 2010-05-16 |
GB2462465A (en) | 2010-02-10 |
JP2011527792A (en) | 2011-11-04 |
KR20100139044A (en) | 2010-12-31 |
GB2462465B (en) | 2013-02-13 |
US10191551B2 (en) | 2019-01-29 |
WO2010016627A1 (en) | 2010-02-11 |
KR101228512B1 (en) | 2013-01-31 |
CA2769930A1 (en) | 2010-02-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10191551B2 (en) | Touch sensitive device | |
Shultz et al. | The application of tactile, audible, and ultrasonic forces to human fingertips using broadband electroadhesion | |
US10082874B2 (en) | Haptic effect conversion system using granular synthesis | |
EP2483760B1 (en) | Touch sensitive device generating a haptic feedback that simulates a click feeling | |
CN102365609B (en) | Touch sensitive device | |
Wiertlewski et al. | A high-fidelity surface-haptic device for texture rendering on bare finger | |
KR20110139732A (en) | Touch sensitive device | |
CA2778089A1 (en) | Touch sensitive device | |
CN110546594B (en) | Time reversal interface for producing acoustic lubrication | |
Papetti et al. | Interactive vibrotactile feedback enhances the perceived quality of a surface for musical expression and the playing experience | |
Torres et al. | Design and control of an ultrasonic surface haptic device for longitudinal and transverse mode comparison | |
JP7252430B2 (en) | Cutaneous sensation presentation device, ultrasonic modulation device and ultrasonic modulation method | |
Grigorii et al. | Comparison of wide-band vibrotactile and friction modulation surface gratings | |
Sinclair | Velocity-driven audio-haptic interaction with real-time digital acoustic models | |
de Jong | Making grains tangible: microtouch for microsound | |
US20200379563A1 (en) | Tactile stimulation interface using time reversal and providing enhanced sensations | |
Lindsay | A Mechanical Amplifier for Haptic Feedback | |
Vitarius | Acoustic transfer functions for pressure sense lines | |
JPH03192395A (en) | Electronic musical instrument |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NISSHA PRINTING CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HARRIS, NEIL JOHN;REEL/FRAME:025475/0789 Effective date: 20100913 |
|
AS | Assignment |
Owner name: HIWAVE TECHNOLOGIES (UK) LIMITED, UNITED KINGDOM Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NISSHA PRINTING CO., LTD;REEL/FRAME:027768/0427 Effective date: 20120125 |
|
AS | Assignment |
Owner name: NVF TECH LTD., UNITED KINGDOM Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HIWAVE TECHNOLOGIES (UK) LIMITED;REEL/FRAME:033107/0256 Effective date: 20140613 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: GOOGLE LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NVF TECH LTD.;REEL/FRAME:050233/0949 Effective date: 20190821 |