US20060017691A1 - System and method for controlling audio output associated with haptic effects - Google Patents
System and method for controlling audio output associated with haptic effects Download PDFInfo
- Publication number
- US20060017691A1 US20060017691A1 US10/983,231 US98323104A US2006017691A1 US 20060017691 A1 US20060017691 A1 US 20060017691A1 US 98323104 A US98323104 A US 98323104A US 2006017691 A1 US2006017691 A1 US 2006017691A1
- Authority
- US
- United States
- Prior art keywords
- frequency
- haptic
- effect
- haptic effect
- output
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
- A63F2300/1037—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/60—Methods for processing data by generating or executing the game program
- A63F2300/6063—Methods for processing data by generating or executing the game program for sound processing
- A63F2300/6081—Methods for processing data by generating or executing the game program for sound processing generating an output signal, e.g. under timing constraints, for spatialization
Definitions
- the invention relates generally to haptic feedback devices. More specifically, the invention relates to a system and method for controlling audio output associated with haptic effects.
- haptic-enabled controllers such as mouse devices, can be configured to provide haptic feedback to a user while the user interacts with an operating system (OS), or other application.
- OS operating system
- FIG. 1 is a block diagram of a processor system, according to an embodiment of the invention.
- FIG. 2 is a diagram illustrating a haptic device, a controller, and a sensor, according to an embodiment of the invention.
- FIG. 3 is a block diagram of a haptic device, according to an embodiment of the invention.
- FIG. 4 is a diagram of multiple frequency ranges of haptic effects output by a haptic device, according to an embodiment of the invention.
- FIG. 5 is a plot of a magnitude versus frequency response of a haptic device, according to an embodiment of the invention.
- FIG. 6 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention.
- FIG. 7 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention.
- FIG. 8 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention.
- FIG. 9 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention
- FIG. 10 is a diagram of linearization of voltages of a haptic device, according to an embodiment of the invention.
- FIG. 11 is a diagram of various parameters associated with a smooth effect according to an embodiment of the invention.
- FIG. 12 is a diagram of various parameters associated with a strong effect according to an embodiment of the invention.
- FIG. 13 is a diagram of various parameters associated with a sharp effect according to an embodiment of the invention.
- a system and method for controlling audio output associated with haptic effects are described. More specifically, audio output associated with haptic effects can be controlled to modify a perceived experience of the haptic effects. For example, by modifying the audio output, a user can be made to perceive a frequency of a haptic effect as being different than the actual frequency.
- control signals can be configured to cause haptic effects to be output across a wide range of frequencies.
- These control signals can independently control haptic effects within any frequency range from among multiple frequency ranges. This can occur, for example, using either a single controller or multiple controllers configured to output control signals from each frequency range.
- a single controller can output control signals that independently control haptic effects in each of multiple frequency ranges.
- multiple controllers can be used, such that each controller outputs control signals within a single frequency range of multiple frequency ranges, each controller being uniquely associated with each frequency range.
- Audio output associated with a haptic effect is generated in at least one frequency range of multiple frequency ranges when that haptic effect is output in response to a corresponding control signal.
- the haptic device can also create audible sound or, in other words, an audio output based on the movement of the haptic device.
- the audio signal heard by a user can correspond to a frequency of a haptic effect that is beyond the tactile detection capabilities of the user. In other words, although a user cannot feel a difference in the frequency of a haptic effect above a certain threshold frequency, the user can hear audio associated with such higher frequencies.
- One or more embodiments of the invention provide an extended perceived frequency range of haptic effects. More specifically, in addition to the range of haptic effects that can be tactilely detected by the user, a range of effects that are detected audibly by a user can be can be added such that the perceived overall experience has a greater frequency range. Because a user is able to sense an increased range of frequencies, more information can be communicated to the user using such combination of haptic effects and audio output.
- control signals having a constant average energy can be used to provide a variety of different audio output frequencies.
- the different audio output frequencies can cause a user to believe that he or she is sensing tactilely a different frequency of a haptic effect, even though tactilely sensing such a difference would not be possible.
- a variety of overall experiences (each having a haptic component and an audio component) caused by a corresponding variety of control signals, each of which has the same average energy, is able to be sensed by a user via a combination of the haptic effect and the associated audio output, even though the variety of haptic effects alone would be perceived as having the same feel without the audio output.
- the average energy can be maintained constant by varying the frequency and/or duty cycle of a control signal inversely with the magnitude of a control signal.
- the frequency of the control signal is increased, the magnitude decreases and, conversely, as the frequency of the control signal decreases, the magnitude increases, to maintain a constant average energy of the carrier signal.
- FIG. 1 is a block diagram of a processor system, according to an embodiment of the invention.
- the processor system 110 illustrated in FIG. 1 can be, for example, a commercially available personal computer, portable electronic device, or a less complex computing or processing device (e.g., a device that is dedicated to performing one or more specific tasks).
- the processor system can be a mobile telephone, a PDA, a portable gaming system, an MP3 player, or the like.
- the processor system 110 can be a terminal dedicated to providing an interactive virtual reality environment, such as a gaming system, or the like.
- each component of the processor system 110 is shown as being a single component in FIG. 1 , the processor system 110 can include multiple numbers of any components illustrated in FIG. 1 . Additionally, multiple components of the processor system 1 10 can be combined as a single component.
- the processor system 110 includes a processor 112 , which according to one or more embodiments of the invention, can be a commercially available microprocessor capable of performing general processing operations.
- the processor 112 can be an application-specific integrated circuit (ASIC) or a combination of ASICs, which is designed to achieve one or more specific functions, or enable one or more specific devices or applications.
- the processor 112 can be an analog or digital circuit, or a combination of multiple circuits.
- the processor 112 can optionally include one or more individual sub-processors or coprocessors.
- the processor can include a graphics coprocessor that is capable of rendering graphics, a math coprocessor that is capable of efficiently performing complex calculations, a controller that is capable of controlling one or more devices, a sensor interface that is capable of receiving sensory input from one or more sensing devices, and so forth.
- the processor system 110 can also include a memory component 114 .
- the memory component 114 can include one or more types of memory.
- the memory component 114 can include a read only memory (ROM) component 114 a and a random access memory (RAM) component 114 b.
- the memory component 114 can also include other types of memory not illustrated in FIG. 1 that are suitable for storing data in a form retrievable by the processor 112 .
- EPROM electronically programmable read only memory
- EEPROM erasable electrically programmable read only memory
- flash memory as well as other suitable forms of memory can be included within the memory component 114 .
- the processor system 110 can also include a variety of other components, depending upon the desired functionality of the processor system 110 .
- the processor 112 is in communication with the memory component 114 , and can store data in the memory component 114 or retrieve data previously stored in the memory component 114 .
- the processor system 110 can also include a haptic device 116 , which is capable of providing a variety of haptic output.
- the haptic device 116 can be configured to output basis haptic effects, such as periodic effects, magnitude-sweep effects, or timeline haptic effects, each of which is described in greater detail below.
- the haptic device 116 can include one or more force-applying mechanisms, which are capable of outputting haptic effects or force, to a user of the processor system 110 (e.g., via the housing of the processor system 110 ).
- effects or forces can be transmitted, for example, in the form of vibrational movement caused by the haptic device 116 (e.g., caused by a rotating mass, a piezo-electric device, or other vibrating actuator), or in the form of resistive force caused by the haptic device 116 .
- the processor system 110 can also, according to one or more embodiments of the invention, include a sensor 118 that is capable of receiving input from a user, the haptic device 116 , or is otherwise capable of sensing one or more physical parameters.
- a sensor 118 can be configured to measure speed, intensity, acceleration, or other parameters associated with a haptic effect output by the haptic device 116 .
- the sensor 118 can be configured to sense environmental or ambient conditions of the processor system's surroundings.
- the sensor 118 can interface and communicate with the processor 112 by way of a sensor interface (not shown) within the processor 112 .
- the processor system 110 can also include a controller 120 , which can optionally be internal to the processor 112 , or external thereto, as shown in FIG. 1 .
- the controller 120 can be configured to control the haptic device 116 when the processor 112 is not directly controlling the haptic device 116 .
- the controller 120 can control the memory 114 and/or the sensor 118 , as well as devices external to the processor system 110 by way of an input/output (I/O) component 124 (described below).
- I/O input/output
- the various components of the processor system 110 can communicate with one another via a bus 122 , which is capable of carrying instructions from the processor 112 and/or the controller 120 to other components, and which is capable of carrying data between the various components of the processor system 110 . Additionally, signals received via the sensor 118 can be communicated to the processor 112 or the controller 120 by way of the bus 122 . Data retrieved from or written to memory 114 can be carried by the bus 122 , as are instructions to the haptic device 116 . Instructions to the haptic device 116 can be provided in the form of haptic-effect signals (e.g., basis haptic-effect signals), for example, which can be provided by the processor 112 , the controller 120 , or devices external to the processor system 110 .
- haptic-effect signals e.g., basis haptic-effect signals
- the components of the processor system 110 can communicate with devices external to the processor system 110 by way of an input/output (I/O) component 124 (accessed via the bus 122 ).
- I/O component 124 can include a variety of suitable communication interfaces.
- the I/O component 124 can include, for example, wireless connections, such as infrared ports, optical ports, Bluetooth wireless ports, wireless LAN ports, or the like.
- the I/O component 124 can include, wired connections, such as standard serial ports, parallel ports, universal serial bus (USB) ports, S-video ports, large area network (LAN) ports, small computer system interface (SCSI) ports, and so forth.
- FIG. 2 is a diagram illustrating a haptic device, a controller, and a sensor, according to an embodiment of the invention.
- FIG. 2 also shows data values provided to the system (e.g., user input 202 and control parameters 204 ).
- the elements shown in FIG. 2 can be used with the processor system 110 , or a similar device.
- the controller 120 is configured to output control signals that are configured to cause haptic effects to be output by the haptic device 116 .
- user input 202 can optionally be provided (e.g., via the I/O component 124 shown in FIG. 1 ) and/or received by an optional sensor 118 .
- the user input 202 can also optionally be provided directly to a controller 120 (e.g., by way of the sensor 118 , or some other devices configured to accept and convey user input).
- the sensor 118 can also optionally receive information from the haptic device 116 . For example, the sensor 118 can sense the actual movements of the haptic device 116 .
- the controller 120 can optionally receive data from the sensor 118 , and can optionally receive user input 202 and control parameters 204 . Based on any data received from the sensor 118 , any received user input 202 , and/or any received control parameters 204 , the controller 120 controls the haptic output of the haptic device 116 (e.g., the controller 120 sends control signals configured to cause haptic effects). For example, the controller 120 can execute a feedback algorithm, controlling the haptic device 116 based on feedback received from the haptic device 116 . The controller 120 controls the output of the haptic device 116 by a control signal that the controller 120 outputs to the haptic device 116 .
- the controller 120 controls the haptic output of the haptic device 116 by a control signal that the controller 120 outputs to the haptic device 116 .
- control signal output by the controller 120 can be based on a number of parameters, including, for example, control parameters 204 .
- control parameters 204 and other parameters that can be used by the controller 120 to control the haptic device 116 can be stored in the memory component 114 of the processor system 110 , or by another suitable memory component.
- the control parameters 204 can include input from an electronic system, a portable gaming device, a cellular telephone, or the like.
- the controller receives control parameters (e.g., gaming device input, cellular telephone input, etc.), and does not include a sensor.
- user input can optionally be received directly by the controller, or can be omitted entirely, depending upon the desired function of the system in which the controller is used.
- the system shown in FIG. 2 can be used in a stand-alone device, such as a mobile telephone, portable electronic device (e.g., a PDA, etc.), or other device.
- haptic output can be provided in the form of haptic effects via the haptic device 116 in response to status events (e.g., a message received signal, a network indicator signal, etc.), user input (e.g., mode changes, keypad dialing, option selections, etc.), incoming calls, or other events.
- the system shown in FIG. 2 can be used in a configuration where a processor, such as the processor 112 of the processor system 110 shown in FIG. 1 , can be connected to an external device, and the processing tasks can be divided among the devices, as desired.
- the controller 120 can generate a variety of different control signals to drive the haptic device 116 , several of which will be described in greater detail below.
- the controller 120 can send a control signal to the haptic device 116 , which is configured to cause the haptic device 116 to output a corresponding haptic effect.
- control signals include, pulse width modulation (PWM) signals (e.g., pulse signals having a given duty cycle), sinusoidal signals, and other periodic signals (e.g., triangle waves, square waves, etc.).
- PWM pulse width modulation
- the controller 120 can modulate control signals using one or more haptic envelopes.
- the controller 120 also can be configured to provide a lead-in pulse at the beginning of a control signal, and/or a braking pulse, at the end of a control signal, which are configured to decrease response time of the haptic device 116 .
- the lead-in signal reduces the time for the haptic device 116 to initiate outputting a haptic effect associated with the control signal.
- the braking pulse decreases the time it takes for the haptic device 116 to cease a haptic effect currently being output.
- the controller 120 can output a variety of other control signals, such as non-periodic signals, that are configured to cause the haptic device 116 to output haptic effects.
- FIG. 3 is a block diagram of a haptic device 116 shown in FIGS. 1 and 2 .
- the haptic device 116 includes an actuator 302 , an elastic member 304 and a mass 306 .
- the haptic device 116 is configured to provide haptic feedback.
- the actuator 302 is operably connected to the elastic member 304
- the elastic member 304 is operably connected to the mass 306 .
- the actuator 302 can include, for example, a motor (e.g., a brush motor, a brushless motor, etc.).
- the elastic member can provide a desired amount of coupling rigidity between the actuator and the mass 306 .
- the actuator 302 When control signals are received by the haptic device 116 , the actuator 302 provides force to the elastic member 304 . Some of the force applied to the elastic member 304 is translated to the mass 306 , and causes the mass 306 to move. By causing the mass 306 to move, haptic effects commanded by the control signals are output by the haptic device, and can be output to a user.
- the actuator 302 can be configured, for example, to cause the mass to rotate in response to the control signals received by the haptic device. Alternatively, the actuator can move the mass 306 in other directions (e.g., vibrating the mass, moving the mass laterally, etc.).
- the configuration shown in FIG. 3 is only one example of a configuration of a haptic device 116 .
- Other configurations that vary from the configuration shown in FIG. 3 can be used as a haptic device 116 , according to one or more embodiments of the invention.
- the elastic member 304 can be coupled to the mass 306 by a flexible coupling; the elastic member 304 can be coupled to the actuator 302 by a flexible coupling.
- the elastic member can be coupled between actuator and a mechanical ground, and the actuator can be directly coupled to the actuator.
- haptic devices examples include an eccentric-rotating-mass (ERM) haptic device and a harmonic ERM (HERM) haptic device, described in detail in copending U.S. patent application Ser. No. 10/301,809, which is incorporated by reference herein in its entirety.
- ERM eccentric-rotating-mass
- HERM harmonic ERM
- FIG. 4 is a diagram of multiple frequency ranges of haptic effects that can be output by a haptic device 116 , according to an embodiment of the invention.
- a low-frequency range extends from approximately DC (i.e., 0 Hz) to a low-frequency limit f L , which can vary depending upon the control signal being used to cause a haptic effect and the desired characteristics of the haptic effect.
- a mid-frequency range extends from the low-frequency threshold frequency f L to a high-frequency threshold frequency f H , which can vary depending upon the control signal being used to cause a haptic effect and the desired characteristics of the haptic effect.
- a high-frequency range extends from the high-frequency threshold frequency f H to all higher frequencies.
- At least one frequency range from the frequency ranges shown in FIG. 4 can have an audio output associated with the haptic effect.
- the audio output can occur for haptic effects having a frequency within the at least one frequency range or for haptic effects having a frequency beyond the at least one frequency range, depending upon the desired performance of the system.
- the mid-frequency range shown in FIG. 4 can have an audio output associated with a haptic effect having a frequency within the mid-frequency range.
- the haptic effect having a frequency within the mid-frequency range can be varied in a manner such that the associated audio output varies, while any changes in the frequency of the haptic effect remains tactilely undetectable to a user. Because the audio output varies (e.g., changes frequency of the audio output), a user aurally detects the change in the audio output, and believes that he or she has tactilely detected a change in the haptic effect.
- the user may perceive that the overall effect (the combination of a haptic effect and an audio effect) has changed and attribute such a change, at least in part, to the user's tactile experience.
- the average energy of a control signal used to cause the haptic effect to be output can be maintained substantially the same while the associated audio output is varied, causing a user to detect an increase in the audio output and believe that he or she has tactilely perceived a change in the haptic effect.
- FIG. 4 illustrates only three frequency ranges, the number of frequency ranges used according to one or more embodiments of the invention can be varied. For example, many more frequency ranges can be used, among which multiple frequency ranges can include an audio output associated with the haptic effects having frequencies within those frequency ranges.
- Haptic effects having frequencies within each of the frequency ranges shown in FIG. 4 can be separately controlled. This can occur, for example, using a single controller, that separately controls the haptic effects associated with each of the frequency ranges shown in FIG. 4 .
- each frequency range shown in FIG. 4 can have a uniquely associated controller, which is configured to control haptic effects having frequencies within that frequency range.
- FIG. 5 is a plot of a magnitude versus frequency of a haptic effect, according to an embodiment of the invention.
- the plot shown in FIG. 5 is not drawn to scale, and is intended only as an example of the correlation between the magnitude and frequency of a haptic effect, and how a user perceives them.
- the magnitude versus frequency response shown in FIG. 5 indicates that, as the frequency of a haptic effect is increased, the magnitude of that haptic effect appears also to increase to a user.
- a first portion 510 of the curve shown in FIG. 5 represents a region of haptic effect frequencies within which a user can detect changes in frequency. Within some region of frequencies (e.g., beginning near the high-frequency threshold frequency f H in the plot of FIG.
- a user perceives some increases in the frequency of a haptic effect as increases in magnitude (and not the frequency) without being able to detect tactilely the increases in frequency.
- a second portion 520 of the curve shown in FIG. 5 represents this region, where the user has difficulty tactilely detecting changes in frequency.
- the diminished-sensitivity region begins (i.e., the area of intersection of the first portion 510 and the second portion 520 of the curve shown in FIG. 5 )
- a user's perception of increasing frequencies of combination of a haptic effect and audio output is illustrated using a line 520 . Similar lines can be drawn to illustrate a user's perception of increasing frequencies of a combination of a haptic effect and an audio output is generated by the haptic device.
- a pulse-like, periodic control signal is configured to cause the haptic effects to be output having frequencies within each of the ranges shown in FIG. 4 . Examples of such signals are discussed in greater detail below.
- the period between pulse features of the control signal corresponds to a low-frequency component (e.g., a haptic envelope) of a haptic effect at lower frequencies. At lower frequencies, it is these low-frequency components (perceived as pulses) that are most easily tactilely detected by a user.
- the haptic device 116 reaches a state where it is moving almost the entire period, even during the portions of the period when no pulse in the control signal exists. Over increasing control signal frequencies where this begins to occur, the haptic device 116 is said to be operating in “saturation mode.” For example, in the case of a rotating-mass device, when the haptic device 116 reaches the saturation mode, despite the fact that the control signal pulses are not continuously on and, therefore, are causing low-frequency components in the haptic effect, the mass of the device continues to rotate.
- the saturation mode may or may not correspond to the diminished-sensitivity region, depending upon the physical characteristics of the haptic device 116 or other parameters.
- a user perceives the frequency of the haptic effect to increase due to an increase in the frequency of the audio output. This is illustrated, for example, by a line 530 extending from the magnitude versus frequency curve shown in FIG. 5 .
- This perceived change in frequency of the haptic effect due to the audio output can occur, for example, at the beginning of the diminished-sensitivity region (i.e., where it begins to be difficult for a user to tactilely detect variations in frequency).
- the perceived increased frequency of the haptic effect occurs when the haptic device is being driven within the mid-frequency range (i.e., a frequency between f L and f H ), as shown in FIG. 5 .
- the audio output to increase the frequency range a user perceives a haptic device to have allows a user to experience an increased perceived frequency range in the overall experience, and specifically the perceived haptic effect, without being limited by the performance range of the haptic device.
- multiple lines similar to the illustrated line 530 can be used to represent an audio output changing the frequency that a user perceives either within or outside of the diminished-sensitivity region.
- the line 530 representing a frequency perceived by a user indicates an essential constant perceived magnitude, it also is possible to change the magnitude perceived by a user, depending upon the audio output that is produced.
- control signals that can be used in connection with one or more embodiments of the invention are described in detail in copending U.S. patent application Ser. Nos. 09/669,029 and 10/671,465, each of which is incorporated by reference herein in its entirety.
- other control signals, as well as haptic devices that can be used in connection with one or more embodiments of the invention are described in detail in U.S. Pat. No. 6,275,213, which is incorporated by reference herein in its entirety.
- FIG. 6 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention.
- the control signal shown in FIG. 6 can be used to closely control the frequency of a haptic effect.
- the control signal shown in FIG. 6 is a series of pulses, each having a positive on-time.
- Changes in magnitude of a haptic effect caused by the control signal shown in FIG. 6 can be conveyed by proportional changes of magnitude of the pulses of that control signal.
- a change in magnitude of the output haptic effect that is based on the control signal shown in FIG. 6 varies proportionately to the change in magnitude of the control signal.
- the duration of the pulses i.e., the positive on-time or duty cycle
- Table 1 Duty cycle/on-time values for control signal of FIG.
- the frequency ranges shown in Table 1 above can correspond to the three ranges shown in FIG. 4 (with “other” being included in the high-frequency range), according to one or more embodiments of the invention.
- the duty cycle of the control signal is increased. For example, in the transition from 10 Hz to 100 Hz in the high-frequency range, the duty cycle increases from 50% to 100%. This increase in duty cycle can be a linearly increase, or another type of increase, if desired.
- the duration of the pulses (i.e., the positive on-time, or duty cycle) can alternatively be selected according to the values shown in Table 2 below to provide a strong haptic effect magnitude, depending upon the frequency range of the haptic effect being selected.
- Table 2 Duty cycle/on-time values for control signal of FIG. 6 Frequency Range Frequencies Duty Cycle/On-Time Low f D ⁇ 10 Hz 75 ms Mid 10 Hz ⁇ f D ⁇ 16 Hz 75% High 16 Hz ⁇ f D ⁇ 100 Hz 50% @ 10 Hz f D > 100 Hz 100% @ 100 Hz and above
- the frequency ranges shown in Table 2 above can also correspond to the three ranges shown in FIG. 4 (with “other” being included in the high-frequency range), according to one or more embodiments of the invention.
- the length of the duty cycle of the control signal is increased in the high-frequency range.
- the duty cycle can be increased linearly, or in some other desirable manner.
- FIG. 7 is a diagram of a control signal used to control a haptic device, according to another embodiment of the invention.
- the control signal shown in FIG. 7 is a bi-directional control signal that includes multiple bi-directional pulses, and is configured to create a haptic effect with a strong magnitude.
- These bi-directional pulses are periodic, and have a period corresponding to the desired frequency f D of the haptic effect to be output (defined by Equation 1 above). Seventy-five percent of the bi-directional pulse is a positive pulse portion, and twenty-five percent of the bi-directional pulse is a negative pulse portion. Values associated with the pulse size for various frequency ranges are shown below in Table 3. TABLE 3 Duty cycle values for control signal of FIG.
- the three frequency ranges can correspond, for example, to the three frequency ranges shown in FIG. 4 .
- the magnitude of the pulse is varied proportionally to the desired increase or decrease of magnitude. Haptic effects to be output in the high-frequency range cause a user to perceive that the magnitude and frequency change proportionally to any changes in the desired frequency of the control signal shown in FIG. 7 .
- FIG. 8 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention.
- the control signal shown in FIG. 8 is a magnitude-sweep control signal (also referred to as a “mag-sweep” signal), which sweeps through a variety of magnitude values to cause a desired haptic effect to be output.
- the magnitude-sweep control signal shown in FIG. 8 can cause a corresponding haptic effect to be output, or can be used to modulate another control signal (e.g., a periodic signal such as the signals shown in FIG. 6 and FIG. 7 ).
- a lead-in pulse is provided at the beginning of the control signal shown in FIG. 8 , which improves response time of the haptic device being controlled by the control signal.
- the duration t p of the pulse is varied as a function of the magnitude level of the beginning of the effect (e.g., at a level desired to begin the impulse portion of the signal, or the ramp-up portion of the signal). The smaller the magnitude levels at the beginning of the effect, the shorter the duration t p of the lead-in pulse and, similarly, the larger the magnitude levels at the beginning of the signal, the longer the duration t p of the lead-in pulse.
- the duration t p of the pulse can be varied according to the percentage of maximum control signal magnitude (e.g., the voltage magnitude) that one desires to reach by the end of the pulse (e.g., at the value indicated as “level” in FIG. 9 and denoted L in Table 4), as shown below in Table 4.
- % t P 0% ⁇ L ⁇ 47% 0 47% ⁇ L ⁇ 70% 25 ms 70% ⁇ L ⁇ 100% 50 ms
- FIG. 9 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention.
- the control signal shown in FIG. 9 is a magnitude sweep signal, similar to the control signal shown in FIG. 8 , but having both a lead-in pulse and a braking pulse. Because the braking pulse is a negative pulse, the control signal shown in FIG. 9 can also be considered a bi-directional signal.
- the braking pulse generally is executed to cause a large change in voltage to stop the actuation of a haptic device (i.e., to stop a device from outputting a haptic effect currently being output, such as stopping the rotation of a rotational haptic device).
- the braking pulse is of opposite polarity to the lead-in pulse, and the rest of the signal.
- the duration t b of the braking pulse varies as a function of the magnitude level of the signal at the end of the effect (i.e., immediately prior to initiating the braking pulse). The smaller the magnitude level at the end of the effect (i.e., at the end of the control signal shown in FIG. 9 ), the shorter the duration of the braking pulse that is required.
- Various lengths of possible braking pulses are shown below in Table 5 according to the corresponding percentage of voltage magnitude (e.g., the voltage magnitude) at the end of the signal (e.g., at the value indicated as “End level” in FIG. 10 and denoted EL in Table 5) shown in FIG. 9 .
- FIG. 10 is a diagram of linearization of voltages of a haptic device, according to an embodiment of the invention.
- the actuator of the haptic device may require increased voltage over the desired voltage to initiate and achieve the intended effects.
- FIG. 10 illustrates an example of a linearization table for such an actuator. As shown in the linearization table of FIG. 10 , the percentage of motor voltage to begin movement of the motor can be about twenty percent.
- Each of the control signals described above in connection with FIGS. 9-10 can be used with a haptic device that produces an audio output, in addition to outputting a haptic effect, when driven by the control signal.
- the audio output also can change. This can occur, for example, by varying a magnitude, frequency, and/or pulse of the control signal.
- the control signal can cause a user to sense a change in the frequency in the overall effect and to perceive a change in the haptic effect output by the haptic device.
- Table 6 shows the number of available controller input frequencies for different frequency ranges for the smooth controller.
- Table 7 shows the number of available controller input frequencies for different frequency ranges for the strong and sharp controllers.
- Table 8 shows motor inputs for various frequency ranges for smooth, strong and sharp controllers.
- Table 9 shows desired perceived frequency and perceived magnitude for various frequency ranges for smooth, strong and sharp controllers.
- Table 10 shows actual acceleration frequency, perceived frequency and perceived magnitude for various frequency ranges for smooth, strong and sharp controllers.
- Controller input frequencies for the smooth controller # of available # of available Controller Frequency frequencies at 200 Hz frequencies at 1 kHz Range Range sampling sampling Low ⁇ 6.6 Hz 170 between (1 Hz 850 between (1 Hz and 8 Hz) and 6.6 Hz) Transition 6.6-10 Hz 10 50 High 10-100 Hz 19 90
- Controller input frequencies for the strong and sharp controllers # of available # of available Controller Frequency frequencies at frequencies at 1 kHz Range Range 200 Hz sampling sampling Low ⁇ 10 Hz 180 between (1 Hz 900 between (1 Hz and 10 Hz) and 10 Hz Transition 10-16 Hz 8 37 High 16-100 Hz 11 52
- Motor input for various frequency ranges for smooth, strong and sharp controllers
- Motor input Motor input: Motor input: Controller Range Frequency Range Smooth Strong Sharp Low ⁇ 6.6 Hz Unidir Pulse Unidir Pulse @ Bidir Pulse @ Smooth @ 6.6 Hz with 10 Hz with 75% 10 Hz with ⁇ 10 Hz Strong, 50% duty duty cycle 75% duty cycle Sharp cycle Transition 6.6-10 Hz Unidir Pulse at Unidir Pulse at Bidir Pulse at Smooth chosen input chosen input 10-16 Hz, frequency at frequency at frequency at frequency at Strong Sharp 50% duty 75% duty cycle 75% duty cycle High 10-100 Hz Unidir Pulse at Unidir Pulse at Bidir Pulse at Smooth chosen input chosen input 16-100 Hz, frequency and frequency and frequency and Strong Sharp duty cycle duty cycle duty cycle increases from increases from increases from increases from increases from 50% to 100% 75% to 100% 75% to 100% 75% to 100% 75% to 100%
- Controller Frequency Desired Perceived Desired Perceived Range Range Frequency Magnitude Comments Low ⁇ 6.6 Hz Smooth ⁇ 10 Hz Strong, Sharp Controller input frequency matches, actual and perceived output frequency. Perceived magnitude is variable with PWM Transition 6.6-10 Hz Smooth 10-16 Hz Strong, Sharp Controller input frequency creates actual frequency with envelope. (If continuous spinning, then input frequency does not match output frequency.) Average energy delivered is the same. High 10-100 Hz Smooth 16-100 Hz, Strong Sharp Controller input frequency creates continuous spinning where that does not match output frequency Average energy delivered is increasing.
- FIG. 11 is a diagram of parameters associated with a smooth effect according to an embodiment of the invention.
- FIG. 12 is a diagram of parameters associated with a strong effect according to an embodiment of the invention.
- FIG. 13 is a diagram of parameters associated with a sharp effect according to an embodiment of the invention.
- a system and method for controlling audio output associated with haptic effects are discussed. Specific embodiments have been described above in connection with separately controlling multiple frequencies, either using a single controller or using multiple controllers, each of the multiple controllers being equally associated with a frequency range. Additionally, other embodiments have been discussed in connection with controlling an audio output associated with a haptic effect in at least one of the frequency ranges. Also, the audio effect can be changed, such that a user senses a change in frequency in the overall effect and perceives that change in a haptic effect; in some cases, this can occur above the frequency range where a user can tactilely detect variations in frequency (e.g., within a diminished-sensitivity region).
- the user perceives an increase in a frequency of the haptic effect associated with the audio effect, even where such an increase results in a change in haptic frequencies within the diminished-sensitivity region.
- the frequency of the audio effect is decreased, the user perceives a decrease in frequency of the corresponding haptic effect, even where such changes result in variations of haptic effect frequencies, which are undetectable to a user (e.g., within the diminished-sensitivity region).
Abstract
An apparatus, a processor-readable medium, and a method are provided that are configured to cause a haptic effect and an audio effect to be output substantially concurrently. The haptic effect has a frequency and the audio effect has a frequency different from the frequency of the haptic effect. At least one of the frequency of the haptic effect and the frequency of the audio effect is varied while maintaining substantially constant an average energy of the haptic effect. Varying the frequency of the audio effect can cause a perceived frequency of the haptic effect to change.
Description
- This application claims the benefit under 35 U.S.C. 119(e) of U.S. Provisional Application Ser. No. 60/590,432, entitled “System and Method for Controlling Audio Output Associated with Haptic Effects,” filed Jul. 23, 2004, which is incorporated herein by reference in its entirety
- The invention relates generally to haptic feedback devices. More specifically, the invention relates to a system and method for controlling audio output associated with haptic effects.
- Devices that provide haptics, such as tactile feedback, have enjoyed increased popularity in recent years. These devices are used in a variety of different applications. For example, devices providing haptics are popular in various applications, where the haptic feedback enhances the overall gaming experience of a user. For example, haptic-enabled controllers, such as mouse devices, can be configured to provide haptic feedback to a user while the user interacts with an operating system (OS), or other application.
- Existing devices, however, do not effectively control audio output associated with haptic feedback. Accordingly, it would be desirable to control effectively audio output associated with haptic effects.
-
FIG. 1 is a block diagram of a processor system, according to an embodiment of the invention. -
FIG. 2 is a diagram illustrating a haptic device, a controller, and a sensor, according to an embodiment of the invention. -
FIG. 3 is a block diagram of a haptic device, according to an embodiment of the invention. -
FIG. 4 is a diagram of multiple frequency ranges of haptic effects output by a haptic device, according to an embodiment of the invention. -
FIG. 5 is a plot of a magnitude versus frequency response of a haptic device, according to an embodiment of the invention. -
FIG. 6 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention. -
FIG. 7 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention. -
FIG. 8 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention. -
FIG. 9 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention -
FIG. 10 is a diagram of linearization of voltages of a haptic device, according to an embodiment of the invention. -
FIG. 11 is a diagram of various parameters associated with a smooth effect according to an embodiment of the invention. -
FIG. 12 is a diagram of various parameters associated with a strong effect according to an embodiment of the invention. -
FIG. 13 is a diagram of various parameters associated with a sharp effect according to an embodiment of the invention. - A system and method for controlling audio output associated with haptic effects are described. More specifically, audio output associated with haptic effects can be controlled to modify a perceived experience of the haptic effects. For example, by modifying the audio output, a user can be made to perceive a frequency of a haptic effect as being different than the actual frequency.
- According to one or more embodiments of the invention, control signals can be configured to cause haptic effects to be output across a wide range of frequencies. These control signals can independently control haptic effects within any frequency range from among multiple frequency ranges. This can occur, for example, using either a single controller or multiple controllers configured to output control signals from each frequency range. For example, a single controller can output control signals that independently control haptic effects in each of multiple frequency ranges. Alternatively, multiple controllers can be used, such that each controller outputs control signals within a single frequency range of multiple frequency ranges, each controller being uniquely associated with each frequency range.
- Audio output associated with a haptic effect is generated in at least one frequency range of multiple frequency ranges when that haptic effect is output in response to a corresponding control signal. For example, when a haptic effect is output by a haptic device in response to a control signal, the haptic device can also create audible sound or, in other words, an audio output based on the movement of the haptic device. The audio signal heard by a user can correspond to a frequency of a haptic effect that is beyond the tactile detection capabilities of the user. In other words, although a user cannot feel a difference in the frequency of a haptic effect above a certain threshold frequency, the user can hear audio associated with such higher frequencies. Thus, although varying such tactile-imperceptible frequencies will not cause a user to feel a difference in a frequency of a haptic effect, the user will be able to hear such a variation. Because a user can hear an increase or decrease in frequency of the audio output, the user will perceive that the haptic effect has changed, and in many cases will believe that he or she has felt the change in the overall experience.
- One or more embodiments of the invention provide an extended perceived frequency range of haptic effects. More specifically, in addition to the range of haptic effects that can be tactilely detected by the user, a range of effects that are detected audibly by a user can be can be added such that the perceived overall experience has a greater frequency range. Because a user is able to sense an increased range of frequencies, more information can be communicated to the user using such combination of haptic effects and audio output.
- When pulse-like control signals are used to generate haptic effects, control signals having a constant average energy can be used to provide a variety of different audio output frequencies. The different audio output frequencies can cause a user to believe that he or she is sensing tactilely a different frequency of a haptic effect, even though tactilely sensing such a difference would not be possible. Thus, according to one or more embodiments of the invention, a variety of overall experiences (each having a haptic component and an audio component) caused by a corresponding variety of control signals, each of which has the same average energy, is able to be sensed by a user via a combination of the haptic effect and the associated audio output, even though the variety of haptic effects alone would be perceived as having the same feel without the audio output. The average energy can be maintained constant by varying the frequency and/or duty cycle of a control signal inversely with the magnitude of a control signal. Thus, as the frequency of the control signal is increased, the magnitude decreases and, conversely, as the frequency of the control signal decreases, the magnitude increases, to maintain a constant average energy of the carrier signal.
-
FIG. 1 is a block diagram of a processor system, according to an embodiment of the invention. Theprocessor system 110 illustrated inFIG. 1 can be, for example, a commercially available personal computer, portable electronic device, or a less complex computing or processing device (e.g., a device that is dedicated to performing one or more specific tasks). For example, the processor system can be a mobile telephone, a PDA, a portable gaming system, an MP3 player, or the like. Alternatively, theprocessor system 110 can be a terminal dedicated to providing an interactive virtual reality environment, such as a gaming system, or the like. Although each component of theprocessor system 110 is shown as being a single component inFIG. 1 , theprocessor system 110 can include multiple numbers of any components illustrated inFIG. 1 . Additionally, multiple components of theprocessor system 1 10 can be combined as a single component. - The
processor system 110 includes aprocessor 112, which according to one or more embodiments of the invention, can be a commercially available microprocessor capable of performing general processing operations. Alternatively, theprocessor 112 can be an application-specific integrated circuit (ASIC) or a combination of ASICs, which is designed to achieve one or more specific functions, or enable one or more specific devices or applications. In yet another alternative, theprocessor 112 can be an analog or digital circuit, or a combination of multiple circuits. - Alternatively, the
processor 112 can optionally include one or more individual sub-processors or coprocessors. For example, the processor can include a graphics coprocessor that is capable of rendering graphics, a math coprocessor that is capable of efficiently performing complex calculations, a controller that is capable of controlling one or more devices, a sensor interface that is capable of receiving sensory input from one or more sensing devices, and so forth. - The
processor system 110 can also include amemory component 114. As shown inFIG. 1 , thememory component 114 can include one or more types of memory. For example, thememory component 114 can include a read only memory (ROM)component 114 a and a random access memory (RAM)component 114 b. Thememory component 114 can also include other types of memory not illustrated inFIG. 1 that are suitable for storing data in a form retrievable by theprocessor 112. For example, electronically programmable read only memory (EPROM), erasable electrically programmable read only memory (EEPROM), flash memory, as well as other suitable forms of memory can be included within thememory component 114. Theprocessor system 110 can also include a variety of other components, depending upon the desired functionality of theprocessor system 110. Theprocessor 112 is in communication with thememory component 114, and can store data in thememory component 114 or retrieve data previously stored in thememory component 114. - The
processor system 110 can also include ahaptic device 116, which is capable of providing a variety of haptic output. For example, thehaptic device 116 can be configured to output basis haptic effects, such as periodic effects, magnitude-sweep effects, or timeline haptic effects, each of which is described in greater detail below. According to one or more embodiments of the invention, thehaptic device 116 can include one or more force-applying mechanisms, which are capable of outputting haptic effects or force, to a user of the processor system 110 (e.g., via the housing of the processor system 110). These effects or forces can be transmitted, for example, in the form of vibrational movement caused by the haptic device 116 (e.g., caused by a rotating mass, a piezo-electric device, or other vibrating actuator), or in the form of resistive force caused by thehaptic device 116. - The
processor system 110 can also, according to one or more embodiments of the invention, include asensor 118 that is capable of receiving input from a user, thehaptic device 116, or is otherwise capable of sensing one or more physical parameters. For example, according to one or more embodiments of the invention, asensor 118 can be configured to measure speed, intensity, acceleration, or other parameters associated with a haptic effect output by thehaptic device 116. Similarly, thesensor 118 can be configured to sense environmental or ambient conditions of the processor system's surroundings. Thesensor 118 can interface and communicate with theprocessor 112 by way of a sensor interface (not shown) within theprocessor 112. - The
processor system 110 can also include acontroller 120, which can optionally be internal to theprocessor 112, or external thereto, as shown inFIG. 1 . Thecontroller 120 can be configured to control thehaptic device 116 when theprocessor 112 is not directly controlling thehaptic device 116. Similarly, thecontroller 120 can control thememory 114 and/or thesensor 118, as well as devices external to theprocessor system 110 by way of an input/output (I/O) component 124 (described below). - The various components of the
processor system 110 can communicate with one another via abus 122, which is capable of carrying instructions from theprocessor 112 and/or thecontroller 120 to other components, and which is capable of carrying data between the various components of theprocessor system 110. Additionally, signals received via thesensor 118 can be communicated to theprocessor 112 or thecontroller 120 by way of thebus 122. Data retrieved from or written tomemory 114 can be carried by thebus 122, as are instructions to thehaptic device 116. Instructions to thehaptic device 116 can be provided in the form of haptic-effect signals (e.g., basis haptic-effect signals), for example, which can be provided by theprocessor 112, thecontroller 120, or devices external to theprocessor system 110. - The components of the
processor system 110 can communicate with devices external to theprocessor system 110 by way of an input/output (I/O) component 124 (accessed via the bus 122). According to one or more embodiments of the invention, the I/O component 124 can include a variety of suitable communication interfaces. For example, the I/O component 124 can include, for example, wireless connections, such as infrared ports, optical ports, Bluetooth wireless ports, wireless LAN ports, or the like. Additionally, the I/O component 124 can include, wired connections, such as standard serial ports, parallel ports, universal serial bus (USB) ports, S-video ports, large area network (LAN) ports, small computer system interface (SCSI) ports, and so forth. -
FIG. 2 is a diagram illustrating a haptic device, a controller, and a sensor, according to an embodiment of the invention.FIG. 2 also shows data values provided to the system (e.g.,user input 202 and control parameters 204). The elements shown inFIG. 2 can be used with theprocessor system 110, or a similar device. - The
controller 120 is configured to output control signals that are configured to cause haptic effects to be output by thehaptic device 116. As shown inFIG. 2 ,user input 202 can optionally be provided (e.g., via the I/O component 124 shown inFIG. 1 ) and/or received by anoptional sensor 118. Theuser input 202 can also optionally be provided directly to a controller 120 (e.g., by way of thesensor 118, or some other devices configured to accept and convey user input). Thesensor 118 can also optionally receive information from thehaptic device 116. For example, thesensor 118 can sense the actual movements of thehaptic device 116. - According to an arrangement of the system shown in
FIG. 2 , thecontroller 120 can optionally receive data from thesensor 118, and can optionally receiveuser input 202 andcontrol parameters 204. Based on any data received from thesensor 118, any receiveduser input 202, and/or any receivedcontrol parameters 204, thecontroller 120 controls the haptic output of the haptic device 116 (e.g., thecontroller 120 sends control signals configured to cause haptic effects). For example, thecontroller 120 can execute a feedback algorithm, controlling thehaptic device 116 based on feedback received from thehaptic device 116. Thecontroller 120 controls the output of thehaptic device 116 by a control signal that thecontroller 120 outputs to thehaptic device 116. - The control signal output by the
controller 120 can be based on a number of parameters, including, for example,control parameters 204. For example,control parameters 204 and other parameters that can be used by thecontroller 120 to control thehaptic device 116 can be stored in thememory component 114 of theprocessor system 110, or by another suitable memory component. For example, thecontrol parameters 204 can include input from an electronic system, a portable gaming device, a cellular telephone, or the like. According to one or more embodiments of the invention, the controller receives control parameters (e.g., gaming device input, cellular telephone input, etc.), and does not include a sensor. According to such embodiments, user input can optionally be received directly by the controller, or can be omitted entirely, depending upon the desired function of the system in which the controller is used. - According to one or more embodiments of the invention, the system shown in
FIG. 2 can be used in a stand-alone device, such as a mobile telephone, portable electronic device (e.g., a PDA, etc.), or other device. In a mobile telephone embodiment, for example, haptic output can be provided in the form of haptic effects via thehaptic device 116 in response to status events (e.g., a message received signal, a network indicator signal, etc.), user input (e.g., mode changes, keypad dialing, option selections, etc.), incoming calls, or other events. Alternatively, the system shown inFIG. 2 can be used in a configuration where a processor, such as theprocessor 112 of theprocessor system 110 shown inFIG. 1 , can be connected to an external device, and the processing tasks can be divided among the devices, as desired. - The
controller 120 can generate a variety of different control signals to drive thehaptic device 116, several of which will be described in greater detail below. For example, thecontroller 120 can send a control signal to thehaptic device 116, which is configured to cause thehaptic device 116 to output a corresponding haptic effect. Examples of such control signals include, pulse width modulation (PWM) signals (e.g., pulse signals having a given duty cycle), sinusoidal signals, and other periodic signals (e.g., triangle waves, square waves, etc.). Additionally, thecontroller 120 can modulate control signals using one or more haptic envelopes. - The
controller 120 also can be configured to provide a lead-in pulse at the beginning of a control signal, and/or a braking pulse, at the end of a control signal, which are configured to decrease response time of thehaptic device 116. For example, the lead-in signal reduces the time for thehaptic device 116 to initiate outputting a haptic effect associated with the control signal. The braking pulse, on the other hand, decreases the time it takes for thehaptic device 116 to cease a haptic effect currently being output. In addition to signals described above, such as periodic signals, thecontroller 120 can output a variety of other control signals, such as non-periodic signals, that are configured to cause thehaptic device 116 to output haptic effects. -
FIG. 3 is a block diagram of ahaptic device 116 shown inFIGS. 1 and 2 . As shown inFIG. 3 , thehaptic device 116 includes anactuator 302, anelastic member 304 and amass 306. Thehaptic device 116 is configured to provide haptic feedback. Theactuator 302 is operably connected to theelastic member 304, and theelastic member 304 is operably connected to themass 306. Theactuator 302 can include, for example, a motor (e.g., a brush motor, a brushless motor, etc.). The elastic member can provide a desired amount of coupling rigidity between the actuator and themass 306. - When control signals are received by the
haptic device 116, theactuator 302 provides force to theelastic member 304. Some of the force applied to theelastic member 304 is translated to themass 306, and causes themass 306 to move. By causing themass 306 to move, haptic effects commanded by the control signals are output by the haptic device, and can be output to a user. Theactuator 302 can be configured, for example, to cause the mass to rotate in response to the control signals received by the haptic device. Alternatively, the actuator can move themass 306 in other directions (e.g., vibrating the mass, moving the mass laterally, etc.). - The configuration shown in
FIG. 3 is only one example of a configuration of ahaptic device 116. Other configurations that vary from the configuration shown inFIG. 3 can be used as ahaptic device 116, according to one or more embodiments of the invention. For example, theelastic member 304 can be coupled to themass 306 by a flexible coupling; theelastic member 304 can be coupled to theactuator 302 by a flexible coupling. In an alternative embodiment, the elastic member can be coupled between actuator and a mechanical ground, and the actuator can be directly coupled to the actuator. Examples of haptic devices that can be used in connection with one or more embodiments of the invention include an eccentric-rotating-mass (ERM) haptic device and a harmonic ERM (HERM) haptic device, described in detail in copending U.S. patent application Ser. No. 10/301,809, which is incorporated by reference herein in its entirety. -
FIG. 4 is a diagram of multiple frequency ranges of haptic effects that can be output by ahaptic device 116, according to an embodiment of the invention. A low-frequency range extends from approximately DC (i.e., 0 Hz) to a low-frequency limit fL, which can vary depending upon the control signal being used to cause a haptic effect and the desired characteristics of the haptic effect. A mid-frequency range extends from the low-frequency threshold frequency fL to a high-frequency threshold frequency fH, which can vary depending upon the control signal being used to cause a haptic effect and the desired characteristics of the haptic effect. A high-frequency range extends from the high-frequency threshold frequency fH to all higher frequencies. - According to one or more embodiments of the invention, at least one frequency range from the frequency ranges shown in
FIG. 4 can have an audio output associated with the haptic effect. The audio output can occur for haptic effects having a frequency within the at least one frequency range or for haptic effects having a frequency beyond the at least one frequency range, depending upon the desired performance of the system. - For example, according to one or more embodiments of the invention, the mid-frequency range shown in
FIG. 4 can have an audio output associated with a haptic effect having a frequency within the mid-frequency range. The haptic effect having a frequency within the mid-frequency range can be varied in a manner such that the associated audio output varies, while any changes in the frequency of the haptic effect remains tactilely undetectable to a user. Because the audio output varies (e.g., changes frequency of the audio output), a user aurally detects the change in the audio output, and believes that he or she has tactilely detected a change in the haptic effect. Said another way, by varying the audio output, the user may perceive that the overall effect (the combination of a haptic effect and an audio effect) has changed and attribute such a change, at least in part, to the user's tactile experience. Additionally, in one or more embodiments, the average energy of a control signal used to cause the haptic effect to be output can be maintained substantially the same while the associated audio output is varied, causing a user to detect an increase in the audio output and believe that he or she has tactilely perceived a change in the haptic effect. - Although
FIG. 4 illustrates only three frequency ranges, the number of frequency ranges used according to one or more embodiments of the invention can be varied. For example, many more frequency ranges can be used, among which multiple frequency ranges can include an audio output associated with the haptic effects having frequencies within those frequency ranges. - Haptic effects having frequencies within each of the frequency ranges shown in
FIG. 4 can be separately controlled. This can occur, for example, using a single controller, that separately controls the haptic effects associated with each of the frequency ranges shown inFIG. 4 . Alternatively, each frequency range shown inFIG. 4 can have a uniquely associated controller, which is configured to control haptic effects having frequencies within that frequency range. -
FIG. 5 is a plot of a magnitude versus frequency of a haptic effect, according to an embodiment of the invention. The plot shown inFIG. 5 is not drawn to scale, and is intended only as an example of the correlation between the magnitude and frequency of a haptic effect, and how a user perceives them. The magnitude versus frequency response shown inFIG. 5 indicates that, as the frequency of a haptic effect is increased, the magnitude of that haptic effect appears also to increase to a user. Afirst portion 510 of the curve shown inFIG. 5 represents a region of haptic effect frequencies within which a user can detect changes in frequency. Within some region of frequencies (e.g., beginning near the high-frequency threshold frequency fH in the plot ofFIG. 5 ), referred to as a diminished-sensitivity region, a user perceives some increases in the frequency of a haptic effect as increases in magnitude (and not the frequency) without being able to detect tactilely the increases in frequency. Asecond portion 520 of the curve shown inFIG. 5 represents this region, where the user has difficulty tactilely detecting changes in frequency. Near where the diminished-sensitivity region begins (i.e., the area of intersection of thefirst portion 510 and thesecond portion 520 of the curve shown inFIG. 5 ), a user's perception of increasing frequencies of combination of a haptic effect and audio output is illustrated using aline 520. Similar lines can be drawn to illustrate a user's perception of increasing frequencies of a combination of a haptic effect and an audio output is generated by the haptic device. - According to one or more embodiments of the invention, a pulse-like, periodic control signal is configured to cause the haptic effects to be output having frequencies within each of the ranges shown in
FIG. 4 . Examples of such signals are discussed in greater detail below. Generally speaking, the period between pulse features of the control signal corresponds to a low-frequency component (e.g., a haptic envelope) of a haptic effect at lower frequencies. At lower frequencies, it is these low-frequency components (perceived as pulses) that are most easily tactilely detected by a user. As the period between the pulses of the control signal decreases (i.e., the frequency of the pulses increases), thehaptic device 116 reaches a state where it is moving almost the entire period, even during the portions of the period when no pulse in the control signal exists. Over increasing control signal frequencies where this begins to occur, thehaptic device 116 is said to be operating in “saturation mode.” For example, in the case of a rotating-mass device, when thehaptic device 116 reaches the saturation mode, despite the fact that the control signal pulses are not continuously on and, therefore, are causing low-frequency components in the haptic effect, the mass of the device continues to rotate. The saturation mode may or may not correspond to the diminished-sensitivity region, depending upon the physical characteristics of thehaptic device 116 or other parameters. - When an audio output associated with a haptic effect is output at the same time as the haptic effect, a user perceives the frequency of the haptic effect to increase due to an increase in the frequency of the audio output. This is illustrated, for example, by a
line 530 extending from the magnitude versus frequency curve shown inFIG. 5 . This perceived change in frequency of the haptic effect due to the audio output can occur, for example, at the beginning of the diminished-sensitivity region (i.e., where it begins to be difficult for a user to tactilely detect variations in frequency). According to one or more embodiments of the invention, the perceived increased frequency of the haptic effect occurs when the haptic device is being driven within the mid-frequency range (i.e., a frequency between fL and fH), as shown inFIG. 5 . Using the audio output to increase the frequency range a user perceives a haptic device to have allows a user to experience an increased perceived frequency range in the overall experience, and specifically the perceived haptic effect, without being limited by the performance range of the haptic device. Although not shown, multiple lines similar to the illustratedline 530 can be used to represent an audio output changing the frequency that a user perceives either within or outside of the diminished-sensitivity region. Also, although theline 530 representing a frequency perceived by a user indicates an essential constant perceived magnitude, it also is possible to change the magnitude perceived by a user, depending upon the audio output that is produced. - Several signals are described below in greater detail. These signals are only examples, however, and it should be recognized that there are many other signals that are suitable for acting as control signals, depending upon the desired haptic effects to be output and audio output to be produced. Examples of control signals that can be used in connection with one or more embodiments of the invention are described in detail in copending U.S. patent application Ser. Nos. 09/669,029 and 10/671,465, each of which is incorporated by reference herein in its entirety. Similarly, other control signals, as well as haptic devices that can be used in connection with one or more embodiments of the invention are described in detail in U.S. Pat. No. 6,275,213, which is incorporated by reference herein in its entirety.
-
FIG. 6 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention. The control signal shown inFIG. 6 can be used to closely control the frequency of a haptic effect. The control signal shown inFIG. 6 is a series of pulses, each having a positive on-time. The pulses are periodic, having a period that corresponds to the desired frequency of the haptic effect to be output, as defined byEquation 1 below:
where T is the period of the control signal (i.e., the time period between two adjacent pulses of the control signal), and fD is the desired frequency of the output of the haptic effect. - Changes in magnitude of a haptic effect caused by the control signal shown in
FIG. 6 can be conveyed by proportional changes of magnitude of the pulses of that control signal. A change in magnitude of the output haptic effect that is based on the control signal shown inFIG. 6 varies proportionately to the change in magnitude of the control signal. The duration of the pulses (i.e., the positive on-time or duty cycle) can be selected according to the values shown in Table 1 below to provide a distinct frequency pattern, depending upon the frequency range of the haptic effect being selected.TABLE 1 Duty cycle/on-time values for control signal of FIG. 6 Frequency Range Frequencies Duty Cycle/On-Time Low fD ≦ 6.66 Hz 75 ms Mid 6.66 Hz < fD ≦ 10 Hz 50% High 10 Hz < fD ≦ 100 Hz 50% @ 10 Hz fD > 100 H 100% @ 100 Hz and above - The frequency ranges shown in Table 1 above can correspond to the three ranges shown in
FIG. 4 (with “other” being included in the high-frequency range), according to one or more embodiments of the invention. To achieve higher desired frequencies fD of a haptic effect, the duty cycle of the control signal is increased. For example, in the transition from 10 Hz to 100 Hz in the high-frequency range, the duty cycle increases from 50% to 100%. This increase in duty cycle can be a linearly increase, or another type of increase, if desired. - The duration of the pulses (i.e., the positive on-time, or duty cycle) can alternatively be selected according to the values shown in Table 2 below to provide a strong haptic effect magnitude, depending upon the frequency range of the haptic effect being selected.
TABLE 2 Duty cycle/on-time values for control signal of FIG. 6 Frequency Range Frequencies Duty Cycle/On-Time Low fD ≦ 10 Hz 75 ms Mid 10 Hz < fD ≦ 16 Hz 75% High 16 Hz < fD ≦ 100 Hz 50% @ 10 Hz fD > 100 Hz 100% @ 100 Hz and above - The frequency ranges shown in Table 2 above can also correspond to the three ranges shown in
FIG. 4 (with “other” being included in the high-frequency range), according to one or more embodiments of the invention. To achieve greater magnitude of a haptic effect, the length of the duty cycle of the control signal is increased in the high-frequency range. As discussed above, the duty cycle can be increased linearly, or in some other desirable manner. -
FIG. 7 is a diagram of a control signal used to control a haptic device, according to another embodiment of the invention. The control signal shown inFIG. 7 is a bi-directional control signal that includes multiple bi-directional pulses, and is configured to create a haptic effect with a strong magnitude. These bi-directional pulses are periodic, and have a period corresponding to the desired frequency fD of the haptic effect to be output (defined byEquation 1 above). Seventy-five percent of the bi-directional pulse is a positive pulse portion, and twenty-five percent of the bi-directional pulse is a negative pulse portion. Values associated with the pulse size for various frequency ranges are shown below in Table 3.TABLE 3 Duty cycle values for control signal of FIG. 7 Frequency Range Frequencies Duty Cycle/On-Time Low fD ≦ 10 Hz 10 Hz (75% V+ pulse, 25% V− pulse) Mid 10 Hz < fD ≦ 16 Hz 75% High 16 Hz < fD ≦ 100 Hz 75% @ 16 Hz fD > 100 Hz 100% @ 100 Hz and above - In Table 3, the three frequency ranges can correspond, for example, to the three frequency ranges shown in
FIG. 4 . To vary the magnitude of a haptic effect, the magnitude of the pulse is varied proportionally to the desired increase or decrease of magnitude. Haptic effects to be output in the high-frequency range cause a user to perceive that the magnitude and frequency change proportionally to any changes in the desired frequency of the control signal shown inFIG. 7 . -
FIG. 8 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention. The control signal shown inFIG. 8 is a magnitude-sweep control signal (also referred to as a “mag-sweep” signal), which sweeps through a variety of magnitude values to cause a desired haptic effect to be output. The magnitude-sweep control signal shown inFIG. 8 can cause a corresponding haptic effect to be output, or can be used to modulate another control signal (e.g., a periodic signal such as the signals shown inFIG. 6 andFIG. 7 ). - A lead-in pulse is provided at the beginning of the control signal shown in
FIG. 8 , which improves response time of the haptic device being controlled by the control signal. The duration tp of the pulse is varied as a function of the magnitude level of the beginning of the effect (e.g., at a level desired to begin the impulse portion of the signal, or the ramp-up portion of the signal). The smaller the magnitude levels at the beginning of the effect, the shorter the duration tp of the lead-in pulse and, similarly, the larger the magnitude levels at the beginning of the signal, the longer the duration tp of the lead-in pulse. The duration tp of the pulse can be varied according to the percentage of maximum control signal magnitude (e.g., the voltage magnitude) that one desires to reach by the end of the pulse (e.g., at the value indicated as “level” inFIG. 9 and denoted L in Table 4), as shown below in Table 4.TABLE 4 Duration tP of the lead-in pulse in FIG. 8 |Maximum Voltage| % t P 0% ≦ L ≦ 47% 0 47% ≦ L ≦ 70% 25 ms 70% ≦ L ≦ 100% 50 ms -
FIG. 9 is a diagram of a control signal used to control a haptic device, according to an embodiment of the invention. The control signal shown inFIG. 9 is a magnitude sweep signal, similar to the control signal shown inFIG. 8 , but having both a lead-in pulse and a braking pulse. Because the braking pulse is a negative pulse, the control signal shown inFIG. 9 can also be considered a bi-directional signal. The braking pulse generally is executed to cause a large change in voltage to stop the actuation of a haptic device (i.e., to stop a device from outputting a haptic effect currently being output, such as stopping the rotation of a rotational haptic device). - The braking pulse is of opposite polarity to the lead-in pulse, and the rest of the signal. The duration tb of the braking pulse varies as a function of the magnitude level of the signal at the end of the effect (i.e., immediately prior to initiating the braking pulse). The smaller the magnitude level at the end of the effect (i.e., at the end of the control signal shown in
FIG. 9 ), the shorter the duration of the braking pulse that is required. Various lengths of possible braking pulses are shown below in Table 5 according to the corresponding percentage of voltage magnitude (e.g., the voltage magnitude) at the end of the signal (e.g., at the value indicated as “End level” inFIG. 10 and denoted EL in Table 5) shown inFIG. 9 .TABLE 5 Duration tb of the braking pulse in FIG. 9 |Maximum Voltage| % t b 0% ≦ EL ≦ 47% 0 47% ≦ EL ≦ 70% 25 ms 70% ≦ EL ≦ 100% 50 ms -
FIG. 10 is a diagram of linearization of voltages of a haptic device, according to an embodiment of the invention. The actuator of the haptic device may require increased voltage over the desired voltage to initiate and achieve the intended effects.FIG. 10 illustrates an example of a linearization table for such an actuator. As shown in the linearization table ofFIG. 10 , the percentage of motor voltage to begin movement of the motor can be about twenty percent. - Each of the control signals described above in connection with
FIGS. 9-10 can be used with a haptic device that produces an audio output, in addition to outputting a haptic effect, when driven by the control signal. As the control signal changes the rate of movement associated with such haptic devices, the audio output also can change. This can occur, for example, by varying a magnitude, frequency, and/or pulse of the control signal. Thus, by using the control signal to vary the audio output produced by a haptic device, the control signal can cause a user to sense a change in the frequency in the overall effect and to perceive a change in the haptic effect output by the haptic device. - Additional information and examples regarding control signals according to one or more embodiments of the invention are illustrated below in Tables 6-10. For example, Table 6 shows the number of available controller input frequencies for different frequency ranges for the smooth controller. Table 7 shows the number of available controller input frequencies for different frequency ranges for the strong and sharp controllers. Table 8 shows motor inputs for various frequency ranges for smooth, strong and sharp controllers. Table 9 shows desired perceived frequency and perceived magnitude for various frequency ranges for smooth, strong and sharp controllers. Table 10 shows actual acceleration frequency, perceived frequency and perceived magnitude for various frequency ranges for smooth, strong and sharp controllers.
TABLE 6 Controller input frequencies for the smooth controller # of available # of available Controller Frequency frequencies at 200 Hz frequencies at 1 kHz Range Range sampling sampling Low <6.6 Hz 170 between (1 Hz 850 between (1 Hz and 8 Hz) and 6.6 Hz) Transition 6.6-10 Hz 10 50 High 10-100 Hz 19 90 -
TABLE 7 Controller input frequencies for the strong and sharp controllers # of available # of available Controller Frequency frequencies at frequencies at 1 kHz Range Range 200 Hz sampling sampling Low <10 Hz 180 between (1 Hz 900 between (1 Hz and 10 Hz) and 10 Hz Transition 10-16 Hz 8 37 High 16-100 Hz 11 52 -
TABLE 8 Motor input for various frequency ranges for smooth, strong and sharp controllers Motor input: Motor input: Motor input: Controller Range Frequency Range Smooth Strong Sharp Low <6.6 Hz Unidir Pulse Unidir Pulse @ Bidir Pulse @ Smooth @ 6.6 Hz with 10 Hz with 75% 10 Hz with <10 Hz Strong, 50% duty duty cycle 75% duty cycle Sharp cycle Transition 6.6-10 Hz Unidir Pulse at Unidir Pulse at Bidir Pulse at Smooth chosen input chosen input chosen input 10-16 Hz, frequency at frequency at frequency at Strong Sharp 50 % duty 75 % duty cycle 75% duty cycle cycle High 10-100 Hz Unidir Pulse at Unidir Pulse at Bidir Pulse at Smooth chosen input chosen input chosen input 16-100 Hz, frequency and frequency and frequency and Strong Sharp duty cycle duty cycle duty cycle increases from increases from increases from 50% to 100% 75% to 100% 75% to 100% -
TABLE 9 Desired perceived frequency and perceived magnitude for various frequency ranges for smooth, strong and sharp controllers Controller Frequency Desired Perceived Desired Perceived Range Range Frequency Magnitude Comments Low <6.6 Hz Smooth <10 Hz Strong, Sharp Controller input frequency matches, actual and perceived output frequency. Perceived magnitude is variable with PWM Transition 6.6-10 Hz Smooth 10-16 Hz Strong, Sharp Controller input frequency creates actual frequency with envelope. (If continuous spinning, then input frequency does not match output frequency.) Average energy delivered is the same. High 10-100 Hz Smooth 16-100 Hz, Strong Sharp Controller input frequency creates continuous spinning where that does not match output frequency Average energy delivered is increasing. -
TABLE 10 Actual acceleration frequency, perceived frequency and perceived magnitude for various frequency ranges for smooth, strong and sharp controllers Controller Input Actual Frequency Acceleration Perceived Perceived Controller Range Range frequency frequency Magnitude Low <6.6 Hz Smooth Controller input Controller input Perceived <10 Hz Strong, frequency frequency magnitude is Sharp matches, actual. matches variable with perceived. PWM Transition 6.6-10 Hz Controller input User perceives Perceived Smooth frequency envelope magnitude is 10-16 Hz matches frequency variable with Strong, Sharp envelope PWM. frequency. High 10-100 Hz Controller input Controller input As controller Smooth frequency does frequency does input frequency 16-100 Hz, NOT match NOT match is increased, Strong Sharp actual perceived magnitude is frequency. frequency. perceived to increase. -
FIG. 11 is a diagram of parameters associated with a smooth effect according to an embodiment of the invention.FIG. 12 is a diagram of parameters associated with a strong effect according to an embodiment of the invention.FIG. 13 is a diagram of parameters associated with a sharp effect according to an embodiment of the invention. - A system and method for controlling audio output associated with haptic effects are discussed. Specific embodiments have been described above in connection with separately controlling multiple frequencies, either using a single controller or using multiple controllers, each of the multiple controllers being equally associated with a frequency range. Additionally, other embodiments have been discussed in connection with controlling an audio output associated with a haptic effect in at least one of the frequency ranges. Also, the audio effect can be changed, such that a user senses a change in frequency in the overall effect and perceives that change in a haptic effect; in some cases, this can occur above the frequency range where a user can tactilely detect variations in frequency (e.g., within a diminished-sensitivity region). Thus, as the frequency of the audio effect is increased, the user perceives an increase in a frequency of the haptic effect associated with the audio effect, even where such an increase results in a change in haptic frequencies within the diminished-sensitivity region. Similarly, as the frequency of the audio effect is decreased, the user perceives a decrease in frequency of the corresponding haptic effect, even where such changes result in variations of haptic effect frequencies, which are undetectable to a user (e.g., within the diminished-sensitivity region).
- It will be appreciated, however, that embodiments of the invention can be in other specific forms without departing from the spirit or essential characteristics thereof. For example, while some embodiments have been described in the context of periodic or magnitude sweep control signals for causing haptic effects, any suitable signal can be used. Also, although control signals have been described as square-waves or PWM signals having square-wave-like shapes, other pulse shapes can be used. Additionally, although a specific reference has been made to devices configured to output periodic haptic effect (e.g., rotating haptic devices such spinning mass motors, etc.), any type of haptic device capable of outputting haptic effects associated with an audio output can be used according to one or more embodiments of the invention.
- The presently disclosed embodiments are, therefore, considered in all respects to be illustrative and not restrictive.
Claims (37)
1. A processor-readable medium comprising code representing instructions to cause a processor to:
send a signal configured to cause a haptic effect and an audio effect to be output substantially concurrently, the haptic effect having a frequency and the audio effect having a frequency different from the frequency of the haptic effect, the signal being further configured to vary at least one of the frequency of the haptic effect and the frequency of the audio effect while maintaining substantially constant an average energy of the haptic effect.
2. The processor-readable medium of claim 1 , wherein the signal is further configured to cause the frequency of the audio effect to vary while causing the frequency of the haptic effect to remain substantially constant.
3. The processor-readable medium of claim 1 , wherein the signal includes a plurality of pulses, the signal being configured to cause a frequency of the plurality of pulses to vary while causing a magnitude of the plurality of pulses to vary inversely.
4. The processor-readable medium of claim 1 , the frequency of the haptic effect being a first frequency of the haptic effect, wherein the audio effect is configured to cause a user to perceive the haptic effect as having a second frequency different from the first frequency of the haptic effect.
5. The processor-readable medium of claim 1 , the frequency of the haptic effect being a first frequency of the haptic effect, wherein the audio effect is configured to cause a user to perceive the haptic effect as having a second frequency different from the first frequency of the haptic effect, the second frequency being higher than the first frequency.
6. The processor-readable medium of claim 1 , wherein the frequency of the audio effect is higher than the frequency of the haptic effect.
7. The processor-readable medium of claim 1 , wherein the haptic effect has a frequency from a frequency range from a plurality of frequency ranges, each frequency range from the plurality of frequency ranges being uniquely associated with a control scheme.
8. An apparatus, comprising:
a controller configured to output a control signal, the control signal being configured to cause a haptic effect and an audio effect to be output substantially concurrently, the haptic effect having a frequency and the audio effect having a frequency different from the frequency of the haptic effect, the control signal being further configured to vary at least one of the frequency of the haptic effect and the frequency of the audio effect while maintaining substantially constant an average energy of the haptic effect; and
an interface component coupled to the controller and configured to be coupled to a component external to the controller, the interface component configured to provide a haptic instruction to the component based at least partially on the control signal.
9. The apparatus of claim 8 , wherein the control signal is further configured to cause the frequency of the audio effect to vary while causing the frequency of the haptic effect to remain substantially constant.
10. The apparatus of claim 8 , wherein the control signal includes a plurality of pulses, the controller being configured to cause a frequency of the plurality of pulses to vary while causing a magnitude of the plurality of pulses to vary inversely.
11. The apparatus of claim 8 , wherein the audio effect is configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect.
12. The apparatus of claim 8 , wherein the audio effect is configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect, the perceived frequency being higher than the frequency of the haptic effect.
13. The apparatus of claim 8 , wherein the control signal is configured to cause a plurality of haptic effects including the haptic effect, each of the plurality of haptic effects having a frequency within at least two frequency ranges from a plurality of frequency ranges, the control signal being configured to cause the haptic effect to be output having a frequency in an intermediate frequency range from the plurality of frequency ranges, the intermediate frequency range being between the at least two frequency ranges.
14. The apparatus of claim 8 , further comprising:
a haptic device coupled to the controller, the haptic device being configured to receive the control signal and to output the haptic effect and the audio effect.
15. The apparatus of claim 8 , further comprising:
a haptic device coupled to the controller, the haptic device including an actuator, the haptic device being configured to receive the control signal and to output the haptic effect and the audio effect via the actuator.
16. The apparatus of claim 8 , further comprising:
a haptic device coupled to the controller, the haptic device including a actuator and an audio output device substantially collocated with the actuator and to receive the control signal, the haptic device being configured to output the haptic effect via the actuator, the haptic device being configured to output the audio effect via the audio output device.
17. The apparatus of claim 8 , further comprising:
a plurality of controllers including the controller, each controller from the plurality of controllers being associated with a frequency range from a plurality of frequency ranges, each controller from the plurality of controllers being configured to output an associated control signal, the associated control signal output by each controller from the plurality of controllers being configured to cause a haptic effect to be output having a frequency within the frequency range associated with that controller, at least one controller from the plurality of controllers being configured to output the control signal configured to cause the haptic effect and the audio effect to be output substantially concurrently.
18. The apparatus of claim 8 , further comprising:
a resonant vibrotactile haptic device coupled to the controller, the resonant vibrotactile haptic device being configured to output the haptic effect having a frequency within a pre-determined operational frequency range, the pre-determined operational frequency range having a frequency associated with a resonant mode of the resonant vibrotactile haptic device.
19. The apparatus of claim 8 , wherein the control signal includes a plurality of pulses, the control signal being configured to cause the haptic effect to be output at a desired output frequency, each pulse from the plurality of pulses having a width associated with the desired output frequency.
20. The apparatus of claim 8 , further comprising:
a plurality of controllers including the controller, each controller from the plurality of controllers being configured to output a control signal from a plurality of control signals, each control signal from the plurality of control signals being uniquely associated with a frequency range from a plurality of frequency ranges and being configured to cause a haptic effect to be output, a first control signal from the plurality of control signals being output by a first controller, the first control signal being configured to cause the haptic effect to be output substantially concurrently with the audio effect, the audio effect being configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect.
21. The apparatus of claim 8 , further comprising:
a plurality of controllers including the controller, each controller from the plurality of controllers being configured to output a control signal from a plurality of control signals, each control signal from the plurality of control signals being uniquely associated with a frequency range from a plurality of frequency ranges and being configured to cause a haptic effect to be output, a first control signal from the plurality of control signals being output by a first controller, the first control signal being configured to cause the haptic effect to be output substantially concurrently with the audio effect, the audio effect being configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect; and
a haptic device coupled to the plurality of controllers, the haptic device being configured to output a plurality of haptic effects associated with the plurality of control signals, the plurality of haptic effects including the haptic effect.
22. The apparatus of claim 8 , further comprising:
a plurality of controllers including the controller, each controller from the plurality of controllers being configured to output a control signal from a plurality of control signals, each control signal from the plurality of control signals being uniquely associated with a frequency range from a plurality of frequency ranges and being configured to cause a haptic effect to be output, a first control signal from the plurality of control signals being output by a first controller, the first control signal being configured to cause the haptic effect to be output substantially concurrently with the audio effect, the audio effect being configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect; and
a plurality of haptic devices, each haptic device from the plurality of haptic devices being uniquely associated with a controller from the plurality of controllers, each haptic device from the plurality of haptic devices being configured to output the haptic effect caused by the control signal output by the associated controller.
23. A method, comprising:
outputting a haptic effect at least partially based on a control instruction; and
outputting an audio effect substantially concurrently with a haptic effect at least partially based on the control instruction, the haptic effect having a frequency, the audio effect having a frequency different from the frequency of the haptic effect, the audio effect and the haptic effect being output by a common device;
varying the frequency of at least one of the frequency of the haptic effect and the frequency of the audio effect while maintaining substantially constant an average energy of the haptic effect.
24. The method of claim 23 , wherein the frequency of the audio effect is varied while the frequency of the haptic effect is maintained substantially constant.
25. The method of claim 23 , wherein outputting the haptic effect includes:
varying a frequency of pulses configured to cause the haptic effect to be output while inversely varying a magnitude of the pulses configured to output the haptic effect.
26. The method of claim 23 , wherein the audio effect is configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect.
27. The method of claim 23 , wherein the audio effect is configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect, the perceived frequency of the haptic effect being higher than the frequency of the haptic effect.
28. The method of claim 23 , further comprising:
outputting a plurality of haptic effects, the plurality of haptic effects including the haptic effect, each haptic effect from the plurality of haptic effects having a different corresponding frequency, each corresponding frequency being within a frequency range from a plurality of frequency ranges.
29. A method, comprising:
receiving an output instruction; and
outputting a control signal based on the received output instruction, the control signal being configured to cause a haptic effect and an audio effect to be output substantially concurrently, the haptic effect having a frequency and the audio effect having a frequency different from the frequency of the haptic effect, the control signal being configured to vary at least one of the frequency of the haptic effect and the frequency of the audio effect while maintaining substantially constant an average energy of the haptic effect.
30. The method of claim 29 , wherein the control signal is configured to cause the frequency of the audio effect to vary while causing the frequency of the haptic effect to remain substantially constant.
31. The method of claim 29 , wherein the control signal includes a plurality of pulses, configured to cause a frequency of the plurality of pulses to vary while causing a magnitude of the plurality of pulses to vary inversely.
32. The method of claim 29 , wherein the control signal is configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect.
33. The method of claim 29 , wherein the control signal is configured to cause a user to perceive the haptic effect as having a perceived frequency different from the frequency of the haptic effect, the perceived frequency of the haptic effect being higher than the frequency of the haptic effect.
34. The method of claim 29 , further comprising:
outputting a plurality of control signals including the control signal, the plurality of control signals being at least partially based on the received output instruction, each control signal from the plurality of control signals being uniquely associated with a frequency range from a plurality of frequency ranges, at least one control signal from the plurality of control signals being configured to cause the haptic effect.
35. The method of 29, wherein the control signal is configured to output the haptic effect and the audio effect via an actuator of a haptic device.
36. The method of 29, wherein the control signal is configured to output the haptic effect via an actuator of a haptic device, the control signal being configured to cause the audio effect to be output by an audio output device of the haptic device, the audio output device being substantially collocated with the actuator.
37. A method, comprising:
receiving an output instruction; and
outputting a control signal based on the received output instruction, the control signal being configured to cause a haptic effect and an audio effect to be output substantially concurrently via an actuator of a haptic device, the haptic effect having a frequency and the audio effect having a frequency different from the frequency of the haptic effect, the audio effect being configured to change a perceived frequency of the haptic effect.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/983,231 US20060017691A1 (en) | 2004-07-23 | 2004-11-08 | System and method for controlling audio output associated with haptic effects |
US14/728,673 US9760171B2 (en) | 2004-07-23 | 2015-06-02 | System and method for controlling audio output associated with haptic effects |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US59043204P | 2004-07-23 | 2004-07-23 | |
US10/983,231 US20060017691A1 (en) | 2004-07-23 | 2004-11-08 | System and method for controlling audio output associated with haptic effects |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/728,673 Continuation US9760171B2 (en) | 2004-07-23 | 2015-06-02 | System and method for controlling audio output associated with haptic effects |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060017691A1 true US20060017691A1 (en) | 2006-01-26 |
Family
ID=35656622
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/983,231 Abandoned US20060017691A1 (en) | 2004-07-23 | 2004-11-08 | System and method for controlling audio output associated with haptic effects |
US14/728,673 Expired - Fee Related US9760171B2 (en) | 2004-07-23 | 2015-06-02 | System and method for controlling audio output associated with haptic effects |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/728,673 Expired - Fee Related US9760171B2 (en) | 2004-07-23 | 2015-06-02 | System and method for controlling audio output associated with haptic effects |
Country Status (1)
Country | Link |
---|---|
US (2) | US20060017691A1 (en) |
Cited By (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080117175A1 (en) * | 2006-11-16 | 2008-05-22 | Nokia Corporation | Method, apparatus, and computer program product providing vibration control interface |
US20080198139A1 (en) * | 2007-02-20 | 2008-08-21 | Immersion Corporation | Haptic Feedback System with Stored Effects |
US20100207746A1 (en) * | 2007-10-12 | 2010-08-19 | Koo-Young Song | Drive system of vibration mouse |
US20100309142A1 (en) * | 2009-06-09 | 2010-12-09 | Immersion Corporation, A Delaware Corporation | Method and apparatus for generating haptic effects using actuators |
US20110065428A1 (en) * | 2009-09-16 | 2011-03-17 | At&T Intellectual Property I, L.P | Systems and methods for selecting an output modality in a mobile device |
CN104049889A (en) * | 2013-03-15 | 2014-09-17 | 英默森公司 | Method and apparatus for encoding and decoding haptic information in multi-media files |
US20150009168A1 (en) * | 2013-07-02 | 2015-01-08 | Immersion Corporation | Systems and Methods For Perceptual Normalization of Haptic Effects |
US20150070144A1 (en) * | 2013-09-06 | 2015-03-12 | Immersion Corporation | Automatic remote sensing and haptic conversion system |
JP2015053044A (en) * | 2013-09-06 | 2015-03-19 | イマージョン コーポレーションImmersion Corporation | Multiplexing and demultiplexing haptic signals |
CN104679251A (en) * | 2009-06-09 | 2015-06-03 | 意美森公司 | Method And Apparatus For Generating Haptic Feedback And A Haptic Interface |
US20150179027A1 (en) * | 2013-12-24 | 2015-06-25 | Samsung Electronics Co., Ltd. | Home appliance and controlling method thereof |
US20150224394A1 (en) * | 2008-09-24 | 2015-08-13 | Immersion Corporation | Multiple Actuation Handheld Device |
US20150251089A1 (en) * | 2014-03-07 | 2015-09-10 | Sony Corporation | Information processing apparatus, information processing system, information processing method, and program |
US20170003744A1 (en) * | 2014-03-27 | 2017-01-05 | Apple Inc. | Adjusting the level of acoustic and haptic output in haptic devices |
US20170062821A1 (en) * | 2014-02-17 | 2017-03-02 | William Marsh Rice University | Laser induced graphene materials and their use in electronic devices |
US20170270756A1 (en) * | 2013-09-06 | 2017-09-21 | Immersion Corporation | Systems and Methods for Generating Haptic Effects Associated With Transitions in Audio Signals |
US9886090B2 (en) | 2014-07-08 | 2018-02-06 | Apple Inc. | Haptic notifications utilizing haptic input devices |
US9891714B2 (en) | 2014-12-24 | 2018-02-13 | Immersion Corporation | Audio enhanced simulation of high bandwidth haptic effects |
KR101902421B1 (en) * | 2009-10-02 | 2018-09-28 | 엔브이에프 테크 리미티드 | Touch sensitive device generating a haptic feedback that simulates a click feeling |
US10140823B2 (en) | 2013-09-06 | 2018-11-27 | Immersion Corporation | Method and system for providing haptic effects based on information complementary to multimedia content |
US10254840B2 (en) | 2015-07-21 | 2019-04-09 | Apple Inc. | Guidance device for the sensory impaired |
US10297120B2 (en) * | 2016-12-13 | 2019-05-21 | Disney Enterprises, Inc. | Haptic effect generation system |
CN110032272A (en) * | 2013-09-06 | 2019-07-19 | 意美森公司 | For exporting the system, method and non-transitory computer-readable medium of haptic effect |
US10366215B2 (en) * | 2012-07-20 | 2019-07-30 | Licentia Group Limited | Authentication method and system |
US10372214B1 (en) | 2016-09-07 | 2019-08-06 | Apple Inc. | Adaptable user-selectable input area in an electronic device |
US10375930B1 (en) * | 2017-07-07 | 2019-08-13 | Chad R. James | Animal training device that controls stimulus using proportional pressure-based input |
US10395488B2 (en) | 2013-09-06 | 2019-08-27 | Immersion Corporation | Systems and methods for generating haptic effects associated with an envelope in audio signals |
US10437359B1 (en) | 2017-02-28 | 2019-10-08 | Apple Inc. | Stylus with external magnetic influence |
US10556252B2 (en) | 2017-09-20 | 2020-02-11 | Apple Inc. | Electronic device having a tuned resonance haptic actuation system |
US10585480B1 (en) | 2016-05-10 | 2020-03-10 | Apple Inc. | Electronic device with an input device having a haptic engine |
US10592653B2 (en) | 2015-05-27 | 2020-03-17 | Licentia Group Limited | Encoding methods and systems |
US10613678B1 (en) | 2018-09-17 | 2020-04-07 | Apple Inc. | Input device with haptic feedback |
US10649529B1 (en) | 2016-06-28 | 2020-05-12 | Apple Inc. | Modification of user-perceived feedback of an input device using acoustic or haptic output |
US10768747B2 (en) | 2017-08-31 | 2020-09-08 | Apple Inc. | Haptic realignment cues for touch-input displays |
US10768738B1 (en) | 2017-09-27 | 2020-09-08 | Apple Inc. | Electronic device having a haptic actuator with magnetic augmentation |
US10775889B1 (en) | 2017-07-21 | 2020-09-15 | Apple Inc. | Enclosure with locally-flexible regions |
US10845878B1 (en) | 2016-07-25 | 2020-11-24 | Apple Inc. | Input device with tactile feedback |
US10936071B2 (en) | 2018-08-30 | 2021-03-02 | Apple Inc. | Wearable electronic device with haptic rotatable input |
US10942571B2 (en) | 2018-06-29 | 2021-03-09 | Apple Inc. | Laptop computing device with discrete haptic regions |
US10966007B1 (en) | 2018-09-25 | 2021-03-30 | Apple Inc. | Haptic output system |
US11024135B1 (en) | 2020-06-17 | 2021-06-01 | Apple Inc. | Portable electronic device having a haptic button assembly |
US11054932B2 (en) | 2017-09-06 | 2021-07-06 | Apple Inc. | Electronic device having a touch sensor, force sensor, and haptic actuator in an integrated module |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018048907A1 (en) | 2016-09-06 | 2018-03-15 | Neosensory, Inc. C/O Tmc+260 | Method and system for providing adjunct sensory information to a user |
CN110011591B (en) * | 2018-12-31 | 2022-07-05 | 瑞声科技(新加坡)有限公司 | Motor drive signal generation method, electronic device, and storage medium |
US11079854B2 (en) * | 2020-01-07 | 2021-08-03 | Neosensory, Inc. | Method and system for haptic stimulation |
US11497675B2 (en) | 2020-10-23 | 2022-11-15 | Neosensory, Inc. | Method and system for multimodal stimulation |
US11862147B2 (en) | 2021-08-13 | 2024-01-02 | Neosensory, Inc. | Method and system for enhancing the intelligibility of information for a user |
Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4334280A (en) * | 1980-06-09 | 1982-06-08 | Texas Instruments Incorporated | System and method for providing an audible sound and a tactile feedback in an electronic data processing system |
US4355348A (en) * | 1980-07-30 | 1982-10-19 | Williams Theodore R | Audio signal responsive optical display |
US4768412A (en) * | 1986-05-09 | 1988-09-06 | Sanderson Stephen N | Low profile keyboard device and system for recording and scoring music |
US4771344A (en) * | 1986-11-13 | 1988-09-13 | James Fallacaro | System for enhancing audio and/or visual presentation |
US5388992A (en) * | 1991-06-19 | 1995-02-14 | Audiological Engineering Corporation | Method and apparatus for tactile transduction of acoustic signals from television receivers |
US5437608A (en) * | 1991-05-03 | 1995-08-01 | Jb Research, Inc. | Massaging apparatus with sequential vibration |
US5438529A (en) * | 1994-01-26 | 1995-08-01 | Immersion Human Interface Corporation | Percussion input device for personal computer systems |
US5451924A (en) * | 1993-01-14 | 1995-09-19 | Massachusetts Institute Of Technology | Apparatus for providing sensory substitution of force feedback |
US5669818A (en) * | 1995-03-23 | 1997-09-23 | Thorner; Craig | Seat-based tactile sensation generator |
US5684722A (en) * | 1994-09-21 | 1997-11-04 | Thorner; Craig | Apparatus and method for generating a control signal for a tactile sensation generator |
US5737144A (en) * | 1996-06-05 | 1998-04-07 | Quantum Corporation | Switched boost voltage generator for actuator retract in disk drive |
US5842162A (en) * | 1996-03-08 | 1998-11-24 | Motorola, Inc. | Method and recognizer for recognizing a sampled sound signal in noise |
US5857986A (en) * | 1996-05-24 | 1999-01-12 | Moriyasu; Hiro | Interactive vibrator for multimedia |
US6160489A (en) * | 1994-06-23 | 2000-12-12 | Motorola, Inc. | Wireless communication device adapted to generate a plurality of distinctive tactile alert patterns |
US6285351B1 (en) * | 1997-04-25 | 2001-09-04 | Immersion Corporation | Designing force sensations for computer applications including sounds |
US6292776B1 (en) * | 1999-03-12 | 2001-09-18 | Lucent Technologies Inc. | Hierarchial subband linear predictive cepstral features for HMM-based speech recognition |
US6496115B2 (en) * | 2001-03-22 | 2002-12-17 | Meiji University Legal Person | System and method for analyzing baby cries |
US20030068053A1 (en) * | 2001-10-10 | 2003-04-10 | Chu Lonny L. | Sound data output and manipulation using haptic feedback |
US20030067440A1 (en) * | 2001-10-09 | 2003-04-10 | Rank Stephen D. | Haptic feedback sensations based on audio output from computer devices |
US20050047621A1 (en) * | 2003-08-28 | 2005-03-03 | Cranfill David B. | Multifunction transducer and method of driving |
US6885876B2 (en) * | 2001-03-12 | 2005-04-26 | Nokia Mobile Phones Ltd. | Mobile phone featuring audio-modulated vibrotactile module |
US6963762B2 (en) * | 2001-05-23 | 2005-11-08 | Nokia Corporation | Mobile phone using tactile icons |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0648364A1 (en) * | 1993-04-30 | 1995-04-19 | Federal Signal Corporation | Tone generating circuit |
US6422941B1 (en) * | 1994-09-21 | 2002-07-23 | Craig Thorner | Universal tactile feedback system for computer video games and simulations |
JP2000501033A (en) | 1995-11-30 | 2000-02-02 | ヴァーチャル テクノロジーズ インコーポレイテッド | Human / machine interface with tactile feedback |
US6659773B2 (en) * | 1998-03-04 | 2003-12-09 | D-Box Technology Inc. | Motion transducer system |
DE20080209U1 (en) | 1999-09-28 | 2001-08-09 | Immersion Corp | Control of haptic sensations for interface devices with vibrotactile feedback |
US6864877B2 (en) * | 2000-09-28 | 2005-03-08 | Immersion Corporation | Directional tactile feedback for haptic feedback interface devices |
US7019621B2 (en) * | 2001-01-02 | 2006-03-28 | Stanley E. Woodard | Methods and apparatus to increase sound quality of piezoelectric devices |
US7161580B2 (en) | 2002-04-25 | 2007-01-09 | Immersion Corporation | Haptic feedback using rotary harmonic moving mass |
US20050134485A1 (en) * | 2003-12-22 | 2005-06-23 | Hein David A. | Touch pad for motor vehicle and sensor therewith |
US7176888B2 (en) * | 2004-03-23 | 2007-02-13 | Fujitsu Limited | Selective engagement of motion detection |
-
2004
- 2004-11-08 US US10/983,231 patent/US20060017691A1/en not_active Abandoned
-
2015
- 2015-06-02 US US14/728,673 patent/US9760171B2/en not_active Expired - Fee Related
Patent Citations (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4334280A (en) * | 1980-06-09 | 1982-06-08 | Texas Instruments Incorporated | System and method for providing an audible sound and a tactile feedback in an electronic data processing system |
US4355348A (en) * | 1980-07-30 | 1982-10-19 | Williams Theodore R | Audio signal responsive optical display |
US4768412A (en) * | 1986-05-09 | 1988-09-06 | Sanderson Stephen N | Low profile keyboard device and system for recording and scoring music |
US4771344A (en) * | 1986-11-13 | 1988-09-13 | James Fallacaro | System for enhancing audio and/or visual presentation |
US5437608A (en) * | 1991-05-03 | 1995-08-01 | Jb Research, Inc. | Massaging apparatus with sequential vibration |
US5388992A (en) * | 1991-06-19 | 1995-02-14 | Audiological Engineering Corporation | Method and apparatus for tactile transduction of acoustic signals from television receivers |
US5451924A (en) * | 1993-01-14 | 1995-09-19 | Massachusetts Institute Of Technology | Apparatus for providing sensory substitution of force feedback |
US5438529A (en) * | 1994-01-26 | 1995-08-01 | Immersion Human Interface Corporation | Percussion input device for personal computer systems |
US6160489A (en) * | 1994-06-23 | 2000-12-12 | Motorola, Inc. | Wireless communication device adapted to generate a plurality of distinctive tactile alert patterns |
US5684722A (en) * | 1994-09-21 | 1997-11-04 | Thorner; Craig | Apparatus and method for generating a control signal for a tactile sensation generator |
US5669818A (en) * | 1995-03-23 | 1997-09-23 | Thorner; Craig | Seat-based tactile sensation generator |
US5842162A (en) * | 1996-03-08 | 1998-11-24 | Motorola, Inc. | Method and recognizer for recognizing a sampled sound signal in noise |
US5857986A (en) * | 1996-05-24 | 1999-01-12 | Moriyasu; Hiro | Interactive vibrator for multimedia |
US5737144A (en) * | 1996-06-05 | 1998-04-07 | Quantum Corporation | Switched boost voltage generator for actuator retract in disk drive |
US6285351B1 (en) * | 1997-04-25 | 2001-09-04 | Immersion Corporation | Designing force sensations for computer applications including sounds |
US6292776B1 (en) * | 1999-03-12 | 2001-09-18 | Lucent Technologies Inc. | Hierarchial subband linear predictive cepstral features for HMM-based speech recognition |
US6885876B2 (en) * | 2001-03-12 | 2005-04-26 | Nokia Mobile Phones Ltd. | Mobile phone featuring audio-modulated vibrotactile module |
US6496115B2 (en) * | 2001-03-22 | 2002-12-17 | Meiji University Legal Person | System and method for analyzing baby cries |
US6963762B2 (en) * | 2001-05-23 | 2005-11-08 | Nokia Corporation | Mobile phone using tactile icons |
US20030067440A1 (en) * | 2001-10-09 | 2003-04-10 | Rank Stephen D. | Haptic feedback sensations based on audio output from computer devices |
US20030068053A1 (en) * | 2001-10-10 | 2003-04-10 | Chu Lonny L. | Sound data output and manipulation using haptic feedback |
US6703550B2 (en) * | 2001-10-10 | 2004-03-09 | Immersion Corporation | Sound data output and manipulation using haptic feedback |
US20050047621A1 (en) * | 2003-08-28 | 2005-03-03 | Cranfill David B. | Multifunction transducer and method of driving |
US7421088B2 (en) * | 2003-08-28 | 2008-09-02 | Motorola, Inc. | Multifunction transducer |
Cited By (82)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080117175A1 (en) * | 2006-11-16 | 2008-05-22 | Nokia Corporation | Method, apparatus, and computer program product providing vibration control interface |
EP1923774A3 (en) * | 2006-11-16 | 2015-12-30 | Nokia Technologies Oy | Method, apparatus, and computer program product providing vibration control interface |
US8120585B2 (en) * | 2006-11-16 | 2012-02-21 | Nokia Corporation | Method, apparatus, and computer program product providing vibration control interface |
US20080198139A1 (en) * | 2007-02-20 | 2008-08-21 | Immersion Corporation | Haptic Feedback System with Stored Effects |
US8098234B2 (en) * | 2007-02-20 | 2012-01-17 | Immersion Corporation | Haptic feedback system with stored effects |
US8619051B2 (en) | 2007-02-20 | 2013-12-31 | Immersion Corporation | Haptic feedback system with stored effects |
US20100207746A1 (en) * | 2007-10-12 | 2010-08-19 | Koo-Young Song | Drive system of vibration mouse |
US20150224394A1 (en) * | 2008-09-24 | 2015-08-13 | Immersion Corporation | Multiple Actuation Handheld Device |
US9545568B2 (en) * | 2008-09-24 | 2017-01-17 | Immersion Corporation | Multiple actuation handheld device with housing and touch screen actuators |
US10401961B2 (en) * | 2009-06-09 | 2019-09-03 | Immersion Corporation | Method and apparatus for generating haptic effects using actuators |
US9891708B2 (en) | 2009-06-09 | 2018-02-13 | Immersion Corporation | Method and apparatus for generating haptic effects using actuators |
CN104679251A (en) * | 2009-06-09 | 2015-06-03 | 意美森公司 | Method And Apparatus For Generating Haptic Feedback And A Haptic Interface |
US20100309142A1 (en) * | 2009-06-09 | 2010-12-09 | Immersion Corporation, A Delaware Corporation | Method and apparatus for generating haptic effects using actuators |
US20110065428A1 (en) * | 2009-09-16 | 2011-03-17 | At&T Intellectual Property I, L.P | Systems and methods for selecting an output modality in a mobile device |
KR101902421B1 (en) * | 2009-10-02 | 2018-09-28 | 엔브이에프 테크 리미티드 | Touch sensitive device generating a haptic feedback that simulates a click feeling |
US10705608B2 (en) | 2009-10-02 | 2020-07-07 | Google Llc | Touch sensitive device |
US11048783B2 (en) | 2012-07-20 | 2021-06-29 | Licentia Group Limited | Authentication method and system |
US10565359B2 (en) | 2012-07-20 | 2020-02-18 | Licentia Group Limited | Authentication method and system |
US11194892B2 (en) | 2012-07-20 | 2021-12-07 | Licentia Group Limited | Authentication method and system |
US11048784B2 (en) | 2012-07-20 | 2021-06-29 | Licentia Group Limited | Authentication method and system |
US10366215B2 (en) * | 2012-07-20 | 2019-07-30 | Licentia Group Limited | Authentication method and system |
US10477202B2 (en) | 2013-03-15 | 2019-11-12 | Immersion Corporation | Method and apparatus for encoding and decoding haptic information in multi-media files |
US9992491B2 (en) * | 2013-03-15 | 2018-06-05 | Immersion Corporation | Method and apparatus for encoding and decoding haptic information in multi-media files |
US20140270681A1 (en) * | 2013-03-15 | 2014-09-18 | Immersion Corporation | Method and apparatus for encoding and decoding haptic information in multi-media files |
CN104049889A (en) * | 2013-03-15 | 2014-09-17 | 英默森公司 | Method and apparatus for encoding and decoding haptic information in multi-media files |
US20150009168A1 (en) * | 2013-07-02 | 2015-01-08 | Immersion Corporation | Systems and Methods For Perceptual Normalization of Haptic Effects |
US9729730B2 (en) * | 2013-07-02 | 2017-08-08 | Immersion Corporation | Systems and methods for perceptual normalization of haptic effects |
US10395490B2 (en) | 2013-09-06 | 2019-08-27 | Immersion Corporation | Method and system for providing haptic effects based on information complementary to multimedia content |
CN110032272A (en) * | 2013-09-06 | 2019-07-19 | 意美森公司 | For exporting the system, method and non-transitory computer-readable medium of haptic effect |
US9910495B2 (en) | 2013-09-06 | 2018-03-06 | Immersion Corporation | Automatic remote sensing and haptic conversion system |
US9443401B2 (en) * | 2013-09-06 | 2016-09-13 | Immersion Corporation | Automatic remote sensing and haptic conversion system |
US10140823B2 (en) | 2013-09-06 | 2018-11-27 | Immersion Corporation | Method and system for providing haptic effects based on information complementary to multimedia content |
US10388122B2 (en) | 2013-09-06 | 2019-08-20 | Immerson Corporation | Systems and methods for generating haptic effects associated with audio signals |
US10416774B2 (en) | 2013-09-06 | 2019-09-17 | Immersion Corporation | Automatic remote sensing and haptic conversion system |
US20150070144A1 (en) * | 2013-09-06 | 2015-03-12 | Immersion Corporation | Automatic remote sensing and haptic conversion system |
US10276004B2 (en) * | 2013-09-06 | 2019-04-30 | Immersion Corporation | Systems and methods for generating haptic effects associated with transitions in audio signals |
US10395488B2 (en) | 2013-09-06 | 2019-08-27 | Immersion Corporation | Systems and methods for generating haptic effects associated with an envelope in audio signals |
JP2015053044A (en) * | 2013-09-06 | 2015-03-19 | イマージョン コーポレーションImmersion Corporation | Multiplexing and demultiplexing haptic signals |
US20170270756A1 (en) * | 2013-09-06 | 2017-09-21 | Immersion Corporation | Systems and Methods for Generating Haptic Effects Associated With Transitions in Audio Signals |
US9542819B2 (en) * | 2013-12-24 | 2017-01-10 | Samsung Electronics Co., Ltd. | Home appliance and controlling method thereof |
US20150179027A1 (en) * | 2013-12-24 | 2015-06-25 | Samsung Electronics Co., Ltd. | Home appliance and controlling method thereof |
US20170062821A1 (en) * | 2014-02-17 | 2017-03-02 | William Marsh Rice University | Laser induced graphene materials and their use in electronic devices |
US20150251089A1 (en) * | 2014-03-07 | 2015-09-10 | Sony Corporation | Information processing apparatus, information processing system, information processing method, and program |
US10238964B2 (en) * | 2014-03-07 | 2019-03-26 | Sony Corporation | Information processing apparatus, information processing system, and information processing method |
US10261585B2 (en) * | 2014-03-27 | 2019-04-16 | Apple Inc. | Adjusting the level of acoustic and haptic output in haptic devices |
US20170003744A1 (en) * | 2014-03-27 | 2017-01-05 | Apple Inc. | Adjusting the level of acoustic and haptic output in haptic devices |
US9886090B2 (en) | 2014-07-08 | 2018-02-06 | Apple Inc. | Haptic notifications utilizing haptic input devices |
US10429933B2 (en) | 2014-12-24 | 2019-10-01 | Immersion Corporation | Audio enhanced simulation of high bandwidth haptic effects |
US9891714B2 (en) | 2014-12-24 | 2018-02-13 | Immersion Corporation | Audio enhanced simulation of high bandwidth haptic effects |
US11036845B2 (en) | 2015-05-27 | 2021-06-15 | Licentia Group Limited | Authentication methods and systems |
US10740449B2 (en) | 2015-05-27 | 2020-08-11 | Licentia Group Limited | Authentication methods and systems |
US10592653B2 (en) | 2015-05-27 | 2020-03-17 | Licentia Group Limited | Encoding methods and systems |
US11048790B2 (en) | 2015-05-27 | 2021-06-29 | Licentia Group Limited | Authentication methods and systems |
US10254840B2 (en) | 2015-07-21 | 2019-04-09 | Apple Inc. | Guidance device for the sensory impaired |
US10664058B2 (en) | 2015-07-21 | 2020-05-26 | Apple Inc. | Guidance device for the sensory impaired |
US10585480B1 (en) | 2016-05-10 | 2020-03-10 | Apple Inc. | Electronic device with an input device having a haptic engine |
US10890978B2 (en) | 2016-05-10 | 2021-01-12 | Apple Inc. | Electronic device with an input device having a haptic engine |
US11762470B2 (en) | 2016-05-10 | 2023-09-19 | Apple Inc. | Electronic device with an input device having a haptic engine |
US10649529B1 (en) | 2016-06-28 | 2020-05-12 | Apple Inc. | Modification of user-perceived feedback of an input device using acoustic or haptic output |
US10845878B1 (en) | 2016-07-25 | 2020-11-24 | Apple Inc. | Input device with tactile feedback |
US10372214B1 (en) | 2016-09-07 | 2019-08-06 | Apple Inc. | Adaptable user-selectable input area in an electronic device |
US10297120B2 (en) * | 2016-12-13 | 2019-05-21 | Disney Enterprises, Inc. | Haptic effect generation system |
US10437359B1 (en) | 2017-02-28 | 2019-10-08 | Apple Inc. | Stylus with external magnetic influence |
US10375930B1 (en) * | 2017-07-07 | 2019-08-13 | Chad R. James | Animal training device that controls stimulus using proportional pressure-based input |
US11470816B2 (en) | 2017-07-07 | 2022-10-18 | Cje Products, Llc | Animal training device that controls stimulus using proportional pressure-based input |
US11805754B2 (en) * | 2017-07-07 | 2023-11-07 | Cje Products, Llc | Animal training device that controls stimulus using proportional pressure-based input |
US20190246604A1 (en) * | 2017-07-07 | 2019-08-15 | Chad R. James | Animal training device that controls stimulus using proportional pressure-based input |
US20230042381A1 (en) * | 2017-07-07 | 2023-02-09 | Cje Products, Llc | Animal training device that controls stimulus using proportional pressure-based input |
US11487362B1 (en) | 2017-07-21 | 2022-11-01 | Apple Inc. | Enclosure with locally-flexible regions |
US10775889B1 (en) | 2017-07-21 | 2020-09-15 | Apple Inc. | Enclosure with locally-flexible regions |
US10768747B2 (en) | 2017-08-31 | 2020-09-08 | Apple Inc. | Haptic realignment cues for touch-input displays |
US11054932B2 (en) | 2017-09-06 | 2021-07-06 | Apple Inc. | Electronic device having a touch sensor, force sensor, and haptic actuator in an integrated module |
US11460946B2 (en) | 2017-09-06 | 2022-10-04 | Apple Inc. | Electronic device having a touch sensor, force sensor, and haptic actuator in an integrated module |
US10556252B2 (en) | 2017-09-20 | 2020-02-11 | Apple Inc. | Electronic device having a tuned resonance haptic actuation system |
US10768738B1 (en) | 2017-09-27 | 2020-09-08 | Apple Inc. | Electronic device having a haptic actuator with magnetic augmentation |
US10942571B2 (en) | 2018-06-29 | 2021-03-09 | Apple Inc. | Laptop computing device with discrete haptic regions |
US10936071B2 (en) | 2018-08-30 | 2021-03-02 | Apple Inc. | Wearable electronic device with haptic rotatable input |
US10613678B1 (en) | 2018-09-17 | 2020-04-07 | Apple Inc. | Input device with haptic feedback |
US10966007B1 (en) | 2018-09-25 | 2021-03-30 | Apple Inc. | Haptic output system |
US11805345B2 (en) | 2018-09-25 | 2023-10-31 | Apple Inc. | Haptic output system |
US11024135B1 (en) | 2020-06-17 | 2021-06-01 | Apple Inc. | Portable electronic device having a haptic button assembly |
US11756392B2 (en) | 2020-06-17 | 2023-09-12 | Apple Inc. | Portable electronic device having a haptic button assembly |
Also Published As
Publication number | Publication date |
---|---|
US20160070350A1 (en) | 2016-03-10 |
US9760171B2 (en) | 2017-09-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9760171B2 (en) | System and method for controlling audio output associated with haptic effects | |
JP6588951B2 (en) | System and method using multiple actuators to achieve texture | |
JP6410860B2 (en) | System and method for a texture engine | |
KR100960637B1 (en) | System and method for controlling haptic devices having multiple operational modes | |
JP5779508B2 (en) | System and method for a texture engine | |
JP5606462B2 (en) | System and method using multiple actuators to achieve texture | |
JP6498863B2 (en) | Haptic system with increased LRA bandwidth | |
KR100787792B1 (en) | Information processing terminal | |
US20170052593A1 (en) | Haptic device with linear resonant actuator | |
EP2590067A2 (en) | Systems and methods for multi-pressure interaction on touch-sensitive surfaces | |
US20050134561A1 (en) | System and method for mapping instructions associated with haptic feedback | |
CN105892921A (en) | Systems and methods for friction displays and additional haptic effects | |
EP3582083A1 (en) | Systems and methods for controlling actuator drive signals for improving transient response characteristics | |
KR101992070B1 (en) | Systems and methods for a texture engine |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: IMMERSION CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CRUZ-HERNANDEZ, JUAN MANUEL;GRANT, DANNY A.;REEL/FRAME:015979/0429;SIGNING DATES FROM 20041101 TO 20041102 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |