WO2001024158A1 - Controlling haptic sensations for vibrotactile feedback interface devices - Google Patents

Controlling haptic sensations for vibrotactile feedback interface devices Download PDF

Info

Publication number
WO2001024158A1
WO2001024158A1 PCT/US2000/026615 US0026615W WO0124158A1 WO 2001024158 A1 WO2001024158 A1 WO 2001024158A1 US 0026615 W US0026615 W US 0026615W WO 0124158 A1 WO0124158 A1 WO 0124158A1
Authority
WO
WIPO (PCT)
Prior art keywords
vibration
control signal
magnitude
recited
vibrotactile
Prior art date
Application number
PCT/US2000/026615
Other languages
French (fr)
Inventor
Alex S. Goldenberg
Ramon Alarcon
Original Assignee
Immersion Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Immersion Corporation filed Critical Immersion Corporation
Priority to DE20080209U priority Critical patent/DE20080209U1/en
Priority to AU77256/00A priority patent/AU7725600A/en
Publication of WO2001024158A1 publication Critical patent/WO2001024158A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/70Game security or game management aspects
    • A63F13/77Game security or game management aspects involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/23Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/24Constructional details thereof, e.g. game controllers with detachable joystick handles
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25Output arrangements for video game devices
    • A63F13/28Output arrangements for video game devices responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25Output arrangements for video game devices
    • A63F13/28Output arrangements for video game devices responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light
    • A63F13/285Generating tactile feedback signals via the game input device, e.g. force feedback
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55Controlling game characters or game objects based on the game progress
    • A63F13/57Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/90Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1025Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals details of the interface with the game device, e.g. USB version detection
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1037Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1043Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being characterized by constructional details
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/30Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
    • A63F2300/302Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device specially adapted for receiving control signals not targeted to a display device or game input means, e.g. vibrating driver's seat, scent dispenser
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/60Methods for processing data by generating or executing the game program
    • A63F2300/64Methods for processing data by generating or executing the game program for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F9/00Games not otherwise provided for
    • A63F9/24Electric games; Games using electronic circuits not otherwise provided for
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01Indexing scheme relating to G06F3/01
    • G06F2203/013Force feedback applied to a game

Definitions

  • the present invention relates generally to control techniques for human-computer interface devices, and more specifically to controlling vibrotactile haptic sensations from a vibrotactile feedback device.
  • Computer devices are widely used for entertainment activities such as playing games.
  • popular gaming computer devices include game consoles connected to a home television set, such as the Nintendo® 64 from Nintendo Corp., the Playstation® from Sony Corp., and the DreamcastTM from Sega Corp.
  • Gaming computer devices also include personal computers, such as Windows PCs, Macintosh computers, and others.
  • portable computer devices are often used for entertainment purposes, such as Game Boy® from Nintendo, personal digital assistants such as PalmPilot® from Palm Computing, and laptop computers.
  • haptic feedback is intended to include both tactile (or vibrotactile) feedback (forces transmitted to user skin surfaces) and kinesthetic
  • haptic feedback interface devices In the game console market, products are typically produced in high volume and low cost. Therefore, haptic feedback interface devices have been typically limited to simpler embodiments that provide more limited forms of haptic feedback.
  • Existing force feedback "gamepad” controllers (or add-on hardware for gamepad controllers) that are used to interface with games running on game consoles include the Dual ShockTM from Sony Corp., the Rumble PakTM from Nintendo Corp., and the Jump Pack from Sega Corp. These devices are vibrotactile feedback controllers which employ one or more motors to shake the housing of the controller and thus provide output vibrations to the user which are correlated to game events and interactions.
  • an eccentric rotating mass (ERM) motor i.e., pager motor, is used to generate vibration on the controller and thus to the user.
  • ECM eccentric rotating mass
  • This type of motor provides a mass on a rotating shaft offset from the axis of rotation, so that when the shaft is rotated, the mass rocks the motor and the gamepad housing back and forth.
  • Controller 10 similar to the Dual Shock controller is shown in FIGURE 1.
  • Controller 10 includes a d-pad 12 for directional user input, buttons 14 for button user input, and one or more joysticks 16 for directional user input.
  • Controller 10 also includes one or more motors integrated with the housing of the controller, as explained below with reference to Fig. 3. The user grasps both grips 11 while operating the controller to feel vibrations through the housing of the controller.
  • the gamepad controller 22 provides the directional and button input similarly to the controller 10, but does not include motors for vibrotactile feedback.
  • Add-on unit 24 can include one or more motors and can be connected to an interface port 26 of the controller 22 to provide vibrotactile feedback to the housing of controller 22.
  • Controller 10 uses two individual ERM motors.
  • one grip 26 of the controller is a small rotary motor 28 which includes a small mass 30 coupled to its rotating shaft.
  • a large rotary motor 34 which includes a large mass 36.
  • Both motors 28 and 34 are typically driven with a single transistor voltage-mode amplifier, where each motor can be activated to rotate its associated mass in one direction at a single speed.
  • the large motor 34 is activated to display large amplitudes or lower frequencies of vibrations to the user, and the small motor 28 is activated to convey higher frequencies or smaller amplitude vibrations to the user.
  • the existing Dual Shock specification allows 8-bit variability of voltage to the large motor 34 and therefore a variable range of output vibration magnitude.
  • the voltage to the small motor 28 is specified as on or off and thus only allows a single magnitude of output vibration.
  • only one motor 28 or 34 in the controller is activated at one time, but the activated motor may be user-selectable in some games.
  • the large motor speed can be varied, many game programs tend to send DC commands to the large motor, where each command can cause the motor to deliver one of 255 available vibrations (e.g., there are 255 different available frequencies of vibrations).
  • Individual games on the host console unit control when the motors are turned on and off by sending commands or signals to the controllers to correlate vibrotactile feedback with game events.
  • the Rumble Pak and Jump Pack tactile add-on units for the Nintendo and Sega systems use similar ERM motors but currently provide a single motor for vibration output.
  • the motor of the Rumble Pak is typically either turned on (at a maximum or a predetermined frequency) or turned off, and does not provide variable vibration frequencies.
  • the Jump Pack uses an analog value to control variable vibration magnitude. Still other controllers, such as the Mad
  • Catz Dual Force Racing Wheel are handheld controllers that are compatible with existing gaming consoles and provide similar vibrotactile feedback using ERM motors.
  • One problem with existing vibrotactile controllers is that the frequency of output vibrations is closely coupled to the magnitude of the vibrations.
  • the Dual Shock controller when any one of the available vibrations is output, magnitude and frequency are tightly coupled — they cannot be varied independently.
  • a particular vibration frequency (mass rotation speed) is commanded, and that frequency has a particular vibration magnitude associated with it, as felt by the user, which cannot be changed.
  • these devices are very limited in their range of output tactile feedback; only high-magnitude, high-frequency vibrations, or low-magnitude, low- frequency vibrations can be output, with no ability to independently vary these parameters.
  • While vibrotactile controllers for gaming consoles are substantially simpler than more sophisticated force feedback devices for personal computers such as the PC, tactile feedback in console games is still very effective and interesting to users.
  • the console vibrotactile devices are much less expensive than more sophisticated kinesthetic personal computer force feedback devices, such as steering wheels and flight simulator joysticks, which provide forces in the axes of motion of these devices.
  • the console vibrotactile devices when provided for personal computers, can present an attractive option to computer users who wish to experience haptic feedback but do not wish to spend a relatively large amount of money on an interface device.
  • vibrotactile feedback is typically not readily controllable with existing force feedback control methods used for personal computers.
  • haptic feedback effects on personal computers are provided using the Immersion protocol from Immersion Corporation and DirectX from Microsoft Corp., which allow games to command vibration forces using independent frequency and magnitude parameters.
  • the existing gamepad vibrotactile devices and control schemes do not allow independent control of frequency and magnitude of vibration.
  • the existing computer protocols allow a game to output a variety of force feedback effects including not only vibrations but vector forces, spring forces, damping forces, texture forces, and other force effects which are unknown to gamepad controllers or other vibrotactile controllers.
  • What is needed is a method of controlling vibrotactile feedback to allow a greater variety of vibrotactile sensations, and also to allow a vibrotactile controller to be compatible with an application program which expects to be outputting effects to a more sophisticated, kinesthetic force feedback device.
  • the present invention is directed to mapping kinesthetic force sensations to vibrotactile sensations output by a vibrotactile feedback device connected to a computer.
  • Sophisticated vibrations can be output from a haptic feedback device that includes driver electronics driving a rotating mass in one direction, and allowing the magnitude and frequency of the resulting vibrations to be controlled independently.
  • Other kinesthetic force effects can be mapped to other vibrations and motor control methods. More specifically, one aspect of the present invention relates to a method for controlling a vibrotactile interface device, such as a gamepad, mouse, etc., from a host microprocessor.
  • a desired haptic effect to be output by the vibrotactile interface device to a user is determined, and effect information is provided to the vibrotactile device, the information describing a magnitude and a frequency that are independent of each other.
  • a periodic control signal based on the effect information drives a rotary motor of the vibrotactile device to cause a mass to rotate, where the rotation of the mass causes a vibration having the magnitude and frequency to be output to the user.
  • the magnitude of the vibration is based on a duty cycle of the control signal, and the frequency of the vibration is based on a frequency of the control signal.
  • the control signal can be either on or off, where the control signal is turned on for an on-time for each period of the vibration, which determines the magnitude of the vibration.
  • the provided information can include a higher level command and at least one parameter that is parsed by the device, or the information can be substantially equivalent to the control signal.
  • the control signal may drive the motor in only one direction.
  • a method for providing a vibration for a haptic feedback device coupled to a host microprocessor includes providing an actuator for the haptic feedback device, the actuator including a rotatable mass, and receiving information at the haptic feedback device which causes a control signal to be produced.
  • the control signal controls the actuator to rotate the mass about an axis of rotation such that the mass rotation induces a vibration in the device, where a magnitude and a frequency of the vibration can be adjusted independently of each other by adjusting the control signal.
  • a magnitude of the vibration is based on a duty cycle of the control signal and a frequency of the vibration is based on a frequency of the control signal.
  • the control signal can be applied at a predetermined point in time of every period of the vibration.
  • An on-time of the control signal can be determined as a percentage of a period of the vibration, or as a predetermined amount of time for each period.
  • Methods can be used to ensure the mass is rotating, such as sending an initial control signal to said actuator to start the mass rotating before the vibration is output.
  • a haptic feedback device including similar features is also disclosed. Some embodiments can cause the mass to move bidirectionally.
  • a method for allowing kinesthetic haptic effects to be output on a vibrotactile interface device includes receiving a command describing a kinesthetic haptic effect that is intended to cause forces to be output in at least one degree of freedom of a manipulandum of a kinesthetic haptic feedback device that is manipulated by a user.
  • the kinesthetic haptic effect is mapped to a vibrotactile haptic effect that is intended to cause vibrotactile forces to be output to a user contacting the vibrotactile device.
  • the vibrotactile haptic effect is provided to be output by the vibrotactile interface device.
  • the kinesthetic haptic effect can be a periodic effect having a specified magnitude and frequency, where the mapped vibrotactile haptic effect is a vibration having an equivalent magnitude and frequency.
  • the kinesthetic haptic effect can also be a nonperiodic effect, such as a spring, damper, obstruction, or vector force, and the mapped vibrotactile haptic effect can be a vibration having a magnitude based on a magnitude of the nonperiodic effect.
  • a vibrotactile feedback device of the present invention can independently control and specify the magnitude and frequency of vibrations output by a eccentric rotating mass motor using duty cycle and frequency of a control signal and can use low-cost uni-directional amplifiers and control circuitry.
  • Techniques for mapping kinesthetic force effects to vibrotactile effects allow software designed to control kinesthetic force feedback devices to control haptic effects for vibrotactile devices. Less-expensive vibrotactile devices thus are a viable alternative to a purchaser to experience compelling haptic feedback in a variety of existing computer applications.
  • FIGURES 1 and 2 are perspective views of gamepad vibrotactile interface devices which can be used with the present invention
  • FIGURE 3 is a top view of an interior of the gamepad device of Fig. 1;
  • FIGURE 4 is a block diagram of a vibrotactile interface device suitable for use with the present invention.
  • FIGURE 5 is a graph illustrating a control signal of the present invention and resulting output vibration
  • FIGURE 6 is a graph illustrating a control signal of the present invention modified from Fig. 5 and resulting output vibration
  • FIGURES 7, 8 and 9 are graphs illustrating control signals of different frequencies and the resulting vibrations.
  • FIGURES 10a and 10b are force profiles illustrating a kinesthetic spring force and a vibrotactile sensation mapped to the kinesthetic spring force, respectively.
  • This application describes techniques for controlling a vibrotactile interface controller, such as a haptic feedback gamepad, using more sophisticated force effects, as well as techniques for enhancing the vibrotactile feedback output by such vibrotactile controllers.
  • a vibrotactile interface controller such as a haptic feedback gamepad
  • vibrotactile device or “vibrotactile feedback device” is intended to refer to any controller or interface device that outputs vibrations to the user of the device, and can include gamepads, handheld steering wheels, fishing-type controllers, joysticks, mice, trackballs, adult devices, grips, remote controls, handheld game devices, flat screens, styluses, etc.
  • the term “kinesthetic device” or similar term is intended to refer to devices that provide forces along the axes or degrees of freedom of motion of a manipulandum of the device.
  • FIGURE 4 is a block diagram illustrating a haptic system 100 suitable for use with the present invention.
  • System 100 includes a host computer 102 and a vibrotactile device 104.
  • Host computer 102 is any of a variety of computing or electronic devices.
  • computer 102 is a personal computer, game console, or workstation, such as a PC compatible computer or Macintosh personal computer, or game console system from Nintendo
  • host computer 102 can be a "set top box” which can be used, for example, to provide interactive television functions to users, or a "network-" or “internet-computer” which allows users to interact with a local or global network using standard connections and protocols such as used for the Internet and World Wide
  • Some embodiments may provide a host computer 102 within the same casing or housing as the interface device or manipulandum that is held or contacted by the user, e.g. hand-held video game units, portable computers, arcade game machines, etc.
  • Host computer preferably includes a host microprocessor, random access memory (RAM), read only memory (ROM), input/output (I/O) circuitry, an audio output device, and other components of computers well-known to those skilled in the art.
  • RAM random access memory
  • ROM read only memory
  • I/O input/output
  • Other types of peripherals can also be coupled to host computer 102, such as storage devices (hard disk drive, CD ROM drive, floppy disk drive, etc.), printers, and other input and output devices.
  • a display device 106 is preferably connected or part of the computer 102 and displays images of a graphical environment, such as a game environment, operating system application, simulation, etc.
  • Display device 106 can be any of a variety of types of devices, such as LCD displays, LED displays, CRT's, flat panel screens, display goggles, etc.
  • Host computer 102 preferably implements a host application program with which a user is interacting via the interface device 104 and other peripherals, if appropriate.
  • the host application program can be a video game, word processor or spreadsheet, Web page or browser that implements HTML or VRML instructions, scientific analysis program, virtual reality training program or application, or other application program that utilizes input of device 104 and outputs haptic feedback commands to the device 104 (or a different layer can output such commands, such as an API or driver program on the host).
  • the host program checks for input signals received from the electronics and sensors of device 104, and outputs force values and/or commands to be converted into forces output for device 104.
  • Suitable software drivers which interface such simulation software with computer input/output (I/O) devices are available from Immersion Corporation of San Jose, California.
  • ⁇ layers can be running on the host computer 102.
  • one or more application programs can be running, such as a game program, word processing program, etc.
  • Several sub-layers can also be provided, such as an
  • API Application Programming Interface
  • driver layers e.g. used in Windows from Microsoft Corp.
  • the application program can command forces directly, or a driver program can monitor interactions within an application program and command haptic effects when predetermined conditions are met.
  • a haptic feedback driver program can receive kinesthetic haptic commands from an application program and can map the commands to vibrotactile commands and effects, and then send the necessary information to the interface device 104.
  • Vibrotactile interface device 104 is coupled to host computer 102 by a bi-directional bus 108.
  • the bi-directional bus sends signals in either direction between host computer 102 and the interface device.
  • bus 108 can be a serial interface bus, such as an RS232 serial interface, RS-422, Universal Serial Bus (USB), MIDI, or other protocols well known to those skilled in the art; or a parallel bus or wireless link.
  • USB Universal Serial Bus
  • the USB standard provides a relatively high speed interface that can also provide power to actuators of device 104.
  • Vibrotactile device 104 can, in many embodiments, include a local microprocessor 110.
  • Local microprocessor 110 can optionally be included within the housing of device 104 to allow efficient communication with other components of the device.
  • Processor 110 is considered local to device 104, where "local” herein refers to processor 110 being a separate microprocessor from any processors in host computer 102. "Local” also preferably refers to processor 110 being dedicated to haptic feedback and sensor I/O of device 104.
  • Microprocessor 110 can be provided with software instructions to wait for commands or requests from host 102, decode or parse the command or request, and handle/control input and output signals according to the command or request.
  • processor 110 can operate independently of host computer 102 by reading sensor signals and calculating appropriate forces from those sensor signals, time signals, and stored or relayed instructions selected in accordance with a high level host command.
  • Suitable microprocessors for use as local microprocessor 110 include the MC68HC711E9 by
  • Microprocessor 110 can include one microprocessor chip, multiple processors and/or co-processor chips, and/or digital signal processor (DSP) capability.
  • DSP digital signal processor
  • the control techniques described for the present invention can be implemented within firmware of an Immersion TouchSense Processor, where the local microprocessor block 110 includes related components, such as encoder processing circuitry, communication circuitry, and PWM circuitry as well as a microprocessor.
  • Various techniques for playing more sophisticated periodics and other sensations (such as defined by Immersion protocols) with a eccentric rotating mass (ERM) motor can be provided in the firmware of the microprocessor 110.
  • Microprocessor 110 can receive signals from sensors 112 and provide signals to actuators 120 and 122 in accordance with instructions provided by host computer 102 over bus 108.
  • the microprocessor 110 provides a signal 111 which is defined as a "Control Signal” herein.
  • the Control Signal is a PWM signal the firmware of processor 110 can generate and send to the amplifier in actuator interface 124.
  • Another term used herein is "on-time,” which is the amount of time the Control Signal remains high once it goes high.
  • Control Signal provides the drive signal for the actuators, as described in greater detail below.
  • host computer 102 provides high level supervisory commands to microprocessor 110 over bus 108, and microprocessor 110 decodes the commands and manages low level force control loops to sensors and the actuator in accordance with the high level commands and independently of the host computer 102.
  • This operation is described in greater detail in US Patents 5,739,811 and 5,734,373, both inco ⁇ orated by reference herein.
  • force commands are output from the host computer to microprocessor 110 and instruct the microprocessor to output a force or force sensation having specified characteristics.
  • the local microprocessor 110 reports data to the host computer, such as locative data that describes the position of the mouse in one or more provided degrees of freedom. The data can also describe the states of buttons or other devices of device 104.
  • the host computer uses the data to update executed programs.
  • actuator signals are provided from the microprocessor 110 to the actuators and sensor signals are provided from the sensors 112 and other input devices to the microprocessor 110.
  • the microprocessor 110 can process inputted sensor signals to determine appropriate output actuator control signals by following stored instructions.
  • the microprocessor may use sensor signals in the local determination of forces to be output, as well as reporting locative data derived from the sensor signals to the host computer.
  • other hardware can be provided locally to device 104 to provide functionality similar to microprocessor 110.
  • a hardware state machine incorporating fixed logic can be used to provide signals to the actuators and receive sensor signals from sensors
  • host computer 102 can provide low-level force commands over bus 108, which are directly transmitted to the actuator via microprocessor 110 or other circuitry (if no microprocessor 110 is present). Host computer 102 thus directly controls and processes all signals to and from the device 104, e.g. the host computer directly controls the forces output by actuator 120 or 122 and directly receives sensor signals from sensor 112 and input devices 126. This embodiment may be desirable to reduce the cost of the force feedback device yet further, since no complex local microprocessor 110 or other processing circuitry need be included in the device 104.
  • the host 102 can also stream force values that are sent to the actuators, as described in U.S. Patent No. 5,959,613, inco ⁇ orated herein by reference.
  • the signal from the host to the device can be a single bit that indicates whether to pulse the actuator at a predefined frequency and magnitude.
  • the signal from the host can include a magnitude, giving the strength of the desired pulse, and/or a frequency.
  • a local processor can also be used to receive a simple command from the host that indicates a desired force value to apply over time, so that the microprocessor outputs the force value for the specified time period based on the one command.
  • the host command may also specify an "envelope" to provide on a force output to allow force shaping, as disclosed in U.S. Patent No. 5,959,613.
  • force values (magnitudes, etc.) can be streamed from the host computer to be output in real time by the device, as described in
  • Local memory such as RAM and/or ROM
  • microprocessor 110 can be coupled to microprocessor 110 in device 104 to store instructions for microprocessor 110 and store temporary and other data.
  • force profiles can be stored in memory, such as a sequence of stored force values that can be output by the microprocessor, or a look-up table of force values to be output based on current events in the application program or other conditions.
  • a local clock can be coupled to the microprocessor 110 to provide timing data, which might be required, for example, to compute forces output by actuator (e.g., forces dependent on time dependent factors).
  • timing data for microprocessor 110 can be alternatively retrieved from the USB signal.
  • Sensors 112 sense the position or motion of a manipulandum 114 of the device 104 and provides signals to microprocessor 110 (or host 102) including information representative of the position or motion.
  • the manipulandum is a small joystick provided on a gamepad controller and moved by a user in two rotary or linear degrees of freedom to provide control input to the host computer.
  • the manipulandum can be a direction pad having four or more directions which can provide input to the host computer.
  • the manipulandum can also be a rotary dial, linear slider, wheel, finger receptacle, cylinder, or other controlling member.
  • the manipulandum can also be the housing of the device itself, as in the case of a mouse or if sensing the position of a gamepad or other controller in 3-D space.
  • Sensors suitable for detecting motion of a joystick or other manipulandum include digital optical encoders frictionally coupled to a rotating ball or cylinder, as is well known to those skilled in the art.
  • Optional sensor interface 116 can be used to convert sensor signals to signals that can be inte ⁇ reted by the microprocessor 110 and/or host computer 102, as is well known to those skilled in the art.
  • Actuators 120 and 122 transmit forces to the user of the device 104 in response to signals or commands received from microprocessor 110 and/or host computer 102.
  • two actuators are provided, a small actuator 120 and a larger actuator 122.
  • these actuators are eccentric rotating mass (ERM) DC motors, which are rotary motors having an eccentric mass coupled to the rotating shaft of the motor. When rotated, the inertial forces from the rotating mass cause an oscillation or vibration in the housing or other member coupled to the motor housing, thus producing vibrotactile sensations to the user who is holding or otherwise contacting the housing.
  • Actuators 120 and 122 thus provide "informative" or "effect” vibrotactile forces.
  • the small motor 120 can provide high frequency, low magnitude forces, while the large motor 122 can provide lower frequency, higher magnitude forces.
  • Alternate embodiments can employ a single actuator, or two or more actuators of the same or differing sizes for providing vibrotactile sensations or forces to the user of the device 104.
  • actuators can be used, e.g. any type of actuator which can rotate an eccentric mass in a direction, such as voice coil actuators, moving magnet actuators, hydraulic or pneumatic actuators, torquers, brushed or brushless motors, etc.
  • additional actuators can be included to provide kinesthetic force feedback in the manipulandum 114.
  • Actuator interface 124 can be optionally connected between actuators 120 and 122 and microprocessor 110 to convert signals from microprocessor 110 into signals appropriate to drive the actuators.
  • Interface 124 can include power amplifiers, switches, digital to analog controllers (DACs), analog to digital controllers (ADCs), and other components, as is well known to those skilled in the art.
  • the actuators 120 and 122 are off-the-shelf ERM motors which are driven unidirectionally.
  • Uni-directional voltage mode amplifiers are low cost components that can be used in actuator interface 124 to drive the motors.
  • Other input devices 118 are included in device 104 and send input signals to microprocessor 110 or to host 102 when manipulated by the user. Such input devices can include buttons, dials, switches, scroll wheels, or other controls or mechanisms.
  • Power supply 120 can optionally be included in or coupled to device 104, and can be coupled to actuator interface 124 and/or actuators 120 and 122 to provide electrical power to the actuators. Alternatively, and more preferably, power can be drawn from the bus 108, e.g. using a USB or other bus. Also, received power can be stored and regulated by device 104 and thus used when needed to drive actuators 120 and 122 or used in a supplementary fashion.
  • a power storage device may be required in the mouse device to ensure that peak forces can be applied (as described in U.S. Patent No. 5,929,607, inco ⁇ orated herein by reference). For example, power can be stored over time in a capacitor or battery and then immediately dissipated to output a haptic sensation.
  • this technology can be employed in a wireless device 104, in which case battery power can be used to drive the tactile actuator.
  • an existing console gamepad interface device such as those shown in Figs. 1-3, can be used as interface device 104.
  • an existing gamepad controller can be provided with a more sophisticated local microprocessor to allow more sophisticated haptic sensations to be controlled and output by the interface device.
  • a Sony Dual Shock-compatible product for the PlayStation console can be modified to use an Immersion Touchsense Processor- compatible microprocessor and retain the remaining existing motors and electronics.
  • the vibrotactile controller of the present invention need retain from a standard prior art console implementation; different components can be used as necessary or to provide a better haptic output.
  • the Dual Shock controller uses a special PlayStation bus and proprietary protocol, while a PC version can use Universal Serial Bus (USB) or a Serial protocol and be DirectX/Immersion compatible.
  • USB Universal Serial Bus
  • DirectX from Microsoft Co ⁇ oration prescribes a set of conditions and haptic effects for the Windows operating system provided on a PC.
  • other standard protocols and interfaces for those platforms can be used.
  • vibrotactile interface devices which output vibrations to the user can be used with the vibrotactile control features described herein, such as gamepads, handheld or surface-grounded steering wheels, fishing-type controllers, joysticks, mice, trackballs, adult devices, grips, remote controls, handheld game devices, flat screens, styluses, etc.
  • the ERM motors described above are low cost and thus desirable for use in mass market haptic feedback devices.
  • the ERM motors can be driven in different ways to provide haptic sensations, as described below.
  • two or more of these drive methods can be employed in a single device, depending on the type of haptic sensations desired.
  • the preferred drive method is to drive the motors uni-directionally, since uni-directional amplifiers and other components are lower in cost than more sophisticated components.
  • a harmonic drive (bi-directional, like a speaker) may be applied to an ERM motor to produce exact frequencies with independently varying magnitude using a forcing function control signal.
  • the eccentric mass of the motor is driven in pendulous motion in two directions and need not ever make a complete, full rotation.
  • Such a drive has a substantially increased power demand, typically provides sensations having less magnitude, and usually requires more expensive driver circuitry than uni-directional embodiments, but may provide more precisely-controlled haptic sensations.
  • the first method is to use a uni-directional amplifier to drive the ERM in circular motion, and to pulse the mass instead of changing its direction in order to play the frequency content of a vibration.
  • the second method is to use a uni-directional amplifier to drive the ERM in pendulous (bi-directional) motion, but relying on gravity to move the mass in the opposite direction and provide the return torque instead of the amplifier, since the motor is only driven in one direction (although the motor can be slowed by commanding a voltage less than the back EMF of the motor, as is well known to those skilled in the art).
  • both of these drive schemes are utilized by the control method described below, where pendulous motion is exhibited when the commanded magnitude is sufficiently low to be unable to drive a full mass revolution.
  • the parameters that are passed to the local microprocessor 110 can determine whether the ERM spins or follows a pendulous motion.
  • a third drive method employs an interfering member, such as a stop and/or a spring, to ensure oscillatory motion of the rotating mass.
  • a stop can be placed in the path of the ERM which prevents the ERM from making a complete rotation, and which can transmit vibrations to the housing of the device from the impact of the ERM with the stop.
  • a spring material can also or alternatively be used, where the spring can be placed between the stop and the ERM.
  • ERM hits the spring (such as foam, rubber, coiled spring, flexure, or other resilient or compliant material), the spring's resilience causes the ERM to move back with greater force than without the spring.
  • the actuator force can be dynamically adjusted to provide desired vibration magnitude and simultaneously overcome spring force.
  • the spring adds energy back into the system once the actuator force is removed. The result is that a low-cost uni-directional amplifier is able to actively spin the mass in both directions.
  • a central issue addressed by the present invention is how to translate existing kinesthetic haptic effects to vibrotactile haptic effects.
  • Immersion StudioTM from Immersion Co ⁇ . is a software tool that allows a developer to design a wide variety of haptic sensations, many of which are kinesthetic effects such as spring forces, damping forces, and obstruction forces.
  • kinesthetic effects such as spring forces, damping forces, and obstruction forces.
  • kinesthetic force feedback mouse such as the Wingman Force Feedback Mouse from Logitech, Inc.
  • Many types of kinesthetic haptic effects such as springs, dampers, and obstructions, may have no meaning in a vibrotactile device if output without modification, since most vibrotactile devices do not output forces to resist or assist motion of a manipulandum of the device.
  • the present invention allows kinesthetic effects to be output by vibrotactile devices by providing mappings, where a particular vibrotactile effect is designated to be the "equivalent" of a particular kinesthetic force effect.
  • Some kinesthetic effects can be mapped almost directly to vibrotactile effects, such as vibrations.
  • Other kinesthetic effects must be approximated in ways greatly differing from the original effect due to the inability of the vibrotactile device to output directional forces on the motion of a manipulandum.
  • a particular collision effect in DirectX can be implemented as a vector force provided in a particular direction of movement of the user manipulatable object.
  • a particular vibrotactile device may not be able to output forces in a particular direction, so that the vector force must be approximated with a different vibrotactile sensation.
  • a particular vibrotactile effect can be designated to be a replacement for vector kinesthetic effects in all applications and circumstances in which that kinesthetic effect would have been output on a kinesthetic device.
  • different vibrotactile effects can be mapped to a particular kinesthetic effect based on the particular context or application in which the kinesthetic effect is used. For example, a first vibrotactile effect might be used to approximate a kinesthetic vector force in a racing game, while a second, different vibrotactile effect can be used to approximate that kinesthetic vector force in a fighting game or a flight simulator game. In many cases, such a scheme involving specific applications can provide more realistic results since the vibrotactile effects are tailored for the most realism in a particular application.
  • Kinesthetic force effects can be organized into periodic and non-periodic effects. Periodic effects such as vibrations are discussed first since these effects can be most easily translated from kinesthetic to vibrotactile domains, while other, non-periodic effect mappings are described at a later point.
  • the program when an application program is run on the host computer, the program detects what type of controller device is being used with the application program, and selects an appropriate set of effects for that device. If a vibrotactile device is used, the mappings for vibrotactile effects are used; if a kinesthetic device is used, then standard kinesthetic effects can be output.
  • the application (e.g. game) program can be completely unaware of the type of device connected (or can know the type of device but can be unaware of mappings described herein), so that the mappings are handled by a lower-level driver running on the host computer and/or by the local microprocessor/firmware on the device.
  • a vibrotactile-specific driver running on the host can evaluate outgoing commands and map them to vibrotactile commands, if appropriate, so that the device need perform no mapping tasks.
  • the local microprocessor can include a look-up table or other data to map certain kinesthetic force effects to vibrotactile effects and control the motors appropriately.
  • One or more mappings of vibrotactile forces to kinesthetic forces can (in any embodiment) also be selectively applied based on the application or circumstances in which the force effect is commanded. For example, if a kinesthetic vector force has been output for a collision occurring in a game, the equivalent vibration forces may be appropriate. But if the vector force is being used to simulate a dynamic spring return on the steering wheel during a turn, then the vibration force may not be appropriate and is not commanded to be output.
  • the command protocol can include one or more parameters indicating the pu ⁇ ose for the command, such as "collision,” "turn,” or “spin out.”
  • a specialized vibrotactile driver on the host (or the local microprocessor) can examine the pu ⁇ ose parameter to determine whether to output a force based on stored instructions governing such output.
  • other methods are preferred.
  • a set of force effects which are intended to be used with vibrotactile devices can be available to developers when programming haptic effects and available to be installed by users on a computer so that mappings are available to a driver.
  • Each vibrotactile set of haptic effects can be intended for a particular broad category of applications, such as driving games, flight simulation, fighting games, a word processor, Internet browser, DVD interface, etc.
  • a set of mappings can be specific to one particular application program, such as a particular game.
  • the user obtains the set of effects for the application and loads the set to the operating system or otherwise makes the set available to the vibrotactile driver (or local microprocessor).
  • a set of effects for all automobile driving games would map vibrotactile effects to particular kinesthetic forces which are appropriate and effective for driving circumstances in these games.
  • These sets of pre-mapped vibrotactile effects can be available from a central provider or developer, e.g. downloaded from a website or provided with an application program.
  • a set of mappings may also include settings for overall magnitudes provided in a "control panel.” For example, a control panel driver can be available on the host computer to allow the user to adjust the overall magnitudes of particular force sensations (or an overall gain of all force sensations) to a desired level.
  • a set of mappings for vibrotactile effects can also include settings for these control panel force sensations to maximize the effectiveness of the vibrotactile effects for that type of application program.
  • the mappings for a driving game might also set spring forces to a lower overall magnitude and increase vibrations to a greater overall magnitude.
  • Some embodiments can allow the device (or a haptic driver) to ignore all kinesthetic effects that do not directly translate to the vibrotactile domain when a vibrotactile device is being used. Thus, all spring, damper, attraction, and obstruction forces can be ignored, allowing mainly jolts and vibrations to be only output. However, since some types of kinesthetic forces, such as some obstructions, can be effectively simulated or approximated using vibrations, the types of kinesthetic forces that are ignored can vary depending on the embodiment or application.
  • some particular considerations can make the vibrotactile device simple with a low production cost, for those embodiments with such cost constraints.
  • a stock eccentric mass motor can be used to take advantage of existing supplies of these motors.
  • the motor drivers should be assumed to operate uni-directionally, since that is the most common type of motor operation in current vibrotactile devices.
  • uni-directional voltage-mode amplifiers are significantly cheaper than sophisticated current-mode amplifiers used in kinesthetic devices, and they can be built with a single transistor. Since torque control is not crucial to most embodiments, a current-mode amplifier need not be used.
  • sensors for detecting the position of the rotating mass are expensive and can be omitted for cost reasons, as in prior art gamepad vibrotactile controllers. Such a sensor can be added in some embodiments if necessary.
  • the product can use the Immersion Touchsense Processor from Immersion Co ⁇ . or similar microprocessor. No power limitations are preferably imposed on actuator output, and power from the communication bus is preferably sufficient for device operation (e.g. using a USB bus).
  • the device have minimal power requirements so as to avoid the use of bulky power supplies and/or to preserve the life of limited power sources such as batteries.
  • gamepad vibrotactile devices can meet the Universal Serial Bus (USB) power specification of maximum 0.5 A at 5 volts, thereby allowing all power to be received over the USB.
  • USB Universal Serial Bus
  • Other measures can be taken as well.
  • each motor's power requirements can be quantified. In some embodiments, it may be necessary to remove one of the two motors from the system entirely to meet power requirements. If the motors are of different sizes, then the small motor can be removed and the large motor can be used to reach about 35 Hz with independent magnitude and frequency control. Beyond 35 Hz, frequency changes may become imperceptible.
  • the small motor can provide fuller bandwidth but may not be able to meet magnitude requirements within that bandwidth.
  • both motors can be turned on simultaneously when each period begins. However, if one motor's phase is offset so that the it does not get pulsed until the other motor's on-time has completed, then the two motors are prevented from operating simultaneously, and peak power may be reduced. It should be noted that the phases of the two motors should not be offset by 180 degrees, or the resulting vibration may feel to the user as if it is double the intended frequency. Preferably, one motor is offset by the maximum "on-time per period" of the other motor (see below).
  • a dynamic algorithm e.g., running on the local microprocessor
  • a final scheme to reduce power reduces the magnitude of the control signal in favor of increasing the on-time of the control signal. This should deliver the same amount of energy to the motor, though this technique may undesirably lessen the magnitude and impact of the vibrations.
  • Periodic effects are those force feedback effects which are based on time and controlled with a periodic waveform.
  • a vibration is a basic periodic effect in which a waveform type, such as a sine wave or square wave, is commanded to output the vibration at a particular frequency (period) and magnitude.
  • the effect does not require other data such as velocity or position of a user manipulandum or a direction of output.
  • Single, non-directional jolts can be output as a single period of a vibration or part of a period of a vibration.
  • ERMs rotate ERMs at a fixed magnitude and frequency, the two of which are tightly coupled.
  • high frequency vibrations are necessarily high magnitude
  • low frequency vibrations are necessarily low frequency.
  • the present inventive method of control allows independent variation of magnitude and frequency of a 1 degree-of-freedom (DOF), unidirectionally-driven rotary actuator.
  • DOF 1 degree-of-freedom
  • the technique is significant because it enables an ERM to create complex vibrations, like decaying sinusoids or superimposed waveforms.
  • An ERM in prior devices had a magnitude that was roughly linearly coupled to its speed.
  • the present invention utilizing no extra clutches, circuitry, or mechanical parts and using only these control methods (e.g.
  • control methods described herein can be applied not only to rotational motors but other types of 1 DOF actuators, rotational and linear, including moving magnet motors, solenoids, voice coil actuators, etc.
  • a frequency command, a magnitude command, and a function are supplied as parameters or inputs to the firmware.
  • a vibration is controlled with magnitude, frequency, and function type parameters (and additional parameters, if desired). Therefore, a direct mapping between kinesthetic and vibrotactile devices is possible with the present invention, i.e., no mapping may be needed, where the existing periodic effect can be commanded directly to the kinesthetic or vibrotactile device, unchanged.
  • FIG. 5 An example is illustrated in the graph 150 shown in FIGURE 5, showing a time vs. magnitude relationship.
  • a sine wave 152 of frequency 5 Hz and 50% magnitude is shown as the desired vibration to be output by the device (this, and all subsequent similar figures, capture 1 second of input and output signals).
  • the present control method determines where each period of the waveform begins (or should begin), then raises a control signal 154 to a high or "on" level for a specific duration once per period; the control signal 154 is "off or low during the other times.
  • the periodic control signal has a frequency based on the desired (commanded) frequency.
  • control signal 154 By pulsing the actuator once per period using control signal 154, the perception of a vibration with specified frequency is conveyed to the user.
  • the control signal can be raised to a high level at the beginning of a period, as shown, or at a different time within the period.
  • Magnitude of the periodic effect is portrayed by adjusting a duty cycle of the control signal, e.g. the duration at each period ("on-time per period") of the control signal 154.
  • the control signal 154 is either on or off, but the amount of time per period for which the control signal remains on is determined by the magnitude command or parameter.
  • a sine wave with 50% of the possible magnitude is requested. According to the present invention, this requested magnitude generates a control signal 154 that comes on every 250ms for 15ms duration.
  • a 100% magnitude waveform with the same frequency is offered in FIGURE 6, which shows a similar graph 160 to graph 154.
  • the control signal 164 comes on at the same interval as the control signal 154 described above, since the frequency command has not changed.
  • control signal 164 stays on twice as long to produce the feeling in the user of twice the magnitude of vibration.
  • the longer that the control signal is on the longer the actuator spends accelerating.
  • the ERM reaches a larger angular velocity and, since force is proportional to the square of angular velocity, larger forces are perceived at the user's hand.
  • the mass is never allowed to stop rotating, so that static friction need be overcome only once, at the start of rotation. If the control signal remains on too long, the rotating mass will make multiple revolutions and eventually reach its natural (resonant) frequency. At that point, the user will perceive the natural frequency of the system and not the commanded frequency.
  • on-time can be regulated as a "percentage of period.” If the control signal comes on for a fixed percentage of each period, as frequency increases, on-time per period decreases. Yet the control signal comes on more often. The end result is that the control signal spends the same amount of time on each second, no matter what the frequency. This technique offers the advantage that, as frequency increases, constant power is added to the actuator, and perceived magnitude stays the same over the frequency range.
  • the mass may not actually rotate in one direction, but may move bi-directionally due to the small driving time and gravity that causes the mass to move back in the opposite direction once the control signal is turned off. However, this does not adversely affect the output vibration sensed by the user, which is perceived at the desired frequency and magnitude regardless of whether the mass completes full rotations in one direction or is moving bi-directionally.
  • FIGURES 7 and 8 illustrate graphs 170 and 180, respectively, showing different control signals and the resulting sine wave for vibration.
  • the frequency of the sine wave 172 is 5 Hz and the magnitude is 100%.
  • the frequency of the sine wave 182 is 10 Hz and the magnitude is 100%.
  • the control signal 174 of Fig. 7 is on for about 40 ms at each period, and the control signal 184 of Fig. 8 is on for about 20 ms at each period. Comparing the control signals of Figs.
  • FIGURE 9 is a graph 190 showing 1 second of a sine waveform 192 that produces a vibration of low frequency (1 Hz) and 100%o magnitude; only one period is provided for the 1 second duration.
  • the control signal 194 is set high at the beginning of the period, for 125 ms.
  • Both graphs 180 of Fig. 8 and 190 of Fig. 9 set the control signal high for the same total amount of time per second.
  • Fig. 9 shows 1 second of a sine waveform 192 that produces a vibration of low frequency (1 Hz) and 100%o magnitude; only one period is provided for the 1 second duration.
  • the control signal 194 is set high at the beginning of the period, for 125 ms.
  • Both graphs 180 of Fig. 8 and 190 of Fig. 9 set the control signal high for the same total amount of time per second.
  • a second method of the present invention can avoid this problem at low frequencies and thus may provide a more suitable way to output vibrations for many vibrotactile devices.
  • the second method sets the control signal high for a fixed maximum amount of time per period, not a percentage of the period.
  • the on-time for 100% magnitude for any frequency is the same.
  • the on-time for commanded magnitudes less than 100% are lower in proportion to the amount of the commanded magnitude under 100%. This effectively establishes a maximum on-time per period, prohibiting the actuator from coming on long enough to make multiple revolutions during one continuous on-time.
  • the actuator is allowed to make multiple revolutions (e.g., more than about 2 or 3 in some embodiments)
  • the user will perceive a higher frequency (e.g., about 25 Hz for some motors) based on the rotation speed of the actuator rather than the commanded frequency (e.g., which might be less than 10 Hz), so this method prevents that result.
  • a request of 100% magnitude at a lower frequency for a particular motor can be equated with the on-time that causes the mass to rotate just under the number of revolutions that cause the user to feel more than one pulse for a single period (such as 2-3 revolutions); this on-time can be determined empirically.
  • a drawback to the second technique is that as frequency increases, the separate on-times get closer together, and the actuator is eventually, in effect, requested to remain on for longer than one period. At that point, the control signal is always being asserted, the mass rotates continuously, and frequency and magnitude no longer vary independently.
  • one preferred embodiment of the present invention combines or blends the two techniques to avoid the drawbacks in each method.
  • the second method is used only when commanded frequencies are below a particular blend threshold frequency and the first method can be used for commanded frequencies above that threshold frequency. Blending is possible even if the magnitude of the control signal also varies.
  • the blend threshold is chosen based on dynamics of the system; the blend frequency is the frequency at which the on-time will be the longest, so a blend frequency should be chosen that will provide one vibration pulse (e.g. less than two mass revolutions) per period for an on-time corresponding to 100% magnitude at that frequency.
  • 10 Hz can be used as a blend threshold frequency.
  • the first method (“percentage of period”) is used to calculate the on-time of the control signal, and for commanded frequencies below 10 Hz, the second method (“fixed time per period”) can be used.
  • Other thresholds can be used in other embodiments.
  • scalars are chosen so that maximum magnitude for the two methods matches at the blend threshold frequency, i.e. the transition between methods is smooth. For example, a 25 ms control signal on-time at 10 Hz may generate a 10 Hz, 100%) magnitude vibration.
  • the "percentage of period" method is scaled to generate 25 ms on-times at 10 Hz, and those scalars used are retained and applied to this method for frequencies above 10 Hz.
  • more advanced blending techniques can be used such as those that mimic band pass filters in the region of the blend, or low pass/high pass combinations on either side of the blend threshold frequency.
  • a third method to allow the command of magnitude independent of frequency is to vary the amplitude of the control signal proportionally to requested magnitude, rather than having only two levels for the control signal. This can be performed alone or in conjunction with either or both of the first and second methods described above. For example, other types of waveforms having varying amplitude might be used as control signals (sine wave, triangle wave, etc.).
  • One efficient way to set or vary the amplitude of the control signal is to provide pulse-width modulation (PWM) during the chosen on-time for control signals as presented above, or to vary the control signal duty cycle during the on-time using some other method.
  • PWM pulse-width modulation
  • PWM may require a separate PWM module, which can add cost to the device.
  • the first and second methods described above can be implemented by bit-banging, in which the local microprocessor outputs the control signal directly to the actuators without using a PWM module.
  • bit-banging does not allow the control signal magnitude to be directly controlled, but removes the requirement for a PWM module and potentially reduces processor or interface device cost.
  • Dual Shock gamepad uses two motors, and both can be used according to the present invention to reach up toward 100 Hz dynamic range.
  • the large mass/motor combination can use both control techniques over its frequency range, with a blend threshold at 10 Hz.
  • the small mass/motor combination can use only the second method ("fixed-time per period" technique) to add power to the small motor with increased frequency (since on-time does not decrease with frequency increase using the second method). This reduces the tendency of the larger amplitude vibrations of the large motor to drown out the small motor's smaller, high frequency vibrations.
  • filters can be used to achieve the desired balance between large and small motor contribution over the frequency range.
  • the small motor may deliver so little magnitude that the vibration is almost undetectable by the user.
  • the frequency dynamic range can be shortened to 25Hz by defining the device to have a maximum capable frequency of 25 Hz, choosing scalars appropriately. This allows every haptic effect to be played using the large motor, and the small motor is then unnecessary.
  • a better 5V solution may be to increase the small ERM to a more moderate size, allowing stronger vibrations to be output.
  • a more simplified control method might be used for periodic effects without using the methods described above.
  • a commanded magnitude can be ignored, i.e. only the frequency command is used.
  • a predetermined threshold frequency can be used to determine which motor is used to output vibrations. For example, if a vibration above 15 Hz is commanded, the small motor/mass is rotated. The small motor/mass can be commanded with a frequency proportional to the commanded frequency, or, alternatively, may only rotated at one frequency. This effect can be used for such events as the engine of vehicle revving in a driving game. If a vibration below 15
  • the large motor/mass can be rotated at a predetermined frequency or by a frequency proportional to the commanded frequency (where the commanded frequency is scaled to a frequency in the range of the large motor). This effect can be used for such events as moving over bumpy ground or collisions in a game.
  • Nonperiodic effects are determined based not on time and magnitude parameters, but on other parameters instead of or in addition to these parameters. Nonperiodic effects are not as simply mapped to vibrotactile devices as periodic effects. For example, a vector force is a non- period effect which is output in a specified direction for a specified length of time. Since vibrotactile devices generally cannot output forces in particular directions, the mapping of the vector force to a vibrotactile force sensation is not as direct as with periodic effects.
  • Condition effects are closed-loop force feedback effects which can be commanded on more sophisticated kinesthetic force feedback devices, as opposed to the open-loop effects such as vibrations and jolts.
  • Condition effects typically require position and/or velocity (and/or acceleration) data of a user manipulandum (such as a wheel, joystick, or mouse) in the determination of an output force.
  • a user manipulandum such as a wheel, joystick, or mouse
  • spring condition forces are dependent on the current position of the user manipulandum and damper condition forces are dependent on the current velocity of the user manipulandum.
  • conditions can be simply ignored, so that if an application program commands a condition force to be output, the local microprocessor of the vibrotactile device (or a driver or other program on the host) simply ignores the command and no forces are output.
  • conditions can be mapped to other vibrotactile sensations that allow a user to experience at least some force feedback that approximates a condition force effect or signifies an event. Descriptions of examples of mappings for non-periodic force effects follow.
  • Vector forces are kinesthetic force effects which provide a force in a specified direction in one or more degrees of freedom of the manipulandum. For example, in a racing or driving game, a vector force is used to output a collision sensation in the rotation of a steering wheel when the car impacts an object such as a wall or other car, a shift sensation when a stickshift in moved, or a spin out sensation when the car loses control. The magnitude of the vector force is different based on the severity of the event or other factors.
  • a directional force cannot typically be output and must be approximated using vibrations or jolts.
  • a jolt or vibration output from a vibrotactile device can be similar to a vector force.
  • a vibration is output having a low frequency and high magnitude; for example, a single jolt can be output by commanding a 1 Hz frequency with a duration of 1 second, to provide a single rotation.
  • commands for particular vector forces can be ignored for vibrotactile devices.
  • low magnitude vector forces might be ignored by the vibrotactile device since such forces are often used to provide subtle driving forces that are not easily duplicated with vibrotactile effects.
  • the large motor can be caused to rotate to cause a vibration corresponding to the vector force; or both motors can be rotated.
  • a vector force can be directly mapped to a vibration, where the vibration has a magnitude corresponding to the magnitude of the vector force.
  • the magnitude is preferably independently controlled using the methods described above with respect to periodic forces.
  • the magnitude can be proportional to or otherwise based on the magnitude parameter of the vector force command.
  • the frequency of the vibration can be a constant frequency predefined at a particular value; for example, the fixed frequency can be the highest frequency the device is capable of outputting.
  • the vibration frequency for vector forces can also be user selectable.
  • vector force magnitude directly to motor command, i.e. the magnitude of the vector force, as expressed by a duty cycle, is used as a command to rotate the motor, e.g. a 50% magnitude becomes a 50% duty cycle motor command. This causes a pulse of force that substitutes for the vector force.
  • mapping kinesthetic vector force magnitude commands instead of mapping kinesthetic vector force magnitude commands to the full range of motor amplitude commands, the vector force magnitudes can be mapped to only the upper half of the motor amplitude command range. This allows high enough magnitudes to be commanded to always cause a full mass revolution, and thus makes the feel response more linear.
  • amplitude command range can be used to always provide harmonic mass motion.
  • magnitude thresholds for the requested magnitude can be used to determine when and how to approximate vector forces for devices having multiple motors.
  • a first threshold can be provided at 75% magnitude (i.e., 75% of the maximum magnitude that can be output).
  • the large motor can be controlled to output a vibration (or single jolt) that has a magnitude proportional to the commanded vector force magnitude. If a commanded vector force has a magnitude of 76% to 100%> of maximum, then both the large and small motors can be used to provide vibration forces having the maximum possible magnitude.
  • thresholds such as described above can be used in conjunction with the methods used with periodics to independently control magnitude and frequency of vibrations within particular thresholds. Such combination techniques can be used with all the types of force effects described herein.
  • the mapping to vector forces can (in any embodiment) also be selectively applied based on the application or pu ⁇ ose commanded. For example, if the vector force has been output for a collision occurring in a game, the equivalent vibration forces may be appropriate. But if the vector force is being used to simulate a dynamic spring return on the steering wheel during a turn, then the vibration force may not be appropriate.
  • directional forces may be able to be output.
  • a rotating or moving mass can be controlled to rotate so that a directional force is approximated in a particular direction of the degree of freedom of the mass.
  • the motors can be controlled to rotate in phase with each other to cause a force in a particular direction.
  • a vector force can be mapped to a directional force on the vibrotactile device if a force can be output in or near the direction of the vector force.
  • spring forces can be used to center the steering wheel to the center position, where the user feels a force having a magnitude proportional to the amount of rotation of the wheel from the center position and in a direction towards the center position.
  • the spring command can be ignored by the vibrotactile device since the spring may not be effectively duplicated using vibrations or jolts.
  • the spring force can often be ignored when it is used to center a user manipulatable object in a degree of freedom, since a vibration or jolt would simply be distracting to the user in such a circumstance.
  • a vibration force can be output for the spring force.
  • a centering spring force tends to move the steering wheel toward the center.
  • a spring force can be output to simulate this effect for a kinesthetic device.
  • a vibration can be output.
  • FIGURES 10a and 10b One example is shown in FIGURES 10a and 10b.
  • Fig. 10a is a graph showing a typical spring force output on a kinesthetic force feedback device, where displacement of a manipulandum along the x-axis from the origin O causes a spring force resisting that motion to be output, having a magnitude increasing with the amount of displacement.
  • Fig. 10b shows an example of a mapping to a vibrotactile domain.
  • a vibration is output only above a particular spring command magnitude threshold, such as 75%, and may have a fixed, predetermined frequency (or user- selectable frequency). Thus, the vibration is output only for higher-magnitude spring commands.
  • the vibration is only output at manipulandum positions (x-axis) corresponding to a 75% magnitude or greater on the spring force of Fig. 10a.
  • the vibration magnitude can be increased at a steep rate in proportion to the spring magnitude command, up to the maximum magnitude, as shown. This can be used to simulate the wobbling or shimmying of a steering wheel at high-speed turns in a vehicle; thus, the spring force effect is not directly simulated, but a different effect is output as a substitute for the spring force effect.
  • a vibration can be output for all magnitudes of the spring command (e.g., dependent on the position of the user manipulandum).
  • Springs can also be played as vector forces as described above, e.g. on only one motor, such as the smaller motor. This may be helpful to some users while gaming since it may provide feedback about the joystick position; however, other users may find it disconcerting. If condition forces are played on an ERM-based vibrotactile controller, the user should preferably be given the opportunity to turn them off or alter them per situation, perhaps in a profiler program presented at the application layer or other software layer on the host computer.
  • Obstructions such as walls are often simulated with force feedback in position control applications by using a spring force.
  • an encounter with a wall or other object can be simulated using a jolt or low frequency vibration.
  • the force is output at close to maximum magnitude when the initial contact with the wall is made.
  • the vibration can be continued if the user-controlled graphical object (e.g. cursor, car, etc.) continues to be in contact with the wall.
  • the vibration can be ramped down to an eventual zero magnitude using the control methods described above for periodic effects.
  • Such control methods work with envelopes applied to periodic waves, such as to slowly ramp down magnitude of a vibration.
  • Such use of periodics with attack and fade parameters are described in greater detail in U.S. Patent No. 5,959,613.
  • a damping force can be output on a steering wheel when the controlled car moves into a simulated pool of oil or water, indicating the sluggishness of steering in the body of liquid.
  • a damping force can be mapped to high frequency vibrations.
  • the small motor/mass can be controlled to rotate to a small degree or at a low frequency, e.g., just enough to indicate a change in the terrain type to the user in a driving game.
  • This can simulate "sloshing" or other vibration effects such as when a vehicle moves through a liquid.
  • a damping force can be mapped to a vibration that has a magnitude in proportion to the current velocity of the controlled object (such as a vehicle or cursor), or proportional to the current velocity of a user manipulandum on the device, such as a small joystick.
  • the vibration magnitude can be implemented as described above for periodics.
  • a damping command can be ignored by a vibrotactile device. For example, if a damping force is used to provide greater control to the user in a game, a vibration would probably not provide such greater control and should not be output.
  • Any other types of kinesthetic force effects can also be mapped to vibrotactile effects in similar ways as described above.
  • friction forces might be mapped using a high frequency vibration, which has a magnitude correlated with velocity of a controlled graphical object or with the velocity of the user object.
  • Inertia forces might be mapped using low frequency or subtle vibrations to indicate the change of conditions.
  • Texture forces provide jolts, pulses, or other forces to a user based on a manipulandum 's movement through a texture field having features causing the jolts at particular locations.
  • Such texture forces can be simulated by outputting vibrations with the ERM motor having a frequency approximately corresponding with the rate of encountering the texture features by the user.
  • mappings are provided between kinesthetic effects and vibrotactile effects to allow a vibrotactile device to respond to full-featured application force commands.
  • a novel control method is described to play periodics on simple ERM motors moving uni-directionally, where frequency and magnitude are varied independently. Other mappings provide compelling vibrotactile effects in place of kinesthetic effects.
  • the end result is a control method that allows a low-cost vibrotactile product to play at least a subset of kinesthetic forces. Since conditions and other kinesthetic force effects may be greatly reduced in effectiveness or not played at all on lower-cost vibrotactile devices but are easily played on more expensive kinesthetic haptic feedback devices, this allows for product differentiation in the market. Customers are further able to choose how much haptic feedback functionality and sophistication they want based on how much they are willing to pay for a haptic feedback interface device.
  • the hardware topology shown in Fig. 4 supports all of the control schemes described above.
  • there is no sensor feedback to the host computer or local microprocessor regarding the position of the motors i.e. the system is open loop.
  • the control methods discussed above need no feedback regarding the positions of the motors. Since no special sensors are required to provide motor position to a controller, the open loop system allows the interface device to be very cost effective.
  • One method of ensuring that the mass is rotating when it should be, thereby providing consistent forces is to model system dynamics. This method does not require feedback sensors; rather, a model of the physical system is developed beforehand and the model is used to predict the ERM's behavior. The predictions are then used to improve ERM performance as the ERM starts and continues to move.
  • a different technique more easily implemented is to command a motor to spin at maximum magnitude for an initial period of a commanded force sensation to start the motor. This causes the motor to behave in a consistent manner if the worst case scenario is planned for, e.g. an orientation of the interface device causing the ERM to move against gravity where gravity is strongest.
  • the ERM is still forced to spin, thus allowing the commanded sensation to be output more easily once the ERM is in motion.
  • this initial period of maximum rotation is commanded using the "fixed time per period" technique described above for periodic force sensations.
  • the output reverts to the commanded parameters that specify a particular magnitude and frequency (and if the frequency is low enough, the "fixed time per period" method is still used). No sensor or feedback loop is required for this embodiment, allowing the device to be reduced in cost.
  • Yet another technique to ensure desired operation, without the use of feedback sensors, is to carefully quantify desired motor operation over the entire frequency/magnitude domain beforehand, then maintain a counter during operation. If a haptic effect should cause the motor to fully rotate, and the counter ⁇ "max" (the maximum count value of the counter), then the counter is incremented and the requested control signal is output. If the haptic effect is known to not rotate the motor (known through empirical testing) but is desired to cause it to rotate, and the counter > 0, the counter is decremented, and the requested control signal is output. If a haptic effect is in the regime (i.e.
  • low-cost specifications can be relaxed and a sensor can be provided to sense the position and/or motion of the motor.
  • the most important use for a sensor is to close a feedback loop around the actuator and ensure it is moving properly (either oscillating or spinning).
  • a quadrature encoder on the motor shaft can be used for a sensor, but its cost and advanced functionality may be excessive for this application.
  • a simple Boolean sensor either an IR sensor blocked by the ERM, or Hall effect sensor with flux path through a steel or other metal ERM, can be used. This simple type of sensor can indicate to the local microprocessor whether the mass is making full revolutions.
  • the senor can be positioned at top dead center, i.e. at the point of highest travel of the mass, where gravity has the greatest adverse force against reaching this point. This ensures that the ERM triggers the sensor only when the ERM has traveled over the top and completed a full revolution, not just a large pendulum oscillation.
  • the sensor position can be recorded and stored. After a fixed amount of time that is dependent on the inertia of the ERM and the frequency of the commanded effect, the sensor output is checked to determine if it has toggled or otherwise sensed the passage of the ERM. If the ERM has not crossed the top center position in the allotted time, the actuator force can be increased beyond the amount commanded by the periodic.
  • a Boolean sensor can also be used to make sure the mass is oscillating with a minimum magnitude (as opposed to rotating) by following a similar process as just described but by locating the sensor at bottom dead center. If the ERM oscillates enough to cause the sensor to toggle briefly before the ERM swings back, a minimum magnitude of oscillation can be ensured. If an encoder is used, control can become more sophisticated to further "robustify" and ensure vibration output, since the encoder has a greater resolution and can sense the ERM at many positions. A position controller can ensure that the ERM has rotated fully when it should, or that oscillations (when the ERM is not commanded to cross top dead center) are of exact magnitude.
  • velocity control can be performed to convey vibration magnitude instead of controlling open-loop control signal on-times.
  • the encoder can detect the velocity of the motor and determine whether it is at the commanded velocity. The sensed velocity is fed back to the commanding controller and the controller adjusts the command to provide the desired motor velocity.
  • This technique though simple, can add a significant amount of repeatability to each periodic command, no matter how the motor had been positioned or moving beforehand.
  • the initial pulse is barely detectable to the user and helps the ERM accelerate to speed quickly.
  • One disadvantage with this method is that some force effects having increasing envelopes may have a very low magnitude or frequency initially, and the initial kick may be too high a magnitude and/or frequency to subtly output the desired effect. Also, if the initial kick occurs too soon, the ERM may settle into a resting state before the force effect can be output.

Abstract

The invention entails a haptic system (100) controlling haptic sensations from a vibrotactile feedback device (104) connected to a computer (102). The vibrotactile device (104) includes an actuator (120) having a rotatator mass, and receives information which causes a periodic control signal, which then controls the actuator to rotate the mass to induce a vibration in the device, wherein magnitude and frequency can be adjusted independently of each other by adjusting the control signal. Vibration magnitude is based on control signal duty cycle, and vibration frequency is based on control signal frequency. Kinesthetic haptic effect that causes vibrotactile forces to be output. The kinesthetic haptic effect can be a periodic or nonperiodic effect.

Description

CONTROLLING HAPTIC SENSATIONS FOR VIBROTACTILE FEEDBACK INTERFACE DEVICES
BACKGROUND OF THE INVENTION
The present invention relates generally to control techniques for human-computer interface devices, and more specifically to controlling vibrotactile haptic sensations from a vibrotactile feedback device.
Computer devices are widely used for entertainment activities such as playing games. Currently, popular gaming computer devices include game consoles connected to a home television set, such as the Nintendo® 64 from Nintendo Corp., the Playstation® from Sony Corp., and the Dreamcast™ from Sega Corp. Gaming computer devices also include personal computers, such as Windows PCs, Macintosh computers, and others. Also, portable computer devices are often used for entertainment purposes, such as Game Boy® from Nintendo, personal digital assistants such as PalmPilot® from Palm Computing, and laptop computers.
Users of these computer devices typically interact with a game or other application program using an interface device connected to the host computer (e.g. game console). Such interface devices may include joysticks, gamepads, mice, trackballs, styluses, steering wheels, or other devices. A user moves a user manipulatable object (manipulandum), such as a joystick, wheel, mouse, button, or other object, which is sensed by the host computer and used to manipulate a graphical environment displayed by the host computer. Recently, haptic feedback in interface devices has become available, where the host computer and/or a microprocessor on the interface device controls one or more motors to output forces to the user. These forces are correlated with events or objects in the graphical environment to further immerse the user in the gaming experience or interface task. Herein, the term "haptic feedback" is intended to include both tactile (or vibrotactile) feedback (forces transmitted to user skin surfaces) and kinesthetic
(force) feedback (forces provided in degree(s) of freedom of motion of the manipulandum).
In the game console market, products are typically produced in high volume and low cost. Therefore, haptic feedback interface devices have been typically limited to simpler embodiments that provide more limited forms of haptic feedback. Existing force feedback "gamepad" controllers (or add-on hardware for gamepad controllers) that are used to interface with games running on game consoles include the Dual Shock™ from Sony Corp., the Rumble Pak™ from Nintendo Corp., and the Jump Pack from Sega Corp. These devices are vibrotactile feedback controllers which employ one or more motors to shake the housing of the controller and thus provide output vibrations to the user which are correlated to game events and interactions. Typically, an eccentric rotating mass (ERM) motor, i.e., pager motor, is used to generate vibration on the controller and thus to the user. This type of motor provides a mass on a rotating shaft offset from the axis of rotation, so that when the shaft is rotated, the mass rocks the motor and the gamepad housing back and forth.
For example, a gamepad controller 10 similar to the Dual Shock controller is shown in FIGURE 1. Controller 10 includes a d-pad 12 for directional user input, buttons 14 for button user input, and one or more joysticks 16 for directional user input. Controller 10 also includes one or more motors integrated with the housing of the controller, as explained below with reference to Fig. 3. The user grasps both grips 11 while operating the controller to feel vibrations through the housing of the controller. A different controller assembly 20, similar to a Nintendo gamepad controller and Rumble Pak add on, is shown in FIGURE 2. In this configuration, the gamepad controller 22 provides the directional and button input similarly to the controller 10, but does not include motors for vibrotactile feedback. Add-on unit 24 can include one or more motors and can be connected to an interface port 26 of the controller 22 to provide vibrotactile feedback to the housing of controller 22.
A sectional view of controller 10 is shown in FIGURE 3. Controller 10 uses two individual ERM motors. In one grip 26 of the controller is a small rotary motor 28 which includes a small mass 30 coupled to its rotating shaft. In the other grip 32 is a large rotary motor 34 which includes a large mass 36. Both motors 28 and 34 are typically driven with a single transistor voltage-mode amplifier, where each motor can be activated to rotate its associated mass in one direction at a single speed. The large motor 34 is activated to display large amplitudes or lower frequencies of vibrations to the user, and the small motor 28 is activated to convey higher frequencies or smaller amplitude vibrations to the user. The existing Dual Shock specification allows 8-bit variability of voltage to the large motor 34 and therefore a variable range of output vibration magnitude. The voltage to the small motor 28 is specified as on or off and thus only allows a single magnitude of output vibration. Typically, only one motor 28 or 34 in the controller is activated at one time, but the activated motor may be user-selectable in some games. Though the large motor speed can be varied, many game programs tend to send DC commands to the large motor, where each command can cause the motor to deliver one of 255 available vibrations (e.g., there are 255 different available frequencies of vibrations). Individual games on the host console unit control when the motors are turned on and off by sending commands or signals to the controllers to correlate vibrotactile feedback with game events.
The Rumble Pak and Jump Pack tactile add-on units for the Nintendo and Sega systems use similar ERM motors but currently provide a single motor for vibration output. For example, the motor of the Rumble Pak is typically either turned on (at a maximum or a predetermined frequency) or turned off, and does not provide variable vibration frequencies. The Jump Pack uses an analog value to control variable vibration magnitude. Still other controllers, such as the Mad
Catz Dual Force Racing Wheel, are handheld controllers that are compatible with existing gaming consoles and provide similar vibrotactile feedback using ERM motors. One problem with existing vibrotactile controllers is that the frequency of output vibrations is closely coupled to the magnitude of the vibrations. For example, in the Dual Shock controller, when any one of the available vibrations is output, magnitude and frequency are tightly coupled — they cannot be varied independently. A particular vibration frequency (mass rotation speed) is commanded, and that frequency has a particular vibration magnitude associated with it, as felt by the user, which cannot be changed.. Thus, these devices are very limited in their range of output tactile feedback; only high-magnitude, high-frequency vibrations, or low-magnitude, low- frequency vibrations can be output, with no ability to independently vary these parameters.
While vibrotactile controllers for gaming consoles are substantially simpler than more sophisticated force feedback devices for personal computers such as the PC, tactile feedback in console games is still very effective and interesting to users. Furthermore, the console vibrotactile devices are much less expensive than more sophisticated kinesthetic personal computer force feedback devices, such as steering wheels and flight simulator joysticks, which provide forces in the axes of motion of these devices. The console vibrotactile devices, when provided for personal computers, can present an attractive option to computer users who wish to experience haptic feedback but do not wish to spend a relatively large amount of money on an interface device. However, one problem is that vibrotactile feedback is typically not readily controllable with existing force feedback control methods used for personal computers. For example, most haptic feedback effects on personal computers are provided using the Immersion protocol from Immersion Corporation and DirectX from Microsoft Corp., which allow games to command vibration forces using independent frequency and magnitude parameters. However, the existing gamepad vibrotactile devices and control schemes do not allow independent control of frequency and magnitude of vibration. Furthermore, the existing computer protocols allow a game to output a variety of force feedback effects including not only vibrations but vector forces, spring forces, damping forces, texture forces, and other force effects which are unknown to gamepad controllers or other vibrotactile controllers. What is needed is a method of controlling vibrotactile feedback to allow a greater variety of vibrotactile sensations, and also to allow a vibrotactile controller to be compatible with an application program which expects to be outputting effects to a more sophisticated, kinesthetic force feedback device.
SUMMARY OF THE INVENTION
The present invention is directed to mapping kinesthetic force sensations to vibrotactile sensations output by a vibrotactile feedback device connected to a computer. Sophisticated vibrations can be output from a haptic feedback device that includes driver electronics driving a rotating mass in one direction, and allowing the magnitude and frequency of the resulting vibrations to be controlled independently. Other kinesthetic force effects can be mapped to other vibrations and motor control methods. More specifically, one aspect of the present invention relates to a method for controlling a vibrotactile interface device, such as a gamepad, mouse, etc., from a host microprocessor. A desired haptic effect to be output by the vibrotactile interface device to a user is determined, and effect information is provided to the vibrotactile device, the information describing a magnitude and a frequency that are independent of each other. A periodic control signal based on the effect information drives a rotary motor of the vibrotactile device to cause a mass to rotate, where the rotation of the mass causes a vibration having the magnitude and frequency to be output to the user. The magnitude of the vibration is based on a duty cycle of the control signal, and the frequency of the vibration is based on a frequency of the control signal. In one embodiment, the control signal can be either on or off, where the control signal is turned on for an on-time for each period of the vibration, which determines the magnitude of the vibration. The provided information can include a higher level command and at least one parameter that is parsed by the device, or the information can be substantially equivalent to the control signal. The control signal may drive the motor in only one direction.
In another aspect of the present invention, a method for providing a vibration for a haptic feedback device coupled to a host microprocessor includes providing an actuator for the haptic feedback device, the actuator including a rotatable mass, and receiving information at the haptic feedback device which causes a control signal to be produced. The control signal controls the actuator to rotate the mass about an axis of rotation such that the mass rotation induces a vibration in the device, where a magnitude and a frequency of the vibration can be adjusted independently of each other by adjusting the control signal. A magnitude of the vibration is based on a duty cycle of the control signal and a frequency of the vibration is based on a frequency of the control signal. The control signal can be applied at a predetermined point in time of every period of the vibration. An on-time of the control signal can be determined as a percentage of a period of the vibration, or as a predetermined amount of time for each period. Methods can be used to ensure the mass is rotating, such as sending an initial control signal to said actuator to start the mass rotating before the vibration is output. A haptic feedback device including similar features is also disclosed. Some embodiments can cause the mass to move bidirectionally.
In another aspect of the present invention, a method for allowing kinesthetic haptic effects to be output on a vibrotactile interface device includes receiving a command describing a kinesthetic haptic effect that is intended to cause forces to be output in at least one degree of freedom of a manipulandum of a kinesthetic haptic feedback device that is manipulated by a user. The kinesthetic haptic effect is mapped to a vibrotactile haptic effect that is intended to cause vibrotactile forces to be output to a user contacting the vibrotactile device. The vibrotactile haptic effect is provided to be output by the vibrotactile interface device. The kinesthetic haptic effect can be a periodic effect having a specified magnitude and frequency, where the mapped vibrotactile haptic effect is a vibration having an equivalent magnitude and frequency. The kinesthetic haptic effect can also be a nonperiodic effect, such as a spring, damper, obstruction, or vector force, and the mapped vibrotactile haptic effect can be a vibration having a magnitude based on a magnitude of the nonperiodic effect.
The present invention advantageously provides methods for outputting more sophisticated force effects on a vibrotactile interface device. A vibrotactile feedback device of the present invention can independently control and specify the magnitude and frequency of vibrations output by a eccentric rotating mass motor using duty cycle and frequency of a control signal and can use low-cost uni-directional amplifiers and control circuitry. Techniques for mapping kinesthetic force effects to vibrotactile effects allow software designed to control kinesthetic force feedback devices to control haptic effects for vibrotactile devices. Less-expensive vibrotactile devices thus are a viable alternative to a purchaser to experience compelling haptic feedback in a variety of existing computer applications.
These and other advantages of the present invention will become apparent to those skilled in the art upon a reading of the following specification of the invention and a study of the several figures of the drawing.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGURES 1 and 2 are perspective views of gamepad vibrotactile interface devices which can be used with the present invention;
FIGURE 3 is a top view of an interior of the gamepad device of Fig. 1;
FIGURE 4 is a block diagram of a vibrotactile interface device suitable for use with the present invention;
FIGURE 5 is a graph illustrating a control signal of the present invention and resulting output vibration;
FIGURE 6 is a graph illustrating a control signal of the present invention modified from Fig. 5 and resulting output vibration;
FIGURES 7, 8 and 9 are graphs illustrating control signals of different frequencies and the resulting vibrations; and
FIGURES 10a and 10b are force profiles illustrating a kinesthetic spring force and a vibrotactile sensation mapped to the kinesthetic spring force, respectively. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
This application describes techniques for controlling a vibrotactile interface controller, such as a haptic feedback gamepad, using more sophisticated force effects, as well as techniques for enhancing the vibrotactile feedback output by such vibrotactile controllers. Herein, the term
"vibrotactile device" or "vibrotactile feedback device" is intended to refer to any controller or interface device that outputs vibrations to the user of the device, and can include gamepads, handheld steering wheels, fishing-type controllers, joysticks, mice, trackballs, adult devices, grips, remote controls, handheld game devices, flat screens, styluses, etc. In contrast, the term "kinesthetic device" or similar term is intended to refer to devices that provide forces along the axes or degrees of freedom of motion of a manipulandum of the device.
FIGURE 4 is a block diagram illustrating a haptic system 100 suitable for use with the present invention. System 100 includes a host computer 102 and a vibrotactile device 104.
Host computer 102 is any of a variety of computing or electronic devices. In one preferred embodiment, computer 102 is a personal computer, game console, or workstation, such as a PC compatible computer or Macintosh personal computer, or game console system from Nintendo
Corp., Sega Corp., Sony Corp., or Microsoft Corp. In other embodiments, host computer 102 can be a "set top box" which can be used, for example, to provide interactive television functions to users, or a "network-" or "internet-computer" which allows users to interact with a local or global network using standard connections and protocols such as used for the Internet and World Wide
Web. Some embodiments may provide a host computer 102 within the same casing or housing as the interface device or manipulandum that is held or contacted by the user, e.g. hand-held video game units, portable computers, arcade game machines, etc. Host computer preferably includes a host microprocessor, random access memory (RAM), read only memory (ROM), input/output (I/O) circuitry, an audio output device, and other components of computers well-known to those skilled in the art. Other types of peripherals can also be coupled to host computer 102, such as storage devices (hard disk drive, CD ROM drive, floppy disk drive, etc.), printers, and other input and output devices.
A display device 106 is preferably connected or part of the computer 102 and displays images of a graphical environment, such as a game environment, operating system application, simulation, etc. Display device 106 can be any of a variety of types of devices, such as LCD displays, LED displays, CRT's, flat panel screens, display goggles, etc.
Host computer 102 preferably implements a host application program with which a user is interacting via the interface device 104 and other peripherals, if appropriate. For example, the host application program can be a video game, word processor or spreadsheet, Web page or browser that implements HTML or VRML instructions, scientific analysis program, virtual reality training program or application, or other application program that utilizes input of device 104 and outputs haptic feedback commands to the device 104 (or a different layer can output such commands, such as an API or driver program on the host). The host program checks for input signals received from the electronics and sensors of device 104, and outputs force values and/or commands to be converted into forces output for device 104. Suitable software drivers which interface such simulation software with computer input/output (I/O) devices are available from Immersion Corporation of San Jose, California.
Several different layers of programs can be running on the host computer 102. For example, at an application layer, one or more application programs can be running, such as a game program, word processing program, etc. Several sub-layers can also be provided, such as an
Application Programming Interface (API) layer (e.g. used in Windows from Microsoft Corp.), and different driver layers. The application program can command forces directly, or a driver program can monitor interactions within an application program and command haptic effects when predetermined conditions are met. In one embodiment, a haptic feedback driver program can receive kinesthetic haptic commands from an application program and can map the commands to vibrotactile commands and effects, and then send the necessary information to the interface device 104.
Vibrotactile interface device 104 is coupled to host computer 102 by a bi-directional bus 108. The bi-directional bus sends signals in either direction between host computer 102 and the interface device. For example, bus 108 can be a serial interface bus, such as an RS232 serial interface, RS-422, Universal Serial Bus (USB), MIDI, or other protocols well known to those skilled in the art; or a parallel bus or wireless link. For example, the USB standard provides a relatively high speed interface that can also provide power to actuators of device 104.
Vibrotactile device 104 can, in many embodiments, include a local microprocessor 110. Local microprocessor 110 can optionally be included within the housing of device 104 to allow efficient communication with other components of the device. Processor 110 is considered local to device 104, where "local" herein refers to processor 110 being a separate microprocessor from any processors in host computer 102. "Local" also preferably refers to processor 110 being dedicated to haptic feedback and sensor I/O of device 104. Microprocessor 110 can be provided with software instructions to wait for commands or requests from host 102, decode or parse the command or request, and handle/control input and output signals according to the command or request. In some embodiments, processor 110 can operate independently of host computer 102 by reading sensor signals and calculating appropriate forces from those sensor signals, time signals, and stored or relayed instructions selected in accordance with a high level host command. Suitable microprocessors for use as local microprocessor 110 include the MC68HC711E9 by
Motorola, the PIC16C74 by Microchip, and the 82930AX by Intel Corp., for example, as well as more sophisticated force feedback processors such as the Immersion Touchsense™ Processor, current versions of which are used with personal computers such as PC's. Microprocessor 110 can include one microprocessor chip, multiple processors and/or co-processor chips, and/or digital signal processor (DSP) capability. For example, the control techniques described for the present invention can be implemented within firmware of an Immersion TouchSense Processor, where the local microprocessor block 110 includes related components, such as encoder processing circuitry, communication circuitry, and PWM circuitry as well as a microprocessor. Various techniques for playing more sophisticated periodics and other sensations (such as defined by Immersion protocols) with a eccentric rotating mass (ERM) motor can be provided in the firmware of the microprocessor 110.
Microprocessor 110 can receive signals from sensors 112 and provide signals to actuators 120 and 122 in accordance with instructions provided by host computer 102 over bus 108. The microprocessor 110 provides a signal 111 which is defined as a "Control Signal" herein. In one embodiment, the Control Signal is a PWM signal the firmware of processor 110 can generate and send to the amplifier in actuator interface 124. There is preferably one Control Signal per motor, such that two control signals are provided in the described embodiment. Another term used herein is "on-time," which is the amount of time the Control Signal remains high once it goes high. The
Control Signal provides the drive signal for the actuators, as described in greater detail below.
In one local control embodiment, host computer 102 provides high level supervisory commands to microprocessor 110 over bus 108, and microprocessor 110 decodes the commands and manages low level force control loops to sensors and the actuator in accordance with the high level commands and independently of the host computer 102. This operation is described in greater detail in US Patents 5,739,811 and 5,734,373, both incoφorated by reference herein. In the host control loop, force commands are output from the host computer to microprocessor 110 and instruct the microprocessor to output a force or force sensation having specified characteristics. The local microprocessor 110 reports data to the host computer, such as locative data that describes the position of the mouse in one or more provided degrees of freedom. The data can also describe the states of buttons or other devices of device 104. The host computer uses the data to update executed programs. In the local control loop, actuator signals are provided from the microprocessor 110 to the actuators and sensor signals are provided from the sensors 112 and other input devices to the microprocessor 110. The microprocessor 110 can process inputted sensor signals to determine appropriate output actuator control signals by following stored instructions. The microprocessor may use sensor signals in the local determination of forces to be output, as well as reporting locative data derived from the sensor signals to the host computer.
In yet other embodiments, other hardware can be provided locally to device 104 to provide functionality similar to microprocessor 110. For example, a hardware state machine incorporating fixed logic can be used to provide signals to the actuators and receive sensor signals from sensors
112, and to output tactile signals according to a predefined sequence, algorithm, or process.
Techniques for implementing logic with desired functions in hardware are well known to those skilled in the art. Such hardware can be well suited to less complex force feedback devices, such as the device of the present invention.
In a different, host-controlled embodiment, host computer 102 can provide low-level force commands over bus 108, which are directly transmitted to the actuator via microprocessor 110 or other circuitry (if no microprocessor 110 is present). Host computer 102 thus directly controls and processes all signals to and from the device 104, e.g. the host computer directly controls the forces output by actuator 120 or 122 and directly receives sensor signals from sensor 112 and input devices 126. This embodiment may be desirable to reduce the cost of the force feedback device yet further, since no complex local microprocessor 110 or other processing circuitry need be included in the device 104. The host 102 can also stream force values that are sent to the actuators, as described in U.S. Patent No. 5,959,613, incoφorated herein by reference.
In the simplest host control embodiment, the signal from the host to the device can be a single bit that indicates whether to pulse the actuator at a predefined frequency and magnitude. In more complex embodiments, the signal from the host can include a magnitude, giving the strength of the desired pulse, and/or a frequency. A local processor can also be used to receive a simple command from the host that indicates a desired force value to apply over time, so that the microprocessor outputs the force value for the specified time period based on the one command. The host command may also specify an "envelope" to provide on a force output to allow force shaping, as disclosed in U.S. Patent No. 5,959,613. Furthermore, force values (magnitudes, etc.) can be streamed from the host computer to be output in real time by the device, as described in
U.S. Patent No. 5,959,613. A combination of numerous methods described above can also be used for a single device 104.
Local memory, such as RAM and/or ROM, can be coupled to microprocessor 110 in device 104 to store instructions for microprocessor 110 and store temporary and other data. For example, force profiles can be stored in memory, such as a sequence of stored force values that can be output by the microprocessor, or a look-up table of force values to be output based on current events in the application program or other conditions. In addition, a local clock can be coupled to the microprocessor 110 to provide timing data, which might be required, for example, to compute forces output by actuator (e.g., forces dependent on time dependent factors). In embodiments using the USB communication interface, timing data for microprocessor 110 can be alternatively retrieved from the USB signal.
Sensors 112 sense the position or motion of a manipulandum 114 of the device 104 and provides signals to microprocessor 110 (or host 102) including information representative of the position or motion. In some embodiments, the manipulandum is a small joystick provided on a gamepad controller and moved by a user in two rotary or linear degrees of freedom to provide control input to the host computer. In other embodiments, the manipulandum can be a direction pad having four or more directions which can provide input to the host computer. The manipulandum can also be a rotary dial, linear slider, wheel, finger receptacle, cylinder, or other controlling member. The manipulandum can also be the housing of the device itself, as in the case of a mouse or if sensing the position of a gamepad or other controller in 3-D space. Sensors suitable for detecting motion of a joystick or other manipulandum include digital optical encoders frictionally coupled to a rotating ball or cylinder, as is well known to those skilled in the art.
Mechanical switches, linear optical encoders, potentiometers, optical sensors, velocity sensors, acceleration sensors, strain gauges, or other types of sensors can also be used, and either relative or absolute sensors can be provided. Optional sensor interface 116 can be used to convert sensor signals to signals that can be inteφreted by the microprocessor 110 and/or host computer 102, as is well known to those skilled in the art.
Actuators 120 and 122 transmit forces to the user of the device 104 in response to signals or commands received from microprocessor 110 and/or host computer 102. In one described embodiment, two actuators are provided, a small actuator 120 and a larger actuator 122. In the preferred embodiment, these actuators are eccentric rotating mass (ERM) DC motors, which are rotary motors having an eccentric mass coupled to the rotating shaft of the motor. When rotated, the inertial forces from the rotating mass cause an oscillation or vibration in the housing or other member coupled to the motor housing, thus producing vibrotactile sensations to the user who is holding or otherwise contacting the housing. Actuators 120 and 122 thus provide "informative" or "effect" vibrotactile forces. The small motor 120 can provide high frequency, low magnitude forces, while the large motor 122 can provide lower frequency, higher magnitude forces.
Alternate embodiments can employ a single actuator, or two or more actuators of the same or differing sizes for providing vibrotactile sensations or forces to the user of the device 104. Many different types of actuators can be used, e.g. any type of actuator which can rotate an eccentric mass in a direction, such as voice coil actuators, moving magnet actuators, hydraulic or pneumatic actuators, torquers, brushed or brushless motors, etc. Furthermore, additional actuators can be included to provide kinesthetic force feedback in the manipulandum 114.
Actuator interface 124 can be optionally connected between actuators 120 and 122 and microprocessor 110 to convert signals from microprocessor 110 into signals appropriate to drive the actuators. Interface 124 can include power amplifiers, switches, digital to analog controllers (DACs), analog to digital controllers (ADCs), and other components, as is well known to those skilled in the art. For example, in one embodiment the actuators 120 and 122 are off-the-shelf ERM motors which are driven unidirectionally. Uni-directional voltage mode amplifiers are low cost components that can be used in actuator interface 124 to drive the motors.
Other input devices 118 are included in device 104 and send input signals to microprocessor 110 or to host 102 when manipulated by the user. Such input devices can include buttons, dials, switches, scroll wheels, or other controls or mechanisms. Power supply 120 can optionally be included in or coupled to device 104, and can be coupled to actuator interface 124 and/or actuators 120 and 122 to provide electrical power to the actuators. Alternatively, and more preferably, power can be drawn from the bus 108, e.g. using a USB or other bus. Also, received power can be stored and regulated by device 104 and thus used when needed to drive actuators 120 and 122 or used in a supplementary fashion. Because of the limited power supply capabilities of USB, a power storage device may be required in the mouse device to ensure that peak forces can be applied (as described in U.S. Patent No. 5,929,607, incoφorated herein by reference). For example, power can be stored over time in a capacitor or battery and then immediately dissipated to output a haptic sensation. Alternatively, this technology can be employed in a wireless device 104, in which case battery power can be used to drive the tactile actuator.
For many of the various control features of the present invention described below, an existing console gamepad interface device, such as those shown in Figs. 1-3, can be used as interface device 104. In other embodiments, an existing gamepad controller can be provided with a more sophisticated local microprocessor to allow more sophisticated haptic sensations to be controlled and output by the interface device. For example, a Sony Dual Shock-compatible product for the PlayStation console can be modified to use an Immersion Touchsense Processor- compatible microprocessor and retain the remaining existing motors and electronics. However, there are no components that the vibrotactile controller of the present invention need retain from a standard prior art console implementation; different components can be used as necessary or to provide a better haptic output. For example, the Dual Shock controller uses a special PlayStation bus and proprietary protocol, while a PC version can use Universal Serial Bus (USB) or a Serial protocol and be DirectX/Immersion compatible. DirectX from Microsoft Coφoration prescribes a set of conditions and haptic effects for the Windows operating system provided on a PC. In other implementations, such as for Macintosh or other computers or systems, other standard protocols and interfaces for those platforms can be used. In yet other embodiments, a variety of other types of vibrotactile interface devices which output vibrations to the user can be used with the vibrotactile control features described herein, such as gamepads, handheld or surface-grounded steering wheels, fishing-type controllers, joysticks, mice, trackballs, adult devices, grips, remote controls, handheld game devices, flat screens, styluses, etc.
Drive Methods
The ERM motors described above are low cost and thus desirable for use in mass market haptic feedback devices. The ERM motors, however, can be driven in different ways to provide haptic sensations, as described below. In some embodiments, two or more of these drive methods can be employed in a single device, depending on the type of haptic sensations desired.
The preferred drive method is to drive the motors uni-directionally, since uni-directional amplifiers and other components are lower in cost than more sophisticated components. However, in a different bi-directional embodiment, a harmonic drive (bi-directional, like a speaker) may be applied to an ERM motor to produce exact frequencies with independently varying magnitude using a forcing function control signal. In the bi-directional embodiment, the eccentric mass of the motor is driven in pendulous motion in two directions and need not ever make a complete, full rotation. Such a drive has a substantially increased power demand, typically provides sensations having less magnitude, and usually requires more expensive driver circuitry than uni-directional embodiments, but may provide more precisely-controlled haptic sensations.
However, as described herein, there are other ways to provide haptic sensations similar to those produced by a harmonic drive but without the added cost and power demand. The first method is to use a uni-directional amplifier to drive the ERM in circular motion, and to pulse the mass instead of changing its direction in order to play the frequency content of a vibration. The second method is to use a uni-directional amplifier to drive the ERM in pendulous (bi-directional) motion, but relying on gravity to move the mass in the opposite direction and provide the return torque instead of the amplifier, since the motor is only driven in one direction (although the motor can be slowed by commanding a voltage less than the back EMF of the motor, as is well known to those skilled in the art). Both of these drive schemes are utilized by the control method described below, where pendulous motion is exhibited when the commanded magnitude is sufficiently low to be unable to drive a full mass revolution. For example, the parameters that are passed to the local microprocessor 110 (or other controller) can determine whether the ERM spins or follows a pendulous motion.
A third drive method employs an interfering member, such as a stop and/or a spring, to ensure oscillatory motion of the rotating mass. A stop can be placed in the path of the ERM which prevents the ERM from making a complete rotation, and which can transmit vibrations to the housing of the device from the impact of the ERM with the stop. A spring material can also or alternatively be used, where the spring can be placed between the stop and the ERM. When the
ERM hits the spring (such as foam, rubber, coiled spring, flexure, or other resilient or compliant material), the spring's resilience causes the ERM to move back with greater force than without the spring. The actuator force can be dynamically adjusted to provide desired vibration magnitude and simultaneously overcome spring force. The spring adds energy back into the system once the actuator force is removed. The result is that a low-cost uni-directional amplifier is able to actively spin the mass in both directions.
Mapping Vibrotactile Effects to Kinesthetic Force Effects
A central issue addressed by the present invention is how to translate existing kinesthetic haptic effects to vibrotactile haptic effects. Many existing types of kinesthetic effects exist in current devices. For example, Immersion Studio™ from Immersion Coφ. is a software tool that allows a developer to design a wide variety of haptic sensations, many of which are kinesthetic effects such as spring forces, damping forces, and obstruction forces. A problem exists when a vibrotactile device is used with an application program or other software that was meant to be used for a kinesthetic haptic device such as a steering wheel, force feedback joystick (such as the
Wingman Force Feedback Joystick from Logitech, Inc.), or kinesthetic force feedback mouse (such as the Wingman Force Feedback Mouse from Logitech, Inc.). Many types of kinesthetic haptic effects, such as springs, dampers, and obstructions, may have no meaning in a vibrotactile device if output without modification, since most vibrotactile devices do not output forces to resist or assist motion of a manipulandum of the device.
The present invention allows kinesthetic effects to be output by vibrotactile devices by providing mappings, where a particular vibrotactile effect is designated to be the "equivalent" of a particular kinesthetic force effect. Some kinesthetic effects can be mapped almost directly to vibrotactile effects, such as vibrations. Other kinesthetic effects must be approximated in ways greatly differing from the original effect due to the inability of the vibrotactile device to output directional forces on the motion of a manipulandum. For example, a particular collision effect in DirectX can be implemented as a vector force provided in a particular direction of movement of the user manipulatable object. A particular vibrotactile device, however, may not be able to output forces in a particular direction, so that the vector force must be approximated with a different vibrotactile sensation.
Mappings between vibrotactile effects and kinesthetic effects can be made in general terms or more specific terms. For example, a particular vibrotactile effect can be designated to be a replacement for vector kinesthetic effects in all applications and circumstances in which that kinesthetic effect would have been output on a kinesthetic device. Alternatively, different vibrotactile effects can be mapped to a particular kinesthetic effect based on the particular context or application in which the kinesthetic effect is used. For example, a first vibrotactile effect might be used to approximate a kinesthetic vector force in a racing game, while a second, different vibrotactile effect can be used to approximate that kinesthetic vector force in a fighting game or a flight simulator game. In many cases, such a scheme involving specific applications can provide more realistic results since the vibrotactile effects are tailored for the most realism in a particular application.
Kinesthetic force effects can be organized into periodic and non-periodic effects. Periodic effects such as vibrations are discussed first since these effects can be most easily translated from kinesthetic to vibrotactile domains, while other, non-periodic effect mappings are described at a later point.
In one embodiment, when an application program is run on the host computer, the program detects what type of controller device is being used with the application program, and selects an appropriate set of effects for that device. If a vibrotactile device is used, the mappings for vibrotactile effects are used; if a kinesthetic device is used, then standard kinesthetic effects can be output. Alternatively, the application (e.g. game) program can be completely ignorant of the type of device connected (or can know the type of device but can be ignorant of mappings described herein), so that the mappings are handled by a lower-level driver running on the host computer and/or by the local microprocessor/firmware on the device. For example, a vibrotactile-specific driver running on the host can evaluate outgoing commands and map them to vibrotactile commands, if appropriate, so that the device need perform no mapping tasks. Alternatively (or in addition), the local microprocessor can include a look-up table or other data to map certain kinesthetic force effects to vibrotactile effects and control the motors appropriately.
One or more mappings of vibrotactile forces to kinesthetic forces can (in any embodiment) also be selectively applied based on the application or circumstances in which the force effect is commanded. For example, if a kinesthetic vector force has been output for a collision occurring in a game, the equivalent vibration forces may be appropriate. But if the vector force is being used to simulate a dynamic spring return on the steering wheel during a turn, then the vibration force may not be appropriate and is not commanded to be output.
To determine the application or circumstances for which a particular kinesthetic force is being used, different methods can be employed. For example, in one embodiment, the command protocol can include one or more parameters indicating the puφose for the command, such as "collision," "turn," or "spin out." A specialized vibrotactile driver on the host (or the local microprocessor) can examine the puφose parameter to determine whether to output a force based on stored instructions governing such output. However, since such an embodiment would require a change to existing protocols, other methods are preferred.
In one embodiment, for example, a set of force effects which are intended to be used with vibrotactile devices can be available to developers when programming haptic effects and available to be installed by users on a computer so that mappings are available to a driver. Each vibrotactile set of haptic effects can be intended for a particular broad category of applications, such as driving games, flight simulation, fighting games, a word processor, Internet browser, DVD interface, etc.
Alternatively, a set of mappings can be specific to one particular application program, such as a particular game. When a user wishes to include haptic feedback effects for one of these applications, the user obtains the set of effects for the application and loads the set to the operating system or otherwise makes the set available to the vibrotactile driver (or local microprocessor).
Or, the application program itself can perform the mappings, and thus the set would be available to the application program. Thus, a set of effects for all automobile driving games (or a particular driving game) would map vibrotactile effects to particular kinesthetic forces which are appropriate and effective for driving circumstances in these games. These sets of pre-mapped vibrotactile effects can be available from a central provider or developer, e.g. downloaded from a website or provided with an application program. Furthermore, a set of mappings may also include settings for overall magnitudes provided in a "control panel." For example, a control panel driver can be available on the host computer to allow the user to adjust the overall magnitudes of particular force sensations (or an overall gain of all force sensations) to a desired level. A set of mappings for vibrotactile effects can also include settings for these control panel force sensations to maximize the effectiveness of the vibrotactile effects for that type of application program. For example, the mappings for a driving game might also set spring forces to a lower overall magnitude and increase vibrations to a greater overall magnitude.
Some embodiments can allow the device (or a haptic driver) to ignore all kinesthetic effects that do not directly translate to the vibrotactile domain when a vibrotactile device is being used. Thus, all spring, damper, attraction, and obstruction forces can be ignored, allowing mainly jolts and vibrations to be only output. However, since some types of kinesthetic forces, such as some obstructions, can be effectively simulated or approximated using vibrations, the types of kinesthetic forces that are ignored can vary depending on the embodiment or application.
Embodiment Constraints
In providing control techniques to best portray DirectX/Immersion kinesthetic effects in a vibrotactile gamepad device, some particular considerations can make the vibrotactile device simple with a low production cost, for those embodiments with such cost constraints. For example, a stock eccentric mass motor can be used to take advantage of existing supplies of these motors. The motor drivers should be assumed to operate uni-directionally, since that is the most common type of motor operation in current vibrotactile devices. In addition, uni-directional voltage-mode amplifiers are significantly cheaper than sophisticated current-mode amplifiers used in kinesthetic devices, and they can be built with a single transistor. Since torque control is not crucial to most embodiments, a current-mode amplifier need not be used. Furthermore, sensors for detecting the position of the rotating mass are expensive and can be omitted for cost reasons, as in prior art gamepad vibrotactile controllers. Such a sensor can be added in some embodiments if necessary. The product can use the Immersion Touchsense Processor from Immersion Coφ. or similar microprocessor. No power limitations are preferably imposed on actuator output, and power from the communication bus is preferably sufficient for device operation (e.g. using a USB bus).
It is preferable in many embodiments of the interface device that the device have minimal power requirements so as to avoid the use of bulky power supplies and/or to preserve the life of limited power sources such as batteries. For example, where appropriate, gamepad vibrotactile devices can meet the Universal Serial Bus (USB) power specification of maximum 0.5 A at 5 volts, thereby allowing all power to be received over the USB. Other measures can be taken as well. For example, in devices having two motors, each motor's power requirements can be quantified. In some embodiments, it may be necessary to remove one of the two motors from the system entirely to meet power requirements. If the motors are of different sizes, then the small motor can be removed and the large motor can be used to reach about 35 Hz with independent magnitude and frequency control. Beyond 35 Hz, frequency changes may become imperceptible.
If the large motor is removed, the small motor can provide fuller bandwidth but may not be able to meet magnitude requirements within that bandwidth.
If enough power is available for two or more motors in such a device, there are additional steps that can be taken to reduce peak power. In the described methods above, both motors can be turned on simultaneously when each period begins. However, if one motor's phase is offset so that the it does not get pulsed until the other motor's on-time has completed, then the two motors are prevented from operating simultaneously, and peak power may be reduced. It should be noted that the phases of the two motors should not be offset by 180 degrees, or the resulting vibration may feel to the user as if it is double the intended frequency. Preferably, one motor is offset by the maximum "on-time per period" of the other motor (see below). Later, a dynamic algorithm (e.g., running on the local microprocessor) can monitor power and only turn both motors on if the first motor allows enough power headroom. A final scheme to reduce power reduces the magnitude of the control signal in favor of increasing the on-time of the control signal. This should deliver the same amount of energy to the motor, though this technique may undesirably lessen the magnitude and impact of the vibrations.
Controlling Periodic Effects
Periodic effects are those force feedback effects which are based on time and controlled with a periodic waveform. For example, a vibration is a basic periodic effect in which a waveform type, such as a sine wave or square wave, is commanded to output the vibration at a particular frequency (period) and magnitude. The effect does not require other data such as velocity or position of a user manipulandum or a direction of output. Single, non-directional jolts can be output as a single period of a vibration or part of a period of a vibration.
When considering periodic effects, the problem becomes how frequency and magnitude of a periodic effect can be varied independently and displayed on a single degree of freedom actuator, such as (but not restricted to) an ERM motor.
Many standard gamepad vibrotactile devices rotate ERMs at a fixed magnitude and frequency, the two of which are tightly coupled. For example, high frequency vibrations are necessarily high magnitude, while low frequency vibrations are necessarily low frequency. The present inventive method of control allows independent variation of magnitude and frequency of a 1 degree-of-freedom (DOF), unidirectionally-driven rotary actuator. The technique is significant because it enables an ERM to create complex vibrations, like decaying sinusoids or superimposed waveforms. An ERM in prior devices had a magnitude that was roughly linearly coupled to its speed. The present invention, utilizing no extra clutches, circuitry, or mechanical parts and using only these control methods (e.g. implemented in firmware of microprocessor 110 or other controller), can play a range of frequencies at any amplitude using an ERM motor. Note that the control methods described herein can be applied not only to rotational motors but other types of 1 DOF actuators, rotational and linear, including moving magnet motors, solenoids, voice coil actuators, etc.
In this inventive method of control, a frequency command, a magnitude command, and a function (i.e. sine wave, square wave, triangle wave) are supplied as parameters or inputs to the firmware. This follows the existing Immersion/DirectX protocol used in PC's, in which a vibration is controlled with magnitude, frequency, and function type parameters (and additional parameters, if desired). Therefore, a direct mapping between kinesthetic and vibrotactile devices is possible with the present invention, i.e., no mapping may be needed, where the existing periodic effect can be commanded directly to the kinesthetic or vibrotactile device, unchanged.
An example is illustrated in the graph 150 shown in FIGURE 5, showing a time vs. magnitude relationship. A sine wave 152 of frequency 5 Hz and 50% magnitude is shown as the desired vibration to be output by the device (this, and all subsequent similar figures, capture 1 second of input and output signals). The present control method determines where each period of the waveform begins (or should begin), then raises a control signal 154 to a high or "on" level for a specific duration once per period; the control signal 154 is "off or low during the other times.
Thus, the periodic control signal has a frequency based on the desired (commanded) frequency. By pulsing the actuator once per period using control signal 154, the perception of a vibration with specified frequency is conveyed to the user. The control signal can be raised to a high level at the beginning of a period, as shown, or at a different time within the period.
Magnitude of the periodic effect is portrayed by adjusting a duty cycle of the control signal, e.g. the duration at each period ("on-time per period") of the control signal 154. The control signal 154 is either on or off, but the amount of time per period for which the control signal remains on is determined by the magnitude command or parameter. In Fig. 5, a sine wave with 50% of the possible magnitude is requested. According to the present invention, this requested magnitude generates a control signal 154 that comes on every 250ms for 15ms duration. For comparison, a 100% magnitude waveform with the same frequency is offered in FIGURE 6, which shows a similar graph 160 to graph 154. The control signal 164 comes on at the same interval as the control signal 154 described above, since the frequency command has not changed. However, the control signal 164 stays on twice as long to produce the feeling in the user of twice the magnitude of vibration. The longer that the control signal is on, the longer the actuator spends accelerating. In our case, the ERM reaches a larger angular velocity and, since force is proportional to the square of angular velocity, larger forces are perceived at the user's hand. Preferably, the mass is never allowed to stop rotating, so that static friction need be overcome only once, at the start of rotation. If the control signal remains on too long, the rotating mass will make multiple revolutions and eventually reach its natural (resonant) frequency. At that point, the user will perceive the natural frequency of the system and not the commanded frequency.
Thus, according to the present invention, 1) how often the control signal comes on depends directly on frequency command, and 2) how long the control signal remains on (control signal on- time) is related to the magnitude command. The determination of the on-time of the control signal can be accomplished in different ways. Two different ways are presented here. First, on-time can be regulated as a "percentage of period." If the control signal comes on for a fixed percentage of each period, as frequency increases, on-time per period decreases. Yet the control signal comes on more often. The end result is that the control signal spends the same amount of time on each second, no matter what the frequency. This technique offers the advantage that, as frequency increases, constant power is added to the actuator, and perceived magnitude stays the same over the frequency range. It should be noted that, if on-time is decreased sufficiently, the mass may not actually rotate in one direction, but may move bi-directionally due to the small driving time and gravity that causes the mass to move back in the opposite direction once the control signal is turned off. However, this does not adversely affect the output vibration sensed by the user, which is perceived at the desired frequency and magnitude regardless of whether the mass completes full rotations in one direction or is moving bi-directionally.
FIGURES 7 and 8 illustrate graphs 170 and 180, respectively, showing different control signals and the resulting sine wave for vibration. In Fig. 7, the frequency of the sine wave 172 is 5 Hz and the magnitude is 100%. In Fig. 8, the frequency of the sine wave 182 is 10 Hz and the magnitude is 100%. The control signal 174 of Fig. 7 is on for about 40 ms at each period, and the control signal 184 of Fig. 8 is on for about 20 ms at each period. Comparing the control signals of Figs. 7 and 8, it is apparent that the total on-time per second remains constant between the two graphs while the frequency varies: as the frequency command doubles, the figures show how the on-time per period of the control signal is halved yet the on-time per second remains the same.
A problem with this "percentage of period" technique for commanding a desired vibration is that in many embodiments it may not work well at lower frequencies. At low frequencies (e.g., less than 2 Hz for the large ERM motor in the Sony Dual Shock gamepad), too much power is delivered to the actuator at once. For example, FIGURE 9 is a graph 190 showing 1 second of a sine waveform 192 that produces a vibration of low frequency (1 Hz) and 100%o magnitude; only one period is provided for the 1 second duration. The control signal 194 is set high at the beginning of the period, for 125 ms. Both graphs 180 of Fig. 8 and 190 of Fig. 9 set the control signal high for the same total amount of time per second. However, in Fig. 9 all the power from a 1 second period is delivered in a continuous 125ms at the beginning of the period. During this on- time, the rotating actuators make several revolutions while the control signal is held high, so that vibrations (pulses) output during this 125ms are perceived by the user at the frequency of the actuator's rotation speed, not the commanded frequency. The vibration output by the device thus may not correspond with the commanded (low) frequency. This problem was not evident in the vibration produced as shown in Fig. 8, where the on-time of the control signal 184 is divided among multiple periods.
A second method of the present invention can avoid this problem at low frequencies and thus may provide a more suitable way to output vibrations for many vibrotactile devices. The second method sets the control signal high for a fixed maximum amount of time per period, not a percentage of the period. Thus, the on-time for 100% magnitude for any frequency is the same. The on-time for commanded magnitudes less than 100% are lower in proportion to the amount of the commanded magnitude under 100%. This effectively establishes a maximum on-time per period, prohibiting the actuator from coming on long enough to make multiple revolutions during one continuous on-time. If the actuator is allowed to make multiple revolutions (e.g., more than about 2 or 3 in some embodiments), the user will perceive a higher frequency (e.g., about 25 Hz for some motors) based on the rotation speed of the actuator rather than the commanded frequency (e.g., which might be less than 10 Hz), so this method prevents that result. In some embodiments, a request of 100% magnitude at a lower frequency for a particular motor can be equated with the on-time that causes the mass to rotate just under the number of revolutions that cause the user to feel more than one pulse for a single period (such as 2-3 revolutions); this on-time can be determined empirically. A drawback to the second technique is that as frequency increases, the separate on-times get closer together, and the actuator is eventually, in effect, requested to remain on for longer than one period. At that point, the control signal is always being asserted, the mass rotates continuously, and frequency and magnitude no longer vary independently.
Since the two techniques for mapping magnitude to on-time of the control signal are good for different portions of the frequency range, one preferred embodiment of the present invention combines or blends the two techniques to avoid the drawbacks in each method. In the combination method, the second method is used only when commanded frequencies are below a particular blend threshold frequency and the first method can be used for commanded frequencies above that threshold frequency. Blending is possible even if the magnitude of the control signal also varies. First, the blend threshold is chosen based on dynamics of the system; the blend frequency is the frequency at which the on-time will be the longest, so a blend frequency should be chosen that will provide one vibration pulse (e.g. less than two mass revolutions) per period for an on-time corresponding to 100% magnitude at that frequency. For example, when using the large motor/mass combination as described above, 10 Hz can be used as a blend threshold frequency. For commanded frequencies above 10 Hz, the first method ("percentage of period") is used to calculate the on-time of the control signal, and for commanded frequencies below 10 Hz, the second method ("fixed time per period") can be used. Other thresholds can be used in other embodiments. To blend the two methods, scalars are chosen so that maximum magnitude for the two methods matches at the blend threshold frequency, i.e. the transition between methods is smooth. For example, a 25 ms control signal on-time at 10 Hz may generate a 10 Hz, 100%) magnitude vibration. If the commanded frequency is approaching the blend frequency from below 10 Hz, then the "percentage of period" method is scaled to generate 25 ms on-times at 10 Hz, and those scalars used are retained and applied to this method for frequencies above 10 Hz. Depending on the desired effect, more advanced blending techniques can be used such as those that mimic band pass filters in the region of the blend, or low pass/high pass combinations on either side of the blend threshold frequency.
A third method to allow the command of magnitude independent of frequency is to vary the amplitude of the control signal proportionally to requested magnitude, rather than having only two levels for the control signal. This can be performed alone or in conjunction with either or both of the first and second methods described above. For example, other types of waveforms having varying amplitude might be used as control signals (sine wave, triangle wave, etc.). One efficient way to set or vary the amplitude of the control signal is to provide pulse-width modulation (PWM) during the chosen on-time for control signals as presented above, or to vary the control signal duty cycle during the on-time using some other method. However, PWM may require a separate PWM module, which can add cost to the device. To avoid a PWM scheme, the first and second methods described above can be implemented by bit-banging, in which the local microprocessor outputs the control signal directly to the actuators without using a PWM module. Such direct control is often not feasible when using more sophisticated force feedback, but can be viable for vibrotactile feedback devices. Bit-banging does not allow the control signal magnitude to be directly controlled, but removes the requirement for a PWM module and potentially reduces processor or interface device cost.
The techniques described above for independently varying the magnitude and frequency of a vibration can be used on a single actuator or on multiple actuators simultaneously. With multiple actuators, motor/mass combinations can work together to achieve a fuller bandwidth. This is analogous to the woofer/tweeter relationship in an audio speaker. For example, the Sony
Dual Shock gamepad uses two motors, and both can be used according to the present invention to reach up toward 100 Hz dynamic range. In one implementation, the large mass/motor combination can use both control techniques over its frequency range, with a blend threshold at 10 Hz. The small mass/motor combination can use only the second method ("fixed-time per period" technique) to add power to the small motor with increased frequency (since on-time does not decrease with frequency increase using the second method). This reduces the tendency of the larger amplitude vibrations of the large motor to drown out the small motor's smaller, high frequency vibrations. Again, filters can be used to achieve the desired balance between large and small motor contribution over the frequency range.
In some embodiments, such as at 5 volts, the small motor may deliver so little magnitude that the vibration is almost undetectable by the user. The frequency dynamic range can be shortened to 25Hz by defining the device to have a maximum capable frequency of 25 Hz, choosing scalars appropriately. This allows every haptic effect to be played using the large motor, and the small motor is then unnecessary. A better 5V solution may be to increase the small ERM to a more moderate size, allowing stronger vibrations to be output.
In alternate embodiments, a more simplified control method might be used for periodic effects without using the methods described above. A commanded magnitude can be ignored, i.e. only the frequency command is used. For example, in a two-motor vibrotactile device, a predetermined threshold frequency can be used to determine which motor is used to output vibrations. For example, if a vibration above 15 Hz is commanded, the small motor/mass is rotated. The small motor/mass can be commanded with a frequency proportional to the commanded frequency, or, alternatively, may only rotated at one frequency. This effect can be used for such events as the engine of vehicle revving in a driving game. If a vibration below 15
Hz is commanded, then the large motor/mass can be rotated at a predetermined frequency or by a frequency proportional to the commanded frequency (where the commanded frequency is scaled to a frequency in the range of the large motor). This effect can be used for such events as moving over bumpy ground or collisions in a game.
Controlling Non-Periodic Effects with Vibrotactile Devices
The above-described methods are used to play fully featured periodic force effects on an ERM. Other, non-periodic kinesthetic effects, including some condition effects, may also be mapped to vibrotactile effects that can be played on vibrotactile devices.
Nonperiodic effects are determined based not on time and magnitude parameters, but on other parameters instead of or in addition to these parameters. Nonperiodic effects are not as simply mapped to vibrotactile devices as periodic effects. For example, a vector force is a non- period effect which is output in a specified direction for a specified length of time. Since vibrotactile devices generally cannot output forces in particular directions, the mapping of the vector force to a vibrotactile force sensation is not as direct as with periodic effects.
Some non-periodic kinesthetic effects are conditions. "Conditions" are closed-loop force feedback effects which can be commanded on more sophisticated kinesthetic force feedback devices, as opposed to the open-loop effects such as vibrations and jolts. Condition effects typically require position and/or velocity (and/or acceleration) data of a user manipulandum (such as a wheel, joystick, or mouse) in the determination of an output force. For example, spring condition forces are dependent on the current position of the user manipulandum and damper condition forces are dependent on the current velocity of the user manipulandum.
For the most part, conditions are appropriate for force feedback (kinesthetic) devices in which a manipulandum is moved and forces are provided in the degrees of freedom of the manipulandum. Conditions are open for inteφretation when trying to display them on a solely vibrotactile-feedback device.
In one embodiment, conditions can be simply ignored, so that if an application program commands a condition force to be output, the local microprocessor of the vibrotactile device (or a driver or other program on the host) simply ignores the command and no forces are output.
However, in other embodiments, conditions can be mapped to other vibrotactile sensations that allow a user to experience at least some force feedback that approximates a condition force effect or signifies an event. Descriptions of examples of mappings for non-periodic force effects follow.
Vector Forces: Vector forces are kinesthetic force effects which provide a force in a specified direction in one or more degrees of freedom of the manipulandum. For example, in a racing or driving game, a vector force is used to output a collision sensation in the rotation of a steering wheel when the car impacts an object such as a wall or other car, a shift sensation when a stickshift in moved, or a spin out sensation when the car loses control. The magnitude of the vector force is different based on the severity of the event or other factors.
In most vibrotactile devices, a directional force cannot typically be output and must be approximated using vibrations or jolts. A jolt or vibration output from a vibrotactile device, however, can be similar to a vector force. Preferably, a vibration is output having a low frequency and high magnitude; for example, a single jolt can be output by commanding a 1 Hz frequency with a duration of 1 second, to provide a single rotation. In some embodiments, commands for particular vector forces can be ignored for vibrotactile devices. For example, in a driving game, low magnitude vector forces might be ignored by the vibrotactile device since such forces are often used to provide subtle driving forces that are not easily duplicated with vibrotactile effects. In a vibrotactile gamepad device having two motors, the large motor can be caused to rotate to cause a vibration corresponding to the vector force; or both motors can be rotated.
In another preferred embodiment, a vector force can be directly mapped to a vibration, where the vibration has a magnitude corresponding to the magnitude of the vector force. The magnitude is preferably independently controlled using the methods described above with respect to periodic forces. The magnitude can be proportional to or otherwise based on the magnitude parameter of the vector force command. The frequency of the vibration can be a constant frequency predefined at a particular value; for example, the fixed frequency can be the highest frequency the device is capable of outputting. In some embodiments, the vibration frequency for vector forces can also be user selectable.
Another technique to map vector forces to vibrations is to map vector force magnitude directly to motor command, i.e. the magnitude of the vector force, as expressed by a duty cycle, is used as a command to rotate the motor, e.g. a 50% magnitude becomes a 50% duty cycle motor command. This causes a pulse of force that substitutes for the vector force. In some embodiments, instead of mapping kinesthetic vector force magnitude commands to the full range of motor amplitude commands, the vector force magnitudes can be mapped to only the upper half of the motor amplitude command range. This allows high enough magnitudes to be commanded to always cause a full mass revolution, and thus makes the feel response more linear.
Alternatively, only a lower portion of the amplitude command range can be used to always provide harmonic mass motion.
In a simpler alternate embodiment, magnitude thresholds for the requested magnitude can be used to determine when and how to approximate vector forces for devices having multiple motors. For example, a first threshold can be provided at 75% magnitude (i.e., 75% of the maximum magnitude that can be output). In one example for devices having a small motor and a large motor, for vector forces between 0 and 75% magnitude, the large motor can be controlled to output a vibration (or single jolt) that has a magnitude proportional to the commanded vector force magnitude. If a commanded vector force has a magnitude of 76% to 100%> of maximum, then both the large and small motors can be used to provide vibration forces having the maximum possible magnitude. It should also be noted that thresholds such as described above can be used in conjunction with the methods used with periodics to independently control magnitude and frequency of vibrations within particular thresholds. Such combination techniques can be used with all the types of force effects described herein.
The mapping to vector forces can (in any embodiment) also be selectively applied based on the application or puφose commanded. For example, if the vector force has been output for a collision occurring in a game, the equivalent vibration forces may be appropriate. But if the vector force is being used to simulate a dynamic spring return on the steering wheel during a turn, then the vibration force may not be appropriate.
In some vibrotactile devices, directional forces may be able to be output. For example, a rotating or moving mass can be controlled to rotate so that a directional force is approximated in a particular direction of the degree of freedom of the mass. For example, in devices having multiple motors, the motors can be controlled to rotate in phase with each other to cause a force in a particular direction. In such embodiments, a vector force can be mapped to a directional force on the vibrotactile device if a force can be output in or near the direction of the vector force. Spring Forces: Spring forces are conditions which are typically based on the formula F = kX, indicating that the output force F is based on the current position X of the user manipulandum in particular degree of freedom, and a spring constant k. For example, in driving games, spring forces can be used to center the steering wheel to the center position, where the user feels a force having a magnitude proportional to the amount of rotation of the wheel from the center position and in a direction towards the center position.
When mapping a spring kinesthetic force to a vibrotactile force effect, different approaches can be taken. In some circumstances, the spring command can be ignored by the vibrotactile device since the spring may not be effectively duplicated using vibrations or jolts. For example, the spring force can often be ignored when it is used to center a user manipulatable object in a degree of freedom, since a vibration or jolt would simply be distracting to the user in such a circumstance.
Alternatively, or in other circumstances, a vibration force can be output for the spring force. For example, when driving a vehicle and a turn is made, a centering spring force tends to move the steering wheel toward the center. A spring force can be output to simulate this effect for a kinesthetic device. In a vibrotactile device, a vibration can be output.
One example is shown in FIGURES 10a and 10b. Fig. 10a is a graph showing a typical spring force output on a kinesthetic force feedback device, where displacement of a manipulandum along the x-axis from the origin O causes a spring force resisting that motion to be output, having a magnitude increasing with the amount of displacement. Fig. 10b shows an example of a mapping to a vibrotactile domain. A vibration is output only above a particular spring command magnitude threshold, such as 75%, and may have a fixed, predetermined frequency (or user- selectable frequency). Thus, the vibration is output only for higher-magnitude spring commands. For vibrotactile devices having a manipulandum, such as a joystick on a gamepad, the vibration is only output at manipulandum positions (x-axis) corresponding to a 75% magnitude or greater on the spring force of Fig. 10a. For command magnitudes greater than the threshold, the vibration magnitude can be increased at a steep rate in proportion to the spring magnitude command, up to the maximum magnitude, as shown. This can be used to simulate the wobbling or shimmying of a steering wheel at high-speed turns in a vehicle; thus, the spring force effect is not directly simulated, but a different effect is output as a substitute for the spring force effect. In other embodiments, a vibration can be output for all magnitudes of the spring command (e.g., dependent on the position of the user manipulandum).
Springs (or dampers, below) can also be played as vector forces as described above, e.g. on only one motor, such as the smaller motor. This may be helpful to some users while gaming since it may provide feedback about the joystick position; however, other users may find it disconcerting. If condition forces are played on an ERM-based vibrotactile controller, the user should preferably be given the opportunity to turn them off or alter them per situation, perhaps in a profiler program presented at the application layer or other software layer on the host computer.
Obstruction forces: Obstructions such as walls are often simulated with force feedback in position control applications by using a spring force. In vibrotactile applications, an encounter with a wall or other object can be simulated using a jolt or low frequency vibration. Preferably, the force is output at close to maximum magnitude when the initial contact with the wall is made. After the initial contact, the vibration can be continued if the user-controlled graphical object (e.g. cursor, car, etc.) continues to be in contact with the wall. Or the vibration can be ramped down to an eventual zero magnitude using the control methods described above for periodic effects. Such control methods work with envelopes applied to periodic waves, such as to slowly ramp down magnitude of a vibration. Such use of periodics with attack and fade parameters are described in greater detail in U.S. Patent No. 5,959,613.
Damping Forces: Damping forces are condition forces which are typically based on the formula F = Bv, indicating that the force F is based on the current velocity v of the user manipulandum in a particular degree of freedom, and a damping constant B. This force tends to subject the user manipulandum to resistance as it moves, where the faster it is moved, the greater resistance is output. For example, in a driving game, a damping force can be output on a steering wheel when the controlled car moves into a simulated pool of oil or water, indicating the sluggishness of steering in the body of liquid.
For a vibrotactile device, a damping force can be mapped to high frequency vibrations.
For example, in a two-motor device, the small motor/mass can be controlled to rotate to a small degree or at a low frequency, e.g., just enough to indicate a change in the terrain type to the user in a driving game. This can simulate "sloshing" or other vibration effects such as when a vehicle moves through a liquid. In other embodiments, a damping force can be mapped to a vibration that has a magnitude in proportion to the current velocity of the controlled object (such as a vehicle or cursor), or proportional to the current velocity of a user manipulandum on the device, such as a small joystick. The vibration magnitude can be implemented as described above for periodics. In other circumstances, a damping command can be ignored by a vibrotactile device. For example, if a damping force is used to provide greater control to the user in a game, a vibration would probably not provide such greater control and should not be output.
Any other types of kinesthetic force effects can also be mapped to vibrotactile effects in similar ways as described above. For example, friction forces might be mapped using a high frequency vibration, which has a magnitude correlated with velocity of a controlled graphical object or with the velocity of the user object. Inertia forces might be mapped using low frequency or subtle vibrations to indicate the change of conditions. Texture forces provide jolts, pulses, or other forces to a user based on a manipulandum 's movement through a texture field having features causing the jolts at particular locations. Such texture forces can be simulated by outputting vibrations with the ERM motor having a frequency approximately corresponding with the rate of encountering the texture features by the user.
In summary, mappings are provided between kinesthetic effects and vibrotactile effects to allow a vibrotactile device to respond to full-featured application force commands. A novel control method is described to play periodics on simple ERM motors moving uni-directionally, where frequency and magnitude are varied independently. Other mappings provide compelling vibrotactile effects in place of kinesthetic effects. The end result is a control method that allows a low-cost vibrotactile product to play at least a subset of kinesthetic forces. Since conditions and other kinesthetic force effects may be greatly reduced in effectiveness or not played at all on lower-cost vibrotactile devices but are easily played on more expensive kinesthetic haptic feedback devices, this allows for product differentiation in the market. Customers are further able to choose how much haptic feedback functionality and sophistication they want based on how much they are willing to pay for a haptic feedback interface device.
Providing Consistent Force Effects using an ERM Motor
The hardware topology shown in Fig. 4 supports all of the control schemes described above. In one embodiment, there is no sensor feedback to the host computer or local microprocessor regarding the position of the motors, i.e. the system is open loop. The control methods discussed above need no feedback regarding the positions of the motors. Since no special sensors are required to provide motor position to a controller, the open loop system allows the interface device to be very cost effective.
However, without sensors to close a feedback control loop, concern about robustness may arise. For example, it can sometimes be difficult to determine whether an ERM motor is actually spinning when it should be. For instance, some magnitude/frequency combinations can be commanded which should cause the motor to turn but may only actually cause the motor to turn if the motor were already turning. Or, the motor may not turn if the user is shaking, banging, or tilting the interface device or manipulandum in some way. Or, the motor may only turn if the user is holding the interface device in a particular orientation where gravity is less of a factor on the ERM. This can cause the same commanded force effects to feel differently to the user depending on controller or ERM orientation or other conditions. Such a result is undesired in most applications where a force effect should feel consistent each time it is commanded by an application program or other controller, regardless of how the device is held or oriented.
One method of ensuring that the mass is rotating when it should be, thereby providing consistent forces, is to model system dynamics. This method does not require feedback sensors; rather, a model of the physical system is developed beforehand and the model is used to predict the ERM's behavior. The predictions are then used to improve ERM performance as the ERM starts and continues to move. A different technique more easily implemented is to command a motor to spin at maximum magnitude for an initial period of a commanded force sensation to start the motor. This causes the motor to behave in a consistent manner if the worst case scenario is planned for, e.g. an orientation of the interface device causing the ERM to move against gravity where gravity is strongest. In such a worst case, the ERM is still forced to spin, thus allowing the commanded sensation to be output more easily once the ERM is in motion. Preferably, this initial period of maximum rotation is commanded using the "fixed time per period" technique described above for periodic force sensations. After the first period of the vibration, the output reverts to the commanded parameters that specify a particular magnitude and frequency (and if the frequency is low enough, the "fixed time per period" method is still used). No sensor or feedback loop is required for this embodiment, allowing the device to be reduced in cost.
In another embodiment, when the motor is operated in stall and it is known that the motor should rotate, voltage across the motor can be measured and fed back into an AID converter on the local microprocessor. This signal will indicate whether the motor is stalled. This feedback loop can aid in meeting power requirements without adding much cost to the product.
Yet another technique to ensure desired operation, without the use of feedback sensors, is to carefully quantify desired motor operation over the entire frequency/magnitude domain beforehand, then maintain a counter during operation. If a haptic effect should cause the motor to fully rotate, and the counter < "max" (the maximum count value of the counter), then the counter is incremented and the requested control signal is output. If the haptic effect is known to not rotate the motor (known through empirical testing) but is desired to cause it to rotate, and the counter > 0, the counter is decremented, and the requested control signal is output. If a haptic effect is in the regime (i.e. has an amplitude and frequency) where it should cause the motor to rotate but is not able to reach the activation energy to start the rotation, and the counter < max, increment the counter. The requested control signal is not output; instead, the maximum amplitude control signal is output. And, if a haptic effect is in this same regime, but the counter = max, the requested control signal is output.
Finally, in some embodiments, low-cost specifications can be relaxed and a sensor can be provided to sense the position and/or motion of the motor. The most important use for a sensor is to close a feedback loop around the actuator and ensure it is moving properly (either oscillating or spinning). A quadrature encoder on the motor shaft can be used for a sensor, but its cost and advanced functionality may be excessive for this application. Alternatively, a simple Boolean sensor, either an IR sensor blocked by the ERM, or Hall effect sensor with flux path through a steel or other metal ERM, can be used. This simple type of sensor can indicate to the local microprocessor whether the mass is making full revolutions.
For example, the sensor can be positioned at top dead center, i.e. at the point of highest travel of the mass, where gravity has the greatest adverse force against reaching this point. This ensures that the ERM triggers the sensor only when the ERM has traveled over the top and completed a full revolution, not just a large pendulum oscillation. Before a periodic effect is first started, the sensor position can be recorded and stored. After a fixed amount of time that is dependent on the inertia of the ERM and the frequency of the commanded effect, the sensor output is checked to determine if it has toggled or otherwise sensed the passage of the ERM. If the ERM has not crossed the top center position in the allotted time, the actuator force can be increased beyond the amount commanded by the periodic.
A Boolean sensor can also be used to make sure the mass is oscillating with a minimum magnitude (as opposed to rotating) by following a similar process as just described but by locating the sensor at bottom dead center. If the ERM oscillates enough to cause the sensor to toggle briefly before the ERM swings back, a minimum magnitude of oscillation can be ensured. If an encoder is used, control can become more sophisticated to further "robustify" and ensure vibration output, since the encoder has a greater resolution and can sense the ERM at many positions. A position controller can ensure that the ERM has rotated fully when it should, or that oscillations (when the ERM is not commanded to cross top dead center) are of exact magnitude.
Lastly, when using encoders for feedback, velocity control (trajectories) can be performed to convey vibration magnitude instead of controlling open-loop control signal on-times. Thus, the encoder can detect the velocity of the motor and determine whether it is at the commanded velocity. The sensed velocity is fed back to the commanding controller and the controller adjusts the command to provide the desired motor velocity.
In many embodiments, there may not be a need for such above-described mass rotation safeguards when the motor bus is run at a higher voltage. For example, a region of frequency/magnitude combinations can be identified where the ERM will spin if already spinning but will not spin up from stop. At a lower motor voltage, such as 5V, this region may be larger, placing more of a demand on rotation safeguards. One safeguard technique is to initially "kick" or pulse the rotating mass for a known amount of time (e.g., 30 ms) during the first period of any periodic effect. This ensures that the mass will spin if the force command calls for spinning; otherwise the mass will not be able to rotate past its highest position and will fall into pendulous motion. This technique, though simple, can add a significant amount of repeatability to each periodic command, no matter how the motor had been positioned or moving beforehand. The initial pulse is barely detectable to the user and helps the ERM accelerate to speed quickly. One disadvantage with this method is that some force effects having increasing envelopes may have a very low magnitude or frequency initially, and the initial kick may be too high a magnitude and/or frequency to subtly output the desired effect. Also, if the initial kick occurs too soon, the ERM may settle into a resting state before the force effect can be output.
While this invention has been described in terms of several preferred embodiments, it is contemplated that alterations, permutations and equivalents thereof will become apparent to those skilled in the art upon a reading of the specification and study of the drawings. For example, many different types of vibrotactile devices can be used with the control features of the present invention, including gamepads, remote controls, joystick handles, mice, steering wheels, grips, or other manipulandums or devices. Furthermore, certain terminology has been used for the puφoses of descriptive clarity, and not to limit the present invention. It is therefore intended that the following appended claims include alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
What is claimed is:

Claims

1. A method for controlling a vibrotactile interface device from a host microprocessor, the method comprising:
determining a desired haptic effect to be output by said vibrotactile interface device to a user operating said vibrotactile interface device; and
providing information to said vibrotactile interface device, said information describing a magnitude and a frequency that are independent of each other, wherein a periodic control signal based on said information drives a rotary motor of said vibrotactile interface device to cause a mass to rotate in said one direction, wherein said rotation of said mass causes a vibration having said magnitude and said frequency to be output to said user, wherein said magnitude of said vibration is based on a duty cycle of said control signal, and wherein said frequency of said vibration is based on a frequency of said control signal.
2. A method as recited in claim 1 wherein said mass is an eccentric mass.
3. A method as recited in claim 1 wherein said control signal can be either on or off, and wherein said control signal is turned on for an on-time for each period of said vibration, said on- time being a fraction of a period of said vibration, and said on-time determining a magnitude of said vibration.
4. A method as recited in claim 3 wherein said control signal is applied at a predetermined time of every period of said vibration to allow said magnitude and frequency of said vibration to be specified independently of each other.
5. A method as recited in claim 4 wherein said control signal is applied at a beginning point of every period of said vibration.
6. A method as recited in claim 1 wherein said information includes a high level command and at least one parameter, wherein said vibrotactile interface device includes a local microprocessor, separate from said host microprocessor, for parsing said high level command.
7. A method as recited in claim 6 wherein said high level command is a vibration command causing said vibration, and wherein said at least one parameter includes a magnitude parameter and a frequency parameter for said vibration.
8. A method as recited in claim 6 wherein said determining a desired vibrotactile effect includes mapping said vibrotactile effect from a kinesthetic effect that was commanded by a program running on said host computer.
9. A method as recited in claim 8 wherein said kinesthetic effect is one of a spring effect, damper effect, and obstruction effect.
10. A method as recited in claim 8 wherein said kinesthetic effect is a periodic effect having a separately-specified magnitude and frequency.
11. A method as recited in claim 1 wherein said determination of a desired vibrotactile effect is performed by a force feedback driver program running on said host computer.
12. A method as recited in claim 1 wherein said vibrotactile interface device is a gamepad controller grasped by a hand of a user, and wherein said haptic effect is correlated with at least one event or interaction occurring within a graphical environment implemented by said host computer.
13. A method as recited in claim 12 wherein said gamepad controller includes a joystick having two degrees of freedom and providing input to said host computer when manipulated by said user.
14. A method as recited in claim 1 wherein said information provided to said vibrotactile interface device is said control signal which is provided to said actuator.
15. A method as recited in claim 1 wherein said control signal drives said motor in only one direction.
16. A method for providing a vibration for a haptic feedback device coupled to a host microprocessor, the method comprising:
providing an actuator for said haptic feedback device, said actuator including a rotatable mass; and
receiving information at said haptic feedback device and producing a periodic control signal based on said information, said control signal driving said actuator in one direction to rotate said mass in said one direction about an axis of rotation of said actuator such that said rotation of said mass induces a vibration in said haptic feedback device and felt by a user contacting said haptic feedback device, wherein a magnitude and a frequency of said vibration can be adjusted independently of each other by adjusting said control signal, wherein said magnitude of said vibration is based on a duty cycle of said control signal, and wherein said frequency of said vibration is based on a frequency of said control signal.
17. A method as recited in claim 16 wherein said control signal is either on or off, wherein an on-time of said control signal is adjusted to adjust said magnitude of said vibration.
18. A method as recited in claim 17 wherein said control signal is applied at a predetermined point of time of every period of said vibration to adjust said magnitude and frequency of said vibration independently of each other.
19. A method as recited in claim 18 wherein said control signal is turned on for a fraction of each period of said vibration.
20. A method as recited in claim 18 wherein said control signal is applied at a beginning of every period of said vibration.
21. A method as recited in claim 17 wherein said on-time of said control signal is determined as a percentage of a period of said vibration, wherein said percentage is proportional to a desired magnitude of said vibration.
22. A method as recited in claim 17 wherein said on-time of said control signal is determined as a predetermined amount of time for each period of said vibration.
23. A method as recited in claim 17 wherein an amplitude of said control signal is varied to provide said independent magnitude and frequency.
24. A method as recited in claim 17 wherein said on-time of said control signal is determined based on a commanded frequency of said vibration, wherein if said commanded frequency is below a predetermined threshold frequency, said on-time is determined using a first method, and if said commanded frequency is above said predetermined threshold frequency, said on-time is determined using a second method.
25. A method as recited in claim 24 wherein said first method determines said on- time as a percentage of a period of said vibration, wherein said percentage is proportional to a desired magnitude of said vibration, and wherein said second method determines said on-time as a predetermined amount of time for each period of said vibration.
26. A method as recited in claim 16 wherein said vibrotactile interface device is a gamepad controller grasped by a hand of a user and said host microprocessor is provided in a host computer, and wherein said vibration is correlated with at least one event or interaction occurring within a graphical environment implemented by said host computer.
27. A method as recited in claim 16 further comprising monitoring a position of said mass about said axis of rotation to ensure said mass rotates when commanded to by said control signal.
28. A method as recited in claim 16 wherein said information received at said haptic feedback device is a host command including magnitude and frequency parameters describing a desired magnitude and frequency of said vibration, wherein circuitry on said haptic feedback device bases said control signal on said magnitude and frequency parameters to cause said magnitude and frequency of said vibration to substantially equal said desired magnitude and frequency.
29. A method as recited in claim 16 wherein said information received at said haptic feedback device is substantially equivalent to said control signal such that said host microprocessor directly controls said actuator.
30. A method as recited in claim 16 wherein said information describes a kinesthetic effect, and wherein said control signal is provided from said kinesthetic effect information by mapping a vibrotactile effect to said kinesthetic effect.
31. A method as recited in claim 30 wherein a local microprocessor in said haptic feedback device maps said vibrotactile effect to said kinesthetic effect.
32. A method as recited in claim 16 further comprising sending an initial control signal to said actuator to start said mass rotating before said vibration is started to be output, thereby ensuring that said mass rotates when said vibration is commanded to be output.
33. A method as recited in claim 32 wherein said initial control signal is sent for a duration of one period of said vibration at a maximum commanded magnitude.
34. A method as recited in claim 16 wherein said actuator is one of multiple actuators included in said haptic feedback device, wherein each of said actuators is controlled to output a vibration.
35. A method as recited in claim 16 wherein said control signal is modified by envelope parameters received from said host computer, said envelope parameters modifying said magnitude of said vibration.
36. A method as recited in claim 16 wherein said control signal drives said motor in only one direction.
37. A method for providing a vibration for a haptic feedback device coupled to a host microprocessor, the method comprising:
providing an actuator for said haptic feedback device, said actuator including a rotatable mass; and receiving information at said haptic feedback device and producing a periodic control signal based on said information, said control signal driving said actuator in one direction to rotate said mass in said one direction about an axis of rotation of said actuator such that said rotation of said mass induces a vibration in said haptic feedback device and felt by a user contacting said haptic feedback device, wherein a magnitude and a frequency of said vibration can be adjusted independently of each other by adjusting said control signal, wherein said magnitude of said vibration is based on a duty cycle of said control signal, and wherein said frequency of said vibration is based on a frequency of said control signal.
38. A haptic feedback device for providing vibrotactile sensations to a user, said haptic feedback device coupled to a host microprocessor and comprising:
a housing contacted by said user;
an actuator coupled to said housing and including an eccentric mass positioned offset on a rotating shaft of said actuator, wherein said mass can be rotated by said actuator; and
a circuit coupled to and controlling said actuator and receiving a control signal, said circuit including an amplifier coupled to said actuator, said amplifier controlling said actuator to rotate said mass about an axis of rotation and induce a vibration in said housing in accordance with said control signal, said vibration experienced by said user as vibrotactile sensations, wherein a frequency and a magnitude of said vibration can be adjusted independently of each other by adjusting said control signal, wherein said magnitude of said vibration is based on a duty cycle of said control signal, and wherein said frequency of said vibration is based on a frequency of said control signal.
39. A haptic feedback device as recited in claim 38 wherein said amplifier is a unidirectional amplifier.
40. A haptic feedback device as recited in claim 38 wherein said circuit includes a local microprocessor that receives information from said host microprocessor and outputs said control signal.
41. A haptic feedback device as recited in claim 38 further comprising a sensor for determining a position of said mass in a rotational degree of freedom of said mass.
42. A haptic feedback device as recited in claim 38 wherein said haptic feedback device is a gamepad controller, said gamepad controller receiving information from said host microprocessor which determines when said vibrotactile sensations are to be output based on events occurring within a graphical environment implemented and displayed by said host microprocessor.
43. A haptic feedback device as recited in claim 40 wherein said gamepad controller includes a joystick having two degrees of freedom and providing input to said host computer when manipulated by said user.
44. A haptic feedback device as recited in claim 38 wherein said haptic feedback device includes a plurality of actuators, wherein said circuit causes each of said plurality of actuators to rotate a mass associated each actuator and induce a vibration in said housing.
45. A haptic feedback device as recited in claim 38 wherein said actuator receives power over a interface bus connecting said haptic feedback device to said host microprocessor.
46. A method for allowing kinesthetic haptic effects to be output on a vibrotactile interface device, the method comprising:
receiving a command describing a kinesthetic haptic effect, said kinesthetic haptic effect intended to cause forces to be output in at least one degree of freedom of a manipulandum of a kinesthetic haptic feedback device that is manipulated by a user; and
mapping said kinesthetic haptic effect to a vibrotactile haptic effect, said vibrotactile haptic effect intended to cause vibrotactile forces to be output to a user contacting said vibrotactile interface device; and
providing said vibrotactile haptic effect to be output by said vibrotactile interface device.
47. A method as recited in claim 46 wherein said kinesthetic haptic effect is a periodic effect having a specified magnitude and frequency, and wherein said mapped vibrotactile haptic effect is a vibration having an equivalent magnitude and frequency.
48. A method as recited in claim 47 wherein said mapped vibration is commanded using a control signal provided to an actuator, said control signal being provided during a fraction of each period of said vibration.
49. A method as recited in claim 46 wherein said kinesthetic haptic effect is a nonperiodic effect, and wherein said mapped vibrotactile haptic effect is a vibration having a magnitude based on a magnitude of said kinesthetic haptic effect.
50. A method as recited in claim 46 wherein said kinesthetic haptic effect is a spring effect, and wherein said mapped vibrotactile haptic effect is a vibration that is output if said spring effect has a magnitude above a predetermined threshold.
51. A method as recited in claim 46 wherein said kinesthetic haptic effect is a damper effect, and wherein said mapped vibrotactile haptic effect is a vibration having a low frequency.
52. A method as recited in claim 46 wherein said kinesthetic haptic effect is a vector force effect, and wherein said mapped vibrotactile haptic effect is a vibration that is output using a small motor of said vibrotactile interface device if said vector force effect has a magnitude below a first threshold, said vibration is output using a large motor of said vibrotactile interface device if said vector force effect has a magnitude above said first threshold, and said vibration is output using both said large motor and said small motor if said vector force effect has a magnitude above a second threshold.
53. A haptic feedback device for providing vibrotactile sensations to a user, said haptic feedback device coupled to a host microprocessor and comprising:
a housing contacted by said user;
an actuator coupled to said housing and including an eccentric mass positioned offset on a rotating shaft of said actuator, wherein said mass can be rotated by said actuator; and
a circuit that drives said actuator in one direction only, said circuit receiving a control signal and causing said actuator to rotate said mass about an axis of rotation; and
an obstacle member that prevents said mass from making a complete rotation, said obstacle member causing said mass to move in a direction opposite to said direction driven by said circuit after said mass impacts said obstacle, wherein a vibration is caused in said housing in accordance with said control signal and at least in part by said mass impacting said obstacle member, wherein said control signal allows a frequency and magnitude of said vibration to be controlled independently of each other.
54. A haptic feedback device as recited in claim 53 wherein said obstacle member is a hard stop provided in a path of motion of said mass.
55. A haptic feedback device as recited in claim 53 wherein said obstacle member is a spring member including a compliance to increase energy in said movement of said mass in said opposite direction.
PCT/US2000/026615 1999-09-28 2000-09-27 Controlling haptic sensations for vibrotactile feedback interface devices WO2001024158A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE20080209U DE20080209U1 (en) 1999-09-28 2000-09-27 Control of haptic sensations for interface devices with vibrotactile feedback
AU77256/00A AU7725600A (en) 1999-09-28 2000-09-27 Controlling haptic sensations for vibrotactile feedback interface devices

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15635499P 1999-09-28 1999-09-28
US60/156,354 1999-09-28

Publications (1)

Publication Number Publication Date
WO2001024158A1 true WO2001024158A1 (en) 2001-04-05

Family

ID=22559218

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/026615 WO2001024158A1 (en) 1999-09-28 2000-09-27 Controlling haptic sensations for vibrotactile feedback interface devices

Country Status (2)

Country Link
AU (1) AU7725600A (en)
WO (1) WO2001024158A1 (en)

Cited By (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7082570B1 (en) 2002-08-29 2006-07-25 Massachusetts Institute Of Technology Distributed haptic interface system and method
CN101391146A (en) * 2007-09-21 2009-03-25 美国索尼电脑娱乐公司 Method and apparatus for enhancing entertainment software through haptic insertion
WO2010056466A2 (en) * 2008-11-12 2010-05-20 Immersion Corporation Haptic effect generation with an eccentric rotating mass actuator
US7755602B2 (en) 1995-11-30 2010-07-13 Immersion Corporation Tactile feedback man-machine interface device
US8007282B2 (en) 2001-07-16 2011-08-30 Immersion Corporation Medical simulation interface apparatus and method
US8441444B2 (en) 2000-09-28 2013-05-14 Immersion Corporation System and method for providing directional tactile sensations
US8576174B2 (en) 2002-04-25 2013-11-05 Immersion Corporation Haptic devices having multiple operational modes including at least one resonant mode
CN103522262A (en) * 2012-06-29 2014-01-22 罗伯特·博世有限公司 Cotrolling a battery-operated handheld power tool
US9134795B2 (en) 2000-09-28 2015-09-15 Immersion Corporation Directional tactile feedback for haptic feedback interface devices
WO2015187758A1 (en) * 2014-06-06 2015-12-10 Microsoft Technology Licensing, Llc Systems and methods for controlling feedback for multiple haptic zones
US9245428B2 (en) 2012-08-02 2016-01-26 Immersion Corporation Systems and methods for haptic remote control gaming
EP2507685A4 (en) * 2009-12-02 2016-04-13 Samsung Electronics Co Ltd Mobile device and control method thereof
EP3088064A1 (en) * 2015-04-21 2016-11-02 Nintendo Co., Ltd. Vibration signal generation program, vibration signal generation system, vibration signal generation apparatus, vibration signal generation method, and data output program
US9492847B2 (en) 1999-09-28 2016-11-15 Immersion Corporation Controlling haptic sensations for vibrotactile feedback interface devices
US9625905B2 (en) 2001-03-30 2017-04-18 Immersion Corporation Haptic remote control for toys
US9678571B1 (en) 2016-09-06 2017-06-13 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
CN107077233A (en) * 2014-12-23 2017-08-18 意美森公司 Control the power distribution to haptic output devices
US9830784B2 (en) 2014-09-02 2017-11-28 Apple Inc. Semantic framework for variable haptic output
US9864432B1 (en) 2016-09-06 2018-01-09 Apple Inc. Devices, methods, and graphical user interfaces for haptic mixing
DK201670725A1 (en) * 2016-09-06 2018-03-19 Apple Inc Devices, Methods, and Graphical User Interfaces for Generating Tactile Outputs
US9984539B2 (en) 2016-06-12 2018-05-29 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US9996157B2 (en) 2016-06-12 2018-06-12 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
EP3392746A1 (en) * 2010-03-31 2018-10-24 Immersion Corporation System and method for providing haptic stimulus based on position
EP3462282A1 (en) * 2017-09-27 2019-04-03 Vestel Elektronik Sanayi ve Ticaret A.S. Vibration-generating device and method of operation
WO2019204749A1 (en) * 2018-04-20 2019-10-24 Immersion Corporation Kinesthetic haptic feedback for haptically-enabled surfaces
US10508883B2 (en) 2012-05-22 2019-12-17 Haptech, Inc. Method and apparatus for firearm recoil simulation
EP3582073A1 (en) * 2018-06-15 2019-12-18 Immersion Corporation Generation and braking of vibrations
US10671186B2 (en) 2016-06-15 2020-06-02 Microsoft Technology Licensing, Llc Autonomous haptic stylus
US10852093B2 (en) 2012-05-22 2020-12-01 Haptech, Inc. Methods and apparatuses for haptic systems
EP3766768A1 (en) * 2014-11-28 2021-01-20 Haptech, Inc. Methods and apparatuses for haptic systems
CN112933590A (en) * 2021-03-31 2021-06-11 歌尔股份有限公司 Vibration control method of terminal device, terminal device and storage medium
US11314330B2 (en) 2017-05-16 2022-04-26 Apple Inc. Tactile feedback for locked device user interfaces

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5619180A (en) * 1993-01-14 1997-04-08 Massachusetts Inst Technology Apparatus for providing vibrotactile sensory substitution of force feedback
US5694013A (en) * 1996-09-06 1997-12-02 Ford Global Technologies, Inc. Force feedback haptic interface for a three-dimensional CAD surface

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5619180A (en) * 1993-01-14 1997-04-08 Massachusetts Inst Technology Apparatus for providing vibrotactile sensory substitution of force feedback
US5694013A (en) * 1996-09-06 1997-12-02 Ford Global Technologies, Inc. Force feedback haptic interface for a three-dimensional CAD surface

Cited By (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9690379B2 (en) 1995-11-30 2017-06-27 Immersion Corporation Tactile feedback interface device
US7755602B2 (en) 1995-11-30 2010-07-13 Immersion Corporation Tactile feedback man-machine interface device
US9492847B2 (en) 1999-09-28 2016-11-15 Immersion Corporation Controlling haptic sensations for vibrotactile feedback interface devices
US9134795B2 (en) 2000-09-28 2015-09-15 Immersion Corporation Directional tactile feedback for haptic feedback interface devices
US8441444B2 (en) 2000-09-28 2013-05-14 Immersion Corporation System and method for providing directional tactile sensations
US9625905B2 (en) 2001-03-30 2017-04-18 Immersion Corporation Haptic remote control for toys
US8007282B2 (en) 2001-07-16 2011-08-30 Immersion Corporation Medical simulation interface apparatus and method
US8576174B2 (en) 2002-04-25 2013-11-05 Immersion Corporation Haptic devices having multiple operational modes including at least one resonant mode
US7082570B1 (en) 2002-08-29 2006-07-25 Massachusetts Institute Of Technology Distributed haptic interface system and method
CN101391146A (en) * 2007-09-21 2009-03-25 美国索尼电脑娱乐公司 Method and apparatus for enhancing entertainment software through haptic insertion
EP2039404A3 (en) * 2007-09-21 2010-05-12 Sony Computer Entertainment America, Inc. Method and apparatus for enhancing entertainment software through haptic insertion
WO2010056466A3 (en) * 2008-11-12 2011-01-13 Immersion Corporation Haptic effect generation with an eccentric rotating mass actuator
US8884884B2 (en) 2008-11-12 2014-11-11 Immersion Corporation Haptic effect generation with an eccentric rotating mass actuator
WO2010056466A2 (en) * 2008-11-12 2010-05-20 Immersion Corporation Haptic effect generation with an eccentric rotating mass actuator
EP2507685A4 (en) * 2009-12-02 2016-04-13 Samsung Electronics Co Ltd Mobile device and control method thereof
EP3392746A1 (en) * 2010-03-31 2018-10-24 Immersion Corporation System and method for providing haptic stimulus based on position
US11512919B2 (en) 2012-05-22 2022-11-29 Haptech, Inc. Methods and apparatuses for haptic systems
US10508883B2 (en) 2012-05-22 2019-12-17 Haptech, Inc. Method and apparatus for firearm recoil simulation
US10852093B2 (en) 2012-05-22 2020-12-01 Haptech, Inc. Methods and apparatuses for haptic systems
CN103522262A (en) * 2012-06-29 2014-01-22 罗伯特·博世有限公司 Cotrolling a battery-operated handheld power tool
US9245428B2 (en) 2012-08-02 2016-01-26 Immersion Corporation Systems and methods for haptic remote control gaming
US9753540B2 (en) 2012-08-02 2017-09-05 Immersion Corporation Systems and methods for haptic remote control gaming
WO2015187758A1 (en) * 2014-06-06 2015-12-10 Microsoft Technology Licensing, Llc Systems and methods for controlling feedback for multiple haptic zones
US11790739B2 (en) 2014-09-02 2023-10-17 Apple Inc. Semantic framework for variable haptic output
US10504340B2 (en) 2014-09-02 2019-12-10 Apple Inc. Semantic framework for variable haptic output
US10977911B2 (en) 2014-09-02 2021-04-13 Apple Inc. Semantic framework for variable haptic output
US10417879B2 (en) 2014-09-02 2019-09-17 Apple Inc. Semantic framework for variable haptic output
US9928699B2 (en) 2014-09-02 2018-03-27 Apple Inc. Semantic framework for variable haptic output
US9830784B2 (en) 2014-09-02 2017-11-28 Apple Inc. Semantic framework for variable haptic output
US10089840B2 (en) 2014-09-02 2018-10-02 Apple Inc. Semantic framework for variable haptic output
EP3766768A1 (en) * 2014-11-28 2021-01-20 Haptech, Inc. Methods and apparatuses for haptic systems
CN107077233A (en) * 2014-12-23 2017-08-18 意美森公司 Control the power distribution to haptic output devices
US10268286B2 (en) 2014-12-23 2019-04-23 Immersion Corporation Controlling power distribution to haptic output devices
EP3197577A4 (en) * 2014-12-23 2018-05-16 Immersion Corporation Controlling power distribution to haptic output devices
US10452170B2 (en) 2014-12-23 2019-10-22 Immersion Corporation Controlling power distribution to haptic output devices
US11167210B2 (en) 2015-04-21 2021-11-09 Nintendo Co., Ltd. Storage medium having stored therein vibration signal generation program, vibration signal generation system, vibration signal generation apparatus, vibration signal generation method, and storage medium having stored therein data output program
EP3088064A1 (en) * 2015-04-21 2016-11-02 Nintendo Co., Ltd. Vibration signal generation program, vibration signal generation system, vibration signal generation apparatus, vibration signal generation method, and data output program
US10156903B2 (en) 2016-06-12 2018-12-18 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US9984539B2 (en) 2016-06-12 2018-05-29 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US10276000B2 (en) 2016-06-12 2019-04-30 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US11735014B2 (en) 2016-06-12 2023-08-22 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US10175759B2 (en) 2016-06-12 2019-01-08 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US11468749B2 (en) 2016-06-12 2022-10-11 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US11379041B2 (en) 2016-06-12 2022-07-05 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US10139909B2 (en) 2016-06-12 2018-11-27 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US9996157B2 (en) 2016-06-12 2018-06-12 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US11037413B2 (en) 2016-06-12 2021-06-15 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US10692333B2 (en) 2016-06-12 2020-06-23 Apple Inc. Devices, methods, and graphical user interfaces for providing haptic feedback
US10671186B2 (en) 2016-06-15 2020-06-02 Microsoft Technology Licensing, Llc Autonomous haptic stylus
US10175762B2 (en) 2016-09-06 2019-01-08 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US9678571B1 (en) 2016-09-06 2017-06-13 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US10372221B2 (en) 2016-09-06 2019-08-06 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US11662824B2 (en) 2016-09-06 2023-05-30 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
DK201670725A1 (en) * 2016-09-06 2018-03-19 Apple Inc Devices, Methods, and Graphical User Interfaces for Generating Tactile Outputs
US10901513B2 (en) 2016-09-06 2021-01-26 Apple Inc. Devices, methods, and graphical user interfaces for haptic mixing
US10901514B2 (en) 2016-09-06 2021-01-26 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US9864432B1 (en) 2016-09-06 2018-01-09 Apple Inc. Devices, methods, and graphical user interfaces for haptic mixing
US9690383B1 (en) 2016-09-06 2017-06-27 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US10620708B2 (en) 2016-09-06 2020-04-14 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US11221679B2 (en) 2016-09-06 2022-01-11 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US9753541B1 (en) 2016-09-06 2017-09-05 Apple Inc. Devices, methods, and graphical user interfaces for generating tactile outputs
US10528139B2 (en) 2016-09-06 2020-01-07 Apple Inc. Devices, methods, and graphical user interfaces for haptic mixing
US11314330B2 (en) 2017-05-16 2022-04-26 Apple Inc. Tactile feedback for locked device user interfaces
EP3462282A1 (en) * 2017-09-27 2019-04-03 Vestel Elektronik Sanayi ve Ticaret A.S. Vibration-generating device and method of operation
WO2019204749A1 (en) * 2018-04-20 2019-10-24 Immersion Corporation Kinesthetic haptic feedback for haptically-enabled surfaces
CN110609604A (en) * 2018-06-15 2019-12-24 意美森公司 Vibration generation and braking
EP3582073A1 (en) * 2018-06-15 2019-12-18 Immersion Corporation Generation and braking of vibrations
CN112933590A (en) * 2021-03-31 2021-06-11 歌尔股份有限公司 Vibration control method of terminal device, terminal device and storage medium

Also Published As

Publication number Publication date
AU7725600A (en) 2001-04-30

Similar Documents

Publication Publication Date Title
US9492847B2 (en) Controlling haptic sensations for vibrotactile feedback interface devices
WO2001024158A1 (en) Controlling haptic sensations for vibrotactile feedback interface devices
EP1330811B1 (en) Directional tactile feedback for haptic feedback interface devices
US7979797B2 (en) Device having selective directional tactile feedback capability
US9411420B2 (en) Increasing force transmissibility for tactile feedback interface devices
US7084854B1 (en) Actuator for providing tactile sensations and device for directional tactile sensations
US7561142B2 (en) Vibrotactile haptic feedback devices
US6693622B1 (en) Vibrotactile haptic feedback devices
US6686901B2 (en) Enhancing inertial tactile feedback in computer interface devices having increased mass
JP5417663B2 (en) Tactile feedback using rotary harmonic motion mass
JP2003199974A6 (en) Directional contact feedback for haptic feedback interface devices
US20090295552A1 (en) Vibrotactile Haptic Feedback Devices
US20040183782A1 (en) Low-cost haptic mouse implementations
US20080316171A1 (en) Low-Cost Haptic Mouse Implementations
WO2001013354A9 (en) Vibrotactile haptic feedback devices
WO2001003105A9 (en) Controlling vibrotactile sensations for haptic feedback devices
WO2001003105A1 (en) Controlling vibrotactile sensations for haptic feedback devices

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DE DK DZ EE ES FI GB GE GH GM HR HU ID IL IS JP KE KG KP KR KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

ENP Entry into the national phase

Ref country code: JP

Ref document number: 2001 600023

Kind code of ref document: A

Format of ref document f/p: F

ENP Entry into the national phase

Ref country code: JP

Ref document number: 2001 600023

Kind code of ref document: A

Format of ref document f/p: F

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase