US20150175172A1 - Gesture based input system in a vehicle with haptic feedback - Google Patents
Gesture based input system in a vehicle with haptic feedback Download PDFInfo
- Publication number
- US20150175172A1 US20150175172A1 US14/137,540 US201314137540A US2015175172A1 US 20150175172 A1 US20150175172 A1 US 20150175172A1 US 201314137540 A US201314137540 A US 201314137540A US 2015175172 A1 US2015175172 A1 US 2015175172A1
- Authority
- US
- United States
- Prior art keywords
- vehicle
- gesture
- haptic
- driver
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W50/16—Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/10—Interpretation of driver requests or demands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
- G06F3/005—Input arrangements through a video camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
Definitions
- the present invention relates generally to gesture based input system for a vehicle, and more particularly to a gesture based input system for controlling vehicle systems and including haptic feedback to provide haptic effects to the user to confirm the gesture based inputs.
- Vehicles continue to add technology to assist, entertain, and/or provide a comfortable environment for occupants of the vehicle.
- navigation system using global positioning systems (GPS), environmental or climate control systems (such as HVAC controls), radios, CD players, auxiliary device players, video screens, DVD players, telephone systems, and other similar systems are frequently incorporated into vehicles.
- Controls for these systems are provided in various forms, such as, but not limited to, knobs, buttons, dials, and touch screens. More and more of the control systems are being incorporated into touch screens. Using these controls may require the driver to take his or her eyes off the road, which is dangerous. This may be particularly problematic with the increasing use of touch screen devices using virtual buttons and sliders displayed on a screen as an alternative to physical inputs.
- buttons, sliders, scrollers, and the like Users may interface with such devices almost exclusively by touching and/or otherwise manipulating the virtual buttons, sliders, scrollers, and the like on the screen with one or more finger(s). Because there often is no physical or “feel” distinction between virtual buttons, slides, etc., a user cannot “feel” for the location of a virtual button as was possible with physical buttons or knobs, for example. Further, graphic displays on the screen provide visual feedback responsive to the manipulation which further tempts the driver to look at the screen instead of the road.
- gesture based interfaces wherein a computer system can react to gestures or hand motions of a user.
- gestures based interfaces in a vehicle may not alleviate the problem of driver distraction as drivers may tend to look at a display screen or other device to ensure that a gesture based command is properly interpreted by the system.
- accidental activation of the systems needs to be avoided. Accordingly, it would be desirable to incorporate a gesture based input system into a vehicle and to provide feedback in a manner that a user such as the driver is not distracted from the primary task of driving the vehicle.
- Embodiments hereof are directed to systems for controlling vehicle systems including a haptic actuator, a gesture detection device, a controller, and a plurality of vehicle systems.
- the gesture detection device captures movements of a vehicle driver or passenger and outputs data representing a gesture made by the vehicle driver or passenger.
- the controller is coupled to the gesture detection device and receives a signal from the gesture detection device indicative of the gesture.
- the controller is coupled to the haptic actuator, and selects a haptic effect based on the detected gesture and sends a signal to the haptic actuator to provide the selected haptic effect.
- the controller is also coupled to the vehicle systems, and based on the detected gesture, the controller sends a signal to one of the vehicle systems to perform a particular action. The particular action of the vehicle system is matched with the selected haptic effect.
- Embodiments hereof are also directed to systems for controlling vehicle systems including a haptic actuator, a gesture detection device, an audio detection device, a controller, and a plurality of vehicle systems.
- the gesture detection device captures movements of a vehicle driver or passenger and outputs data representing a gesture made by the vehicle driver or passenger.
- the audio detection device detects voice commands from the vehicle driver or passenger and outputs data representing the voice command to the controller.
- the controller is coupled to the gesture detection device and the audio detection device and receives signals from the gesture detection device indicative of the gesture and signals from the audio detection device indicative of the voice command.
- the controller is coupled to the haptic actuator, and selects a haptic effect based on the detected gesture or detected voice command and sends a signal to the haptic actuator to provide the selected haptic effect.
- the controller is also coupled to the vehicle systems, and based on the detected gesture or detected voice command, the controller sends a signal to one of the vehicle systems to perform a particular action. The particular action of the vehicle system is matched with the selected haptic effect.
- FIG. 1 is schematic illustration of a vehicle with various systems illustrating an embodiment of a gesture based input system and haptic feedback system hereof.
- FIG. 2 is a schematic illustration of one of the seats of the vehicle of FIG. 1 .
- FIG. 3 is a schematic illustration of the center console and armrest between the driver seat and front passenger seat of the vehicle of FIG. 1
- FIG. 4 is a block diagram of an embodiment of a gesture based input and haptic feedback system hereof.
- FIG. 5 is a schematic illustration of a display device of a vehicle.
- FIG. 6 is a block diagram of an embodiment of a gesture and audio based input and haptic feedback system hereof.
- FIG. 1 is a schematic illustration of an interior of a vehicle V.
- Vehicle V includes a driver seat 10 including a head rest 12 , a back rest 14 , and a seat base 16 .
- a passenger side front seat 20 similarly includes a head rest 22 , a back rest 24 , and a seat base 26 .
- a driver side rear seat 30 includes a head rest 32 , a back rest 34 , and a seat base 36 , and a passenger side rear seat 40 includes a head rest 42 , a back rest 44 , and a seat base 46 .
- vehicle V can include more seats (as shown in FIG. 1 ) or fewer seats.
- Vehicle V also includes a steering wheel 18 , and an arm rest 28 disposed between driver seat 10 and passenger side front seat 20 , as known in the art.
- Vehicle V may also include an electronic control and display system 50 disposed in the front dash between driver seat 10 and passenger side front seat 20 .
- Electronic control and display system 50 may be similar to a conventional control and display system such that it can be used to control and display functions of systems such as, but not limited to, a radio, a navigation system, a vehicle climate control system, blue-tooth systems, auxiliary systems such as DVD players, CD players and cellular phones, information systems, and other systems known to those skilled in the art.
- a driver side rear control and display system 38 may be attached to driver seat head rest 12 and face driver side rear seat 30 .
- a passenger side rear control and display system 48 may be attached to passenger side front seat head rest 22 and face passenger side rear seat 40 .
- Control and display systems 38 , 48 may include conventional control and display systems to control and display functions of systems such as, but not limited to, rear climate systems, rear entertainment systems such as DVD players and CD players, blue tooth systems, and other systems known to those skilled in the art.
- Control and display system 50 may include a camera or other gesture input detector 60 configured to detect movements of a driver and/or a passenger.
- control and display systems 38 , 48 may include cameras or other gesture input detectors 39 , 49 , respectively, to detect movements of passengers in rear seats 30 , 40 , as explained in more detail below.
- a camera 60 , 39 , 49 is shown associated with each control and display system 50 , 38 , 48 , the present disclosure is not limited to such an embodiment. Instead, any manner in which the gestures of the driver and passengers may be detected and used as described in detail below may be used. For example, and not by way of limitation, several cameras located in the interior of vehicle V may track movements of the entire vehicle interior.
- Such a group of cameras can detect location, orientation, and movements of each of the passengers, including such details as movements of the passengers' hands and fingers.
- Cameras used in the present embodiment may be motion capture cameras used for optical detection.
- other types of camera or detection systems may be used, such as, but not limited to, infra-red, RFID, and electromagnetic.
- systems such as those used for motion detection and processing in games such a XBOX KinectTM or similar systems may be used.
- FIG. 2 shows further details of driver seat 10 of FIG. 1 .
- driver seat 10 may include one or more haptic actuators 110 in portions of driver seat 10 .
- haptic actuators 110 are incorporated into head rest 12 , back rest 14 , and seat base 16 .
- haptic actuators 110 may be incorporated in more or fewer locations in driver seat 10 .
- Haptic actuators 110 are used in conjunction with a gesture based input system, as described in more detail below. Further, while FIG.
- haptic actuators 110 may also be incorporated into passenger seat 20 , driver side rear seat 30 , and passenger side rear seat 40 , and, more particularly, in their respect head rests, 22 , 32 , 42 , back rests, 24 , 34 , 44 , and seat bases 26 , 36 , 46 .
- FIG. 3 shows a portion of arm rest 28 and center console 29 disposed between driver seat 10 and passenger seat 20 .
- Arm rest 28 may include a haptic actuator 110 incorporated therein.
- center console 29 may include a camera or other gesture input detector 62 .
- a gesture input detector 62 In the situation where a gesture input detector 62 is used in center console 29 , it may take the place of the camera 60 .
- camera or gesture input detector 62 may include a limited field of view such that user needs to perform the gestures in the vicinity of camera 62 . Such a situation may be desirable such that inadvertent movements of a user are not detected as gestures to control the vehicle systems.
- separating the camera or gesture input detector 62 from the display system 50 psychologically trains the user to separate the gestures from looking at the display. Thus, a driver is more likely to keep his or her eyes on the road instead of viewing display 50 , which will also be aided by the haptic feedback system described in more detail below.
- Haptic actuators 110 may be provided in other common contact surfaces of vehicle V.
- steering wheel 18 may include haptic actuators 110 , as shown in FIG. 1 .
- other surfaces such as door arm rests, arm rests between rear passenger seats, seat belts, and floors of the vehicle may include haptic actuators 110 .
- haptic actuators may be provided in objects or items that are wearable by the driver or passengers of vehicle V.
- haptic actuators 110 may be provided in gloves, hats, necklaces, wrist bands or wristlets, and other similar items that the driver or passenger may wear such that the haptic effect is felt directly by the driver or passenger.
- haptic actuators 110 may be vibrotactile devices that generate vibrations on the object in which actuator 110 is located, for example, within portions of seat 10 , 20 , 30 , 40 , on arm rest 28 , steering wheel 18 , or the wearable object, etc.
- Other types of haptic feedback may be provided.
- haptic actuators 110 may provide electrostatic friction or ultrasonic friction to the user.
- Other types of haptic feedback may be generated and provided to the user, including kinesthetic feedback (e.g., active and resistive force feedback), deformation, and/or other types of tactile feedback such as texture. As shown in FIGS.
- Haptic actuators 110 may include electromagnetic motors, eccentric rotating mass (“ERM”) actuators in which an eccentric mass is moved by a motor, linear resonant actuators (“LRAs”) in which a mass attached to a spring is driven back and forth, shape memory alloys, electro-active polymers that deform in response to signals, mechanisms for changing stiffness, vibrotactile actuators, inertial actuators, piezoelectric actuators, moving magnet actuators, or other suitable types of actuating devices.
- ECM eccentric rotating mass
- LRAs linear resonant actuators
- FIG. 4 shows a block diagram of an embodiment of a gesture based input and haptic feedback system 100 and a flow chart for using the system 100 .
- gesture input block 102 represents a gesture by a user of the vehicle, such as the driver. It is contemplated that such gestures are hand and/or finger gestures by the user. Other gestures may also be used.
- camera or other gesture detector 60 detects the gesture by the user. As explained above, gesture detector 60 may be a single camera, multiple cameras, or other devices configured to detect location, orientation, and movements of the user.
- the camera or gesture detector 60 When the camera or gesture detector 60 detects a gesture, it sends a signal to a gesture input controller/translator 104 .
- Gesture input controller/translator 104 converts the camera signal into a gesture signal and transmits the signal to a computer/controller 106 .
- camera 60 may provide data in the form of location of the detected hands/fingers, speed/direction of motion, and other data regarding position, orientation, and movement of a users hands/fingers.
- Gesture input controller/translator 104 converts this data into a particular gesture from a table of potential gestures.
- possible gestures may include a hand wave in particular directions, finger movement left, right, up, or down, hand/finger rotation, finger pinch/unpinch, rapid waving, finger movement towards camera, and other gestures known to those skilled in the art.
- sophisticated gesture detection systems may detect individual fingers such that movement by the pointer finger may be distinguished by movement from movement by the pinkie finger, for example.
- an interactive haptic feedback system as described in more detail below, allows for a system which does not need to recognize or distinguish between minor variations in movements. Accordingly, a less costly gesture recognition system can be used in combination with the interactive haptic feedback system.
- gesture input controller/translator 104 converts the data
- the gesture signal is sent to a computer/controller 106 .
- a computer/controller 106 may be combined with computer/controller 106 .
- computer/controller 106 processes the data to provide signals to three different devices.
- computer/controller 106 matches the gesture signal to a haptic effect that is selected from a haptic table 108 .
- the haptic effect is generally a vibratory effect
- a particular vibratory effect in terms of degree, magnitude, frequency, and/or duration, is selected.
- the computer/controller 106 selects to which user the haptic effect will be delivered.
- different haptic actuators 110 may be selected for an individual user.
- activation of a haptic actuator 110 in head rest 12 of driver seat 10 may have a different feedback meaning that activation of a haptic actuator 110 in seat base 16 of driver seat 10 .
- the computer/controller 106 selects to which haptic actuator 110 a signal will be sent.
- computer/controller 106 selects a particular haptic effect from haptic table 108 and a particular haptic actuator 110 to which the effect will be applied. A signal for the selected haptic effect is then sent to the selected haptic actuator 110 .
- computer/controller 106 sends a signal to display 50 to show the action being taken. For example, and not by way of limitation, if a user selects an icon (e.g., radio, navigation, climate, etc.), computer/controller 106 sends a signal to display 50 for the selected icon to be highlighted. Further, any sub-menus associated with the selected icon will be displayed. For example, and not by way of limitation, FIG. 5 shows an example display with a main menu 52 and a sub menu 54 . As described in more detail below, a gesture may be interpreted as selecting the climate control icon on main menu 52 , as shown in FIG. 5 .
- an icon e.g., radio, navigation, climate, etc.
- FIG. 5 shows an example display with a main menu 52 and a sub menu 54 .
- a gesture may be interpreted as selecting the climate control icon on main menu 52 , as shown in FIG. 5 .
- computer/controller 106 sends a signal to display to highlight the climate control icon, as shown in FIG. 5 .
- the sub-menu 54 associate with the climate controls appears.
- a sub-menu associated with that main menu will appear.
- computer/controller 106 also sends a signal to the vehicle system 112 to perform the selected control. For example, and not by way of limitation, upon selecting the climate icon of main menu 52 , if a gesture to increase the driver side temperature is recognized by the camera 60 and processed as described above, computer/controller 106 sends a signal to the climate control system of the vehicle to increase the temperature air blown through the driver side vents, for example. In another example, if the radio icon is selected from main menu 52 and a particular gesture for increasing the volume is recognized, computer/controller 106 sends a signal to the radio control system 112 to increase the volume.
- Such signals will be sent to other vehicle control systems 112 , such as for a CD player, navigation system, phone, DVD player, gaming system, or other vehicle system or sub-system, or auxiliary system coupled to a vehicle system.
- vehicle control systems 112 such as for a CD player, navigation system, phone, DVD player, gaming system, or other vehicle system or sub-system, or auxiliary system coupled to a vehicle system.
- display 50 is one of the vehicle systems and need not be identified separately. It is identified separately in this description for convenience because some actions result only in action on the display, such as highlighting an icon, whereas other actions result in action either only on the other vehicle systems, or in both the other vehicle systems and the display.
- the haptic feedback described above can be used confirm or describe actions that are being taken based on the gesture based input.
- Table 1 summarized in Table 1 below and described with respect to a system such as the system shown in FIG. 5 , a driver of a vehicle V makes gestures that are monitored by camera 60 .
- the example below is described with respect to a driver making gestures and particular haptic effects described below being felt by the driver, those skilled in the art will recognize that similar gestures may be made by passengers in the vehicle, that such gestures may be detected by camera 60 or other cameras, and that the haptic effect may be provided to the passenger making the gesture.
- haptic effects may be desirable for passengers to enhance their experience when interacting with entertainment systems in the vehicle, such as gaming systems or video systems.
- the driver waves his or her hand/finger to wake up the system. Accordingly, computer/controller 106 sends a signal to select the haptic effect of a short vibration from table 108 and sends it to a haptic actuator 110 associated with the driver, such as in driver seat 10 .
- the driver may begin moving a finger or hand to track across main menu 52 .
- a light continuous vibration alerts the drive that a pointer on display 50 is moving across the screen.
- the pointer reaches a selectable area on main menu 52 , such as the climate icon, the intensity of the vibration is raised to be a medium continuous vibration.
- the driver can select the icon by pinching his or her fingers.
- Computer/controller 106 upon recognizing the gesture, will send a signal to select multiple short vibrations from the haptic effect table 108 and send a signal for such vibrations to a haptic actuator 110 associated with the driver. Simultaneously, a signal is sent to display 50 to highlight the climate icon and to display the climate sub-menu options on sub-menu 54 , as shown in FIG. 5 .
- the haptic effect allows the user to know that the icon has been selected. In order to further avoid the need for the driver to look at display 50 , when the driver activates the system, display 50 may always start at a certain icon. This allows the user to become accustomed to the order in which the icons are selected.
- each icon can be designated with its own haptic effect, such as by the particular effect or the location the effect is felt by the driver.
- the haptic effect may be delivered via a haptic actuator 110 in the head rest 12 of driver seat 10
- the haptic effect may be delivered via a haptic actuator in the seat base 14 of driver seat 10 .
- the number of systems controlled other variations would likely be necessary.
- each icon associated with sound may be associated with a haptic effect delivered to haptic actuator 110 in head rest 12 , with each having its own haptic effect, such as one vibration for the radio, two for the CD player, and three for the phone.
- haptic actuator 110 in head rest 12
- each having its own haptic effect such as one vibration for the radio, two for the CD player, and three for the phone.
- selection of the climate icon may result in one vibration to the seat base 12
- selection of the navigation icon may result in two vibrations to the seat base 14 .
- variation in the haptic effect could be used to distinguish between the icons.
- the driver may make further gestures to control the selected system as displayed on sub-menu 54 .
- the driver may move a finger to select items along the top bar of sub menu 54 in a similar fashion as described above with respect to main menu 52 , and the appropriate haptic effect is provided.
- the driver may rotate his or her hand clockwise to increase the temperature and counter-clockwise to decrease the temperature, similar to a rotary dial or knob.
- computer/controller 106 sends a signal to select the haptic effect of a continuous vibration with increasing magnitude from haptic effect table 108 and the signal for such an effect is sent to a haptic actuator 110 associated with the driver, who feels the effect. Simultaneously, a signal is sent to display 50 to highlight the up arrow in the temperature slide and to increase the temperature setting displayed. Also, a signal is sent to the vehicle climate control system to set the temperature to the selected temperature.
- FIG. 6 shows an embodiment wherein gesture based input and audio input are combined, with haptic effect feedback used to confirm commands, as in the embodiment above.
- FIG. 6 shows a block diagram of an embodiment of a gesture and audio based input and feedback system 200 and a flow chart for using the system 200 .
- gesture input block 202 represents a gesture by a user of the vehicle, such as the driver. It is contemplated that such gestures are hand and/or finger gestures by the user. Other gestures may also be used. As the gesture is being made by the user, camera of other gesture detector 60 detects the gesture by the user. As explained above, gesture detector 60 may be a single camera, multiple cameras, or other devices configured to detect location, orientation, and movements of the user.
- voice/audio input block 214 represents a sound or voice command made by a user of the vehicle, such as the driver. It is contemplated that such voice commands will be simple voice commands, for example, the simple voice commands shown in Table 2 below. Other voice commands may also be used.
- An audio input detector or microphone 64 detects the audio/voice command. Microphone 64 may be located adjacent to camera 60 , as shown in FIG. 1 , or elsewhere in vehicle V, as known to those skilled in the art.
- the camera or gesture detector 60 When the camera or gesture detector 60 detects a gesture, it sends a signal to a gesture input controller/translator 204 .
- Gesture input controller/translator 204 converts the camera signal into a gesture signal and transmits the signal to a computer/controller 206 .
- camera 60 may provide data in the form of location of the detected hand/finger, speed/direction of motion, and other data regarding position, orientation, and movement of a users hand/finger.
- Gesture input controller/translator 204 converts this data into a particular gesture from a table of potential gestures.
- possible gestures may include a hand wave in particular directions, finger movement left, right, up, or down, hand/finger rotation, finger pinch/unpinch, rapid waving, finger movement towards or away from the camera, and other gestures known to those skilled in the art.
- sophisticated gesture detection systems may detect individual fingers such that movement by the pointer finger may be distinguished by movement from movement by the pinkie finger, for example.
- an interactive haptic feedback system as described in more detail below, allows for a system which does not need to recognize or distinguish between minor variations in movements. Accordingly, a less costly gesture recognition system can be used in combination with the interactive haptic feedback system.
- Audio input controller/translator 218 converts the voice signal into a data signal representative of the voice command and transmits the signal to computer/controller 206 .
- gesture input controller/translator 204 and/or audio input controller/translator converts 218 the data
- the signal is sent to a computer/controller 206 .
- a computer/controller 206 may be combined without departing from the present invention.
- computer/controller 206 when computer/controller 206 receives a gesture or audio signal, computer/controller 206 processes the data to provide signals to three different devices.
- computer/controller 206 matches the gesture or audio signal to a haptic effect that is selected from a haptic table 208 .
- the computer/controller 206 selects to which user the haptic effect will be delivered.
- different haptic actuators 110 may be selected for an individual user.
- activation of a haptic actuator 110 in head rest 12 of driver seat 10 may have a different feedback meaning than activation of a haptic actuator 110 in seat base 16 of driver seat 10 .
- the computer/controller 206 selects to which haptic actuator 110 a signal will be sent.
- computer/controller 206 selects a particular haptic effect from haptic table 208 and a particular haptic actuator 110 to which the effect will be applied.
- a signal for the selected haptic effect is then sent to the selected haptic actuator 110 .
- computer/controller 206 sends a signal to display 50 to show the action being taken. For example, and not by way of limitation, if a user selects an icon (e.g., radio, navigation, climate, etc.), computer/controller 206 sends a signal to display 50 for the selected icon to be highlighted. Further, any sub-menus associated with the selected icon will be displayed. For example, and not by way of limitation, FIG. 5 shows an example display with a main menu 52 and a sub-menu 54 . If the climate control icon is selected on main menu 52 , such as by using a gesture or voice command, computer/controller 206 sends a signal to display 50 to highlight the climate icon, as shown in FIG. 5 . With the climate icon selected, the sub-menu 54 associated with the climate control appears. Similarly, if any of the other icons on the main menu 52 are selected, a sub-menu associated with that main menu will appear.
- an icon e.g., radio, navigation, climate, etc.
- FIG. 5 shows an
- computer/controller 206 also sends a signal to the vehicle system 212 to perform the selected control. For example, and not by way of limitation, upon selecting the climate icon of main menu 52 , if a gesture or audio command is used to increase the driver side temperature is recognized by the camera 60 /microphone 64 and processed as described above, computer/controller 206 sends a signal to the climate control system of the vehicle to increase the temperature air blown through the driver side vents, for example. In another example, if the radio icon is selected from main menu 52 and a particular gesture or audio command is given for increasing the volume, computer/controller 206 sends a signal to the radio control system 212 to increase the volume.
- Such signals will be sent to other vehicle control system 212 , such as for a CD player, navigation system, phone, DVD player, gaming system, or other vehicle system or sub-system, or auxiliary system coupled to a vehicle system.
- vehicle control system 212 such as for a CD player, navigation system, phone, DVD player, gaming system, or other vehicle system or sub-system, or auxiliary system coupled to a vehicle system.
- display 50 is one of the vehicle systems and need not be identified separately. It is identified separately in this description for convenience because some actions result only in action on the display, such as highlighting an icon, whereas other actions result in action either only on the other vehicle systems, or in both the other vehicle systems and the display.
- the haptic feedback described above can be used confirm or describe actions that are being taken based on the gesture based input.
- Table 2 summarized in Table 2 below and described with respect to a system such as the system shown in FIG. 5 , a driver of a vehicle V makes gestures that are monitored by camera 60 and gives voice commands that are received by microphone 64 .
- the example below is described with respect to a driver making gestures and giving voice commands, and particular haptic effects described below, those skilled in the art will recognize that similar gestures and voice commands may be made by passengers in the vehicle, that such gestures and voice commands may be detected by camera 60 /microphone 64 or other cameras/microphones, and that the haptic effect may be provided to the user making the gesture/giving the voice command.
- haptic effects may be desirable for passengers to enhance their experience when interacting with entertainment systems in the vehicle, such as gaming systems or video systems.
- Table 2 shows various hand gestures and voice commands to control various vehicle system, and haptic effects associated with such actions.
- Table 2 will be explained with respect to the exemplary display 50 show in FIG. 5 .
- the driver may wave a hand or finger to activate the system, as described above with respect to Table 1.
- computer/controller 206 sends a signal to select the haptic effect of a short vibration from table 208 and sends it to a haptic actuator 110 associated with the driver, such as in driver seat 10 .
- the driver may begin moving a finger or hand to track across main menu 52 .
- a light continuous vibration alerts the drive when the drivers hand/finger is over an icon in main menu 52 , as shown in Table 2 under “Basic Navigation”.
- the driver can then select the icon by giving a voice command, such as “select” or “yes”, as shown in Table 2.
- Computer/controller 206 upon recognizing the voice command, will send a signal to select a short vibration from the haptic effect table and send a signal for such a vibration to a haptic actuator 110 associated with the driver. Simultaneously, a signal is sent to display 50 to highlight the climate icon and to display the climate sub-menu options on sub-menu 54 , as shown in FIG. 5 .
- the haptic effect allows the driver to know that the icon has been selected.
- several options for haptic feedback may be used to ensure that the driver is aware that he or she has selected the desired icon, such as different haptic effects associate with each vehicle system on main menu 52 .
- the driver may make further gestures to control the selected system as displayed on sub-menu 54 .
- the driver may move a finger to select items along the top bar of sub-menu 54 in a similar fashion as described above with respect to main menu 52 , and the appropriate haptic effect is provided.
- the driver may move a hand/finger over the icons along top bar of sub menu 54 in a similar fashion as described above with respect to main menu 52 , and the appropriate haptic effect is provided.
- the driver moves a hand/finger over the icons along top bar of sub menu 54 , the “Basic Navigation” features of Table 2 are used.
- haptic actuator 110 if the driver's hand/finger hovers over the “Auto” icon, a light continuous vibration is felt from haptic actuator 110 . Further, the driver may select the “Auto” icon by voice command of “select” or “yes”, as described above. Upon recognition of the voice command, computer/controller 206 sends a signal to haptic actuator 110 for a short vibration and a signal to display 50 to highlight the “Auto” icon.
- the driver can then perform other activities in the sub menu. For example, the driver's fingers can be moved over the driver side temperature reading of 72 in FIG. 5 . A haptic effect such as a light continuous vibration confirms to the driver that the temperature is selected. The driver can then change the value of the driver's side temperature by voice command, such as “74”, as shown in the “Change value, direct input, text input” row of Table 2. Accordingly, computer/controller 206 sends a signal to haptic actuator 110 for a short vibration, a signal to display 50 to change the displayed driver's side temperature setting from “72” to “74”, and a signal to the climate control system to have the vehicle HVAC system provide a 74 degree temperature to the driver's side.
- voice command such as “74”
- computer/controller 206 sends a signal to haptic actuator 110 for a short vibration, a signal to display 50 to change the displayed driver's side temperature setting from “72” to “74”, and a signal to the climate control system to have the vehicle HVAC system
- Table 2 shows how the gesture and voice inputs and haptic effect feedback may be used in combination to make selection and control of various vehicle functions relatively simple without the need for the driver to look at display 50 . While a particular sub-menu selection was described above, those skilled in the art would recognize that the other vehicle systems may be controlled as described in Table 2, such as by selecting the radio icon on main menu 52 and using a voice command to change the station or raise and lower the volume, with the corresponding haptic effects shown in Table 2 providing feedback that the command has been processed y computer/controller 206 .
Abstract
Description
- The present invention relates generally to gesture based input system for a vehicle, and more particularly to a gesture based input system for controlling vehicle systems and including haptic feedback to provide haptic effects to the user to confirm the gesture based inputs.
- Vehicles continue to add technology to assist, entertain, and/or provide a comfortable environment for occupants of the vehicle. For example, and not by way of limitation, navigation system using global positioning systems (GPS), environmental or climate control systems (such as HVAC controls), radios, CD players, auxiliary device players, video screens, DVD players, telephone systems, and other similar systems are frequently incorporated into vehicles. Controls for these systems are provided in various forms, such as, but not limited to, knobs, buttons, dials, and touch screens. More and more of the control systems are being incorporated into touch screens. Using these controls may require the driver to take his or her eyes off the road, which is dangerous. This may be particularly problematic with the increasing use of touch screen devices using virtual buttons and sliders displayed on a screen as an alternative to physical inputs. Users may interface with such devices almost exclusively by touching and/or otherwise manipulating the virtual buttons, sliders, scrollers, and the like on the screen with one or more finger(s). Because there often is no physical or “feel” distinction between virtual buttons, slides, etc., a user cannot “feel” for the location of a virtual button as was possible with physical buttons or knobs, for example. Further, graphic displays on the screen provide visual feedback responsive to the manipulation which further tempts the driver to look at the screen instead of the road.
- Recent developments have allowed for voice-controlled systems for some of the vehicle systems. However, voice-controlled systems disrupt passengers in the vehicle, especially in situations where the microphone for receiving the voice command does not accurately receive or understand the voice command. This results in repetition of the command, and often causes the user to shout the command, thereby disturbing other passengers in the vehicle. Further, ambient noise such as the vehicle's entertainment system and road noise may interfere with the microphone receiving the command.
- There has been some recent development in gesture based interfaces wherein a computer system can react to gestures or hand motions of a user. However, gestures based interfaces in a vehicle may not alleviate the problem of driver distraction as drivers may tend to look at a display screen or other device to ensure that a gesture based command is properly interpreted by the system. Further, accidental activation of the systems needs to be avoided. Accordingly, it would be desirable to incorporate a gesture based input system into a vehicle and to provide feedback in a manner that a user such as the driver is not distracted from the primary task of driving the vehicle.
- Embodiments hereof are directed to systems for controlling vehicle systems including a haptic actuator, a gesture detection device, a controller, and a plurality of vehicle systems. The gesture detection device captures movements of a vehicle driver or passenger and outputs data representing a gesture made by the vehicle driver or passenger. The controller is coupled to the gesture detection device and receives a signal from the gesture detection device indicative of the gesture. The controller is coupled to the haptic actuator, and selects a haptic effect based on the detected gesture and sends a signal to the haptic actuator to provide the selected haptic effect. The controller is also coupled to the vehicle systems, and based on the detected gesture, the controller sends a signal to one of the vehicle systems to perform a particular action. The particular action of the vehicle system is matched with the selected haptic effect.
- Embodiments hereof are also directed to systems for controlling vehicle systems including a haptic actuator, a gesture detection device, an audio detection device, a controller, and a plurality of vehicle systems. The gesture detection device captures movements of a vehicle driver or passenger and outputs data representing a gesture made by the vehicle driver or passenger. The audio detection device detects voice commands from the vehicle driver or passenger and outputs data representing the voice command to the controller. The controller is coupled to the gesture detection device and the audio detection device and receives signals from the gesture detection device indicative of the gesture and signals from the audio detection device indicative of the voice command. The controller is coupled to the haptic actuator, and selects a haptic effect based on the detected gesture or detected voice command and sends a signal to the haptic actuator to provide the selected haptic effect. The controller is also coupled to the vehicle systems, and based on the detected gesture or detected voice command, the controller sends a signal to one of the vehicle systems to perform a particular action. The particular action of the vehicle system is matched with the selected haptic effect.
- The foregoing and other features and advantages of the invention will be apparent from the following description of embodiments thereof as illustrated in the accompanying drawings. The accompanying drawings, which are incorporated herein and form a part of the specification, further serve to explain the principles of the invention and to enable a person skilled in the pertinent art to make and use the invention. The drawings are not to scale.
-
FIG. 1 is schematic illustration of a vehicle with various systems illustrating an embodiment of a gesture based input system and haptic feedback system hereof. -
FIG. 2 is a schematic illustration of one of the seats of the vehicle ofFIG. 1 . -
FIG. 3 is a schematic illustration of the center console and armrest between the driver seat and front passenger seat of the vehicle ofFIG. 1 -
FIG. 4 is a block diagram of an embodiment of a gesture based input and haptic feedback system hereof. -
FIG. 5 is a schematic illustration of a display device of a vehicle. -
FIG. 6 is a block diagram of an embodiment of a gesture and audio based input and haptic feedback system hereof. - Specific embodiments of the present invention are now described with reference to the figures, wherein like reference numbers indicate identical or functionally similar elements. The following detailed description is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Although descriptions of embodiments hereof are in the context of a gesture based input and haptic feedback system in a vehicle, the invention may also be used in any other applications where it is deemed useful. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or the following detailed description.
-
FIG. 1 is a schematic illustration of an interior of a vehicle V. Several typical features of a vehicle interior will be described with respect toFIGS. 1-3 . Vehicle V includes adriver seat 10 including ahead rest 12, aback rest 14, and aseat base 16. A passengerside front seat 20 similarly includes ahead rest 22, aback rest 24, and aseat base 26. A driverside rear seat 30 includes ahead rest 32, aback rest 34, and aseat base 36, and a passenger siderear seat 40 includes ahead rest 42, a back rest 44, and aseat base 46. Those skilled in the art would recognize that vehicle V can include more seats (as shown inFIG. 1 ) or fewer seats. Further, all seats do not need to include all the features described herein. For example, and not by way of limitation, head rests need not be provided. Vehicle V also includes asteering wheel 18, and anarm rest 28 disposed betweendriver seat 10 and passengerside front seat 20, as known in the art. - Vehicle V may also include an electronic control and
display system 50 disposed in the front dash betweendriver seat 10 and passengerside front seat 20. Electronic control anddisplay system 50 may be similar to a conventional control and display system such that it can be used to control and display functions of systems such as, but not limited to, a radio, a navigation system, a vehicle climate control system, blue-tooth systems, auxiliary systems such as DVD players, CD players and cellular phones, information systems, and other systems known to those skilled in the art. A driver side rear control anddisplay system 38 may be attached to driverseat head rest 12 and face driver siderear seat 30. Similarly, a passenger side rear control anddisplay system 48 may be attached to passenger side frontseat head rest 22 and face passenger siderear seat 40. Control anddisplay systems - Control and
display system 50 may include a camera or othergesture input detector 60 configured to detect movements of a driver and/or a passenger. Similarly, control anddisplay systems gesture input detectors rear seats camera display system -
FIG. 2 shows further details ofdriver seat 10 ofFIG. 1 . In particular,driver seat 10 may include one or morehaptic actuators 110 in portions ofdriver seat 10. In the embodiment shown inFIG. 2 ,haptic actuators 110 are incorporated intohead rest 12, backrest 14, andseat base 16. However, as would be understood by those skill in the art,haptic actuators 110 may be incorporated in more or fewer locations indriver seat 10.Haptic actuators 110 are used in conjunction with a gesture based input system, as described in more detail below. Further, whileFIG. 2 showshaptic actuators 110 incorporated intodriver seat 10, haptic actuators may also be incorporated intopassenger seat 20, driver siderear seat 30, and passenger siderear seat 40, and, more particularly, in their respect head rests, 22, 32, 42, back rests, 24, 34, 44, andseat bases -
FIG. 3 shows a portion ofarm rest 28 andcenter console 29 disposed betweendriver seat 10 andpassenger seat 20.Arm rest 28 may include ahaptic actuator 110 incorporated therein. Further,center console 29 may include a camera or othergesture input detector 62. In the situation where agesture input detector 62 is used incenter console 29, it may take the place of thecamera 60. Further, camera orgesture input detector 62 may include a limited field of view such that user needs to perform the gestures in the vicinity ofcamera 62. Such a situation may be desirable such that inadvertent movements of a user are not detected as gestures to control the vehicle systems. Also, separating the camera orgesture input detector 62 from thedisplay system 50 psychologically trains the user to separate the gestures from looking at the display. Thus, a driver is more likely to keep his or her eyes on the road instead of viewingdisplay 50, which will also be aided by the haptic feedback system described in more detail below. -
Haptic actuators 110 may be provided in other common contact surfaces of vehicle V. For example, and not by way of limitation,steering wheel 18 may includehaptic actuators 110, as shown inFIG. 1 . Further, other surfaces, such as door arm rests, arm rests between rear passenger seats, seat belts, and floors of the vehicle may includehaptic actuators 110. Further, haptic actuators may be provided in objects or items that are wearable by the driver or passengers of vehicle V. For example, and not by way of limitation,haptic actuators 110 may be provided in gloves, hats, necklaces, wrist bands or wristlets, and other similar items that the driver or passenger may wear such that the haptic effect is felt directly by the driver or passenger. - In an embodiment,
haptic actuators 110 may be vibrotactile devices that generate vibrations on the object in which actuator 110 is located, for example, within portions ofseat arm rest 28,steering wheel 18, or the wearable object, etc. Other types of haptic feedback may be provided. For example, and not by way of limitation,haptic actuators 110 may provide electrostatic friction or ultrasonic friction to the user. Other types of haptic feedback may be generated and provided to the user, including kinesthetic feedback (e.g., active and resistive force feedback), deformation, and/or other types of tactile feedback such as texture. As shown inFIGS. 1-3 , multiple haptic actuators may be incorporated in different areas of vehicle V for providing haptic effects to different users or to signify different events, as explained in more detail below.Haptic actuators 110 may include electromagnetic motors, eccentric rotating mass (“ERM”) actuators in which an eccentric mass is moved by a motor, linear resonant actuators (“LRAs”) in which a mass attached to a spring is driven back and forth, shape memory alloys, electro-active polymers that deform in response to signals, mechanisms for changing stiffness, vibrotactile actuators, inertial actuators, piezoelectric actuators, moving magnet actuators, or other suitable types of actuating devices. -
FIG. 4 shows a block diagram of an embodiment of a gesture based input andhaptic feedback system 100 and a flow chart for using thesystem 100. In particular,gesture input block 102 represents a gesture by a user of the vehicle, such as the driver. It is contemplated that such gestures are hand and/or finger gestures by the user. Other gestures may also be used. As the gesture is being made by the user, camera orother gesture detector 60 detects the gesture by the user. As explained above,gesture detector 60 may be a single camera, multiple cameras, or other devices configured to detect location, orientation, and movements of the user. - When the camera or
gesture detector 60 detects a gesture, it sends a signal to a gesture input controller/translator 104. Gesture input controller/translator 104 converts the camera signal into a gesture signal and transmits the signal to a computer/controller 106. For example, and not by way of limitation,camera 60 may provide data in the form of location of the detected hands/fingers, speed/direction of motion, and other data regarding position, orientation, and movement of a users hands/fingers. Gesture input controller/translator 104 converts this data into a particular gesture from a table of potential gestures. For example, and not by way of limitation, possible gestures may include a hand wave in particular directions, finger movement left, right, up, or down, hand/finger rotation, finger pinch/unpinch, rapid waving, finger movement towards camera, and other gestures known to those skilled in the art. If desired, sophisticated gesture detection systems may detect individual fingers such that movement by the pointer finger may be distinguished by movement from movement by the pinkie finger, for example. However, the use of an interactive haptic feedback system, as described in more detail below, allows for a system which does not need to recognize or distinguish between minor variations in movements. Accordingly, a less costly gesture recognition system can be used in combination with the interactive haptic feedback system. - After the gesture input controller/
translator 104 converts the data, the gesture signal is sent to a computer/controller 106. Those skilled in the art would recognize that gesture input controller/translator 104 may be combined with computer/controller 106. In the embodiment shown inFIG. 4 , when computer/controller 106 receives a gesture signal, computer/controller 106 processes the data to provide signals to three different devices. In particular, computer/controller 106 matches the gesture signal to a haptic effect that is selected from a haptic table 108. For example, and not by way of limitation, if the haptic effect is generally a vibratory effect, a particular vibratory effect, in terms of degree, magnitude, frequency, and/or duration, is selected. Further, if the gesture based input system allows for input from multiple users, the computer/controller 106 selects to which user the haptic effect will be delivered. Further, in an embodiment, differenthaptic actuators 110 may be selected for an individual user. For example, and not by way of limitation, activation of ahaptic actuator 110 inhead rest 12 ofdriver seat 10 may have a different feedback meaning that activation of ahaptic actuator 110 inseat base 16 ofdriver seat 10. In essence, the computer/controller 106 selects to which haptic actuator 110 a signal will be sent. Thus, computer/controller 106 selects a particular haptic effect from haptic table 108 and a particularhaptic actuator 110 to which the effect will be applied. A signal for the selected haptic effect is then sent to the selectedhaptic actuator 110. - Simultaneously, computer/
controller 106 sends a signal to display 50 to show the action being taken. For example, and not by way of limitation, if a user selects an icon (e.g., radio, navigation, climate, etc.), computer/controller 106 sends a signal to display 50 for the selected icon to be highlighted. Further, any sub-menus associated with the selected icon will be displayed. For example, and not by way of limitation,FIG. 5 shows an example display with amain menu 52 and asub menu 54. As described in more detail below, a gesture may be interpreted as selecting the climate control icon onmain menu 52, as shown inFIG. 5 . With such a selection, computer/controller 106 sends a signal to display to highlight the climate control icon, as shown inFIG. 5 . With the climate control icon selected, the sub-menu 54 associate with the climate controls appears. Similarly, if any of the other icons on themain menu 52 are selected, a sub-menu associated with that main menu will appear. - Further, if the gesture by the user is used to control a particular system, computer/
controller 106 also sends a signal to thevehicle system 112 to perform the selected control. For example, and not by way of limitation, upon selecting the climate icon ofmain menu 52, if a gesture to increase the driver side temperature is recognized by thecamera 60 and processed as described above, computer/controller 106 sends a signal to the climate control system of the vehicle to increase the temperature air blown through the driver side vents, for example. In another example, if the radio icon is selected frommain menu 52 and a particular gesture for increasing the volume is recognized, computer/controller 106 sends a signal to theradio control system 112 to increase the volume. Such signals will be sent to othervehicle control systems 112, such as for a CD player, navigation system, phone, DVD player, gaming system, or other vehicle system or sub-system, or auxiliary system coupled to a vehicle system. Further, those skilled in the art will recognize that althoughdisplay 50 has been identified separately from other vehicle systems,display 50 is one of the vehicle systems and need not be identified separately. It is identified separately in this description for convenience because some actions result only in action on the display, such as highlighting an icon, whereas other actions result in action either only on the other vehicle systems, or in both the other vehicle systems and the display. - Accordingly, the haptic feedback described above can be used confirm or describe actions that are being taken based on the gesture based input. In one example, summarized in Table 1 below and described with respect to a system such as the system shown in
FIG. 5 , a driver of a vehicle V makes gestures that are monitored bycamera 60. Although the example below is described with respect to a driver making gestures and particular haptic effects described below being felt by the driver, those skilled in the art will recognize that similar gestures may be made by passengers in the vehicle, that such gestures may be detected bycamera 60 or other cameras, and that the haptic effect may be provided to the passenger making the gesture. Of course, those skilled in the art will also recognize that in some embodiments, only the driver of the vehicle senses haptic feedback, while the passengers may receive other types of feedback, such as audio or visual, because the passengers do not need to keep their eyes on the road. However, haptic effects may be desirable for passengers to enhance their experience when interacting with entertainment systems in the vehicle, such as gaming systems or video systems. - In the particular example discussed with respect to Table 1, the driver waves his or her hand/finger to wake up the system. Accordingly, computer/
controller 106 sends a signal to select the haptic effect of a short vibration from table 108 and sends it to ahaptic actuator 110 associated with the driver, such as indriver seat 10. Upon feeling the short vibration, the driver may begin moving a finger or hand to track acrossmain menu 52. As the driver is moving a finger/hand, a light continuous vibration alerts the drive that a pointer ondisplay 50 is moving across the screen. When the pointer reaches a selectable area onmain menu 52, such as the climate icon, the intensity of the vibration is raised to be a medium continuous vibration. The driver can select the icon by pinching his or her fingers. Computer/controller 106, upon recognizing the gesture, will send a signal to select multiple short vibrations from the haptic effect table 108 and send a signal for such vibrations to ahaptic actuator 110 associated with the driver. Simultaneously, a signal is sent to display 50 to highlight the climate icon and to display the climate sub-menu options onsub-menu 54, as shown inFIG. 5 . The haptic effect allows the user to know that the icon has been selected. In order to further avoid the need for the driver to look atdisplay 50, when the driver activates the system,display 50 may always start at a certain icon. This allows the user to become accustomed to the order in which the icons are selected. For example, ifdisplay 50 always starts with the radio icon, the driver knows that when the driver has felt the fourth medium continuous vibration, the pointer is on the climate icon. Many other variations are possible for allowing the driver to recognize which particular icon is selected. For example, and not by way of limitation, each icon can be designated with its own haptic effect, such as by the particular effect or the location the effect is felt by the driver. For example and not by way of limitation, when the radio icon is selected, the haptic effect may be delivered via ahaptic actuator 110 in thehead rest 12 ofdriver seat 10, whereas when the climate icon is selected, the haptic effect may be delivered via a haptic actuator in theseat base 14 ofdriver seat 10. Of course, depending on the number of systems controlled, other variations would likely be necessary. For example, and not by way of limitation, each icon associated with sound may be associated with a haptic effect delivered tohaptic actuator 110 inhead rest 12, with each having its own haptic effect, such as one vibration for the radio, two for the CD player, and three for the phone. Similarly, selection of the climate icon may result in one vibration to theseat base 12, and selection of the navigation icon may result in two vibrations to theseat base 14. Similarly, if all haptic effects are delivered via a single haptic actuator, then variation in the haptic effect could be used to distinguish between the icons. - Once the icon has been selected from
main menu 52 and the sub-menu 54 for the selected system has appeared, as described above, the driver may make further gestures to control the selected system as displayed onsub-menu 54. Thus, in the example provided inFIG. 5 and Table 1, upon selection of the climate icon, the driver may move a finger to select items along the top bar ofsub menu 54 in a similar fashion as described above with respect tomain menu 52, and the appropriate haptic effect is provided. For example, and not by way of limitation, upon selection of “Auto” in the top bar ofsub-menu 54, the driver may rotate his or her hand clockwise to increase the temperature and counter-clockwise to decrease the temperature, similar to a rotary dial or knob. When such a gesture is recognized bycamera 60, computer/controller 106 sends a signal to select the haptic effect of a continuous vibration with increasing magnitude from haptic effect table 108 and the signal for such an effect is sent to ahaptic actuator 110 associated with the driver, who feels the effect. Simultaneously, a signal is sent to display 50 to highlight the up arrow in the temperature slide and to increase the temperature setting displayed. Also, a signal is sent to the vehicle climate control system to set the temperature to the selected temperature. -
TABLE 1 GESTURE INPUT SYSTEM ACTION HAPTIC EFFECT EXAMPLE Waving hand/finger Ready for input/wake up Short vibration Finger/hand tracking Pointer present on screen Light continuous vibration Finger/hand tracking Pointer on selectable area Medium continuous vibration Hand/finger movement from left Next menu, track, or other Short vibration to right option, accept, higher value Hand/finger movement from Previous menu, track, option, or Short vibration right to left home, reject, lower value Hand/finger movement Scroll down, more menu, lower Short vibration on value downward value change/scroll action Hand/finger movement upward Scroll up, more menu, higher Short vibration on value value, top menu change/scroll action Pinch fingers/close hand Icon/menu selection, zoom Short vibration or multiple short change, advance menu, back, vibrations accept Open fingers/open hand Icon/menu selection, zoom Short vibration or multiple short change, advance menu, back, vibrations accept Waving finger/hand (in menu) Reject, back, home, clear Continuous vibration Finger/hand movement toward Accept, select, next, zoom Short or medium length vibration screen Rotate hand clockwise/Move Change virtual dial value, rotate Continuous vibration of increasing fingers in a clockwise arc menu/screen/icon magnitude or vibration on change of value/location Rotate hand Change virtual dial value, rotate Continuous vibration of counterclockwise/Move fingers menu/screen/icon decreasing magnitude or vibration in a counter clockwise arc on change of value/location - Those skilled in the art would recognize that the description and Table 1 above provides only examples of the types of gestures and haptic effects that can be used. Similar hand gestures and haptic effects can be used to control the other vehicle systems. Further, different gestures and haptic effects may be used. The desired purpose of the gesture identification and haptic effect feedback system is to allow a simple method for the driver or passenger to control the vehicle systems without reaching towards or looking at a display that may distract the driver, and with sufficient haptic effect feedback such that the driver or passenger is comfortable that the desired action is taking place in response to the gesture command. Further, providing such haptic feedback can be used to enhance the experience of the driver and/or passenger(s).
- Further, although the embodiment described above relates to a gesture based input system with haptic effect feedback, those skilled in the art would recognize that detecting advanced gestures may be costly. Accordingly,
FIG. 6 and Table 2 below show an embodiment wherein gesture based input and audio input are combined, with haptic effect feedback used to confirm commands, as in the embodiment above. In particular,FIG. 6 shows a block diagram of an embodiment of a gesture and audio based input andfeedback system 200 and a flow chart for using thesystem 200. - In
FIG. 6 ,gesture input block 202 represents a gesture by a user of the vehicle, such as the driver. It is contemplated that such gestures are hand and/or finger gestures by the user. Other gestures may also be used. As the gesture is being made by the user, camera ofother gesture detector 60 detects the gesture by the user. As explained above,gesture detector 60 may be a single camera, multiple cameras, or other devices configured to detect location, orientation, and movements of the user. - Similarly, voice/
audio input block 214 represents a sound or voice command made by a user of the vehicle, such as the driver. It is contemplated that such voice commands will be simple voice commands, for example, the simple voice commands shown in Table 2 below. Other voice commands may also be used. An audio input detector ormicrophone 64 detects the audio/voice command.Microphone 64 may be located adjacent tocamera 60, as shown inFIG. 1 , or elsewhere in vehicle V, as known to those skilled in the art. - When the camera or
gesture detector 60 detects a gesture, it sends a signal to a gesture input controller/translator 204. Gesture input controller/translator 204 converts the camera signal into a gesture signal and transmits the signal to a computer/controller 206. For example, and not by way of limitation,camera 60 may provide data in the form of location of the detected hand/finger, speed/direction of motion, and other data regarding position, orientation, and movement of a users hand/finger. Gesture input controller/translator 204 converts this data into a particular gesture from a table of potential gestures. For example, and not by way of limitation, possible gestures may include a hand wave in particular directions, finger movement left, right, up, or down, hand/finger rotation, finger pinch/unpinch, rapid waving, finger movement towards or away from the camera, and other gestures known to those skilled in the art. If desired, sophisticated gesture detection systems may detect individual fingers such that movement by the pointer finger may be distinguished by movement from movement by the pinkie finger, for example. However, the use of an interactive haptic feedback system, as described in more detail below, allows for a system which does not need to recognize or distinguish between minor variations in movements. Accordingly, a less costly gesture recognition system can be used in combination with the interactive haptic feedback system. - Similarly, when
microphone 64 detects a sound, it sends a signal to an audio input controller/translator 218. Audio input controller/translator 218 converts the voice signal into a data signal representative of the voice command and transmits the signal to computer/controller 206. - After the gesture input controller/
translator 204 and/or audio input controller/translator converts 218 the data, the signal is sent to a computer/controller 206. Those skilled in the art would recognize that gesture input controller/translator 204, audio input controller/translator 218, and computer/controller 206 may be combined without departing from the present invention. In the embodiment shown inFIG. 6 , when computer/controller 206 receives a gesture or audio signal, computer/controller 206 processes the data to provide signals to three different devices. In particular, computer/controller 206 matches the gesture or audio signal to a haptic effect that is selected from a haptic table 208. For example, and not by way of limitation, if the haptic effect is generally a vibratory effect, a particular vibratory effect, in terms of degree, magnitude, frequency, and/or duration is selected. Further, if the gesture and audio based input system allows for input from multiple users, the computer/controller 206 selects to which user the haptic effect will be delivered. Further, in an embodiment, differenthaptic actuators 110 may be selected for an individual user. For example, and not by way of limitation, activation of ahaptic actuator 110 inhead rest 12 ofdriver seat 10 may have a different feedback meaning than activation of ahaptic actuator 110 inseat base 16 ofdriver seat 10. In essence, the computer/controller 206 selects to which haptic actuator 110 a signal will be sent. Thus, computer/controller 206 selects a particular haptic effect from haptic table 208 and a particularhaptic actuator 110 to which the effect will be applied. A signal for the selected haptic effect is then sent to the selectedhaptic actuator 110. - Simultaneously, computer/
controller 206 sends a signal to display 50 to show the action being taken. For example, and not by way of limitation, if a user selects an icon (e.g., radio, navigation, climate, etc.), computer/controller 206 sends a signal to display 50 for the selected icon to be highlighted. Further, any sub-menus associated with the selected icon will be displayed. For example, and not by way of limitation,FIG. 5 shows an example display with amain menu 52 and asub-menu 54. If the climate control icon is selected onmain menu 52, such as by using a gesture or voice command, computer/controller 206 sends a signal to display 50 to highlight the climate icon, as shown inFIG. 5 . With the climate icon selected, the sub-menu 54 associated with the climate control appears. Similarly, if any of the other icons on themain menu 52 are selected, a sub-menu associated with that main menu will appear. - Further, if the gesture or audio command by the user is used to control a particular system, computer/
controller 206 also sends a signal to thevehicle system 212 to perform the selected control. For example, and not by way of limitation, upon selecting the climate icon ofmain menu 52, if a gesture or audio command is used to increase the driver side temperature is recognized by thecamera 60/microphone 64 and processed as described above, computer/controller 206 sends a signal to the climate control system of the vehicle to increase the temperature air blown through the driver side vents, for example. In another example, if the radio icon is selected frommain menu 52 and a particular gesture or audio command is given for increasing the volume, computer/controller 206 sends a signal to theradio control system 212 to increase the volume. Such signals will be sent to othervehicle control system 212, such as for a CD player, navigation system, phone, DVD player, gaming system, or other vehicle system or sub-system, or auxiliary system coupled to a vehicle system. Further, those skilled in the art will recognize that althoughdisplay 50 has been identified separately from other vehicle systems,display 50 is one of the vehicle systems and need not be identified separately. It is identified separately in this description for convenience because some actions result only in action on the display, such as highlighting an icon, whereas other actions result in action either only on the other vehicle systems, or in both the other vehicle systems and the display. - Accordingly, the haptic feedback described above can be used confirm or describe actions that are being taken based on the gesture based input. In one example, summarized in Table 2 below and described with respect to a system such as the system shown in
FIG. 5 , a driver of a vehicle V makes gestures that are monitored bycamera 60 and gives voice commands that are received bymicrophone 64. Although the example below is described with respect to a driver making gestures and giving voice commands, and particular haptic effects described below, those skilled in the art will recognize that similar gestures and voice commands may be made by passengers in the vehicle, that such gestures and voice commands may be detected bycamera 60/microphone 64 or other cameras/microphones, and that the haptic effect may be provided to the user making the gesture/giving the voice command. Of course, those skilled in the art will also recognize that in some embodiments, only the driver of the vehicle senses haptic feedback, while the passengers may receive other types of feedback, such as audio or visual, because the passengers do not need to keep their eyes on the road. However, haptic effects may be desirable for passengers to enhance their experience when interacting with entertainment systems in the vehicle, such as gaming systems or video systems. - In the particular example of combining gesture-based input, audio-based input, and haptic effect feedback, Table 2 shows various hand gestures and voice commands to control various vehicle system, and haptic effects associated with such actions. Table 2 will be explained with respect to the
exemplary display 50 show inFIG. 5 . Accordingly, the driver may wave a hand or finger to activate the system, as described above with respect to Table 1. Accordingly, computer/controller 206 sends a signal to select the haptic effect of a short vibration from table 208 and sends it to ahaptic actuator 110 associated with the driver, such as indriver seat 10. Upon feeling the short vibration, the driver may begin moving a finger or hand to track acrossmain menu 52. As the driver is moving a finger/hand, a light continuous vibration alerts the drive when the drivers hand/finger is over an icon inmain menu 52, as shown in Table 2 under “Basic Navigation”. The driver can then select the icon by giving a voice command, such as “select” or “yes”, as shown in Table 2. Computer/controller 206, upon recognizing the voice command, will send a signal to select a short vibration from the haptic effect table and send a signal for such a vibration to ahaptic actuator 110 associated with the driver. Simultaneously, a signal is sent to display 50 to highlight the climate icon and to display the climate sub-menu options onsub-menu 54, as shown inFIG. 5 . The haptic effect allows the driver to know that the icon has been selected. As described above, several options for haptic feedback may be used to ensure that the driver is aware that he or she has selected the desired icon, such as different haptic effects associate with each vehicle system onmain menu 52. - Once the icon has been selected from
main menu 52 and the sub-menu 54 for the selected system has appeared, as described above, the driver may make further gestures to control the selected system as displayed onsub-menu 54. Thus, in the example provided inFIG. 5 and Table 2, upon selection of the climate icon, the driver may move a finger to select items along the top bar ofsub-menu 54 in a similar fashion as described above with respect tomain menu 52, and the appropriate haptic effect is provided. For example, and not by way of limitation, as the driver moves a hand/finger over the icons along top bar ofsub menu 54, the “Basic Navigation” features of Table 2 are used. Thus, if the driver's hand/finger hovers over the “Auto” icon, a light continuous vibration is felt fromhaptic actuator 110. Further, the driver may select the “Auto” icon by voice command of “select” or “yes”, as described above. Upon recognition of the voice command, computer/controller 206 sends a signal tohaptic actuator 110 for a short vibration and a signal to display 50 to highlight the “Auto” icon. - When the sub-menu is selected, the driver can then perform other activities in the sub menu. For example, the driver's fingers can be moved over the driver side temperature reading of 72 in
FIG. 5 . A haptic effect such as a light continuous vibration confirms to the driver that the temperature is selected. The driver can then change the value of the driver's side temperature by voice command, such as “74”, as shown in the “Change value, direct input, text input” row of Table 2. Accordingly, computer/controller 206 sends a signal tohaptic actuator 110 for a short vibration, a signal to display 50 to change the displayed driver's side temperature setting from “72” to “74”, and a signal to the climate control system to have the vehicle HVAC system provide a 74 degree temperature to the driver's side. - Table 2 shows how the gesture and voice inputs and haptic effect feedback may be used in combination to make selection and control of various vehicle functions relatively simple without the need for the driver to look at
display 50. While a particular sub-menu selection was described above, those skilled in the art would recognize that the other vehicle systems may be controlled as described in Table 2, such as by selecting the radio icon onmain menu 52 and using a voice command to change the station or raise and lower the volume, with the corresponding haptic effects shown in Table 2 providing feedback that the command has been processed y computer/controller 206. - In another embodiment described with respect to Table 3 below, interactions between the front and rear passengers are incorporated into the haptic feedback system. The embodiments shown in
FIG. 4 orFIG. 6 can be used to implement the responses shown in Table 3. -
TABLE 3 FIRST ACTION SECOND ACTION HAPTIC EFFECT Front passenger selects None Light vibration media/game for rear passenger Front passenger interrupts None Light vibration media/game for rear passenger Rear passenger interrupts None Light vibration media/game Rear passenger selects Front passenger Medium vibration media/game authorizes media/ game playback One rear passenger interacts Other rear passenger Vibrations with media/game interacts with corresponding to media/game media/game Rear passenger media/game None Medium vibration system has issues Rear seat belt unbuckled None Vibration which increases intensity with time Door open/opened None Continuous vibration - As described above, the gestures, commands, and specific haptic effects described with respect to Tables 1-3 are merely examples. Infinite variations of particular gestures, commands, and resulting haptic effects can be contemplated by those skilled in the art in keeping with the scope and content of the present invention.
- Accordingly, while various embodiments have been described above, it should be understood that they have been presented only as illustrations and examples of the present invention, and not by way of limitation. It will be apparent to persons skilled in the relevant art that various changes in form and detail can be made therein without departing from the spirit and scope of the invention. Thus, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the appended claims and their equivalents. It will also be understood that each feature of each embodiment discussed herein, and of each reference cited herein, can be used in combination with the features of any other embodiment. All patents and publications discussed herein are incorporated by reference herein in their entirety.
Claims (23)
Priority Applications (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/137,540 US9248840B2 (en) | 2013-12-20 | 2013-12-20 | Gesture based input system in a vehicle with haptic feedback |
EP14197941.9A EP2889723B1 (en) | 2013-12-20 | 2014-12-15 | Gesture based input system in a vehicle with haptic feedback |
EP20161782.6A EP3696651A1 (en) | 2013-12-20 | 2014-12-15 | Gesture based input system in a vehicle with haptic feedback |
CN201410795247.7A CN104731319A (en) | 2013-12-20 | 2014-12-18 | Gesture based input system in a vehicle with haptic feedback |
KR1020140184343A KR20150073857A (en) | 2013-12-20 | 2014-12-19 | Gesture based input system in a vehicle with haptic feedback |
JP2014257045A JP6935164B2 (en) | 2013-12-20 | 2014-12-19 | Gesture-based input system with tactile feedback in the vehicle |
US15/011,921 US9718477B2 (en) | 2013-12-20 | 2016-02-01 | Gesture based input system in a vehicle with haptic feedback |
US15/664,192 US10449972B2 (en) | 2013-12-20 | 2017-07-31 | Gesture based input system in a vehicle with haptic feedback |
JP2021097299A JP2021166058A (en) | 2013-12-20 | 2021-06-10 | Gesture based input system using tactile feedback in vehicle |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/137,540 US9248840B2 (en) | 2013-12-20 | 2013-12-20 | Gesture based input system in a vehicle with haptic feedback |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/011,921 Continuation US9718477B2 (en) | 2013-12-20 | 2016-02-01 | Gesture based input system in a vehicle with haptic feedback |
Publications (2)
Publication Number | Publication Date |
---|---|
US20150175172A1 true US20150175172A1 (en) | 2015-06-25 |
US9248840B2 US9248840B2 (en) | 2016-02-02 |
Family
ID=52282392
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/137,540 Active US9248840B2 (en) | 2013-12-20 | 2013-12-20 | Gesture based input system in a vehicle with haptic feedback |
US15/011,921 Active US9718477B2 (en) | 2013-12-20 | 2016-02-01 | Gesture based input system in a vehicle with haptic feedback |
US15/664,192 Active US10449972B2 (en) | 2013-12-20 | 2017-07-31 | Gesture based input system in a vehicle with haptic feedback |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/011,921 Active US9718477B2 (en) | 2013-12-20 | 2016-02-01 | Gesture based input system in a vehicle with haptic feedback |
US15/664,192 Active US10449972B2 (en) | 2013-12-20 | 2017-07-31 | Gesture based input system in a vehicle with haptic feedback |
Country Status (5)
Country | Link |
---|---|
US (3) | US9248840B2 (en) |
EP (2) | EP2889723B1 (en) |
JP (2) | JP6935164B2 (en) |
KR (1) | KR20150073857A (en) |
CN (1) | CN104731319A (en) |
Cited By (59)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150097798A1 (en) * | 2011-11-16 | 2015-04-09 | Flextronics Ap, Llc | Gesture recognition for on-board display |
JP2015125780A (en) * | 2013-12-26 | 2015-07-06 | ビステオン グローバル テクノロジーズ インコーポレイテッド | System and method for providing tactile feedback for gesture based inputs |
US20150199019A1 (en) * | 2014-01-16 | 2015-07-16 | Denso Corporation | Gesture based image capturing system for vehicle |
US20150309573A1 (en) * | 2014-04-28 | 2015-10-29 | Ford Global Technologies, Llc | Automotive touchscreen controls with simulated texture for haptic feedback |
US20160107570A1 (en) * | 2014-10-20 | 2016-04-21 | Immersion Corporation | Systems and methods for enhanced continuous awareness in vehicles using haptic feedback |
US20160162092A1 (en) * | 2014-12-08 | 2016-06-09 | Fujitsu Ten Limited | Operation device |
US20160179366A1 (en) * | 2014-11-13 | 2016-06-23 | Hyundai Motor Company | Touch input device and vehicle including the same |
US20160266655A1 (en) * | 2015-03-10 | 2016-09-15 | Robert Bosh Gmbh | Method for activating an actuator of a motor vehicle, device configured to carry out the method, and computer program product |
US20170060245A1 (en) * | 2015-08-31 | 2017-03-02 | Fujitsu Ten Limited | Input device, integrated input system, input device control method, and program |
US9639323B2 (en) * | 2015-04-14 | 2017-05-02 | Hon Hai Precision Industry Co., Ltd. | Audio control system and control method thereof |
USD788169S1 (en) * | 2015-09-03 | 2017-05-30 | Continental Automotive Gmbh | Display screen with icon |
US20170168630A1 (en) * | 2015-12-11 | 2017-06-15 | Immersion Corporation | Systems and Methods for Position-Based Haptic Effects |
DE102016214523A1 (en) * | 2016-08-05 | 2018-02-08 | Bayerische Motoren Werke Aktiengesellschaft | Device for gesture-based control of a system |
DE102016216318A1 (en) | 2016-08-30 | 2018-03-01 | Continental Automotive Gmbh | Method and device for operating an electronic device |
DE102017200717A1 (en) * | 2016-12-23 | 2018-06-28 | Audi Ag | Non-contact operator control device for a motor vehicle and motor vehicle and operating method for the operating device |
US10013858B2 (en) * | 2016-02-26 | 2018-07-03 | At&T Intellectual Property I, L.P. | Notification system with haptic feedback garment and methods for use therewith |
US20180199716A1 (en) * | 2017-01-13 | 2018-07-19 | Palm Beach Technology Llc | Smart chair |
US20180226077A1 (en) * | 2015-08-05 | 2018-08-09 | Lg Electronics Inc. | Vehicle driving assist and vehicle having same |
US10099608B2 (en) * | 2015-01-16 | 2018-10-16 | Ford Global Technologies, Llc | Haptic vehicle alert based on wearable device |
US10116873B1 (en) * | 2015-11-09 | 2018-10-30 | Ambarella, Inc. | System and method to adjust the field of view displayed on an electronic mirror using real-time, physical cues from the driver in a vehicle |
US10124648B2 (en) * | 2015-03-16 | 2018-11-13 | Thunder Power New Energy Vehicle Development Company Limited | Vehicle operating system using motion capture |
US20180326902A1 (en) * | 2015-11-27 | 2018-11-15 | Clarion Co., Ltd. | Vehicle notification device, vehicle notification method, and notification signals |
US20180348869A1 (en) * | 2015-11-27 | 2018-12-06 | Valeo Schalter Und Sensoren Gmbh | Operator apparatus for a motor vehicle with drive device for outputting a haptic feedback, and motor vehicle |
CN109249940A (en) * | 2017-07-04 | 2019-01-22 | 现代自动车株式会社 | Vehicular system and its control method |
US20190073040A1 (en) * | 2017-09-05 | 2019-03-07 | Future Mobility Corporation Limited | Gesture and motion based control of user interfaces |
US20190077311A1 (en) * | 2017-06-28 | 2019-03-14 | Honda Motor Co., Ltd. | Haptic function leather component and method of making the same |
US20190124290A1 (en) * | 2017-10-20 | 2019-04-25 | Shenzhen Matego Electronics Technology Co., Ltd. | Dashboard Camera |
US10372216B2 (en) | 2016-03-04 | 2019-08-06 | Nxp B.V. | Gesture feedback |
US20190241199A1 (en) * | 2015-01-13 | 2019-08-08 | Ck Materials Lab Co., Ltd. | Haptic information provision device |
US20190283780A1 (en) * | 2016-07-22 | 2019-09-19 | Harman International Industries, Incorporated | Haptic driving guidance system |
US10579252B2 (en) | 2014-04-28 | 2020-03-03 | Ford Global Technologies, Llc | Automotive touchscreen with simulated texture for the visually impaired |
US10682952B2 (en) | 2017-06-28 | 2020-06-16 | Honda Motor Co., Ltd. | Embossed smart functional premium natural leather |
US10682578B1 (en) | 2018-12-14 | 2020-06-16 | Universal City Studios Llc | Ride vehicle restraint system |
US20200226892A1 (en) * | 2019-01-11 | 2020-07-16 | Drift Net | Security System for Detecting Hazardous Events and Occupants in a Building |
US10742061B2 (en) | 2017-06-28 | 2020-08-11 | Honda Motor Co., Ltd. | Smart functional leather for recharging a portable electronic device |
CN111971201A (en) * | 2018-05-04 | 2020-11-20 | 本田技研工业株式会社 | Functional leather member and method for producing same |
WO2021004683A1 (en) | 2019-07-08 | 2021-01-14 | Volkswagen Aktiengesellschaft | Method and operating system for detecting a user input for a device of a vehicle |
WO2021004682A1 (en) | 2019-07-08 | 2021-01-14 | Volkswagen Aktiengesellschaft | Method for operating an operating system in a vehicle and operating system in a vehicle |
WO2021004681A1 (en) | 2019-07-08 | 2021-01-14 | Volkswagen Aktiengesellschaft | Method for operating an operator control system, and operator control system |
US10946797B2 (en) | 2017-06-28 | 2021-03-16 | Honda Motor Co., Ltd. | Smart functional leather for steering wheel and dash board |
US11059495B2 (en) * | 2016-09-12 | 2021-07-13 | Sony Corporation | Information presentation apparatus, steering apparatus, and information presentation method |
US11144129B2 (en) * | 2020-03-04 | 2021-10-12 | Panasonic Avionics Corporation | Depth sensing infrared input device and associated methods thereof |
US20210319191A1 (en) * | 2019-05-06 | 2021-10-14 | Stmicroelectronics (Rousset) Sas | Method for compensating for a phase shift between the signal emitted by an object and that received from a reader equipped with an envelope detector and corresponding object |
US11182072B2 (en) * | 2019-09-09 | 2021-11-23 | Hyundai Motor Company | Touch screen, a vehicle having the same, and a method of controlling the vehicle |
US11225191B2 (en) | 2017-06-28 | 2022-01-18 | Honda Motor Co., Ltd. | Smart leather with wireless power |
CN114258367A (en) * | 2020-07-23 | 2022-03-29 | 日产自动车株式会社 | Control system, gesture recognition system, vehicle, and method of controlling gesture recognition system |
US11314331B2 (en) * | 2016-09-09 | 2022-04-26 | Behr-Hella Thermocontrol Gmbh | Operating unit for a device, in particular for a vehicle component |
US11345309B2 (en) | 2018-12-07 | 2022-05-31 | Universal City Studios Llc | Ride vehicle restraint system |
US20220185296A1 (en) * | 2017-12-18 | 2022-06-16 | Plusai, Inc. | Method and system for human-like driving lane planning in autonomous driving vehicles |
US20220308671A1 (en) * | 2019-09-09 | 2022-09-29 | Japan Science And Technology Agency | Force sense presentation device and force sense presentation method |
WO2022241015A1 (en) * | 2021-05-12 | 2022-11-17 | Polestar Performance Ab | Electroactive polymer seat |
US11521470B2 (en) * | 2020-10-28 | 2022-12-06 | Cerence Operating Company | Haptic communication with a vehicle's occupant |
US11535268B2 (en) * | 2019-01-07 | 2022-12-27 | Hyundai Motor Company | Vehicle and control method thereof |
US20230049900A1 (en) * | 2021-08-16 | 2023-02-16 | Toyota Research Institute, Inc. | Displaced haptic feedback |
US11625145B2 (en) | 2014-04-28 | 2023-04-11 | Ford Global Technologies, Llc | Automotive touchscreen with simulated texture for the visually impaired |
US11665830B2 (en) | 2017-06-28 | 2023-05-30 | Honda Motor Co., Ltd. | Method of making smart functional leather |
US20230264570A1 (en) * | 2020-06-30 | 2023-08-24 | Daimler Ag | Operating unit comprising a touch-sensitive operating area |
US11751337B2 (en) | 2019-04-26 | 2023-09-05 | Honda Motor Co., Ltd. | Wireless power of in-mold electronics and the application within a vehicle |
US11899448B2 (en) * | 2019-02-21 | 2024-02-13 | GM Global Technology Operations LLC | Autonomous vehicle that is configured to identify a travel characteristic based upon a gesture |
Families Citing this family (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10146320B2 (en) | 2007-10-29 | 2018-12-04 | The Boeing Company | Aircraft having gesture-based control for an onboard passenger service unit |
US9248840B2 (en) * | 2013-12-20 | 2016-02-02 | Immersion Corporation | Gesture based input system in a vehicle with haptic feedback |
EP3106343A1 (en) * | 2015-06-19 | 2016-12-21 | Continental Automotive GmbH | Gesture based user input device for car electronics |
JP2017026230A (en) * | 2015-07-23 | 2017-02-02 | パナソニックIpマネジメント株式会社 | Air conditioning control device, air conditioner, air conditioning control method, air conditioning control system, air conditioning control program and recording medium |
JP6585431B2 (en) * | 2015-08-31 | 2019-10-02 | 株式会社デンソーテン | INPUT DEVICE, INTEGRATED INPUT SYSTEM, INPUT DEVICE CONTROL METHOD, AND PROGRAM |
US9891711B1 (en) | 2016-07-26 | 2018-02-13 | Toyota Motor Engineering & Manufacturing North America, Inc. | Human machine interface with haptic response based on phased array LIDAR |
DE102016214692B4 (en) * | 2016-08-08 | 2018-02-15 | Audi Ag | Method and user interface relating to haptic feedback for a screen |
EP3290284A1 (en) * | 2016-08-30 | 2018-03-07 | Honda Research Institute Europe GmbH | Vehicle with a driver seat and at least one passenger seat and a method for providing a co-driver and/or at least one fellow passenger with information on a currently experienced driving situation |
GB2555088A (en) * | 2016-10-11 | 2018-04-25 | Jaguar Land Rover Ltd | Interface apparatus and method |
DE112016007259T5 (en) * | 2016-10-24 | 2019-06-06 | Ford Motor Company | GESTURIZED USER INTERFACE |
CN109997096A (en) * | 2016-11-17 | 2019-07-09 | 索尼公司 | Vibrate presentation device, vibration rendering method and program |
DE102017201236B4 (en) | 2017-01-26 | 2023-09-07 | Volkswagen Aktiengesellschaft | Method for operating a control system, control system and vehicle with a control system |
US10732714B2 (en) | 2017-05-08 | 2020-08-04 | Cirrus Logic, Inc. | Integrated haptic system |
US11259121B2 (en) | 2017-07-21 | 2022-02-22 | Cirrus Logic, Inc. | Surface speaker |
KR101968620B1 (en) * | 2017-09-11 | 2019-04-12 | (주)신오전자 | HMI System and Method for Vehicle |
US10433630B2 (en) | 2018-01-05 | 2019-10-08 | L'oreal | Cosmetic applicator system including trackable cosmetic device, and client device to assist users in makeup application |
US10810902B2 (en) | 2018-01-05 | 2020-10-20 | L'oreal | Trackable cosmetic device to assist users in makeup application |
US10455339B2 (en) | 2018-01-19 | 2019-10-22 | Cirrus Logic, Inc. | Always-on detection systems |
US10620704B2 (en) | 2018-01-19 | 2020-04-14 | Cirrus Logic, Inc. | Haptic output systems |
US11139767B2 (en) | 2018-03-22 | 2021-10-05 | Cirrus Logic, Inc. | Methods and apparatus for driving a transducer |
US10795443B2 (en) | 2018-03-23 | 2020-10-06 | Cirrus Logic, Inc. | Methods and apparatus for driving a transducer |
US10820100B2 (en) | 2018-03-26 | 2020-10-27 | Cirrus Logic, Inc. | Methods and apparatus for limiting the excursion of a transducer |
US10667051B2 (en) | 2018-03-26 | 2020-05-26 | Cirrus Logic, Inc. | Methods and apparatus for limiting the excursion of a transducer |
US10832537B2 (en) | 2018-04-04 | 2020-11-10 | Cirrus Logic, Inc. | Methods and apparatus for outputting a haptic signal to a haptic transducer |
US11069206B2 (en) | 2018-05-04 | 2021-07-20 | Cirrus Logic, Inc. | Methods and apparatus for outputting a haptic signal to a haptic transducer |
EP3613638A1 (en) | 2018-08-10 | 2020-02-26 | Lg Electronics Inc. | Vehicle display system for vehicle |
US11269415B2 (en) | 2018-08-14 | 2022-03-08 | Cirrus Logic, Inc. | Haptic output systems |
US10800433B2 (en) | 2018-09-14 | 2020-10-13 | Honda Motor Co., Ltd. | Seat haptic system and method of equalizing haptic output |
KR102178939B1 (en) * | 2018-10-17 | 2020-11-13 | 유병훈 | Apparatus for controlling environment and method of controlling environment |
WO2020085521A1 (en) * | 2018-10-23 | 2020-04-30 | 엘지전자 주식회사 | Input/output device and vehicle comprising same |
GB201817495D0 (en) | 2018-10-26 | 2018-12-12 | Cirrus Logic Int Semiconductor Ltd | A force sensing system and method |
US11509292B2 (en) | 2019-03-29 | 2022-11-22 | Cirrus Logic, Inc. | Identifying mechanical impedance of an electromagnetic load using least-mean-squares filter |
US10726683B1 (en) | 2019-03-29 | 2020-07-28 | Cirrus Logic, Inc. | Identifying mechanical impedance of an electromagnetic load using a two-tone stimulus |
US11644370B2 (en) | 2019-03-29 | 2023-05-09 | Cirrus Logic, Inc. | Force sensing with an electromagnetic load |
US10828672B2 (en) | 2019-03-29 | 2020-11-10 | Cirrus Logic, Inc. | Driver circuitry |
US10992297B2 (en) | 2019-03-29 | 2021-04-27 | Cirrus Logic, Inc. | Device comprising force sensors |
US10955955B2 (en) | 2019-03-29 | 2021-03-23 | Cirrus Logic, Inc. | Controller for use in a device comprising force sensors |
US20200313529A1 (en) | 2019-03-29 | 2020-10-01 | Cirrus Logic International Semiconductor Ltd. | Methods and systems for estimating transducer parameters |
US11238671B2 (en) | 2019-04-18 | 2022-02-01 | Honda Motor Co., Ltd. | System and method for providing haptic alerts within a vehicle |
CN111942242B (en) * | 2019-05-14 | 2022-10-25 | 奥迪股份公司 | System for pacifying a child in a vehicle, corresponding method and medium |
US10976825B2 (en) | 2019-06-07 | 2021-04-13 | Cirrus Logic, Inc. | Methods and apparatuses for controlling operation of a vibrational output system and/or operation of an input sensor system |
US11150733B2 (en) | 2019-06-07 | 2021-10-19 | Cirrus Logic, Inc. | Methods and apparatuses for providing a haptic output signal to a haptic actuator |
KR20220024091A (en) | 2019-06-21 | 2022-03-03 | 시러스 로직 인터내셔널 세미컨덕터 리미티드 | Method and apparatus for configuring a plurality of virtual buttons on a device |
US11408787B2 (en) | 2019-10-15 | 2022-08-09 | Cirrus Logic, Inc. | Control methods for a force sensor system |
US11380175B2 (en) | 2019-10-24 | 2022-07-05 | Cirrus Logic, Inc. | Reproducibility of haptic waveform |
US11545951B2 (en) | 2019-12-06 | 2023-01-03 | Cirrus Logic, Inc. | Methods and systems for detecting and managing amplifier instability |
US11873000B2 (en) | 2020-02-18 | 2024-01-16 | Toyota Motor North America, Inc. | Gesture detection for transport control |
US20210276575A1 (en) * | 2020-03-04 | 2021-09-09 | Ford Global Technologies, Llc | Vehicle component identification system |
US11662821B2 (en) | 2020-04-16 | 2023-05-30 | Cirrus Logic, Inc. | In-situ monitoring, calibration, and testing of a haptic actuator |
US11933822B2 (en) | 2021-06-16 | 2024-03-19 | Cirrus Logic Inc. | Methods and systems for in-system estimation of actuator parameters |
US11908310B2 (en) | 2021-06-22 | 2024-02-20 | Cirrus Logic Inc. | Methods and systems for detecting and managing unexpected spectral content in an amplifier system |
US11765499B2 (en) | 2021-06-22 | 2023-09-19 | Cirrus Logic Inc. | Methods and systems for managing mixed mode electromechanical actuator drive |
US11552649B1 (en) | 2021-12-03 | 2023-01-10 | Cirrus Logic, Inc. | Analog-to-digital converter-embedded fixed-phase variable gain amplifier stages for dual monitoring paths |
KR102559138B1 (en) * | 2022-12-28 | 2023-07-25 | 에이아이다이콤 (주) | Non-contact control system for vehicle |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7519223B2 (en) * | 2004-06-28 | 2009-04-14 | Microsoft Corporation | Recognizing gestures and using gestures for interacting with software applications |
US20130261871A1 (en) * | 2012-04-02 | 2013-10-03 | Google Inc. | Gesture-Based Automotive Controls |
US20140218187A1 (en) * | 2013-02-04 | 2014-08-07 | Anthony L. Chun | Assessment and management of emotional state of a vehicle operator |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090273574A1 (en) | 1995-06-29 | 2009-11-05 | Pryor Timothy R | Programmable tactile touch screen displays and man-machine interfaces for improved vehicle instrumentation and telematics |
EP1052593B1 (en) * | 1999-05-13 | 2015-07-15 | Canon Kabushiki Kaisha | Form search apparatus and method |
US7050606B2 (en) | 1999-08-10 | 2006-05-23 | Cybernet Systems Corporation | Tracking and gesture recognition system particularly suited to vehicular control applications |
AU2001294852A1 (en) * | 2000-09-28 | 2002-04-08 | Immersion Corporation | Directional tactile feedback for haptic feedback interface devices |
US6703999B1 (en) * | 2000-11-13 | 2004-03-09 | Toyota Jidosha Kabushiki Kaisha | System for computer user interface |
JP2003131785A (en) * | 2001-10-22 | 2003-05-09 | Toshiba Corp | Interface device, operation control method and program product |
US7196790B2 (en) * | 2002-03-18 | 2007-03-27 | Honeywell International Inc. | Multiple wavelength spectrometer |
US7295904B2 (en) | 2004-08-31 | 2007-11-13 | International Business Machines Corporation | Touch gesture based interface for motor vehicle |
DE102004044880B3 (en) * | 2004-09-14 | 2006-06-14 | Siemens Ag | Operating element for a heating and air conditioning system |
US20090278915A1 (en) | 2006-02-08 | 2009-11-12 | Oblong Industries, Inc. | Gesture-Based Control System For Vehicle Interfaces |
US7792328B2 (en) | 2007-01-12 | 2010-09-07 | International Business Machines Corporation | Warning a vehicle operator of unsafe operation behavior based on a 3D captured image stream |
JP4605170B2 (en) * | 2007-03-23 | 2011-01-05 | 株式会社デンソー | Operation input device |
US8508486B2 (en) * | 2007-10-01 | 2013-08-13 | Immersion Corporation | Directional haptic effects for a handheld device |
US8031172B2 (en) | 2007-10-12 | 2011-10-04 | Immersion Corporation | Method and apparatus for wearable remote interface device |
DE102008051756A1 (en) | 2007-11-12 | 2009-05-14 | Volkswagen Ag | Multimodal user interface of a driver assistance system for entering and presenting information |
CN102473034B (en) | 2009-07-22 | 2015-04-01 | 意美森公司 | System and method for providing complex haptic stimulation during input of control gestures, and relating to control of virtual equipment |
JP5847407B2 (en) * | 2010-03-16 | 2016-01-20 | イマージョン コーポレーションImmersion Corporation | System and method for pre-touch and true touch |
WO2012112277A1 (en) | 2011-01-26 | 2012-08-23 | Novodigit Sarl | Breath-sensitive digital interface |
ES2791722T3 (en) | 2012-02-02 | 2020-11-05 | Airbus Helicopters Espana Sa | Virtual mockup with haptic portable aid |
KR102024006B1 (en) * | 2012-02-10 | 2019-09-24 | 삼성전자주식회사 | Apparatus and method for controlling vibration flow between vibration devices |
US9248840B2 (en) * | 2013-12-20 | 2016-02-02 | Immersion Corporation | Gesture based input system in a vehicle with haptic feedback |
US9891711B1 (en) * | 2016-07-26 | 2018-02-13 | Toyota Motor Engineering & Manufacturing North America, Inc. | Human machine interface with haptic response based on phased array LIDAR |
-
2013
- 2013-12-20 US US14/137,540 patent/US9248840B2/en active Active
-
2014
- 2014-12-15 EP EP14197941.9A patent/EP2889723B1/en active Active
- 2014-12-15 EP EP20161782.6A patent/EP3696651A1/en not_active Withdrawn
- 2014-12-18 CN CN201410795247.7A patent/CN104731319A/en active Pending
- 2014-12-19 JP JP2014257045A patent/JP6935164B2/en active Active
- 2014-12-19 KR KR1020140184343A patent/KR20150073857A/en not_active Application Discontinuation
-
2016
- 2016-02-01 US US15/011,921 patent/US9718477B2/en active Active
-
2017
- 2017-07-31 US US15/664,192 patent/US10449972B2/en active Active
-
2021
- 2021-06-10 JP JP2021097299A patent/JP2021166058A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7519223B2 (en) * | 2004-06-28 | 2009-04-14 | Microsoft Corporation | Recognizing gestures and using gestures for interacting with software applications |
US20130261871A1 (en) * | 2012-04-02 | 2013-10-03 | Google Inc. | Gesture-Based Automotive Controls |
US20140218187A1 (en) * | 2013-02-04 | 2014-08-07 | Anthony L. Chun | Assessment and management of emotional state of a vehicle operator |
Cited By (86)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9449516B2 (en) * | 2011-11-16 | 2016-09-20 | Autoconnect Holdings Llc | Gesture recognition for on-board display |
US20150097798A1 (en) * | 2011-11-16 | 2015-04-09 | Flextronics Ap, Llc | Gesture recognition for on-board display |
JP2015125780A (en) * | 2013-12-26 | 2015-07-06 | ビステオン グローバル テクノロジーズ インコーポレイテッド | System and method for providing tactile feedback for gesture based inputs |
US20150199019A1 (en) * | 2014-01-16 | 2015-07-16 | Denso Corporation | Gesture based image capturing system for vehicle |
US9430046B2 (en) * | 2014-01-16 | 2016-08-30 | Denso International America, Inc. | Gesture based image capturing system for vehicle |
US20150309573A1 (en) * | 2014-04-28 | 2015-10-29 | Ford Global Technologies, Llc | Automotive touchscreen controls with simulated texture for haptic feedback |
US11625145B2 (en) | 2014-04-28 | 2023-04-11 | Ford Global Technologies, Llc | Automotive touchscreen with simulated texture for the visually impaired |
US9829979B2 (en) * | 2014-04-28 | 2017-11-28 | Ford Global Technologies, Llc | Automotive touchscreen controls with simulated texture for haptic feedback |
US10579252B2 (en) | 2014-04-28 | 2020-03-03 | Ford Global Technologies, Llc | Automotive touchscreen with simulated texture for the visually impaired |
US20160107570A1 (en) * | 2014-10-20 | 2016-04-21 | Immersion Corporation | Systems and methods for enhanced continuous awareness in vehicles using haptic feedback |
US10232773B2 (en) | 2014-10-20 | 2019-03-19 | Immersion Corporation | Systems and methods for awareness in vehicles using haptic effects |
US9827904B2 (en) * | 2014-10-20 | 2017-11-28 | Immersion Corporation | Systems and methods for enhanced continuous awareness in vehicles using haptic feedback |
US20160179366A1 (en) * | 2014-11-13 | 2016-06-23 | Hyundai Motor Company | Touch input device and vehicle including the same |
US10678425B2 (en) * | 2014-11-13 | 2020-06-09 | Hyundai Motor Company | Touch input device and vehicle including the same |
US11474687B2 (en) * | 2014-11-13 | 2022-10-18 | Hyundai Motor Company | Touch input device and vehicle including the same |
US20160162092A1 (en) * | 2014-12-08 | 2016-06-09 | Fujitsu Ten Limited | Operation device |
US20190241199A1 (en) * | 2015-01-13 | 2019-08-08 | Ck Materials Lab Co., Ltd. | Haptic information provision device |
US10099608B2 (en) * | 2015-01-16 | 2018-10-16 | Ford Global Technologies, Llc | Haptic vehicle alert based on wearable device |
US20160266655A1 (en) * | 2015-03-10 | 2016-09-15 | Robert Bosh Gmbh | Method for activating an actuator of a motor vehicle, device configured to carry out the method, and computer program product |
US10124648B2 (en) * | 2015-03-16 | 2018-11-13 | Thunder Power New Energy Vehicle Development Company Limited | Vehicle operating system using motion capture |
US9639323B2 (en) * | 2015-04-14 | 2017-05-02 | Hon Hai Precision Industry Co., Ltd. | Audio control system and control method thereof |
US20180226077A1 (en) * | 2015-08-05 | 2018-08-09 | Lg Electronics Inc. | Vehicle driving assist and vehicle having same |
US20170060245A1 (en) * | 2015-08-31 | 2017-03-02 | Fujitsu Ten Limited | Input device, integrated input system, input device control method, and program |
USD788169S1 (en) * | 2015-09-03 | 2017-05-30 | Continental Automotive Gmbh | Display screen with icon |
US10116873B1 (en) * | 2015-11-09 | 2018-10-30 | Ambarella, Inc. | System and method to adjust the field of view displayed on an electronic mirror using real-time, physical cues from the driver in a vehicle |
US20180326902A1 (en) * | 2015-11-27 | 2018-11-15 | Clarion Co., Ltd. | Vehicle notification device, vehicle notification method, and notification signals |
US20180348869A1 (en) * | 2015-11-27 | 2018-12-06 | Valeo Schalter Und Sensoren Gmbh | Operator apparatus for a motor vehicle with drive device for outputting a haptic feedback, and motor vehicle |
US10479271B2 (en) * | 2015-11-27 | 2019-11-19 | Clarion Co., Ltd. | Vehicle notification device and vehicle notification method |
US9990078B2 (en) * | 2015-12-11 | 2018-06-05 | Immersion Corporation | Systems and methods for position-based haptic effects |
US20170168630A1 (en) * | 2015-12-11 | 2017-06-15 | Immersion Corporation | Systems and Methods for Position-Based Haptic Effects |
US10551964B2 (en) | 2015-12-11 | 2020-02-04 | Immersion Corporation | Systems and methods for manipulating a graphical user interface through gestures in real space and providing associated haptic effects |
US10185441B2 (en) | 2015-12-11 | 2019-01-22 | Immersion Corporation | Systems and methods for position-based haptic effects |
US10013858B2 (en) * | 2016-02-26 | 2018-07-03 | At&T Intellectual Property I, L.P. | Notification system with haptic feedback garment and methods for use therewith |
US10482731B2 (en) * | 2016-02-26 | 2019-11-19 | At&T Intellectual Property I, L.P. | Notification system with haptic feedback garment and methods for use therewith |
US10235850B2 (en) * | 2016-02-26 | 2019-03-19 | At&T Intellectual Property I, L.P. | Notification system with haptic feedback garment and methods for use therewith |
US10372216B2 (en) | 2016-03-04 | 2019-08-06 | Nxp B.V. | Gesture feedback |
US20190283780A1 (en) * | 2016-07-22 | 2019-09-19 | Harman International Industries, Incorporated | Haptic driving guidance system |
US10890975B2 (en) | 2016-07-22 | 2021-01-12 | Harman International Industries, Incorporated | Haptic guidance system |
US11126263B2 (en) | 2016-07-22 | 2021-09-21 | Harman International Industries, Incorporated | Haptic system for actuating materials |
US10671170B2 (en) * | 2016-07-22 | 2020-06-02 | Harman International Industries, Inc. | Haptic driving guidance system |
US10915175B2 (en) | 2016-07-22 | 2021-02-09 | Harman International Industries, Incorporated | Haptic notification system for vehicles |
US11275442B2 (en) | 2016-07-22 | 2022-03-15 | Harman International Industries, Incorporated | Echolocation with haptic transducer devices |
US11392201B2 (en) | 2016-07-22 | 2022-07-19 | Harman International Industries, Incorporated | Haptic system for delivering audio content to a user |
DE102016214523A1 (en) * | 2016-08-05 | 2018-02-08 | Bayerische Motoren Werke Aktiengesellschaft | Device for gesture-based control of a system |
DE102016216318A1 (en) | 2016-08-30 | 2018-03-01 | Continental Automotive Gmbh | Method and device for operating an electronic device |
US11314331B2 (en) * | 2016-09-09 | 2022-04-26 | Behr-Hella Thermocontrol Gmbh | Operating unit for a device, in particular for a vehicle component |
US11059495B2 (en) * | 2016-09-12 | 2021-07-13 | Sony Corporation | Information presentation apparatus, steering apparatus, and information presentation method |
DE102017200717A1 (en) * | 2016-12-23 | 2018-06-28 | Audi Ag | Non-contact operator control device for a motor vehicle and motor vehicle and operating method for the operating device |
US20180199716A1 (en) * | 2017-01-13 | 2018-07-19 | Palm Beach Technology Llc | Smart chair |
US10742061B2 (en) | 2017-06-28 | 2020-08-11 | Honda Motor Co., Ltd. | Smart functional leather for recharging a portable electronic device |
US10953793B2 (en) * | 2017-06-28 | 2021-03-23 | Honda Motor Co., Ltd. | Haptic function leather component and method of making the same |
US11665830B2 (en) | 2017-06-28 | 2023-05-30 | Honda Motor Co., Ltd. | Method of making smart functional leather |
US11225191B2 (en) | 2017-06-28 | 2022-01-18 | Honda Motor Co., Ltd. | Smart leather with wireless power |
US11827143B2 (en) | 2017-06-28 | 2023-11-28 | Honda Motor Co., Ltd. | Embossed smart functional premium natural leather |
US10682952B2 (en) | 2017-06-28 | 2020-06-16 | Honda Motor Co., Ltd. | Embossed smart functional premium natural leather |
US10946797B2 (en) | 2017-06-28 | 2021-03-16 | Honda Motor Co., Ltd. | Smart functional leather for steering wheel and dash board |
US20190077311A1 (en) * | 2017-06-28 | 2019-03-14 | Honda Motor Co., Ltd. | Haptic function leather component and method of making the same |
US11027647B2 (en) | 2017-06-28 | 2021-06-08 | Honda Motor Co., Ltd. | Embossed smart functional premium natural leather |
CN109249940A (en) * | 2017-07-04 | 2019-01-22 | 现代自动车株式会社 | Vehicular system and its control method |
US20190073040A1 (en) * | 2017-09-05 | 2019-03-07 | Future Mobility Corporation Limited | Gesture and motion based control of user interfaces |
US20190124290A1 (en) * | 2017-10-20 | 2019-04-25 | Shenzhen Matego Electronics Technology Co., Ltd. | Dashboard Camera |
US20220185296A1 (en) * | 2017-12-18 | 2022-06-16 | Plusai, Inc. | Method and system for human-like driving lane planning in autonomous driving vehicles |
CN111971201A (en) * | 2018-05-04 | 2020-11-20 | 本田技研工业株式会社 | Functional leather member and method for producing same |
US11345309B2 (en) | 2018-12-07 | 2022-05-31 | Universal City Studios Llc | Ride vehicle restraint system |
US10682578B1 (en) | 2018-12-14 | 2020-06-16 | Universal City Studios Llc | Ride vehicle restraint system |
US11535268B2 (en) * | 2019-01-07 | 2022-12-27 | Hyundai Motor Company | Vehicle and control method thereof |
US20200226892A1 (en) * | 2019-01-11 | 2020-07-16 | Drift Net | Security System for Detecting Hazardous Events and Occupants in a Building |
US10964177B1 (en) | 2019-01-11 | 2021-03-30 | Drift Net | Security system for detecting hazardous events and occupants in a building |
US10810845B2 (en) * | 2019-01-11 | 2020-10-20 | Drift Net | Security system for detecting hazardous events and occupants in a building |
US11899448B2 (en) * | 2019-02-21 | 2024-02-13 | GM Global Technology Operations LLC | Autonomous vehicle that is configured to identify a travel characteristic based upon a gesture |
US11751337B2 (en) | 2019-04-26 | 2023-09-05 | Honda Motor Co., Ltd. | Wireless power of in-mold electronics and the application within a vehicle |
US20210319191A1 (en) * | 2019-05-06 | 2021-10-14 | Stmicroelectronics (Rousset) Sas | Method for compensating for a phase shift between the signal emitted by an object and that received from a reader equipped with an envelope detector and corresponding object |
US11544480B2 (en) * | 2019-05-06 | 2023-01-03 | Stmicroelectronics (Rousset) Sas | Method for compensating for a phase shift between the signal emitted by an object and that received from a reader equipped with an envelope detector and corresponding object |
WO2021004681A1 (en) | 2019-07-08 | 2021-01-14 | Volkswagen Aktiengesellschaft | Method for operating an operator control system, and operator control system |
WO2021004682A1 (en) | 2019-07-08 | 2021-01-14 | Volkswagen Aktiengesellschaft | Method for operating an operating system in a vehicle and operating system in a vehicle |
WO2021004683A1 (en) | 2019-07-08 | 2021-01-14 | Volkswagen Aktiengesellschaft | Method and operating system for detecting a user input for a device of a vehicle |
US20220308671A1 (en) * | 2019-09-09 | 2022-09-29 | Japan Science And Technology Agency | Force sense presentation device and force sense presentation method |
US11182072B2 (en) * | 2019-09-09 | 2021-11-23 | Hyundai Motor Company | Touch screen, a vehicle having the same, and a method of controlling the vehicle |
US11947725B2 (en) * | 2019-09-09 | 2024-04-02 | Japan Science And Technology Agency | Force sense presentation device and force sense presentation method |
US11144129B2 (en) * | 2020-03-04 | 2021-10-12 | Panasonic Avionics Corporation | Depth sensing infrared input device and associated methods thereof |
US20230264570A1 (en) * | 2020-06-30 | 2023-08-24 | Daimler Ag | Operating unit comprising a touch-sensitive operating area |
US11938823B2 (en) * | 2020-06-30 | 2024-03-26 | Mercedes-Benz Group AG | Operating unit comprising a touch-sensitive operating area |
CN114258367A (en) * | 2020-07-23 | 2022-03-29 | 日产自动车株式会社 | Control system, gesture recognition system, vehicle, and method of controlling gesture recognition system |
US11521470B2 (en) * | 2020-10-28 | 2022-12-06 | Cerence Operating Company | Haptic communication with a vehicle's occupant |
WO2022241015A1 (en) * | 2021-05-12 | 2022-11-17 | Polestar Performance Ab | Electroactive polymer seat |
US20230049900A1 (en) * | 2021-08-16 | 2023-02-16 | Toyota Research Institute, Inc. | Displaced haptic feedback |
Also Published As
Publication number | Publication date |
---|---|
CN104731319A (en) | 2015-06-24 |
KR20150073857A (en) | 2015-07-01 |
EP2889723B1 (en) | 2020-03-11 |
JP2021166058A (en) | 2021-10-14 |
US9718477B2 (en) | 2017-08-01 |
US20160144868A1 (en) | 2016-05-26 |
US10449972B2 (en) | 2019-10-22 |
EP3696651A1 (en) | 2020-08-19 |
JP6935164B2 (en) | 2021-09-15 |
US9248840B2 (en) | 2016-02-02 |
EP2889723A1 (en) | 2015-07-01 |
JP2015133109A (en) | 2015-07-23 |
US20180015933A1 (en) | 2018-01-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10449972B2 (en) | Gesture based input system in a vehicle with haptic feedback | |
CN110045825B (en) | Gesture recognition system for vehicle interaction control | |
US10394375B2 (en) | Systems and methods for controlling multiple displays of a motor vehicle | |
US10579252B2 (en) | Automotive touchscreen with simulated texture for the visually impaired | |
US9481246B2 (en) | Vehicle control apparatus and method thereof | |
CN106427571B (en) | Interactive operating device and method for operating same | |
US20190073040A1 (en) | Gesture and motion based control of user interfaces | |
KR20180041049A (en) | Contextual pressure sensing haptic responses | |
KR20120046059A (en) | Method and apparatus for gesture recognition | |
JP2005174356A (en) | Direction detection method | |
US20140379212A1 (en) | Blind control system for vehicle | |
US20190212910A1 (en) | Method for operating a human-machine interface and human-machine interface | |
US20190250776A1 (en) | Vehicular display apparatus | |
US20230049900A1 (en) | Displaced haptic feedback | |
US20190391736A1 (en) | Input device and input method | |
JP2005135439A (en) | Operation input device | |
US20150185885A1 (en) | Devices, Systems, and Methods For Using Corrugated Tessellation To Create Surface Features | |
EP3223130A1 (en) | Method of controlling an input device for navigating a hierarchical menu | |
CN110850975B (en) | Electronic system with palm recognition, vehicle and operation method thereof | |
Park et al. | Gaze-directed hands-free interface for mobile interaction | |
JP6739864B2 (en) | In-vehicle information processing system | |
CN116783087A (en) | Method for improving safety during operation of a device | |
JP2018163555A (en) | Information input system and information input program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: IMMERSION CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TRUONG, DAT;REEL/FRAME:031934/0855 Effective date: 20131220 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |