WO2012036753A1 - Control of an electronic apparatus using micro-impulse radar - Google Patents

Control of an electronic apparatus using micro-impulse radar Download PDF

Info

Publication number
WO2012036753A1
WO2012036753A1 PCT/US2011/001629 US2011001629W WO2012036753A1 WO 2012036753 A1 WO2012036753 A1 WO 2012036753A1 US 2011001629 W US2011001629 W US 2011001629W WO 2012036753 A1 WO2012036753 A1 WO 2012036753A1
Authority
WO
WIPO (PCT)
Prior art keywords
computer
micro
impulse radar
selecting
movement
Prior art date
Application number
PCT/US2011/001629
Other languages
French (fr)
Inventor
Mahalaxmi Gita Bangera
Roderick A. Hyde
Muriel Y. Ishikawa
Edward K.Y. Jung
Jordin T. Kare
Eric C. Leuthardt
Nathan P. Myhrvold
Elizabeth A. Sweeney
Clarence T. Tegreene
David B. Tuckerman
Jr. Lowell L. Wood
Victoria Y.H. Wood
Original Assignee
Searete Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Searete Llc filed Critical Searete Llc
Publication of WO2012036753A1 publication Critical patent/WO2012036753A1/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/0209Systems with very large relative bandwidth, i.e. larger than 10 %, e.g. baseband, pulse, carrier-free, ultrawideband
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/04Systems determining presence of a target
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/41Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G01S7/415Identification of targets based on measurements of movement associated with the target
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/422Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42201Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213Monitoring of end-user related data
    • H04N21/44218Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program

Definitions

  • a computer with micro-impulse radar (MIR) feedback includes a processing unit including processing hardware and an operating system configured to run one or more application programs.
  • a display under control of the processing unit is configured to display images to a person located in a first region.
  • a MIR is operatively coupled to the processing unit and configured to probe a second region to detect all or a portion of one or more individuals, and produce a corresponding MIR signal.
  • At least one of the processing hardware, operating system, or application program is configured to receive information associated with the MIR signal and determine content or characteristics of the images displayed on the display responsive to one or more characteristics of the MIR information.
  • a computer method includes receiving a sequence of MIR signals corresponding to a person, extracting at least one of a physiological or movement parameter from the sequence of MIR signals, correlating the physiological or movement parameter to a predicted emotional state of the person, and conditionally selecting a program execution path responsive to the predicted emotional state of the person.
  • a tangible computer-readable medium carries computer-executable instructions that cause a computer to receive a sequence of MIR signals corresponding to a person, extract at least one of a physiological or movement parameter from the sequence of MIR signals correlate the physiological or movement parameter to a predicted emotional state of the person, and conditionally select a program execution path responsive to the predicted emotional state of the person.
  • a method for controlling a computer includes receiving one or more MIR signals from one or more regions; performing analysis on the one or more MIR signals to determine presence, movement, and/or at least one physiological process of each of one or more persons in the one or more regions; and selecting an operation parameter of at least one computer responsive to the presence movement, and/or the physiological process(es) of the one or more persons.
  • a tangible computer-readable medium carries computer-executable instructions that cause a computer to receive one or more MIR signals from one or more regions; perform analysis on the one or more MIR signals to determine presence, movement, and/or at least one physiological process of each of one or more persons in the one or more regions; and responsively select an operation parameter of one or more computers (which may include only the computer executing the instructions).
  • an entertainment system is configured to adapt to personal preferences.
  • the entertainment system includes a media output apparatus, such as a video monitor and/or loudspeakers, configured to present media content according to one or more program options.
  • a MIR is operatively coupled to the media output apparatus and is configured to probe a region proximate the media output apparatus and output a MIR signal.
  • a controller is configured to receive the MIR signal, and select the one or more program options responsive to presence, movement, and/or physiological parameter(s) corresponding to one or more persons in the probed region.
  • FIG. 1 is a simplified block diagram of a micro-impulse radar (MIR), according to an embodiment.
  • MIR micro-impulse radar
  • FIG. 2 is a flow chart showing an illustrative process for determining the presence of a person in a region with the MIR of FIG. 1, according to an embodiment.
  • FIG. 3 is a flow chart showing an illustrative process for determining a physiological parameter of a person in a region with the MIR of FIG. 1, according to an embodiment.
  • FIG. 4 is a block diagram of a system including a computer with MIR feedback, according to an embodiment.
  • FIG. 5 is a block diagram of a system including a computer with MIR feedback, according to another embodiment.
  • FIG. 6 is a block diagram of a computer architecture having an operatively coupled MIR, according to an embodiment.
  • FIG. 7 is a flow chart illustrating a method for controlling a computer using parameters determined by a MIR, according to an embodiment.
  • FIG. 8 is a flow chart illustrating a method for operating a computer responsive to an emotional state of a person, according to an embodiment.
  • FIG. 9 is a block diagram of an entertainment system configured to adapt to personal preferences, according to an embodiment.
  • FIG. 1 is a simplified block diagram of a micro-impulse radar (MIR) 101 , according to an embodiment.
  • a pulse generator 102 is configured to output a relatively short voltage pulse that is applied to a transmit antenna 104.
  • a typical transmitted pulse width can be between about two hundred picoseconds and about 5 nanoseconds, for example.
  • the voltage pulse can be conditioned and amplified (or attenuated) for output by a transmitter 108.
  • the transmitter 108 can transmit the voltage pulse or can further condition the pulse, such as by differentiating a leading and/or trailing edge to produce a short sub-nanosecond transmitted pulses.
  • the voltage pulse is typically not modulated onto a carrier frequency. Rather, the voltage pulse transmission spectrum is the frequency domain transform of the emitted pulse.
  • the MIR 101 can probe a region 1 10 by emitting a series of spaced voltage pulses.
  • the series of voltage pulses can be spaced between about 100 nanoseconds and 100 microseconds apart.
  • the pulse generator 102 emits the voltage pulses with non-uniform spacing such as random or pseudo-random spacing, although constant spacing can be used if interference or compliance is not a concern. Spacing between the series of voltage pulses can be varied responsive to detection of one or more persons 1 12 in the region 1 10. For example, the spacing between pulses can be relatively large when a person 1 12 is not detected in the region 1 12. Spacing between pulses can be decreased (responsive to one or more commands from a controller 106) when a person 1 12 is detected in the region 1 10. For example, the decreased time between pulses can result in faster MIR data generation for purposes of more quickly determining information about one or more persons 1 12 in the region 1 10.
  • the emitted series of voltage pulses can be characterized by spectral components having high penetration that can pass through a range of materials and geometries in the region 1 10.
  • An object 1 12 (such as a person) in the probed region 1 10 can selectively reflect, refract, absorb, and/or otherwise scatter the emitted pulses.
  • a return signal including a reflected, refracted, absorbed, and/or otherwise scattered signal can be received by a receive antenna 1 14.
  • the receive antenna 1 14 and transmit antenna 104 can be combined into a single antenna.
  • a filter (not shown) can be used to separate the return signal from the emitted pulse.
  • a probed region 1 10 can be defined according to an angular extent and distance from the transmit antenna 104 and the receive antenna 1 14. Distance can be determined by a range delay 1 16 configured to trigger a receiver 1 18 operatively coupled to the receive antenna 1 14.
  • the receiver 1 18 can include a voltage detector such as a capture-and-hold capacitor or network.
  • the range delay corresponds to distance into the region 1 10. Range delay can be modulated to capture information corresponding to different distances.
  • a signal processor 120 can be configured to receive detection signals or data from the receiver 1 18 and the analog to digital converter 122, and by correlating range delay to the detection signal, extract data corresponding to the probed region 1 10 including the object 1 12.
  • the MIR 101 can include a second receive antenna 1 14b.
  • the second receive antenna can be operatively coupled to a second receiver 1 18b coupled to an output of the range delay 1 16 or a separate range delay (not shown) configured to provide a delay selected for a depth into the region 1 10.
  • the signal processor 120 can further receive output from a second A/D converter 122b operatively coupled to the second receiver 1 18b.
  • the signal processor 120 can be configured to compare detection signals received by the antennas 1 14, 1 14b. For example, the signal processor 120 can search for common signal characteristics such as similar reflected static signal strength or spectrum, similar (or corresponding) Doppler shift, and/or common periodic motion components, and compare the respective range delays corresponding to detection by the respective antennas 1 14, 1 14b. Signals sharing one or more characteristics can be correlated to triangulate to a location of one or more objects 1 12 in the region 1 10 relative to known locations of the antennas 1 14, 1 14b. The triangulated locations can be output as computed ranges of angle or computed ranges of extent.
  • a first signal corresponding to a reflected pulse received by an antenna element 1 14 can be digitized by an analog-to-digital converter (A/D) 122 to form a first digitized waveform.
  • a second signal corresponding to the reflected pulse received by a second antenna element 1 14b can similarly be digitized by and A/D 122b (or alternatively by the same A/D converter 122) to form a second digitized waveform.
  • the signal processor 120 can compare the first and second digitized waveforms and deduce angular information from the first and second digitized waveforms and known geometry of the first and second antenna elements.
  • a second pulse can be received at a second range delay 1 16 value and can be similarly signal processed to produce a second set of angular information that maps a second surface at a different distance. Depth within a given range delay can be inferred from a strength of the reflected signal. A greater number of signals can be combined to provide additional depth information. A series of pulses can be combined to form a time series of signals corresponding to the object 1 12 that includes movement information of the object 1 12 through the region 1 10.
  • the object 1 12 described herein can include one or more persons.
  • the signal processor 120 outputs MIR data.
  • the MIR data can include object location information, object shape information, object velocity information, information about inclusion of high density and/or conductive objects such as jewelry, cell phones, glasses including metal, etc., and physiological information related to periodic motion.
  • the MIR data can include spatial information, time-domain motion information, and/or frequency domain information.
  • the MIR data can be output in the form of an image.
  • MIR data in the form of an image can include a surface slice made of pixels or a volume made of voxels.
  • the image can include vector information.
  • the MIR data from the signal processor 120 is output to a signal analyzer 124.
  • the signal analyzer 124 can be integrated with the signal processor 120 and/or can be included in the same MIR 101 , as shown.
  • the signal processor 120 can output MIR data through an interface to a signal analyzer 124 included in an apparatus separate from the MIR 101.
  • a signal analyzer 124 can be configured to extract desired information from MIR data received from the signal processor 120. Data corresponding to the extracted information can be saved in a memory for access by a data interface 126 or can be pushed out the data interface 126.
  • the signal analyzer 124 can be configured to determine the presence of a person 1 12 in the region 1 10.
  • MIR data from the signal processor can include data having a static spectrum at a location in the region 1 10, and a periodic motion spectrum corresponding to the location characteristic of a human physiological process (e.g.
  • the signal analyzer 124 can be configured to determine a number of persons 1 12 in the region 1 10.
  • the signal analyzer 124 can be configured to determine the size of a person and/or relative size of anatomical features of a person 1 12 in the region 1 10.
  • the signal analyzer 124 can be configured to determine the presence of an animal 1 12 in the region 1 10.
  • the signal analyzer 124 can be configured to determine movement and/or speed of movement of a person 1 12 through the region 1 10.
  • the signal analyzer 124 can be configured to determine or infer the orientation of a person 1 12 such as the direction a person is facing relative to the region 1 10.
  • the signal analyzer 124 can be configured to determine one or more physiological aspects of a person 1 12 in the region 1 10.
  • the signal analyzer 124 can determine presence of a personal appliance such as a cell phone, PDA, etc. and/or presence of metallized objects such as credit cards, smart cards, access cards, etc.
  • the signal analyzer 124 can infer the gender and age of one or more persons based on returned MIR data.
  • male bodies can generally be characterized by higher mass density than female bodies, and thus can be characterized by somewhat greater reflectivity at a given range.
  • Adult female bodies can exhibit relatively greater harmonic motion ("jiggle") responsive to movements, and can thus be correlated to harmonic spectra characteristics. Older persons generally move differently than younger persons, allowing an age inference based on detected movement in the region 1 10.
  • the signal analyzer 124 can determine a demographic of one or more persons 1 12 in the region 1 10.
  • MIR data can include movement corresponding to the beating heart of one or more persons 1 12 in the region 1 10.
  • the signal analyzer 124 can filter the MIR data to remove information not corresponding to a range of heart rates, and determine one or more heart rates by comparing movement of the heart surface to the MIR signal rate.
  • the one or more heart rates can further be characterized according to a confidence factor, depending on statistical certainty regarding the determined one or more heart rates.
  • the signal analyzer 124 can determine one or more respiration rates by measuring movement corresponding to the chest or diaphragm of one or more persons 1 12.
  • the signal analyzer 124 can determine movement, a direction of movement, and/or a rate of movement of one or more persons 1 12 in the region 1 10. Operation of the signal analyzer 124 is described in greater detail below by reference to FIGS. 2 and 3.
  • An electronic controller 106 can be operatively coupled to the pulse generator 102, the transmitter 108, the range delay 1 16, the receiver 1 18, the analog-to-digital converter 122, the signal processor 120, and/or the signal analyzer 124 to control the operation of the components of the MIR 101.
  • the electronic controller 106 can also be operatively coupled to the second receiver 1 18b, and the second analog-to-digital converter 122b.
  • the data interface 126 can include a high speed interface configured to output of data from the signal analyzer 124. Alternatively, for cases where signals are analyzed externally to the MIR, the data interface 126 can include a high speed interface configured to output MIR data from the signal processor 120.
  • the data interface 126 can include an interface to the controller 106.
  • the controller 106 can be interfaced to external systems via a separate interface (not shown).
  • FIG. 2 is a flow chart showing an illustrative process 201 for determining the presence of one or more persons 1 12 in the region 1 10 with the signal analyzer 124 of the MIR 101 , according to an embodiment.
  • MIR data is received as described above in conjunction with FIG. 1.
  • the MIR data can correspond to a plurality of probes of the region 1 10.
  • the MIR data can be enhanced to facilitate processing. For example, grayscale data corresponding to static reflection strength as a function of triangulated position can be adjusted, compressed, quantized, and/or expanded to meet a desired average signal brightness and range.
  • velocity information corresponding to Doppler shift, and/or frequency transform information corresponding to periodically varying velocity can similarly be adjusted, compressed, quantized, and/or expanded.
  • Systematic, large scale variations in brightness can be balanced, such as to account for side-to-side variations in antenna coupling to the region. Contrast can be enhanced such as to amplify reflectance variations in the region.
  • a spatial filter can be applied.
  • Application of a spatial filter can reduce processing time and/or capacity requirements for subsequent steps described below.
  • the spatial filter may, for example, include a computed angle or computed extent filter configured to remove information corresponding to areas of contrast, velocity, or frequency component(s) having insufficient physical extent to be large enough to be an object of interest.
  • the spatial filter may, for example, identify portions of the region 1 10 having sufficient physical extent to correspond to body parts or an entire body of a person 1 12, and remove features corresponding to smaller objects such as small animals, leaves of plants, or other clutter.
  • the spatial filter can remove information corresponding to areas of contrast, velocity, or frequency component(s) having physical extent greater than a maximum angle or extent that is likely to correspond to a person or persons 1 12.
  • the spatial filter applied in step 206 can eliminate small, low contrast features, but retain small, high contrast features such as jewelry, since such body ornamentation can be useful in some subsequent processes.
  • the step of applying the spatial filter 206 can further include removing background features from the MIR data. For example, a wall lying between an antenna 104, 1 14 and the region 1 10 can cast a shadow such as a line in every MIR signal. Removal of such constant features can reduce subsequent processing
  • an edge-finder can identify edges of objects 1 12 in the region 1 10. For example, a global threshold, local threshold, second derivative, or other algorithm can identify edge candidates. Object edges can be used, for example, to identify object shapes, and thus relieve subsequent processes from operating on grayscale data. Alternatively, step 208 can be omitted and the process of identifying objects can be performed on the grayscale MIR data.
  • processed data corresponding to the MIR data is compared to a database to determine a match.
  • the object data received from step 202 can be compared to corresponding data for known objects in a shape database.
  • Step 210 can be performed on a grayscale signal, but for simplicity of description it will be assumed that optional step 208 was performed and matching is performed using object edges, velocity, and/or spectrum values.
  • the edge of an object 1 12 in the region 1 10 can include a line corresponding to the outline of the head and torso, cardiac spectrum, and movements characteristic of a young adult male.
  • a first shape in the shape database can include the outline of the head and torso, cardiac spectrum, density, and movements characteristic of a young adult female and/or the head and torso outline, cardiac spectrum, density, and movements characteristic of a generic human.
  • the differences between the MIR data and the shape database shape can be measured and characterized to derive a probability value. For example, a least-squares difference can be calculated.
  • the object shape from the MIR data can be stepped across, magnified, and stepped up and down the shape database data to minimize a sum-of-squares difference between the MIR shape and the first shape in the shape database.
  • the minimum difference corresponds to the probability value for the first shape.
  • step 212 if the probability value for the first shape is the best probability yet encountered, the process proceeds to step 214.
  • the first probability value is the best probability yet encountered. If an earlier tested shape had a higher probability to the MIR data, the process loops back from step 212 to step 210 and the fit comparison is repeated for the next shape from the shape database.
  • the object type for the compared shape from the shape database and the best probability value for the compared shape are temporarily stored for future comparison and/or output.
  • the compared shape from the shape database can be identified by metadata that is included in the database or embedded in the comparison data. Proceeding to step 216, the process either loops back to step 210 or proceeds to step 218, depending on whether a test is met. If the most recently compared shape is the last shape available for comparison, then the process proceeds to step 218. Optionally, if the most recently compared shape is the last shape that the process has time to compare (for example, if a new MIR data is received and/or if another process requires output data from the process 201) then the process proceeds to step 218. In step 218, the object type and the probability value is output. The process can then loop back to step 202 and the process 201 can be repeated.
  • the process 201 loops from step 216 back to step 210.
  • the next comparison shape from a shape database is loaded.
  • the comparison can proceed from the last tested shape in the shape database. In this way, if the step 218 to 202 loop occurs more rapidly than all objects in the shape database can be compared, the process eventually works its way through the entire shape database.
  • the shape database can include multiple copies of the same object at different orientations, distances, and positions within the region. This can be useful to reduce processing associated with stepping the MIR shape across the shape database shape and/or changing magnification.
  • the object type can include determination of a number of persons 1 12 in the region 1 10.
  • the shape database can include outlines, cardiac and/or respiration spectra, density, and movement characteristics for plural numbers of persons.
  • the shape library can include shapes not corresponding to persons. This can aid in identification of circumstances where no person 212 is in the region 210.
  • process 201 can be performed using plural video frames such as averaged video frames or a series of video frames.
  • steps 212, 214, and 216 can be replaced by a single decision step that compares the probability to a predetermined value and proceeds to step 218 if the probability meets the predetermined value. This can be useful, for example, in embodiments where simple presence or absence of a person 212 in the region 210 is sufficient information.
  • the signal analysis process 201 of FIG. 2 can be performed using conventional software running on a general-purpose microprocessor.
  • the process 201 using various combinations of hardware, firmware, and software and can include use of a digital signal processor.
  • FIG. 3 is a flow chart showing an illustrative process 301 for determining one or more particular physiological parameters of a person 1 12 in the region 1 10 with the signal analyzer 124 of the MIR 101 , according to an embodiment.
  • the process 301 of FIG. 3 can be performed conditional to the results of another process such as the process 201 of FIG. 2. For example, if the process 201 determines that no person 1 12 is in the region 1 10, then it can be preferable to continue to repeat process 201 rather than execute process 301 in an attempt to extract one or more particular physiological parameters from a person that is not present.
  • a series of MIR time series data is received. While the received time series data need not be purely sequential, the process 301 generally needs the time series data received in step 302 to have a temporal capture relationship appropriate for extracting time-based information.
  • the MIR time series data can have a frame rate between about 16 frames per second and about 120 frames per second. Higher capture rate systems can benefit from depopulating frames, such as by dropping every other frame, to reduce data processing capacity requirements.
  • step 304 the MIR video frames can be enhanced in a manner akin to that described in conjunction with step 204 of FIG. 2.
  • step 304 can include averaging and/or smoothing across multiple MIR time series data.
  • a frequency filter can be applied. The frequency filter can operate by comparing changes between MIR time series data to a reference frequency band for extracting a desired physical parameter. For example, if a desired physiological parameter is a heart rate, then it can be useful to apply a pass band for periodic movements having a frequency between about 20 cycles per minute and about 200 cycles per minute, since periodic motion beyond those limits is unlikely to be related to a human heart rate.
  • step 304 can include a high pass filter that removes periodic motion below a predetermined limit, but retains higher frequency information that can be useful for determining atypical physiological parameters.
  • a spatial filter can be applied.
  • the spatial filter may, for example, include a pass band filter configured to remove information corresponding to areas of contrast having insufficient physical extent to be large enough to be an object of interest, and remove information corresponding to areas too large to be an object of interest.
  • the spatial filter may, for example, identify portions of the region 1 10 having sufficient physical extent to correspond to the heart, diaphragm, or chest of a person 1 12, and remove signal features corresponding to smaller or larger objects.
  • the step of applying the spatial filter 308 can further include removing background features from the MIR data. For example, a wall lying between an antenna 104, 1 14 (1 14b) and the region 1 10 can cast a shadow such as a line in every instance of MIR data. Removal of such constant features can reduce subsequent processing requirements.
  • movement such as periodic movement in the MIR time series data is measured.
  • a periodic motion is to be measured, a time- to-frequency domain transform can be performed on selected signal elements.
  • a rate of movement of selected signal elements can be determined.
  • periodic and/or non-periodic motion can be measured in space vs. time.
  • Arrhythmic movement features can be measured as spread in frequency domain bright points or can be determined as motion vs. time.
  • subsets of the selected signal elements can be analyzed for arrhythmic features.
  • plural subsets of selected signal elements can be cross-correlated for periodic and/or arrhythmic features.
  • one or more motion phase relationships between plural subsets of selected signal features, between a subset of a selected signal feature and the signal feature, or between signal features can be determined.
  • a person with a hiccup can be detected as a non-periodic or arrhythmic motion superimposed over periodic motion of a signal element corresponding to the diaphragm of the person.
  • a physiological parameter can be calculated. For example,
  • Step 312 can include determining one or more heart rates by comparing movement of the heart surface to the MIR signal rate. The one or more heart rates can further be characterized according to a confidence factor, depending on statistical certainty regarding the determined one or more heart rates. Similarly, step 312 can include determining one or more respiration rates by measuring movement corresponding to the chest or diaphragm of one or more person.
  • the physiological parameter can be output. Proceeding to step 316, if there are more locations to measure, the process 301 can loop back to execute step 308. If there are not more locations to measure, the process can proceed to step 318. In step 318, if there are more physiological parameters to measure, the process 301 can loop back to execute step 306. If there are not more physiological parameters to measure, the process 301 can loop back to step 302, and the process 301 of FIG. 3 can be repeated.
  • FIG. 4 is a block diagram of a system 401 including a computer with MIR feedback, according to an embodiment.
  • the computer with micro-impulse radar feedback 401 includes a processing unit 402 including processing hardware and an operating system configured to run one or more application programs.
  • a display 404 is under control of the processing unit 402 and configured to display images to a person 1 12 located in a first region 406. The first region 406 can be considered the viewing region.
  • a MIR 101 is operatively coupled to the processing unit 402 and configured to probe a second region 1 10 to detect all or a portion of one or more individuals 1 12.
  • the one or more individuals 1 12 can include a user of the computer system 401. The user can input commands to run the operating system and/or application programs on the processing unit 402.
  • the MIR 101 is configured to produce a MIR signal.
  • the MIR signal can be substantially the same as MIR data output by the signal processor 120 described above.
  • at least a portion of the MIR signal processing and/or analysis can be performed by the computer processing unit 402, and the MIR signal can be more primitive than output of MIR analysis.
  • the MIR signal can correspond to a signal output by the signal analyzer 122.
  • At least one of the processing hardware, operating system, or application program portions of the processing unit 402 is configured to receive information associated with the MIR signal and determine content or characteristics of the images displayed on the display 404 responsive to one or more characteristics of the micro-impulse radar information.
  • the first region 406 and second region 1 10 can be substantially coincident, such as when a majority of the regions 406 and 1 10 overlap.
  • the system 501 can alternatively be configured such that the second region 1 10 is a portion of the first region 406.
  • the MIR probe region 1 10 can be configured to measure characteristics of a portion 1 12a of the person 1 12. If the portion 1 12a of the person 1 12 corresponds to a hand, for example, the MIR 101 can receive signals corresponding to hand gestures such as gestures corresponding to operation of a virtual pointing device, operation of a virtual keyboard, American Sign Language (ASL), or other gesture convention.
  • a person 1 12 shall be understood to include one or more portions 1 12a.
  • the first region 406 can be a portion of the second region 1 10.
  • the MIR can be configured to probe a second region 1 10 that is larger than the region 406 from which the display 404 can be viewed.
  • the computer processing unit 402 can be configured to select a parameter and output an image on the display 404 in anticipation of one or more persons 1 10 entering the viewing region 406.
  • the first and second regions 406, 1 10 can be substantially non-coincident.
  • the MIR can probe a person 1 12 traveling through a second region 1 10, and the computer processing unit 402 can output an image to the display 404 at a time
  • the MIR signals can include a MIR image.
  • MIR signals can correspond to MIR data, such as MIR data output by the signal processor 120.
  • the MIR 101 of FIGS. 4 and 5 can include a transmitter 108 configured to transmit electromagnetic pulses toward the second region 1 12.
  • a pulse delay gate 1 16 can be configured to delay the pulses to trigger at least one receiver 1 18.
  • the receiver 1 18 can be synchronized to the pulse delay gate 1 16 and configured to receive electromagnetic energy scattered from the pulses as they encounter objects, such as one or more persons 1 12, in the region 1 10.
  • a signal processor 120 can be configured to receive signals or data from the receiver 1 18 and perform signal processing on the signals or data to extract one or more signals corresponding to at least one of human presence, human movement, or human physiological processes.
  • the MIR 101 can be configured as a separate device from and operatively coupled to the computer processing unit 402.
  • the MIR 101 can be configured to communicate with the computer process via an exposed interface such as usb, IEEE 802.1 lx, line level inputs, or other conventional data interface.
  • the MIR 101 can be integrated into the computer processing unit 402.
  • Various levels of integration and partitioning are contemplated.
  • the signal processor 120 can be integrated into the computer processing unit 402.
  • the signal processor 120 can be integrated into the MIR 101 . This distinction may be moot in cases where the MIR 101 is integrated into the computer processing unit 402.
  • the signal processor 120 can be configured as a portion of the processing hardware.
  • the signal processor 120 can be embodied as software operable to run on the processing unit 402, a relatively low cost solution when the microprocessor (s) 604 has sufficient bandwidth.
  • the signal processor 120 can similarly include both dedicated hardware and computer executable instructions operable to run on the processing hardware.
  • FIG. 6 is a block diagram of a computer architecture 602 having an operatively coupled MIR 101 , according to an embodiment.
  • the computer 602 typically includes a microprocessor 604 operatively coupled to computer memory 606 (which includes tangible computer-readable media capable of carrying computer-readable instructions), computer storage 608 including a storage medium 610 that forms a tangible computer- readable medium capable of carrying computer-readable instructions, a data interface 612, and one or more human interfaces 614.
  • the human interface can include a keyboard, a computer pointing device such as a mouse, a touch screen, and/or a microphone.
  • the MIR 101 can operate as a human interface and can augment or replace one or more conventional human interface apparatuses 614.
  • the computer 602 can include or be operatively coupled to a display 404 and/or one or more additional output apparatuses (not shown) configured to output program output and/or media to one or more persons 1 12.
  • the display 404 and/or one or more additional output apparatuses (not shown) are configured to output information, entertainment, etc. to the one or more persons 1 12 in a first region 406.
  • the MIR 101 is configured to probe a second region 1 10 and responsively output an MIR signal or MIR data corresponding to presence, movement, and/or physiological processes of one or more persons 1 12 in a region 406.
  • the regions 1 10 and 406 can be substantially coincident, overlapping, disjointed, and/or one region can be a subset of the other region.
  • the MIR can be embedded in a computer motherboard.
  • the MIR 101 can be configured as an expansion card such as a card compliant with ISA, PCI, PCI Express, NuBus, or other standard.
  • the MIR can be physically separate from and operatively coupled to the computer 602, such as through an exposed interface (not shown).
  • the MIR 101 can include an integrated signal processor, which can include Fourier transformation hardware or software.
  • signal processing can be performed using software running on the hardware 602 represented in FIG. 6.
  • the MIR 101 and/or the computer 602 can include a signal analyzer configured to receive signals or data from the signal processor and to perform signal analysis to determine, from the one or more extracted signals, variable data corresponding to human presence, human movement, human posture, and/or the human physiological processes.
  • the computer 602 can control media content and/or program execution responsive to the received variable data corresponding to human presence, human movement, human posture, and/or the human physiological processes.
  • the MIR 101 can include an interface operatively coupled to a signal analyzer and configured to output MIR data including variable data corresponding to human 1 12 presence, human movement, and/or human physiological processes to the processing unit 602.
  • a signal processor can be configured to output MIR data corresponding to human 1 12 presence, human movement, and/or human physiological processes to one or more memory circuit 606 or storage device 608, 610 locations.
  • An operating system running on the computer 602 can be configured to read at least a subset of the MIR data at the one or more memory circuit 606 or storage device 608, 610 locations and responsively adjust one or more operating system parameters. Responsive adjustment of one or more operating system parameters may, for example, include waking the operating system from a sleep mode responsive to MIR data corresponding to human presence in the second region 1 10. Similarly, adjustment of one or more operating system parameters can include entering a sleep mode responsive to MIR data corresponding to departure of an individual 1 12 from the second region 1 10. Additionally or alternatively, an application running on the computer 602 can be configured to read at least a subset of the MIR data at the one or more memory circuit 606 or storage device 608, 610 locations and responsively adjust one or more application parameters.
  • adjustment one or more operating system parameters or application parameters can include selection of terse prompts or fast response responsive to MIR data corresponding to human 1 12 movement or physiological processes characteristic of impatience or urgency. Additionally or alternatively, adjustment of one or more operating system or application parameters can include conversion of the MIR data into cursor movements. For example, adjustment of one or more application or operating system parameters can includes conversion of the MIR data into computer pointer device 614 commands.
  • an application program running on the computer 602 can be configured to read at least a subset of the MIR data at the one or more memory circuit 606 or storage device 608, 610 locations and responsively adjust application program parameters.
  • an application program can be configured to convert the MIR data to program commands.
  • the application program can be configured to convert the MIR data to one or more preferences selections.
  • one or more preferences selections can include automated help, terse response, verbose response, and/or video magnification.
  • information associated with the MIR signal can correspond to a human 1 12 physiological process.
  • a physiological process can include heartbeat and/or breathing, for example.
  • An application program and/or operating system can further be configured to correlate the physiological process to a predicted emotional state of an individual 1 12.
  • the operating system and/or application program can be configured to conditionally select a program execution path as a function of the predicted emotional state.
  • FIG. 7 is a flow chart illustrating a method 701 for controlling a computer using parameters determined by a MIR, according to an embodiment.
  • a first section 702 of the method 701 corresponds to generation of an operating parameter for controlling an operating system and/or application program.
  • Section 702 can be performed by hardware, by an application program, and/or by an operating system, for example.
  • one or more MIR signals is received from one or more regions (e.g., one or more regions 1 10).
  • the MIR signals can include a MIR image.
  • the MIR signals can correspond to MIR data.
  • step 706 analysis is performed on the one or more MIR signals to determine presence, movement, and/or one or more physiological processes of one or more persons in the one or more regions.
  • the one or more persons can include a user of the computer.
  • Approaches to determining presence, movement, posture, and/or physiological processes of persons are described above in conjunction with FIGS. 1-3, for example.
  • physiological processes can include heartbeat, perspiration, tremor, and/or breathing
  • corresponding physiological parameters can include heart rate, moistness of the skin, a periodic movement or shaking, or breathing rate.
  • an operating parameter which can include one or more operating parameters, is selected and output for at least one computer responsive to the presence, movement, and/or one or more physiological processes of the one or more persons in the one or more regions.
  • a physiological parameter can be indicative of a physiological state of the person and determine the physiological state for the computer.
  • Determining an operating parameter can include correlating at least one physiological parameter to a predicted emotional state.
  • correlating the at least one physiological parameter to the predicted emotional state can include correlating the physiological parameter to an autonomic nervous system state and correlating the autonomic nervous system state to the predicted emotional state.
  • a posture can be indicative of a physiological state and/or an intent. For example a person lying prone may be unconscious, asleep, or relaxed. A person who is hunched or collapsed on the ground may be injured, sick, or emotional. A stance may be threatening or precarious. A movement may indicate good balance or a lack of balance. Another movement and/or tremor may correspond to laughing, coughing, or sneezing. A detected movement stability, periodicity, and/or tremor man indicate decreased motor control, shaking, trembling, or undergoing a seizure. Such a posture or movement may be conscious or may be unconscious. Embodiments herein include driving an electronic apparatus responsive to these and other indications of a command, a condition, or an intent that may be extracted from or included in MIR data.
  • a data value corresponding to the operating parameter can be written to computer memory or storage device.
  • line levels can be set at pins of a
  • microprocessor or other hardware a combination of one or more line levels being set according to the analyzed MIR signals.
  • a number of parameters and/or line levels can be selected to correspond to the sophistication of control desired. For example, simple presence or absence of a person in a region can be communicated to the computer by setting a bit or a single line level.
  • a larger number of bits or lines, or a greater number of variables can be set to correspond to control the computer with greater sophistication.
  • an X-Y location or velocity of a digit of a hand can be represented to high precision by two bytes for each axis.
  • a vertical position or velocity of the digit can be represented by as little as a single bit if software filtering is not desired.
  • operating parameters can be determined or output corresponding to one or more of other presence data such as a probability of presence of a particular person, movement such as body movements and/or velocity intended to control a computer game, and/or
  • physiological parameters such as parameters corresponding to heart rate and/or breathing depth and rate.
  • the process section 702 After outputting one or more operating parameters, the process section 702 loops back to step 704 and is repeated.
  • the frequency of looping can be determined according to an operating state. For example, if the system is in a sleep mode and is intended to wake up when a person enters a region, looping can be relatively slow, such as once per second. On the other hand, if the computer is in active operation with movements used to control a cursor, then looping can be relatively rapid, such as once per millisecond to once per 100 milliseconds, depending on desired responsiveness, bandwidth, and/or preferred limits on MIR probe radiation.
  • Section 710 of the process 701 can be performed asynchronously from process 702. For example, section 710 can be performed by a device driver, API, or other operating system-related process.
  • the operating parameter is read. Proceeding to step 714, the new operating parameter can be compared to previous values of the operating parameter or a function of previous values of the operating parameter, if the operating parameter has not changed (e.g. in a statistically significant way), the process can loop back through an optional process 718 and then to step 712.
  • the optional process 718 may, for example, perform filtering such as debounce or other noise reduction.
  • the process 718 can be programmable, with its functionality determined by context. In this way, the test performed in step 714 can be required to be performed a plurality of times prior to the condition being determined true. If step 714 is determined to be true, the process proceeds to step 716, where the parameter is loaded.
  • loading the parameter can include writing a value (either the value of the operating parameter or a corresponding data value) into a memory or storage location that can be accessed by other processes, or can include setting one or more line levels on pins of a microprocessor or other hardware.
  • selecting an operation parameter and/or subsequent performance of section 710 of process 701 can include waking portions of the computer when the one or more persons enters the one or more regions.
  • performing analysis on a sequence of micro- impulse radar signals can determine a probability that the one or more persons has an intent to use the computer. Determining the probability that the one or more persons has an intent to use the computer can include analyzing movements of the one or more persons. For example, the one or more persons can be determined to have a high probability of intent to use the computer when the one or more persons enter a computer operating position. For example, one or more persons can be determined to have a high probability of intent to use the computer when the one or more persons approach near to or reach for a computer keyboard or a computer pointing device and/or approach the computer display.
  • selecting an operation parameter and/or subsequent performance of section 710 of process 701 can including placing one or more portions of the computer in a sleep mode when the one or more persons leaves the one or more regions.
  • performing analysis on a sequence of MIR signals can determine a probability that the one or more persons has an intent to stop using the computer.
  • step 716 When a parameter or parameters are loaded in step 716, the parameter or parameters can be read by an operating system and/or one or more application programs. Proceeding to step 720, which can be performed asynchronously from section 710 of the process 701 , program execution can be performed responsive to the loaded parameter or parameters.
  • selecting an operation parameter includes conditionally selecting computer program logic.
  • sections 702 and 710 can include analyzing movements of the one or more persons. Loading the parameter(s) in step 716 can amount to entering one or more software commands responsive to the movements.
  • the one or more software commands can include commands to operate game software on the computer.
  • the one or more software commands can be selected to drive virtual movements of one or more game entities corresponding to the movements of the one or more persons.
  • the method described in conjunction with FIG. 7 can be physically embodied as computer executable instructions carried by a tangible computer readable medium.
  • the method shown in FIG. 7 can be used in apparatuses other than general purpose computers, such as video gaming systems, entertainment systems, and/or other systems that benefit from MIR input to electronic processing functions.
  • FIG. 8 is a flow chart illustrating a method 801 for operating a computer responsive to an emotional state of a person, according to an embodiment.
  • the person can be a user of the computer.
  • a sequence of MIR signals is received, the signals including characteristics corresponding to a person.
  • the MIR signals can include micro- impulse radar image.
  • the MIR signals can correspond to micro-impulse radar data, such as MIR data produced by a signal analyzer, as described in conjunction with FIGS. 1-3.
  • at least one parameter corresponding to at least one of a physiology, posture, or movement of the person is extracted from the sequence of MIR signals.
  • a physiological parameter can include at least one of
  • physiological parameter (optionally along with posture and/or movement) can determine the physiological state of the person.
  • the at least one parameter extracted in step 804 is correlated to a predicted emotional state of the person.
  • correlating a physiological or movement parameter to the predicted emotional state can include correlating the physiological parameter to an autonomic nervous system state.
  • Step 804 can include labeling the predicted emotional state a positive or negative.
  • a program execution path is conditionally selected responsive to the predicted emotional state of the person.
  • Conditionally selecting a program execution path can include conditionally selecting a program execution path predicted to please the person.
  • Conditionally selecting a program execution path can include to activating a program function correlated to a positive emotional state.
  • conditionally selecting a program execution path can include disabling a program function correlated to a negative emotional state.
  • disabling a program function correlated to a negative emotional state can include disabling an automatic format function correlated to a negative emotional state.
  • step 804 if the person is confused or otherwise uncertain of how to perform a function or what function to perform, he or she can display movement and/or physiological signs such as tensing muscles of the face, cocking his or her head to one side, consulting printed materials near the computer, or otherwise betraying a state of puzzlement, such as can be observable by another person, and which corresponds to detectable information that is extracted from the MIR signals. Accordingly, in step 806, the physiological or movement parameter can be correlated to a prediction of puzzlement.
  • movement and/or physiological signs such as tensing muscles of the face, cocking his or her head to one side, consulting printed materials near the computer, or otherwise betraying a state of puzzlement, such as can be observable by another person, and which corresponds to detectable information that is extracted from the MIR signals.
  • the physiological or movement parameter can be correlated to a prediction of puzzlement.
  • conditionally selecting a program execution path can include activating a help function responsive to the predicted puzzlement.
  • activating a help function can include activating verbose prompts.
  • Verbose prompts can thus provide additional help to aid the user in overcoming his or her puzzlement.
  • conditionally selecting a program execution path can include selecting a path intended to reduce the impatience of the person.
  • terse prompts can be activated responsive to predicting impatience, the terse prompts thus reducing processing and/or interface time, and providing the person less reason for being impatient.
  • the program execution path selected in step 808 can include selecting game logic responsive to the predicted emotional state.
  • the game logic can be selected to provide predicted emotional states including tension and release.
  • Conditionally selecting a program execution path in step 808 responsive to the predicted emotional state of the person can include selecting a game difficulty level responsive to the predicted emotional state.
  • various movement, posture, and/or physiological parameters can indicate different inputs and be used to drive system response in different ways, or in ways that are an interaction product of two or more movement, posture, and/or physiological parameters.
  • an explicit movement of a finger can be used to determine an explicit intent to make a command gesture, while smaller, involuntary movements such as a tremor in the finger can be used to infer a physiological and/or emotional state.
  • posture can indicate a transitory mood, a disposition, and/or an intent of a person.
  • a person taking an aggressive stance can indicate an aggressive mood or, in another context, can drive an aggression parameter in a computer game.
  • a prone posture can indicate relaxation, or in another context, can drive a "play dead" parameter in a computer game.
  • a hunched over posture can indicate physical or emotional distress.
  • interpretation of MIR information can be made context-sensitive, can be disabled, and/or can be used to construct a user profile for future reference.
  • the method described in conjunction with FIG. 8 can be physically embodied as computer executable instructions carried by a tangible computer readable medium.
  • the method shown in FIG. 8 can be used in apparatuses other than general purpose computers, such as video gaming systems, entertainment systems, and/or other systems that benefit from MIR input to electronic processing functions.
  • FIG. 9 is a block diagram of an entertainment system 901 configured to adapt to personal preferences, according to an embodiment.
  • the entertainment system 901 includes a MIR 101 configured to probe a region proximate a media output apparatus and output a MIR signal.
  • the MIR signal can include or consist essentially of MIR data.
  • the MIR signal can include a MIR image.
  • a controller 902 is configured to receive the micro-impulse radar signal from the MIR 101 and select one or more program options responsive to at least one of a presence, movement or a physiological parameter corresponding to one or more persons 1 12 in the probed region.
  • the controller 902 can be configured as an audio and/or video receiver.
  • the MIR 101 can be separate from and operatively coupled to the controller 902 as shown, or optionally the MIR 101 and the controller 802 can be integrated.
  • the controller 902 is configured to drive a media output apparatus 912.
  • the media output apparatus 912 is configured to present media content according to the one or more program options.
  • the media output apparatus 912 can include a television, a video monitor, one or more loudspeakers, a portable audio device, a lighting system, and/or a window treatment actuator.
  • the media output apparatus 912 can include a portable media player carried by the person 1 12.
  • One or more of the MIR 101 and the controller 902 can be thus be located remotely from and operatively coupled to the media output apparatus 912.
  • the entertainment system 901 can include at least one program source 908.
  • Selecting at least one program option can include selecting programming from a plurality of available programming from the at least one program source 908.
  • the program source 908 can include an apparatus for receiving one or more streamed video and/or audio channels, a tuner for receiving one or more television channels, local solid state or rotating drive media track storage, an audio or video server, and/or a single- or multi-disc CD, DVD, or Blue Ray player.
  • One or more program sources 908 can optionally be integrated into the controller 902, or can be operatively coupled to the controller 902 as shown.
  • the at least one program source 908 can optionally be integrated into the output apparatus 912.
  • the controller 902 can include a microcontroller 904.
  • the entertainment system 901 can include at least one codec 906.
  • the codec 906 can optionally be integrated into the controller 902 as shown.
  • the codec 906 can be operatively coupled to the microcontroller 904.
  • the codec 906 can be separate from and operatively coupled to the controller 902.
  • the codec 906 can be integrated into the at least one program source 908.
  • the codec 906 can be integrated into the media output apparatus 912. Selecting at least one program option can include selecting a codec 906 or selecting a codec operating parameter.
  • the entertainment system 901 and the controller 902 can include one or more media output apparatus drivers 910.
  • the one or more media output apparatus drivers 910 can be operatively coupled to the microcontroller 904 and to the at least one program source 908. Selecting at least one program option can include selecting a media output apparatus driver 910 or a media output apparatus driver operating parameter.
  • the one or more media output apparatus driver 910 can include one or more signal amplifiers; and selecting a media output apparatus driver parameter can include selecting an amount of amplification.
  • selecting a media output apparatus driver 910 operating parameter can include selecting between two or more media output apparatuses 912, which can include selecting both or more of the media output apparatuses 912.
  • the entertainment system 901 can be operated in a one or more of a variety of ways, according to embodiments.
  • selecting one or more program options can be performed responsive to presence of one or more persons 1 12. This can include one or more of starting output of media responsive to arrival of one or more persons in the probed region and/or stopping or suspending output of media responsive to departure of one or more persons from the probed region.
  • selecting one or more program options can be performed responsive to movement of one or more persons 1 12. This can include selecting a program option set responsive to detected movement corresponding to attentiveness to a media output apparatus 912.
  • detected movement corresponding to a media output apparatus 912 can include one or more of the one or more persons 1 12 approaching a video display, moving to an optimum listening area in an audio sound field, or remaining still in a viewing area of a video display.
  • the detected movement of the one or more persons 1 12 can include a rhythmic movement of the one or more persons in time with a rhythmic component of a media program.
  • rhythmic movement can include one or more of dancing, head bobbing, or toe tapping.
  • selecting one or more program options can be performed responsive to detecting a physiological parameter corresponding to one or more persons 1 12.
  • a physiological parameter can include a physiological parameter corresponding to a response to media content. Sensing of physiological parameters and changes in physiological parameters is described above.
  • Selecting one or more program options responsive to detected movement or a physiological parameter may, for example, include one or more of increasing audio volume and enabling a video output apparatus responsive to receiving a MIR 101 signal corresponding to attentiveness. Additionally or alternatively, selecting one or more program options responsive to a detected movement or physiological parameter of one or more persons 1 12 corresponding to attentiveness to a media output apparatus 912 can include selecting and/or queuing additional program content having at least one characteristic similar to program content played during the detection of movement or physiological parameter corresponding to attentiveness.

Abstract

A computer or entertainment system is configured to respond to data received from a micro impulse radar configured to detect movement, physiology, presence, and/or absence of a person in one or more regions near the computer or entertainment system.

Description

Control of an Electronic Apparatus Using
Micro-Impulse Radar
SUMMARY
According to an embodiment, a computer with micro-impulse radar (MIR) feedback includes a processing unit including processing hardware and an operating system configured to run one or more application programs. A display under control of the processing unit is configured to display images to a person located in a first region. A MIR is operatively coupled to the processing unit and configured to probe a second region to detect all or a portion of one or more individuals, and produce a corresponding MIR signal. At least one of the processing hardware, operating system, or application program is configured to receive information associated with the MIR signal and determine content or characteristics of the images displayed on the display responsive to one or more characteristics of the MIR information.
According to an embodiment, a computer method includes receiving a sequence of MIR signals corresponding to a person, extracting at least one of a physiological or movement parameter from the sequence of MIR signals, correlating the physiological or movement parameter to a predicted emotional state of the person, and conditionally selecting a program execution path responsive to the predicted emotional state of the person. According to an embodiment, a tangible computer-readable medium carries computer-executable instructions that cause a computer to receive a sequence of MIR signals corresponding to a person, extract at least one of a physiological or movement parameter from the sequence of MIR signals correlate the physiological or movement parameter to a predicted emotional state of the person, and conditionally select a program execution path responsive to the predicted emotional state of the person.
According to an embodiment, a method for controlling a computer includes receiving one or more MIR signals from one or more regions; performing analysis on the one or more MIR signals to determine presence, movement, and/or at least one physiological process of each of one or more persons in the one or more regions; and selecting an operation parameter of at least one computer responsive to the presence movement, and/or the physiological process(es) of the one or more persons.
According to an embodiment, a tangible computer-readable medium carries computer-executable instructions that cause a computer to receive one or more MIR signals from one or more regions; perform analysis on the one or more MIR signals to determine presence, movement, and/or at least one physiological process of each of one or more persons in the one or more regions; and responsively select an operation parameter of one or more computers (which may include only the computer executing the instructions).
According to an embodiment, an entertainment system is configured to adapt to personal preferences. The entertainment system includes a media output apparatus, such as a video monitor and/or loudspeakers, configured to present media content according to one or more program options. A MIR is operatively coupled to the media output apparatus and is configured to probe a region proximate the media output apparatus and output a MIR signal. A controller is configured to receive the MIR signal, and select the one or more program options responsive to presence, movement, and/or physiological parameter(s) corresponding to one or more persons in the probed region.
The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE FIGURES
FIG. 1 is a simplified block diagram of a micro-impulse radar (MIR), according to an embodiment.
FIG. 2 is a flow chart showing an illustrative process for determining the presence of a person in a region with the MIR of FIG. 1, according to an embodiment.
FIG. 3 is a flow chart showing an illustrative process for determining a physiological parameter of a person in a region with the MIR of FIG. 1, according to an embodiment.
FIG. 4 is a block diagram of a system including a computer with MIR feedback, according to an embodiment.
FIG. 5 is a block diagram of a system including a computer with MIR feedback, according to another embodiment.
FIG. 6 is a block diagram of a computer architecture having an operatively coupled MIR, according to an embodiment.
FIG. 7 is a flow chart illustrating a method for controlling a computer using parameters determined by a MIR, according to an embodiment.
FIG. 8 is a flow chart illustrating a method for operating a computer responsive to an emotional state of a person, according to an embodiment.
FIG. 9 is a block diagram of an entertainment system configured to adapt to personal preferences, according to an embodiment.
DETAILED DESCRIPTION In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here.
FIG. 1 is a simplified block diagram of a micro-impulse radar (MIR) 101 , according to an embodiment. A pulse generator 102 is configured to output a relatively short voltage pulse that is applied to a transmit antenna 104. A typical transmitted pulse width can be between about two hundred picoseconds and about 5 nanoseconds, for example. The voltage pulse can be conditioned and amplified (or attenuated) for output by a transmitter 108. For example, the transmitter 108 can transmit the voltage pulse or can further condition the pulse, such as by differentiating a leading and/or trailing edge to produce a short sub-nanosecond transmitted pulses. The voltage pulse is typically not modulated onto a carrier frequency. Rather, the voltage pulse transmission spectrum is the frequency domain transform of the emitted pulse. The MIR 101 can probe a region 1 10 by emitting a series of spaced voltage pulses. For example, the series of voltage pulses can be spaced between about 100 nanoseconds and 100 microseconds apart.
Typically, the pulse generator 102 emits the voltage pulses with non-uniform spacing such as random or pseudo-random spacing, although constant spacing can be used if interference or compliance is not a concern. Spacing between the series of voltage pulses can be varied responsive to detection of one or more persons 1 12 in the region 1 10. For example, the spacing between pulses can be relatively large when a person 1 12 is not detected in the region 1 12. Spacing between pulses can be decreased (responsive to one or more commands from a controller 106) when a person 1 12 is detected in the region 1 10. For example, the decreased time between pulses can result in faster MIR data generation for purposes of more quickly determining information about one or more persons 1 12 in the region 1 10. The emitted series of voltage pulses can be characterized by spectral components having high penetration that can pass through a range of materials and geometries in the region 1 10.
An object 1 12 (such as a person) in the probed region 1 10 can selectively reflect, refract, absorb, and/or otherwise scatter the emitted pulses. A return signal including a reflected, refracted, absorbed, and/or otherwise scattered signal can be received by a receive antenna 1 14. Optionally, the receive antenna 1 14 and transmit antenna 104 can be combined into a single antenna. In a single antenna embodiment, a filter (not shown) can be used to separate the return signal from the emitted pulse.
A probed region 1 10 can be defined according to an angular extent and distance from the transmit antenna 104 and the receive antenna 1 14. Distance can be determined by a range delay 1 16 configured to trigger a receiver 1 18 operatively coupled to the receive antenna 1 14. For example, the receiver 1 18 can include a voltage detector such as a capture-and-hold capacitor or network. The range delay corresponds to distance into the region 1 10. Range delay can be modulated to capture information corresponding to different distances.
A signal processor 120 can be configured to receive detection signals or data from the receiver 1 18 and the analog to digital converter 122, and by correlating range delay to the detection signal, extract data corresponding to the probed region 1 10 including the object 1 12.
Optionally, the MIR 101 can include a second receive antenna 1 14b. The second receive antenna can be operatively coupled to a second receiver 1 18b coupled to an output of the range delay 1 16 or a separate range delay (not shown) configured to provide a delay selected for a depth into the region 1 10. The signal processor 120 can further receive output from a second A/D converter 122b operatively coupled to the second receiver 1 18b.
The signal processor 120 can be configured to compare detection signals received by the antennas 1 14, 1 14b. For example, the signal processor 120 can search for common signal characteristics such as similar reflected static signal strength or spectrum, similar (or corresponding) Doppler shift, and/or common periodic motion components, and compare the respective range delays corresponding to detection by the respective antennas 1 14, 1 14b. Signals sharing one or more characteristics can be correlated to triangulate to a location of one or more objects 1 12 in the region 1 10 relative to known locations of the antennas 1 14, 1 14b. The triangulated locations can be output as computed ranges of angle or computed ranges of extent.
For example, a first signal corresponding to a reflected pulse received by an antenna element 1 14 can be digitized by an analog-to-digital converter (A/D) 122 to form a first digitized waveform. A second signal corresponding to the reflected pulse received by a second antenna element 1 14b can similarly be digitized by and A/D 122b (or alternatively by the same A/D converter 122) to form a second digitized waveform. The signal processor 120 can compare the first and second digitized waveforms and deduce angular information from the first and second digitized waveforms and known geometry of the first and second antenna elements.
A second pulse can be received at a second range delay 1 16 value and can be similarly signal processed to produce a second set of angular information that maps a second surface at a different distance. Depth within a given range delay can be inferred from a strength of the reflected signal. A greater number of signals can be combined to provide additional depth information. A series of pulses can be combined to form a time series of signals corresponding to the object 1 12 that includes movement information of the object 1 12 through the region 1 10. The object 1 12 described herein can include one or more persons.
The signal processor 120 outputs MIR data. The MIR data can include object location information, object shape information, object velocity information, information about inclusion of high density and/or conductive objects such as jewelry, cell phones, glasses including metal, etc., and physiological information related to periodic motion. The MIR data can include spatial information, time-domain motion information, and/or frequency domain information. Optionally, the MIR data can be output in the form of an image. MIR data in the form of an image can include a surface slice made of pixels or a volume made of voxels. Optionally, the image can include vector information.
The MIR data from the signal processor 120 is output to a signal analyzer 124. The signal analyzer 124 can be integrated with the signal processor 120 and/or can be included in the same MIR 101 , as shown. Alternatively, the signal processor 120 can output MIR data through an interface to a signal analyzer 124 included in an apparatus separate from the MIR 101.
A signal analyzer 124 can be configured to extract desired information from MIR data received from the signal processor 120. Data corresponding to the extracted information can be saved in a memory for access by a data interface 126 or can be pushed out the data interface 126. The signal analyzer 124 can be configured to determine the presence of a person 1 12 in the region 1 10. For example, MIR data from the signal processor can include data having a static spectrum at a location in the region 1 10, and a periodic motion spectrum corresponding to the location characteristic of a human physiological process (e.g.
heartbeat and/or breathing). From the correspondence of such MIR data, it can be deduced that a person 1 12 is at the location in the region 1 10. The signal analyzer 124 can be configured to determine a number of persons 1 12 in the region 1 10. The signal analyzer 124 can be configured to determine the size of a person and/or relative size of anatomical features of a person 1 12 in the region 1 10. The signal analyzer 124 can be configured to determine the presence of an animal 1 12 in the region 1 10. The signal analyzer 124 can be configured to determine movement and/or speed of movement of a person 1 12 through the region 1 10. The signal analyzer 124 can be configured to determine or infer the orientation of a person 1 12 such as the direction a person is facing relative to the region 1 10. The signal analyzer 124 can be configured to determine one or more physiological aspects of a person 1 12 in the region 1 10. The signal analyzer 124 can determine presence of a personal appliance such as a cell phone, PDA, etc. and/or presence of metallized objects such as credit cards, smart cards, access cards, etc. The signal analyzer 124 can infer the gender and age of one or more persons based on returned MIR data. For example, male bodies can generally be characterized by higher mass density than female bodies, and thus can be characterized by somewhat greater reflectivity at a given range. Adult female bodies can exhibit relatively greater harmonic motion ("jiggle") responsive to movements, and can thus be correlated to harmonic spectra characteristics. Older persons generally move differently than younger persons, allowing an age inference based on detected movement in the region 1 10.
By determination of one or more such aspects and/or combinations of aspects, the signal analyzer 124 can determine a demographic of one or more persons 1 12 in the region 1 10.
For example, MIR data can include movement corresponding to the beating heart of one or more persons 1 12 in the region 1 10. The signal analyzer 124 can filter the MIR data to remove information not corresponding to a range of heart rates, and determine one or more heart rates by comparing movement of the heart surface to the MIR signal rate. The one or more heart rates can further be characterized according to a confidence factor, depending on statistical certainty regarding the determined one or more heart rates.
Similarly, the signal analyzer 124 can determine one or more respiration rates by measuring movement corresponding to the chest or diaphragm of one or more persons 1 12. The signal analyzer 124 can determine movement, a direction of movement, and/or a rate of movement of one or more persons 1 12 in the region 1 10. Operation of the signal analyzer 124 is described in greater detail below by reference to FIGS. 2 and 3.
An electronic controller 106 can be operatively coupled to the pulse generator 102, the transmitter 108, the range delay 1 16, the receiver 1 18, the analog-to-digital converter 122, the signal processor 120, and/or the signal analyzer 124 to control the operation of the components of the MIR 101. For embodiments so equipped, the electronic controller 106 can also be operatively coupled to the second receiver 1 18b, and the second analog-to-digital converter 122b. The data interface 126 can include a high speed interface configured to output of data from the signal analyzer 124. Alternatively, for cases where signals are analyzed externally to the MIR, the data interface 126 can include a high speed interface configured to output MIR data from the signal processor 120. The data interface 126 can include an interface to the controller 106. Optionally, the controller 106 can be interfaced to external systems via a separate interface (not shown).
FIG. 2 is a flow chart showing an illustrative process 201 for determining the presence of one or more persons 1 12 in the region 1 10 with the signal analyzer 124 of the MIR 101 , according to an embodiment. Beginning with step 202, MIR data is received as described above in conjunction with FIG. 1. The MIR data can correspond to a plurality of probes of the region 1 10. Proceeding to optional step 204, the MIR data can be enhanced to facilitate processing. For example, grayscale data corresponding to static reflection strength as a function of triangulated position can be adjusted, compressed, quantized, and/or expanded to meet a desired average signal brightness and range.
Additionally or alternatively, velocity information corresponding to Doppler shift, and/or frequency transform information corresponding to periodically varying velocity can similarly be adjusted, compressed, quantized, and/or expanded. Systematic, large scale variations in brightness can be balanced, such as to account for side-to-side variations in antenna coupling to the region. Contrast can be enhanced such as to amplify reflectance variations in the region.
Proceeding to optional step 206, a spatial filter can be applied. Application of a spatial filter can reduce processing time and/or capacity requirements for subsequent steps described below. The spatial filter may, for example, include a computed angle or computed extent filter configured to remove information corresponding to areas of contrast, velocity, or frequency component(s) having insufficient physical extent to be large enough to be an object of interest. The spatial filter may, for example, identify portions of the region 1 10 having sufficient physical extent to correspond to body parts or an entire body of a person 1 12, and remove features corresponding to smaller objects such as small animals, leaves of plants, or other clutter. According to an embodiment, the spatial filter can remove information corresponding to areas of contrast, velocity, or frequency component(s) having physical extent greater than a maximum angle or extent that is likely to correspond to a person or persons 1 12. In other embodiments, the spatial filter applied in step 206 can eliminate small, low contrast features, but retain small, high contrast features such as jewelry, since such body ornamentation can be useful in some subsequent processes. The step of applying the spatial filter 206 can further include removing background features from the MIR data. For example, a wall lying between an antenna 104, 1 14 and the region 1 10 can cast a shadow such as a line in every MIR signal. Removal of such constant features can reduce subsequent processing
requirements.
Proceeding to optional step 208, an edge-finder can identify edges of objects 1 12 in the region 1 10. For example, a global threshold, local threshold, second derivative, or other algorithm can identify edge candidates. Object edges can be used, for example, to identify object shapes, and thus relieve subsequent processes from operating on grayscale data. Alternatively, step 208 can be omitted and the process of identifying objects can be performed on the grayscale MIR data.
Proceeding to step 210, processed data corresponding to the MIR data is compared to a database to determine a match. The object data received from step 202 (and optionally steps 204, 206, and/or 208) can be compared to corresponding data for known objects in a shape database. Step 210 can be performed on a grayscale signal, but for simplicity of description it will be assumed that optional step 208 was performed and matching is performed using object edges, velocity, and/or spectrum values. For example, the edge of an object 1 12 in the region 1 10 can include a line corresponding to the outline of the head and torso, cardiac spectrum, and movements characteristic of a young adult male. A first shape in the shape database can include the outline of the head and torso, cardiac spectrum, density, and movements characteristic of a young adult female and/or the head and torso outline, cardiac spectrum, density, and movements characteristic of a generic human. The differences between the MIR data and the shape database shape can be measured and characterized to derive a probability value. For example, a least-squares difference can be calculated.
Optionally, the object shape from the MIR data can be stepped across, magnified, and stepped up and down the shape database data to minimize a sum-of-squares difference between the MIR shape and the first shape in the shape database. The minimum difference corresponds to the probability value for the first shape.
Proceeding to step 212, if the probability value for the first shape is the best probability yet encountered, the process proceeds to step 214. For the first shape tested, the first probability value is the best probability yet encountered. If an earlier tested shape had a higher probability to the MIR data, the process loops back from step 212 to step 210 and the fit comparison is repeated for the next shape from the shape database.
In step 214, the object type for the compared shape from the shape database and the best probability value for the compared shape are temporarily stored for future comparison and/or output. For example, the compared shape from the shape database can be identified by metadata that is included in the database or embedded in the comparison data. Proceeding to step 216, the process either loops back to step 210 or proceeds to step 218, depending on whether a test is met. If the most recently compared shape is the last shape available for comparison, then the process proceeds to step 218. Optionally, if the most recently compared shape is the last shape that the process has time to compare (for example, if a new MIR data is received and/or if another process requires output data from the process 201) then the process proceeds to step 218. In step 218, the object type and the probability value is output. The process can then loop back to step 202 and the process 201 can be repeated.
Otherwise, the process 201 loops from step 216 back to step 210. Again, in step 210, the next comparison shape from a shape database is loaded. According to an embodiment, the comparison can proceed from the last tested shape in the shape database. In this way, if the step 218 to 202 loop occurs more rapidly than all objects in the shape database can be compared, the process eventually works its way through the entire shape database. According to an embodiment, the shape database can include multiple copies of the same object at different orientations, distances, and positions within the region. This can be useful to reduce processing associated with stepping the MIR shape across the shape database shape and/or changing magnification.
The object type can include determination of a number of persons 1 12 in the region 1 10. For example, the shape database can include outlines, cardiac and/or respiration spectra, density, and movement characteristics for plural numbers of persons. According to embodiments, the shape library can include shapes not corresponding to persons. This can aid in identification of circumstances where no person 212 is in the region 210. Optionally, process 201 can be performed using plural video frames such as averaged video frames or a series of video frames. Optionally, steps 212, 214, and 216 can be replaced by a single decision step that compares the probability to a predetermined value and proceeds to step 218 if the probability meets the predetermined value. This can be useful, for example, in embodiments where simple presence or absence of a person 212 in the region 210 is sufficient information.
According to an embodiment, the signal analysis process 201 of FIG. 2 can be performed using conventional software running on a general-purpose microprocessor. Optionally, the process 201 using various combinations of hardware, firmware, and software and can include use of a digital signal processor.
FIG. 3 is a flow chart showing an illustrative process 301 for determining one or more particular physiological parameters of a person 1 12 in the region 1 10 with the signal analyzer 124 of the MIR 101 , according to an embodiment. Optionally, the process 301 of FIG. 3 can be performed conditional to the results of another process such as the process 201 of FIG. 2. For example, if the process 201 determines that no person 1 12 is in the region 1 10, then it can be preferable to continue to repeat process 201 rather than execute process 301 in an attempt to extract one or more particular physiological parameters from a person that is not present.
Beginning with step 302, a series of MIR time series data is received. While the received time series data need not be purely sequential, the process 301 generally needs the time series data received in step 302 to have a temporal capture relationship appropriate for extracting time-based information. According to an embodiment, the MIR time series data can have a frame rate between about 16 frames per second and about 120 frames per second. Higher capture rate systems can benefit from depopulating frames, such as by dropping every other frame, to reduce data processing capacity requirements.
Proceeding to step 304, the MIR video frames can be enhanced in a manner akin to that described in conjunction with step 204 of FIG. 2. Optionally, step 304 can include averaging and/or smoothing across multiple MIR time series data. Proceeding to optional step 306, a frequency filter can be applied. The frequency filter can operate by comparing changes between MIR time series data to a reference frequency band for extracting a desired physical parameter. For example, if a desired physiological parameter is a heart rate, then it can be useful to apply a pass band for periodic movements having a frequency between about 20 cycles per minute and about 200 cycles per minute, since periodic motion beyond those limits is unlikely to be related to a human heart rate. Alternatively, step 304 can include a high pass filter that removes periodic motion below a predetermined limit, but retains higher frequency information that can be useful for determining atypical physiological parameters.
Proceeding to optional step 308, a spatial filter can be applied. The spatial filter may, for example, include a pass band filter configured to remove information corresponding to areas of contrast having insufficient physical extent to be large enough to be an object of interest, and remove information corresponding to areas too large to be an object of interest. The spatial filter may, for example, identify portions of the region 1 10 having sufficient physical extent to correspond to the heart, diaphragm, or chest of a person 1 12, and remove signal features corresponding to smaller or larger objects. The step of applying the spatial filter 308 can further include removing background features from the MIR data. For example, a wall lying between an antenna 104, 1 14 (1 14b) and the region 1 10 can cast a shadow such as a line in every instance of MIR data. Removal of such constant features can reduce subsequent processing requirements.
Proceeding to step 310, movement such as periodic movement in the MIR time series data is measured. For example, when a periodic motion is to be measured, a time- to-frequency domain transform can be performed on selected signal elements. For example, when a non-periodic motion such as translation or rotation is to be measured, a rate of movement of selected signal elements can be determined. Optionally, periodic and/or non-periodic motion can be measured in space vs. time. Arrhythmic movement features can be measured as spread in frequency domain bright points or can be determined as motion vs. time. Optionally, subsets of the selected signal elements can be analyzed for arrhythmic features. Optionally plural subsets of selected signal elements can be cross-correlated for periodic and/or arrhythmic features. Optionally, one or more motion phase relationships between plural subsets of selected signal features, between a subset of a selected signal feature and the signal feature, or between signal features can be determined.
For example, a person with a hiccup can be detected as a non-periodic or arrhythmic motion superimposed over periodic motion of a signal element corresponding to the diaphragm of the person.
Proceeding to step 312, a physiological parameter can be calculated. For example,
MIR date can include data having a periodic motion spectrum corresponding to the location characteristic of a human physiological process (e.g. heartbeat and/or breathing). Step 312 can include determining one or more heart rates by comparing movement of the heart surface to the MIR signal rate. The one or more heart rates can further be characterized according to a confidence factor, depending on statistical certainty regarding the determined one or more heart rates. Similarly, step 312 can include determining one or more respiration rates by measuring movement corresponding to the chest or diaphragm of one or more person.
Proceeding to step 314, the physiological parameter can be output. Proceeding to step 316, if there are more locations to measure, the process 301 can loop back to execute step 308. If there are not more locations to measure, the process can proceed to step 318. In step 318, if there are more physiological parameters to measure, the process 301 can loop back to execute step 306. If there are not more physiological parameters to measure, the process 301 can loop back to step 302, and the process 301 of FIG. 3 can be repeated.
FIG. 4 is a block diagram of a system 401 including a computer with MIR feedback, according to an embodiment. The computer with micro-impulse radar feedback 401 includes a processing unit 402 including processing hardware and an operating system configured to run one or more application programs. A display 404 is under control of the processing unit 402 and configured to display images to a person 1 12 located in a first region 406. The first region 406 can be considered the viewing region. A MIR 101 is operatively coupled to the processing unit 402 and configured to probe a second region 1 10 to detect all or a portion of one or more individuals 1 12. According to an embodiment, the one or more individuals 1 12 can include a user of the computer system 401. The user can input commands to run the operating system and/or application programs on the processing unit 402.
As described in FIGS. 1-3, the MIR 101 is configured to produce a MIR signal.
Depending on how the MIR 101 and computer processing unit 402 are configured, the MIR signal can be substantially the same as MIR data output by the signal processor 120 described above. Alternatively, at least a portion of the MIR signal processing and/or analysis can be performed by the computer processing unit 402, and the MIR signal can be more primitive than output of MIR analysis. Alternatively, the MIR signal can correspond to a signal output by the signal analyzer 122.
According to approaches described herein, at least one of the processing hardware, operating system, or application program portions of the processing unit 402 is configured to receive information associated with the MIR signal and determine content or characteristics of the images displayed on the display 404 responsive to one or more characteristics of the micro-impulse radar information.
As indicated in FIG. 4, the first region 406 and second region 1 10 can be substantially coincident, such as when a majority of the regions 406 and 1 10 overlap. Referring to FIG. 5, the system 501 can alternatively be configured such that the second region 1 10 is a portion of the first region 406. For example, the MIR probe region 1 10 can be configured to measure characteristics of a portion 1 12a of the person 1 12. If the portion 1 12a of the person 1 12 corresponds to a hand, for example, the MIR 101 can receive signals corresponding to hand gestures such as gestures corresponding to operation of a virtual pointing device, operation of a virtual keyboard, American Sign Language (ASL), or other gesture convention. A person 1 12 shall be understood to include one or more portions 1 12a.
Alternatively, the first region 406 can be a portion of the second region 1 10. The MIR can be configured to probe a second region 1 10 that is larger than the region 406 from which the display 404 can be viewed. For example, the computer processing unit 402 can be configured to select a parameter and output an image on the display 404 in anticipation of one or more persons 1 10 entering the viewing region 406. In another alternative, the first and second regions 406, 1 10 can be substantially non-coincident. For example, the MIR can probe a person 1 12 traveling through a second region 1 10, and the computer processing unit 402 can output an image to the display 404 at a time
substantially coincident with the person 1 12 entering the viewing region 406.
Referring again to FIGS. 1-3, the MIR signals can include a MIR image. The
MIR signals can correspond to MIR data, such as MIR data output by the signal processor 120.
Referring to FIG. 1, the MIR 101 of FIGS. 4 and 5 can include a transmitter 108 configured to transmit electromagnetic pulses toward the second region 1 12. A pulse delay gate 1 16 can be configured to delay the pulses to trigger at least one receiver 1 18. Thus, the receiver 1 18 can be synchronized to the pulse delay gate 1 16 and configured to receive electromagnetic energy scattered from the pulses as they encounter objects, such as one or more persons 1 12, in the region 1 10. A signal processor 120 can be configured to receive signals or data from the receiver 1 18 and perform signal processing on the signals or data to extract one or more signals corresponding to at least one of human presence, human movement, or human physiological processes.
As shown in FIG. 4, the MIR 101 can be configured as a separate device from and operatively coupled to the computer processing unit 402. For example, the MIR 101 can be configured to communicate with the computer process via an exposed interface such as usb, IEEE 802.1 lx, line level inputs, or other conventional data interface. Alternatively, as indicated in FIG. 5, the MIR 101 can be integrated into the computer processing unit 402. Various levels of integration and partitioning are contemplated. The signal processor 120 can be integrated into the computer processing unit 402.
Similarly, the signal processor 120 can be integrated into the MIR 101 . This distinction may be moot in cases where the MIR 101 is integrated into the computer processing unit 402.
Referring to FIG. 6, according to an embodiment, the signal processor 120 can be configured as a portion of the processing hardware. The signal processor 120 can be embodied as software operable to run on the processing unit 402, a relatively low cost solution when the microprocessor (s) 604 has sufficient bandwidth. The signal processor 120 can similarly include both dedicated hardware and computer executable instructions operable to run on the processing hardware.
FIG. 6 is a block diagram of a computer architecture 602 having an operatively coupled MIR 101 , according to an embodiment. The computer 602 typically includes a microprocessor 604 operatively coupled to computer memory 606 (which includes tangible computer-readable media capable of carrying computer-readable instructions), computer storage 608 including a storage medium 610 that forms a tangible computer- readable medium capable of carrying computer-readable instructions, a data interface 612, and one or more human interfaces 614. For example, the human interface can include a keyboard, a computer pointing device such as a mouse, a touch screen, and/or a microphone. As indicated above, the MIR 101 can operate as a human interface and can augment or replace one or more conventional human interface apparatuses 614. The computer 602 can include or be operatively coupled to a display 404 and/or one or more additional output apparatuses (not shown) configured to output program output and/or media to one or more persons 1 12. The display 404 and/or one or more additional output apparatuses (not shown) are configured to output information, entertainment, etc. to the one or more persons 1 12 in a first region 406. The MIR 101 is configured to probe a second region 1 10 and responsively output an MIR signal or MIR data corresponding to presence, movement, and/or physiological processes of one or more persons 1 12 in a region 406. As described in conjunction with FIGS. 4 and 5, the regions 1 10 and 406 can be substantially coincident, overlapping, disjointed, and/or one region can be a subset of the other region.
The MIR can be embedded in a computer motherboard. Optionally, the MIR 101 can be configured as an expansion card such as a card compliant with ISA, PCI, PCI Express, NuBus, or other standard. Optionally, the MIR can be physically separate from and operatively coupled to the computer 602, such as through an exposed interface (not shown). Optionally, the MIR 101 can include an integrated signal processor, which can include Fourier transformation hardware or software. Optionally, signal processing can be performed using software running on the hardware 602 represented in FIG. 6.
The MIR 101 and/or the computer 602 can include a signal analyzer configured to receive signals or data from the signal processor and to perform signal analysis to determine, from the one or more extracted signals, variable data corresponding to human presence, human movement, human posture, and/or the human physiological processes. The computer 602 can control media content and/or program execution responsive to the received variable data corresponding to human presence, human movement, human posture, and/or the human physiological processes.
The MIR 101 can include an interface operatively coupled to a signal analyzer and configured to output MIR data including variable data corresponding to human 1 12 presence, human movement, and/or human physiological processes to the processing unit 602. A signal processor can be configured to output MIR data corresponding to human 1 12 presence, human movement, and/or human physiological processes to one or more memory circuit 606 or storage device 608, 610 locations.
An operating system running on the computer 602 can be configured to read at least a subset of the MIR data at the one or more memory circuit 606 or storage device 608, 610 locations and responsively adjust one or more operating system parameters. Responsive adjustment of one or more operating system parameters may, for example, include waking the operating system from a sleep mode responsive to MIR data corresponding to human presence in the second region 1 10. Similarly, adjustment of one or more operating system parameters can include entering a sleep mode responsive to MIR data corresponding to departure of an individual 1 12 from the second region 1 10. Additionally or alternatively, an application running on the computer 602 can be configured to read at least a subset of the MIR data at the one or more memory circuit 606 or storage device 608, 610 locations and responsively adjust one or more application parameters.
For example, adjustment one or more operating system parameters or application parameters can include selection of terse prompts or fast response responsive to MIR data corresponding to human 1 12 movement or physiological processes characteristic of impatience or urgency. Additionally or alternatively, adjustment of one or more operating system or application parameters can include conversion of the MIR data into cursor movements. For example, adjustment of one or more application or operating system parameters can includes conversion of the MIR data into computer pointer device 614 commands.
As indicated above, an application program running on the computer 602 can be configured to read at least a subset of the MIR data at the one or more memory circuit 606 or storage device 608, 610 locations and responsively adjust application program parameters. For example, an application program can be configured to convert the MIR data to program commands. Alternatively or additionally, the application program can be configured to convert the MIR data to one or more preferences selections. For example, one or more preferences selections can include automated help, terse response, verbose response, and/or video magnification.
As indicated above, information associated with the MIR signal can correspond to a human 1 12 physiological process. Such a physiological process can include heartbeat and/or breathing, for example. An application program and/or operating system can further be configured to correlate the physiological process to a predicted emotional state of an individual 1 12. The operating system and/or application program can be configured to conditionally select a program execution path as a function of the predicted emotional state.
FIG. 7 is a flow chart illustrating a method 701 for controlling a computer using parameters determined by a MIR, according to an embodiment. A first section 702 of the method 701 corresponds to generation of an operating parameter for controlling an operating system and/or application program. Section 702 can be performed by hardware, by an application program, and/or by an operating system, for example.
Beginning at step 704, one or more MIR signals is received from one or more regions (e.g., one or more regions 1 10). For example, the MIR signals can include a MIR image. According to an embodiment, the MIR signals can correspond to MIR data.
Proceeding to step 706, analysis is performed on the one or more MIR signals to determine presence, movement, and/or one or more physiological processes of one or more persons in the one or more regions. For example, the one or more persons can include a user of the computer. Approaches to determining presence, movement, posture, and/or physiological processes of persons are described above in conjunction with FIGS. 1-3, for example. For example, physiological processes can include heartbeat, perspiration, tremor, and/or breathing, and corresponding physiological parameters can include heart rate, moistness of the skin, a periodic movement or shaking, or breathing rate.
Proceeding to step 708, an operating parameter, which can include one or more operating parameters, is selected and output for at least one computer responsive to the presence, movement, and/or one or more physiological processes of the one or more persons in the one or more regions.
For example, a physiological parameter can be indicative of a physiological state of the person and determine the physiological state for the computer. Determining an operating parameter can include correlating at least one physiological parameter to a predicted emotional state. For example, correlating the at least one physiological parameter to the predicted emotional state can include correlating the physiological parameter to an autonomic nervous system state and correlating the autonomic nervous system state to the predicted emotional state.
Similarly, a posture can be indicative of a physiological state and/or an intent. For example a person lying prone may be unconscious, asleep, or relaxed. A person who is hunched or collapsed on the ground may be injured, sick, or emotional. A stance may be threatening or precarious. A movement may indicate good balance or a lack of balance. Another movement and/or tremor may correspond to laughing, coughing, or sneezing. A detected movement stability, periodicity, and/or tremor man indicate decreased motor control, shaking, trembling, or undergoing a seizure. Such a posture or movement may be conscious or may be unconscious. Embodiments herein include driving an electronic apparatus responsive to these and other indications of a command, a condition, or an intent that may be extracted from or included in MIR data.
A data value corresponding to the operating parameter can be written to computer memory or storage device. Alternatively, line levels can be set at pins of a
microprocessor or other hardware, a combination of one or more line levels being set according to the analyzed MIR signals. A number of parameters and/or line levels can be selected to correspond to the sophistication of control desired. For example, simple presence or absence of a person in a region can be communicated to the computer by setting a bit or a single line level.
A larger number of bits or lines, or a greater number of variables can be set to correspond to control the computer with greater sophistication. For example, an X-Y location or velocity of a digit of a hand (representing a computer pointer, e.g. mouse, position) can be represented to high precision by two bytes for each axis. A vertical position or velocity of the digit (representing a pointer device selection, e.g. mouse click) can be represented by as little as a single bit if software filtering is not desired. Similarly, operating parameters can be determined or output corresponding to one or more of other presence data such as a probability of presence of a particular person, movement such as body movements and/or velocity intended to control a computer game, and/or
physiological parameters such as parameters corresponding to heart rate and/or breathing depth and rate.
After outputting one or more operating parameters, the process section 702 loops back to step 704 and is repeated. The frequency of looping can be determined according to an operating state. For example, if the system is in a sleep mode and is intended to wake up when a person enters a region, looping can be relatively slow, such as once per second. On the other hand, if the computer is in active operation with movements used to control a cursor, then looping can be relatively rapid, such as once per millisecond to once per 100 milliseconds, depending on desired responsiveness, bandwidth, and/or preferred limits on MIR probe radiation. Section 710 of the process 701 can be performed asynchronously from process 702. For example, section 710 can be performed by a device driver, API, or other operating system-related process. Starting with step 712, the operating parameter is read. Proceeding to step 714, the new operating parameter can be compared to previous values of the operating parameter or a function of previous values of the operating parameter, if the operating parameter has not changed (e.g. in a statistically significant way), the process can loop back through an optional process 718 and then to step 712. The optional process 718 may, for example, perform filtering such as debounce or other noise reduction. The process 718 can be programmable, with its functionality determined by context. In this way, the test performed in step 714 can be required to be performed a plurality of times prior to the condition being determined true. If step 714 is determined to be true, the process proceeds to step 716, where the parameter is loaded. For example, loading the parameter can include writing a value (either the value of the operating parameter or a corresponding data value) into a memory or storage location that can be accessed by other processes, or can include setting one or more line levels on pins of a microprocessor or other hardware.
For example, selecting an operation parameter and/or subsequent performance of section 710 of process 701 can include waking portions of the computer when the one or more persons enters the one or more regions. According to an embodiment, prior to waking the portions of the computer, performing analysis on a sequence of micro- impulse radar signals can determine a probability that the one or more persons has an intent to use the computer. Determining the probability that the one or more persons has an intent to use the computer can include analyzing movements of the one or more persons. For example, the one or more persons can be determined to have a high probability of intent to use the computer when the one or more persons enter a computer operating position. For example, one or more persons can be determined to have a high probability of intent to use the computer when the one or more persons approach near to or reach for a computer keyboard or a computer pointing device and/or approach the computer display.
Similarly, selecting an operation parameter and/or subsequent performance of section 710 of process 701 can including placing one or more portions of the computer in a sleep mode when the one or more persons leaves the one or more regions. According to an embodiment, prior to putting portions of the computer to sleep, performing analysis on a sequence of MIR signals can determine a probability that the one or more persons has an intent to stop using the computer.
When a parameter or parameters are loaded in step 716, the parameter or parameters can be read by an operating system and/or one or more application programs. Proceeding to step 720, which can be performed asynchronously from section 710 of the process 701 , program execution can be performed responsive to the loaded parameter or parameters.
According to another embodiment, selecting an operation parameter includes conditionally selecting computer program logic. For example, sections 702 and 710 can include analyzing movements of the one or more persons. Loading the parameter(s) in step 716 can amount to entering one or more software commands responsive to the movements.
Referring to step 720, the one or more software commands can include commands to operate game software on the computer. For example, the one or more software commands can be selected to drive virtual movements of one or more game entities corresponding to the movements of the one or more persons.
The method described in conjunction with FIG. 7 can be physically embodied as computer executable instructions carried by a tangible computer readable medium.
According to embodiments, the method shown in FIG. 7 can be used in apparatuses other than general purpose computers, such as video gaming systems, entertainment systems, and/or other systems that benefit from MIR input to electronic processing functions.
FIG. 8 is a flow chart illustrating a method 801 for operating a computer responsive to an emotional state of a person, according to an embodiment. The person can be a user of the computer.
Beginning with step 802, a sequence of MIR signals is received, the signals including characteristics corresponding to a person. The MIR signals can include micro- impulse radar image. The MIR signals can correspond to micro-impulse radar data, such as MIR data produced by a signal analyzer, as described in conjunction with FIGS. 1-3. Proceeding to step 804, at least one parameter corresponding to at least one of a physiology, posture, or movement of the person is extracted from the sequence of MIR signals. For example, a physiological parameter can include at least one of
heartbeat, perspiration on the skin of the person, tremor, or respiration. The
physiological parameter (optionally along with posture and/or movement) can determine the physiological state of the person.
Proceeding to step 806 the at least one parameter extracted in step 804 is correlated to a predicted emotional state of the person. For example, correlating a physiological or movement parameter to the predicted emotional state can include correlating the physiological parameter to an autonomic nervous system state. Step 804 can include labeling the predicted emotional state a positive or negative.
Proceeding to step 808 a program execution path is conditionally selected responsive to the predicted emotional state of the person. Conditionally selecting a program execution path can include conditionally selecting a program execution path predicted to please the person.
Conditionally selecting a program execution path can include to activating a program function correlated to a positive emotional state. Alternatively, conditionally selecting a program execution path can include disabling a program function correlated to a negative emotional state. For example, disabling a program function correlated to a negative emotional state can include disabling an automatic format function correlated to a negative emotional state.
Referring to step 804, if the person is confused or otherwise uncertain of how to perform a function or what function to perform, he or she can display movement and/or physiological signs such as tensing muscles of the face, cocking his or her head to one side, consulting printed materials near the computer, or otherwise betraying a state of puzzlement, such as can be observable by another person, and which corresponds to detectable information that is extracted from the MIR signals. Accordingly, in step 806, the physiological or movement parameter can be correlated to a prediction of puzzlement.
In step 808, conditionally selecting a program execution path can include activating a help function responsive to the predicted puzzlement. For example, activating a help function can include activating verbose prompts. Verbose prompts can thus provide additional help to aid the user in overcoming his or her puzzlement.
Referring again to step 804, if a person is impatient, then he or she can tap a finger, fidget, exhibit increased heart rate and/or a staccato breathing motion, and so the extracted physiological or movement parameter extracted from the MIR signal may, in step 806, be correlated to a prediction of impatience. Accordingly, conditionally selecting a program execution path can include selecting a path intended to reduce the impatience of the person. For example, terse prompts can be activated responsive to predicting impatience, the terse prompts thus reducing processing and/or interface time, and providing the person less reason for being impatient.
According to some embodiments, it can be desirable for an application to produce a target emotional state in a person, such as in a computer game. For example, the program execution path selected in step 808 can include selecting game logic responsive to the predicted emotional state. According to an embodiment, the game logic can be selected to provide predicted emotional states including tension and release.
Conditionally selecting a program execution path in step 808 responsive to the predicted emotional state of the person can include selecting a game difficulty level responsive to the predicted emotional state.
According to embodiments, various movement, posture, and/or physiological parameters can indicate different inputs and be used to drive system response in different ways, or in ways that are an interaction product of two or more movement, posture, and/or physiological parameters. For example, an explicit movement of a finger can be used to determine an explicit intent to make a command gesture, while smaller, involuntary movements such as a tremor in the finger can be used to infer a physiological and/or emotional state. Similarly, posture can indicate a transitory mood, a disposition, and/or an intent of a person. For example, a person taking an aggressive stance can indicate an aggressive mood or, in another context, can drive an aggression parameter in a computer game. A prone posture can indicate relaxation, or in another context, can drive a "play dead" parameter in a computer game. A hunched over posture can indicate physical or emotional distress. As may be appreciated, interpretation of MIR information can be made context-sensitive, can be disabled, and/or can be used to construct a user profile for future reference.
The method described in conjunction with FIG. 8 can be physically embodied as computer executable instructions carried by a tangible computer readable medium.
According to embodiments, the method shown in FIG. 8 can be used in apparatuses other than general purpose computers, such as video gaming systems, entertainment systems, and/or other systems that benefit from MIR input to electronic processing functions.
FIG. 9 is a block diagram of an entertainment system 901 configured to adapt to personal preferences, according to an embodiment. The entertainment system 901 includes a MIR 101 configured to probe a region proximate a media output apparatus and output a MIR signal. The MIR signal can include or consist essentially of MIR data. The MIR signal can include a MIR image. A controller 902 is configured to receive the micro-impulse radar signal from the MIR 101 and select one or more program options responsive to at least one of a presence, movement or a physiological parameter corresponding to one or more persons 1 12 in the probed region. For example, the controller 902 can be configured as an audio and/or video receiver. The MIR 101 can be separate from and operatively coupled to the controller 902 as shown, or optionally the MIR 101 and the controller 802 can be integrated.
The controller 902 is configured to drive a media output apparatus 912. The media output apparatus 912 is configured to present media content according to the one or more program options. For example, the media output apparatus 912 can include a television, a video monitor, one or more loudspeakers, a portable audio device, a lighting system, and/or a window treatment actuator. According to an embodiment, the media output apparatus 912 can include a portable media player carried by the person 1 12. One or more of the MIR 101 and the controller 902 can be thus be located remotely from and operatively coupled to the media output apparatus 912.
The entertainment system 901 can include at least one program source 908.
Selecting at least one program option can include selecting programming from a plurality of available programming from the at least one program source 908. For example, the program source 908 can include an apparatus for receiving one or more streamed video and/or audio channels, a tuner for receiving one or more television channels, local solid state or rotating drive media track storage, an audio or video server, and/or a single- or multi-disc CD, DVD, or Blue Ray player. One or more program sources 908 can optionally be integrated into the controller 902, or can be operatively coupled to the controller 902 as shown. The at least one program source 908 can optionally be integrated into the output apparatus 912.
The controller 902 can include a microcontroller 904. The entertainment system 901 can include at least one codec 906. The codec 906 can optionally be integrated into the controller 902 as shown. The codec 906 can be operatively coupled to the microcontroller 904. Optionally, the codec 906 can be separate from and operatively coupled to the controller 902. According to an embodiment, the codec 906 can be integrated into the at least one program source 908. According to an embodiment, the codec 906 can be integrated into the media output apparatus 912. Selecting at least one program option can include selecting a codec 906 or selecting a codec operating parameter.
The entertainment system 901 and the controller 902 can include one or more media output apparatus drivers 910. The one or more media output apparatus drivers 910 can be operatively coupled to the microcontroller 904 and to the at least one program source 908. Selecting at least one program option can include selecting a media output apparatus driver 910 or a media output apparatus driver operating parameter. For example, the one or more media output apparatus driver 910 can include one or more signal amplifiers; and selecting a media output apparatus driver parameter can include selecting an amount of amplification. Optionally, selecting a media output apparatus driver 910 operating parameter can include selecting between two or more media output apparatuses 912, which can include selecting both or more of the media output apparatuses 912.
The entertainment system 901 can be operated in a one or more of a variety of ways, according to embodiments.
For example, selecting one or more program options can be performed responsive to presence of one or more persons 1 12. This can include one or more of starting output of media responsive to arrival of one or more persons in the probed region and/or stopping or suspending output of media responsive to departure of one or more persons from the probed region.
Additionally or alternatively, selecting one or more program options can be performed responsive to movement of one or more persons 1 12. This can include selecting a program option set responsive to detected movement corresponding to attentiveness to a media output apparatus 912. For example, detected movement corresponding to a media output apparatus 912 can include one or more of the one or more persons 1 12 approaching a video display, moving to an optimum listening area in an audio sound field, or remaining still in a viewing area of a video display. Additionally or alternatively, the detected movement of the one or more persons 1 12 can include a rhythmic movement of the one or more persons in time with a rhythmic component of a media program. For example, rhythmic movement can include one or more of dancing, head bobbing, or toe tapping.
Additionally or alternatively, selecting one or more program options can be performed responsive to detecting a physiological parameter corresponding to one or more persons 1 12. Such a physiological parameter can include a physiological parameter corresponding to a response to media content. Sensing of physiological parameters and changes in physiological parameters is described above.
Selecting one or more program options responsive to detected movement or a physiological parameter may, for example, include one or more of increasing audio volume and enabling a video output apparatus responsive to receiving a MIR 101 signal corresponding to attentiveness. Additionally or alternatively, selecting one or more program options responsive to a detected movement or physiological parameter of one or more persons 1 12 corresponding to attentiveness to a media output apparatus 912 can include selecting and/or queuing additional program content having at least one characteristic similar to program content played during the detection of movement or physiological parameter corresponding to attentiveness.
While particular aspects of the present subject matter described herein have been shown and described, it will be apparent that, based upon the teachings herein, changes and modifications may be made without departing from the subject matter described herein and its broader aspects and, therefore, the appended claims are to encompass within their scope all such changes and modifications as are within the true spirit and scope of the subject matter described herein. Furthermore, it is to be understood that the invention is defined by the appended claims. It will be understood that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but is not limited to," etc.). If a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., "a" and/or "an" should typically be interpreted to mean "at least one" or "one or more"); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., " a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to "at least one of A, B, or C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., " a system having at least one of A, B, or C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B."
With respect to the appended claims, those skilled in the art will appreciate that recited operations therein may generally be performed in any order. Examples of such alternate orderings may include overlapping, interleaved, interrupted, reordered, incremental, preparatory, supplemental, simultaneous, reverse, or other variant orderings, unless context dictates otherwise. With respect to context, even terms like "responsive to," "related to," or other past-tense adjectives are generally not intended to exclude such variants, unless context dictates otherwise.
While various aspects and embodiments have been disclosed herein, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

CLAIMS What is claimed is:
1. A computer with micro-impulse radar feedback, comprising:
a processing unit including processing hardware and an operating system configured to run one or more application programs;
a display under control of the processing unit and configured to display images to a person located in a first region; and
a micro-impulse radar operatively coupled to the processing unit and configured to probe a second region to detect all or a portion of one or more individuals and produce a micro-impulse radar signal;
wherein at least one of the processing hardware, operating system, or application program is configured to read at least a subset of information associated with the micro- impulse radar signal and responsively adjust one or more processing hardware, operating system, or application program parameters to determine content or characteristics of the images displayed on the display responsive to one or more characteristics of the micro- impulse radar information.
2. The computer with micro-impulse radar feedback of claim 1 , wherein the second region is a portion of the first region.
3. The computer with micro-impulse radar feedback of claim 1 , wherein the first region and second region are substantially coincident.
4. The computer with micro-impulse radar feedback of claim 1 , wherein the first region is a portion of the second region.
5. The computer with micro-impulse radar feedback of claim 1 , wherein the micro- impulse radar signals include a micro-impulse radar image.
6. The computer with micro-impulse radar feedback of claim 1, wherein the micro- impulse radar signals correspond to micro-impulse radar data.
7. The computer with micro-impulse radar feedback of claim 1, wherein the one or more individuals includes a user of the processing unit
8. The computer with micro-impulse radar feedback of claim 1, wherein the micro- impulse radar includes:
a transmitter configured to transmit electromagnetic pulses toward the second region;
a range delay gate configured to delay clock times corresponding to the pulses; and
a receiver synchronized to the pulse delay gate and configured to receive electromagnetic energy scattered from the pulses.
9. The computer with micro-impulse radar feedback of claim 1 , further comprising: a signal processor configured to receive signals or data from the micro-impulse radar and to perform signal processing on the signals or data to extract one or more signals corresponding to at least one of human presence, human movement, or human physiological processes.
10. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor is configured as a portion of the processing hardware.
1 1. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor is embodied as software operable to run on the processing unit.
12. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor includes dedicated hardware and computer executable instructions operable to run on the processing hardware.
13. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor includes Fourier transformation hardware or software.
14. The computer with micro-impulse radar feedback of claim 9, further comprising: a signal analyzer configured to receive signals or data from the signal processor and to perform signal analysis to determine, from the one or more extracted signals, variable data corresponding to at least one of the human presence, the human movement, or the human physiological processes, the content or characteristics of the images displayed on the display being controlled responsive to the variable data.
15. The computer with micro-impulse radar feedback of claim 14, further comprising: an interface operatively coupled to the signal analyzer and configured to output micro-impulse radar data including the variable data corresponding to at least one of the human presence, the human movement, or the human physiological processes to the processing unit.
16. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor is integrated into the processing unit.
17. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor is integrated into the micro-impulse radar.
18. The computer with micro-impulse radar feedback of claim 9, wherein the signal processor is configured to output micro-impulse radar data corresponding to the at least one of human presence, human movement, or human physiological processes to one or more memory circuit or storage device locations.
19. The computer with micro-impulse radar feedback of claim 18, wherein the operating system is configured to read at least a subset of the micro-impulse radar data at the one or more memory circuit or storage device locations and responsively adjust one or more operating system parameters.
20. The computer with micro-impulse radar feedback of claim 19, wherein responsive adjustment of one or more operating system parameters includes waking the operating system from a sleep mode responsive to micro-impulse radar data corresponding to human presence in the second region.
21 . The computer with micro-impulse radar feedback of claim 19, wherein adjustment of one or more operating system parameters includes selection of terse prompts or fast response responsive to micro-impulse radar data corresponding to human movement or physiological processes characteristic of impatience or urgency.
22. The computer with micro-impulse radar feedback of claim 19, wherein adjustment of one or more operating system parameters includes entering a sleep mode responsive to micro-impulse radar data corresponding to departure of an individual from the second region.
23. The computer with micro-impulse radar feedback of claim 19, wherein adjustment of one or more operating system parameters includes conversion of the micro- impulse radar data into cursor movements.
24. The computer with micro-impulse radar feedback of claim 19, wherein adjustment of one or more operating system parameters includes conversion of the micro- impulse radar data into computer pointer device commands.
25. The computer with micro-impulse radar feedback of claim 18, wherein an application program is configured to read at least a subset of the micro-impulse radar data at the one or more memory circuit or storage device locations and responsively adjust application program parameters.
26. The computer with micro-impulse radar feedback of claim 25, wherein the application program is configured to convert the micro-impulse radar data to program commands.
27. The computer with micro-impulse radar feedback of claim 25, wherein the application program is configured to convert the micro-impulse radar data to one or more preferences selections.
28. The computer with micro-impulse radar feedback of claim 27, wherein the one or more preferences selections includes at least one of automated help, terse response, verbose response, or magnification.
29. The computer with micro-impulse radar feedback of claim 1 , wherein the information associated with the micro-impulse radar signal corresponds to a human physiological process.
30. The computer with micro-impulse radar feedback of claim 29, wherein the physiological process includes at least one of heartbeat, perspiration, tremor, or breathing.
31. The computer with micro-impulse radar feedback of claim 29, further comprising: correlating the physiological process to a predicted emotional state of an individual.
32. The computer with micro-impulse radar feedback of claim 31 , wherein the operating system or application program is configured to conditionally select a program execution path as a function of the predicted emotional state.
33. A tangible computer-readable medium carrying computer-executable instructions that, when executed, cause a computer to:
receive a sequence of micro-impulse radar signals corresponding to a person; extract at least one of a physiological or movement parameter from the sequence of micro-impulse radar signals;
correlate the at least one physiological or movement parameter to at least one predicted emotional state of the person; and
conditionally select a program execution path responsive to the at least one predicted emotional state of the person.
34. The tangible computer-readable medium of claim 33, wherein the micro-impulse radar signals include micro-impulse radar image.
35. The tangible computer-readable medium of claim 33, wherein the micro-impulse radar signals correspond to micro-impulse radar data.
36. The tangible computer-readable medium of claim 33, wherein the person is a user of a computer.
37. The tangible computer-readable medium of claim 33, wherein the at least one physiological parameter includes at least one of heartbeat, perspiration, tremor, or respiration.
38. The tangible computer-readable medium of claim 33, wherein the at least one physiological, posture, or movement parameter determines the physiological state of the person.
39. The tangible computer-readable medium of claim 33, wherein the correlating the at least one physiological, posture, or movement parameter to the at least one predicted emotional state includes correlating the at least one physiological, posture, or movement parameter to an autonomic nervous system state.
40. The tangible computer-readable medium of claim 33, wherein the conditionally selecting a program execution path includes conditionally selecting a program execution path predicted to please the person.
41 . The tangible computer-readable medium of claim 33, wherein the computer- executable instructions, when executed, further cause the computer to label the at least one predicted emotional state positive or negative.
42. The tangible computer-readable medium of claim 41 , wherein the computer- executable instructions, when executed, further cause the computer to activate a program function correlated to a positive emotional state.
43. The tangible computer-readable medium of claim 41 , wherein the computer- executable instructions, when executed, further cause the computer to disable a program function correlated to a negative emotional state.
44. The tangible computer-readable medium of claim 41 , wherein the computer- executable instructions, when executed, further cause the computer to disable an automatic format function correlated to a negative emotional state.
45. The tangible computer-readable medium of claim 33, wherein the at least one predicted emotional state includes a state of puzzlement; and
wherein conditionally selecting a program execution path includes activating a help function responsive to predicting puzzlement.
46. The tangible computer-readable medium of claim 45, wherein the help function includes activating verbal prompts.
47. The tangible computer-readable medium of claim 41 , wherein the at least one predicted emotional state includes a state of impatience; and wherein conditionally selecting a program execution path includes activating terse prompts responsive to predicting impatience.
48. The tangible computer-readable medium of claim 33, wherein the computer- executable instructions, when executed, further cause the computer to execute game logic associated with a computer game; and
wherein the game logic is selected responsive to the at least one predicted emotional state.
49. The tangible computer-readable medium of claim 33, wherein the computer- executable instructions, when executed, further cause the computer to execute game logic associated with a computer game; and
wherein the game logic is selected to provide predicted emotional states including tension and release.
50. The tangible computer-readable medium of claim 33, wherein the computer- executable instructions, when executed, further cause the computer to execute game logic associated with a computer game; and
wherein conditionally selecting a program execution path responsive to the at least one predicted emotional state of the person includes selecting a game difficulty level responsive to the at least one predicted emotional state.
51. A method for controlling a computer, comprising:
receiving one or more micro-impulse radar signals from one or more regions; performing analysis on the one or more micro-impulse radar signals to determine one or more of presence, movement, or at least one physiological process of one or more persons in the one or more regions; and
selecting an operation parameter of the computer responsive to one or more of the presence, the movement, or the at least one physiological process of the one or more persons in the one or more regions.
52. The method for controlling a computer of claim 5 1 , wherein the micro-impulse radar signals include a micro-impulse radar image.
53. The method for controlling a computer of claim 51 , wherein the micro-impulse radar signals correspond to micro-impulse radar data.
54. The method for controlling a computer of claim 5 1 , wherein the one or more persons include a user of the computer.
55. The method for controlling a computer of claim 5 1 , wherein performing the analysis on the micro-impulse radar signals includes determining at least one physiological parameter corresponding to the one or more persons.
56. The method for controlling a computer of claim 55, wherein the at least one physiological parameter includes at least one of heart rate, perspiration, tremor, or breathing rate.
57. The method for controlling a computer of claim 55, wherein the at least one movement, posture, or physiological parameter determines the physiological state of the person.
58. The method for controlling a computer of claim 55, further comprising:
correlating the at least one movement, posture, or physiological parameter to a predicted emotional state.
59. The method for controlling a computer of claim 58, wherein correlating the at least one movement, posture, or physiological parameter to the predicted emotional state includes:
correlating one or more of the movement, posture, or physiological parameter to an autonomic nervous system state; and
correlating the autonomic nervous system state to the predicted emotional state.
60. The method for controlling a computer of claim 5 1 , wherein selecting an operation parameter includes conditionally selecting computer program logic.
61. The method for controlling a computer of claim 51, wherein selecting an operation parameter includes waking portions of the computer when the one or more persons enters the one or more regions.
62. The method for controlling a computer of claim 61 , wherein the computer is located in one of the regions.
63. The method for controlling a computer of claim 61 , further comprising, prior to waking the portions of the computer, performing analysis on a sequence of micro- impulse radar signals to determine a probability that the one or more persons has an intent to use the computer.
64. The method for controlling a computer of claim 63, wherein determining the probability that the one or more persons has an intent to use the computer includes analyzing movements of the one or more persons.
65. The method for controlling a computer of claim 64, wherein the one or more persons is determined to have a high probability of intent to use the computer when the one or more persons enter a computer operating position or approach a chair near the computer.
66. The method for controlling a computer of claim 64, wherein the one or more persons is determined to have a high probability of intent to use the computer when the one or more persons near or reach for a computer keyboard or a computer pointing device or near a computer display.
67. The method for controlling a computer of claim 51 , wherein selecting an operation parameter includes putting portions of the computer in a sleep mode when the one or more persons leaves the one or more regions.
68. The method for controlling a computer of claim 51 , further comprising:
analyzing movements of the one or more persons; and
wherein selecting an operating parameter includes entering one or more software commands responsive to the movements.
69. The method for controlling a computer of claim 68, wherein the one or more software commands includes commands to operate game software on the computer.
70. The method for controlling a computer of claim 69, wherein the one or more software commands are selected to drive virtual movements of one or more game entities corresponding to the movements of the one or more persons.
71. An entertainment system configured to adapt to personal preferences, comprising: a media output apparatus configured to present media content according to one or more program options;
a micro-impulse radar configured to probe a region proximate the media output apparatus and output a micro-impulse radar signal; and
a controller configured to receive the micro-impulse radar signal, and select the one or more program options responsive to at least one of a presence, movement or physiological parameter corresponding to one or more persons in the probed region.
72. The entertainment system of claim 71 , further comprising:
at least one program source;
wherein selecting one or more program options includes selecting programming from the program source.
73. The entertainment system of claim 71 , further comprising: at least one code;
wherein selecting one or more program options includes selecting a codec or selecting a codec operating parameter.
74. The entertainment system of claim 71 , further comprising:
one or more media output apparatus drivers; and
wherein selecting one or more program options includes selecting a media output apparatus driver or a media output apparatus driver parameter.
75. The entertainment system of claim 74, wherein the one or more media output apparatus drivers include one or more signal amplifiers; and
wherein selecting a media output apparatus driver parameter includes selecting an amount of amplification.
76. The entertainment system of claim 74, wherein selecting a media output apparatus driver parameter includes selecting between two or more media output apparatuses.
77. The entertainment system of claim 71 , wherein the micro-impulse radar signal includes micro-impulse radar data.
78. The entertainment system of claim 71 , wherein the micro-impulse radar signal includes a micro-impulse radar image.
79. The entertainment system of claim 71 , wherein selecting one or more program options responsive to presence includes one or more of starting output of media responsive to arrival of one or more persons in the probed region and stopping or suspending output of media responsive to departure of one or more persons from the probed region.
80. The entertainment system of claim 71 , wherein selecting one or more program options responsive to movement, posture, or a physiological parameter includes selecting one or more program options responsive to detected movement or physiological parameter corresponding to attentiveness to a media output apparatus.
81. The entertainment system of claim 80, wherein selecting one or more program options responsive to detected movement or a physiological parameter corresponding to attentiveness to a media output apparatus includes one or more of increasing audio volume and enabling a video output apparatus.
82. The entertainment system of claim 80, wherein the detected movement corresponding to attentiveness to a media output device includes one or more of the one or more persons approaching a video display, moving to an optimum listening area in an audio sound field, changing orientation relative to a video display, or remaining still in a viewing area of a video display.
83. The entertainment system of claim 80, wherein the movement includes a rhythmic movement of the one or more persons in time with a rhythmic component of a media program.
84. The entertainment system of claim 83, wherein the rhythmic movement includes one or more of dancing, head bobbing, or toe tapping.
85. The entertainment system of claim 80, wherein selecting one or more program options responsive to detected movement or physiological parameter corresponding to attentiveness to a media output apparatus includes selecting additional program content having at least one characteristic similar to program content played during detection of movement corresponding to attentiveness.
86. The entertainment system of claim 71 , wherein the micro-impulse radar and the controller are integrated.
87. The entertainment system of claim 71, wherein the media output apparatus includes a portable media player carried by the person.
88. The entertainment system of claim 87, wherein one or more of the micro-impulse radar and the controller are located remotely from the portable media player.
89. The entertainment system of claim 71, wherein the physiological parameter includes a physiological parameter corresponding to a response to the media content.
PCT/US2011/001629 2010-09-17 2011-09-19 Control of an electronic apparatus using micro-impulse radar WO2012036753A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/924,036 US9069067B2 (en) 2010-09-17 2010-09-17 Control of an electronic apparatus using micro-impulse radar
US12/924,036 2010-09-17

Publications (1)

Publication Number Publication Date
WO2012036753A1 true WO2012036753A1 (en) 2012-03-22

Family

ID=45817257

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/001629 WO2012036753A1 (en) 2010-09-17 2011-09-19 Control of an electronic apparatus using micro-impulse radar

Country Status (2)

Country Link
US (1) US9069067B2 (en)
WO (1) WO2012036753A1 (en)

Families Citing this family (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9151834B2 (en) 2011-04-29 2015-10-06 The Invention Science Fund I, Llc Network and personal electronic devices operatively coupled to micro-impulse radars
US9103899B2 (en) 2011-04-29 2015-08-11 The Invention Science Fund I, Llc Adaptive control of a personal electronic device responsive to a micro-impulse radar
US9000973B2 (en) * 2011-04-29 2015-04-07 The Invention Science Fund I, Llc Personal electronic device with a micro-impulse radar
US9264660B1 (en) * 2012-03-30 2016-02-16 Google Inc. Presenter control during a video conference
US9344773B2 (en) * 2013-02-05 2016-05-17 Microsoft Technology Licensing, Llc Providing recommendations based upon environmental sensing
GB2521833A (en) * 2014-01-02 2015-07-08 Nokia Technologies Oy An apparatus, method and computer program for enabling a user to make user inputs
US9921657B2 (en) * 2014-03-28 2018-03-20 Intel Corporation Radar-based gesture recognition
US10436888B2 (en) * 2014-05-30 2019-10-08 Texas Tech University System Hybrid FMCW-interferometry radar for positioning and monitoring and methods of using same
US9575560B2 (en) 2014-06-03 2017-02-21 Google Inc. Radar-based gesture-recognition through a wearable device
US9811164B2 (en) 2014-08-07 2017-11-07 Google Inc. Radar-based gesture sensing and data transmission
US9921660B2 (en) 2014-08-07 2018-03-20 Google Llc Radar-based gesture recognition
US9588625B2 (en) 2014-08-15 2017-03-07 Google Inc. Interactive textiles
US10268321B2 (en) 2014-08-15 2019-04-23 Google Llc Interactive textiles within hard objects
US9778749B2 (en) 2014-08-22 2017-10-03 Google Inc. Occluded gesture recognition
US11169988B2 (en) 2014-08-22 2021-11-09 Google Llc Radar recognition-aided search
US9600080B2 (en) 2014-10-02 2017-03-21 Google Inc. Non-line-of-sight radar-based gesture recognition
KR20160057127A (en) * 2014-11-13 2016-05-23 삼성전자주식회사 Display apparatus and control method thereof
US10016162B1 (en) 2015-03-23 2018-07-10 Google Llc In-ear health monitoring
US9983747B2 (en) 2015-03-26 2018-05-29 Google Llc Two-layer interactive textiles
WO2016170011A1 (en) 2015-04-20 2016-10-27 Resmed Sensor Technologies Limited Gesture recognition with sensors
EP3885882A1 (en) 2015-04-30 2021-09-29 Google LLC Rf-based micro-motion tracking for gesture tracking and recognition
CN107466389B (en) 2015-04-30 2021-02-12 谷歌有限责任公司 Method and apparatus for determining type-agnostic RF signal representation
CN111880650A (en) 2015-04-30 2020-11-03 谷歌有限责任公司 Gesture recognition based on wide field radar
US9693592B2 (en) 2015-05-27 2017-07-04 Google Inc. Attaching electronic components to interactive textiles
US10088908B1 (en) 2015-05-27 2018-10-02 Google Llc Gesture detection and interactions
US10817065B1 (en) 2015-10-06 2020-10-27 Google Llc Gesture recognition using multiple antenna
US9837760B2 (en) 2015-11-04 2017-12-05 Google Inc. Connectors for connecting electronics embedded in garments to external devices
US10118696B1 (en) 2016-03-31 2018-11-06 Steven M. Hoffberg Steerable rotating projectile
US10492302B2 (en) 2016-05-03 2019-11-26 Google Llc Connecting an electronic component to an interactive textile
WO2017200570A1 (en) 2016-05-16 2017-11-23 Google Llc Interactive object with multiple electronics modules
US10579150B2 (en) 2016-12-05 2020-03-03 Google Llc Concurrent detection of absolute distance and relative movement for sensing action gestures
DE102017216622C5 (en) 2017-09-20 2023-11-16 BSH Hausgeräte GmbH Household appliance with a sensor
US11520028B2 (en) * 2018-01-10 2022-12-06 Richwave Technology Corp. Occupancy detection using multiple antenna motion sensing
US11712637B1 (en) 2018-03-23 2023-08-01 Steven M. Hoffberg Steerable disk or ball
US10775482B2 (en) * 2018-04-11 2020-09-15 Infineon Technologies Ag Human detection and identification in a setting using millimeter-wave radar
US11183772B2 (en) 2018-09-13 2021-11-23 Infineon Technologies Ag Embedded downlight and radar system
US11442550B2 (en) * 2019-05-06 2022-09-13 Samsung Electronics Co., Ltd. Methods for gesture recognition and control
US11191466B1 (en) 2019-06-28 2021-12-07 Fitbit Inc. Determining mental health and cognitive state through physiological and other non-invasively obtained data
IL269198B (en) * 2019-09-09 2022-01-01 Goodtechcom Ltd System and method for translocating and buffering cellular radiation source
KR20240004262A (en) 2021-02-25 2024-01-11 체리쉬 헬스, 인크. Technology for tracking objects within a defined area

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5519400A (en) * 1993-04-12 1996-05-21 The Regents Of The University Of California Phase coded, micro-power impulse radar motion sensor
US6211863B1 (en) * 1998-05-14 2001-04-03 Virtual Ink. Corp. Method and software for enabling use of transcription system as a mouse
US20030033449A1 (en) * 2001-08-13 2003-02-13 Frantz Gene A. Universal decoder for use in a network media player
US20050163302A1 (en) * 2004-01-22 2005-07-28 Mock Von A. Customer service system and method using physiological data
US6954145B2 (en) * 2002-02-25 2005-10-11 Omron Corporation Proximate sensor using micro impulse waves for monitoring the status of an object, and monitoring system employing the same
US20070149282A1 (en) * 2005-12-27 2007-06-28 Industrial Technology Research Institute Interactive gaming method and apparatus with emotion perception ability
US20070214371A1 (en) * 2006-03-10 2007-09-13 Hon Hai Precision Industry Co., Ltd. Computer sleep/awake circuit
US20080221401A1 (en) * 2006-10-27 2008-09-11 Derchak P Alexander Identification of emotional states using physiological responses
US20090328089A1 (en) * 2007-05-16 2009-12-31 Neurofocus Inc. Audience response measurement and tracking system
US20100234720A1 (en) * 2003-06-04 2010-09-16 Tupin Jr Joe Paul System and method for extracting physiological data using ultra-wideband radar and improved signal processing techniques

Family Cites Families (100)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB1378754A (en) 1971-09-07 1974-12-27 Peak Technologies Ltd Patient monitoring
US6778672B2 (en) 1992-05-05 2004-08-17 Automotive Technologies International Inc. Audio reception control arrangement and method for a vehicle
US7663502B2 (en) 1992-05-05 2010-02-16 Intelligent Technologies International, Inc. Asset system control arrangement and method
US4513748A (en) 1983-08-30 1985-04-30 Rca Corporation Dual frequency heart rate monitor utilizing doppler radar
US4958638A (en) 1988-06-30 1990-09-25 Georgia Tech Research Corporation Non-contact vital signs monitor
US4931865A (en) * 1988-08-24 1990-06-05 Sebastiano Scarampi Apparatus and methods for monitoring television viewers
US5226425A (en) 1991-09-10 1993-07-13 Ralin, Inc. Portable ECG monitor/recorder
US5305748A (en) 1992-06-05 1994-04-26 Wilk Peter J Medical diagnostic system and related method
DE4241664C2 (en) 1992-12-04 1995-04-06 Borus Spezialverfahren Electronic life detection system
US5774091A (en) 1993-04-12 1998-06-30 The Regents Of The University Of California Short range micro-power impulse radar with high resolution swept range gate with damped transmit and receive cavities
US5361070B1 (en) 1993-04-12 2000-05-16 Univ California Ultra-wideband radar motion sensor
DE4329898A1 (en) 1993-09-04 1995-04-06 Marcus Dr Besson Wireless medical diagnostic and monitoring device
DE9400950U1 (en) 1994-01-20 1995-08-24 Selectronic Vertriebs Gmbh Device for detecting living bodies and their use
US5507291A (en) 1994-04-05 1996-04-16 Stirbl; Robert C. Method and an associated apparatus for remotely determining information as to person's emotional state
US5573012A (en) 1994-08-09 1996-11-12 The Regents Of The University Of California Body monitoring and imaging apparatus and method
US5853364A (en) 1995-08-07 1998-12-29 Nellcor Puritan Bennett, Inc. Method and apparatus for estimating physiological parameters using model-based adaptive filtering
US5850470A (en) 1995-08-30 1998-12-15 Siemens Corporate Research, Inc. Neural network for locating and recognizing a deformable object
US6292688B1 (en) * 1996-02-28 2001-09-18 Advanced Neurotechnologies, Inc. Method and apparatus for analyzing neurological response to emotion-inducing stimuli
US6837436B2 (en) 1996-09-05 2005-01-04 Symbol Technologies, Inc. Consumer interactive shopping system
US5905436A (en) 1996-10-24 1999-05-18 Gerontological Solutions, Inc. Situation-based monitoring system
US6062216A (en) 1996-12-27 2000-05-16 Children's Medical Center Corporation Sleep apnea detector system
US6011477A (en) 1997-07-23 2000-01-04 Sensitive Technologies, Llc Respiration and movement monitoring system
US7196720B2 (en) * 1998-03-06 2007-03-27 Intel Corporation Method and apparatus for powering on an electronic device with a video camera that detects motion
US6501393B1 (en) 1999-09-27 2002-12-31 Time Domain Corporation System and method for using impulse radio technology to track and monitor vehicles
US6492906B1 (en) 1998-03-23 2002-12-10 Time Domain Corporation System and method using impulse radio technology to track and monitor people under house arrest
US6466125B1 (en) 1998-03-23 2002-10-15 Time Domain Corporation System and method using impulse radio technology to track and monitor people needing health care
US6489893B1 (en) 1998-03-23 2002-12-03 Time Domain Corporation System and method for tracking and monitoring prisoners using impulse radio technology
US6454708B1 (en) 1999-04-15 2002-09-24 Nexan Limited Portable remote patient telemonitoring system using a memory card or smart card
US6351246B1 (en) 1999-05-03 2002-02-26 Xtremespectrum, Inc. Planar ultra wide band antenna with integrated electronics
US6177903B1 (en) 1999-06-14 2001-01-23 Time Domain Corporation System and method for intrusion detection using a time domain radar array
US7592944B2 (en) 1999-06-14 2009-09-22 Time Domain Corporation System and method for intrusion detection using a time domain radar array
US6218979B1 (en) 1999-06-14 2001-04-17 Time Domain Corporation Wide area time domain radar array
DE19929328A1 (en) 1999-06-26 2001-01-04 Daimlerchrysler Aerospace Ag Device for long-term medical monitoring of people
US6608910B1 (en) 1999-09-02 2003-08-19 Hrl Laboratories, Llc Computer vision method and apparatus for imaging sensors for recognizing and tracking occupants in fixed environments under variable illumination
US6730023B1 (en) 1999-10-15 2004-05-04 Hemopet Animal genetic and health profile database management
US6661345B1 (en) 1999-10-22 2003-12-09 The Johns Hopkins University Alertness monitoring system
US6524239B1 (en) 1999-11-05 2003-02-25 Wcr Company Apparatus for non-instrusively measuring health parameters of a subject and method of use thereof
US6611783B2 (en) 2000-01-07 2003-08-26 Nocwatch, Inc. Attitude indicator and activity monitoring device
JP4877696B2 (en) 2000-04-25 2012-02-15 ガネット サテライト インフォメーション ネットワーク インコーポレイテッド Information portal
ES2260245T3 (en) 2000-06-23 2006-11-01 Bodymedia, Inc. SYSTEM TO CONTROL HEALTH, WELFARE AND EXERCISE.
JP2002112969A (en) 2000-09-02 2002-04-16 Samsung Electronics Co Ltd Device and method for recognizing physical and emotional conditions
US7106885B2 (en) 2000-09-08 2006-09-12 Carecord Technologies, Inc. Method and apparatus for subject physical position and security determination
US6445298B1 (en) 2000-12-21 2002-09-03 Isaac Shepher System and method for remotely monitoring movement of individuals
US6611206B2 (en) 2001-03-15 2003-08-26 Koninklijke Philips Electronics N.V. Automatic system for monitoring independent person requiring occasional assistance
US6993378B2 (en) * 2001-06-25 2006-01-31 Science Applications International Corporation Identification by analysis of physiometric variation
US7728870B2 (en) * 2001-09-06 2010-06-01 Nice Systems Ltd Advanced quality management and recording solutions for walk-in environments
US7023499B2 (en) * 2001-09-21 2006-04-04 Williams Cassandra S Television receiver with motion sensor
US6753780B2 (en) 2002-03-15 2004-06-22 Delphi Technologies, Inc. Vehicle occupant detection system and method using radar motion sensor
US8947347B2 (en) * 2003-08-27 2015-02-03 Sony Computer Entertainment Inc. Controlling actions in a video game unit
EP1545680B1 (en) 2002-07-25 2010-09-08 Boston Scientific Limited Medical device for navigation through anatomy
WO2004013611A2 (en) 2002-08-01 2004-02-12 California Institute Of Technology Remote-sensing method and device
US20070100666A1 (en) 2002-08-22 2007-05-03 Stivoric John M Devices and systems for contextual and physiological-based detection, monitoring, reporting, entertainment, and control of other devices
DE10259522A1 (en) 2002-12-19 2004-07-01 Robert Bosch Gmbh Radar-based sensing of the position and / or movement of the body or in the body of living beings
EP1667579A4 (en) 2003-09-12 2008-06-11 Bodymedia Inc Method and apparatus for measuring heart related parameters
US20060061504A1 (en) 2004-09-23 2006-03-23 The Regents Of The University Of California Through wall detection and tracking system
US7683883B2 (en) * 2004-11-02 2010-03-23 Pierre Touma 3D mouse and game controller based on spherical coordinates system and system for use
US20060218244A1 (en) 2005-03-25 2006-09-28 Rasmussen Jung A Methods and systems for automating the control of objects within a defined human environment
US20060001545A1 (en) * 2005-05-04 2006-01-05 Mr. Brian Wolf Non-Intrusive Fall Protection Device, System and Method
WO2006126186A2 (en) 2005-05-26 2006-11-30 Yissum Research Development Company Of The Hebrew University Of Jerusalem Method and system for determination of physiological conditions and emotional states
JP2009501044A (en) 2005-07-15 2009-01-15 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Device for detecting heart activity
US7414705B2 (en) * 2005-11-29 2008-08-19 Navisense Method and system for range measurement
CA2531533C (en) 2005-12-28 2013-08-06 Bce Inc. Session-based public key infrastructure
US20100201512A1 (en) 2006-01-09 2010-08-12 Harold Dan Stirling Apparatus, systems, and methods for evaluating body movements
US8502729B2 (en) 2006-01-30 2013-08-06 Lawrence Livermore National Security, Llc Ultra-wideband radar sensors and networks
WO2007106806A2 (en) 2006-03-13 2007-09-20 Nielsen Media Research, Inc. Methods and apparatus for using radar to monitor audiences in media environments
US7567200B1 (en) 2006-04-27 2009-07-28 Josef Osterweil Method and apparatus for body position monitor and fall detect ion using radar
US7916066B1 (en) 2006-04-27 2011-03-29 Josef Osterweil Method and apparatus for a body position monitor and fall detector using radar
US20080074307A1 (en) 2006-05-17 2008-03-27 Olga Boric-Lubecke Determining presence and/or physiological motion of one or more subjects within a doppler radar system
US7525434B2 (en) 2006-06-09 2009-04-28 Intelleflex Corporation RF systems and methods for tracking and singulating tagged items
US7757803B2 (en) 2006-07-14 2010-07-20 Richard Fiske Motor vehicle operator identification and maximum speed limiter
US20080240379A1 (en) 2006-08-03 2008-10-02 Pudding Ltd. Automatic retrieval and presentation of information relevant to the context of a user's conversation
US20100106475A1 (en) 2006-08-04 2010-04-29 Auckland Uniservices Limited Biophysical virtual model database and applications
EP2053963A2 (en) 2006-08-17 2009-05-06 Koninklijke Philips Electronics N.V. Dynamic body state display device
BRPI0716106A2 (en) 2006-09-07 2014-07-01 Procter & Gamble METHODS FOR MEASURING EMOTIONAL RESPONSE AND PREFERENCE OF CHOICE
US20080098448A1 (en) * 2006-10-19 2008-04-24 Sony Computer Entertainment America Inc. Controller configured to track user's level of anxiety and other mental and physical attributes
US8577446B2 (en) 2006-11-06 2013-11-05 Bobby Kyle Stress detection device and methods of use thereof
US8157730B2 (en) 2006-12-19 2012-04-17 Valencell, Inc. Physiological and environmental monitoring systems and methods
US20090017910A1 (en) * 2007-06-22 2009-01-15 Broadcom Corporation Position and motion tracking of an object
WO2008121221A1 (en) 2007-03-30 2008-10-09 Seesaw Networks Inc. Measuring a location based advertising campaign
US7865916B2 (en) * 2007-07-20 2011-01-04 James Beser Audience determination for monetizing displayable content
US8243141B2 (en) * 2007-08-20 2012-08-14 Greenberger Hal P Adjusting a content rendering system based on user occupancy
US20090112697A1 (en) 2007-10-30 2009-04-30 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Providing personalized advertising
CN101925916B (en) 2007-11-21 2013-06-19 高通股份有限公司 Method and system for controlling electronic device based on media preferences
US7999741B2 (en) 2007-12-04 2011-08-16 Avaya Inc. Systems and methods for facilitating a first response mission at an incident scene using precision location
US8636670B2 (en) 2008-05-13 2014-01-28 The Invention Science Fund I, Llc Circulatory monitoring systems and methods
US20090164287A1 (en) 2007-12-24 2009-06-25 Kies Jonathan K Method and apparatus for optimizing presentation of media content on a wireless device based on user behavior
US8284990B2 (en) 2008-05-21 2012-10-09 Honeywell International Inc. Social network construction based on data association
US20090296997A1 (en) * 2008-06-03 2009-12-03 James Rocheford Method and apparatus for securing a computer
US7692573B1 (en) 2008-07-01 2010-04-06 The United States Of America As Represented By The Secretary Of The Navy System and method for classification of multiple source sensor measurements, reports, or target tracks and association with uniquely identified candidate targets
US8125331B2 (en) 2008-08-27 2012-02-28 The Invention Science Fund I, Llc Health-related signaling via wearable items
US8094009B2 (en) 2008-08-27 2012-01-10 The Invention Science Fund I, Llc Health-related signaling via wearable items
US8130095B2 (en) 2008-08-27 2012-03-06 The Invention Science Fund I, Llc Health-related signaling via wearable items
US8284046B2 (en) 2008-08-27 2012-10-09 The Invention Science Fund I, Llc Health-related signaling via wearable items
US20100259395A1 (en) 2009-04-08 2010-10-14 General Electric Company Patient monitoring system and method
WO2011011413A2 (en) * 2009-07-20 2011-01-27 University Of Florida Research Foundation, Inc. Method and apparatus for evaluation of a subject's emotional, physiological and/or physical state with the subject's physiological and/or acoustic data
US8629938B2 (en) * 2009-10-05 2014-01-14 Sony Corporation Multi-point television motion sensor system and method
US8260269B2 (en) 2009-11-25 2012-09-04 Visa International Service Association Input device with an accelerometer
US20110307210A1 (en) 2010-06-14 2011-12-15 International Business Machines Corporation System and method for tracking a mobile node
US9095316B2 (en) 2011-04-20 2015-08-04 Masimo Corporation System for generating alarms based on alarm patterns
US20120326873A1 (en) 2011-06-10 2012-12-27 Aliphcom Activity attainment method and apparatus for a wellness application using data from a data-capable band

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5519400A (en) * 1993-04-12 1996-05-21 The Regents Of The University Of California Phase coded, micro-power impulse radar motion sensor
US6211863B1 (en) * 1998-05-14 2001-04-03 Virtual Ink. Corp. Method and software for enabling use of transcription system as a mouse
US20030033449A1 (en) * 2001-08-13 2003-02-13 Frantz Gene A. Universal decoder for use in a network media player
US6954145B2 (en) * 2002-02-25 2005-10-11 Omron Corporation Proximate sensor using micro impulse waves for monitoring the status of an object, and monitoring system employing the same
US20100234720A1 (en) * 2003-06-04 2010-09-16 Tupin Jr Joe Paul System and method for extracting physiological data using ultra-wideband radar and improved signal processing techniques
US20050163302A1 (en) * 2004-01-22 2005-07-28 Mock Von A. Customer service system and method using physiological data
US20070149282A1 (en) * 2005-12-27 2007-06-28 Industrial Technology Research Institute Interactive gaming method and apparatus with emotion perception ability
US20070214371A1 (en) * 2006-03-10 2007-09-13 Hon Hai Precision Industry Co., Ltd. Computer sleep/awake circuit
US20080221401A1 (en) * 2006-10-27 2008-09-11 Derchak P Alexander Identification of emotional states using physiological responses
US20090328089A1 (en) * 2007-05-16 2009-12-31 Neurofocus Inc. Audience response measurement and tracking system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MICHAHELLES, F. ET AL.: "Less Contact: Heart-rate detection without even touching the user.", EIGHTH INTEMATIONAL SYMPOSIUM ON WEARABLE COMPUTERS, 2004 (ISWC 2004), vol. 1, 31 October 2004 (2004-10-31), pages 4 - 7, XP010749620, Retrieved from the Internet <URL:http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=8amumber=1364682&isnumber-29895> [retrieved on 20111226], DOI: doi:10.1109/ISWC.2004.27 *

Also Published As

Publication number Publication date
US20120068876A1 (en) 2012-03-22
US9069067B2 (en) 2015-06-30

Similar Documents

Publication Publication Date Title
US9069067B2 (en) Control of an electronic apparatus using micro-impulse radar
US9164167B2 (en) Personal electronic device with a micro-impulse radar
US9024814B2 (en) Tracking identities of persons using micro-impulse radar
US9103899B2 (en) Adaptive control of a personal electronic device responsive to a micro-impulse radar
US20110166937A1 (en) Media output with micro-impulse radar feedback of physiological response
US10671231B2 (en) Electromagnetic interference signal detection
US10141929B2 (en) Processing electromagnetic interference signal using machine learning
US20210365124A1 (en) Radar-Enabled Sensor Fusion
US20110166940A1 (en) Micro-impulse radar detection of a human demographic and delivery of targeted media content
US20210103348A1 (en) Facilitating User-Proficiency in Using Radar Gestures to Interact with an Electronic Device
US20210064144A1 (en) Methods for Reliable Acceptance of User Non-Contact Gesture Inputs for a Mobile Device
Zhao et al. Towards low-cost sign language gesture recognition leveraging wearables
Tang Automated detection of puffing and smoking with wrist accelerometers
US10101869B2 (en) Identifying device associated with touch event
US9019149B2 (en) Method and apparatus for measuring the motion of a person
US20210232228A1 (en) Adaptive thresholding and noise reduction for radar data
US20210142214A1 (en) Machine-learning based gesture recognition using multiple sensors
EP3335099B1 (en) Electromagnetic interference signal detection
US20230325719A1 (en) Machine-learning based gesture recognition using multiple sensors
US20160106329A1 (en) Utilizing different color channels for rgb-image-based differential heart rate detection
US9151834B2 (en) Network and personal electronic devices operatively coupled to micro-impulse radars
KR102084209B1 (en) Electromagnetic interference signal detection
Nadig Techniques for Detecting and Classifying User Behavior Through the Fusion of Ultrasonic Proximity Data and Doppler-Shift Velocity Data
EP3335317B1 (en) Processing electromagnetic interference signal using machine learning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11825573

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11825573

Country of ref document: EP

Kind code of ref document: A1