WO1994003020A1

WO1994003020A1 - Voice operated remote control device

Info

Publication number: WO1994003020A1
Application number: PCT/US1993/006627
Authority: WO
Inventors: W. Michael Bissonnette; Douglas L. Turner
Original assignee: Voice Powered Technology International, Inc.
Priority date: 1992-07-17
Filing date: 1993-07-15
Publication date: 1994-02-03
Also published as: MX9304336A; AU4773193A

Abstract

A voice operated remote control device (figure 5A, 5B) accepts voice commands spoken by the user (301, 307, 310), performs voice recognition pattern matching on the spoken word comparing the same against pretrained templates to determine the appropriate corresponding command (308, 311), determines the specific IR remote control code or set of IR codes that represent the function corresponding to the command and transmits the IR remote control codes to the electronic equipement to be controlled (309, 313). The voice operated remote control device can be operated by multiple users either by voice or by manual control of the selected function of the remote control. The voice commands can represent either real time commands, which are transmitted immediately after the determination of the voice command, or programmed delay events which delay the transmission of the remote control codes or set of remote control codes for a preset period of time specified by the user.

Description

VOICE OPERATED REMOTE CONTROL DEVICE

FIELD OF THE INVENTION

The present invention is directed to a remote control apparatus and in particular to a portable universal remote control which is voice operated.

BACKGROUND OF THE INVENTION

Voice recognition has long been accepted as one of the more user friendly interfaces between man and machine. Nevertheless, a number of impediments have prevented voice recognition from being more widely used in applications where portability is a significant factor. These impediments can be broken down generally into software and hardware limitations inherent in prior art voice recognition techniques. For example, many companies as well as government agencies such as DARPA have invested years of research and resources in developing complex voice recognition software algorithms, but such algorithms ultimately require the performance of a large personal computer (PC) or mainframe to implement. Voice recognition algorithms also typically require sophisticated hardware in the form of either digital signal processors (DSPs) or 16 bit microprocessors. This hardware is expensive, requires numerous support devices, is bulky and consumes a significant amount of power. These considerations explain why voice recognition has traditionally been confined to larger computer-based environments. For example, published U.K. Patent Application No. GB 2220290A (Harvey) uses voice recognition for control of equipment, but again the voice recognition is implemented on a large and bulky PC based platform which is not suited for portable applications.

There are, nevertheless, many portable low cost applications where voice recognition would be desirable. For example, it is often desirable to remotely control home entertainment electronic components, such as a video cassette recorder (VCR) , television, cable box, satellite dish controller, or stereo with a portable remote control. These components typically include their own separate remote controls. These remote controls typically use transmitted infra-red (IR) or radio-frequency (RF) signals to effectuate control of the various electronic components. Since each component has its own separate remote control, the number of separate remote controls can soon become unwieldy and cluttersome. In order to reduce the number of separate remote controls, a universal remote control must be used which transmits the equivalent signals of each of the separate remote controls. But, as explained above, the limitations of the available voice recognition systems preclude such systems from being used in smaller, portable environments. While the prior art includes remote control devices, these typically lack voice recognition capability for the reasons mentioned above. An example of such a typical prior art remote control device can be seen in U.S. Patent No. 4,856,081 (Smith).

Accordingly, an objective of the present invention is to provide a portable, low-power voice-operated remote control device. The term "portable" as used throughout the specification generally means that the present invention is easily carried but this does not mean that the teachings of the present invention could not be used in a compact embodiment that is mounted on a wall for example.

SUMMARY OF THE INVENTION

In accordance with one exemplary embodiment of the present invention, there is provided a portable remote control device operated by a voice command for controlling one or more electronic components each which in turn is responsive to a set of respective remote control codes. This device includes a voice input circuit for receiving the voice command and generating a voice signal based on the voice command; an analog to digital converter coupled to the input circuit for converting the voice signal into a first digital voice signal; a reference memory (a RAM) for storing a plurality of reference digital voice templates and the remote control codes; a program memory (a ROM) for storing a control program; a processor coupled to the reference memory, converter and program memory for converting the first digital voice signal into a first voice template and for executing the control program to determine whether the first voice template is substantially equivalent to one of the reference voice templates, and for selecting one of the remote control codes based on the first voice template; and finally, a remote control code transmitter coupled to the processor for transmitting the selected remote control code.

In addition, an input capture circuit may be coupled to the processor and used for capturing the remote control codes from the separate remote controllers. A keyboard with a plurality of control keys is also coupled to the processor. When a key is depressed by the user, the processor selects an appropriate remote control code to be output. In this manner, the device can be controlled manually as well as by voice command. Furthermore, a liquid crystal display (LCD) is provided for displaying the voice command to the user

As used herein, the term "component system" (or system of components) is used generally to refer to a home entertainment system comprised of at least two separate electronic components such as a TV, VCR, cable box, satellite receiver, etc. A "component function" refers generally to functions that require control of only a single electronic component, such as a "channel" function, which should only affect one component in the component system. When a component command is entered by the user (either by voice or by means of a keyboard) the component function is generated by outputting IR remote control codes to the particular component. A "system function" conversely refers generally to functions that require control of more than one of the electronic components in the component system, such as the "record" function described above, which requires that the VCR as well as the cable box be controlled. When a system command is entered by the user (again either by voice or by means of a keyboard) the system function is generated by outputting a specific sequence of IR remote control codes to two or more components.

As mentioned, a typical component system includes a television, a videocassette recorder which can be programmed to perform a record operation to record information on a videocassette, and a cable box converter. Often, these components share one or more functions in common (such as a channel select for example) . The present device, in response to a user's voice command to perform one of the functions which the components share in common, selects and outputs one or more remote control codes to only one of the electronic components to effectuate the requested functional operation. In this manner, for example, only the cable box channel would be changed in response to a "channel" voice command. Furthermore, the present device also includes means to intelligently select, in response to a user's voice command to perform a record operation, a sequence of whichever remote control codes are necessary to transmit to the electronic components to effectuate a system command, such as a record operation. In this manner, for example, a "record" voice system command would cause the present invention to output a sequence of remote control codes to cause a VCR to perform a record operation, but also to cause a remote control code to be sent to either the VCR or cable box converter for purposes of setting the recorded channel. Finally, in accordance with a more specific embodiment of the present invention, the user may also enter, as part of the system command, programmed sequence timing data to effectuate remote control of the system components or single components at a later time, such as, for example, to record a future TV broadcast on a VCR. The present invention then generates the necessary sequence of remote control codes at the later time based on the timing data. For a future record operation, for example, a first portion of the sequence of remote control codes is generated at a first time based on the timing data (for example at the start time) while a second portion of the sequence of remote control codes is generated at a second time based on the timing data (such as at the stop time) . Thus, the present invention provides a simple way for a user to control a number of home entertainment electronic components by means of a voice-operated, hand-held, portable, battery-operated universal remote control device. While the present invention is described in connection with electronic components used in a home entertainment system, it will become apparent to skilled artisans that the invention is applicable for any number of environments requiring control of separate electronic components. BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, features and advantages of the present invention will become evident from the ensuing "Detailed Description of the Invention" when read in conjunction with the accompanying drawings in which:

FIG.l is a block diagram of a voice operated remote control device in accordance with the present invention;

FIG.2 is a flow diagram detailing the setup mode software routine implemented in a control program forming part of a program ROM;

FIG.3 is a flow diagram describing the voice operation mode software routine implemented in the control program forming part of a program ROM; FIG.4 is a flow diagram describing the manual operation mode software routine implemented in the control program forming part of a program ROM;

FIGs. 5A and 5B show a housing which may be used to enclose the present invention, including casing, keyboard keys, sliding door and LCD;

FIGs. 6A and 6B show the LCD used in the present invention;

FIG. 7 is a flow diagram describing the channel control configuration software routine implemented in the control program forming part of a program ROM;

FIG. 8 is a flow diagram describing the volume control configuration software routine implemented in the control program forming part of a program ROM;

FIG. 9 is a flow diagram describing the playback and stop control configuration software routine implemented in the control program forming part of a program ROM. DETAILED DESCRIPTION OF THE INVENTION

A. STRUCTURE OF THE VOICE CONTROLLER

The hardware of the present invention consists of several functional sections as shown in Figure 1. As can be seen therein, a Voice Controller 1 embodying the present invention consists generally of an Analog Voice Input 2, a Keyboard 3, a Battery Circuit 4, IR Input 5, a ROM Code Library 7, an External DRAM 8, a Shift Register 9, an LCD 10, a Piezo Beeper 11, an IR Output 12 and a Microcontroller 20. These circuits are described in detail below.

Looking at Figure l and proceeding counterclockwise, the first functional block is IR Input 5. This circuit is used to capture IR remote control codes for later control of each unit of the user's equipment, such as their TV, VCR, cable box, satellite dish receiver and the like. The present invention includes sufficient RAM capacity to learn and store the IR codes from several separate remote control units. While the present embodiment is directed to an infrared (IR) remote control signal capturing circuit, it will be apparent to the skilled artisan that a radio-frequency (RF) capturing circuit could easily be used instead for learning RF codes from RF remote controllers. Moreover the IR or RF capturing circuit could easily consist of a direct electrically connected interface between the user's remote controller and the present invention in lieu of the present wireless embodiment.

IR Input 5 is comprised of well-known IR remote control capture circuits. A typical example of such a circuit can be found in U.S. Patent No. 4,857,898 (Smith) which is hereby incorporated by reference as if fully set forth herein. In the present invention, this circuit consists of a conventional input photodiode and a transistor amplifier (not shown) . IR Input 5 is activated to capture IR remote control codes when placed into the "SETUP" mode by the user as described below. This section of the hardware is normally powered down to minimize power consumption from the battery. To learn the IR remote control codes from the user's remote controllers, the user points the transmitting end of the remote controller to be learned at the input IR window (not shown) of IR Input 5. The IR code from the teaching remote control passes through the IR window to a photodiode in IR Input 5 which converts the IR signal to an electrical signal. The output of the IR Input 5 photodiode is signal conditioned by a conventional two transistor amplifier which converts the analog electrical signal to a digital electrical signal. IR Input 5 handles IR remote control codes with carrier frequencies from 20 KHz to 70 KHz and IR pulse codes with long and short "on" times. The digital signal translated from the IR code from the user's remote control is input into an interrupt input of the Microcontroller 20 which reads the code, converts the code to a particular format and stores the converted code in RAM 23 which is a reference memory for later use to control the separate components (TV, VCR and cable box) of the user's systems. While one embodiment of IR Input 5 has been shown, it would be apparent to one skilled in the art that a number of acceptable alternatives which capture remote control codes could be used in place of the circuit shown. The IR remote control codes from the user's separate remote controllers are stored by the Microcontroller 20 until such time as the user desires to control one of the separate components in their system. The IR remote control codes are output by IR Output 12. IR Output 12, as with IR input 5, is also conventional, and consists of a well-known circuit including three (3) infra-red light emitting diodes (LEDs) driven by two transistors which in turn are driven from an output port of the Microcontroller 20. A representative example of the state of the art in such circuits can be seen in the Smith patent referred to above, and also in U.S. Patent No. 4,425,647

(Collins et. al.) which is also hereby incorporated by reference.

In the present invention, three (3) light emitting diodes provide the coverage needed for operation in the various physical positions that the present invention may be used. Two LEDs are positioned at 90 degree angles from each other, while the third light emitting diode is positioned at the center of the 90 degree angle, 45 degrees from the two end diodes. The IR diodes used are model No. QED243 by Quality Technology, which are known for their ability to provide a wide dispersion IR beam of light. The axial dispersion of the 3 IR LEDs are 130 degrees with relatively full coverage in-between the diodes In operation, the 3 LEDs, which are in series with each other, are driven in a well-known circuit configuration as explained earlier. A first series pass transistor (not shown) is also in series with the diodes and is driven directly from an unregulated battery voltage from Battery Circuit 4. The LEDs are configured in the circuit to provide their own current limiting. A second transistor (not shown) buffers the output port of the Microcontroller 20, driving the base of the first series pass transistor thus driving the LEDs. The drive circuit to the LEDs also provides protection to ensure that the LEDs will not remain on all the time in the event Microcontroller 20 fails to toggle its I/O line to turn the LEDs off.

While one embodiment of IR Output 12 has been shown, a number of equivalent circuits which output IR remote control codes could be used in place of the circuit shown. Moreover, as with the remote control capture circuit described above, the remote control output circuit (IR Output 12) of the present invention could also easily be implemented by a skilled artisan to accommodate RF rather than IR signals. Moreover, IR Output 12 could easily output the remote control codes to the user's separate electronic components via wire rather than through wireless air transmission.

The present invention may be operated by way of voice commands as explained further below. An Analog Voice Input 2 converts the audio information in the user's voice to an analog electrical signal and also conditions this electrical signal for processing by Microcontroller 20. The present invention may be enclosed in a plastic casing (Fig. 5A) . Referring to Figure 1 again, in a preferred embodiment of the present invention. Analog Voice Input 2 includes a microphone that is mounted against the front of the casing with a small opening through the plastic located at the center of the microphone. The microphone is physically mounted to a printed circuit board containing the present invention with a rubber grommet (not shown) . This grommet not only provides a means to physically mount the microphone to the printed circuit board but also provides mechanical isolation required between the hardware and the microphone. This mechanical isolation isolates the microphone from any mechanical noise induced within the unit when the user depresses the voice switch as well as mechanical noise when holding the plastic. In operation, the voice signal output of the microphone is fed into an analog input section of Analog Voice Input 2. The signal is then conditioned by well known electronic circuits that amplify and filter the voice input signal from the microphone prior to going to an analog to digital converter (ADC) 24 in Microcontroller 20. An example of a typical prior art speech recognition amplifying and conditioning circuit can be seen in U.S. Patent No. 4,054,749 (Suzuki et. al.) which is incorporated by reference herein. In a preferred embodiment. Analog Voice Input 2 consists of three stages of gain and filtering. A first stage provides a signal gain of 40 with frequency emphasis characteristics of 6 db per octave at the upper end of the band pass. The frequency emphasis is used to amplify the voice information at the upper end of the frequency spectrum which has been determined by the inventors to enhance the voice recognition capability of the present invention. A second stage of Analog Voice Input 2 consists of an amplifier circuit that provides for analog band pass filtering. This filtering band passes maximum useful voice information while filtering out unwanted noise outside the band pass. The band pass section of this analog circuit has minimal gain with a frequency response roll off characteristic of 18 db per octave. The overall frequency response of the analog section is 300 to 4800 Hz. A third and final stage of Analog Voice Input 2 provides for analog gain control (AGC) of the voice input signal. Microcontroller 20 can adjust the level of the ADC input signal for maximum signal to noise ratio, thus enhancing recognition performance. The AGC compensates for variations in audio levels as the user speaks and also compensates volume variations which can result from the user speaking from various distances into the microphone. To maximize on battery life, Analog Voice Input 2 is also powered up only when the user activates one of three voice keys described below to speak into the unit. While one embodiment of Analog Voice Input 2 is shown herein, it would be apparent to one skilled in the art that any equivalent circuit for conditioning audio voice information could be used in lieu thereof.

The output of Analog Voice Input 2 feeds into an 8-bit ADC 24 within Microcontroller 20 which samples the data at 9.6 Khz. ADC 24 then outputs a digital signal representing the input analog voice signal from Analog Voice Input 2. Microcontroller 20 then processes the digital voice signal by means of microprocessor 21 and a voice recognition software routine that is part of a control program stored in ROM 22. The digital voice signal is converted into a voice template that is compared against previously stored voice templates of the user's voice. The program then decodes the voice templates as explained further below. Because of its compact and efficient design the present invention consumes minimal electrical power and can be powered entirely by conventional batteries. Batteries, however, eventually lose their charge, and can render the device inoperable. The present invention also includes a mechanism for informing the user at an early stage when the power level of the batteries is running low. As can be seen in Figure 1, Battery Circuit 4 provides an analog output to the Microcontroller 20 level that is read by the software program in ROM 22 to determine when the present invention is operating below a preset first voltage value. In a preferred embodiment, the first value is set to 5.2 volts. The output of Battery Circuit 4 is a digital signal driven by a operational amplifier configured as a comparator and read through a second analog to digital converter port of the Microcontroller 20. If the output of the analog to digital converter read by the software in ROM 22 is below the specified battery voltage, Microcontroller 20 outputs a warning message to the user on LCD 10.

When the battery voltage falls below a second preset value, a second output designated "change battery" is output to Microcontroller 20. This output indicates that the battery level has fallen below 4.9 volts. A latched output signal produced by a voltage regulator within Battery Circuit 4 prevents the present invention from going into and out of the "change battery" mode each time the system is operated. This could happen, for example during operation when IR Output 12 photodiodes are transmitting, causing the battery voltage typically to drop lower than when sitting in an idle or clock mode. Without the latch mode and accompanying hysteresis, the battery voltage would momentarily go low then high causing Microcontroller 20 to go into and out of the "change battery" condition. Again, while one embodiment of the Battery Circuit 5 has been described, a number of alternatives that provide battery information to the user could be implemented instead.

Next, it is often desirable to provide audible feedback to the user of the present invention, such as when the device has finished learning a specific IR remote control code from one of the user's separate remote controllers. For this reason, the present invention also includes a Piezo Output circuit 11 for providing audible feedback. The Piezo Output 11 consists of a simple single transistor circuit driving a piezo beeper from an output port of the Microcontroller 20. The output port of the Microcontroller 20 outputs a 4 KHz signal with an on time controlled by the software program in ROM 22. Finally, the majority of the electronic data processing and control is performed by 8-bit CMOS Microcontroller 20 which has several input and output ports interfacing to the various hardware sections described above. As explained earlier, included within Microcontroller 20 is a microprocessor 21 with an associated program read-only memory (ROM) 22 and an reference random access memory (RAM) 23 which perform voice recognition and other functions described herein. For this purpose, ROM 24 contains a 24 kbyte control program consisting of microcode instructions executed by Microprocessor 21 to effectuate the aforementioned functions. An additional 1.5 kbytes of reference memory is found in RAM 23 for temporary storage and is used by Microprocessor 21 for computing and for storage of information needed frequently. Microcontroller 20 also includes ADC 24 explained above for converting analog voice signals from Analog Voice Input 2 into digital voice signals. Microcontroller 20 also controls all input/output (I/O) in the present invention (such as Keyboard 3, IR input and output 5 and 12) and drives liquid crystal display (LCD) 10 as described below. In a preferred embodiment, Microcontroller 20 is a Panasonic integrated circuit part no. NN 1872410, but it is apparent that any number of acceptable Microcontrollers and or Microcomputers could be used instead.

Microcontroller 20 runs off of two standard crystals: a first 32.768 KHz crystal for the clock mode which is used to maintain the real time clock while consuming minimal power (150 microamps) from the battery supply and a second crystal running at 8.38 MHz for performing voice recognition and transmitting IR remote control codes. In the higher speed mode the power consumption from the battery increases to approximately 5 milliamps. The input ports of Microcontroller 20 thus include: a voice data port for receiving the analog voice signal from Analog Voice Input 2; a key strobe port for reading the Keyboard 3; an interrupt input (IRQ) for reading the IR remote control codes from IR Input 5; a memory data port for reading data from ROM Code Library 7 and External DRAM 8, and low battery ports for reading the status of the Battery Circuit 5 for low battery and change battery conditions. The output ports of Microcontroller 20 then include: an LCD driver port to drive a 200 segment LCD 10; an IR port for the IR remote control code transmission by IR Output 12; a piezo port for the Piezo Output 11; an AGC output to Analog Voice Input 2; and a key strobe port for driving keyboard decode lines on Keyboard 3.

Microcontroller 20 also has access to 1 MEG of External DRAM 8 for additional storage of information where larger memory capacity is desired. In a preferred embodiment. External DRAM 8 is a pseudo static 1 MEG DRAM which consumes lower power to operate then conventional DRAMs and at the same time has a smaller foot print and is far less expensive then static RAMS. To address the External DRAM 8, an 8-bit Shift Register 9 ^'is used between Microcontroller 20 and External DRAM 8 to minimize the number of I/O ports used to address External DRAM 8. In other words, using Shift Register 9 allows use of only two output port lines to address External DRAM 8 instead of the normal 8 address lines. A clock output port line from the Microcontroller 20 controls the data clock input of Shift Register 9, and a second output port serial address line provides the address data to be shifted into Shift Register 9. The 8 output lines of Shift Register 9 provide the RAS and CAS address lines to External DRAM. Because the address lines are shifted into Shift Register 9 in a serial manner, External DRAM 8 access time is significantly longer. Thus, Microcontroller 20 works out of its high speed internal memory RAM 23 any operations requiring high speed memory access such as for voice recognition or IR code capture or transmission.

In addition to External DRAM 8 the present invention accommodates an additional ROM Code Library 7 to support a code library of known manufacturer IR remote control codes. ROM Code Library 7 allows the user to select an IR code associated with their equipment without having to learn the remote control code through IR Input 5. To address ROM Code Library 7, a second 8-bit shift register is used in the same manner used to address External DRAM 8 allowing two I/O port lines to be used instead of eight. ROM Code Library 7 provides 8 kbytes of additional IR remote control code storage.

Figures 5A and 5B are depictions of the exterior of the casing which can physically incorporate the electronics of the present invention. As can be seen in Figure 5A, the present invention being approximately 7 inches by 2 inches by 1 inch is small, compact and portable. The exterior embodying the present invention includes: LCD 10 which provides visual feedback to the user; a plastic outer casing 30; a Keyboard 3 (sliding door 31 slides to expose part of the keys of Keyboard 3 that are not used frequently) ; and a microphone 32 into which the user speaks the voice commands. Keyboard 3 is divided into three logical groups of keys: voice-related keys, non-voice related upper keyboard keys, and lower keys (covered by sliding door 31) . The voice-related keys include the following: VOICE VCR key 33, VOICE RECORD key 34 and VOICE CHANNEL key 37. By "voice-related" it is meant that these keys are depressed by the user prior to speaking a voice command into microphone 32. The remaining non-voice related keys on the upper portion of Keyboard 3 include: VCR ON key 35, TV ON key 36, (VOICE and NON-VOICE) CHANNEL key 37, VOLUME and MUTE key 38, TRAIN key 39 and REVIEW key 40. The lower keyboard keys covered by sliding door 31 include: USER key 41; RECORD key 42; SETUP key 43; FEATURE key 44; DELETE key 45; SYSTEM key 46; TV/VCR key 47; CABLE key 48; PLAY key 49; REW (Rewind) Key 50; PAUSE Key 51; F. FWD (fast forward) key 52; and STOP key 53. When sliding door 31 is moved to cover the lower keyboard keys, a STOP key 54 is still available to the user. In the closed position seen in Figure 5B, STOP key 54 covers and is operatively engaged with STOP key 53.

Finally, Figures 6A and 6B show LCD 10 of the present invention. As can be seen in Figure 6A, LCD includes five vertically distinct status lines which visually display information to the user. LCD 10 is shown as it would appear after depressing REVIEW key 40 discussed below. In Figure 6A, a VCR programming event has been scheduled on April 16. The selected channel

(4) day (Tuesday), start time (8 p.m.), stop time (9:30 p.m.) and system (1) are also displayed. In Figure 6B, the status of LCD 10 is shown after the user has selected the idle mode after setting up the Voice Controller as discussed further below. The selected user's name is shown (in this case "Michael") as well as the day (Friday) and time (10:37 a.m.) .

B. OPERATION OF THE VOICE CONTROLLER The basic hardware of the present invention is shown in Figure 1. Microprocessor 21 and ROM 22 within Microcontroller 20 form a conventional digital processing circuit. Microprocessor 21 processes and executes instructions retrieved from the software program contained within ROM 22 in order to monitor and/or control the various hardware sections (IR Input 5, Keyboard 3 etc.). Accordingly, the remainder of the specification discusses the operation of the control program stored in ROM 22 of the present invention. Setup - Figure 2

Referring to the flow diagram of the Setup Modes in Figure 2, it can be seen that the control program is normally in a mode designated Idle 200. Before the user operates the present invention, he or she first goes through an initial setup procedure. As seen in Figure 2, the setup procedure consists of a series of setup modes: Clock Mode 201 where the clock is set at 202, to include the day, date and the real time clock; Name Mode 203 where the user's names are entered; a voice training mode 205 where voice templates are created for the recognition of the user voices; and various modes 206-214 for teaching the present invention the IR codes for the user's equipment (such as a TV, VCR and cable box) .

—Clock Mode 201

Referring back to the operation of the program as seen in Figure 2, the real time clock is set at 202 by first pressing SETUP key 43 (Fig.5A) until LCD (Fig.5A) displays the word "CLOCK". Referring to Figure 5A, to get into "CLOCK" mode 201 (Fig. 2) the user presses RIGHT arrow key (F.FWD key 52 in Fig. 5A) which changes the display to read "YEAR 92", for example, with "92" flashing. The user can set the year by pressing UP (PLAY key 49 in Fig. 5A) or DOWN arrow (PAUSE key 51 in Fig. 5A) keys which changes the displayed year. After the user sets the correct year on the display, the user then proceeds to enter the correct month, day and time in the same manner. If the user needs to correct a setting they made, the user presses the LEFT or RIGHT arrow keys 50, 52 to first select the field to be changed and then uses the UP or DOWN arrow keys 49, 51 to change the entry. When the user decides he has set everything properly, the user now presses SETUP key 43 which saves all of the users settings. —Name Mode 203—

Continuing in Figure 2 in Name Mode 203, the next item of information the user enters is the user's "NAME" for identification of the active voice templates. Referring to Figure 5A, pressing SETUP key 43 now advances the user from the "CLOCK" setup mode 201 (Fig. 2) displayed to the users "NAME" setup mode 203 (Fig. 2). With "NAME" displayed on LCD the user presses the RIGHT arrow key 52 which advances the user into the name set up field. The user sees "NAME 1" displayed at this time. To set the user's name (step 204 in Fig. 2) , the user presses the UP arrow key 49 and the display changes to "A " with the letter

"A" flashing. The display allows the user to enter their name using up to 8 characters. The user presses the UP or DOWN arrow keys 49, 51 which scrolls through the letters of the alphabet (from A to Z) until they have located and then entered the first letter of their name. To enter the second letter, the user presses the RIGHT arrow key 52 which advances the display to the second letter of the name field at which time the user again uses the UP or DOWN arrow keys 49, 51 to enter the second letter of their name. The user continues this process until their name has been entered. After the user has entered their name at 204 (Fig. 2) , the user presses SETUP key 43 which then saves all entries made.

—Voice Template Training 205—

Next in Figure 2 as part of the Setup Mode, after the user enters his or her name at 204, he or she then enters the necessary user voice templates at 205. The present invention is trained with the user's voice templates by way of a method described in more detail in U.S. application serial no. 07/915,938 which has also been filed on behalf of the present assignee concurrently herewith and which is hereby incorporated by reference as if fully set forth herein.

As a brief summary, however, referring to Figures 5A and 6A, to enter voice templates the user presses TRAIN key 39 at which time LCD automatically prompts the user to say the first of a series of a library of command words (voice command words) into the microphone. LCD 10 displays the prompted command word on the top line of the display as seen in Figure 6A. To create the voice template corresponding to the command word, the user presses VOICE RECORD key 34, which is one of three voice keys, and speaks the first prompted command word on LCD 10 into microphone 34. A digital voice signal is generated by ADC 24 as explained above. Microprocessor 21 executing a voice recognition routine within the control program in ROM 22 then generates a voice template from the digital voice signal corresponding to the command word and stores this template in External DRAM 8 (as well as internal reference RAM 23) . The control program then prompts the user to speak the next command word into the unit by flashing that word on LCD 10.

After the user has gone through the first pass of the entire set of command words, the control program then prompts the user to go through the vocabulary set a second time. During this second pass the voice recognition software in the control program in ROM 22 compares the templates from the second pass against the templates from the first pass to determine if there is a match between each of the templates. If the software determines that the scoring (closeness) of the first pass voice template and second pass template for any command word is not within certain acceptable limits (which the skilled artisan can manifestly vary depending on whether high recognitions rate or high ease of use are desired by the user) the control program of the present invention prompts the user on LCD 10 to repeat the command word. If after the user repeats the command word the scoring is still not within the acceptable range, the user is prompted to make another retry which results in a third voice template which is then compared against both the first and second voice templates. If there is still no match, the user is prompted one more time for a fourth template at which time he control program then selects and stores whichever two of the four voice templates score the best match with each other. When the user has gone through the second pass of each of command words the present invention stores all of the voice templates in External DRAM 8 and momentarily displays "SAVED" on LCD 10 before going back to the "Idle" mode display. This completes the entering of voice templates at 205 (Fig. 2) . The voice templates for a second user are retrieved from External DRAM 8 and entered into internal RAM 8 when selecting users with the USER key 41 described below. The training process continues at 205 until the entire set of command words have been prompted to the user, spoken into the unit by the user, and a voice template generated for each.

—IR Training; TV Mode 206, VCR Mode 208 and Cable 210 Looking further at the next mode shown in Figure 2, the user next trains the present invention to identify the infrared remote control code signals from the user's separate remote controllers for each of their TV, VCR and/or cable box components at 206, 208 and 210, respectively. In lieu of learning such codes, it is possible of course to use a code library as explained earlier, however. To learn the IR codes, the user presses SETUP key 43 (Fig. 5A) until "TV" is displayed in the top line of LCD 10. The present invention then learns the TV IR codes at 207. The user presses RIGHT arrow key 52 (Fig. 5A) at which time the word "POWER" is displayed on the top line of LCD 10. This word corresponds to the IR remote control code from the user's remote controller needed to turn the TV power on. To learn this IR remote control code, the user points the IR transmitting end of their TV remote controller into a window which is part of IR Input 5, and presses the prompted key shown in LCD 10 (which in this case is the "POWER" key) on the TV remote controller. When the control program detects the end of the IR code being transmitted by the TV remote control, the Microcontroller 20 generates an output to Piezo Output 11 to make an audible beep, prompting the user to release the key being pressed on the TV remote controller. The TV IR remote control codes are stored directly in External DRAM 8 and internal RAM 22. The control program then indicates on LCD 10 to the user the next function from the TV remote to be learned. This process continues at 207 until the user has trained the present invention with all the keys prompted on LCD 10 for the TV. Using this training method, the user never has to remember which remote control codes have been learned, since the control program automatically takes the user through the list of available TV remote control commands. If the user's remote does not have some of the keys prompted, the user goes past the prompted key by pressing UP arrow key 49a (Fig. 5A) . After the last key prompted has been trained the word "SAVED" is displayed on LCD 10 momentarily prior to going back to Idle mode.

The user then trains the present invention with the IR remote control codes from the user's VCR remote controller at 208 and 209. The user presses SETUP key 43 (Fig. 5A) until "VCR" is displayed. The user then presses RIGHT arrow key 52 (Fig. 5A) at which time the display prompts the user to press the "POWER" key on the VCR remote control. After pressing the "POWER" key, the user is prompted on LCD 10 to press the rest of the VCR remote control keys in the same manner as described in the TV IR section. This process continues until all prompted keys have been trained and the LCD displays "SAVED" momentarily and stores the VCR remote control IR codes as well in RAM 23. In addition to the TV and VCR IR codes, the user also trains the IR remote control codes from the user's cable box at 210 and 211 should they have one. To do so the user presses SETUP key 43 (Fig. 5A) until "CABLE" is displayed. The user then presses RIGHT arrow key 52 (Fig.5A) and LCD 10 changes to display the word "POWER" prompting the user to press the "POWER" key on their remote control for the cable box. Again this process continues until all prompted keys have been trained and the LCD displays "SAVED" momentarily and stores the IR remote control code in memory. The above process merely represents one embodiment of the invention wherein the user's system includes a TV, VCR and cable box. As is apparent, the present invention could easily learn the IR remote control codes for any other type of electronics entertainment system, such as a stereo system or satellite receiver system for example. The control program within ROM 22 of the Microcontroller 20 could then be appropriately modified to handle the training of this additional piece of equipment or some other similar component.

—IR Training: Channel and Volume System Configuration Data—

While training the present invention with IR remote control codes, the control program within ROM 22 generates a portion of the system configuration data. The remainder of the configuration data is generated during the Confirm Mode 213 described in more detail below. The system configuration data is stored in internal RAM 23. This system configuration data is based on the particular types of components that were learned during the IR training modes, and the functions relating to each of the components that were learned. This system configuration data is used by the control program of the present invention to self configure itself in order to later control the appropriate functions on each component by outputting the correct remote control code. For example, as indicated above, in a system that includes components which share a common function (such as a volume function) the user would prefer that the appropriate volume remote control code be output automatically rather than having to manually select the appropriate component before implementing the volume function. The remainder of the system configuration data is used by the control program as described further below to generate system functions which consist of a sequence of multiple transmitted commands. The first portion of the system configuration data consists of information relating to channel control. The operation of the channel control configuration routine of the control program is illustrated in the flow diagram of Figure 7. During IR training (that is, referring to Figure 2 again, after learning the IR code at 207 (TV), 209 (VCR) or 211 (Cable Box)), the control program decides from which piece of equipment to control the channel selection for viewing TV channels and from which piece of equipment to control the channel selection when recording programs.

Referring to Figure 7, if a cable box was learned at 701, the control program sets a flag in the TV channel control register at 702 indicating that TV channels are to be controlled by the cable box. In addition the control program sets a flag in a record channel control register at 703 indicating that the record channel is to be controlled by the cable box. At 704, if no cable box remote control was learned the control program next determines if a VCR remote control was learned. If a VCR remote was learned, the control program sets a flag in the record channel control register at 705 indicating that the record channel is to be controlled by the VCR tuner. The control program then next checks at 706 to see if TV channels were learned. If this is true, then the control program sets a flag in a TV channel control register at 707 indicating that the TV channels are to be controlled by the TV tuner. If no TV channels were learned, the control program sets a flag in the TV channel control register at 708 indicating that the TV channels are to be controlled by the VCR tuner.

At 709, if no VCR channel selection was learned and a TV channel selection was learned the control program sets a flag in the TV channel control register at 710 indicating the TV channels are to be controlled by the TV tuner. The next portion of system configuration information that is generated relates to volume control. The creation of system volume control configuration data is handled in a similar manner as illustrated in Figure 8. During the IR setup mode the control program first determines at 801 the TV volume control was learned. If the volume was learned from the TV, the control program sets a flag in a volume control register at 802 indicating that the volume is to be controlled by the TV. At 803, if the volume was not learned by the TV but was by the cable box, the control program sets a flag in the volume control register at 804 indicating the volume is to be controlled by the cable box. Based on this system configuration data, later it can be seen that the present invention, during remote control operation, automatically outputs the correct remote control code even in the case where there is overlap of functionality between components. When the user enters (using Keyboard 3 or via a voice command) a particular component command corresponding to one of the component functions (such as volume for example) , processor 21, automatically selects both the appropriate component and the remote control code corresponding to the component function based on the component command entered and the configuration data stored in reference RAM 23. This remote control code is then transmitted by IR output 12 to the component to perform the function corresponding to the component command.

The above examples are merely representative, and it will be apparent to skilled artisans that other overlapping component functions may be implemented in a similar manner. The last portion of system configuration data pertaining to playback and stop functions is explained further below. —Confirm Mode 212— As the next step in Setup Mode in Figure 2, the user sets up a number of configuration features for his or her system shown as "Confirm Mode" at 212. This option is used to confirm the status of the configuration of the user's system. The user first enters the channels that the VCR and cable box transmit on. To set the transmitting channel, the user presses SETUP key 43 (Fig. 5A) until he has selected "CONFIRM" on the top portion of LCD 10. Pressing RIGHT arrow key 52 (Fig. 5A) advances the display to "VCR" with a " 3 " flashing above the channel annunciation displayed on the lower left portion of LCD 10. The user now presses UP key 49 or DOWN key 51 (Fig. 5A) to instruct the present invention on whether the VCR is set to transmit on channel 3 or on channel 4. The user then presses

RIGHT arrow key 52 (Fig. 5A) to advance to setting the cable box transmission channel next. After pressing the "RIGHT" arrow, LCD 10 displays "CABLE" with "3" flashing above the channel annunciation displayed on the lower left portion of the display. If the users cable box transmits on channel 2 or channel 4 instead of channel 3, the user presses the UP or DOWN arrow keys 49 and 51 (Fig. 5A) to select the correct channel. After the channels have been properly set, the user presses RIGHT arrow key 52 (Fig. 5A) to set the TV/VCR automatic select function. LCD 10 displays "POWER" at this time prompting the user to power on both his TV and VCR. After powering the TV and the VCR on, the user then changes channels on the TV using the present invention. If the picture on the TV is clear when changing the channels, the tuner on the VCR was properly set to the TV tuner when powered on. With a clear picture on the TV the user presses SETUP key 43 (Fig. 5A) which saves the user's configuration entries. If the TV did not produce a clear picture when changing the TV channels, the VCR tuner was set to VCR when powered on. If this occurs, the user is instructed at 213 to press the TV/VCR key on keyboard 3 which informs the control program of the state of the VCR tuner when powered on. Thereafter, each time the present invention powers the VCR on during normal operation, the present invention also outputs the IR remote control code to the VCR for the TV/VCR function which places the VCR tuner into the proper state for normal use.

—Confirm Mode 212: Playback and Stop Control System Configuration Data—

As explained earlier, a portion of the system configuration data is generated by the control program during the IR training mode. The remainder of the system configuration data is generated during Confirm Mode 212.

This configuration data is generated because of the following problems found in prior art devices. As explained above, when controlling a component system, system functions may require multiple key selections to control that specific function. For example, to record a TV broadcast onto a videotape, the user must perform the following steps with a remote control: (1) select cable box; (2) change channel; (3) select VCR; (4) set the VCR tuner to the cable box broadcasting channel; and finally (5) record. Similarly, to view a tape on the VCR, the user may have to first select the TV; then channel 3 or 4 on the TV (or if the TV and VCR are configured for the video input, the user would have to select the video input instead of channel 3 or 4) before selecting the VCR PLAYBACK function on the VCR. When the user no longer wanted to view the VCR tape the user would first select the VCR STOP function, if the user had a cable box they may have to select the TV channel corresponding to the cable box broadcast channel or the TV input if using the video input option of the TV before they can watch the desired TV channel. Many VCRs when first powered on enable their internal VCR tuner requiring the user to select the TV/VCR function on the VCR remote control to eliminate the snow on their TV and for the TV tuner to change channels. The configuration data generated during Confirm

Mode 212 is used to eliminate these prior art problems. Specifically, the configuration data is used by the control program to generate sequences of remote control codes in order to simultaneously control more than one of the electronic components without having to resort to complicated manual user component selection.

The implementation of the system configuration data in connection with playback and stop channel selection features typically found in component systems is illustrated in Figure 9. As illustrated in Figure 9 the playback and stop channel selection on the TV is determined by first checking to see at 901 if the TV Video Input feature had been learned. If learned the control program sets a playback control register to Video Input from the TV at 902. In addition the control program at 903 sets a stop control register to return to the TV input for the TV.

At 904, if the video input was not learned, the control program checks to see what VCR channel was selected in Confirm Mode 212 as described above. If channel 3 was selected the control program sets a playback control register to TV channel 3 control at 905. However, if channel 4 was selected instead the control program sets the playback control register at 912 to TV channel 4.

Next, the control program checks to see if a cable box was learned at 906. If a cable box was learned the control program checks at 907 to see if channel 2 was selected. Should this be the case, the control program at 908 sets the stop control register to channel 2. If not, the control program then checks at 909 to see if channel 3 way set. Should this be the case, the control program sets the stop control register to channel 3 at 910. If channel 3 was not selected the control program sets the Stop control register to channel 4 at 911. When a cable box was not learned, the control program does not change channels during playback as shown at 913.

Finally, the control program also checks to see if the TV/VCR tuner control was selected in Confirm Mode 212. If selected, the control program sets a flag in a TV/VCR tuner control register indicating that the selection is needed.

During later remote control operation, the present invention automatically outputs the correct sequence of remote control codes required to effectuate a particular system function. For example, when the user enters (using Keyboard 3 or via a voice command) a particular system command (such as PLAY for example to play a videocassette on a VCR) , processor 21 automatically generates a sequence of remote control codes based on the PLAY command and the configuration data described above. This sequence would include remote control codes for setting the TV to the appropriate channel depending on the configuration data and playing a videocassette in the VCR IR Output 12 then transmits the sequence of remote control codes to the TV/cable box and VCR.

It can thus be seen that the present invention reduces significantly the time and effort required to effectuate system functions in a system of components. While prior art remote controllers as described above would typically require the manual entry of five (5) separate items of information to effectuate a record function the present invention at most requires two (2) , the rest being performed automatically by the control program. Furthermore, as described below, the present invention also permits a user to program a VCR to record both current and future TV shows. When entering this program timing data as part of the system command, the present invention generates the necessary sequence of remote control codes at the later time designated in the timing data. Moreover, it is apparent that when a start and stop time is designated by the user for recording, the present invention generates a portion of the sequence at the start time (for example, the

"record" remote control codes) and the remainder of the sequence at the stop time (for example, the "stop") command.

The above examples are merely representative, and it will be apparent to skilled artisans that other system functions may be implemented in the sequence fashion described. --Options Mode 214—

Lastly in Figure 2, the last setup mode is "Options Mode" 214. The user typically will not go into the options field for setups. The options field handles unusual equipment configurations and changes to default settings for unusual equipment. To get into the options mode the user presses SETUP key 43 (Fig. 5A) after leaving the Confirm Mode. Referring to Figure 5A, to get into the options field the user presses RIGHT arrow key 52 and LCD 10 displays "OPT 01'" with the option setting flashing. To change the option setting the user presses UP key 49 or DOWN key 51. To advance to the next option the user presses RIGHT arrow key 52 again and continues pressing RIGHT arrow key 52 until the desired option is displayed. After setting the option at 215 the user presses SETUP key 43 and the display goes back to the Idle Mode 200 display. Voice Mode Control - Figure 3

After the user has properly set up the present invention in the fashion described above, he or she may then remotely control any and all of the separate electronic components in the user's system. As explained earlier, the present invention is voice or keyboard operated. Figure 3 shows some of the functions that can be operated by voice commands; obviously, others could be implemented as required by any particular application. The list of functions that can be operated by manual (keyboard) entry are described below in connection with Figure 4. —voice recognition software operatio —

As explained above, the control program in ROM 22 of the present invention includes software for carrying out voice recognition. Voice recognition is well known in the art, and consists of various computationally intensive processing steps that are unavoidable. In order to implement real-time, high quality voice recognition on a low cost 8 bit microcontroller of the type used in the present invention, and in a small code size (24K) , the voice recognition software of the present invention uses an approach which reduces significantly two of the most computationally intensive processing steps required during voice recognition. This software is embodied in a computer program registered in its entirety with the U.S. Copyright Office as registration no. TXu488458 on September 13, 1991 and is hereby incorporated by reference as if fully set forth herein.

As mentioned, voice recognition algorithms are well-known in the art. A first requirement of voice recognition is that the voice signal must be converted from a time domain into a frequency domain in order to be able to work with the signal. This conversion is normally accomplished by the prior art either in analog or digital form. In analog schemes a bank of bandpass filters, each with a rectifier and low pass filter separates the voice signal into contiguous frequency bands. This process operates in real time but requires a significant number of discrete components external to the microcontroller. If the process is carried out digitally, a mathematical technique such as a Fourier transform may be used. Even though a variety of high speed implementations of this transform have been developed, the non-stationary nature of the voice signal normally requires that many such transforms be computed and then averaged to form a single frequency spectrum that is representative of a given time segment of the signal. This transform process normally requires a high speed processor with specific instructions designed for digital signal processing.

The approach used in the present invention uses a 32 point Fourier transform, resulting in the equivalent of 16 frequency bands. The voice commands are received, filtered and conditioned by Analog Voice Input 2. The conditioned voice signal is then converted by ADC 24 into a digital voice signal. A set of samples from this digital voice signal in a given time segment (20 ms) is used to yield a representative spectrum of that segment. The samples are selected based on a detection of the peak energy within any particular segment. This approach allows the number of transforms necessary to accurately describe the voice signal to be dramatically reduced which in turn allows a much simpler and smaller processor to be used.

The present invention also reduces the computation time normally required for voice recognition in a second way now described. First, each spoken word by the user is converted to a fixed length sequence of normalized frequency spectra (a voice template) by the transform process described. In order to recognize a spoken word or phrase, this voice template is compared to the reference voice templates previously trained and stored in RAM 23 which represent the vocabulary of word commands used in the system. In order to allow for the likely misalignment of these voice templates in the time domain, it is desirable to use some form of non-linear time warping to find the alignment that produces the best match. Several forms of this time alignment compensation are normally used in prior art voice recognition systems but these require very fast processors in order to operate in real time.

The present invention utilizes a two pass voice template matching scheme. First, a low resolution, high speed matching process is used to select the four (4) most likely candidates. This is done using a simple linear pattern match with consecutive time segments of the voice templates compared by a distance metric. Second, a much more extensive time warping comparison of the type described above is then used only to decide among the most likely candidates found by the first pass. This allows a high quality recognition approach to be utilized on a relatively slow, small and simple 8 bit processor. — rocessing of voice commands— Referring generally to Figure 3, to operate the present invention by use of voice commands the user selects one of three voice keys (shown as 33, 34 or 37 in Fig. 5A) depending on the desired function. Using the voice keys the user can voice control: (1) all of the VCR playback functions; (2) channel numbers; and (3) programming of recording events.

Beginning at the top of the flow diagram in Figure 3, the control program of the present invention is normally in idle mode 200. At 301, VOICE RECORD key 34 is used to program both current and future TV shows (as well as to record the voice templates in the setup mode described above) . If the user wants to record a current show on his VCR and is not concerned with setting a stop time the user presses VOICE RECORD key 34 at which time LCD 10 displays "RECORD" and then prompts the user to enter the channel number by flashing the channel number digit field. Channel numbers are entered as single digit entries. If the user desired to select channel 13, for example, the user presses VOICE RECORD key 34 (Fig. 5A) and says "1", pauses until 1 is displayed on LCD 10 and then says "3." The reason for handling channel numbers as double digit numbers instead of single numbers such as thirteen in this case, is to minimize the amount of time it takes the user to train the present invention with voice commands. For example, if the system handles up to 99 channels the user would have had to say 1 through 99 twice when voice training which is not practical.

After the user enters the channel number to be recorded, the program automatically prompts the user to enter in the day of the week by flashing the day annunciation and the day field above the annunciation on LCD 10. The user enters the day of the week by voice and the present invention next prompts the user to enter the start time hour to record. The user then has two choices at step 302. As a first choice, if the user wants to record now (immediately) the user says "START" at which time the present invention starts the recording process by outputting the necessary IR remote control codes to both the VCR and cable box at step 304.

Prior to this, however, a step entitled "device select" 303 is executed. In this step, the control program performs a routine to make certain that the proper devices (components) are selected and configured properly for the requested recording operation. This device select routine is based on the system data configuration data described earlier.

Briefly, if the user has a cable box that is used to select the channels on the VCR for recording, the present invention first outputs the channel number to the cable box, then, outputs channel 2, 3 or 4 to the VCR (whichever is necessary to monitor the cable box) and then outputs the record signal to the VCR. If the user does not have a cable box, the present invention outputs the channel numbers to the VCR instead followed by the record command. Looking further at Figure 3 at step 302, the second choice available to the user is to program the present invention to set a programmed (future) event at step 305. To record a show in the future, the user follows the same steps as above to set the channel numbers and the day of the week the program is to be recorded on. After entering the day of the week by voice, the program prompts the user to enter the start time hour. The user has the option of entering the hours 1 through 12. Unlike the channel numbers, the start time hours 10, 11, 12 are treated as a single number and not as two distinct digits. This is because the user only had to voice train 3 additional numbers unlike what would have been required for the 99 channel number. After entering the start time hour the user has a choice of entering "AM", "PM" or "30". The start and stop time has 30 minute resolution by voice to minimize the time for voice training. However, for applications requiring greater resolution this can be accomplished simply by increasing the amount of voice training or as implemented, manually setting the time as described below. If the user desires to start the recording on the hour the user says either "AM" or "PM". If the user desired to start recording on the half hour the user first says "30" and then either "AM" or "PM". After entering "AM" or "PM" for the start time the user is then prompted to enter the hour of the stop time, which is entered in a similar manner as that discussed at the start time.

Next, the user has the capability at step 306 to characterize the future programmed event in three possible ways. First, if the user wants the event to be recorded as a one time event the user releases VOICE RECORD key 34 (Fig. 5A) and the control program saves the recording event into External DRAM 8 after 8 seconds. The user also has the options of recording the programmed event on a daily (Monday through Friday every week) or weekly (one day a week every week) basis. To select a recording for either daily or weekly, the user presses and holds RIGHT arrow key 52 (Fig. 5A) while the recording event is still displayed on LCD 10 until both the "DAILY" and "WEEKLY" annunciation appear in the upper right corner of LCD 10. To select the daily option, the user presses UP arrow key 49 (Fig. 5A) once at which time only the "DAILY" annunciation is displayed. If the user desired to select weekly instead of daily, the user presses UP arrow key 49 (Fig. 5A) a second time and the "WEEKLY" annunciation is displayed instead. The present invention automatically saves the user's entries 8 seconds after release of VOICE RECORD key 34. The user can program the present invention for a number of programmed recording events; the limit is simply a design choice determined in part by the amount of available memory in External DRAM 8.

It should be emphasized that it is the present invention, not the user's VCR or any other device, that stores the programmed events. When it is time to record, the present invention outputs the necessary IR remote control codes to the VCR and cable box in the manner described above. Thus, in this manner, the user still has the additional independent programming flexibility of his VCR should he or she then choose to use it. The present invention operates completely independent of any other equipment controller.

As indicated earlier, the pattern covered by the IR transmitting LEDs is 130 degrees along the vertical axis with full coverage throughout that range. For this reason, the present invention is fairly omni-directional and can be left on a coffee table, for example, and only needs to be pointed in the general direction and still be able to output the necessary IR remote control codes to the user's VCR and have these signals received properly by such VCR. Looking again at Figure 3, the next box 307 in the flow diagram shows that in addition to recording events by voice commands, the user can control all of the other VCR playback functions by voice. To do so the user presses VOICE VCR key 33 (Fig. 5A) at which time LCD 10 displays "VCR". The user at 308 can now say any of the VCR voice commands which include "PLAY", "STOP", "FAST FORWARD", "REWIND", "PAUSE", and "ZAP-IT". These voice commands are received and processed at 309 in the same manner as the "START" command described above. The user can either say a single VCR command and release the voice key or the user can hold VOICE VCR key 33 down and continuously speak commands without releasing the voice key between words. As an example, if the user wanted to put his VCR in the search mode the user presses VOICE VCR key 33 (Fig. 5A) and says

"PLAY" which sends the play command to the VCR to start playing the tape. The user without releasing the VCR voice key can then say "FAST FORWARD" or "REWIND" to search either in the forward or the reverse direction. The user can then say either "PLAY" to abort the search mode and continue watching the tape or "STOP" to stop the tape, again without releasing the voice key. In this manner, the present invention interprets an uninterrupted string of continuous voice commands with pauses between the commands and translates them into corresponding remote control codes.

The "PLAY" key generates a system command that can consist of a sequence of remote control codes for multiple components depending on the user's component configuration. When the PLAY command is entered (via voice or keyboard) , the present invention first outputs a remote control code to the TV to select either Video input, channel 3 or channel 4, again, depending on the system configuration data. After outputting the TV control the present invention then outputs the play command to the VCR. The Stop command also consists of multiple controls again depending on the user's configuration. When the Stop command is given the present invention outputs a remote control code to the TV (corresponding to either TV Input or channel 2, 3 or 4 depending on what the cable channel transmission was set for) . Following the TV remote control code, the present invention then outputs remote control codes to the VCR corresponding to the Stop and TV/VCR commands.

A "ZAP-IT" voice command causes the VCR, during playback, to fast forward past any commercials that may have been recorded. Looking at the third and last operation of step 310 in Figure 3, the user can also set the channel number by voice command. The user presses the center of the (VOICE) CHANNEL key 37 (Fig. 5A) and LCD 10 displays "CHANNEL" with two channel spaces flashing. The user then speaks the channel number into microphone 32 (Fig. 5A) , the voice recognition software within the control program processes and decodes the spoken word at 311, performs a device select at 312, and then outputs the appropriate IR remote control code at 313. As described earlier in the voice record sequence, the user enters the channel numbers as two separate digits. Channel 13 would be spoken as "1" followed by "3" instead of thirteen. When the program detects the entry of a second digit it immediately outputs the IR remote control code associated with the channel number entered. If a single digit channel number is entered by voice, the control program waits for 3 seconds after the first digit to see if a second digit is entered and if a second digit is not entered the present invention outputs the single digit channel number the user entered. Again, referring to step 312 in Figure 3, after performing voice recognition of the voice command at 311, the control program also performs a further device select of the type described briefly above. This device select routine is used in the following scenarios. For example, in a system that includes a TV, VCR and cable box, all three components could include a "channel" IR remote control code (assuming all three have remote controls) . The control program, when told by the user to change the channel (say to channel "13") automatically decides which IR code should be output (i.e., whether the cable box, TV or VCR "channel" IR remote control code should be output) . The process by which the control program makes this decision is as follows: (1) in an system that includes all three components (TV/VCR/cable box) , the IR remote control code for the cable box is output; (2) in a system without a cable box but with a TV and VCR (which have remote controllers) the present invention outputs the IR remote control code for the TV; and (3) in a system which has neither a cable box nor a TV remote control, the present invention outputs the VCR IR channel remote control codes. In this manner, the present invention receives a component command and intelligently coordinates and selects the appropriate component in the user's system that should perform the requested channel change function.

Other overlapping functions are handled similarly, such as, for example, when the user depresses VOLUME key 38 (Fig. 5A) .

Manual Mode Control - Figure 4 —Real Time Key Control—

In addition to being controllable by way of voice commands, the present invention can also be operated manually as shown in Figure 4 by way of Keyboard 3 (keys shown in Fig. 5A) . As shown in Figure 4 the program is normally in idle mode 200 or 400. When a key is depressed at 401 it is decoded at 402, and a corresponding IR remote control code is output at 403. Some of the keys that can be controlled at step 401 are now described. Referring again to Figure 5A, there are two sets of keys available for the user to operate the present invention. The more frequently used keys are located on the upper surface of Keyboard 3. These keys include TV ON key 36 for turning the power to the TV on or off and VCR ON key 35 for turning the power to the VCR on or off. When powering the VCR on the present invention first checks the TV/VCR status register to determine if the TV/VCR status flag was set in the Confirm mode. If set the present invention outputs the TV/VCR IR command following the output of the VCR power command. This feature eliminates the snow viewed on the TV when first powering on many VCRs. CHANNEL key 37 is an up and down rocker key for changing the viewing channels up or down. This key also is used for voice control and controls either the TV, the cable box or the VCR channel selection depending on the user's equipment setup as described previously. VOLUME key 38, like the channel key, is a rocker key with an up and down position for controlling the listening volume. As with the aforementioned channel key, the volume key also includes device select logic to handle component commands which include overlapping component functions. In other words, if the user's TV has a remote controller with volume control, the present invention sends the IR volume remote control code to the TV. If the user's TV does not have a remote control with volume control but the cable box has the remote controlled volume control, the present invention sends the IR volume control code to the cable box for control of the volume instead. Continuing in Figure 5A with the keys on Keyboard 3, TRAIN key 39 is used initially by the user in the Setup Mode described above to train the present invention with the user's voice. After the user enters his or her name, the user pressed TRAIN key 39 which starts the prompting process for entering the voice commands. After the user's voice templates have been created during the initial setup (Fig. 2) , the user can press TRAIN key 39 to retrain a voice command that may not be working optimally for them. Next, REVIEW key 40 is used for reviewing any stored programmed events. A single depression of REVIEW key 40 allows the user to see on LCD how many programmed events have been stored. If no events have been programmed into the present invention by the user, LCD displays "NONE". Each additional depression of the "REVIEW" key displays the programmed events in chronological order of the time they are to be recorded.

A secondary set of keys are located under sliding door 31 at the lower end of the casing 30 (Fig. 5A) . These keys are typically not used in everyday usage. USER key 41 allows the user to select which set of voice templates are to be active, since more than one user can use the present inventions. If the name displayed on LCD is not the correct user's name, the user can select their name and corresponding voice templates by pressing USER key 41 until their name is displayed.

SETUP key 43 is special and has been described above.

DELETE key 45 is used for several functions. For example, if the user wants to change one of the user's names, the name can be deleted by selecting the name in Setup Mode (Fig. 2) and pressing DELETE key 45 for two seconds at which time a beep is heard and the word "DELETED" is displayed momentarily followed by "NAME #" at which time the user can enter a new name if so desired. If the user has replaced their TV or other piece of equipment which has a different remote controller, the user trains the present invention (Fig. 2) with the new IR remote control code for that component. To do so, the user goes into Setup Mode and selects the learn mode associated with the piece of equipment they are replacing. After selecting the component, the user presses DELETE key 45 and holds the key down for 2 seconds at which time Piezo Beeper 11 beeps and LCD 10 displays "DELETED" clearing the memory field for that device. The third function that DELETE key 45 can delete is a programmed event. If the user wants to cancel a programmed event that they previously entered, the user selects the programmed event with REVIEW key 40 and while the event is displayed presses DELETE key 45 for 2 seconds which deletes the programmed event. To stop a programmed event that is recording on the VCR, pressing DELETE key 45 both deletes the event as well as stops the VCR recording process and powers off the VCR.

TV/VCR key 47 functions in the exact manner that the TV/VCR key functions on a typical VCR remote controller. Pressing TV/VCR key 47 sends the IR remote control code signal to the VCR to change the input selection tuner on the VCR from TV to VCR or vice-versa. This key is also used in the "Confirm" section of Setup Mode (Fig. 2) to instruct the program on the state of the VCR tuner when the VCR is powered up.

CABLE key 48 is a power on and off key for the cable box. Pressing this key toggles the power on the cable box. The present invention assumes that the cable box is powered on all the time so this key is rarely needed.

SYSTEM key 46 switches the present invention between two different systems of electronic components generally designated as System 1 and System 2. When the user initially trains the present invention the IR remote control codes from the separate remote controllers, all devices (e.g., TV, VCR, cable box) are trained on System 1. If the user has another system in another room they would also like to operate from the present invention, they train this new set of IR remote control codes on System 2. To train the present invention to handle the second set of equipment, the user selects System 2 with SYSTEM key 46 prior to training the IR remote control codes from the second set of equipment. To operate the second set of equipment, the user selects System 2 using SYSTEM key 46 and then proceeds to use the present invention in the normal manner. When the user programs an event, the user first selects either System 1 or 2 prior to setting the event. When the present invention outputs the IR remote control codes to control the VCR and cable box for recording, the system number that was set when the event was entered is used

FEATURE key 44 is an uncommitted key that the user can use to control a feature not already programmed. The feature can be any function used on any of the user's separate remote controllers. For example, the user can program in a "Sleep" feature from their TV remote, an additional fast forward or search feature from the VCR remote, or last channel recall from the cable box remote control. The user can have a separate feature for each set of equipment he is controlling. To train the feature into the present invention, the user first selects the proper system number, presses SETUP key 43 and while pressing SETUP key 43 presses FEATURE key 44 at which time LCD 10 displays "FEATURE" . The user then trains the user selected IR remote control code following the same instructions described earlier in the train section. To use the feature, the user simply presses FEATURE key 44 and the correct IR remote control code is transmitted to the device being controlled.

UP (PLAY) arrow key 49, DOWN (PAUSE) arrow key 51, LEFT (REW) arrow key 43 and RIGHT (F.FWD) arrow key 52 serve multiple functions depending on the mode of the present invention. For example, during Setup Mode (Fig.2) and the manual record sequence LEFT arrow key 43 and RIGHT arrow key 52 are used to change the various fields to be set. RIGHT arrow key 52 allows the user to advance to the next field to be set while LEFT arrow key 43 allows the user to back up to a prior field to correct any errors that may have been made. In this same Setup Mode UP arrow key 49 (Fig. 2) and DOWN arrow key 51 are used to make the settings within the selected field. Specifically, UP arrow key 49 allows the user to increment the setting in the field while DOWN arrow key 51 allows the user to decrement the setting in the field. When setting a programmed event for recording, the arrow keys are used in manner similar to what was just described, that is, selecting and setting the desired fields.

Conversely, during Manual (Fig4) or Voice Control (Fig3) modes of operation of the present invention, RIGHT arrow key 52 operates to fast forward (F. FWD) the videocassette tape, LEFT arrow keys 43 operates to rewind (REW) the videocassette tape, UP arrow key 49 is used to initiate the play (PLAY) function on the VCR, and DOWN arrow key 51 pauses (PAUSE) the playing of the videocassette tape.

STOP key 53 also serves more then one function, depending on what mode the present invention is operating in. When in Setup Mode STOP key 53 is used as an escape key. If the user is in the middle of setting up a particular function and wishes to abort what is being done, STOP key 53 takes the program back to the prior mode. When setting a programmed event, either by voice or manually, if the user wishes to abort the process for whatever reason the user can again press STOP key 53 which takes the program back to the idle mode. As with PLAY key 49, when the present invention is being used to control the operation of the VCR, STOP key 53 when pressed first sends the appropriate IR remote control code to the VCR to stop the tape followed by either channel 3 or 4 IR remote control code to the TV if the VCR is connected to the RF section of the TV, or the RF input selector of the TV if the VCR is connected to the Video input of the TV. —Record Key—

The next operation which can be performed manually is the RECORD KEY CONTROL shown in Figure 4 as block 404.

Referring to Figure 5A again, RECORD key 42 under sliding door 31 provides the user with an additional manual way of setting both current and future recording events. Unlike VOICE RECORD key 34, RECORD key 42 does not allow voice entry to program events. This manual recording feature allows the user the added ability, however, of setting the start time of a programmed event to the minute (instead of on the half hour of hour by and the stop time to the nearest 15 minutes (again instead of the half hour or hour by voice) .

At 405 (Fig. 4) if the user wants to record a show presently being broadcast, the user presses RECORD key 42 at which time LCD displays "RECORD" and prompts the user to enter the first digit of the channel number. The user presses UP or DOWN arrow keys 49 and 51 respectively until the first digit of the channel they are setting is displayed. After setting the first digit of the channel the user presses RIGHT arrow key 52 which advances the display to the second digit field which is flashing. After selecting the second digit field the user again uses UP and DOWN arrow keys 49 and 51 to select the second digit of the channel to be recorded. When the user has entered the channel number the user presses RECORD key 42 again, at which time the present invention at 406 (Fig. 4) sends the necessary IR remote control codes to the VCR and cable box if applicable.

At 407 (Fig. 4) to set future programmed events, the user first sets the channel number as described above and then sets the day of the week. The day of the week is set by pressing RIGHT arrow key 52 after selecting the last digit of the channel number which advances the display to the day field of the display. The user sets the day of the week by pressing UP or DOWN arrow keys 49 and 51 until the correct day is displayed. After setting the day, the user presses RIGHT arrow key 52 to select the start time field on the display. With the field selected, the user presses UP or DOWN arrow keys 49 and 51 to set the hour. To set either "AM" or "PM" of the recording time, the user continues to press UP or DOWN arrow keys 49 and 51 past the 12th hour which advances the display from AM to PM or PM to AM. Pressing RIGHT arrow key 52 at this time advances the display to prompt the user to fill in the minutes of the start time field of the display. Again the user presses UP or DOWN arrow key 49 and 51 to set the minutes. The start time minutes field has a resolution of 1 minute allowing the user to start the recording exactly when desired. Pressing RIGHT arrow key 52 advances the display to prompt the hours field of the stop time. To set the hours the user presses UP or DOWN arrow keys 49 and 51 until the hour desired is set. After setting the hours field of the stop time the user presses RIGHT arrow key 52 advancing the display to the minutes field of the stop time. Unlike the start time, the stop time minutes field has a resolution of 15 minutes and not l minute. The minutes field is set by pressing UP or DOWN arrow keys 49 and 51. The user now has three choices for setting the type of event they have just programmed. If the user wants to record a single one time event with the settings they have just entered, the user presses RECORD key 42 which saves all entries. If the user wants to set the event as a daily or weekly occurrence, the user presses and holds RIGHT arrow key 52 for 2 seconds until LCD 10 displays the two annunciators "DAILY" and "WEEKLY" with both annunciators flashing. After the "DAILY" and "WEEKLY" annunciators are displayed on LCD 10, the user selects the specific one desired by pressing UP or DOWN arrow keys 49 and 51. The record sequence is now completed and the user can save the settings by now pressing RECORD key 42 again. —Setup Key—

The last item in Figure 4 is SETUP KEY CONTROL 408. Referring to Figure 5A, SETUP key 43 is used to initially set up the present invention at 409 by the user as discussed in previous sections. SETUP key 43 serves two functions. SETUP key 43 is used to select the various modes as well as to save users entries.

C. SUMMARY The present invention is a voice operated remote control for home entertainment electronic components, such as VCRs, televisions, cable boxes or stereos but is equally applicable to other environments requiring remote control. The invention provides portable, voice-activated control of electronic components. While one embodiment of the invention has been presented above, the scope of the invention is to be determined by the following claims.

Claims

What is claims is:

1. A portable remote control apparatus operated by a voice command for controlling one or more electronic components each which in turn is responsive to a set of respective remote control codes, said apparatus comprising: an analog input circuit for receiving the voice command and generating an analog voice signal based on the voice command; and an analog to digital converter coupled to the input circuit for converting the analog voice signal into a first digital voice signal; and a reference memory for storing a plurality of reference digital voice templates and the remote control codes; and a program memory for storing a control program; and a processor coupled to the converter, reference memory and program memory for generating a first digital voice template from the first digital voice signal, and for executing the control program to determine whether the first digital voice template is substantially equivalent to one of the reference digital voice templates, and for selecting one of the remote control codes based on the first digital voice template; and a remote control code transmitter coupled to the processor for transmitting the selected remote control code.

2. The apparatus of claim 1, further including a input capture circuit coupled to the processor for capturing the remote control codes.

3. The apparatus of claim 1, further including a keyboard with a plurality of control keys coupled to the processor, and wherein the processor selects one of the remote control codes based on which of the control keys is depressed.

4. The apparatus of claim 1, further including a display coupled to the processor for displaying the voice command.

5. The apparatus of claim 1, wherein the electronic components include a television, a videocassette recorder which can be programmed to perform a record operation to record information on a videocassette, and a cable box converter, and which components share one or more functional operations in common.

6. The apparatus of claim 5, wherein in response to a user's voice command to perform the record operation the processor selects and outputs one or more remote control codes to each of the electronic components to effectuate the record operation

7. The apparatus of claim 5, wherein in response to a user's voice command to perform one of the functional operations which the components share in common, the processor selects and outputs one or more remote control codes to only one of the electronic components to effectuate the requested functional operation.

8. A portable remote control apparatus comprising: a keyboard for entering a manual control command; and an analog input circuit for receiving a voice control command and generating an analog voice signal based on the voice control command; and an analog to digital converter coupled to the analog input circuit for converting the analog voice signal into a first digital voice signal; and a remote control code capture circuit for capturing remote control codes; and a RAM for storing a plurality of reference digital voice templates and the captured remote control codes; and a ROM for storing a microcode control a program; and a processor coupled to the keyboard, converter, capture circuit, RAM and ROM for generating a first voice template from the first digital voice signal and for executing the control program in order to select one of the captured remote control codes when a manual control command is entered, or when the first digital voice template is substantially equivalent to one of the reference digital templates; and a remote control code transmitter coupled to the processor for transmitting the selected captured remote control code; and a display for visually displaying the selected manual control or voice control command

9. The apparatus of claim 8, wherein the electronic components include a television, a videocassette recorder which can be programmed to perform a record operation to record information on a videocassette, and a cable box converter, and which components share one or more functional operations in common.

10. The apparatus of claim 9, wherein in response to a user's voice command to perform the record operation the processor selects and outputs one or more remote control codes to each of the electronic components to effectuate the record operation.

11. The apparatus of claim 9, wherein in response to a user's voice command to perform one of the functional operations which the components share in common, the processor selects and outputs one or more remote control codes to only one of the electronic components to effectuate the requested functional operation.

12. A method for controlling by voice one or more electronic components each which in turn is responsive to a set of respective remote control codes, said method comprising the steps of: (a) generating a reference analog voice signal based on a reference voice control command;

(b) converting the reference analog voice signal into a reference digital voice template and storing the same in a reference memory;

(c) repeating steps (a) and (b) until a set of reference digital voice templates is obtained;

(d) capturing and storing the remote control codes from one or more remote controllers;

(e) generating an analog voice signal based on a voice control command;

(f) converting the analog voice signal into a digital voice template;

(g) selecting one of the captured remote control codes when the digital voice template is substantially equivalent to one of the reference digital templates; (h) transmitting the captured remote control code which has been selected.

13. The method of claim 12, further including the step of visually displaying the selected manual control or voice control command.

14. The method of claim 12, wherein during the step of capturing the remote control codes a visual prompt to depress keys on the remote controllers appears on a display.

15. The method of claim 12, wherein the electronic components include a television, a videocassette recorder which can be programmed to perform a record operation to record information on a videocassette, and a cable box converter, and which components share one or more functional operations in common.

16. The method of claim 15, wherein in response to a user's voice command to perform the record operation one or more remote control codes is selected and output to each of the electronic components to effectuate the record operation.

17. The method of claim 15, wherein in response to a user's voice command to perform one of the functional operations which the components share in common, one or more remote control codes is selected and output to only one of the electronic components to effectuate the requested functional operation.

18. The method of claim 12, wherein a plurality of continuous voice commands are decoded continuously.