US20110082688A1 - Apparatus and Method for Analyzing Intention - Google Patents
Apparatus and Method for Analyzing Intention Download PDFInfo
- Publication number
- US20110082688A1 US20110082688A1 US12/894,846 US89484610A US2011082688A1 US 20110082688 A1 US20110082688 A1 US 20110082688A1 US 89484610 A US89484610 A US 89484610A US 2011082688 A1 US2011082688 A1 US 2011082688A1
- Authority
- US
- United States
- Prior art keywords
- intention
- sentence
- semantic role
- determined
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
- G06F40/35—Discourse or dialogue representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
Definitions
- the following description relates to a technology for analyzing the intention of a user, and more particularly, to an apparatus and method for analyzing the intention of a sentence generated by a user.
- Voice interaction technology is becoming essential for interaction between humans and computer systems.
- Modern voice recognition technology provides high performance for previously defined speeches.
- a grammar-based language model such as context free grammar language model or a statistical language model such as an N-gram language model is used.
- the grammar-based language model advantageously accepts only a grammatically and semantically correct sentence as a recognition result, but cannot recognize a sentence which has not been pre-defined in terms of grammars.
- the statistical language models may recognize some sentences that have not been pre-defined and do not require a user to manually define grammar.
- an apparatus for analyzing intention comprising: a phrase spotter configured to perform phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases; a valid sentence determiner configured to: determine whether the at least one sentence is grammatically valid by applying a dependency grammar to the sentence that has undergone phrase spotting; and filter an invalid sentence; and an intention deducer configured to generate an intention analysis result of a sentence determined to be valid.
- the apparatus may further include that the intention deducer is further configured to: select an intention frame to be the intention analysis result of the sentence determined to be valid; determine a semantic role value of at least one semantic role element included in the selected intention frame; and allocate the determined semantic role value to the semantic role element included in the selected intention frame.
- the apparatus may further include that, in response to the intention deducer allocating the semantic role value, the intention deducer is further configured to: determine the semantic role value from the sentence determined to be valid through phrase chunking; and allocate the determined semantic role value to the semantic role element in the selected intention frame if at least one semantic role element of the sentence determined to be valid matches at least one semantic role element in the selected intention frame.
- the apparatus may further include that, in response to the sentence determined to be valid comprising a semantic role element other than the at least one semantic role element in the intention frame, the intention deducer is further configured to: determine whether the other semantic intention role element can be replaced by the semantic role element in the intention frame using a role network; determine a semantic role value of the semantic role element in the intention frame from the sentence determined to be valid through phrase chunking in response to it being determined that the other semantic intention role element can be replaced by the semantic role element in the intention frame; and allocate the determined semantic role value to the semantic role element in the intention frame.
- the apparatus may further include that the intention deducer is further configured to estimate the semantic role value of the at least one semantic role element in the intention frame using an ontology.
- the apparatus may further include a scorer configured to: calculate a probability that intention analysis has been correctly performed on at least one intention analysis result candidate to which the semantic role value of the semantic role element included in the selected intention frame is allocated; and score the intention analysis result candidate.
- the apparatus may further include an analysis applier configured to: apply the intention analysis result to an application; and generate an intention analysis application result.
- the apparatus may further include a speech recognizer configured to convert an audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted by the speech recognizer.
- a speech recognizer configured to convert an audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted by the speech recognizer.
- a method of analyzing an intention comprising: performing phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases; determining whether the at least one sentence is grammatically valid by: applying a dependency grammar to the sentence that has undergone phrase spotting; and filtering an invalid sentence; and generating an intention analysis result of a sentence determined to be valid.
- the method may further include that the generating of the intention analysis result of the sentence determined to be valid comprises: selecting an intention frame to be the intention analysis result of the sentence determined to be valid; determining semantic role values of semantic role elements included in the selected intention frame; and allocating the determined semantic role values to the semantic role elements included in the selected intention frame.
- the method may further include that the allocating of the semantic role values comprises: determining whether at least one semantic role element of the sentence determined to be valid matches at least one semantic role element in the selected intention frame; and in response to it being determined that the at least one semantic role element of the sentence determined to be valid matches the at least one semantic role element in the selected intention frame: determining the semantic role values from the sentence determined to be valid through phrase chunking; and allocating the determined semantic role values.
- the method may further include that, in response to the semantic role element of the sentence determined to be valid not matching the semantic role element in the selected intention frame, the allocating of the semantic role values further comprises: determining whether the sentence determined to be valid comprises a semantic role element other than the semantic role elements of the intention frame; in response to the sentence determined to be valid comprising a semantic role element other than the semantic role elements of the intention frame, determining whether the other semantic role element can be replaced by the semantic role element in the intention frame using a role network; and in response to it being determined that the other semantic role element can be replaced by the semantic role element in the intention frame: determining the semantic role value of the semantic role element in the intention frame from the sentence determined to be valid through phrase chunking; and allocating the determined semantic role value to the semantic role element in the intention frame.
- the method may further include estimating the semantic role value of the at least one semantic role element in the intention frame using an ontology.
- the method may further include: calculating probabilities that intention analysis has been correctly performed on at least one intention analysis result candidate to which the semantic role value of the semantic role element in the selected intention frame is allocated; and scoring the intention analysis result candidates.
- the method may further include applying the intention analysis result to an application and generating an intention analysis application result.
- the method may further include performing speech recognition on an audio input and converting the audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted through the speech recognition.
- a computer-readable storage medium storing a program that causes a computer to execute a method of analyzing an intention, comprising: performing phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases; determining whether the at least one sentence is grammatically valid by: applying a dependency grammar to the sentence that has undergone phrase spotting; and filtering an invalid sentence; and generating an intention analysis result of a sentence determined to be valid.
- FIG. 1 is a diagram illustrating an example of an apparatus for analyzing an intention.
- FIG. 2 is a diagram illustrating an example of an intention analyzer.
- FIG. 3 is a diagram illustrating an example of an intention deducer.
- FIG. 4 is a flowchart illustrating an example of a method of a semantic role value allocator.
- FIG. 5 is a diagram illustrating an example of context-free grammar.
- FIG. 6 is a diagram illustrating an example of phrase spotting.
- FIG. 7 is a diagram illustrating an example of a phrase spotting operation.
- FIG. 8 is a diagram illustrating an example of dependency grammar.
- FIG. 9 is a diagram illustrating an example of a role network.
- FIG. 10 is a diagram illustrating an example of the allocation of a semantic role value in response to semantic role elements matching.
- FIG. 11 is a diagram illustrating an example of the allocation of a semantic role value in response to semantic role elements not matching.
- FIG. 12 is a diagram illustrating an example of the estimation of a semantic role value through phrase chunking.
- FIG. 13 is a flowchart illustrating an example of a method for analyzing intention.
- FIG. 1 illustrates an example of an apparatus for analyzing an intention.
- FIG. 1 illustrates an example of an apparatus for analyzing an intention implemented in a speech dialogue system that performs speech recognition in response to a user's speech being input and analyzes the intentions of speech.
- apparatus 100 for analyzing an intention includes a preprocessor 110 , a speech recognizer 120 , an acoustic model 130 , a language model 140 , an intention analyzer 150 , an intention analysis database (DB) 160 , and an analysis applier 170 .
- the preprocessor 110 detects a speech section from an input acoustic signal, generates speech feature information from the detected speech section, and transfers the speech feature information to the speech recognizer 120 .
- the speech recognizer 120 converts the input speech feature information into at least one speech recognition candidate sentence using at least one of the acoustic model 130 and the language model 140 .
- the speech recognizer 120 may perform speech recognition alone or using both an acoustic feature and a language model. For example, a statistical language model such as an n-gram model or a grammar-based model such as a context-free grammar may be used as the language model 140 .
- the speech recognizer 120 transfers a set of speech recognition candidate sentences.
- the speech recognition candidate sentences may be expressed by n-best sentences as speech recognition results to the intention analyzer 150 .
- Each sentence output from the speech recognizer 120 may include tag information that indicates features of morphemes in the sentence.
- the intention analyzer 150 may solve these problems and may analyze the intention of a speech pattern, which has not been defined in advance and which may be referred to as an out-of-grammar (OOG) expression.
- OOG out-of-grammar
- the intention analyzer 150 analyzes the intentions of the speech recognition candidate sentences generated by the speech recognizer 120 , and generates and outputs speech recognition result candidates to which the intentions of the sentences are attached. Also, the intention analyzer 150 may verify the speech recognition result candidates, score the verified speech recognition result candidates, and rearrange the speech recognition result candidates based on the respective scores. For example, the intention analyzer may arrange the speech recognition results in a decreasing order based on score.
- the intention analyzer 150 may analyze the intention of a recognized speech, for example, using context-free grammar, dependency grammar, and the like.
- context-free grammar When the context-free grammar is applied to a sentence, semantic roles may be attached to words or phrases of the sentence, and an intention analyzed from the whole sentence may be determined.
- the intention analysis DB 160 stores various information used for intention analysis. The intention analyzer is further described with reference to FIG. 2 .
- the analysis applier 170 may conduct a predetermined action based on an analyzed intention.
- the analysis applier 170 may execute a predetermined application according to the analyzed intention, and generate and provide the application execution results to a user.
- the analyzed intention may be varied according to a field to which speech recognition is applied, such as ticket reservation, performance reservation, and broadcast recording, and the like.
- FIG. 2 illustrates an example of an intention analyzer.
- the intention analyzer may be the intention analyzer 150 of the apparatus 100 of FIG. 1 .
- the intention analyzer 150 includes a sentence analyzer 210 , a phrase spotter 220 , a valid sentence determiner 230 , an intention deducer 240 , a scorer 250 , a context-free grammar DB 151 , a dependency grammar DB 152 , a phrase chunking DB 153 , an ontology DB 154 , and a role network DB 155 .
- the context-free grammar DB 151 , the dependency grammar DB 152 , the phrase chunking DB 153 , the ontology DB 154 , and the role network DB 155 may be included in the intention analysis DB 160 of FIG. 1 .
- the sentence analyzer 210 may apply information stored in the context-free grammar DB 151 to at least one sentence generated by a user's speech, to analyze the intention of each sentence. When phrase spotting is performed on all input sentences, the sentence analyzer 210 may not be included in the intention analyzer 150 .
- intention analysis is successful, the results of successful intention analysis may be stored, and the intention of a next recognition candidate sentence may be analyzed using the context-free grammar. A speech recognition candidate sentence whose intention has been successfully analyzed and the intention analysis results may be transferred to the scorer 250 .
- FIG. 5 illustrates an example of context-free grammar.
- Context-free grammar information stored in the context-free grammar DB 151 may include information on the semantic role of each word or phrase and grammatical relationships between words or phrases. By applying the context-free grammar to a sentence, it is possible to determine whether the sentence is in an intention frame that is defined in the context-free grammar.
- the context-free grammar DB 151 may be expressed by a context-free grammar network 620 as shown in FIG. 6 .
- the intention frame refers to a format representing the intention of a user that may be obtained by applying the context-free grammar to a sentence.
- An intention frame may include an intention name and at least one semantic role element that are included in the intention frame. However, in cases, the intention frame may not include any semantic role. For example, a sentence “Turn TV on” has “Turn on TV” as an intention frame and has no semantic role element.
- At least one intention frame may be defined in advance for various fields, for example, a newspaper article search, a ticket reservation, a weather search, and the like.
- FIG. 5 illustrates an example of information stored in the context-free grammar DB 151 about the field of a news search.
- search(@object, @day, @section) being determined as the intention frame of newspaper article search
- the sentence spoken by the user may be determined to have an intention name “search” and indicate an order to search for articles about an object (@object) in a section (@section) from a day (@day) of the week.
- the sentence analyzer 210 may produce the analysis results as intention analysis results.
- phrases spotter 220 refers to semantic phrase spotting.
- Phrase spotting refers to semantic phrase spotting.
- the phrase spotter 220 applies the context-free grammar to each word or combination of words rather than the whole sentence.
- results of partial phrase spotting that is, the semantic roles of respective words or phrases, and at least one intention frame to which the semantic role of each word or phrase belongs may be determined in units.
- the partial phrase spotting may determine an intention frame based on a word or a phrase from the sentence.
- phrase spotting is to perform an intention analysis of a sentence including an OOG expression.
- intention analysis is performed using the context-free grammar alone, like conventional intention analysis algorithms, only sentences suited for the context-free grammar may be analyzed, and it may be difficult to analyze the intentions of a user's general speeches that are sometimes ungrammatical or not recognized.
- FIG. 6 illustrates an example of phrase spotting.
- phrases spotter 220 matches a speech recognition candidate sentence with nodes of a context-free grammar network using a grammar made according to the context-free grammar.
- a matching level between the sentence and nodes of the context-free grammar network may be determined in units of words, phrases, and the like.
- Each phrase in one sentence may be interpreted to have various semantic roles, and one phrase may overlap and belong to several intention frames. Thus, one sentence may have several phrase spotting results.
- phrase spotting is performed on a sentence 610 consisting of ⁇ circle around (a) ⁇ - ⁇ circle around (b) ⁇ - ⁇ circle around (c) ⁇ - ⁇ circle around (d) ⁇ - ⁇ circle around (x) ⁇ - ⁇ circle around (y) ⁇ - ⁇ circle around (z) ⁇ with reference to the context-free grammar network 620 .
- respective nodes ⁇ circle around (a) ⁇ , ⁇ circle around (b) ⁇ , ⁇ circle around (c) ⁇ , ⁇ circle around (d) ⁇ , ⁇ circle around (x) ⁇ , ⁇ circle around (y) ⁇ , and ⁇ circle around (z) ⁇ of the context-free grammar network 620 denote words of a sentence.
- the context-free grammar network 620 may be a context-free grammar expressed as a network of semantic roles.
- Semantic roles for example, a day of the week (@day), an object (@object), a section (@section), and a time (@time), indicate semantic roles of words in a sentence.
- arrows indicate that origination nodes of the arrows appear prior to destination nodes of the arrows in the sentence.
- sets of nodes connected by arrows may be defined as intention frames.
- the semantic role of @time is mapped to example words “today” and “tomorrow” in FIG. 5
- several example words may be mapped onto one semantic role in the context-free grammar network 620 .
- ⁇ circle around (a) ⁇ - ⁇ circle around (b) ⁇ - ⁇ circle around (c) ⁇ - ⁇ circle around (d) ⁇ - ⁇ circle around (x) ⁇ - ⁇ circle around (y) ⁇ - ⁇ circle around (z) ⁇ may be determined to correspond to node paths 621 , 622 and 623 in the context-free grammar network 620 .
- an intention frame 1 and intention frame k may be determined as candidate intention frames of the sentence 610 .
- FIG. 7 illustrates an example of a phrase spotting operation.
- a speech recognition candidate sentence output recognized by the speech recognizer 120 is “Reserve a train for Kansas City at three o'clock,” it may be presumed that “reserve a train (@object) for Kansas City (@region) at three o'clock (@startTime)” is output from the context-free grammar network 620 as a result of applying the context-free grammar. Accordingly, one or more candidate intention analysis results may be determined as phrase spotting results.
- an intention frame MakeReservation(@object, @startTime, @destination) 720 and an intention frame Getweather(@region) 730 match the speech recognition candidate sentence in a high matching level of semantic roles.
- the valid sentence determiner 230 examines the grammatical and semantic validity of a sentence using the dependency grammar.
- the dependency grammar may be in a form as shown in FIG. 8 .
- PV, NP, NC, NC, JCM, and NR refer to morpheme class tag information, each of which indicates a type of morpheme.
- the dependency grammar indicates what type of dependency relation is established between respective parts (words or phrases) of a sentence.
- the valid sentence determiner 230 may examine dependency relations between respective parts of a sentence. Also, the valid sentence determiner 230 may examine whether respective phrases having semantic roles and respective phrases not having semantic roles are dependent upon each other. For example, word classes, words, meanings, and the like may be used as elements of the dependency grammar, and one or more of them may be used.
- a sentence that has undergone phrase spotting and that has been determined to be valid according to the dependency grammar may be temporarily stored in a predetermined storage space where it may undergo an intention deduction process by the intention deducer 240 .
- a sentence that has been determined to be invalid according to the dependency grammar is an ungrammatical sentence or a semantically incorrect sentence and may be filtered. In other words, among speech recognition candidate sentences that have undergone phrase spotting, an ungrammatical or semantically incorrect sentence may be ignored.
- the intention deducer 240 determines one final intention frame among one or more intention frames that may be selected for a sentence that has undergone phrase spotting and been determined to be valid among speech recognition candidate sentences.
- the intention deducer 240 allocates semantic role values to semantic role elements which are components of the intention frame, and generates intention analysis results.
- the intention deducer 240 may estimate the semantic role values by applying an ontology such as WORDNET® to words that are not in the intention frame.
- WORDNET® an ontology
- the intention deducer 240 may deduce whether the words that are not in the intention frame correspond to semantic roles of the intention frame, and what kinds of semantic roles correspond to the words of the intention frame.
- the ontology denotes semantic relationships between words
- the role network denotes relationship between semantic roles.
- FIG. 9 illustrates an example of a role network.
- @region denotes the semantic role of a region
- @destination denotes the semantic role of a destination
- @origin denotes the semantic role of a point of origin.
- @region, @destination, and @origin have different semantic roles.
- @destination and @origin are disposed at lower nodes of @region in the semantic role network and may have a semantic relationship with each other.
- the intention deducer 240 is described later with reference to FIGS. 3 and 4 .
- the scorer 250 may calculate the probability that intention analysis results are speech recognition results and/or the probability that intention analysis has been correctly performed for the intention analysis results, and perform scoring based on the calculated probability.
- one of the intention analysis results is generated by the sentence analyzer 210 using the context-free grammar.
- the other intention analysis result is processed by the phrase spotter 220 , the valid sentence determiner 230 , and the intention deducer 240 because its intention frame has not been determined by the sentence analyzer 210 .
- the following elements may be used for scoring:
- phrase spotting such as information about how many network paths words match the context-free grammar network
- elements used for intention frame selection such as the matching level between words, the matching level between word categories, the matching level between semantic role elements, and the matching level between headwords;
- a sentence interpreted according to the context-free grammar and/or a sentence having undergone phrase spotting is correct, such as a variety of contexts (the field of current conversation, a field of interest to a user, previous speeches, a previous system response, and the like.
- the scorer 250 transfers at least one intention frame for each speech recognition candidate sentence to which a score has been given to the analysis applier 170 .
- a recognition candidate sentence whose overall intention has not been analyzed by the sentence analyzer 210 may be processed by the phrase spotter 220 , the valid sentence determiner 230 , and the intention deducer 240 . Also, the intentions of n-best sentences output from the speech recognizer 120 may be directly analyzed by the phrase spotter 220 without the sentence analyzer 210 .
- Analyzing the intention of a recognition candidate sentence that the sentence analyzer 210 cannot successfully analyze using the phrase spotter 220 may be useful when a probability of an OOG expression occurring is low and it is desirable to use a small amount of resources. It is unnecessary to perform phrase spotting in the method when the intention of a sentence can be analyzed using the context-free grammar, and thus program execution time and required resources are reduced.
- Analyzing the respective intentions of all speech recognition candidate sentences by performing phrase spotting using the phrase spotter 220 without using the sentence analyzer 210 from the beginning may be useful when a probability of an OOG expression occurring is high and one unified intention analysis structure is needed.
- intention analysis may be performed using the context-free grammar DB 152 once, unlike a case in which the sentence analyzer 210 is used.
- time or resources may be wasted.
- FIG. 3 illustrates an example of an intention deducer, for example, the intention deducer 240 of FIG. 2 .
- the intention deducer 240 includes an intention frame selector 310 and a semantic role value allocator 320 .
- the intention frame selector 310 selects an intention frame that is an intention analysis result for each speech recognition candidate sentence.
- the intention frame selector 310 may compare intention frames of the context-free grammar with the phrase spotting result of a sentence that is determined to be valid.
- Various elements may be compared, for example, whether or not headwords of sentences match each other, whether or not semantic role elements match each other, whether or not words match each other, and the like.
- the headword of a sentence may be a word that is determined to have the largest number of dependency relation with other words.
- the semantic role value allocator 320 may allocate a semantic role value to at least one semantic role element included in the selected intention frame.
- FIG. 4 illustrates an example of a method of a semantic role value allocator, for example, the semantic role value allocator 320 of the intention deducer 240 of FIG. 3 .
- the semantic role value allocator 320 determines whether at least one semantic role element in an intention frame selected by the intention frame selector 310 matches at least one semantic role element of a speech recognition candidate sentence that has undergone phrase spotting.
- the speech recognition candidate sentence that has undergone phrase spotting is a sentence that has been determined to be grammatically valid.
- the semantic role value allocator 320 may allocate phrases corresponding to respective semantic roles of the speech recognition candidate sentence that has undergone phrase spotting as the semantic role values of semantic role elements in the intention frame.
- phrase chunking may be performed on the word together with the adjacent words using the phrase chunking DB 153 that stores information for phrase chunking to determine the range of the semantic role values.
- Phrase chunking refers to a natural language process that segments a sentence into sub-parts, for example, a noun, a verb, a prepositional phrase, and the like.
- a semantic role value is allocated, at least one intention analysis result candidate may be generated. An example of this process is described with reference to FIG. 10 .
- FIG. 10 illustrates an example of the allocation of a semantic role value in response to semantic role elements matching.
- a speech recognition candidate sentence that has undergone phrase spotting is “I want to reserve a train ticket (@object) for Seoul (@destination)” and a selected intention frame is “MakeReservation(@destination, @object),” Accordingly, semantic role elements of the speech recognition candidate sentence that has undergone phrase spotting match those in the selected intention frame, that is, @destination and @object.
- the semantic role value allocator 320 determines whether a semantic role element that is not in the intention frame is in the sentence that has undergone phrase spotting.
- the semantic role value allocator 320 may determine relationships between semantic roles with reference to a role network from the role network DB 155 . In response to the semantic roles having a parent-child relationship in the role network, it may be determined that the semantic role is replaceable. In response to the semantic role being determined to be replaceable, in operation 450 the semantic role value allocator 320 may determine the range of a semantic role value through phrase chunking and allocate the semantic role value that belongs to the selected intention frame.
- FIG. 11 illustrates an example of the allocation of a semantic role value in response to semantic role elements not matching.
- phrase spotting result When a phrase spotting result is “reserve a [train](@object) for [Kansas City](@region) at [three o'clock](@startTime),” and an intention frame is “MakeReservation(@object, @startTime, @destination),” the phrase spotting result has @region that is not in the intention frame.
- @region and @destination are in a parent-children relationship referring to a role network as shown in FIG. 9 . Accordingly, @region and @destination may be replaced with each other.
- the semantic role value allocator 320 may estimate a semantic role value through phrase chunking using the ontology and may allocate the semantic role value. The estimation of the semantic role value may be performed in response to it being determined that there is a semantic role element in the intention frame but not in the phrase spotting result.
- the semantic role value allocator 320 may check the positions of words that are not matching the intention frame in the phrase spotting result, and may determine the range of semantic role values through phrase chunking and allocate the semantic role values in response to it being determined that the words are at positions that may have semantic role values in the sentence.
- the categories of words in the speech recognition candidate sentence that has undergone phrase spotting are compared with those of words corresponding to the semantic role elements of the intention frame.
- Semantic role values may be determined in response to the words in the speech recognition candidate sentence that has undergone phrase spotting and the words corresponding to the semantic role elements of the intention frame being in the same categories or in a parent-child relationship. Comparison of word categories may be performed using the ontology. Also, in response to a phrase being likely to be a proper noun, a semantic role value may be allocated without the category comparison process. An example of this process is described with reference to FIG. 12 .
- FIG. 12 illustrates an example of the estimation of a semantic role value through phrase chunking.
- the semantic role of “Lovers in Paris” in the phrase spotting result may not be determined even with reference to an ontology.
- the semantic role value allocator 320 may determine “Lovers in Paris” as a proper noun and allocate “Lovers in Paris” to @object of the intention frame as a semantic role value.
- FIG. 13 illustrates an example of a method for analyzing intention.
- the phrase spotter 220 performs phrase spotting on at least one sentence by applying the context-free grammar to the at least one sentence.
- the valid sentence determiner 230 determines whether the sentences are grammatically valid by applying the dependency grammar to the sentences that have undergone phrase spotting, and filters an invalid sentence.
- the intention deducer 240 generates the intention analysis result of a sentence determined to be valid. For example, the intention deducer 240 may select an intention frame to be the intention analysis result of the sentence that has undergone phrase spotting, determine a semantic role value for a semantic role element included in the intention frame from the sentence that has undergone phrase spotting, and allocate the determined semantic role value to the semantic role element in the selected intention frame.
- the apparatus 100 for analyzing an intention can be applied not only to sentences that are recognized by speech recognition but also to general sentences that are not recognized by speech recognition, and employed in systems having various forms for a variety of purposes.
- an OOG expression can be processed to increase the degree of freedom of speech of a user, and the rate of success in intention analysis and the overall performance of a speech dialogue system can be increased in comparison with a conventional speech dialogue system that performs speech recognition using predetermined speech only.
- the processes, functions, methods and/or software described above may be recorded, stored, or fixed in one or more computer-readable storage media that includes program instructions to be implemented by a computer to cause a processor to execute or perform the program instructions.
- the media may also include, alone or in combination with the program instructions, data files, data structures, and the like.
- the media and program instructions may be those specially designed and constructed, or they may be of the kind well-known and available to those having skill in the computer software arts.
- Examples of computer-readable media include magnetic media, such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks and DVDs; magneto-optical media, such as optical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like.
- Examples of program instructions include machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.
- the described hardware devices may be configured to act as one or more software modules in order to perform the operations and methods described above, or vice versa.
- a computer-readable storage medium may be distributed among computer systems connected through a network and computer-readable codes or program instructions may be stored and executed in a decentralized manner.
- a computing system or a computer may include a microprocessor that is electrically connected with a bus, a user interface, and a memory controller. It may further include a flash memory device. The flash memory device may store N-bit data via the memory controller. The N-bit data is processed or will be processed by the microprocessor and N may be 1 or an integer greater than 1 . Where the computing system or computer is a mobile apparatus, a battery may be additionally provided to supply operation voltage of the computing system or computer.
- the computing system or computer may further include an application chipset, a camera image processor (CIS), a mobile Dynamic Random Access Memory (DRAM), and the like.
- the memory controller and the flash memory device may constitute a solid state drive/disk (SSD) that uses a non-volatile memory to store data.
- SSD solid state drive/disk
Abstract
An apparatus and system for analyzing intention are provided. The apparatus for analyzing an intention applies a context-free grammar to each of one or more sentences in units of one or more phrases to perform phrase spotting on each sentence, thereby extending a recognition range for an out-of-grammar (OOG) expression. Meanwhile, the apparatus for analyzing an intention determines whether sentences that have undergone phrase spotting are grammatically valid by applying a dependency grammar to the sentences to filter an invalid sentence, and generates the intention analysis result of a valid sentence, thereby and grammatically and/or semantically verifying a sentence that has undergone speech recognition while extending a speech recognition range.
Description
- This application claims the benefit under 35 U.S.C. §119(a) of Korean Patent Application No. 10-2009-0094019 filed on Oct. 1, 2009, the entire disclosure of which is incorporated herein by reference for all purposes.
- 1. Field
- The following description relates to a technology for analyzing the intention of a user, and more particularly, to an apparatus and method for analyzing the intention of a sentence generated by a user.
- 2. Description of the Related Art
- Voice interaction technology is becoming essential for interaction between humans and computer systems. Modern voice recognition technology provides high performance for previously defined speeches.
- Generally, to model a user's speech, a grammar-based language model such as context free grammar language model or a statistical language model such as an N-gram language model is used.
- The grammar-based language model advantageously accepts only a grammatically and semantically correct sentence as a recognition result, but cannot recognize a sentence which has not been pre-defined in terms of grammars. The statistical language models may recognize some sentences that have not been pre-defined and do not require a user to manually define grammar.
- However, because the statistical language model cannot take into consideration a structure of a whole sentence in the course of speech recognition, an ungrammatical sentence may be output as a recognition result. Also, a large amount of training data is needed to generate a language model. Due to these drawbacks, it is difficult to use the current speech dialogue system in a real-world application.
- In one general aspect, there is provided an apparatus for analyzing intention, the apparatus comprising: a phrase spotter configured to perform phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases; a valid sentence determiner configured to: determine whether the at least one sentence is grammatically valid by applying a dependency grammar to the sentence that has undergone phrase spotting; and filter an invalid sentence; and an intention deducer configured to generate an intention analysis result of a sentence determined to be valid.
- The apparatus may further include that the intention deducer is further configured to: select an intention frame to be the intention analysis result of the sentence determined to be valid; determine a semantic role value of at least one semantic role element included in the selected intention frame; and allocate the determined semantic role value to the semantic role element included in the selected intention frame.
- The apparatus may further include that, in response to the intention deducer allocating the semantic role value, the intention deducer is further configured to: determine the semantic role value from the sentence determined to be valid through phrase chunking; and allocate the determined semantic role value to the semantic role element in the selected intention frame if at least one semantic role element of the sentence determined to be valid matches at least one semantic role element in the selected intention frame.
- The apparatus may further include that, in response to the sentence determined to be valid comprising a semantic role element other than the at least one semantic role element in the intention frame, the intention deducer is further configured to: determine whether the other semantic intention role element can be replaced by the semantic role element in the intention frame using a role network; determine a semantic role value of the semantic role element in the intention frame from the sentence determined to be valid through phrase chunking in response to it being determined that the other semantic intention role element can be replaced by the semantic role element in the intention frame; and allocate the determined semantic role value to the semantic role element in the intention frame.
- The apparatus may further include that the intention deducer is further configured to estimate the semantic role value of the at least one semantic role element in the intention frame using an ontology.
- The apparatus may further include a scorer configured to: calculate a probability that intention analysis has been correctly performed on at least one intention analysis result candidate to which the semantic role value of the semantic role element included in the selected intention frame is allocated; and score the intention analysis result candidate.
- The apparatus may further include an analysis applier configured to: apply the intention analysis result to an application; and generate an intention analysis application result.
- The apparatus may further include a speech recognizer configured to convert an audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted by the speech recognizer.
- In another general aspect, there is provided a method of analyzing an intention, the method comprising: performing phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases; determining whether the at least one sentence is grammatically valid by: applying a dependency grammar to the sentence that has undergone phrase spotting; and filtering an invalid sentence; and generating an intention analysis result of a sentence determined to be valid.
- The method may further include that the generating of the intention analysis result of the sentence determined to be valid comprises: selecting an intention frame to be the intention analysis result of the sentence determined to be valid; determining semantic role values of semantic role elements included in the selected intention frame; and allocating the determined semantic role values to the semantic role elements included in the selected intention frame.
- The method may further include that the allocating of the semantic role values comprises: determining whether at least one semantic role element of the sentence determined to be valid matches at least one semantic role element in the selected intention frame; and in response to it being determined that the at least one semantic role element of the sentence determined to be valid matches the at least one semantic role element in the selected intention frame: determining the semantic role values from the sentence determined to be valid through phrase chunking; and allocating the determined semantic role values.
- The method may further include that, in response to the semantic role element of the sentence determined to be valid not matching the semantic role element in the selected intention frame, the allocating of the semantic role values further comprises: determining whether the sentence determined to be valid comprises a semantic role element other than the semantic role elements of the intention frame; in response to the sentence determined to be valid comprising a semantic role element other than the semantic role elements of the intention frame, determining whether the other semantic role element can be replaced by the semantic role element in the intention frame using a role network; and in response to it being determined that the other semantic role element can be replaced by the semantic role element in the intention frame: determining the semantic role value of the semantic role element in the intention frame from the sentence determined to be valid through phrase chunking; and allocating the determined semantic role value to the semantic role element in the intention frame.
- The method may further include estimating the semantic role value of the at least one semantic role element in the intention frame using an ontology.
- The method may further include: calculating probabilities that intention analysis has been correctly performed on at least one intention analysis result candidate to which the semantic role value of the semantic role element in the selected intention frame is allocated; and scoring the intention analysis result candidates.
- The method may further include applying the intention analysis result to an application and generating an intention analysis application result.
- The method may further include performing speech recognition on an audio input and converting the audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted through the speech recognition.
- In another general aspect, there is provided a computer-readable storage medium storing a program that causes a computer to execute a method of analyzing an intention, comprising: performing phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases; determining whether the at least one sentence is grammatically valid by: applying a dependency grammar to the sentence that has undergone phrase spotting; and filtering an invalid sentence; and generating an intention analysis result of a sentence determined to be valid.
- Other features and aspects may be apparent from the following description, the drawings, and the claims.
-
FIG. 1 is a diagram illustrating an example of an apparatus for analyzing an intention. -
FIG. 2 is a diagram illustrating an example of an intention analyzer. -
FIG. 3 is a diagram illustrating an example of an intention deducer. -
FIG. 4 is a flowchart illustrating an example of a method of a semantic role value allocator. -
FIG. 5 is a diagram illustrating an example of context-free grammar. -
FIG. 6 is a diagram illustrating an example of phrase spotting. -
FIG. 7 is a diagram illustrating an example of a phrase spotting operation. -
FIG. 8 is a diagram illustrating an example of dependency grammar. -
FIG. 9 is a diagram illustrating an example of a role network. -
FIG. 10 is a diagram illustrating an example of the allocation of a semantic role value in response to semantic role elements matching. -
FIG. 11 is a diagram illustrating an example of the allocation of a semantic role value in response to semantic role elements not matching. -
FIG. 12 is a diagram illustrating an example of the estimation of a semantic role value through phrase chunking. -
FIG. 13 is a flowchart illustrating an example of a method for analyzing intention. - Throughout the drawings and the description, unless otherwise described, the same drawing reference numerals should be understood to refer to the same elements, features, and structures. The relative size and depiction of these elements may be exaggerated for clarity, illustration, and convenience.
- The following description is provided to assist the reader in gaining a comprehensive understanding of the methods, apparatuses, and/or systems described herein. Accordingly, various changes, modifications, and equivalents of the methods, apparatuses, and/or systems described herein may be suggested to those of ordinary skill in the art. The progression of processing steps and/or operations described is an example; however, the sequence of steps and/or operations is not limited to that set forth herein and may be changed as is known in the art, with the exception of steps and/or operations necessarily occurring in a certain order. Also, descriptions of well-known functions and constructions may be omitted for increased clarity and conciseness.
-
FIG. 1 illustrates an example of an apparatus for analyzing an intention. -
FIG. 1 illustrates an example of an apparatus for analyzing an intention implemented in a speech dialogue system that performs speech recognition in response to a user's speech being input and analyzes the intentions of speech. - In this example,
apparatus 100 for analyzing an intention includes apreprocessor 110, aspeech recognizer 120, anacoustic model 130, alanguage model 140, anintention analyzer 150, an intention analysis database (DB) 160, and an analysis applier 170. - The
preprocessor 110 detects a speech section from an input acoustic signal, generates speech feature information from the detected speech section, and transfers the speech feature information to thespeech recognizer 120. - The speech recognizer 120 converts the input speech feature information into at least one speech recognition candidate sentence using at least one of the
acoustic model 130 and thelanguage model 140. The speech recognizer 120 may perform speech recognition alone or using both an acoustic feature and a language model. For example, a statistical language model such as an n-gram model or a grammar-based model such as a context-free grammar may be used as thelanguage model 140. The speech recognizer 120 transfers a set of speech recognition candidate sentences. The speech recognition candidate sentences may be expressed by n-best sentences as speech recognition results to the intention analyzer 150. Each sentence output from thespeech recognizer 120 may include tag information that indicates features of morphemes in the sentence. - When the
speech recognizer 120 performs speech recognition using theacoustic model 130 or a statistical language model of thelanguage model 140, the overall sentence structure and the meaning may not be taken into consideration. Also, when a frequently used n-gram model for speech recognition is applied, an ungrammatical sentence may be output as a speech recognition result. Theintention analyzer 150 may solve these problems and may analyze the intention of a speech pattern, which has not been defined in advance and which may be referred to as an out-of-grammar (OOG) expression. - The
intention analyzer 150 analyzes the intentions of the speech recognition candidate sentences generated by thespeech recognizer 120, and generates and outputs speech recognition result candidates to which the intentions of the sentences are attached. Also, theintention analyzer 150 may verify the speech recognition result candidates, score the verified speech recognition result candidates, and rearrange the speech recognition result candidates based on the respective scores. For example, the intention analyzer may arrange the speech recognition results in a decreasing order based on score. - The
intention analyzer 150 may analyze the intention of a recognized speech, for example, using context-free grammar, dependency grammar, and the like. When the context-free grammar is applied to a sentence, semantic roles may be attached to words or phrases of the sentence, and an intention analyzed from the whole sentence may be determined. Theintention analysis DB 160 stores various information used for intention analysis. The intention analyzer is further described with reference toFIG. 2 . - The
analysis applier 170 may conduct a predetermined action based on an analyzed intention. Theanalysis applier 170 may execute a predetermined application according to the analyzed intention, and generate and provide the application execution results to a user. The analyzed intention may be varied according to a field to which speech recognition is applied, such as ticket reservation, performance reservation, and broadcast recording, and the like. -
FIG. 2 illustrates an example of an intention analyzer. For example, the intention analyzer may be theintention analyzer 150 of theapparatus 100 ofFIG. 1 . - Referring to
FIG. 2 , theintention analyzer 150 includes asentence analyzer 210, aphrase spotter 220, avalid sentence determiner 230, anintention deducer 240, ascorer 250, a context-free grammar DB 151, adependency grammar DB 152, aphrase chunking DB 153, anontology DB 154, and arole network DB 155. The context-free grammar DB 151, thedependency grammar DB 152, thephrase chunking DB 153, theontology DB 154, and therole network DB 155 may be included in theintention analysis DB 160 ofFIG. 1 . - The
sentence analyzer 210 may apply information stored in the context-free grammar DB 151 to at least one sentence generated by a user's speech, to analyze the intention of each sentence. When phrase spotting is performed on all input sentences, thesentence analyzer 210 may not be included in theintention analyzer 150. When intention analysis is successful, the results of successful intention analysis may be stored, and the intention of a next recognition candidate sentence may be analyzed using the context-free grammar. A speech recognition candidate sentence whose intention has been successfully analyzed and the intention analysis results may be transferred to thescorer 250. -
FIG. 5 illustrates an example of context-free grammar. - Context-free grammar information stored in the context-
free grammar DB 151 may include information on the semantic role of each word or phrase and grammatical relationships between words or phrases. By applying the context-free grammar to a sentence, it is possible to determine whether the sentence is in an intention frame that is defined in the context-free grammar. The context-free grammar DB 151 may be expressed by a context-free grammar network 620 as shown inFIG. 6 . - The intention frame refers to a format representing the intention of a user that may be obtained by applying the context-free grammar to a sentence. An intention frame may include an intention name and at least one semantic role element that are included in the intention frame. However, in cases, the intention frame may not include any semantic role. For example, a sentence “Turn TV on” has “Turn on TV” as an intention frame and has no semantic role element. At least one intention frame may be defined in advance for various fields, for example, a newspaper article search, a ticket reservation, a weather search, and the like.
-
FIG. 5 illustrates an example of information stored in the context-free grammar DB 151 about the field of a news search. For example, in response to “search(@object, @day, @section)” being determined as the intention frame of newspaper article search, the sentence spoken by the user may be determined to have an intention name “search” and indicate an order to search for articles about an object (@object) in a section (@section) from a day (@day) of the week. - In response to a speech recognition candidate sentence corresponding to an intention frame defined by the context-free grammar, and the sentence being analyzed using the context-free grammar, the
sentence analyzer 210 may produce the analysis results as intention analysis results. - Meanwhile, a speech recognition candidate sentence whose overall intention is not analyzed using the context-free grammar is transferred to the
phrase spotter 220 and undergoes semantic phrase spotting. Phrase spotting refers to semantic phrase spotting. For example, when a sentence is not analyzed using the context-free grammar due to an OOG expression included in a user's speech or a speech recognition error, thephrase spotter 220 may be used. Thephrase spotter 220 applies the context-free grammar to each word or combination of words rather than the whole sentence. For example, when a sentence undergoes phrase spotting, results of partial phrase spotting, that is, the semantic roles of respective words or phrases, and at least one intention frame to which the semantic role of each word or phrase belongs may be determined in units. For example, the partial phrase spotting may determine an intention frame based on a word or a phrase from the sentence. - The purpose of phrase spotting is to perform an intention analysis of a sentence including an OOG expression. When intention analysis is performed using the context-free grammar alone, like conventional intention analysis algorithms, only sentences suited for the context-free grammar may be analyzed, and it may be difficult to analyze the intentions of a user's general speeches that are sometimes ungrammatical or not recognized.
-
FIG. 6 illustrates an example of phrase spotting. - Phrase spotting results are obtained only from interpretable words or phrases in a whole sentence. The
phrase spotter 220 matches a speech recognition candidate sentence with nodes of a context-free grammar network using a grammar made according to the context-free grammar. - When an input sentence and the context-free grammar network are matched together, for example, a dynamic programming technique may be used. A matching level between the sentence and nodes of the context-free grammar network may be determined in units of words, phrases, and the like. Each phrase in one sentence may be interpreted to have various semantic roles, and one phrase may overlap and belong to several intention frames. Thus, one sentence may have several phrase spotting results.
- Referring to
FIG. 6 , phrase spotting is performed on asentence 610 consisting of {circle around (a)}-{circle around (b)}-{circle around (c)}-{circle around (d)}-{circle around (x)}-{circle around (y)}-{circle around (z)} with reference to the context-free grammar network 620. In this example, respective nodes {circle around (a)}, {circle around (b)}, {circle around (c)}, {circle around (d)}, {circle around (x)}, {circle around (y)}, and {circle around (z)} of the context-free grammar network 620 denote words of a sentence. The context-free grammar network 620 may be a context-free grammar expressed as a network of semantic roles. - Semantic roles, for example, a day of the week (@day), an object (@object), a section (@section), and a time (@time), indicate semantic roles of words in a sentence. In the context-
free grammar network 620, arrows indicate that origination nodes of the arrows appear prior to destination nodes of the arrows in the sentence. In the context-free grammar network 620, sets of nodes connected by arrows may be defined as intention frames. Just as the semantic role of @time is mapped to example words “today” and “tomorrow” inFIG. 5 , several example words may be mapped onto one semantic role in the context-free grammar network 620. - As shown in
FIG. 6 , the intention of thesentence 610 is not analyzed using the context-free grammar. When phrase spotting is performed on thesentence 610, {circle around (a)}-{circle around (b)}-{circle around (c)}-{circle around (d)}-{circle around (x)}-{circle around (y)}-{circle around (z)} may be determined to correspond tonode paths free grammar network 620. In this example, anintention frame 1 and intention frame k may be determined as candidate intention frames of thesentence 610. -
FIG. 7 illustrates an example of a phrase spotting operation. - When a speech recognition candidate sentence output recognized by the
speech recognizer 120 is “Reserve a train for Kansas City at three o'clock,” it may be presumed that “reserve a train (@object) for Kansas City (@region) at three o'clock (@startTime)” is output from the context-free grammar network 620 as a result of applying the context-free grammar. Accordingly, one or more candidate intention analysis results may be determined as phrase spotting results. - Referring to
FIG. 7 , an intention frame MakeReservation(@object, @startTime, @destination) 720 and an intention frame Getweather(@region) 730 match the speech recognition candidate sentence in a high matching level of semantic roles. InFIG. 7 , “MakeReservation(@object=train, @startTime=three o'clock, @destination=Boston),” “Reserve a train for Boston at three o'clock,” “GetWeather(@region=Kansas City),” and “What's the weather like in Kansas City?” indicate example word information and example sentences about respective intention frames in the context-free grammar network 620. - Referring back to
FIG. 2 , sentences that have undergone phrase spotting by thephrase spotter 220 are input to thevalid sentence determiner 230. Thevalid sentence determiner 230 examines the grammatical and semantic validity of a sentence using the dependency grammar. The dependency grammar may be in a form as shown inFIG. 8 . InFIG. 8 , PV, NP, NC, NC, JCM, and NR refer to morpheme class tag information, each of which indicates a type of morpheme. The dependency grammar indicates what type of dependency relation is established between respective parts (words or phrases) of a sentence. - The
valid sentence determiner 230 may examine dependency relations between respective parts of a sentence. Also, thevalid sentence determiner 230 may examine whether respective phrases having semantic roles and respective phrases not having semantic roles are dependent upon each other. For example, word classes, words, meanings, and the like may be used as elements of the dependency grammar, and one or more of them may be used. - A sentence that has undergone phrase spotting and that has been determined to be valid according to the dependency grammar may be temporarily stored in a predetermined storage space where it may undergo an intention deduction process by the
intention deducer 240. A sentence that has been determined to be invalid according to the dependency grammar is an ungrammatical sentence or a semantically incorrect sentence and may be filtered. In other words, among speech recognition candidate sentences that have undergone phrase spotting, an ungrammatical or semantically incorrect sentence may be ignored. - The
intention deducer 240 determines one final intention frame among one or more intention frames that may be selected for a sentence that has undergone phrase spotting and been determined to be valid among speech recognition candidate sentences. In addition, theintention deducer 240 allocates semantic role values to semantic role elements which are components of the intention frame, and generates intention analysis results. Theintention deducer 240 may estimate the semantic role values by applying an ontology such as WORDNET® to words that are not in the intention frame. Also, using a role network, theintention deducer 240 may deduce whether the words that are not in the intention frame correspond to semantic roles of the intention frame, and what kinds of semantic roles correspond to the words of the intention frame. Like WORDNET®, the ontology denotes semantic relationships between words, and the role network denotes relationship between semantic roles. -
FIG. 9 illustrates an example of a role network. - As shown in
FIG. 9 , @region denotes the semantic role of a region, @destination denotes the semantic role of a destination, and @origin denotes the semantic role of a point of origin. In other words, @region, @destination, and @origin have different semantic roles. However, @destination and @origin are disposed at lower nodes of @region in the semantic role network and may have a semantic relationship with each other. Theintention deducer 240 is described later with reference toFIGS. 3 and 4 . - Referring back to
FIG. 2 , thescorer 250 may calculate the probability that intention analysis results are speech recognition results and/or the probability that intention analysis has been correctly performed for the intention analysis results, and perform scoring based on the calculated probability. In this example, one of the intention analysis results is generated by thesentence analyzer 210 using the context-free grammar. The other intention analysis result is processed by thephrase spotter 220, thevalid sentence determiner 230, and theintention deducer 240 because its intention frame has not been determined by thesentence analyzer 210. The following elements may be used for scoring: - a confidence score calculated by the
speech recognizer 120 using acoustic features; - an element related with phrase spotting, such as information about how many network paths words match the context-free grammar network;
- elements used for intention frame selection, such as the matching level between words, the matching level between word categories, the matching level between semantic role elements, and the matching level between headwords; and
- elements whereby it is possible to determine if a sentence interpreted according to the context-free grammar and/or a sentence having undergone phrase spotting is correct, such as a variety of contexts (the field of current conversation, a field of interest to a user, previous speeches, a previous system response, and the like.
- After performing the scoring, the
scorer 250 transfers at least one intention frame for each speech recognition candidate sentence to which a score has been given to theanalysis applier 170. - In the description above, a recognition candidate sentence whose overall intention has not been analyzed by the
sentence analyzer 210 may be processed by thephrase spotter 220, thevalid sentence determiner 230, and theintention deducer 240. Also, the intentions of n-best sentences output from thespeech recognizer 120 may be directly analyzed by thephrase spotter 220 without thesentence analyzer 210. - Analyzing the intention of a recognition candidate sentence that the
sentence analyzer 210 cannot successfully analyze using thephrase spotter 220 may be useful when a probability of an OOG expression occurring is low and it is desirable to use a small amount of resources. It is unnecessary to perform phrase spotting in the method when the intention of a sentence can be analyzed using the context-free grammar, and thus program execution time and required resources are reduced. - Analyzing the respective intentions of all speech recognition candidate sentences by performing phrase spotting using the
phrase spotter 220 without using thesentence analyzer 210 from the beginning may be useful when a probability of an OOG expression occurring is high and one unified intention analysis structure is needed. In this example, intention analysis may be performed using the context-free grammar DB 152 once, unlike a case in which thesentence analyzer 210 is used. However, when an OOG expression is not included in a sentence, time or resources may be wasted. -
FIG. 3 illustrates an example of an intention deducer, for example, theintention deducer 240 ofFIG. 2 . - Referring to
FIG. 3 , theintention deducer 240 includes anintention frame selector 310 and a semanticrole value allocator 320. - The
intention frame selector 310 selects an intention frame that is an intention analysis result for each speech recognition candidate sentence. Theintention frame selector 310 may compare intention frames of the context-free grammar with the phrase spotting result of a sentence that is determined to be valid. - Various elements may be compared, for example, whether or not headwords of sentences match each other, whether or not semantic role elements match each other, whether or not words match each other, and the like. For example, the headword of a sentence may be a word that is determined to have the largest number of dependency relation with other words.
- When an intention frame is selected, the semantic
role value allocator 320 may allocate a semantic role value to at least one semantic role element included in the selected intention frame. -
FIG. 4 illustrates an example of a method of a semantic role value allocator, for example, the semanticrole value allocator 320 of theintention deducer 240 ofFIG. 3 . - Referring to
FIG. 3 , inoperation 410 the semanticrole value allocator 320 determines whether at least one semantic role element in an intention frame selected by theintention frame selector 310 matches at least one semantic role element of a speech recognition candidate sentence that has undergone phrase spotting. As mentioned above, the speech recognition candidate sentence that has undergone phrase spotting is a sentence that has been determined to be grammatically valid. - In response to at least one semantic role element in the selected intention frame matching at least one semantic role element of a speech recognition candidate sentence that has undergone phrase spotting, in
operation 450 the semanticrole value allocator 320 may allocate phrases corresponding to respective semantic roles of the speech recognition candidate sentence that has undergone phrase spotting as the semantic role values of semantic role elements in the intention frame. - At this time, in response to words that do not match the semantic role elements of the intention frame being adjacent to a word corresponding to a semantic role in the speech recognition candidate sentence that has undergone phrase spotting, phrase chunking may be performed on the word together with the adjacent words using the
phrase chunking DB 153 that stores information for phrase chunking to determine the range of the semantic role values. Phrase chunking refers to a natural language process that segments a sentence into sub-parts, for example, a noun, a verb, a prepositional phrase, and the like. When a semantic role value is allocated, at least one intention analysis result candidate may be generated. An example of this process is described with reference toFIG. 10 . -
FIG. 10 illustrates an example of the allocation of a semantic role value in response to semantic role elements matching. - Referring to the example shown in
FIG. 10 , a speech recognition candidate sentence that has undergone phrase spotting is “I want to reserve a train ticket (@object) for Seoul (@destination)” and a selected intention frame is “MakeReservation(@destination, @object),” Accordingly, semantic role elements of the speech recognition candidate sentence that has undergone phrase spotting match those in the selected intention frame, that is, @destination and @object. Thus, by allocating the semantic role values of the semantic role elements in the speech recognition candidate sentence to the corresponding semantic role elements of the intention frame, an intention analysis result “MakeReservation(@destination=Seoul, @object=train ticket)” may be generated. - Referring back to
FIG. 4 , in response to it being determined inoperation 410 that at least one semantic role element in the selected intention frame does not match at least one semantic role element of a speech recognition candidate sentence that has undergone phrase spotting, inoperation 420 the semanticrole value allocator 320 determines whether a semantic role element that is not in the intention frame is in the sentence that has undergone phrase spotting. - In response to a semantic role element that is not in the intention frame being in the sentence that has undergone phrase spotting, in
operation 430 the semanticrole value allocator 320 may determine relationships between semantic roles with reference to a role network from therole network DB 155. In response to the semantic roles having a parent-child relationship in the role network, it may be determined that the semantic role is replaceable. In response to the semantic role being determined to be replaceable, inoperation 450 the semanticrole value allocator 320 may determine the range of a semantic role value through phrase chunking and allocate the semantic role value that belongs to the selected intention frame. - An example of this process is described with reference Such a case, in which a semantic role element of a speech recognition candidate sentence that has undergone phrase spotting using a role network can replace a semantic role element in an intention frame, may be useful when the number of semantic role elements of the speech recognition candidate sentence that has undergone phrase spotting match that of semantic role elements in the intention frame.
-
FIG. 11 illustrates an example of the allocation of a semantic role value in response to semantic role elements not matching. - When a phrase spotting result is “reserve a [train](@object) for [Kansas City](@region) at [three o'clock](@startTime),” and an intention frame is “MakeReservation(@object, @startTime, @destination),” the phrase spotting result has @region that is not in the intention frame. In this example, @region and @destination are in a parent-children relationship referring to a role network as shown in
FIG. 9 . Accordingly, @region and @destination may be replaced with each other. In response to the role values of the phrase spotting result being allocated to the corresponding semantic role elements of the intention frame, an intention analysis result “MakeReservation(@object=train, @startTime=three o'clock, @destination=Kansas City)” may be generated. - Referring back to
FIG. 4 , in response to it being determined inoperation 420 that a semantic role element that is not in the intention frame is also not in the speech recognition candidate sentence that has undergone phrase spotting, inoperation 440 the semanticrole value allocator 320 may estimate a semantic role value through phrase chunking using the ontology and may allocate the semantic role value. The estimation of the semantic role value may be performed in response to it being determined that there is a semantic role element in the intention frame but not in the phrase spotting result. - For example, in
operation 440 the semanticrole value allocator 320 may check the positions of words that are not matching the intention frame in the phrase spotting result, and may determine the range of semantic role values through phrase chunking and allocate the semantic role values in response to it being determined that the words are at positions that may have semantic role values in the sentence. - For example, the categories of words in the speech recognition candidate sentence that has undergone phrase spotting are compared with those of words corresponding to the semantic role elements of the intention frame. Semantic role values may be determined in response to the words in the speech recognition candidate sentence that has undergone phrase spotting and the words corresponding to the semantic role elements of the intention frame being in the same categories or in a parent-child relationship. Comparison of word categories may be performed using the ontology. Also, in response to a phrase being likely to be a proper noun, a semantic role value may be allocated without the category comparison process. An example of this process is described with reference to
FIG. 12 . -
FIG. 12 illustrates an example of the estimation of a semantic role value through phrase chunking. - In response to a phrase spotting result being “Record Lovers in Paris on Tuesday (@time)” and a selected intention frame is “GetEstablishTime(@time, @object),” the semantic role of “Lovers in Paris” in the phrase spotting result may not be determined even with reference to an ontology. In this example, the semantic
role value allocator 320 may determine “Lovers in Paris” as a proper noun and allocate “Lovers in Paris” to @object of the intention frame as a semantic role value. Thus, an intention analysis result “GetEstablishTime(@time=Tuesday, @object=Lovers in Paris)” may be generated. -
FIG. 13 illustrates an example of a method for analyzing intention. - In
operation 1310, thephrase spotter 220 performs phrase spotting on at least one sentence by applying the context-free grammar to the at least one sentence. - In
operation 1320, thevalid sentence determiner 230 determines whether the sentences are grammatically valid by applying the dependency grammar to the sentences that have undergone phrase spotting, and filters an invalid sentence. - In
operation 1330, theintention deducer 240 generates the intention analysis result of a sentence determined to be valid. For example, theintention deducer 240 may select an intention frame to be the intention analysis result of the sentence that has undergone phrase spotting, determine a semantic role value for a semantic role element included in the intention frame from the sentence that has undergone phrase spotting, and allocate the determined semantic role value to the semantic role element in the selected intention frame. - Thus far, an example in which the
apparatus 100 for analyzing an intention is used in a speech dialogue system has been described. However, theapparatus 100 for analyzing an intention can be applied not only to sentences that are recognized by speech recognition but also to general sentences that are not recognized by speech recognition, and employed in systems having various forms for a variety of purposes. - For example, even when an OOG expression is included in a sentence generated in a user's speech the intention of the speech may be analyzed. Also, a sentence that has undergone speech recognition is grammatically or semantically verified while a speech recognition range is extended by generating the intention analysis result of the grammatically valid sentence. Accordingly, it is possible to prevent a sentence causing a speech recognition error from being output as a speech recognition result. During intention analysis, an OOG expression can be processed to increase the degree of freedom of speech of a user, and the rate of success in intention analysis and the overall performance of a speech dialogue system can be increased in comparison with a conventional speech dialogue system that performs speech recognition using predetermined speech only.
- The processes, functions, methods and/or software described above may be recorded, stored, or fixed in one or more computer-readable storage media that includes program instructions to be implemented by a computer to cause a processor to execute or perform the program instructions. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. The media and program instructions may be those specially designed and constructed, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable media include magnetic media, such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks and DVDs; magneto-optical media, such as optical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. Examples of program instructions include machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations and methods described above, or vice versa. In addition, a computer-readable storage medium may be distributed among computer systems connected through a network and computer-readable codes or program instructions may be stored and executed in a decentralized manner.
- A computing system or a computer may include a microprocessor that is electrically connected with a bus, a user interface, and a memory controller. It may further include a flash memory device. The flash memory device may store N-bit data via the memory controller. The N-bit data is processed or will be processed by the microprocessor and N may be 1 or an integer greater than 1. Where the computing system or computer is a mobile apparatus, a battery may be additionally provided to supply operation voltage of the computing system or computer.
- It will be apparent to those of ordinary skill in the art that the computing system or computer may further include an application chipset, a camera image processor (CIS), a mobile Dynamic Random Access Memory (DRAM), and the like. The memory controller and the flash memory device may constitute a solid state drive/disk (SSD) that uses a non-volatile memory to store data.
- A number of examples have been described above. Nevertheless, it should be understood that various modifications may be made. For example, suitable results may be achieved if the described techniques are performed in a different order and/or if components in a described system, architecture, device, or circuit are combined in a different manner and/or replaced or supplemented by other components or their equivalents. Accordingly, other implementations are within the scope of the following claims.
Claims (17)
1. An apparatus for analyzing intention, the apparatus comprising:
a phrase spotter configured to perform phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases;
a valid sentence determiner configured to:
determine whether the at least one sentence is grammatically valid by applying a dependency grammar to the sentence that has undergone phrase spotting; and
filter an invalid sentence; and
an intention deducer configured to generate an intention analysis result of a sentence determined to be valid.
2. The apparatus of claim 1 , wherein the intention deducer is further configured to:
select an intention frame to be the intention analysis result of the sentence determined to be valid;
determine a semantic role value of at least one semantic role element included in the selected intention frame; and
allocate the determined semantic role value to the semantic role element included in the selected intention frame.
3. The apparatus of claim 2 , wherein, in response to the intention deducer allocating the semantic role value, the intention deducer is further configured to:
determine the semantic role value from the sentence determined to be valid through phrase chunking; and
allocate the determined semantic role value to the semantic role element in the selected intention frame if at least one semantic role element of the sentence determined to be valid matches at least one semantic role element in the selected intention frame.
4. The apparatus of claim 3 , wherein, in response to the sentence determined to be valid comprising a semantic role element other than the at least one semantic role element in the intention frame, the intention deducer is further configured to:
determine whether the other semantic intention role element can be replaced by the semantic role element in the intention frame using a role network;
determine a semantic role value of the semantic role element in the intention frame from the sentence determined to be valid through phrase chunking in response to it being determined that the other semantic intention role element can be replaced by the semantic role element in the intention frame; and
allocate the determined semantic role value to the semantic role element in the intention frame.
5. The apparatus of claim 3 , wherein the intention deducer is further configured to estimate the semantic role value of the at least one semantic role element in the intention frame using an ontology.
6. The apparatus of claim 2 , further comprising a scorer configured to:
calculate a probability that intention analysis has been correctly performed on at least one intention analysis result candidate to which the semantic role value of the semantic role element included in the selected intention frame is allocated; and
score the intention analysis result candidate.
7. The apparatus of claim 1 , further comprising an analysis applier configured to:
apply the intention analysis result to an application; and
generate an intention analysis application result.
8. The apparatus of claim 1 , further comprising a speech recognizer configured to convert an audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted by the speech recognizer.
9. A method of analyzing an intention, the method comprising:
performing phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases;
determining whether the at least one sentence is grammatically valid by:
applying a dependency grammar to the sentence that has undergone phrase spotting; and
filtering an invalid sentence; and
generating an intention analysis result of a sentence determined to be valid.
10. The method of claim 9 , wherein the generating of the intention analysis result of the sentence determined to be valid comprises:
selecting an intention frame to be the intention analysis result of the sentence determined to be valid;
determining semantic role values of semantic role elements included in the selected intention frame; and
allocating the determined semantic role values to the semantic role elements included in the selected intention frame.
11. The method of claim 10 , wherein the allocating of the semantic role values comprises:
determining whether at least one semantic role element of the sentence determined to be valid matches at least one semantic role element in the selected intention frame; and
in response to it being determined that the at least one semantic role element of the sentence determined to be valid matches the at least one semantic role element in the selected intention frame:
determining the semantic role values from the sentence determined to be valid through phrase chunking; and
allocating the determined semantic role values.
12. The method of claim 11 , wherein, in response to the semantic role element of the sentence determined to be valid not matching the semantic role element in the selected intention frame, the allocating of the semantic role values further comprises:
determining whether the sentence determined to be valid comprises a semantic role element other than the semantic role elements of the intention frame;
in response to the sentence determined to be valid comprising a semantic role element other than the semantic role elements of the intention frame, determining whether the other semantic role element can be replaced by the semantic role element in the intention frame using a role network; and
in response to it being determined that the other semantic role element can be replaced by the semantic role element in the intention frame:
determining the semantic role value of the semantic role element in the intention frame from the sentence determined to be valid through phrase chunking; and
allocating the determined semantic role value to the semantic role element in the intention frame.
13. The method of claim 11 , further comprising estimating the semantic role value of the at least one semantic role element in the intention frame using an ontology.
14. The method of claim 10 , further comprising:
calculating probabilities that intention analysis has been correctly performed on at least one intention analysis result candidate to which the semantic role value of the semantic role element in the selected intention frame is allocated; and
scoring the intention analysis result candidates.
15. The method of claim 9 , further comprising applying the intention analysis result to an application and generating an intention analysis application result.
16. The method of claim 9 , further comprising performing speech recognition on an audio input and converting the audio input into at least one sentence, the at least one sentence comprising an n-best sentence converted through the speech recognition.
17. A computer-readable storage medium storing a program that causes a computer to execute a method of analyzing an intention, comprising:
performing phrase spotting on at least one sentence by applying a context-free grammar to the at least one sentence in units of words or phrases;
determining whether the at least one sentence is grammatically valid by:
applying a dependency grammar to the sentence that has undergone phrase spotting; and
filtering an invalid sentence; and
generating an intention analysis result of a sentence determined to be valid.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020090094019A KR20110036385A (en) | 2009-10-01 | 2009-10-01 | Apparatus for analyzing intention of user and method thereof |
KR10-2009-0094019 | 2009-10-01 |
Publications (1)
Publication Number | Publication Date |
---|---|
US20110082688A1 true US20110082688A1 (en) | 2011-04-07 |
Family
ID=43823870
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/894,846 Abandoned US20110082688A1 (en) | 2009-10-01 | 2010-09-30 | Apparatus and Method for Analyzing Intention |
Country Status (2)
Country | Link |
---|---|
US (1) | US20110082688A1 (en) |
KR (1) | KR20110036385A (en) |
Cited By (262)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070186148A1 (en) * | 1999-08-13 | 2007-08-09 | Pixo, Inc. | Methods and apparatuses for display and traversing of links in page character array |
US20080248797A1 (en) * | 2007-04-03 | 2008-10-09 | Daniel Freeman | Method and System for Operating a Multi-Function Portable Electronic Device Using Voice-Activation |
US20090112647A1 (en) * | 2007-10-26 | 2009-04-30 | Christopher Volkert | Search Assistant for Digital Media Assets |
US20090225041A1 (en) * | 2008-03-04 | 2009-09-10 | Apple Inc. | Language input interface on a device |
US20100076767A1 (en) * | 2001-10-22 | 2010-03-25 | Braintexter, Inc. | Text to speech conversion of text messages from mobile communication devices |
US20100082348A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for text normalization for text to speech synthesis |
US20100082347A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for concatenation of words in text to speech synthesis |
US20100082328A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for speech preprocessing in text to speech synthesis |
US20100299138A1 (en) * | 2009-05-22 | 2010-11-25 | Kim Yeo Jin | Apparatus and method for language expression using context and intent awareness |
US20110010179A1 (en) * | 2009-07-13 | 2011-01-13 | Naik Devang K | Voice synthesis and processing |
US20110172994A1 (en) * | 2010-01-13 | 2011-07-14 | Apple Inc. | Processing of voice inputs |
US20120173244A1 (en) * | 2011-01-04 | 2012-07-05 | Kwak Byung-Kwan | Apparatus and method for voice command recognition based on a combination of dialog models |
US20120226492A1 (en) * | 2011-03-03 | 2012-09-06 | International Business Machines Corporation | Information processing apparatus, natural language analysis method, program and recording medium |
US8296383B2 (en) | 2008-10-02 | 2012-10-23 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US8352268B2 (en) | 2008-09-29 | 2013-01-08 | Apple Inc. | Systems and methods for selective rate of speech and speech preferences for text to speech synthesis |
US8352272B2 (en) | 2008-09-29 | 2013-01-08 | Apple Inc. | Systems and methods for text to speech synthesis |
US8359234B2 (en) | 2007-07-26 | 2013-01-22 | Braintexter, Inc. | System to generate and set up an advertising campaign based on the insertion of advertising messages within an exchange of messages, and method to operate said system |
US8380507B2 (en) | 2009-03-09 | 2013-02-19 | Apple Inc. | Systems and methods for determining the language to use for speech generated by a text to speech engine |
US8458278B2 (en) | 2003-05-02 | 2013-06-04 | Apple Inc. | Method and apparatus for displaying information during an instant messaging session |
US8543407B1 (en) | 2007-10-04 | 2013-09-24 | Great Northern Research, LLC | Speech interface system and method for control and interaction with applications on a computing system |
US8583418B2 (en) | 2008-09-29 | 2013-11-12 | Apple Inc. | Systems and methods of detecting language and natural language strings for text to speech synthesis |
US8600743B2 (en) | 2010-01-06 | 2013-12-03 | Apple Inc. | Noise profile determination for voice-related feature |
US8614431B2 (en) | 2005-09-30 | 2013-12-24 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US8620662B2 (en) | 2007-11-20 | 2013-12-31 | Apple Inc. | Context-aware unit selection |
US8639516B2 (en) | 2010-06-04 | 2014-01-28 | Apple Inc. | User-specific noise suppression for voice quality improvements |
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US8660849B2 (en) | 2010-01-18 | 2014-02-25 | Apple Inc. | Prioritizing selection criteria by automated assistant |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US8682649B2 (en) | 2009-11-12 | 2014-03-25 | Apple Inc. | Sentiment prediction from textual data |
US8688446B2 (en) | 2008-02-22 | 2014-04-01 | Apple Inc. | Providing text input using speech data and non-speech data |
US8706472B2 (en) | 2011-08-11 | 2014-04-22 | Apple Inc. | Method for disambiguating multiple readings in language conversion |
US8713021B2 (en) | 2010-07-07 | 2014-04-29 | Apple Inc. | Unsupervised document clustering using latent semantic density analysis |
US8712776B2 (en) | 2008-09-29 | 2014-04-29 | Apple Inc. | Systems and methods for selective text to speech synthesis |
US8719014B2 (en) | 2010-09-27 | 2014-05-06 | Apple Inc. | Electronic device with text error correction based on voice recognition data |
US8719006B2 (en) | 2010-08-27 | 2014-05-06 | Apple Inc. | Combined statistical and rule-based part-of-speech tagging for text-to-speech synthesis |
US8762156B2 (en) | 2011-09-28 | 2014-06-24 | Apple Inc. | Speech recognition repair using contextual information |
US8768702B2 (en) | 2008-09-05 | 2014-07-01 | Apple Inc. | Multi-tiered voice feedback in an electronic device |
US8775442B2 (en) | 2012-05-15 | 2014-07-08 | Apple Inc. | Semantic search using a single-source semantic model |
US8781836B2 (en) | 2011-02-22 | 2014-07-15 | Apple Inc. | Hearing assistance system for providing consistent human speech |
WO2014110281A1 (en) * | 2013-01-11 | 2014-07-17 | Db Networks, Inc. | Systems and methods for detecting and mitigating threats to a structured data storage system |
US8812294B2 (en) | 2011-06-21 | 2014-08-19 | Apple Inc. | Translating phrases from one language into another using an order-based set of declarative rules |
US8862252B2 (en) | 2009-01-30 | 2014-10-14 | Apple Inc. | Audio user interface for displayless electronic device |
US8898568B2 (en) | 2008-09-09 | 2014-11-25 | Apple Inc. | Audio user interface |
US20140365209A1 (en) * | 2013-06-09 | 2014-12-11 | Apple Inc. | System and method for inferring user intent from speech inputs |
US8935167B2 (en) | 2012-09-25 | 2015-01-13 | Apple Inc. | Exemplar-based latent perceptual modeling for automatic speech recognition |
US20150032441A1 (en) * | 2013-07-26 | 2015-01-29 | Nuance Communications, Inc. | Initializing a Workspace for Building a Natural Language Understanding System |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US9053089B2 (en) | 2007-10-02 | 2015-06-09 | Apple Inc. | Part-of-speech tagging using latent analogy |
US9069798B2 (en) | 2012-05-24 | 2015-06-30 | Mitsubishi Electric Research Laboratories, Inc. | Method of text classification using discriminative topic transformation |
US9100291B2 (en) | 2012-01-31 | 2015-08-04 | Db Networks, Inc. | Systems and methods for extracting structured application data from a communications link |
US9104670B2 (en) | 2010-07-21 | 2015-08-11 | Apple Inc. | Customized search or acquisition of digital media assets |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US9292488B2 (en) | 2014-02-01 | 2016-03-22 | Soundhound, Inc. | Method for embedding voice mail in a spoken utterance using a natural language processing computer system |
US9300784B2 (en) | 2013-06-13 | 2016-03-29 | Apple Inc. | System and method for emergency calls initiated by voice command |
US9311043B2 (en) | 2010-01-13 | 2016-04-12 | Apple Inc. | Adaptive audio feedback system and method |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US9330381B2 (en) | 2008-01-06 | 2016-05-03 | Apple Inc. | Portable multifunction device, method, and graphical user interface for viewing and managing electronic calendars |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
US20160188574A1 (en) * | 2014-12-25 | 2016-06-30 | Clarion Co., Ltd. | Intention estimation equipment and intention estimation system |
US9390167B2 (en) | 2010-07-29 | 2016-07-12 | Soundhound, Inc. | System and methods for continuous audio matching |
US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
US9507849B2 (en) | 2013-11-28 | 2016-11-29 | Soundhound, Inc. | Method for combining a query and a communication command in a natural language computer system |
US9519461B2 (en) | 2013-06-20 | 2016-12-13 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on third-party developers |
US9525642B2 (en) | 2012-01-31 | 2016-12-20 | Db Networks, Inc. | Ordering traffic captured on a data connection |
US9535906B2 (en) | 2008-07-31 | 2017-01-03 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
US9564123B1 (en) | 2014-05-12 | 2017-02-07 | Soundhound, Inc. | Method and system for building an integrated user profile |
US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
US9594542B2 (en) | 2013-06-20 | 2017-03-14 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on training by third-party developers |
US9620104B2 (en) | 2013-06-07 | 2017-04-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
US9633317B2 (en) | 2013-06-20 | 2017-04-25 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on a natural language intent interpreter |
US9633674B2 (en) | 2013-06-07 | 2017-04-25 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US9697822B1 (en) | 2013-03-15 | 2017-07-04 | Apple Inc. | System and method for updating an adaptive speech recognition model |
US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
CN107015964A (en) * | 2017-03-22 | 2017-08-04 | 北京光年无限科技有限公司 | The self-defined intention implementation method and device developed towards intelligent robot |
US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
US9733821B2 (en) | 2013-03-14 | 2017-08-15 | Apple Inc. | Voice control to diagnose inadvertent activation of accessibility features |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US9798393B2 (en) | 2011-08-29 | 2017-10-24 | Apple Inc. | Text correction processing |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9922642B2 (en) | 2013-03-15 | 2018-03-20 | Apple Inc. | Training an at least partial voice command system |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9946706B2 (en) | 2008-06-07 | 2018-04-17 | Apple Inc. | Automatic language identification for dynamic text processing |
US9959870B2 (en) | 2008-12-11 | 2018-05-01 | Apple Inc. | Speech recognition involving a mobile device |
US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
US9966068B2 (en) | 2013-06-08 | 2018-05-08 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US9977779B2 (en) | 2013-03-14 | 2018-05-22 | Apple Inc. | Automatic supplementation of word correction dictionaries |
US20180165135A1 (en) * | 2016-12-09 | 2018-06-14 | Fujitsu Limited | Api learning |
US10002189B2 (en) | 2007-12-20 | 2018-06-19 | Apple Inc. | Method and apparatus for searching using an active ontology |
US10019994B2 (en) | 2012-06-08 | 2018-07-10 | Apple Inc. | Systems and methods for recognizing textual identifiers within a plurality of words |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US10078487B2 (en) | 2013-03-15 | 2018-09-18 | Apple Inc. | Context-sensitive handling of interruptions |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10089072B2 (en) | 2016-06-11 | 2018-10-02 | Apple Inc. | Intelligent device arbitration and control |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US10121165B1 (en) | 2011-05-10 | 2018-11-06 | Soundhound, Inc. | System and method for targeting content based on identified audio and multimedia |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10185542B2 (en) | 2013-06-09 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US10199051B2 (en) | 2013-02-07 | 2019-02-05 | Apple Inc. | Voice trigger for a digital assistant |
CN109388802A (en) * | 2018-10-11 | 2019-02-26 | 北京轮子科技有限公司 | A kind of semantic understanding method and apparatus based on deep learning |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US10255566B2 (en) | 2011-06-03 | 2019-04-09 | Apple Inc. | Generating and processing task items that represent tasks to perform |
US10269345B2 (en) | 2016-06-11 | 2019-04-23 | Apple Inc. | Intelligent task discovery |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
US10297253B2 (en) | 2016-06-11 | 2019-05-21 | Apple Inc. | Application integration with a digital assistant |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
US10303715B2 (en) | 2017-05-16 | 2019-05-28 | Apple Inc. | Intelligent automated assistant for media exploration |
US10311144B2 (en) | 2017-05-16 | 2019-06-04 | Apple Inc. | Emoji word sense disambiguation |
US10332518B2 (en) | 2017-05-09 | 2019-06-25 | Apple Inc. | User interface for correcting recognition errors |
US10354011B2 (en) | 2016-06-09 | 2019-07-16 | Apple Inc. | Intelligent automated assistant in a home environment |
US10356243B2 (en) | 2015-06-05 | 2019-07-16 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
CN110096570A (en) * | 2019-04-09 | 2019-08-06 | 苏宁易购集团股份有限公司 | A kind of intension recognizing method and device applied to intelligent customer service robot |
US10395654B2 (en) | 2017-05-11 | 2019-08-27 | Apple Inc. | Text normalization based on a data-driven learning network |
US10403283B1 (en) | 2018-06-01 | 2019-09-03 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
US10403278B2 (en) | 2017-05-16 | 2019-09-03 | Apple Inc. | Methods and systems for phonetic matching in digital assistant services |
US10410637B2 (en) | 2017-05-12 | 2019-09-10 | Apple Inc. | User-specific acoustic models |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US10417266B2 (en) | 2017-05-09 | 2019-09-17 | Apple Inc. | Context-aware ranking of intelligent response suggestions |
US10445429B2 (en) | 2017-09-21 | 2019-10-15 | Apple Inc. | Natural language understanding using vocabularies with compressed serialized tries |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US10474753B2 (en) | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
US10474961B2 (en) | 2013-06-20 | 2019-11-12 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on prompting for additional user input |
US10482874B2 (en) | 2017-05-15 | 2019-11-19 | Apple Inc. | Hierarchical belief states for digital assistants |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10496705B1 (en) | 2018-06-03 | 2019-12-03 | Apple Inc. | Accelerated task performance |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10515147B2 (en) | 2010-12-22 | 2019-12-24 | Apple Inc. | Using statistical language models for contextual lookup |
US10521466B2 (en) | 2016-06-11 | 2019-12-31 | Apple Inc. | Data driven natural language event detection and classification |
US10540976B2 (en) | 2009-06-05 | 2020-01-21 | Apple Inc. | Contextual voice commands |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US10572476B2 (en) | 2013-03-14 | 2020-02-25 | Apple Inc. | Refining a search based on schedule items |
US10592604B2 (en) | 2018-03-12 | 2020-03-17 | Apple Inc. | Inverse text normalization for automatic speech recognition |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
US10636424B2 (en) | 2017-11-30 | 2020-04-28 | Apple Inc. | Multi-turn canned dialog |
US10642574B2 (en) | 2013-03-14 | 2020-05-05 | Apple Inc. | Device, method, and graphical user interface for outputting captions |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US10657328B2 (en) | 2017-06-02 | 2020-05-19 | Apple Inc. | Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10672399B2 (en) | 2011-06-03 | 2020-06-02 | Apple Inc. | Switching between text data and audio data based on a mapping |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10684703B2 (en) | 2018-06-01 | 2020-06-16 | Apple Inc. | Attention aware virtual assistant dismissal |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10726832B2 (en) | 2017-05-11 | 2020-07-28 | Apple Inc. | Maintaining privacy of personal information |
US10733375B2 (en) | 2018-01-31 | 2020-08-04 | Apple Inc. | Knowledge-based framework for improving natural language understanding |
US10733982B2 (en) | 2018-01-08 | 2020-08-04 | Apple Inc. | Multi-directional dialog |
US10733993B2 (en) | 2016-06-10 | 2020-08-04 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10748529B1 (en) | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10748546B2 (en) | 2017-05-16 | 2020-08-18 | Apple Inc. | Digital assistant services based on device capabilities |
US10755051B2 (en) | 2017-09-29 | 2020-08-25 | Apple Inc. | Rule-based natural language processing |
US10755703B2 (en) | 2017-05-11 | 2020-08-25 | Apple Inc. | Offline personal assistant |
US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
US10791176B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Synchronization and task delegation of a digital assistant |
US10791216B2 (en) | 2013-08-06 | 2020-09-29 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
US10789945B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Low-latency intelligent automated assistant |
US10789959B2 (en) | 2018-03-02 | 2020-09-29 | Apple Inc. | Training speaker recognition models for digital assistants |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US10810274B2 (en) | 2017-05-15 | 2020-10-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US10818288B2 (en) | 2018-03-26 | 2020-10-27 | Apple Inc. | Natural assistant interaction |
US10839159B2 (en) | 2018-09-28 | 2020-11-17 | Apple Inc. | Named entity normalization in a spoken dialog system |
US10892996B2 (en) | 2018-06-01 | 2021-01-12 | Apple Inc. | Variable latency device coordination |
US10909331B2 (en) | 2018-03-30 | 2021-02-02 | Apple Inc. | Implicit identification of translation payload with neural machine translation |
US10928918B2 (en) | 2018-05-07 | 2021-02-23 | Apple Inc. | Raise to speak |
US10957310B1 (en) * | 2012-07-23 | 2021-03-23 | Soundhound, Inc. | Integrated programming framework for speech and text understanding with meaning parsing |
US10984780B2 (en) | 2018-05-21 | 2021-04-20 | Apple Inc. | Global semantic word embeddings using bi-directional recurrent neural networks |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US11010127B2 (en) | 2015-06-29 | 2021-05-18 | Apple Inc. | Virtual assistant for media playback |
US11010561B2 (en) | 2018-09-27 | 2021-05-18 | Apple Inc. | Sentiment prediction from textual data |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US11070949B2 (en) | 2015-05-27 | 2021-07-20 | Apple Inc. | Systems and methods for proactively identifying and surfacing relevant content on an electronic device with a touch-sensitive display |
US11140099B2 (en) | 2019-05-21 | 2021-10-05 | Apple Inc. | Providing message response suggestions |
US11145294B2 (en) | 2018-05-07 | 2021-10-12 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US11151899B2 (en) | 2013-03-15 | 2021-10-19 | Apple Inc. | User training by intelligent digital assistant |
US11170166B2 (en) | 2018-09-28 | 2021-11-09 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
US11204787B2 (en) | 2017-01-09 | 2021-12-21 | Apple Inc. | Application integration with a digital assistant |
US11217251B2 (en) | 2019-05-06 | 2022-01-04 | Apple Inc. | Spoken notifications |
US11227589B2 (en) | 2016-06-06 | 2022-01-18 | Apple Inc. | Intelligent list reading |
US11231904B2 (en) | 2015-03-06 | 2022-01-25 | Apple Inc. | Reducing response latency of intelligent automated assistants |
US11237797B2 (en) | 2019-05-31 | 2022-02-01 | Apple Inc. | User activity shortcut suggestions |
US11281993B2 (en) | 2016-12-05 | 2022-03-22 | Apple Inc. | Model and ensemble compression for metric learning |
US11289073B2 (en) | 2019-05-31 | 2022-03-29 | Apple Inc. | Device text to speech |
US11295730B1 (en) | 2014-02-27 | 2022-04-05 | Soundhound, Inc. | Using phonetic variants in a local context to improve natural language understanding |
US11301477B2 (en) | 2017-05-12 | 2022-04-12 | Apple Inc. | Feedback analysis of a digital assistant |
US11307752B2 (en) | 2019-05-06 | 2022-04-19 | Apple Inc. | User configurable task triggers |
US11348573B2 (en) | 2019-03-18 | 2022-05-31 | Apple Inc. | Multimodality in digital assistant systems |
US11360641B2 (en) | 2019-06-01 | 2022-06-14 | Apple Inc. | Increasing the relevance of new available information |
US11386266B2 (en) | 2018-06-01 | 2022-07-12 | Apple Inc. | Text correction |
US11423908B2 (en) | 2019-05-06 | 2022-08-23 | Apple Inc. | Interpreting spoken requests |
US11462215B2 (en) | 2018-09-28 | 2022-10-04 | Apple Inc. | Multi-modal inputs for voice commands |
US11468282B2 (en) | 2015-05-15 | 2022-10-11 | Apple Inc. | Virtual assistant in a communication session |
US11475884B2 (en) | 2019-05-06 | 2022-10-18 | Apple Inc. | Reducing digital assistant latency when a language is incorrectly determined |
US11475898B2 (en) | 2018-10-26 | 2022-10-18 | Apple Inc. | Low-latency multi-speaker speech recognition |
US11488406B2 (en) | 2019-09-25 | 2022-11-01 | Apple Inc. | Text detection using global geometry estimators |
US11496600B2 (en) | 2019-05-31 | 2022-11-08 | Apple Inc. | Remote execution of machine-learned models |
US11495218B2 (en) | 2018-06-01 | 2022-11-08 | Apple Inc. | Virtual assistant operation in multi-device environments |
US11532306B2 (en) | 2017-05-16 | 2022-12-20 | Apple Inc. | Detecting a trigger of a digital assistant |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US11599332B1 (en) | 2007-10-04 | 2023-03-07 | Great Northern Research, LLC | Multiple shell multi faceted graphical user interface |
US11610065B2 (en) | 2020-06-12 | 2023-03-21 | Apple Inc. | Providing personalized responses based on semantic context |
US11638059B2 (en) | 2019-01-04 | 2023-04-25 | Apple Inc. | Content playback on multiple devices |
US11657813B2 (en) | 2019-05-31 | 2023-05-23 | Apple Inc. | Voice identification in digital assistant systems |
US11677875B2 (en) | 2021-07-02 | 2023-06-13 | Talkdesk Inc. | Method and apparatus for automated quality management of communication records |
US11736616B1 (en) | 2022-05-27 | 2023-08-22 | Talkdesk, Inc. | Method and apparatus for automatically taking action based on the content of call center communications |
US11736615B2 (en) | 2020-01-16 | 2023-08-22 | Talkdesk, Inc. | Method, apparatus, and computer-readable medium for managing concurrent communications in a networked call center |
US11755276B2 (en) | 2020-05-12 | 2023-09-12 | Apple Inc. | Reducing description length based on confidence |
US11765209B2 (en) | 2020-05-11 | 2023-09-19 | Apple Inc. | Digital assistant hardware abstraction |
US11783246B2 (en) | 2019-10-16 | 2023-10-10 | Talkdesk, Inc. | Systems and methods for workforce management system deployment |
US11809483B2 (en) | 2015-09-08 | 2023-11-07 | Apple Inc. | Intelligent automated assistant for media search and playback |
US11853536B2 (en) | 2015-09-08 | 2023-12-26 | Apple Inc. | Intelligent automated assistant in a media environment |
US11856140B2 (en) | 2022-03-07 | 2023-12-26 | Talkdesk, Inc. | Predictive communications system |
US11886805B2 (en) | 2015-11-09 | 2024-01-30 | Apple Inc. | Unconventional virtual assistant interactions |
US11943391B1 (en) | 2022-12-13 | 2024-03-26 | Talkdesk, Inc. | Method and apparatus for routing communications within a contact center |
US11971908B2 (en) | 2022-06-17 | 2024-04-30 | Talkdesk, Inc. | Method and apparatus for detecting anomalies in communication data |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108447471B (en) * | 2017-02-15 | 2021-09-10 | 腾讯科技(深圳)有限公司 | Speech recognition method and speech recognition device |
KR102159220B1 (en) * | 2017-05-11 | 2020-09-23 | 경희대학교 산학협력단 | Method for intent-context fusioning in healthcare systems for effective dialogue management |
Citations (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20010020837A1 (en) * | 1999-12-28 | 2001-09-13 | Junichi Yamashita | Information processing device, information processing method and storage medium |
US20030182131A1 (en) * | 2002-03-25 | 2003-09-25 | Arnold James F. | Method and apparatus for providing speech-driven routing between spoken language applications |
US20040243419A1 (en) * | 2003-05-29 | 2004-12-02 | Microsoft Corporation | Semantic object synchronous understanding for highly interactive interface |
US6895377B2 (en) * | 2000-03-24 | 2005-05-17 | Eliza Corporation | Phonetic data processing system and method |
US7146381B1 (en) * | 1997-02-10 | 2006-12-05 | Actioneer, Inc. | Information organization and collaboration tool for processing notes and action requests in computer systems |
US7200559B2 (en) * | 2003-05-29 | 2007-04-03 | Microsoft Corporation | Semantic object synchronous understanding implemented with speech application language tags |
US20070239454A1 (en) * | 2006-04-06 | 2007-10-11 | Microsoft Corporation | Personalizing a context-free grammar using a dictation language model |
US20070239453A1 (en) * | 2006-04-06 | 2007-10-11 | Microsoft Corporation | Augmenting context-free grammars with back-off grammars for processing out-of-grammar utterances |
US7289950B2 (en) * | 2000-09-29 | 2007-10-30 | Apple Inc. | Extended finite state grammar for speech recognition systems |
US7412387B2 (en) * | 2005-01-18 | 2008-08-12 | International Business Machines Corporation | Automatic improvement of spoken language |
US20080270135A1 (en) * | 2007-04-30 | 2008-10-30 | International Business Machines Corporation | Method and system for using a statistical language model and an action classifier in parallel with grammar for better handling of out-of-grammar utterances |
US7460996B2 (en) * | 2005-06-23 | 2008-12-02 | Microsoft Corporation | Using strong data types to express speech recognition grammars in software programs |
US20090076798A1 (en) * | 2007-09-19 | 2009-03-19 | Electronics And Telecommunications Research Institute | Apparatus and method for post-processing dialogue error in speech dialogue system using multilevel verification |
US20090235253A1 (en) * | 2008-03-12 | 2009-09-17 | Apple Inc. | Smart task list/life event annotator |
US7734461B2 (en) * | 2006-03-03 | 2010-06-08 | Samsung Electronics Co., Ltd | Apparatus for providing voice dialogue service and method of operating the same |
US20110218954A1 (en) * | 2005-12-12 | 2011-09-08 | Qin Zhang | Thinking system and method |
-
2009
- 2009-10-01 KR KR1020090094019A patent/KR20110036385A/en not_active Application Discontinuation
-
2010
- 2010-09-30 US US12/894,846 patent/US20110082688A1/en not_active Abandoned
Patent Citations (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7146381B1 (en) * | 1997-02-10 | 2006-12-05 | Actioneer, Inc. | Information organization and collaboration tool for processing notes and action requests in computer systems |
US20010020837A1 (en) * | 1999-12-28 | 2001-09-13 | Junichi Yamashita | Information processing device, information processing method and storage medium |
US6895377B2 (en) * | 2000-03-24 | 2005-05-17 | Eliza Corporation | Phonetic data processing system and method |
US7289950B2 (en) * | 2000-09-29 | 2007-10-30 | Apple Inc. | Extended finite state grammar for speech recognition systems |
US20030182131A1 (en) * | 2002-03-25 | 2003-09-25 | Arnold James F. | Method and apparatus for providing speech-driven routing between spoken language applications |
US20040243419A1 (en) * | 2003-05-29 | 2004-12-02 | Microsoft Corporation | Semantic object synchronous understanding for highly interactive interface |
US7200559B2 (en) * | 2003-05-29 | 2007-04-03 | Microsoft Corporation | Semantic object synchronous understanding implemented with speech application language tags |
US7412387B2 (en) * | 2005-01-18 | 2008-08-12 | International Business Machines Corporation | Automatic improvement of spoken language |
US7460996B2 (en) * | 2005-06-23 | 2008-12-02 | Microsoft Corporation | Using strong data types to express speech recognition grammars in software programs |
US20110218954A1 (en) * | 2005-12-12 | 2011-09-08 | Qin Zhang | Thinking system and method |
US7734461B2 (en) * | 2006-03-03 | 2010-06-08 | Samsung Electronics Co., Ltd | Apparatus for providing voice dialogue service and method of operating the same |
US20070239454A1 (en) * | 2006-04-06 | 2007-10-11 | Microsoft Corporation | Personalizing a context-free grammar using a dictation language model |
US20070239453A1 (en) * | 2006-04-06 | 2007-10-11 | Microsoft Corporation | Augmenting context-free grammars with back-off grammars for processing out-of-grammar utterances |
US20080270135A1 (en) * | 2007-04-30 | 2008-10-30 | International Business Machines Corporation | Method and system for using a statistical language model and an action classifier in parallel with grammar for better handling of out-of-grammar utterances |
US20090076798A1 (en) * | 2007-09-19 | 2009-03-19 | Electronics And Telecommunications Research Institute | Apparatus and method for post-processing dialogue error in speech dialogue system using multilevel verification |
US20090235253A1 (en) * | 2008-03-12 | 2009-09-17 | Apple Inc. | Smart task list/life event annotator |
Cited By (442)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8527861B2 (en) | 1999-08-13 | 2013-09-03 | Apple Inc. | Methods and apparatuses for display and traversing of links in page character array |
US20070186148A1 (en) * | 1999-08-13 | 2007-08-09 | Pixo, Inc. | Methods and apparatuses for display and traversing of links in page character array |
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US9646614B2 (en) | 2000-03-16 | 2017-05-09 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US8718047B2 (en) | 2001-10-22 | 2014-05-06 | Apple Inc. | Text to speech conversion of text messages from mobile communication devices |
US20100076767A1 (en) * | 2001-10-22 | 2010-03-25 | Braintexter, Inc. | Text to speech conversion of text messages from mobile communication devices |
US8345665B2 (en) | 2001-10-22 | 2013-01-01 | Apple Inc. | Text to speech conversion of text messages from mobile communication devices |
US8458278B2 (en) | 2003-05-02 | 2013-06-04 | Apple Inc. | Method and apparatus for displaying information during an instant messaging session |
US10623347B2 (en) | 2003-05-02 | 2020-04-14 | Apple Inc. | Method and apparatus for displaying information during an instant messaging session |
US10348654B2 (en) | 2003-05-02 | 2019-07-09 | Apple Inc. | Method and apparatus for displaying information during an instant messaging session |
US9501741B2 (en) | 2005-09-08 | 2016-11-22 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US10318871B2 (en) | 2005-09-08 | 2019-06-11 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US11928604B2 (en) | 2005-09-08 | 2024-03-12 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US8614431B2 (en) | 2005-09-30 | 2013-12-24 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US9389729B2 (en) | 2005-09-30 | 2016-07-12 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US9958987B2 (en) | 2005-09-30 | 2018-05-01 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US9619079B2 (en) | 2005-09-30 | 2017-04-11 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US8930191B2 (en) | 2006-09-08 | 2015-01-06 | Apple Inc. | Paraphrasing of user requests and results by automated digital assistant |
US9117447B2 (en) | 2006-09-08 | 2015-08-25 | Apple Inc. | Using event alert text as input to an automated assistant |
US8942986B2 (en) | 2006-09-08 | 2015-01-27 | Apple Inc. | Determining user intent based on ontologies of domains |
US11671920B2 (en) | 2007-04-03 | 2023-06-06 | Apple Inc. | Method and system for operating a multifunction portable electronic device using voice-activation |
US11012942B2 (en) | 2007-04-03 | 2021-05-18 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US20080248797A1 (en) * | 2007-04-03 | 2008-10-09 | Daniel Freeman | Method and System for Operating a Multi-Function Portable Electronic Device Using Voice-Activation |
US10568032B2 (en) | 2007-04-03 | 2020-02-18 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US8909545B2 (en) | 2007-07-26 | 2014-12-09 | Braintexter, Inc. | System to generate and set up an advertising campaign based on the insertion of advertising messages within an exchange of messages, and method to operate said system |
US8359234B2 (en) | 2007-07-26 | 2013-01-22 | Braintexter, Inc. | System to generate and set up an advertising campaign based on the insertion of advertising messages within an exchange of messages, and method to operate said system |
US9053089B2 (en) | 2007-10-02 | 2015-06-09 | Apple Inc. | Part-of-speech tagging using latent analogy |
US11599332B1 (en) | 2007-10-04 | 2023-03-07 | Great Northern Research, LLC | Multiple shell multi faceted graphical user interface |
US8543407B1 (en) | 2007-10-04 | 2013-09-24 | Great Northern Research, LLC | Speech interface system and method for control and interaction with applications on a computing system |
US9305101B2 (en) | 2007-10-26 | 2016-04-05 | Apple Inc. | Search assistant for digital media assets |
US8639716B2 (en) | 2007-10-26 | 2014-01-28 | Apple Inc. | Search assistant for digital media assets |
US20090112647A1 (en) * | 2007-10-26 | 2009-04-30 | Christopher Volkert | Search Assistant for Digital Media Assets |
US8364694B2 (en) | 2007-10-26 | 2013-01-29 | Apple Inc. | Search assistant for digital media assets |
US8943089B2 (en) | 2007-10-26 | 2015-01-27 | Apple Inc. | Search assistant for digital media assets |
US8620662B2 (en) | 2007-11-20 | 2013-12-31 | Apple Inc. | Context-aware unit selection |
US10002189B2 (en) | 2007-12-20 | 2018-06-19 | Apple Inc. | Method and apparatus for searching using an active ontology |
US11023513B2 (en) | 2007-12-20 | 2021-06-01 | Apple Inc. | Method and apparatus for searching using an active ontology |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US10381016B2 (en) | 2008-01-03 | 2019-08-13 | Apple Inc. | Methods and apparatus for altering audio output signals |
US10503366B2 (en) | 2008-01-06 | 2019-12-10 | Apple Inc. | Portable multifunction device, method, and graphical user interface for viewing and managing electronic calendars |
US9330381B2 (en) | 2008-01-06 | 2016-05-03 | Apple Inc. | Portable multifunction device, method, and graphical user interface for viewing and managing electronic calendars |
US11126326B2 (en) | 2008-01-06 | 2021-09-21 | Apple Inc. | Portable multifunction device, method, and graphical user interface for viewing and managing electronic calendars |
US9361886B2 (en) | 2008-02-22 | 2016-06-07 | Apple Inc. | Providing text input using speech data and non-speech data |
US8688446B2 (en) | 2008-02-22 | 2014-04-01 | Apple Inc. | Providing text input using speech data and non-speech data |
US8289283B2 (en) | 2008-03-04 | 2012-10-16 | Apple Inc. | Language input interface on a device |
US20090225041A1 (en) * | 2008-03-04 | 2009-09-10 | Apple Inc. | Language input interface on a device |
USRE46139E1 (en) | 2008-03-04 | 2016-09-06 | Apple Inc. | Language input interface on a device |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US9865248B2 (en) | 2008-04-05 | 2018-01-09 | Apple Inc. | Intelligent text-to-speech conversion |
US9626955B2 (en) | 2008-04-05 | 2017-04-18 | Apple Inc. | Intelligent text-to-speech conversion |
US9946706B2 (en) | 2008-06-07 | 2018-04-17 | Apple Inc. | Automatic language identification for dynamic text processing |
US10108612B2 (en) | 2008-07-31 | 2018-10-23 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
US9535906B2 (en) | 2008-07-31 | 2017-01-03 | Apple Inc. | Mobile device having human language translation capability with positional feedback |
US9691383B2 (en) | 2008-09-05 | 2017-06-27 | Apple Inc. | Multi-tiered voice feedback in an electronic device |
US8768702B2 (en) | 2008-09-05 | 2014-07-01 | Apple Inc. | Multi-tiered voice feedback in an electronic device |
US8898568B2 (en) | 2008-09-09 | 2014-11-25 | Apple Inc. | Audio user interface |
US8583418B2 (en) | 2008-09-29 | 2013-11-12 | Apple Inc. | Systems and methods of detecting language and natural language strings for text to speech synthesis |
US20100082328A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for speech preprocessing in text to speech synthesis |
US20100082348A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for text normalization for text to speech synthesis |
US8355919B2 (en) | 2008-09-29 | 2013-01-15 | Apple Inc. | Systems and methods for text normalization for text to speech synthesis |
US8712776B2 (en) | 2008-09-29 | 2014-04-29 | Apple Inc. | Systems and methods for selective text to speech synthesis |
US20100082347A1 (en) * | 2008-09-29 | 2010-04-01 | Apple Inc. | Systems and methods for concatenation of words in text to speech synthesis |
US8396714B2 (en) | 2008-09-29 | 2013-03-12 | Apple Inc. | Systems and methods for concatenation of words in text to speech synthesis |
US8352272B2 (en) | 2008-09-29 | 2013-01-08 | Apple Inc. | Systems and methods for text to speech synthesis |
US8352268B2 (en) | 2008-09-29 | 2013-01-08 | Apple Inc. | Systems and methods for selective rate of speech and speech preferences for text to speech synthesis |
US8762469B2 (en) | 2008-10-02 | 2014-06-24 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US8296383B2 (en) | 2008-10-02 | 2012-10-23 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US8713119B2 (en) | 2008-10-02 | 2014-04-29 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US10643611B2 (en) | 2008-10-02 | 2020-05-05 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US9412392B2 (en) | 2008-10-02 | 2016-08-09 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US11348582B2 (en) | 2008-10-02 | 2022-05-31 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US8676904B2 (en) | 2008-10-02 | 2014-03-18 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US9959870B2 (en) | 2008-12-11 | 2018-05-01 | Apple Inc. | Speech recognition involving a mobile device |
US8862252B2 (en) | 2009-01-30 | 2014-10-14 | Apple Inc. | Audio user interface for displayless electronic device |
US8751238B2 (en) | 2009-03-09 | 2014-06-10 | Apple Inc. | Systems and methods for determining the language to use for speech generated by a text to speech engine |
US8380507B2 (en) | 2009-03-09 | 2013-02-19 | Apple Inc. | Systems and methods for determining the language to use for speech generated by a text to speech engine |
US8560301B2 (en) * | 2009-05-22 | 2013-10-15 | Samsung Electronics Co., Ltd. | Apparatus and method for language expression using context and intent awareness |
US20100299138A1 (en) * | 2009-05-22 | 2010-11-25 | Kim Yeo Jin | Apparatus and method for language expression using context and intent awareness |
US10795541B2 (en) | 2009-06-05 | 2020-10-06 | Apple Inc. | Intelligent organization of tasks items |
US10475446B2 (en) | 2009-06-05 | 2019-11-12 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US10540976B2 (en) | 2009-06-05 | 2020-01-21 | Apple Inc. | Contextual voice commands |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US11080012B2 (en) | 2009-06-05 | 2021-08-03 | Apple Inc. | Interface for a virtual digital assistant |
US10283110B2 (en) | 2009-07-02 | 2019-05-07 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US20110010179A1 (en) * | 2009-07-13 | 2011-01-13 | Naik Devang K | Voice synthesis and processing |
US8682649B2 (en) | 2009-11-12 | 2014-03-25 | Apple Inc. | Sentiment prediction from textual data |
US8600743B2 (en) | 2010-01-06 | 2013-12-03 | Apple Inc. | Noise profile determination for voice-related feature |
US8670985B2 (en) | 2010-01-13 | 2014-03-11 | Apple Inc. | Devices and methods for identifying a prompt corresponding to a voice input in a sequence of prompts |
US9311043B2 (en) | 2010-01-13 | 2016-04-12 | Apple Inc. | Adaptive audio feedback system and method |
US20110172994A1 (en) * | 2010-01-13 | 2011-07-14 | Apple Inc. | Processing of voice inputs |
US8311838B2 (en) | 2010-01-13 | 2012-11-13 | Apple Inc. | Devices and methods for identifying a prompt corresponding to a voice input in a sequence of prompts |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US9548050B2 (en) | 2010-01-18 | 2017-01-17 | Apple Inc. | Intelligent automated assistant |
US8903716B2 (en) | 2010-01-18 | 2014-12-02 | Apple Inc. | Personalized vocabulary for digital assistant |
US8892446B2 (en) | 2010-01-18 | 2014-11-18 | Apple Inc. | Service orchestration for intelligent automated assistant |
US8670979B2 (en) | 2010-01-18 | 2014-03-11 | Apple Inc. | Active input elicitation by intelligent automated assistant |
US10741185B2 (en) | 2010-01-18 | 2020-08-11 | Apple Inc. | Intelligent automated assistant |
US8799000B2 (en) | 2010-01-18 | 2014-08-05 | Apple Inc. | Disambiguation based on active input elicitation by intelligent automated assistant |
US10706841B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Task flow identification based on user intent |
US8660849B2 (en) | 2010-01-18 | 2014-02-25 | Apple Inc. | Prioritizing selection criteria by automated assistant |
US8706503B2 (en) | 2010-01-18 | 2014-04-22 | Apple Inc. | Intent deduction based on previous user interactions with voice assistant |
US8731942B2 (en) | 2010-01-18 | 2014-05-20 | Apple Inc. | Maintaining context information between user interactions with a voice assistant |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
US11423886B2 (en) | 2010-01-18 | 2022-08-23 | Apple Inc. | Task flow identification based on user intent |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10692504B2 (en) | 2010-02-25 | 2020-06-23 | Apple Inc. | User profiling for voice input processing |
US10049675B2 (en) | 2010-02-25 | 2018-08-14 | Apple Inc. | User profiling for voice input processing |
US9190062B2 (en) | 2010-02-25 | 2015-11-17 | Apple Inc. | User profiling for voice input processing |
US9633660B2 (en) | 2010-02-25 | 2017-04-25 | Apple Inc. | User profiling for voice input processing |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US8639516B2 (en) | 2010-06-04 | 2014-01-28 | Apple Inc. | User-specific noise suppression for voice quality improvements |
US10446167B2 (en) | 2010-06-04 | 2019-10-15 | Apple Inc. | User-specific noise suppression for voice quality improvements |
US8713021B2 (en) | 2010-07-07 | 2014-04-29 | Apple Inc. | Unsupervised document clustering using latent semantic density analysis |
US9104670B2 (en) | 2010-07-21 | 2015-08-11 | Apple Inc. | Customized search or acquisition of digital media assets |
US10657174B2 (en) | 2010-07-29 | 2020-05-19 | Soundhound, Inc. | Systems and methods for providing identification information in response to an audio segment |
US10055490B2 (en) | 2010-07-29 | 2018-08-21 | Soundhound, Inc. | System and methods for continuous audio matching |
US9390167B2 (en) | 2010-07-29 | 2016-07-12 | Soundhound, Inc. | System and methods for continuous audio matching |
US8719006B2 (en) | 2010-08-27 | 2014-05-06 | Apple Inc. | Combined statistical and rule-based part-of-speech tagging for text-to-speech synthesis |
US9075783B2 (en) | 2010-09-27 | 2015-07-07 | Apple Inc. | Electronic device with text error correction based on voice recognition data |
US8719014B2 (en) | 2010-09-27 | 2014-05-06 | Apple Inc. | Electronic device with text error correction based on voice recognition data |
US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
US10515147B2 (en) | 2010-12-22 | 2019-12-24 | Apple Inc. | Using statistical language models for contextual lookup |
US8954326B2 (en) * | 2011-01-04 | 2015-02-10 | Samsung Electronics Co., Ltd. | Apparatus and method for voice command recognition based on a combination of dialog models |
US20120173244A1 (en) * | 2011-01-04 | 2012-07-05 | Kwak Byung-Kwan | Apparatus and method for voice command recognition based on a combination of dialog models |
US8781836B2 (en) | 2011-02-22 | 2014-07-15 | Apple Inc. | Hearing assistance system for providing consistent human speech |
US8793121B2 (en) * | 2011-03-03 | 2014-07-29 | International Business Machines Corporation | Information processing apparatus, natural language analysis method, program and recording medium |
US20130060562A1 (en) * | 2011-03-03 | 2013-03-07 | International Business Machines Corporation | Information processing appratus, natural language analysis method, program and recording medium |
US20120226492A1 (en) * | 2011-03-03 | 2012-09-06 | International Business Machines Corporation | Information processing apparatus, natural language analysis method, program and recording medium |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US10417405B2 (en) | 2011-03-21 | 2019-09-17 | Apple Inc. | Device access using voice authentication |
US10102359B2 (en) | 2011-03-21 | 2018-10-16 | Apple Inc. | Device access using voice authentication |
US10832287B2 (en) | 2011-05-10 | 2020-11-10 | Soundhound, Inc. | Promotional content targeting based on recognized audio |
US10121165B1 (en) | 2011-05-10 | 2018-11-06 | Soundhound, Inc. | System and method for targeting content based on identified audio and multimedia |
US11120372B2 (en) | 2011-06-03 | 2021-09-14 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
US10255566B2 (en) | 2011-06-03 | 2019-04-09 | Apple Inc. | Generating and processing task items that represent tasks to perform |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US10706373B2 (en) | 2011-06-03 | 2020-07-07 | Apple Inc. | Performing actions associated with task items that represent tasks to perform |
US10672399B2 (en) | 2011-06-03 | 2020-06-02 | Apple Inc. | Switching between text data and audio data based on a mapping |
US11350253B2 (en) | 2011-06-03 | 2022-05-31 | Apple Inc. | Active transport based notifications |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US8812294B2 (en) | 2011-06-21 | 2014-08-19 | Apple Inc. | Translating phrases from one language into another using an order-based set of declarative rules |
US8706472B2 (en) | 2011-08-11 | 2014-04-22 | Apple Inc. | Method for disambiguating multiple readings in language conversion |
US9798393B2 (en) | 2011-08-29 | 2017-10-24 | Apple Inc. | Text correction processing |
US8762156B2 (en) | 2011-09-28 | 2014-06-24 | Apple Inc. | Speech recognition repair using contextual information |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US9100291B2 (en) | 2012-01-31 | 2015-08-04 | Db Networks, Inc. | Systems and methods for extracting structured application data from a communications link |
US9525642B2 (en) | 2012-01-31 | 2016-12-20 | Db Networks, Inc. | Ordering traffic captured on a data connection |
US9185125B2 (en) | 2012-01-31 | 2015-11-10 | Db Networks, Inc. | Systems and methods for detecting and mitigating threats to a structured data storage system |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US11069336B2 (en) | 2012-03-02 | 2021-07-20 | Apple Inc. | Systems and methods for name pronunciation |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9953088B2 (en) | 2012-05-14 | 2018-04-24 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US11321116B2 (en) | 2012-05-15 | 2022-05-03 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US8775442B2 (en) | 2012-05-15 | 2014-07-08 | Apple Inc. | Semantic search using a single-source semantic model |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US11269678B2 (en) | 2012-05-15 | 2022-03-08 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US9069798B2 (en) | 2012-05-24 | 2015-06-30 | Mitsubishi Electric Research Laboratories, Inc. | Method of text classification using discriminative topic transformation |
US10019994B2 (en) | 2012-06-08 | 2018-07-10 | Apple Inc. | Systems and methods for recognizing textual identifiers within a plurality of words |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US10079014B2 (en) | 2012-06-08 | 2018-09-18 | Apple Inc. | Name recognition system |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US10957310B1 (en) * | 2012-07-23 | 2021-03-23 | Soundhound, Inc. | Integrated programming framework for speech and text understanding with meaning parsing |
US11776533B2 (en) | 2012-07-23 | 2023-10-03 | Soundhound, Inc. | Building a natural language understanding application using a received electronic record containing programming code including an interpret-block, an interpret-statement, a pattern expression and an action statement |
US10996931B1 (en) | 2012-07-23 | 2021-05-04 | Soundhound, Inc. | Integrated programming framework for speech and text understanding with block and statement structure |
US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
US9971774B2 (en) | 2012-09-19 | 2018-05-15 | Apple Inc. | Voice-based media searching |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
US8935167B2 (en) | 2012-09-25 | 2015-01-13 | Apple Inc. | Exemplar-based latent perceptual modeling for automatic speech recognition |
WO2014110281A1 (en) * | 2013-01-11 | 2014-07-17 | Db Networks, Inc. | Systems and methods for detecting and mitigating threats to a structured data storage system |
US11636869B2 (en) | 2013-02-07 | 2023-04-25 | Apple Inc. | Voice trigger for a digital assistant |
US10199051B2 (en) | 2013-02-07 | 2019-02-05 | Apple Inc. | Voice trigger for a digital assistant |
US10978090B2 (en) | 2013-02-07 | 2021-04-13 | Apple Inc. | Voice trigger for a digital assistant |
US10714117B2 (en) | 2013-02-07 | 2020-07-14 | Apple Inc. | Voice trigger for a digital assistant |
US10642574B2 (en) | 2013-03-14 | 2020-05-05 | Apple Inc. | Device, method, and graphical user interface for outputting captions |
US9733821B2 (en) | 2013-03-14 | 2017-08-15 | Apple Inc. | Voice control to diagnose inadvertent activation of accessibility features |
US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
US10572476B2 (en) | 2013-03-14 | 2020-02-25 | Apple Inc. | Refining a search based on schedule items |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US9977779B2 (en) | 2013-03-14 | 2018-05-22 | Apple Inc. | Automatic supplementation of word correction dictionaries |
US11388291B2 (en) | 2013-03-14 | 2022-07-12 | Apple Inc. | System and method for processing voicemail |
US11798547B2 (en) | 2013-03-15 | 2023-10-24 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
US9697822B1 (en) | 2013-03-15 | 2017-07-04 | Apple Inc. | System and method for updating an adaptive speech recognition model |
US10748529B1 (en) | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
US9922642B2 (en) | 2013-03-15 | 2018-03-20 | Apple Inc. | Training an at least partial voice command system |
US10078487B2 (en) | 2013-03-15 | 2018-09-18 | Apple Inc. | Context-sensitive handling of interruptions |
US11151899B2 (en) | 2013-03-15 | 2021-10-19 | Apple Inc. | User training by intelligent digital assistant |
US9633674B2 (en) | 2013-06-07 | 2017-04-25 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
US9620104B2 (en) | 2013-06-07 | 2017-04-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9966060B2 (en) | 2013-06-07 | 2018-05-08 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US10657961B2 (en) | 2013-06-08 | 2020-05-19 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US9966068B2 (en) | 2013-06-08 | 2018-05-08 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US10185542B2 (en) | 2013-06-09 | 2019-01-22 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US20140365209A1 (en) * | 2013-06-09 | 2014-12-11 | Apple Inc. | System and method for inferring user intent from speech inputs |
US20200364411A1 (en) * | 2013-06-09 | 2020-11-19 | Apple Inc. | System and method for inferring user intent from speech inputs |
US11727219B2 (en) * | 2013-06-09 | 2023-08-15 | Apple Inc. | System and method for inferring user intent from speech inputs |
US10176167B2 (en) * | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
US10769385B2 (en) | 2013-06-09 | 2020-09-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
US11048473B2 (en) | 2013-06-09 | 2021-06-29 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US9300784B2 (en) | 2013-06-13 | 2016-03-29 | Apple Inc. | System and method for emergency calls initiated by voice command |
US9633317B2 (en) | 2013-06-20 | 2017-04-25 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on a natural language intent interpreter |
US9519461B2 (en) | 2013-06-20 | 2016-12-13 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on third-party developers |
US10474961B2 (en) | 2013-06-20 | 2019-11-12 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on prompting for additional user input |
US10083009B2 (en) | 2013-06-20 | 2018-09-25 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system planning |
US9594542B2 (en) | 2013-06-20 | 2017-03-14 | Viv Labs, Inc. | Dynamically evolving cognitive architecture system based on training by third-party developers |
US20150032441A1 (en) * | 2013-07-26 | 2015-01-29 | Nuance Communications, Inc. | Initializing a Workspace for Building a Natural Language Understanding System |
US10229106B2 (en) * | 2013-07-26 | 2019-03-12 | Nuance Communications, Inc. | Initializing a workspace for building a natural language understanding system |
US10791216B2 (en) | 2013-08-06 | 2020-09-29 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
US9507849B2 (en) | 2013-11-28 | 2016-11-29 | Soundhound, Inc. | Method for combining a query and a communication command in a natural language computer system |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
US11314370B2 (en) | 2013-12-06 | 2022-04-26 | Apple Inc. | Method for extracting salient dialog usage from live data |
US9292488B2 (en) | 2014-02-01 | 2016-03-22 | Soundhound, Inc. | Method for embedding voice mail in a spoken utterance using a natural language processing computer system |
US9601114B2 (en) | 2014-02-01 | 2017-03-21 | Soundhound, Inc. | Method for embedding voice mail in a spoken utterance using a natural language processing computer system |
US11295730B1 (en) | 2014-02-27 | 2022-04-05 | Soundhound, Inc. | Using phonetic variants in a local context to improve natural language understanding |
US9564123B1 (en) | 2014-05-12 | 2017-02-07 | Soundhound, Inc. | Method and system for building an integrated user profile |
US11030993B2 (en) | 2014-05-12 | 2021-06-08 | Soundhound, Inc. | Advertisement selection by linguistic classification |
US10311858B1 (en) | 2014-05-12 | 2019-06-04 | Soundhound, Inc. | Method and system for building an integrated user profile |
US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
US11810562B2 (en) | 2014-05-30 | 2023-11-07 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US10878809B2 (en) | 2014-05-30 | 2020-12-29 | Apple Inc. | Multi-command single utterance input method |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
US10417344B2 (en) | 2014-05-30 | 2019-09-17 | Apple Inc. | Exemplar-based natural language processing |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US10714095B2 (en) | 2014-05-30 | 2020-07-14 | Apple Inc. | Intelligent assistant for home automation |
US10699717B2 (en) | 2014-05-30 | 2020-06-30 | Apple Inc. | Intelligent assistant for home automation |
US9966065B2 (en) | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
US10657966B2 (en) | 2014-05-30 | 2020-05-19 | Apple Inc. | Better resolution when referencing to concepts |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US10083690B2 (en) | 2014-05-30 | 2018-09-25 | Apple Inc. | Better resolution when referencing to concepts |
US10169329B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Exemplar-based natural language processing |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US11133008B2 (en) | 2014-05-30 | 2021-09-28 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US11699448B2 (en) | 2014-05-30 | 2023-07-11 | Apple Inc. | Intelligent assistant for home automation |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
US10497365B2 (en) | 2014-05-30 | 2019-12-03 | Apple Inc. | Multi-command single utterance input method |
US11257504B2 (en) | 2014-05-30 | 2022-02-22 | Apple Inc. | Intelligent assistant for home automation |
US11670289B2 (en) | 2014-05-30 | 2023-06-06 | Apple Inc. | Multi-command single utterance input method |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US11516537B2 (en) | 2014-06-30 | 2022-11-29 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US9668024B2 (en) | 2014-06-30 | 2017-05-30 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10904611B2 (en) | 2014-06-30 | 2021-01-26 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10431204B2 (en) | 2014-09-11 | 2019-10-01 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US10390213B2 (en) | 2014-09-30 | 2019-08-20 | Apple Inc. | Social reminders |
US9986419B2 (en) | 2014-09-30 | 2018-05-29 | Apple Inc. | Social reminders |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10453443B2 (en) | 2014-09-30 | 2019-10-22 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US10438595B2 (en) | 2014-09-30 | 2019-10-08 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US11556230B2 (en) | 2014-12-02 | 2023-01-17 | Apple Inc. | Data detection |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
US9569427B2 (en) * | 2014-12-25 | 2017-02-14 | Clarion Co., Ltd. | Intention estimation equipment and intention estimation system |
US20160188574A1 (en) * | 2014-12-25 | 2016-06-30 | Clarion Co., Ltd. | Intention estimation equipment and intention estimation system |
US11231904B2 (en) | 2015-03-06 | 2022-01-25 | Apple Inc. | Reducing response latency of intelligent automated assistants |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US10529332B2 (en) | 2015-03-08 | 2020-01-07 | Apple Inc. | Virtual assistant activation |
US10311871B2 (en) | 2015-03-08 | 2019-06-04 | Apple Inc. | Competing devices responding to voice triggers |
US11842734B2 (en) | 2015-03-08 | 2023-12-12 | Apple Inc. | Virtual assistant activation |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US11087759B2 (en) | 2015-03-08 | 2021-08-10 | Apple Inc. | Virtual assistant activation |
US10930282B2 (en) | 2015-03-08 | 2021-02-23 | Apple Inc. | Competing devices responding to voice triggers |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US11468282B2 (en) | 2015-05-15 | 2022-10-11 | Apple Inc. | Virtual assistant in a communication session |
US11070949B2 (en) | 2015-05-27 | 2021-07-20 | Apple Inc. | Systems and methods for proactively identifying and surfacing relevant content on an electronic device with a touch-sensitive display |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US11127397B2 (en) | 2015-05-27 | 2021-09-21 | Apple Inc. | Device voice control |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10681212B2 (en) | 2015-06-05 | 2020-06-09 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US10356243B2 (en) | 2015-06-05 | 2019-07-16 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US11010127B2 (en) | 2015-06-29 | 2021-05-18 | Apple Inc. | Virtual assistant for media playback |
US11947873B2 (en) | 2015-06-29 | 2024-04-02 | Apple Inc. | Virtual assistant for media playback |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US11126400B2 (en) | 2015-09-08 | 2021-09-21 | Apple Inc. | Zero latency digital assistant |
US11853536B2 (en) | 2015-09-08 | 2023-12-26 | Apple Inc. | Intelligent automated assistant in a media environment |
US11809483B2 (en) | 2015-09-08 | 2023-11-07 | Apple Inc. | Intelligent automated assistant for media search and playback |
US11550542B2 (en) | 2015-09-08 | 2023-01-10 | Apple Inc. | Zero latency digital assistant |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US11500672B2 (en) | 2015-09-08 | 2022-11-15 | Apple Inc. | Distributed personal assistant |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US11526368B2 (en) | 2015-11-06 | 2022-12-13 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US11886805B2 (en) | 2015-11-09 | 2024-01-30 | Apple Inc. | Unconventional virtual assistant interactions |
US10354652B2 (en) | 2015-12-02 | 2019-07-16 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10942703B2 (en) | 2015-12-23 | 2021-03-09 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US11853647B2 (en) | 2015-12-23 | 2023-12-26 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US11227589B2 (en) | 2016-06-06 | 2022-01-18 | Apple Inc. | Intelligent list reading |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
US11069347B2 (en) | 2016-06-08 | 2021-07-20 | Apple Inc. | Intelligent automated assistant for media exploration |
US10354011B2 (en) | 2016-06-09 | 2019-07-16 | Apple Inc. | Intelligent automated assistant in a home environment |
US10733993B2 (en) | 2016-06-10 | 2020-08-04 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US11657820B2 (en) | 2016-06-10 | 2023-05-23 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US11037565B2 (en) | 2016-06-10 | 2021-06-15 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
US10521466B2 (en) | 2016-06-11 | 2019-12-31 | Apple Inc. | Data driven natural language event detection and classification |
US10580409B2 (en) | 2016-06-11 | 2020-03-03 | Apple Inc. | Application integration with a digital assistant |
US10089072B2 (en) | 2016-06-11 | 2018-10-02 | Apple Inc. | Intelligent device arbitration and control |
US11749275B2 (en) | 2016-06-11 | 2023-09-05 | Apple Inc. | Application integration with a digital assistant |
US11809783B2 (en) | 2016-06-11 | 2023-11-07 | Apple Inc. | Intelligent device arbitration and control |
US10942702B2 (en) | 2016-06-11 | 2021-03-09 | Apple Inc. | Intelligent device arbitration and control |
US11152002B2 (en) | 2016-06-11 | 2021-10-19 | Apple Inc. | Application integration with a digital assistant |
US10297253B2 (en) | 2016-06-11 | 2019-05-21 | Apple Inc. | Application integration with a digital assistant |
US10269345B2 (en) | 2016-06-11 | 2019-04-23 | Apple Inc. | Intelligent task discovery |
US10474753B2 (en) | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10553215B2 (en) | 2016-09-23 | 2020-02-04 | Apple Inc. | Intelligent automated assistant |
US11281993B2 (en) | 2016-12-05 | 2022-03-22 | Apple Inc. | Model and ensemble compression for metric learning |
US20180165135A1 (en) * | 2016-12-09 | 2018-06-14 | Fujitsu Limited | Api learning |
US10691507B2 (en) * | 2016-12-09 | 2020-06-23 | Fujitsu Limited | API learning |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US11204787B2 (en) | 2017-01-09 | 2021-12-21 | Apple Inc. | Application integration with a digital assistant |
US11656884B2 (en) | 2017-01-09 | 2023-05-23 | Apple Inc. | Application integration with a digital assistant |
CN107015964A (en) * | 2017-03-22 | 2017-08-04 | 北京光年无限科技有限公司 | The self-defined intention implementation method and device developed towards intelligent robot |
US10417266B2 (en) | 2017-05-09 | 2019-09-17 | Apple Inc. | Context-aware ranking of intelligent response suggestions |
US10332518B2 (en) | 2017-05-09 | 2019-06-25 | Apple Inc. | User interface for correcting recognition errors |
US10741181B2 (en) | 2017-05-09 | 2020-08-11 | Apple Inc. | User interface for correcting recognition errors |
US10847142B2 (en) | 2017-05-11 | 2020-11-24 | Apple Inc. | Maintaining privacy of personal information |
US10395654B2 (en) | 2017-05-11 | 2019-08-27 | Apple Inc. | Text normalization based on a data-driven learning network |
US10726832B2 (en) | 2017-05-11 | 2020-07-28 | Apple Inc. | Maintaining privacy of personal information |
US11599331B2 (en) | 2017-05-11 | 2023-03-07 | Apple Inc. | Maintaining privacy of personal information |
US10755703B2 (en) | 2017-05-11 | 2020-08-25 | Apple Inc. | Offline personal assistant |
US11580990B2 (en) | 2017-05-12 | 2023-02-14 | Apple Inc. | User-specific acoustic models |
US10791176B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Synchronization and task delegation of a digital assistant |
US11405466B2 (en) | 2017-05-12 | 2022-08-02 | Apple Inc. | Synchronization and task delegation of a digital assistant |
US10789945B2 (en) | 2017-05-12 | 2020-09-29 | Apple Inc. | Low-latency intelligent automated assistant |
US11301477B2 (en) | 2017-05-12 | 2022-04-12 | Apple Inc. | Feedback analysis of a digital assistant |
US11380310B2 (en) | 2017-05-12 | 2022-07-05 | Apple Inc. | Low-latency intelligent automated assistant |
US10410637B2 (en) | 2017-05-12 | 2019-09-10 | Apple Inc. | User-specific acoustic models |
US10810274B2 (en) | 2017-05-15 | 2020-10-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US10482874B2 (en) | 2017-05-15 | 2019-11-19 | Apple Inc. | Hierarchical belief states for digital assistants |
US10909171B2 (en) | 2017-05-16 | 2021-02-02 | Apple Inc. | Intelligent automated assistant for media exploration |
US11675829B2 (en) | 2017-05-16 | 2023-06-13 | Apple Inc. | Intelligent automated assistant for media exploration |
US10748546B2 (en) | 2017-05-16 | 2020-08-18 | Apple Inc. | Digital assistant services based on device capabilities |
US11532306B2 (en) | 2017-05-16 | 2022-12-20 | Apple Inc. | Detecting a trigger of a digital assistant |
US10403278B2 (en) | 2017-05-16 | 2019-09-03 | Apple Inc. | Methods and systems for phonetic matching in digital assistant services |
US10311144B2 (en) | 2017-05-16 | 2019-06-04 | Apple Inc. | Emoji word sense disambiguation |
US11217255B2 (en) | 2017-05-16 | 2022-01-04 | Apple Inc. | Far-field extension for digital assistant services |
US10303715B2 (en) | 2017-05-16 | 2019-05-28 | Apple Inc. | Intelligent automated assistant for media exploration |
US10657328B2 (en) | 2017-06-02 | 2020-05-19 | Apple Inc. | Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling |
US10445429B2 (en) | 2017-09-21 | 2019-10-15 | Apple Inc. | Natural language understanding using vocabularies with compressed serialized tries |
US10755051B2 (en) | 2017-09-29 | 2020-08-25 | Apple Inc. | Rule-based natural language processing |
US10636424B2 (en) | 2017-11-30 | 2020-04-28 | Apple Inc. | Multi-turn canned dialog |
US10733982B2 (en) | 2018-01-08 | 2020-08-04 | Apple Inc. | Multi-directional dialog |
US10733375B2 (en) | 2018-01-31 | 2020-08-04 | Apple Inc. | Knowledge-based framework for improving natural language understanding |
US10789959B2 (en) | 2018-03-02 | 2020-09-29 | Apple Inc. | Training speaker recognition models for digital assistants |
US10592604B2 (en) | 2018-03-12 | 2020-03-17 | Apple Inc. | Inverse text normalization for automatic speech recognition |
US10818288B2 (en) | 2018-03-26 | 2020-10-27 | Apple Inc. | Natural assistant interaction |
US11710482B2 (en) | 2018-03-26 | 2023-07-25 | Apple Inc. | Natural assistant interaction |
US10909331B2 (en) | 2018-03-30 | 2021-02-02 | Apple Inc. | Implicit identification of translation payload with neural machine translation |
US11487364B2 (en) | 2018-05-07 | 2022-11-01 | Apple Inc. | Raise to speak |
US11169616B2 (en) | 2018-05-07 | 2021-11-09 | Apple Inc. | Raise to speak |
US10928918B2 (en) | 2018-05-07 | 2021-02-23 | Apple Inc. | Raise to speak |
US11854539B2 (en) | 2018-05-07 | 2023-12-26 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US11900923B2 (en) | 2018-05-07 | 2024-02-13 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US11145294B2 (en) | 2018-05-07 | 2021-10-12 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US10984780B2 (en) | 2018-05-21 | 2021-04-20 | Apple Inc. | Global semantic word embeddings using bi-directional recurrent neural networks |
US10684703B2 (en) | 2018-06-01 | 2020-06-16 | Apple Inc. | Attention aware virtual assistant dismissal |
US11495218B2 (en) | 2018-06-01 | 2022-11-08 | Apple Inc. | Virtual assistant operation in multi-device environments |
US10892996B2 (en) | 2018-06-01 | 2021-01-12 | Apple Inc. | Variable latency device coordination |
US11386266B2 (en) | 2018-06-01 | 2022-07-12 | Apple Inc. | Text correction |
US11431642B2 (en) | 2018-06-01 | 2022-08-30 | Apple Inc. | Variable latency device coordination |
US10984798B2 (en) | 2018-06-01 | 2021-04-20 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
US10403283B1 (en) | 2018-06-01 | 2019-09-03 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
US11009970B2 (en) | 2018-06-01 | 2021-05-18 | Apple Inc. | Attention aware virtual assistant dismissal |
US10720160B2 (en) | 2018-06-01 | 2020-07-21 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
US11360577B2 (en) | 2018-06-01 | 2022-06-14 | Apple Inc. | Attention aware virtual assistant dismissal |
US10944859B2 (en) | 2018-06-03 | 2021-03-09 | Apple Inc. | Accelerated task performance |
US10504518B1 (en) | 2018-06-03 | 2019-12-10 | Apple Inc. | Accelerated task performance |
US10496705B1 (en) | 2018-06-03 | 2019-12-03 | Apple Inc. | Accelerated task performance |
US11010561B2 (en) | 2018-09-27 | 2021-05-18 | Apple Inc. | Sentiment prediction from textual data |
US11462215B2 (en) | 2018-09-28 | 2022-10-04 | Apple Inc. | Multi-modal inputs for voice commands |
US11170166B2 (en) | 2018-09-28 | 2021-11-09 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
US10839159B2 (en) | 2018-09-28 | 2020-11-17 | Apple Inc. | Named entity normalization in a spoken dialog system |
CN109388802A (en) * | 2018-10-11 | 2019-02-26 | 北京轮子科技有限公司 | A kind of semantic understanding method and apparatus based on deep learning |
US11475898B2 (en) | 2018-10-26 | 2022-10-18 | Apple Inc. | Low-latency multi-speaker speech recognition |
US11638059B2 (en) | 2019-01-04 | 2023-04-25 | Apple Inc. | Content playback on multiple devices |
US11348573B2 (en) | 2019-03-18 | 2022-05-31 | Apple Inc. | Multimodality in digital assistant systems |
CN110096570A (en) * | 2019-04-09 | 2019-08-06 | 苏宁易购集团股份有限公司 | A kind of intension recognizing method and device applied to intelligent customer service robot |
US11423908B2 (en) | 2019-05-06 | 2022-08-23 | Apple Inc. | Interpreting spoken requests |
US11307752B2 (en) | 2019-05-06 | 2022-04-19 | Apple Inc. | User configurable task triggers |
US11217251B2 (en) | 2019-05-06 | 2022-01-04 | Apple Inc. | Spoken notifications |
US11475884B2 (en) | 2019-05-06 | 2022-10-18 | Apple Inc. | Reducing digital assistant latency when a language is incorrectly determined |
US11705130B2 (en) | 2019-05-06 | 2023-07-18 | Apple Inc. | Spoken notifications |
US11140099B2 (en) | 2019-05-21 | 2021-10-05 | Apple Inc. | Providing message response suggestions |
US11888791B2 (en) | 2019-05-21 | 2024-01-30 | Apple Inc. | Providing message response suggestions |
US11657813B2 (en) | 2019-05-31 | 2023-05-23 | Apple Inc. | Voice identification in digital assistant systems |
US11496600B2 (en) | 2019-05-31 | 2022-11-08 | Apple Inc. | Remote execution of machine-learned models |
US11237797B2 (en) | 2019-05-31 | 2022-02-01 | Apple Inc. | User activity shortcut suggestions |
US11360739B2 (en) | 2019-05-31 | 2022-06-14 | Apple Inc. | User activity shortcut suggestions |
US11289073B2 (en) | 2019-05-31 | 2022-03-29 | Apple Inc. | Device text to speech |
US11360641B2 (en) | 2019-06-01 | 2022-06-14 | Apple Inc. | Increasing the relevance of new available information |
US11488406B2 (en) | 2019-09-25 | 2022-11-01 | Apple Inc. | Text detection using global geometry estimators |
US11783246B2 (en) | 2019-10-16 | 2023-10-10 | Talkdesk, Inc. | Systems and methods for workforce management system deployment |
US11736615B2 (en) | 2020-01-16 | 2023-08-22 | Talkdesk, Inc. | Method, apparatus, and computer-readable medium for managing concurrent communications in a networked call center |
US11765209B2 (en) | 2020-05-11 | 2023-09-19 | Apple Inc. | Digital assistant hardware abstraction |
US11924254B2 (en) | 2020-05-11 | 2024-03-05 | Apple Inc. | Digital assistant hardware abstraction |
US11755276B2 (en) | 2020-05-12 | 2023-09-12 | Apple Inc. | Reducing description length based on confidence |
US11610065B2 (en) | 2020-06-12 | 2023-03-21 | Apple Inc. | Providing personalized responses based on semantic context |
US11677875B2 (en) | 2021-07-02 | 2023-06-13 | Talkdesk Inc. | Method and apparatus for automated quality management of communication records |
US11856140B2 (en) | 2022-03-07 | 2023-12-26 | Talkdesk, Inc. | Predictive communications system |
US11736616B1 (en) | 2022-05-27 | 2023-08-22 | Talkdesk, Inc. | Method and apparatus for automatically taking action based on the content of call center communications |
US11971908B2 (en) | 2022-06-17 | 2024-04-30 | Talkdesk, Inc. | Method and apparatus for detecting anomalies in communication data |
US11943391B1 (en) | 2022-12-13 | 2024-03-26 | Talkdesk, Inc. | Method and apparatus for routing communications within a contact center |
Also Published As
Publication number | Publication date |
---|---|
KR20110036385A (en) | 2011-04-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20110082688A1 (en) | Apparatus and Method for Analyzing Intention | |
US7124080B2 (en) | Method and apparatus for adapting a class entity dictionary used with language models | |
US7043422B2 (en) | Method and apparatus for distribution-based language model adaptation | |
US6606597B1 (en) | Augmented-word language model | |
US10902846B2 (en) | Spoken language understanding apparatus and spoken language understanding method using the same | |
US9442910B2 (en) | Method and system for adding punctuation to voice files | |
US7542907B2 (en) | Biasing a speech recognizer based on prompt context | |
US7529657B2 (en) | Configurable parameters for grammar authoring for speech recognition and natural language understanding | |
US8849668B2 (en) | Speech recognition apparatus and method | |
US20020133346A1 (en) | Method for processing initially recognized speech in a speech recognition session | |
US11043213B2 (en) | System and method for detection and correction of incorrectly pronounced words | |
US10242670B2 (en) | Syntactic re-ranking of potential transcriptions during automatic speech recognition | |
US8255220B2 (en) | Device, method, and medium for establishing language model for expanding finite state grammar using a general grammar database | |
CN109754809A (en) | Audio recognition method, device, electronic equipment and storage medium | |
JP2005024797A (en) | Statistical language model generating device, speech recognition device, statistical language model generating method, speech recognizing method, and program | |
US10152298B1 (en) | Confidence estimation based on frequency | |
Chen et al. | Lightly supervised and data-driven approaches to mandarin broadcast news transcription | |
Bhuvanagiri et al. | An approach to mixed language automatic speech recognition | |
Skantze | Galatea: A discourse modeller supporting concept-level error handling in spoken dialogue systems | |
CN115457938A (en) | Method, device, storage medium and electronic device for identifying awakening words | |
Tran et al. | Joint modeling of text and acoustic-prosodic cues for neural parsing | |
US20220310067A1 (en) | Lookup-Table Recurrent Language Model | |
US6772116B2 (en) | Method of decoding telegraphic speech | |
JP6276516B2 (en) | Dictionary creation apparatus and dictionary creation program | |
US6128595A (en) | Method of determining a reliability measure |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KIM, JUNG EUN;CHO, JEONG MI;REEL/FRAME:025072/0258 Effective date: 20100929 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |