CA2293780C - Method of using a natural language interface to retrieve information from one or more data resources - Google Patents
Method of using a natural language interface to retrieve information from one or more data resources Download PDFInfo
- Publication number
- CA2293780C CA2293780C CA002293780A CA2293780A CA2293780C CA 2293780 C CA2293780 C CA 2293780C CA 002293780 A CA002293780 A CA 002293780A CA 2293780 A CA2293780 A CA 2293780A CA 2293780 C CA2293780 C CA 2293780C
- Authority
- CA
- Canada
- Prior art keywords
- information
- query
- natural language
- telephone number
- language query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/487—Arrangements for providing information services, e.g. recorded voice services or time announcements
- H04M3/493—Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
- H04M3/4931—Directory assistance systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2201/00—Electronic components, circuits, software, systems or apparatus used in telephone systems
- H04M2201/40—Electronic components, circuits, software, systems or apparatus used in telephone systems using speech recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/912—Applications of a database
- Y10S707/913—Multimedia
- Y10S707/916—Audio
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/912—Applications of a database
- Y10S707/917—Text
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/953—Organization of data
- Y10S707/955—Object-oriented
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99934—Query formulation, input preparation, or translation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99935—Query augmenting and refining, e.g. inexact access
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99943—Generating database or data structure, e.g. via user interface
Abstract
A method of using at least one natural language query to retrieve information from one or more data resources and further performing a requested action using the retrieved information is disclosed. At least one natural language query directed to retrieving particular information is received. At least one object from the natural language query is extracted. The relationship between each of the at least one extracted objects is determined. A semantic representation is created from the at least one extracted objects. The semantic representation is compared to a knowledge structure. The knowledge structure is comprised of one or more grammars which are extracted from a plurality of data resources. The semantic representations are matched to the grammar. A database query is generated based on the matched objects. The query is applied to one or more of the data resources and information is retrieved.
The requested action is then performed using the retrieved information.
The requested action is then performed using the retrieved information.
Description
WO 99/53676 PCT/US99/0727$
METHOD OF USING A NATURAL LANGUAGE INTERFACE TO
RETRIEVE INFORMATION FROM ONE OR MORE DATA RESOURCES
Technical Field The invention relates to a method for using a natural language query for retrieving information from one or more data resources and, more particularly, a method for using a natural language query for automatically retrieving and to configuring a destination telephone number from existing data resources and using the destination telephone number to place a telephone call.
Background of the Invention Files or other resources on computers around the world may be publicly 15 available to users of other computers through the collection of networks known as the Internet. The collection of all such publicly available resources, linked together using files written in Hypertext Mark-up Language ("HTML") is known as the World Wide Web ("web") A user of a computer that is connected to the Internet may cause a program 2o known as a client to request resources that are part of the web. Server programs then process the requests to return the specified resources. A standard naming convention has been adopted, known as a Uniform Resource Locator ("URL").
This convention encompasses several types of location names, presently including subclasses such as Hypertext Transport Protocol ("http"), File Transport Protocol 25 ("ftp"), gopher and.Wide Area Information Service ("WAIS") The various resources accessible via the web are created and maintained by many different people on servers located all around the world, and may be created for many different purposes. Many individuals and businesses now have their own web sites that can be visited by people "surfing" the web. These web sites typically 3o provide information on a myriad of subjects such as sports, business, news and even community events. For example, many web sites exist which provide useful information about a particular business establishment such as office locations, customer service telephone numbers and information about the products and/or services that the business offers to the consumer.
In many cases, an individual accessing such a web site is looking for particular information such as information about a particular product.
However, the individual might desire additional information about the particular product, which is not available from the web site. Typically, the individual will place a telephone call to the customer service department of the business entity to obtain the l0 additional information. For example, the individual may want to know if an electronics store carries a particular product, such as a particular brand name television and the price of the television. Such information may not be contained in the web site and the individual may have to call the electronics store to receive the desired information. This multiple step process of searching and retrieving 15 information from the Internet and then using a conventional phone line to call the business establishment for still more information or conducting business is cumbersome and time consuming.
In other cases, the individual may generally know the location of a particular establishment, but may not know the exact address or the name of the 2o establishment. For example, an individual may know that there is an electronics store in their town on Route 4, but may not know the name of the electronics store.
The individual may access a web site that contains information about the particular town to try to identify the exact name and location of the electronics store.
Conventional searching techniques require a hierarchical step by step query 25 approach to locate the desired information. For example, the individual may enter the address of the store, if known, or the type of store to get a listing of relevant information. However, such searching may require many steps and, in many instances, the available search queries may not be compatible with the known information. As such, the searching many require additional unnecessary search 3o queries in order to obtain the desired information.
Summary of the Invention In accordance with the present invention, a method of using at least one natural language query to retrieve information from one or more data resources and further performing a requested action using the retrieved information is 5 disclosed. At least one natural language query directed to retrieving particular information is received. At least one object from the natural language query is extracted. The relationship between each of the at least one extracted objects is determined. A semantic representation is created from the at least one extracted objects. A database query is generated based on the semantic 1o representation using a pre-defined knowledge structure. The query is applied to one or more of the data resources and information is retrieved. The requested action is then performed using the retrieved information.
In another embodiment of the present invention, the information retrieved is a telephone number for a particular destination and the requested 15 action is the placement of a telephone call using the retrieved telephone number.
In another embodiment of the present invention, the requested information is retrieved by querying multiple data resources in which each data resource contains a different type of information. Objects are extracted from 2o the natural language query which correspond to different types of information.
Each information type is associated with a particular data resource. A query is generated for each object and applied to the corresponding data resource. The retrieved information may be integrated into a query to a different data resource to retrieve additional information. The retrieved information may be 25 used to perform a requested action such as the placement of a telephone call.
In accordance with one aspect of the present invention there is provided a method of using at least one natural language query to retrieve information from an data resource and using said information to perform a requested action comprising: receiving at least one natural language query directed to 3o retrieving said information; creating one or more semantic representations from said natural language query; comparing said semantic tags to a 3a knowledge structure associated with said data resource; generating a database query based on said comparison; using said database query to retrieve said information from said data resource; and using said retrieved information to perform said requested action.
Brief Descriution of the Drawings In the drawings, where like numerals refer to like elements throughout the several views:
FIG. 1 is a simplified diagram of an exemplary system embodying the invention;
FIGs. 2a and 2b are flow charts illustrating the implementation of a query by the system of FIG. 1; and FIG. 3 is a flow chart illustrating the creation of a knowledge structure from a natural language query.
Detailed Description of the Invention For purposes of illustration, FIG. 1 is a simplified diagram of an exemplary system 100 embodying the invention. A user desires to establish a real-time connection to a particular establishment 130 (e.g., a telephone call to a representative of the establishment 130) by extracting relevant information from one or more data resources. In accordance with an embodiment of the present invention, the data resources are web sites identified by a particular URL. It is to be understood by those skilled in the art that the data resources can be any type of data file that may be accessed by the user over a packet network 108.
Using a personal computer (PC) 102, a user establishes a connection with packet network 108 via an access server 106. The user may also use a telephone 103 to connect to the packet network 108. Typically a modem connection (not shown) may be used to connect the PC 102 to the packet network 108 in a 2o conventional manner. The packet network 108 can be, for example, the Internet or an Intranet. The packet network 108 may comprise a single packet network or a multiplicity of packet networks, such as, e.g., the "backbone" networks comprising the Internet. The access server 106 may illustratively, be a server connected to the Internet provided by, e.g., an Internet service provider, or may be any other server used for providing access to the packet network 108.
As illustrated, the packet network 108 is connected to a plurality of information servers 110 which host a plurality of information services or web sites.
An information server 110 includes grammars that represent the language (written or spoken) used by customers for accessing the information related to that 3o particular service or site.
When the information server I 10 receives a natural language query, the information server 110 transmits the natural language query and the corresponding grammar to a service host 112 in a predefined protocol that characterizes the service host.
In general the service host 112 can communicate with a plurality of information servers, each one of them representing a particular service or site. Any information server that complies with the protocol defined by a service host can use the services provided by that service host.
The service host 112 coordinates with the information server 110, the 1 o associated data resources, and the natural language (NL) server 114 to process the natural language query. The service host 112 sends the natural language query and the specific grammars provided by the information server 110 to the NL server 114.
The NL server 114 parses the natural language query and sends the resulting semantic tags to the service host 112. When the service host 1 I2 receives the ~ 5 semantic tags, and if the service host 112 decides, based on the rules, that there is enough information to retrieve data from the data resources, generates a search query. The service host 112 uses ad-hoc transducers (a.k.a knowledge structures) to convert semantic tags into search queries (e.g. SQL queries) for each particular data resource. The knowledge structures are stored in the databases 118(a-n). The search 2o query is used to access data resources contained in, typically, a plurality of destination servers 116. It is to be understood by those skilled in the art that more than one data resource may be accessed from a single destination server, or alternatively, multiple data resources contained in different destination servers may be accessed at the same time. These destination servers may be dedicated to the 25 particular service host and/or publicly available and accessed through the packet network I 08 (not shown in Figure 1 ).
An example of a search query that may be generated is a request for a telephone number corresponding to a particular establishment. Once the appropriate telephone number of the establishment is retrieved call connection can 3o be provided by a standard telecom adjunct at the service host 112 i.e., the call to the telephone 122 of the establishment 130 is initiated. Once a connection is established with the called party (e.g., the establishment 130) a direct connection is established between the user and the called party (e.g., employee of establishment 130). Alternatively, instead of a telephone call, the user can connect to the establishment's facsimile machine 124 or personal computer 126.
A more detailed example of how the present invention may be implemented is set forth in FIGS. 2a and 2b. A user may access a particular data resource such as a directory for the town of Westfield by inputting the URL
http://www.westfield.com (step 202). The URL for that Westfield data resource is 1 o inputted into PC 102 either by typing the request using a keyboard 104 or by speaking the request into a microphone 1 O5. Alternatively the information server 110 can be directly accessed through a telephone 103: ( 1 ) an Internet telephone connected to the access server (2) a conventional PSTN telephone connected to the information server assuming that the information server has a telecom adjunct and the capability of processing spoken requests.
Spoken requests either from a PC microphone 105 or from a telephone 103 can be handled by a speech recognition system residing at the information server.
The PC 102 dials into an access server 106 that is connected to the Internet or other database service via a logical network interface (not shown). The logical 2o network interface may be a local area network (LAN), a Serial Line Internet Protocol (SLIP) connection over a modem, an ISDN port or via a connection to a special LAN such as an ATM LAN or a LAN that offers bandwidth reservation.
The invention is independent of the actual modality of call placement. As mentioned before, call placement can occur from a PSTN (made possible by means of a telecom adjunct at the server) or Internet telephone.
In general, the information server that is accessed can either be a specific web site, such as, but not limited to the web site of a company, or a dedicated information resource web site. The dedicated information resource web site can be, for example, a single web site or a combination of web sites which contains a 3o significant amount of general reference information which relates to a particular subject matter, such as a national directory of movie information. A web site can also be dedicated to handle a particular subject matter. For example, a web site can contain movie and theater information for all movie theaters in the United States.
The information server to be accessed may be characterized by a URL and/or a telephone number.
Once an information server is accessed, the user can send a text or a spoken query requesting a particular action or service (step 204), for example: "call the pizza place on Main Street in Westfield". The query is received by the access server 106 and the natural language query is sent to the information server 110 via packet I o network 108. It is to be understood that the packet network 108 may be connected to a plurality of information servers which each relate to one or more particular information services, or there may be a single centralized information server which is accessed by all information services which are capable of receiving and processing natural language queries. The information server 110 is preferably located in one or more adjunct servers. Each information server 110 contains at least some of the data resources (e.g., URLs and associated site/service-specific grammars) capable of receiving and responding to a natural language query.
If an information server is enhanced for voice access, -standard speech application programming interfaces may be used to provide a means for 2o communicating between the user and the web site. Information servers include grammars that reflect the language used by people to request or to describe information services specific to that particular site. The grammar should try to capture all possible reasonable sentences or queries.
Once the information server receives the user's query, it retrieves the relevant grammars for processing the query (step 206). Following this, a communication between the information server 110 and the service host 112 will be established. The information server will send the user's query and the retrieved grammars to the service host (step 208). The service host 112 coordinates the activities of the information server 110 with a NL server 114 and at least one 3o destination server 116 to process the natural language query and perform the *rB
requested action. The service host 112 includes a dialog control program that manages interactions with users over several turns (e.g., it decides when to ask a question, when to give an answer, provides means for clarifying ambiguities, and provides error control and recovery during an interaction). The service host then must determine what type of information is being requested as expressed in the user's natural language query. For example, is the query requiring an action, such as the placement of a telephone call, or requesting specific information, such as the location at which a particular movie is being shown. To parse the natural language query the service host uses a natural language server NL 114: the query and the to grammar are sent to NL (step 210). An NL server typically embodies a parser, an example of which is described in Pieraccini, R., Levin, E., "A Spontaneous-Speech Understanding System for Database Query Applications," ESCA Workshop on Spoken Dialogue Systems - Theories and Applications, May 30, 3une 2, 1995, Vigs , Denmark, which is incorporated by reference. The semantic representations are then received by the service host 112.
The NL server 114 parses the natural language query into a plurality of semantic tags (step 212) as will be described in detail hereinafter. The semantic tags are then sent from the NL server 114 to the service host 112 (step 214).
The service host 112 converts the semantic tags to a string of logical search 2o queries (step 216). The conversion of the semantic tags into a search query is accomplished by a knowledge structure which provides a mapping between the semantic tags and the language used for querying the database in the selected application (e.g., SQL). This mapping is performed by an ad-hoc program developed for the particular application. The resulting knowledge structure is stored in databases 118. Standard techniques may be used for developing the ad-hoc program. For the example if a standard SQL database is used, the mapping will look like:
Input: Action: Call / Action Object: Pizza Restaurant / Location: Main Street / City: Westfield Output: SELECT telephone number FROM pizza restaurants WHERE
location--'Main Street' and 'city='Westfieid';
It is obvious that Call should map into the attribute telephone number, Pizza Restaurant to pizza restaurants, etc.
To summarize, a grammar (either handcrafted or automatically acquired using machine learning algorithms) is used to parse the natural language query to obtain a set of semantic tags that represent the query. The query can be received either as written text or can be translated from speech to text via a speech recognition system. For the query "call the pizza place on Main Street in to Westfield", the parser generates a semantic representation such as "Action:
Call /
Action Object: Pizza Restaurant / Location: Main Street / City: Westfield".
The semantic representation can then be used for generating a search query (e.g., in SQL) that will retrieve the corresponding information. In the case of the exemplary query, the telephone number for the particular pizza restaurant is retrieved from the data resource. An example of a system which is capable of processing such a query is described in Pieraccini, R., Levin, E., "A Spontaneous-Speech Understanding System for Database Query Applications," ESCA Workshop on Spoken Dialogue Systems - Theories and Applications, May 30, June 2, 1995, Vigs , Denmark which is incorporated by reference.
2o The system can implement a dialog control as that described in Pieraccini, R., Levin, E., Eckert, W., "AMICA, the AT&T Mixed Initiative Conversational Architecture," Proc. EUROSPEECH 97, September 1997, Rhodes, Greece which is also incorporated by reference. The above symbolic description of the natural language query is then analyzed according to a predefined set of rules. The rules have to be defined ad-hoc for each application. For instance in the pizza restaurant application there might be rules that say:
If (Action Object=Pizza Restaurant) then User must provide: Location and City So, For instance, if the rule is activated (i.e. when Action Object=Pizza Restaurant), and the user had given the Location already, the system will request the City.
If the service host 112, based on the rules, decides that there is enough information for performing a database access, the database query is generated.
The database query is generally in one of the standard query languages (e.g. SQL).
The service host 112 will have a table that maps topics to URL's, so it knows that if topic=Pizza Restaurant, the URL to send the query to is www.njrestaurants.com.
The search queries are forwarded to the web site associated with the URL
1o transmitted by the user that is located at one of the destination servers 116 (step 218). The web sites process the query and retrieve data that is believed to respond to the query (step 220). The query results are transmitted to the service host 112.
'the service host 112 determines if there are any ambiguities with respect to the response (step 222) and, if so, forwards additional queries to the user to help to resolve the ambiguities (step 224). The service host 112 then sends the responses to the information server 110 (step 226).
If there are too many potential answers (for instance if there are two pizza places on Main Street in Westfield), one or more questions to the user are generated in order to disambiguate the query (e.g. Do you mean "Venezia" or "Bella Roma?"). The answers to the additional questions are used to formulate a new logical search query.
For this there might be additional rules like:
If(Action Object=Pizza Restaurant and Too-Many Answers) then User must provide further clarifying information such as, for example, the name of restaurant OR exact address.
If the user does not provide enough information to achieve a single answer, the service host 112 might then list the possibilities and ask the user to chose one of them.
Parallel search queries can be launched from the original natural language 3o query or parent query in order to fill in information that is missing from the original query. Typically such parallel queries are added by using a logical AND/OR relation. For example, the query "call ITALIAN RESTAURANT in WESTFIELD NEAR the INTERSECTION of EAST and BROAD STREET" can be segmented into two parallel searches queries. The first search query is directed to accessing a restaurant directory to list all Italian restaurants in Westfield. The second parallel search query is directed to accessing a street database to find all streets near the intersection.
Once the service host 112 determines that a single response matches the database query, the requested information is retrieved from the database. In the 1 o case of the present example, the appropriate telephone number of the calling party is retrieved and confirmed using dialog control. The information (i.e., phone number) is received by the service host 112 which then forwards the number to the information server 110 (step 226). The service host 112 then performs the desired action (step 228). In the above example, the service host would establish a telephone connection between the pizza restaurant and the user.
Call connection can be provided by a standard telecom adjunct at the service host, i.e., the call to the telephone 122 of the pizza restaurant is initiated. Once a connection is established with the called party (e.g., the pizza restaurant) a direct connection is established between the user and the called party (e.g., pizza 2o restaurant owner). Alternatively, instead of a telephone call, the user can connect to the called party's facsimile machine 124 or personal computer 126.
In an alternative embodiment, the user may input the natural language query using a telephone. In such a case, the user dials a telephone number to gain access to a particular Internet resource and speaks the natural language query, e.g., "call the pizza place on Main Street in Westfield." The query is received by a speech recognizes 107 connected to the information server 110. An example of a speech recognizes which may be used is disclosed in L. Rabiner and B-H.Juang, "Fundamentals of Speech Recognition" Prentice Hall, Englewood Cliffs, NJ, 1993, which may be incorporated by reference. The speech recognizes translates a spoken 3o query into the corresponding text for and returns it to the information server.
Described herein are further details of the functional aspects of the service host 112. In addition, to providing communication routing, the service host provides controller functionality. An integral functional component of the service host is a controller that uses the semantic representations obtained from the NL
server to create a knowledge structure. The controller acts upon a strategy based on a sequence of operations (actions) and rules (see reference (2)). The sequence of operations determines which of a finite number of predetermined functions is invoked at any step of the processing. One of the predetermined functions that was mentioned previously is the interfacing with the natural language server to obtain 1o a symbolic representation of the user's query i.e., semantic tagging. The other functions include interfacing with a speech generation mechanism to produce a spoken question or answer, and performing a search query to a database using the semantic tags obtained from the natural language server. The decision on which function to invoke is based on rules on the current information content of the controller. The information content corresponds to:
1- the symbols generated by the NL parser that represent the current and the previous user's sentences
METHOD OF USING A NATURAL LANGUAGE INTERFACE TO
RETRIEVE INFORMATION FROM ONE OR MORE DATA RESOURCES
Technical Field The invention relates to a method for using a natural language query for retrieving information from one or more data resources and, more particularly, a method for using a natural language query for automatically retrieving and to configuring a destination telephone number from existing data resources and using the destination telephone number to place a telephone call.
Background of the Invention Files or other resources on computers around the world may be publicly 15 available to users of other computers through the collection of networks known as the Internet. The collection of all such publicly available resources, linked together using files written in Hypertext Mark-up Language ("HTML") is known as the World Wide Web ("web") A user of a computer that is connected to the Internet may cause a program 2o known as a client to request resources that are part of the web. Server programs then process the requests to return the specified resources. A standard naming convention has been adopted, known as a Uniform Resource Locator ("URL").
This convention encompasses several types of location names, presently including subclasses such as Hypertext Transport Protocol ("http"), File Transport Protocol 25 ("ftp"), gopher and.Wide Area Information Service ("WAIS") The various resources accessible via the web are created and maintained by many different people on servers located all around the world, and may be created for many different purposes. Many individuals and businesses now have their own web sites that can be visited by people "surfing" the web. These web sites typically 3o provide information on a myriad of subjects such as sports, business, news and even community events. For example, many web sites exist which provide useful information about a particular business establishment such as office locations, customer service telephone numbers and information about the products and/or services that the business offers to the consumer.
In many cases, an individual accessing such a web site is looking for particular information such as information about a particular product.
However, the individual might desire additional information about the particular product, which is not available from the web site. Typically, the individual will place a telephone call to the customer service department of the business entity to obtain the l0 additional information. For example, the individual may want to know if an electronics store carries a particular product, such as a particular brand name television and the price of the television. Such information may not be contained in the web site and the individual may have to call the electronics store to receive the desired information. This multiple step process of searching and retrieving 15 information from the Internet and then using a conventional phone line to call the business establishment for still more information or conducting business is cumbersome and time consuming.
In other cases, the individual may generally know the location of a particular establishment, but may not know the exact address or the name of the 2o establishment. For example, an individual may know that there is an electronics store in their town on Route 4, but may not know the name of the electronics store.
The individual may access a web site that contains information about the particular town to try to identify the exact name and location of the electronics store.
Conventional searching techniques require a hierarchical step by step query 25 approach to locate the desired information. For example, the individual may enter the address of the store, if known, or the type of store to get a listing of relevant information. However, such searching may require many steps and, in many instances, the available search queries may not be compatible with the known information. As such, the searching many require additional unnecessary search 3o queries in order to obtain the desired information.
Summary of the Invention In accordance with the present invention, a method of using at least one natural language query to retrieve information from one or more data resources and further performing a requested action using the retrieved information is 5 disclosed. At least one natural language query directed to retrieving particular information is received. At least one object from the natural language query is extracted. The relationship between each of the at least one extracted objects is determined. A semantic representation is created from the at least one extracted objects. A database query is generated based on the semantic 1o representation using a pre-defined knowledge structure. The query is applied to one or more of the data resources and information is retrieved. The requested action is then performed using the retrieved information.
In another embodiment of the present invention, the information retrieved is a telephone number for a particular destination and the requested 15 action is the placement of a telephone call using the retrieved telephone number.
In another embodiment of the present invention, the requested information is retrieved by querying multiple data resources in which each data resource contains a different type of information. Objects are extracted from 2o the natural language query which correspond to different types of information.
Each information type is associated with a particular data resource. A query is generated for each object and applied to the corresponding data resource. The retrieved information may be integrated into a query to a different data resource to retrieve additional information. The retrieved information may be 25 used to perform a requested action such as the placement of a telephone call.
In accordance with one aspect of the present invention there is provided a method of using at least one natural language query to retrieve information from an data resource and using said information to perform a requested action comprising: receiving at least one natural language query directed to 3o retrieving said information; creating one or more semantic representations from said natural language query; comparing said semantic tags to a 3a knowledge structure associated with said data resource; generating a database query based on said comparison; using said database query to retrieve said information from said data resource; and using said retrieved information to perform said requested action.
Brief Descriution of the Drawings In the drawings, where like numerals refer to like elements throughout the several views:
FIG. 1 is a simplified diagram of an exemplary system embodying the invention;
FIGs. 2a and 2b are flow charts illustrating the implementation of a query by the system of FIG. 1; and FIG. 3 is a flow chart illustrating the creation of a knowledge structure from a natural language query.
Detailed Description of the Invention For purposes of illustration, FIG. 1 is a simplified diagram of an exemplary system 100 embodying the invention. A user desires to establish a real-time connection to a particular establishment 130 (e.g., a telephone call to a representative of the establishment 130) by extracting relevant information from one or more data resources. In accordance with an embodiment of the present invention, the data resources are web sites identified by a particular URL. It is to be understood by those skilled in the art that the data resources can be any type of data file that may be accessed by the user over a packet network 108.
Using a personal computer (PC) 102, a user establishes a connection with packet network 108 via an access server 106. The user may also use a telephone 103 to connect to the packet network 108. Typically a modem connection (not shown) may be used to connect the PC 102 to the packet network 108 in a 2o conventional manner. The packet network 108 can be, for example, the Internet or an Intranet. The packet network 108 may comprise a single packet network or a multiplicity of packet networks, such as, e.g., the "backbone" networks comprising the Internet. The access server 106 may illustratively, be a server connected to the Internet provided by, e.g., an Internet service provider, or may be any other server used for providing access to the packet network 108.
As illustrated, the packet network 108 is connected to a plurality of information servers 110 which host a plurality of information services or web sites.
An information server 110 includes grammars that represent the language (written or spoken) used by customers for accessing the information related to that 3o particular service or site.
When the information server I 10 receives a natural language query, the information server 110 transmits the natural language query and the corresponding grammar to a service host 112 in a predefined protocol that characterizes the service host.
In general the service host 112 can communicate with a plurality of information servers, each one of them representing a particular service or site. Any information server that complies with the protocol defined by a service host can use the services provided by that service host.
The service host 112 coordinates with the information server 110, the 1 o associated data resources, and the natural language (NL) server 114 to process the natural language query. The service host 112 sends the natural language query and the specific grammars provided by the information server 110 to the NL server 114.
The NL server 114 parses the natural language query and sends the resulting semantic tags to the service host 112. When the service host 1 I2 receives the ~ 5 semantic tags, and if the service host 112 decides, based on the rules, that there is enough information to retrieve data from the data resources, generates a search query. The service host 112 uses ad-hoc transducers (a.k.a knowledge structures) to convert semantic tags into search queries (e.g. SQL queries) for each particular data resource. The knowledge structures are stored in the databases 118(a-n). The search 2o query is used to access data resources contained in, typically, a plurality of destination servers 116. It is to be understood by those skilled in the art that more than one data resource may be accessed from a single destination server, or alternatively, multiple data resources contained in different destination servers may be accessed at the same time. These destination servers may be dedicated to the 25 particular service host and/or publicly available and accessed through the packet network I 08 (not shown in Figure 1 ).
An example of a search query that may be generated is a request for a telephone number corresponding to a particular establishment. Once the appropriate telephone number of the establishment is retrieved call connection can 3o be provided by a standard telecom adjunct at the service host 112 i.e., the call to the telephone 122 of the establishment 130 is initiated. Once a connection is established with the called party (e.g., the establishment 130) a direct connection is established between the user and the called party (e.g., employee of establishment 130). Alternatively, instead of a telephone call, the user can connect to the establishment's facsimile machine 124 or personal computer 126.
A more detailed example of how the present invention may be implemented is set forth in FIGS. 2a and 2b. A user may access a particular data resource such as a directory for the town of Westfield by inputting the URL
http://www.westfield.com (step 202). The URL for that Westfield data resource is 1 o inputted into PC 102 either by typing the request using a keyboard 104 or by speaking the request into a microphone 1 O5. Alternatively the information server 110 can be directly accessed through a telephone 103: ( 1 ) an Internet telephone connected to the access server (2) a conventional PSTN telephone connected to the information server assuming that the information server has a telecom adjunct and the capability of processing spoken requests.
Spoken requests either from a PC microphone 105 or from a telephone 103 can be handled by a speech recognition system residing at the information server.
The PC 102 dials into an access server 106 that is connected to the Internet or other database service via a logical network interface (not shown). The logical 2o network interface may be a local area network (LAN), a Serial Line Internet Protocol (SLIP) connection over a modem, an ISDN port or via a connection to a special LAN such as an ATM LAN or a LAN that offers bandwidth reservation.
The invention is independent of the actual modality of call placement. As mentioned before, call placement can occur from a PSTN (made possible by means of a telecom adjunct at the server) or Internet telephone.
In general, the information server that is accessed can either be a specific web site, such as, but not limited to the web site of a company, or a dedicated information resource web site. The dedicated information resource web site can be, for example, a single web site or a combination of web sites which contains a 3o significant amount of general reference information which relates to a particular subject matter, such as a national directory of movie information. A web site can also be dedicated to handle a particular subject matter. For example, a web site can contain movie and theater information for all movie theaters in the United States.
The information server to be accessed may be characterized by a URL and/or a telephone number.
Once an information server is accessed, the user can send a text or a spoken query requesting a particular action or service (step 204), for example: "call the pizza place on Main Street in Westfield". The query is received by the access server 106 and the natural language query is sent to the information server 110 via packet I o network 108. It is to be understood that the packet network 108 may be connected to a plurality of information servers which each relate to one or more particular information services, or there may be a single centralized information server which is accessed by all information services which are capable of receiving and processing natural language queries. The information server 110 is preferably located in one or more adjunct servers. Each information server 110 contains at least some of the data resources (e.g., URLs and associated site/service-specific grammars) capable of receiving and responding to a natural language query.
If an information server is enhanced for voice access, -standard speech application programming interfaces may be used to provide a means for 2o communicating between the user and the web site. Information servers include grammars that reflect the language used by people to request or to describe information services specific to that particular site. The grammar should try to capture all possible reasonable sentences or queries.
Once the information server receives the user's query, it retrieves the relevant grammars for processing the query (step 206). Following this, a communication between the information server 110 and the service host 112 will be established. The information server will send the user's query and the retrieved grammars to the service host (step 208). The service host 112 coordinates the activities of the information server 110 with a NL server 114 and at least one 3o destination server 116 to process the natural language query and perform the *rB
requested action. The service host 112 includes a dialog control program that manages interactions with users over several turns (e.g., it decides when to ask a question, when to give an answer, provides means for clarifying ambiguities, and provides error control and recovery during an interaction). The service host then must determine what type of information is being requested as expressed in the user's natural language query. For example, is the query requiring an action, such as the placement of a telephone call, or requesting specific information, such as the location at which a particular movie is being shown. To parse the natural language query the service host uses a natural language server NL 114: the query and the to grammar are sent to NL (step 210). An NL server typically embodies a parser, an example of which is described in Pieraccini, R., Levin, E., "A Spontaneous-Speech Understanding System for Database Query Applications," ESCA Workshop on Spoken Dialogue Systems - Theories and Applications, May 30, 3une 2, 1995, Vigs , Denmark, which is incorporated by reference. The semantic representations are then received by the service host 112.
The NL server 114 parses the natural language query into a plurality of semantic tags (step 212) as will be described in detail hereinafter. The semantic tags are then sent from the NL server 114 to the service host 112 (step 214).
The service host 112 converts the semantic tags to a string of logical search 2o queries (step 216). The conversion of the semantic tags into a search query is accomplished by a knowledge structure which provides a mapping between the semantic tags and the language used for querying the database in the selected application (e.g., SQL). This mapping is performed by an ad-hoc program developed for the particular application. The resulting knowledge structure is stored in databases 118. Standard techniques may be used for developing the ad-hoc program. For the example if a standard SQL database is used, the mapping will look like:
Input: Action: Call / Action Object: Pizza Restaurant / Location: Main Street / City: Westfield Output: SELECT telephone number FROM pizza restaurants WHERE
location--'Main Street' and 'city='Westfieid';
It is obvious that Call should map into the attribute telephone number, Pizza Restaurant to pizza restaurants, etc.
To summarize, a grammar (either handcrafted or automatically acquired using machine learning algorithms) is used to parse the natural language query to obtain a set of semantic tags that represent the query. The query can be received either as written text or can be translated from speech to text via a speech recognition system. For the query "call the pizza place on Main Street in to Westfield", the parser generates a semantic representation such as "Action:
Call /
Action Object: Pizza Restaurant / Location: Main Street / City: Westfield".
The semantic representation can then be used for generating a search query (e.g., in SQL) that will retrieve the corresponding information. In the case of the exemplary query, the telephone number for the particular pizza restaurant is retrieved from the data resource. An example of a system which is capable of processing such a query is described in Pieraccini, R., Levin, E., "A Spontaneous-Speech Understanding System for Database Query Applications," ESCA Workshop on Spoken Dialogue Systems - Theories and Applications, May 30, June 2, 1995, Vigs , Denmark which is incorporated by reference.
2o The system can implement a dialog control as that described in Pieraccini, R., Levin, E., Eckert, W., "AMICA, the AT&T Mixed Initiative Conversational Architecture," Proc. EUROSPEECH 97, September 1997, Rhodes, Greece which is also incorporated by reference. The above symbolic description of the natural language query is then analyzed according to a predefined set of rules. The rules have to be defined ad-hoc for each application. For instance in the pizza restaurant application there might be rules that say:
If (Action Object=Pizza Restaurant) then User must provide: Location and City So, For instance, if the rule is activated (i.e. when Action Object=Pizza Restaurant), and the user had given the Location already, the system will request the City.
If the service host 112, based on the rules, decides that there is enough information for performing a database access, the database query is generated.
The database query is generally in one of the standard query languages (e.g. SQL).
The service host 112 will have a table that maps topics to URL's, so it knows that if topic=Pizza Restaurant, the URL to send the query to is www.njrestaurants.com.
The search queries are forwarded to the web site associated with the URL
1o transmitted by the user that is located at one of the destination servers 116 (step 218). The web sites process the query and retrieve data that is believed to respond to the query (step 220). The query results are transmitted to the service host 112.
'the service host 112 determines if there are any ambiguities with respect to the response (step 222) and, if so, forwards additional queries to the user to help to resolve the ambiguities (step 224). The service host 112 then sends the responses to the information server 110 (step 226).
If there are too many potential answers (for instance if there are two pizza places on Main Street in Westfield), one or more questions to the user are generated in order to disambiguate the query (e.g. Do you mean "Venezia" or "Bella Roma?"). The answers to the additional questions are used to formulate a new logical search query.
For this there might be additional rules like:
If(Action Object=Pizza Restaurant and Too-Many Answers) then User must provide further clarifying information such as, for example, the name of restaurant OR exact address.
If the user does not provide enough information to achieve a single answer, the service host 112 might then list the possibilities and ask the user to chose one of them.
Parallel search queries can be launched from the original natural language 3o query or parent query in order to fill in information that is missing from the original query. Typically such parallel queries are added by using a logical AND/OR relation. For example, the query "call ITALIAN RESTAURANT in WESTFIELD NEAR the INTERSECTION of EAST and BROAD STREET" can be segmented into two parallel searches queries. The first search query is directed to accessing a restaurant directory to list all Italian restaurants in Westfield. The second parallel search query is directed to accessing a street database to find all streets near the intersection.
Once the service host 112 determines that a single response matches the database query, the requested information is retrieved from the database. In the 1 o case of the present example, the appropriate telephone number of the calling party is retrieved and confirmed using dialog control. The information (i.e., phone number) is received by the service host 112 which then forwards the number to the information server 110 (step 226). The service host 112 then performs the desired action (step 228). In the above example, the service host would establish a telephone connection between the pizza restaurant and the user.
Call connection can be provided by a standard telecom adjunct at the service host, i.e., the call to the telephone 122 of the pizza restaurant is initiated. Once a connection is established with the called party (e.g., the pizza restaurant) a direct connection is established between the user and the called party (e.g., pizza 2o restaurant owner). Alternatively, instead of a telephone call, the user can connect to the called party's facsimile machine 124 or personal computer 126.
In an alternative embodiment, the user may input the natural language query using a telephone. In such a case, the user dials a telephone number to gain access to a particular Internet resource and speaks the natural language query, e.g., "call the pizza place on Main Street in Westfield." The query is received by a speech recognizes 107 connected to the information server 110. An example of a speech recognizes which may be used is disclosed in L. Rabiner and B-H.Juang, "Fundamentals of Speech Recognition" Prentice Hall, Englewood Cliffs, NJ, 1993, which may be incorporated by reference. The speech recognizes translates a spoken 3o query into the corresponding text for and returns it to the information server.
Described herein are further details of the functional aspects of the service host 112. In addition, to providing communication routing, the service host provides controller functionality. An integral functional component of the service host is a controller that uses the semantic representations obtained from the NL
server to create a knowledge structure. The controller acts upon a strategy based on a sequence of operations (actions) and rules (see reference (2)). The sequence of operations determines which of a finite number of predetermined functions is invoked at any step of the processing. One of the predetermined functions that was mentioned previously is the interfacing with the natural language server to obtain 1o a symbolic representation of the user's query i.e., semantic tagging. The other functions include interfacing with a speech generation mechanism to produce a spoken question or answer, and performing a search query to a database using the semantic tags obtained from the natural language server. The decision on which function to invoke is based on rules on the current information content of the controller. The information content corresponds to:
1- the symbols generated by the NL parser that represent the current and the previous user's sentences
2- the information extracted from the databases)
3- information generated by the controller itself (for instance the count 2o of how many times a question is asked) Each semantic representation is identified as a particular object. The objects are then implemented by one or more knowledge structures which represent an intelligent search strategy that is used to interpret the query and ultimately retrieve the desired information from one or more data resources to perform a desired action. The knowledge structures are stored in one or more databases 118a, 118b, 118n. The knowledge structures are comprised of a plurality of objects. Each object is placed in a class and defined by unique properties such as, but not limited to, location, color, and size. This information serves as an identifier tag for the object. A class is a broad identifier that can describe a group of objects that share some attributes. For example, a class may be restaurants which would include all eateries associated with a particular data resource.
The objects are further arranged in different categories based on the inherent attributes of the particular object. Examples of such categories include, but are not limited to, callable objects, landmark objects, and functional objects.
Callable objects are objects associated with telephone numbers (and/or fax, email) such as, but not limited to, people and businesses i.e., objects upon which an action such as calling or messaging can be performed. Callable objects are organized into classes in a hierarchical database. Examples of callable objects are to restaurants, stores, services, entertainment and churches. Each callable object has a knowledge structure associated with it. An example of a knowledge structure for a restaurant is illustrated below:
<restaurant> -> (diner, American restaurant, fast food, Italian, Chinese) <restaurant> -> (expensive, low price, ~
<restaurant> -> (drive through, take out, formal, ~
<restaurant> -> (name, address) <restaurant> -> (telephone number, fax number, email address) The callable objects are further arranged in a hierarchy that results in the knowledge structure producing a description of the particular callable object that is 2o used to retrieve the desired information from one or more data resources.
An example of a knowledge structure representing a hierarchy of information relating to a particular callable object is shown as follows:
<Italian> IS A <restaurant,> (where IS A specifies inheritance) Now, <Italian> is also associated with attributes such as food types, for example, <Italian> -> (pizza, pasta, general food) Such an organization of objects results in mufti-level inheritance. For example, _> <pizza> IS A <Italian> IS A <restaurant> (two level inheritance) Hence, when a query regarding "pizza" places is made it will automatically include all Italian restaurants as well, all though many of them may not explicitly advertise 3o themselves as pizza restaurants.
The second category of objects are landmark objects. Landmark objects are objects that relate typically in a geographic manner to the desired information but are provided for a contextual basis. Examples of landmark objects include, but are not limited to, streets, parks, ponds, monuments, and important buildings.
Another category of objects are functional objects. Functional objects determine the relationships between one or more objects, such as between callable objects and landmark objects or between two callable objects or two landmark objects. Examples of such functions are "next to", " across from" and "north".
All of these objects are stored in one or more databases. An example of a knowledge to structure that may be used to interpret functional objects is as follows:
<CIblObj> [near] [intersection] OF <South> STREET and <north>
AVENUE
In the above example the function [intersection(<streetl>, <street2>)]
determines geographical coordinates of the intersection of two streets.
Similarly we can define functions denoting proximity, such as "near". For example, the function [near(geogr_coordl, geogr_coordl>)] gives [near] value, perhaps ranging between (0. To 1.) to provide a relative measure of proximity.
AN EXAMPLE USING MULTIPLE RESOURCES
A user calls a toll free number that offers information services for the entire 2o state. The information server component of this service as a basic feature would be configured with names of towns, counties, and other landmark locations.
Further, its grammar would embody general concepts about various information resources:
restaurants, movie theaters, transportation, banks. Notice at information server level, the grammar contains no details of any specific information resource i.e., it knows about "banks, financial institutions, credit unions" and not specific names such as "Fleet Bank, Affinity Credit Union etc.". Similarly, for restaurants the language model embodies something like:
<restaurant> -> (diner, American restaurant, fast food, Italian, Chinese) <restaurant> -> (expensive, low price, ~
<restaurant> -> (drive through, take out, formal, ~
while not including specifics such as, <restaurant> -> {name, address) <restaurant> -> (telephone number, fax number, email address) These kinds of specific information will be retrieved from multiple resources 5 during consecutive actions of the search, real-time, when the query is made.
The reason for this strategy is because these types of information typically (1) are contained in several different database resources (restaurant resources, maps database, financial information resources etc.) (2) tend to change with time and need to be current at the time of query.
to Example 1:
Query: Can you help me locate a pizza place near the intersection of Main and Broad Streets in Westfield?
One way of processing the above query is as follows and is illustrated in FIG.
3.
The first step is transcription of the above query into text form using automatic 15 speech recognition, if it is a spoken utterance (step 302). In the second step the semantic analyzer identifies key concepts such as <pizza place>, <intersection>, <Main Street>, <Broad Street>, <Westfleld> (step 304). In the third step, based on matches with the initial language model information as described above, the town name <Westfield>, <pizza> IS A <restaurant>, <streets> IS A <location> will be 2o identified (step 306). Given that, this will trigger real-time access of the databases (step 308):
( 1 ) Restaurant database, with the retrieval restricted by town name <Westfield>, restaurant type <pizza> and generate all Italian restaurants in Westfield (2) A street map database for Westfield and generate geographical coordinates for intersection of main and broad streets In the fourth step, the addresses from the restaurant database (from ( 1 ) above) is retrieved and mapped with the street map database (from (2) above) (step 310).
Finally, the results of merging these two information sources using the functional objects <near> and <intersection> will result in a ranked list of possible Italian restaurants near the intersection of main and broad streets in Westfield (step 312).
In the fifth step, all the telephone numbers of those restaurants will be accessed from the restaurant database (step 314). Finally, the dialog control manager after confirming with user (and getting the choice of selection if there are more than one selection as a result of the query) will complete the call to the selected restaurant (step 316).
Example 2:
Query: Can you connect me to the loan o~cer of the Fleet Bank in Westfreld?
Like in the previous example, the initial concepts of <bank>, <Westfield> will be 1 o identified. The bank database for the state will be first retrieved and in turn information on Fleet Bank in Westfleld will be retrieved.
Assuming that Fleet Bank has its own website (perhaps with enhanced voice dialing features) the next step will retrieve the appropriate loan officer telephone number for the Fleet Bank in Westfield and complete the telephone call.
(Note dialog control will facilitate the user with confirmations etc. before actual call connection is made).
While the present invention has been described in connection with the illustrated embodiments, it will be appreciated and understood that modifications may be made without departing from the true spirit and scope of the invention.
The objects are further arranged in different categories based on the inherent attributes of the particular object. Examples of such categories include, but are not limited to, callable objects, landmark objects, and functional objects.
Callable objects are objects associated with telephone numbers (and/or fax, email) such as, but not limited to, people and businesses i.e., objects upon which an action such as calling or messaging can be performed. Callable objects are organized into classes in a hierarchical database. Examples of callable objects are to restaurants, stores, services, entertainment and churches. Each callable object has a knowledge structure associated with it. An example of a knowledge structure for a restaurant is illustrated below:
<restaurant> -> (diner, American restaurant, fast food, Italian, Chinese) <restaurant> -> (expensive, low price, ~
<restaurant> -> (drive through, take out, formal, ~
<restaurant> -> (name, address) <restaurant> -> (telephone number, fax number, email address) The callable objects are further arranged in a hierarchy that results in the knowledge structure producing a description of the particular callable object that is 2o used to retrieve the desired information from one or more data resources.
An example of a knowledge structure representing a hierarchy of information relating to a particular callable object is shown as follows:
<Italian> IS A <restaurant,> (where IS A specifies inheritance) Now, <Italian> is also associated with attributes such as food types, for example, <Italian> -> (pizza, pasta, general food) Such an organization of objects results in mufti-level inheritance. For example, _> <pizza> IS A <Italian> IS A <restaurant> (two level inheritance) Hence, when a query regarding "pizza" places is made it will automatically include all Italian restaurants as well, all though many of them may not explicitly advertise 3o themselves as pizza restaurants.
The second category of objects are landmark objects. Landmark objects are objects that relate typically in a geographic manner to the desired information but are provided for a contextual basis. Examples of landmark objects include, but are not limited to, streets, parks, ponds, monuments, and important buildings.
Another category of objects are functional objects. Functional objects determine the relationships between one or more objects, such as between callable objects and landmark objects or between two callable objects or two landmark objects. Examples of such functions are "next to", " across from" and "north".
All of these objects are stored in one or more databases. An example of a knowledge to structure that may be used to interpret functional objects is as follows:
<CIblObj> [near] [intersection] OF <South> STREET and <north>
AVENUE
In the above example the function [intersection(<streetl>, <street2>)]
determines geographical coordinates of the intersection of two streets.
Similarly we can define functions denoting proximity, such as "near". For example, the function [near(geogr_coordl, geogr_coordl>)] gives [near] value, perhaps ranging between (0. To 1.) to provide a relative measure of proximity.
AN EXAMPLE USING MULTIPLE RESOURCES
A user calls a toll free number that offers information services for the entire 2o state. The information server component of this service as a basic feature would be configured with names of towns, counties, and other landmark locations.
Further, its grammar would embody general concepts about various information resources:
restaurants, movie theaters, transportation, banks. Notice at information server level, the grammar contains no details of any specific information resource i.e., it knows about "banks, financial institutions, credit unions" and not specific names such as "Fleet Bank, Affinity Credit Union etc.". Similarly, for restaurants the language model embodies something like:
<restaurant> -> (diner, American restaurant, fast food, Italian, Chinese) <restaurant> -> (expensive, low price, ~
<restaurant> -> (drive through, take out, formal, ~
while not including specifics such as, <restaurant> -> {name, address) <restaurant> -> (telephone number, fax number, email address) These kinds of specific information will be retrieved from multiple resources 5 during consecutive actions of the search, real-time, when the query is made.
The reason for this strategy is because these types of information typically (1) are contained in several different database resources (restaurant resources, maps database, financial information resources etc.) (2) tend to change with time and need to be current at the time of query.
to Example 1:
Query: Can you help me locate a pizza place near the intersection of Main and Broad Streets in Westfield?
One way of processing the above query is as follows and is illustrated in FIG.
3.
The first step is transcription of the above query into text form using automatic 15 speech recognition, if it is a spoken utterance (step 302). In the second step the semantic analyzer identifies key concepts such as <pizza place>, <intersection>, <Main Street>, <Broad Street>, <Westfleld> (step 304). In the third step, based on matches with the initial language model information as described above, the town name <Westfield>, <pizza> IS A <restaurant>, <streets> IS A <location> will be 2o identified (step 306). Given that, this will trigger real-time access of the databases (step 308):
( 1 ) Restaurant database, with the retrieval restricted by town name <Westfield>, restaurant type <pizza> and generate all Italian restaurants in Westfield (2) A street map database for Westfield and generate geographical coordinates for intersection of main and broad streets In the fourth step, the addresses from the restaurant database (from ( 1 ) above) is retrieved and mapped with the street map database (from (2) above) (step 310).
Finally, the results of merging these two information sources using the functional objects <near> and <intersection> will result in a ranked list of possible Italian restaurants near the intersection of main and broad streets in Westfield (step 312).
In the fifth step, all the telephone numbers of those restaurants will be accessed from the restaurant database (step 314). Finally, the dialog control manager after confirming with user (and getting the choice of selection if there are more than one selection as a result of the query) will complete the call to the selected restaurant (step 316).
Example 2:
Query: Can you connect me to the loan o~cer of the Fleet Bank in Westfreld?
Like in the previous example, the initial concepts of <bank>, <Westfield> will be 1 o identified. The bank database for the state will be first retrieved and in turn information on Fleet Bank in Westfleld will be retrieved.
Assuming that Fleet Bank has its own website (perhaps with enhanced voice dialing features) the next step will retrieve the appropriate loan officer telephone number for the Fleet Bank in Westfield and complete the telephone call.
(Note dialog control will facilitate the user with confirmations etc. before actual call connection is made).
While the present invention has been described in connection with the illustrated embodiments, it will be appreciated and understood that modifications may be made without departing from the true spirit and scope of the invention.
Claims (21)
1. A method of using at least one natural language query to retrieve information from an data resource and using said information to perform a requested action comprising:
receiving at least one natural language query directed to retrieving said information;
creating one or more semantic representations from said natural language query;
comparing said semantic tags to a knowledge structure associated with said data resource;
generating a database query based on said comparison;
using said database query to retrieve said information from said data resource; and using said retrieved information to perform said requested action.
receiving at least one natural language query directed to retrieving said information;
creating one or more semantic representations from said natural language query;
comparing said semantic tags to a knowledge structure associated with said data resource;
generating a database query based on said comparison;
using said database query to retrieve said information from said data resource; and using said retrieved information to perform said requested action.
2. The method of claim 1 wherein said step of creating one or more semantic representations further comprises:
extracting at least one object from the natural language query; and comparing the at least one object to a knowledge structure, the knowledge structure comprised of objects categorized from a plurality of data resources.
extracting at least one object from the natural language query; and comparing the at least one object to a knowledge structure, the knowledge structure comprised of objects categorized from a plurality of data resources.
3. The method of claim 1 wherein said information is a telephone number.
4. The method of claim 3 wherein said requested action is the placement of a call to a particular destination using said telephone number.
5. The method of claim 3 wherein said requested action is the sending of a facsimile transmission to a particular destination using said telephone number.
6. The method of claim 1 wherein said data resource is an Internet data resource.
7. The method of claim 1 wherein said natural language query is in text form.
8. The method of claim 1 wherein said natural language query is received as speech.
9. A method of retrieving a telephone number for a particular destination from a data resource and using said telephone number to place a call to the particular destination comprising:
receiving at least one natural language query directed to retrieving a telephone number for a particular destination;
extracting at least one object from the natural language query;
creating one or more semantic representations from the at least one extracted objects;
comparing the one or more semantic representations to a knowledge structure, said knowledge structure comprised of objects categorized from a plurality of data resources;
generating a search query based on said comparison;
transmitting said search query to said data resources;
matching the search query to one or more callable objects, at least one of said callable objects being associated with a telephone number; and placing a telephone call to the destination associated with the matched callable object.
receiving at least one natural language query directed to retrieving a telephone number for a particular destination;
extracting at least one object from the natural language query;
creating one or more semantic representations from the at least one extracted objects;
comparing the one or more semantic representations to a knowledge structure, said knowledge structure comprised of objects categorized from a plurality of data resources;
generating a search query based on said comparison;
transmitting said search query to said data resources;
matching the search query to one or more callable objects, at least one of said callable objects being associated with a telephone number; and placing a telephone call to the destination associated with the matched callable object.
10. The method of claim 9 wherein said data resource is an Internet data resource.
11. The method of claim 9 wherein said natural language query is in text form.
12. The method of claim 9 wherein said natural language query is received as speech.
13. The method of claim 9 wherein said natural language query is generated by a personal computer.
14. The method of claim 13 further comprising the step of:
establishing a connection between said personal computer and said destination.
establishing a connection between said personal computer and said destination.
15. The method of claim 13 wherein said step of matching the search query to one or more callable objects further comprising the steps of:
determining if more than one match arises from said matching step, and if more than one match occurs generating one or more queries which are transmitted to the personal computer.
determining if more than one match arises from said matching step, and if more than one match occurs generating one or more queries which are transmitted to the personal computer.
16. The method of claim 15 further comprising the steps of:
receiving additional information from said personal computer in response to said one or more queries, and using said additional information to eliminate at least some of the matches.
receiving additional information from said personal computer in response to said one or more queries, and using said additional information to eliminate at least some of the matches.
17. A method of using at least one natural language query to receive information from a plurality of data resources, each data resource comprising a particular type of information comprising:
receiving at least one natural language query;
extracting one or more semantic representations from said query;
categorizing each semantic representation by type of information;
matching each semantic representation to one or more elements of a knowledge structure based on type of information;
using said matching to develop a database query;
applying each database query to the plurality of data resources; and retrieving the requested information.
receiving at least one natural language query;
extracting one or more semantic representations from said query;
categorizing each semantic representation by type of information;
matching each semantic representation to one or more elements of a knowledge structure based on type of information;
using said matching to develop a database query;
applying each database query to the plurality of data resources; and retrieving the requested information.
18. The method of claim 17 wherein said step of developing a database query further comprises:
identifying information extracted from a data resource in response to a previous database query; and integrating said extracted information with the semantic representations to generate a new database query.
identifying information extracted from a data resource in response to a previous database query; and integrating said extracted information with the semantic representations to generate a new database query.
19. The method of claim 17 further comprising the step of:
using said retrieved information to perform a requested action.
using said retrieved information to perform a requested action.
20. The method of claim 19 wherein said retrieved information is a telephone number and said requested action is initiation of a telephone call using said telephone number.
21. The method of claim 19 wherein said retrieved information is a telephone number and said requested action is initiation of a facsimile transmission using said telephone number.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/058,107 | 1998-04-09 | ||
US09/058,107 US6173279B1 (en) | 1998-04-09 | 1998-04-09 | Method of using a natural language interface to retrieve information from one or more data resources |
PCT/US1999/007278 WO1999053676A1 (en) | 1998-04-09 | 1999-04-01 | Method of using a natural language interface to retrieve information from one or more data resources |
Publications (2)
Publication Number | Publication Date |
---|---|
CA2293780A1 CA2293780A1 (en) | 1999-10-21 |
CA2293780C true CA2293780C (en) | 2002-08-20 |
Family
ID=22014726
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA002293780A Expired - Lifetime CA2293780C (en) | 1998-04-09 | 1999-04-01 | Method of using a natural language interface to retrieve information from one or more data resources |
Country Status (3)
Country | Link |
---|---|
US (1) | US6173279B1 (en) |
CA (1) | CA2293780C (en) |
WO (1) | WO1999053676A1 (en) |
Families Citing this family (353)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5995921A (en) * | 1996-04-23 | 1999-11-30 | International Business Machines Corporation | Natural language help interface |
US6779358B2 (en) * | 1997-12-30 | 2004-08-24 | International Water Makers, Inc. | Water collection and dispensing machine |
US6173279B1 (en) | 1998-04-09 | 2001-01-09 | At&T Corp. | Method of using a natural language interface to retrieve information from one or more data resources |
US6404877B1 (en) * | 1998-06-02 | 2002-06-11 | At&T Corp. | Automated toll-free telecommunications information service and apparatus |
US6405159B2 (en) * | 1998-06-03 | 2002-06-11 | Sbc Technology Resources, Inc. | Method for categorizing, describing and modeling types of system users |
DE19837102A1 (en) * | 1998-08-17 | 2000-02-24 | Philips Corp Intellectual Pty | Method and arrangement for carrying out a database query |
US6694482B1 (en) * | 1998-09-11 | 2004-02-17 | Sbc Technology Resources, Inc. | System and methods for an architectural framework for design of an adaptive, personalized, interactive content delivery system |
WO2000033226A1 (en) * | 1998-11-30 | 2000-06-08 | Siebel Systems, Inc. | Smart scripting call centers |
US6965925B1 (en) * | 1998-12-31 | 2005-11-15 | Nortel Networks, Ltd | Distributed open architecture for media and telephony services |
US6742021B1 (en) * | 1999-01-05 | 2004-05-25 | Sri International, Inc. | Navigating network-based electronic information using spoken input with multimodal error feedback |
US6513063B1 (en) * | 1999-01-05 | 2003-01-28 | Sri International | Accessing network-based electronic information through scripted online interfaces using spoken input |
US7036128B1 (en) | 1999-01-05 | 2006-04-25 | Sri International Offices | Using a community of distributed electronic agents to support a highly mobile, ambient computing environment |
US6757718B1 (en) * | 1999-01-05 | 2004-06-29 | Sri International | Mobile navigation of network-based electronic information using spoken input |
US6851115B1 (en) | 1999-01-05 | 2005-02-01 | Sri International | Software-based architecture for communication and cooperation among distributed electronic agents |
US6480860B1 (en) * | 1999-02-11 | 2002-11-12 | International Business Machines Corporation | Tagged markup language interface with document type definition to access data in object oriented database |
US6223165B1 (en) * | 1999-03-22 | 2001-04-24 | Keen.Com, Incorporated | Method and apparatus to connect consumer to expert |
US6233561B1 (en) * | 1999-04-12 | 2001-05-15 | Matsushita Electric Industrial Co., Ltd. | Method for goal-oriented speech translation in hand-held devices using meaning extraction and dialogue |
US6604075B1 (en) * | 1999-05-20 | 2003-08-05 | Lucent Technologies Inc. | Web-based voice dialog interface |
US7086007B1 (en) * | 1999-05-27 | 2006-08-01 | Sbc Technology Resources, Inc. | Method for integrating user models to interface design |
US7224790B1 (en) * | 1999-05-27 | 2007-05-29 | Sbc Technology Resources, Inc. | Method to identify and categorize customer's goals and behaviors within a customer service center environment |
US8065155B1 (en) | 1999-06-10 | 2011-11-22 | Gazdzinski Robert F | Adaptive advertising apparatus and methods |
US6912691B1 (en) * | 1999-09-03 | 2005-06-28 | Cisco Technology, Inc. | Delivering voice portal services using an XML voice-enabled web server |
JP4642953B2 (en) * | 1999-09-09 | 2011-03-02 | クラリオン株式会社 | Voice search device and voice recognition navigation device |
US6601026B2 (en) * | 1999-09-17 | 2003-07-29 | Discern Communications, Inc. | Information retrieval by natural language querying |
US6665666B1 (en) * | 1999-10-26 | 2003-12-16 | International Business Machines Corporation | System, method and program product for answering questions using a search engine |
US7392185B2 (en) * | 1999-11-12 | 2008-06-24 | Phoenix Solutions, Inc. | Speech based learning/training system using semantic decoding |
US7050977B1 (en) * | 1999-11-12 | 2006-05-23 | Phoenix Solutions, Inc. | Speech-enabled server for internet website and method |
US9076448B2 (en) | 1999-11-12 | 2015-07-07 | Nuance Communications, Inc. | Distributed real time speech recognition system |
US7725307B2 (en) * | 1999-11-12 | 2010-05-25 | Phoenix Solutions, Inc. | Query engine for processing voice based queries including semantic decoding |
AU2073401A (en) * | 1999-12-10 | 2001-06-18 | Myteam.Com, Inc. | Tools for administering leagues and accessing and populating a community websitestructure |
US6587849B1 (en) * | 1999-12-10 | 2003-07-01 | Art Technology Group, Inc. | Method and system for constructing personalized result sets |
US6988276B2 (en) * | 1999-12-14 | 2006-01-17 | Koninklijke Philips Electronics N.V. | In-house TV to TV channel peeking |
US7177316B1 (en) * | 1999-12-20 | 2007-02-13 | Avaya Technology Corp. | Methods and devices for providing links to experts |
US6829603B1 (en) * | 2000-02-02 | 2004-12-07 | International Business Machines Corp. | System, method and program product for interactive natural dialog |
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US6510417B1 (en) * | 2000-03-21 | 2003-01-21 | America Online, Inc. | System and method for voice access to internet-based information |
US20040006473A1 (en) * | 2002-07-02 | 2004-01-08 | Sbc Technology Resources, Inc. | Method and system for automated categorization of statements |
US7213027B1 (en) | 2000-03-21 | 2007-05-01 | Aol Llc | System and method for the transformation and canonicalization of semantically structured data |
US6778643B1 (en) | 2000-03-21 | 2004-08-17 | Sbc Technology Resources, Inc. | Interface and method of designing an interface |
US6868380B2 (en) | 2000-03-24 | 2005-03-15 | Eliza Corporation | Speech recognition system and method for generating phonotic estimates |
US7370086B2 (en) | 2000-03-24 | 2008-05-06 | Eliza Corporation | Web-based speech recognition with scripting and semantic objects |
US7366766B2 (en) * | 2000-03-24 | 2008-04-29 | Eliza Corporation | Web-based speech recognition with scripting and semantic objects |
DE60130880T2 (en) | 2000-03-24 | 2008-07-17 | Eliza Corp., Beverly | WEB-BASED VOICE RECOGNITION BY SCRIPTING AND SEMANTIC OBJECTS |
US7606706B1 (en) * | 2000-04-20 | 2009-10-20 | Rubin Aviel D | System and method for storage and retrieval of personal communications in a broadband network |
US7962326B2 (en) * | 2000-04-20 | 2011-06-14 | Invention Machine Corporation | Semantic answering system and method |
US6604101B1 (en) * | 2000-06-28 | 2003-08-05 | Qnaturally Systems, Inc. | Method and system for translingual translation of query and search and retrieval of multilingual information on a computer network |
FI109952B (en) * | 2000-06-06 | 2002-10-31 | Sonera Oyj | Provision of services in a telecommunications network |
US7047196B2 (en) * | 2000-06-08 | 2006-05-16 | Agiletv Corporation | System and method of voice recognition near a wireline node of a network supporting cable television and/or video delivery |
US6625595B1 (en) * | 2000-07-05 | 2003-09-23 | Bellsouth Intellectual Property Corporation | Method and system for selectively presenting database results in an information retrieval system |
US20020059204A1 (en) * | 2000-07-28 | 2002-05-16 | Harris Larry R. | Distributed search system and method |
DE60125597T2 (en) * | 2000-08-31 | 2007-05-03 | Hitachi, Ltd. | Device for the provision of services |
US6636590B1 (en) * | 2000-10-30 | 2003-10-21 | Ingenio, Inc. | Apparatus and method for specifying and obtaining services through voice commands |
WO2002041169A1 (en) * | 2000-11-17 | 2002-05-23 | Invention Machine Corporation, Inc. | Semantic answering system and method |
US7289623B2 (en) | 2001-01-16 | 2007-10-30 | Utbk, Inc. | System and method for an online speaker patch-through |
US8095370B2 (en) * | 2001-02-16 | 2012-01-10 | Agiletv Corporation | Dual compression voice recordation non-repudiation system |
US7013280B2 (en) * | 2001-02-27 | 2006-03-14 | International Business Machines Corporation | Disambiguation method and system for a voice activated directory assistance system |
US20020133402A1 (en) * | 2001-03-13 | 2002-09-19 | Scott Faber | Apparatus and method for recruiting, communicating with, and paying participants of interactive advertising |
US7072838B1 (en) * | 2001-03-20 | 2006-07-04 | Nuance Communications, Inc. | Method and apparatus for improving human-machine dialogs using language models learned automatically from personalized data |
US6721737B2 (en) * | 2001-04-04 | 2004-04-13 | International Business Machines Corporation | Method of ranking items using efficient queries |
US6904428B2 (en) * | 2001-04-18 | 2005-06-07 | Illinois Institute Of Technology | Intranet mediator |
US7716235B2 (en) * | 2001-04-20 | 2010-05-11 | Yahoo! Inc. | Phonetic self-improving search engine |
US20030023431A1 (en) * | 2001-07-26 | 2003-01-30 | Marc Neuberger | Method and system for augmenting grammars in distributed voice browsing |
US7065201B2 (en) * | 2001-07-31 | 2006-06-20 | Sbc Technology Resources, Inc. | Telephone call processing in an interactive voice response call management system |
US20030061405A1 (en) * | 2001-08-15 | 2003-03-27 | Open Technologies Group, Inc. | System, method and computer program product for protocol-independent processing of information in an enterprise integration application |
US6704403B2 (en) | 2001-09-05 | 2004-03-09 | Ingenio, Inc. | Apparatus and method for ensuring a real-time connection between users and selected service provider using voice mail |
ITFI20010199A1 (en) | 2001-10-22 | 2003-04-22 | Riccardo Vieri | SYSTEM AND METHOD TO TRANSFORM TEXTUAL COMMUNICATIONS INTO VOICE AND SEND THEM WITH AN INTERNET CONNECTION TO ANY TELEPHONE SYSTEM |
US7580850B2 (en) | 2001-12-14 | 2009-08-25 | Utbk, Inc. | Apparatus and method for online advice customer relationship management |
US7353176B1 (en) | 2001-12-20 | 2008-04-01 | Ianywhere Solutions, Inc. | Actuation system for an agent oriented architecture |
US7937439B2 (en) * | 2001-12-27 | 2011-05-03 | Utbk, Inc. | Apparatus and method for scheduling live advice communication with a selected service provider |
US7305070B2 (en) | 2002-01-30 | 2007-12-04 | At&T Labs, Inc. | Sequential presentation of long instructions in an interactive voice response system |
US6914975B2 (en) * | 2002-02-21 | 2005-07-05 | Sbc Properties, L.P. | Interactive dialog-based training method |
US20030171926A1 (en) * | 2002-03-07 | 2003-09-11 | Narasimha Suresh | System for information storage, retrieval and voice based content search and methods thereof |
US7242758B2 (en) * | 2002-03-19 | 2007-07-10 | Nuance Communications, Inc | System and method for automatically processing a user's request by an automated assistant |
US6895379B2 (en) * | 2002-03-27 | 2005-05-17 | Sony Corporation | Method of and apparatus for configuring and controlling home entertainment systems through natural language and spoken commands using a natural language server |
US20100023481A1 (en) * | 2002-04-02 | 2010-01-28 | Mcgoveran Davd O | Computer-implemented method for deriving, translating, and using definitional expressions for data in a database |
US7707221B1 (en) * | 2002-04-03 | 2010-04-27 | Yahoo! Inc. | Associating and linking compact disc metadata |
US7574427B2 (en) * | 2002-05-17 | 2009-08-11 | Delefevre Patrick Y | Neutral sales consultant |
US7398209B2 (en) | 2002-06-03 | 2008-07-08 | Voicebox Technologies, Inc. | Systems and methods for responding to natural language speech utterance |
US7693720B2 (en) | 2002-07-15 | 2010-04-06 | Voicebox Technologies, Inc. | Mobile systems and methods for responding to natural language speech utterance |
US20040019488A1 (en) * | 2002-07-23 | 2004-01-29 | Netbytel, Inc. | Email address recognition using personal information |
US7421389B2 (en) * | 2002-11-13 | 2008-09-02 | At&T Knowledge Ventures, L.P. | System and method for remote speech recognition |
DE10258033A1 (en) * | 2002-12-12 | 2004-06-24 | Deutsche Telekom Ag | Object-language database |
US6931453B2 (en) * | 2003-01-03 | 2005-08-16 | Nokia Corporation | Method and apparatus for resolving protocol-agnostic schemes in an internet protocol multimedia subsystem |
US20040162724A1 (en) | 2003-02-11 | 2004-08-19 | Jeffrey Hill | Management of conversations |
US7606714B2 (en) * | 2003-02-11 | 2009-10-20 | Microsoft Corporation | Natural language classification within an automated response system |
US7359498B2 (en) * | 2003-06-12 | 2008-04-15 | Utbk, Inc. | Systems and methods for arranging a call |
US7698183B2 (en) * | 2003-06-18 | 2010-04-13 | Utbk, Inc. | Method and apparatus for prioritizing a listing of information providers |
US7886009B2 (en) | 2003-08-22 | 2011-02-08 | Utbk, Inc. | Gate keeper |
US7120235B2 (en) * | 2003-10-06 | 2006-10-10 | Ingenio, Inc. | Method and apparatus to provide pay-per-call performance based advertising |
US7428497B2 (en) | 2003-10-06 | 2008-09-23 | Utbk, Inc. | Methods and apparatuses for pay-per-call advertising in mobile/wireless applications |
US9202220B2 (en) * | 2003-10-06 | 2015-12-01 | Yellowpages.Com Llc | Methods and apparatuses to provide application programming interface for retrieving pay per call advertisements |
US8121898B2 (en) | 2003-10-06 | 2012-02-21 | Utbk, Inc. | Methods and apparatuses for geographic area selections in pay-per-call advertisement |
US8837698B2 (en) * | 2003-10-06 | 2014-09-16 | Yp Interactive Llc | Systems and methods to collect information just in time for connecting people for real time communications |
US9984377B2 (en) | 2003-10-06 | 2018-05-29 | Yellowpages.Com Llc | System and method for providing advertisement |
US7424442B2 (en) | 2004-05-04 | 2008-09-09 | Utbk, Inc. | Method and apparatus to allocate and recycle telephone numbers in a call-tracking system |
US8024224B2 (en) * | 2004-03-10 | 2011-09-20 | Utbk, Inc. | Method and apparatus to provide pay-per-call advertising and billing |
US8027878B2 (en) * | 2003-10-06 | 2011-09-27 | Utbk, Inc. | Method and apparatus to compensate demand partners in a pay-per-call performance based advertising system |
US8140389B2 (en) | 2003-10-06 | 2012-03-20 | Utbk, Inc. | Methods and apparatuses for pay for deal advertisements |
US7366683B2 (en) | 2003-10-06 | 2008-04-29 | Utbk, Inc. | Methods and apparatuses for offline selection of pay-per-call advertisers |
US7427024B1 (en) | 2003-12-17 | 2008-09-23 | Gazdzinski Mark J | Chattel management apparatus and methods |
US7027586B2 (en) * | 2003-12-18 | 2006-04-11 | Sbc Knowledge Ventures, L.P. | Intelligently routing customer communications |
US20050165761A1 (en) * | 2004-01-22 | 2005-07-28 | Chan Eric J. | Method and apparatus for data processing and retrieval |
FI20040296A (en) | 2004-02-25 | 2005-08-26 | Saimaan Ipr Oy | A method and device for requesting a service online |
US7562008B2 (en) * | 2004-06-23 | 2009-07-14 | Ning-Ping Chan | Machine translation method and system that decomposes complex sentences into two or more sentences |
WO2006007194A1 (en) * | 2004-06-25 | 2006-01-19 | Personasearch, Inc. | Dynamic search processor |
US9552599B1 (en) | 2004-09-10 | 2017-01-24 | Deem, Inc. | Platform for multi-service procurement |
FR2881245A1 (en) * | 2005-01-27 | 2006-07-28 | Roger Marx Desenberg | Web site content e.g. good such as refrigerator, locating method for Internet, involves automatically generating computer program code as function of electronic information of site, and transferring code to work station |
US8538768B2 (en) | 2005-02-16 | 2013-09-17 | Ingenio Llc | Methods and apparatuses for delivery of advice to mobile/wireless devices |
US9202219B2 (en) * | 2005-02-16 | 2015-12-01 | Yellowpages.Com Llc | System and method to merge pay-for-performance advertising models |
US7979308B2 (en) * | 2005-03-03 | 2011-07-12 | Utbk, Inc. | Methods and apparatuses for sorting lists for presentation |
US8934614B2 (en) * | 2005-02-25 | 2015-01-13 | YP Interatcive LLC | Systems and methods for dynamic pay for performance advertisements |
WO2006138706A2 (en) * | 2005-06-17 | 2006-12-28 | Computer Associates Think, Inc. | System and method for identifying application resources |
US20070021960A1 (en) * | 2005-07-20 | 2007-01-25 | Mclean Marc | System and method for communicating with a network |
US7640160B2 (en) | 2005-08-05 | 2009-12-29 | Voicebox Technologies, Inc. | Systems and methods for responding to natural language speech utterance |
US7620549B2 (en) | 2005-08-10 | 2009-11-17 | Voicebox Technologies, Inc. | System and method of supporting adaptive misrecognition in conversational speech |
US7949529B2 (en) | 2005-08-29 | 2011-05-24 | Voicebox Technologies, Inc. | Mobile systems and methods of supporting natural language human-machine interactions |
US7634409B2 (en) * | 2005-08-31 | 2009-12-15 | Voicebox Technologies, Inc. | Dynamic speech sharpening |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US8599832B2 (en) | 2005-09-28 | 2013-12-03 | Ingenio Llc | Methods and apparatuses to connect people for real time communications via voice over internet protocol (VOIP) |
US8761154B2 (en) | 2005-09-28 | 2014-06-24 | Ebbe Altberg | Methods and apparatuses to access advertisements through voice over internet protocol (VoIP) applications |
US7797282B1 (en) | 2005-09-29 | 2010-09-14 | Hewlett-Packard Development Company, L.P. | System and method for modifying a training set |
US9117223B1 (en) | 2005-12-28 | 2015-08-25 | Deem, Inc. | Method and system for resource planning for service provider |
US9197479B2 (en) | 2006-01-10 | 2015-11-24 | Yellowpages.Com Llc | Systems and methods to manage a queue of people requesting real time communication connections |
US20070165841A1 (en) * | 2006-01-10 | 2007-07-19 | Scott Faber | Systems and methods to provide guidance during a process to establish a communication connection |
US8681778B2 (en) | 2006-01-10 | 2014-03-25 | Ingenio Llc | Systems and methods to manage privilege to speak |
US7720091B2 (en) * | 2006-01-10 | 2010-05-18 | Utbk, Inc. | Systems and methods to arrange call back |
US8125931B2 (en) * | 2006-01-10 | 2012-02-28 | Utbk, Inc. | Systems and methods to provide availability indication |
US7599861B2 (en) | 2006-03-02 | 2009-10-06 | Convergys Customer Management Group, Inc. | System and method for closed loop decisionmaking in an automated care system |
US7437338B1 (en) * | 2006-03-21 | 2008-10-14 | Hewlett-Packard Development Company, L.P. | Providing information regarding a trend based on output of a categorizer |
US7809663B1 (en) | 2006-05-22 | 2010-10-05 | Convergys Cmg Utah, Inc. | System and method for supporting the utilization of machine language |
US8379830B1 (en) | 2006-05-22 | 2013-02-19 | Convergys Customer Management Delaware Llc | System and method for automated customer service with contingent live interaction |
US7860815B1 (en) * | 2006-07-12 | 2010-12-28 | Venkateswara Prasad Tangirala | Computer knowledge representation format, system, methods, and applications |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US7890326B2 (en) * | 2006-10-13 | 2011-02-15 | Google Inc. | Business listing search |
US7840407B2 (en) | 2006-10-13 | 2010-11-23 | Google Inc. | Business listing search |
US8041568B2 (en) * | 2006-10-13 | 2011-10-18 | Google Inc. | Business listing search |
US8073681B2 (en) * | 2006-10-16 | 2011-12-06 | Voicebox Technologies, Inc. | System and method for a cooperative conversational voice user interface |
US9317855B2 (en) | 2006-10-24 | 2016-04-19 | Yellowpages.Com Llc | Systems and methods to provide voice connections via local telephone numbers |
US8630855B2 (en) * | 2006-12-22 | 2014-01-14 | Anthony Oddo | Call system and method |
US8451825B2 (en) | 2007-02-22 | 2013-05-28 | Utbk, Llc | Systems and methods to confirm initiation of a callback |
US7818176B2 (en) | 2007-02-06 | 2010-10-19 | Voicebox Technologies, Inc. | System and method for selecting and presenting advertisements based on natural language processing of voice-based input |
US20080228571A1 (en) * | 2007-03-16 | 2008-09-18 | Koran Joshua M | Automated recommendation of targeting criteria |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US20080263460A1 (en) * | 2007-04-20 | 2008-10-23 | Utbk, Inc. | Methods and Systems to Connect People for Virtual Meeting in Virtual Reality |
US20080262910A1 (en) * | 2007-04-20 | 2008-10-23 | Utbk, Inc. | Methods and Systems to Connect People via Virtual Reality for Real Time Communications |
US8712844B2 (en) * | 2007-05-08 | 2014-04-29 | Yahoo! Inc. | Use of natural query events to improve online advertising campaigns |
US9277019B2 (en) | 2007-06-18 | 2016-03-01 | Yellowpages.Com Llc | Systems and methods to provide communication references to connect people for real time communications |
US20090063413A1 (en) * | 2007-08-31 | 2009-03-05 | Handysoft Global Corporation | Method and system for tracking allocations of assets and tasks |
US8838476B2 (en) * | 2007-09-07 | 2014-09-16 | Yp Interactive Llc | Systems and methods to provide information and connect people for real time communications |
US8024177B2 (en) * | 2007-09-28 | 2011-09-20 | Cycorp, Inc. | Method of transforming natural language expression into formal language representation |
US9053089B2 (en) * | 2007-10-02 | 2015-06-09 | Apple Inc. | Part-of-speech tagging using latent analogy |
US8140335B2 (en) | 2007-12-11 | 2012-03-20 | Voicebox Technologies, Inc. | System and method for providing a natural language voice user interface in an integrated voice navigation services environment |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8065143B2 (en) | 2008-02-22 | 2011-11-22 | Apple Inc. | Providing text input using speech data and non-speech data |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US8589161B2 (en) * | 2008-05-27 | 2013-11-19 | Voicebox Technologies, Inc. | System and method for an integrated, multi-modal, multi-device natural language voice services environment |
US9305548B2 (en) | 2008-05-27 | 2016-04-05 | Voicebox Technologies Corporation | System and method for an integrated, multi-modal, multi-device natural language voice services environment |
US8464150B2 (en) | 2008-06-07 | 2013-06-11 | Apple Inc. | Automatic language identification for dynamic text processing |
US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
US8768702B2 (en) * | 2008-09-05 | 2014-07-01 | Apple Inc. | Multi-tiered voice feedback in an electronic device |
US8898568B2 (en) | 2008-09-09 | 2014-11-25 | Apple Inc. | Audio user interface |
US8712776B2 (en) | 2008-09-29 | 2014-04-29 | Apple Inc. | Systems and methods for selective text to speech synthesis |
US8676904B2 (en) | 2008-10-02 | 2014-03-18 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
WO2010067118A1 (en) | 2008-12-11 | 2010-06-17 | Novauris Technologies Limited | Speech recognition involving a mobile device |
US8862252B2 (en) * | 2009-01-30 | 2014-10-14 | Apple Inc. | Audio user interface for displayless electronic device |
US8458105B2 (en) * | 2009-02-12 | 2013-06-04 | Decisive Analytics Corporation | Method and apparatus for analyzing and interrelating data |
US20100235314A1 (en) * | 2009-02-12 | 2010-09-16 | Decisive Analytics Corporation | Method and apparatus for analyzing and interrelating video data |
US8326637B2 (en) * | 2009-02-20 | 2012-12-04 | Voicebox Technologies, Inc. | System and method for processing multi-modal device interactions in a natural language voice services environment |
US8380507B2 (en) | 2009-03-09 | 2013-02-19 | Apple Inc. | Systems and methods for determining the language to use for speech generated by a text to speech engine |
US8543401B2 (en) * | 2009-04-17 | 2013-09-24 | Synchronoss Technologies | System and method for improving performance of semantic classifiers in spoken dialog systems |
US10552849B2 (en) * | 2009-04-30 | 2020-02-04 | Deem, Inc. | System and method for offering, tracking and promoting loyalty rewards |
US10540976B2 (en) | 2009-06-05 | 2020-01-21 | Apple Inc. | Contextual voice commands |
US20120311585A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Organizing task items that represent tasks to perform |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US9431006B2 (en) * | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US9646079B2 (en) | 2012-05-04 | 2017-05-09 | Pearl.com LLC | Method and apparatus for identifiying similar questions in a consultation system |
US20130295546A1 (en) * | 2012-05-04 | 2013-11-07 | Pearl.com LLC | Method and apparatus for identifying and eliciting missing question details in a consultation system |
US9904436B2 (en) | 2009-08-11 | 2018-02-27 | Pearl.com LLC | Method and apparatus for creating a personalized question feed platform |
US20110066438A1 (en) * | 2009-09-15 | 2011-03-17 | Apple Inc. | Contextual voiceover |
WO2011059997A1 (en) | 2009-11-10 | 2011-05-19 | Voicebox Technologies, Inc. | System and method for providing a natural language content dedication service |
US9171541B2 (en) * | 2009-11-10 | 2015-10-27 | Voicebox Technologies Corporation | System and method for hybrid processing in a natural language voice services environment |
US8682649B2 (en) * | 2009-11-12 | 2014-03-25 | Apple Inc. | Sentiment prediction from textual data |
US8381107B2 (en) | 2010-01-13 | 2013-02-19 | Apple Inc. | Adaptive audio feedback system and method |
US8311838B2 (en) | 2010-01-13 | 2012-11-13 | Apple Inc. | Devices and methods for identifying a prompt corresponding to a voice input in a sequence of prompts |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
WO2011089450A2 (en) | 2010-01-25 | 2011-07-28 | Andrew Peter Nelson Jerram | Apparatuses, methods and systems for a digital conversation management platform |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US8713021B2 (en) | 2010-07-07 | 2014-04-29 | Apple Inc. | Unsupervised document clustering using latent semantic density analysis |
US8538915B2 (en) * | 2010-07-12 | 2013-09-17 | International Business Machines Corporation | Unified numerical and semantic analytics system for decision support |
US8661018B2 (en) | 2010-08-10 | 2014-02-25 | Lockheed Martin Corporation | Data service response plan generator |
US8719006B2 (en) | 2010-08-27 | 2014-05-06 | Apple Inc. | Combined statistical and rule-based part-of-speech tagging for text-to-speech synthesis |
US8719014B2 (en) | 2010-09-27 | 2014-05-06 | Apple Inc. | Electronic device with text error correction based on voice recognition data |
US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
US10515147B2 (en) | 2010-12-22 | 2019-12-24 | Apple Inc. | Using statistical language models for contextual lookup |
US8781836B2 (en) | 2011-02-22 | 2014-07-15 | Apple Inc. | Hearing assistance system for providing consistent human speech |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US9280535B2 (en) | 2011-03-31 | 2016-03-08 | Infosys Limited | Natural language querying with cascaded conditional random fields |
US9449288B2 (en) | 2011-05-20 | 2016-09-20 | Deem, Inc. | Travel services search |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US20120310642A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Automatically creating a mapping between text data and audio data |
US8812294B2 (en) | 2011-06-21 | 2014-08-19 | Apple Inc. | Translating phrases from one language into another using an order-based set of declarative rules |
US8706472B2 (en) | 2011-08-11 | 2014-04-22 | Apple Inc. | Method for disambiguating multiple readings in language conversion |
US8994660B2 (en) | 2011-08-29 | 2015-03-31 | Apple Inc. | Text correction processing |
US9269063B2 (en) | 2011-09-23 | 2016-02-23 | Elwha Llc | Acquiring and transmitting event related tasks and subtasks to interface devices |
US20130081027A1 (en) * | 2011-09-23 | 2013-03-28 | Elwha LLC, a limited liability company of the State of Delaware | Acquiring, presenting and transmitting tasks and subtasks to interface devices |
US20130086589A1 (en) * | 2011-09-30 | 2013-04-04 | Elwha Llc | Acquiring and transmitting tasks and subtasks to interface |
US20130081053A1 (en) * | 2011-09-23 | 2013-03-28 | Elwha LLC, a limited liability company of the State of Delaware | Acquiring and transmitting tasks and subtasks to interface devices |
US8762156B2 (en) | 2011-09-28 | 2014-06-24 | Apple Inc. | Speech recognition repair using contextual information |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9275038B2 (en) | 2012-05-04 | 2016-03-01 | Pearl.com LLC | Method and apparatus for identifying customer service and duplicate questions in an online consultation system |
US9501580B2 (en) | 2012-05-04 | 2016-11-22 | Pearl.com LLC | Method and apparatus for automated selection of interesting content for presentation to first time visitors of a website |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US8775442B2 (en) | 2012-05-15 | 2014-07-08 | Apple Inc. | Semantic search using a single-source semantic model |
WO2013185109A2 (en) | 2012-06-08 | 2013-12-12 | Apple Inc. | Systems and methods for recognizing textual identifiers within a plurality of words |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US9405821B1 (en) * | 2012-08-03 | 2016-08-02 | tinyclues SAS | Systems and methods for data mining automation |
US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
US8935167B2 (en) | 2012-09-25 | 2015-01-13 | Apple Inc. | Exemplar-based latent perceptual modeling for automatic speech recognition |
US20140200879A1 (en) * | 2013-01-11 | 2014-07-17 | Brian Sakhai | Method and System for Rating Food Items |
KR102516577B1 (en) | 2013-02-07 | 2023-04-03 | 애플 인크. | Voice trigger for a digital assistant |
US10642574B2 (en) | 2013-03-14 | 2020-05-05 | Apple Inc. | Device, method, and graphical user interface for outputting captions |
US9977779B2 (en) | 2013-03-14 | 2018-05-22 | Apple Inc. | Automatic supplementation of word correction dictionaries |
US10572476B2 (en) | 2013-03-14 | 2020-02-25 | Apple Inc. | Refining a search based on schedule items |
US9733821B2 (en) | 2013-03-14 | 2017-08-15 | Apple Inc. | Voice control to diagnose inadvertent activation of accessibility features |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
WO2014144949A2 (en) | 2013-03-15 | 2014-09-18 | Apple Inc. | Training an at least partial voice command system |
CN112230878A (en) | 2013-03-15 | 2021-01-15 | 苹果公司 | Context-sensitive handling of interrupts |
US10748529B1 (en) | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
US11151899B2 (en) | 2013-03-15 | 2021-10-19 | Apple Inc. | User training by intelligent digital assistant |
WO2014144579A1 (en) | 2013-03-15 | 2014-09-18 | Apple Inc. | System and method for updating an adaptive speech recognition model |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
WO2014197336A1 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
EP3008641A1 (en) | 2013-06-09 | 2016-04-20 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
CN105265005B (en) | 2013-06-13 | 2019-09-17 | 苹果公司 | System and method for the urgent call initiated by voice command |
WO2015020942A1 (en) | 2013-08-06 | 2015-02-12 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
US9672552B2 (en) * | 2014-03-27 | 2017-06-06 | GroupBy Inc. | Methods of augmenting search engines for ecommerce information retrieval |
US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
EP3149728B1 (en) | 2014-05-30 | 2019-01-16 | Apple Inc. | Multi-command single utterance input method |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US9898459B2 (en) | 2014-09-16 | 2018-02-20 | Voicebox Technologies Corporation | Integration of domain information into state transitions of a finite state transducer for natural language processing |
EP3195145A4 (en) | 2014-09-16 | 2018-01-24 | VoiceBox Technologies Corporation | Voice commerce |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
CN107003999B (en) | 2014-10-15 | 2020-08-21 | 声钰科技 | System and method for subsequent response to a user's prior natural language input |
US9992352B2 (en) | 2014-11-01 | 2018-06-05 | Somos, Inc. | Toll-free telecommunications and data management platform |
US10560583B2 (en) | 2014-11-01 | 2020-02-11 | Somos, Inc. | Toll-free numbers metadata tagging, analysis and reporting |
CA2965681C (en) | 2014-11-01 | 2020-01-14 | Somos, Inc. | Toll-free telecommunications management platform |
US9553997B2 (en) | 2014-11-01 | 2017-01-24 | Somos, Inc. | Toll-free telecommunications management platform |
US10431214B2 (en) | 2014-11-26 | 2019-10-01 | Voicebox Technologies Corporation | System and method of determining a domain and/or an action related to a natural language input |
US10614799B2 (en) | 2014-11-26 | 2020-04-07 | Voicebox Technologies Corporation | System and method of providing intent predictions for an utterance prior to a system detection of an end of the utterance |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US9578173B2 (en) | 2015-06-05 | 2017-02-21 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US9727554B2 (en) | 2015-11-24 | 2017-08-08 | International Business Machines Corporation | Knowledge-based editor with natural language interface |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US10171410B2 (en) | 2016-03-28 | 2019-01-01 | Microsoft Technology Licensing, Llc | Cross-mode communiation |
US10587708B2 (en) | 2016-03-28 | 2020-03-10 | Microsoft Technology Licensing, Llc | Multi-modal conversational intercom |
US11487512B2 (en) | 2016-03-29 | 2022-11-01 | Microsoft Technology Licensing, Llc | Generating a services application |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
DK179309B1 (en) | 2016-06-09 | 2018-04-23 | Apple Inc | Intelligent automated assistant in a home environment |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
DK179049B1 (en) | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179343B1 (en) | 2016-06-11 | 2018-05-14 | Apple Inc | Intelligent task discovery |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
US10083451B2 (en) * | 2016-07-08 | 2018-09-25 | Asapp, Inc. | Using semantic processing for customer support |
US10453074B2 (en) | 2016-07-08 | 2019-10-22 | Asapp, Inc. | Automatically suggesting resources for responding to a request |
US10331784B2 (en) | 2016-07-29 | 2019-06-25 | Voicebox Technologies Corporation | System and method of disambiguating natural language processing requests |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10109275B2 (en) | 2016-12-19 | 2018-10-23 | Asapp, Inc. | Word hash language model |
US10650311B2 (en) | 2016-12-19 | 2020-05-12 | Asaap, Inc. | Suggesting resources using context hashing |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
DK201770439A1 (en) | 2017-05-11 | 2018-12-13 | Apple Inc. | Offline personal assistant |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK179496B1 (en) | 2017-05-12 | 2019-01-15 | Apple Inc. | USER-SPECIFIC Acoustic Models |
DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
DK201770432A1 (en) | 2017-05-15 | 2018-12-21 | Apple Inc. | Hierarchical belief states for digital assistants |
DK179560B1 (en) | 2017-05-16 | 2019-02-18 | Apple Inc. | Far-field extension for digital assistant services |
US11494395B2 (en) | 2017-07-31 | 2022-11-08 | Splunk Inc. | Creating dashboards for viewing data in a data storage system based on natural language requests |
US10901811B2 (en) | 2017-07-31 | 2021-01-26 | Splunk Inc. | Creating alerts associated with a data storage system based on natural language requests |
US10497004B2 (en) | 2017-12-08 | 2019-12-03 | Asapp, Inc. | Automating communications using an intent classifier |
US10489792B2 (en) | 2018-01-05 | 2019-11-26 | Asapp, Inc. | Maintaining quality of customer support messages |
US10210244B1 (en) | 2018-02-12 | 2019-02-19 | Asapp, Inc. | Updating natural language interfaces by processing usage data |
US10169315B1 (en) | 2018-04-27 | 2019-01-01 | Asapp, Inc. | Removing personal information from text using a neural network |
RU2686000C1 (en) * | 2018-06-20 | 2019-04-23 | Общество с ограниченной ответственностью "Аби Продакшн" | Retrieval of information objects using a combination of classifiers analyzing local and non-local signs |
US11216510B2 (en) | 2018-08-03 | 2022-01-04 | Asapp, Inc. | Processing an incomplete message with a neural network to generate suggested messages |
CA3051556A1 (en) | 2018-08-10 | 2020-02-10 | Somos, Inc. | Toll-free telecommunications data management interface |
US10747957B2 (en) | 2018-11-13 | 2020-08-18 | Asapp, Inc. | Processing communications using a prototype classifier |
US11551004B2 (en) | 2018-11-13 | 2023-01-10 | Asapp, Inc. | Intent discovery with a prototype classifier |
US11580416B2 (en) | 2019-08-14 | 2023-02-14 | International Business Machines Corporation | Improving the accuracy of a compendium of natural language responses |
US11425064B2 (en) | 2019-10-25 | 2022-08-23 | Asapp, Inc. | Customized message suggestion with user embedding vectors |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4797910A (en) * | 1986-05-07 | 1989-01-10 | American Telphone And Telegraph Company, At&T Bell Laboratories | Automated operator assistance calls with voice processing |
US4959855A (en) * | 1986-10-08 | 1990-09-25 | At&T Bell Laboratories | Directory assistance call processing and calling customer remote signal monitoring arrangements |
US4979206A (en) * | 1987-07-10 | 1990-12-18 | At&T Bell Laboratories | Directory assistance systems |
US5331554A (en) * | 1992-12-10 | 1994-07-19 | Ricoh Corporation | Method and apparatus for semantic pattern matching for text retrieval |
US5802511A (en) * | 1996-01-02 | 1998-09-01 | Timeline, Inc. | Data retrieval method and apparatus with multiple source capability |
US5943410A (en) * | 1996-08-02 | 1999-08-24 | Siemens Information And Communications Networks, Inc. | Automated calling of multiple numbers using directory assistance |
US5839107A (en) * | 1996-11-29 | 1998-11-17 | Northern Telecom Limited | Method and apparatus for automatically generating a speech recognition vocabulary from a white pages listing |
US6028917A (en) * | 1997-04-04 | 2000-02-22 | International Business Machines Corporation | Access to extended telephone services via the internet |
US5895464A (en) * | 1997-04-30 | 1999-04-20 | Eastman Kodak Company | Computer program product and a method for using natural language for the description, search and retrieval of multi-media objects |
US5878418A (en) * | 1997-08-12 | 1999-03-02 | Intervoice Limited Partnership | Auto definition of data sets and provisioning interfaces for call automation |
US6173279B1 (en) | 1998-04-09 | 2001-01-09 | At&T Corp. | Method of using a natural language interface to retrieve information from one or more data resources |
-
1998
- 1998-04-09 US US09/058,107 patent/US6173279B1/en not_active Expired - Lifetime
-
1999
- 1999-04-01 WO PCT/US1999/007278 patent/WO1999053676A1/en active Application Filing
- 1999-04-01 CA CA002293780A patent/CA2293780C/en not_active Expired - Lifetime
Also Published As
Publication number | Publication date |
---|---|
WO1999053676A1 (en) | 1999-10-21 |
CA2293780A1 (en) | 1999-10-21 |
US6173279B1 (en) | 2001-01-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2293780C (en) | Method of using a natural language interface to retrieve information from one or more data resources | |
US6240448B1 (en) | Method and system for audio access to information in a wide area computer network | |
US6859776B1 (en) | Method and apparatus for optimizing a spoken dialog between a person and a machine | |
US6600736B1 (en) | Method of providing transfer capability on web-based interactive voice response services | |
US6157705A (en) | Voice control of a server | |
CN100472500C (en) | Conversational browser and conversational systems | |
US7043521B2 (en) | Search agent for searching the internet | |
US6850766B2 (en) | Voice activated wireless locator service | |
JP4398098B2 (en) | Glamor template query system | |
US7437295B2 (en) | Natural language processing for a location-based services system | |
US6922670B2 (en) | User support apparatus and system using agents | |
US6349132B1 (en) | Voice interface for electronic documents | |
US7831688B2 (en) | Method and system for providing electronic user assistance | |
US6583716B2 (en) | System and method for providing location-relevant services using stored location information | |
US8204956B2 (en) | Computer-implemented voice application indexing web site | |
JP2002108683A (en) | Light weight directory access protocol interface for directory assistance system | |
TW201034004A (en) | Systems and methods for interactively accessing hosted services using voice communications | |
US8387074B2 (en) | Enterprise directory service | |
US20010051973A1 (en) | System, method and computer program product for a locator service | |
Pargellis et al. | An automatic dialogue generation platform for personalized dialogue applications | |
US20040267549A1 (en) | Methods and systems for assisting in real estate transactions with automation utilizing verbal communication | |
US20050240409A1 (en) | System and method for providing rules-based directory assistance automation | |
US20060075037A1 (en) | Portal for managing communications of a client over a network | |
KR100488112B1 (en) | Apparatus For Converting Document and Searching in Voice Portal System | |
US20060112078A1 (en) | Information procurement |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
EEER | Examination request | ||
MKEX | Expiry |
Effective date: 20190401 |