WO2002057896A2 - Interactive virtual assistant - Google Patents

Interactive virtual assistant Download PDF

Info

Publication number
WO2002057896A2
WO2002057896A2 PCT/GB2002/000202 GB0200202W WO02057896A2 WO 2002057896 A2 WO2002057896 A2 WO 2002057896A2 GB 0200202 W GB0200202 W GB 0200202W WO 02057896 A2 WO02057896 A2 WO 02057896A2
Authority
WO
WIPO (PCT)
Prior art keywords
user
task
character
iva
input
Prior art date
Application number
PCT/GB2002/000202
Other languages
French (fr)
Other versions
WO2002057896A3 (en
Inventor
Christopher Ball
Michael Hambly
James Robertson
Matthew Caryl
Robert Mcmillan
Original Assignee
Digital Animations Group Plc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Digital Animations Group Plc filed Critical Digital Animations Group Plc
Priority to AU2002225160A priority Critical patent/AU2002225160A1/en
Publication of WO2002057896A2 publication Critical patent/WO2002057896A2/en
Publication of WO2002057896A3 publication Critical patent/WO2002057896A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Definitions

  • the present invention relates to apparatus for implementing an interactive virtual assistant, such as a virtual receptionist or a virtual museum guide; and to a method of implementing such an interactive virtual assistant.
  • the invention further relates to a method of raising revenue for individuals by licensing an image of the individual for use as virtual assistants.
  • Virtual assistant refers to a computer or other processing device generated and animated character which is able to serve a user in an assistant role.
  • the present invention makes use of this system in the generation and implementation of an improved interactive virtual assistant.
  • the invention is, however, not restricted to implementation using the specific virtual character technology, but can make use of any suitable character generation system having similar characteristics.
  • an apparatus for providing an interactive virtual assistant implemented by a virtual character comprising: display means for displaying said virtual character; user interaction means for receiving input from a user; processing means for processing received input; task execution means for performing a task in response to received input; and response generation means for responding to said user including character animation means for generating and animating said virtual character on said display means in response to received input .
  • the present invention allows a user to provide input (such as instructions, questions, or an identification of the user) to an interactive virtual assistant (hereinafter referred to as an IVA) .
  • the IVA then performs a task in response to the input, and a character is animated to provide a response to the user.
  • the response may be visual, audible, tactile, physical, or a combination of any of these.
  • the animated character provides a human- like interface for the user, allowing an easier interaction than if no human were involved.
  • the invention therefore allows an IVA to take over a number of roles which would normally otherwise be performed by a human.
  • the user interaction means may comprise any suitable device for providing an interface between the user and the IVA.
  • a touch sensitive display screen For example, a touch sensitive display screen; a voice recognition device; a smart card or magnetic swipe card reader, which may for example identify a user as they swipe a security card in the reader; a keyboard or numeric keypad; an infra-red communications port, for communicating with, for example, a mobile communications device such as a laptop or palmtop computer, a personal digital assistant (PDA) , a mobile telephone, or the like; a wireless technology (such as Bluetooth) detection device for interacting with similarly-equipped mobile devices; a hardwired communications device, such as a telephone or videophone; sensors such as a pressure-sensitive footpad for detecting the presence of a user. Any or all of these devices may be used in combination: for example, a pressure- sensitive footpad may be used to alert the IVA of the presence of a user, while a voice recognition device may be used for the input of further data or commands.
  • the apparatus may automatically identify the user based on some distinguishing characteristic encoded in the mobile device or smart card. This will enable the apparatus to provide a more efficient and personal service to the user.
  • the processing means comprises a computing device.
  • the computing device may be of conventional type, and may be running known computer software for processing the received input.
  • specific software may be written for the implementation of the IVA, depending on the type of input liable to be received.
  • Hardware or firmware may of course be used in place of computer software.
  • the task execution means will typically comprise a computing device, or a computer program (in the form of hardware or software) . Tasks to be executed may be explicitly included in the user input (for example, a request to find a particular person's telephone number or the like) ,- or may be implicit
  • user input in the form of activation of a pressure sensitive pad may implicitly include the task to be executed of 'activate the IVA and greet the user' ) .
  • Example tasks may include: determining the subsequent content of speech to be made by the IVA (for example, where the input includes the user's name or the like, the implicit task may be to generate a phoneme and viseme output to greet the user by name; or where the input is a selection of one of a range of options, the task may be to provide task-dependent sub-options for example to further specify details of the task) ; alerting a third party to the presence of the user (for example, where the IVA is acting as a virtual receptionist) , the third party may be alerted by email, telephone, or the like; retrieving information for the user, from for example a remote or local database, the internet, an expert system, or the like; contacting and interacting with an IVA belonging to a third party.
  • This last task may be particularly useful in the implementation of a virtual PA' ; a user may instruct the IVA to arrange a meeting with a third party at a mutually convenient time, whereupon the IVA liaises with a similarly-enabled IVA belonging to the third party. Once both IVAs have performed their tasks of arranging a time and place for the meeting, the IVAs then report to their respective users.
  • the particular tasks which are available depend on the purpose for which the IVA is available.
  • the task execution means is implementable in software, such that the available tasks may be readily reconfigured to allow for different functionalities to be implemented.
  • roles for an IVA according to the present invention include, but are not limited to, virtual receptionist; personal secretary; hotel room assistant; environment guide; virtual advisor; virtual trainer; language teacher; e-commerce host; domestic unit; children's assistant. Each of these will be described in more detail later; additional roles are of course possible.
  • the character animation means comprises a computing device together with a suitable computer program (whether hardware or software) .
  • the computer program is preferably as described in our co-pending Patent Application "Character Animation System", referred to above, although any suitable program may be used.
  • the character animation is generated in substantially real-time.
  • the virtual character may be any moving, talking virtual image, not necessarily human. For example, if used as a children's assistant, it may be preferable to use a zooform assistant, although animated with human-characteristic expressions and mannerisms.
  • the apparatus may further comprise means for selecting a virtual character to be displayed.
  • the IVA may be selectable from a range of preset possibilities (for example, characters may be available in a range of ethnicities and genders, as well as species) .
  • the character selection means may also or instead include means for adding new characters to the available options; for example, additional characters may be purchased or rented from the apparatus provider, or new characters may be created based on user-added data.
  • a user may wish to create a character based on themselves to act as a virtual PA or the like.
  • at least a portion of the animation may be dependent on the task performed; for example, where the task is to retrieve particular information from a database, the animation depends on the data thereby retrieved; or if the task is to alert a particular person to the ' presence of the user, the animation is dependent on whether the particular person can be alerted or not.
  • the animated character preferably includes facial movements and expressions corresponding to generated speech; the animation may in addition or alternatively include gestural animations (for example, to provide an output in sign language) , text output (as subtitles, or to present retrieved data which cannot conveniently be presented as speech), still pictures or images, and prerecorded video clips or the like.
  • gestural animations for example, to provide an output in sign language
  • text output as subtitles, or to present retrieved data which cannot conveniently be presented as speech
  • still pictures or images and prerecorded video clips or the like.
  • the display means comprises a display screen.
  • a cathode-ray tube screen for example, a cathode-ray tube screen, an LCD screen, a projection screen, or the like.
  • the display means may comprise a holographic display, three-dimensional display means, or similar device.
  • the apparatus may further include sound generation means, for generating speech sounds .
  • the display means may be local to the character animation means, or may be remote. Where the display means is remote, the display may be physically connected to the character animation means (for example, fibre optic or other data communication cables) , or may be in remote data communication (for example, via radio, infra red, or other electromagnetic communication, such as is used in Bluetooth or similar wireless technology) .
  • a remote display means may be incorporated in some other device, such as a personal digital assistant (PDA), mobile telephone, videophone, or the like.
  • PDA personal digital assistant
  • the remote display means may be portable; this may be particularly useful where the IVA is to be used as a museum guide or the like.
  • Any other components of the apparatus may also be portable along with the display means; for example, a computer processor may be located at a central location, and a user interaction means and display means provided as a portable handset .
  • the apparatus may in certain embodiments comprise a plurality of display means, and may optionally also comprise a plurality of user interaction means.
  • a display screen may be located within each of a plurality of rooms in a building, with user interaction means arranged to alert the IVA to the presence and identity of a user adjacent each screen.
  • the IVA may be aware of a user as they move about a building, and the IVA may be on hand in each room to assist the user.
  • a plurality of distinct virtual characters may of course be generated in such an embodiment, each assisting a different user as they move about a building.
  • a method of providing an interactive virtual assistant to a user comprising the steps of: receiving input from a user; processing the input to determine a task to be performed; performing said task in response to the input; responding to the user at least in part by generating and animating a virtual character in response to the input; and displaying the animated virtual character to the user.
  • a computer program product for providing an interaction to a user, the product comprising a computer-readable medium having thereon computer program code for : receiving input from a user; processing the input to determine a task to be performed; performing said task in response to the input; responding to the user at least in part by generating and animating a virtual character in response to the input; and displaying the animated virtual character to the user.
  • a method of generating licensing revenue for an individual comprising the steps of: reproducing distinctive characteristics of the individual as a virtual character; charging third parties to implement the virtual character in any of the above methods according to the invention; and providing at least a portion of an amount charged to third parties to the individual or a representative thereof.
  • the distinctive characteristics may include the individual's appearance, speech patterns, or mannerisms. Which particular characteristics are reproduced will obviously depend on the individual, and may also depend on the role which the virtual character is to play (for example, if the character is in animal form, the degree of reproduction of the individual's appearance will be lessened, while the reproduction may focus on speech and mannerisms) .
  • a method of generating revenue for an individual comprising the step of charging a third party for reproducing distinctive characteristics of the individual as a virtual character.
  • a method of marketing goods or services comprising the steps of: generating and animating a virtual character having distinctive characteristics of an individual; and using the virtual character to promote said goods or services .
  • the individual is a celebrity.
  • Fig. 1 shows an outline flow chart illustrating the implementation of the method of providing an interaction to a user in accordance with one embodiment of the present invention
  • Fig. 2 shows a sketch of an apparatus for implementing an interactive virtual assistant, in accordance with one embodiment of the present invention.
  • Figs. 3a to 3d show various stages in the implementation of a virtual receptionist, in accordance with one embodiment of the present invention.
  • FIG. 1 shows an outline flow chart illustrating the implementation of a method of providing a virtual assistant, in accordance with one embodiment of the present invention.
  • the figure shows a client or user 12 of the interactive virtual assistant (IVA) interacting with one of a number of possible input devices 1 .
  • the user 12 interacts with an input device 14 to provide information referring to a task to be performed to a central processor 16.
  • the input device 14 is the means by which the user 12 makes his or her requirements known to the IVA.
  • the device 14 can be anything which can convey data, ranging, for example, from simple touch-screen push-buttons, through standard keyboard to speech input.
  • the content of the user's information can range from the simplest ⁇ yes/no' button selection, through menu selection to text input which can be understood by the processor 16 to sophisticated speech recognition devices.
  • the processor 16 analyses the input received from the devices 14, and performs the requested task in an appropriate manner.
  • the processor 16 carries out or sets in motion some kind of task for the user, based on the requirements conveyed by the user, or implied on behalf of the user.
  • 'implied 1 when a visitor walks up to an idling virtual image and steps on a pressure pad, the IVA infers that the visitor wants attention, even although this is not specifically indicated.
  • the task can be any which results in a response to the client, for example, a spoken response by the IVA, or a physical event, or the like.
  • a virtual character 18 is generated and animated.
  • the processor 16 determines what the character 18 will say, and with what mood or attitude, or emotional expressions.
  • the animation itself is conducted by a self-contained program suite associated with the virtual character 18.
  • the generation and animation of the virtual character is substantially as described in our co-pending patent application referred to above; although any computer-generated character technology may be used.
  • the virtual character is any moving, talking virtual image, normally but not necessarily human, which reports a response to the task carried out by the IVA to the user. In most embodiments, only a face or head is used, but a full body image may be used provided some part generates expressions synchronised to the words to provide additional information to a user.
  • Dialogue with the client can take place in any language with which the IVA has been programmed.
  • the virtual character 18 is animated to provide an output 20 to the user 12.
  • the output 20 may be the results of the task, or may be a request for further information or clarification as to the nature of the task from the client; for example, if the client makes an ambiguous request to see an employee, the IVA may ask the client to specify the employee more precisely; or if a database search returns an excessive number of 'hits', the IVA may prompt the client to narrow the field of search by specifying more terms.
  • Further implementation of the method consists of repeating the stages of the method until the user or IVA indicates that the exchange is at an end; for example, by leaving the vicinity of the IVA, or by saying 'goodbye' or the like. While the cycling through the method is being performed, the user has the impression that a sustained dialogue is being conducted with the virtual character, while their requests are being carried out.
  • Fig. 2 shows a sketch of a specific apparatus for providing an interactive virtual assistant .
  • the apparatus 30 can be used to provide a 'virtual receptionist', as will be described below.
  • the apparatus 30 comprises a processing device 32, which performs the interpretation of input, generation of animations, and execution of tasks; a display screen and loudspeaker 34, on which animations of the IVA are displayed to a user; and a touch sensitive portion of the screen 36, by which a user can enter data to the processor 32.
  • the processor 32 is connected via a data communications network to one or more third party computers 38, which are not themselves part of the apparatus.
  • the 'Virtual Receptionist' is an animated character (for example, that illustrated in Fig. 3a) generated by a PC or other processing device 32.
  • the 'public face' of the receptionist is seen on a screen 34 in the reception foyer of the organisation employing the IVA.
  • a visitor arriving on the premises sees a screen 34 showing the image of the IVA shown in Fig. 3a.
  • the visitor also sees an area on a touch sensitive screen 36 containing the message: 'Please touch the screen here for assistance'.
  • Compliance activates a menu containing a list of company personnel names (Fig. 3b) . It also triggers a spoken and animated welcome from the IVA suggesting the visitor select a name from the list.
  • Touching a name prompts the IVA to ask the visitor to type his or her name into a text box (Fig. 3c) .
  • Typing may be conducted via the touch sensitive screen 36, or by means of a separate keyboard provided for the purpose.
  • the IVA sends an internal email or other message to the computer 38 of the required staff member, alerting the staff member to the arrival of the visitor.
  • a preferred embodiment of the invention uses the company telephone network to place a telephone call to the required staff member, alerting them to the presence of the visitor by means of a computer-generated or prerecorded voice message.
  • Use of a telephone call has the advantage that the staff member may provide the IVA with further instructions as to how to deal with the visitor immediately.
  • the IVA invites the visitor to take a seat until the contacted staff member can get there (Fig. 3d) , by speaking a phrase with the visitor's name, for example. If the required staff member cannot be reached; for example, if they do not acknowledge the message within a short time, the IVA goes through the staff list for an alternative, until someone deals with the visitor. This list is in the preferred order appropriate to the original required contact .
  • the IVA can recognise that one or more visitors have arrived, and thus be activated. This can be done with any of a number of techniques including beam interruption, swipe-card use, camera visual interpretation etc. As a default, the touch screen will always be available.
  • the IVA is able to determine the identity of the visitor by associating the time of arrival with the company's day calendar. This is subject to ambiguity if two or more visitors have close appointments, or an unscheduled arrival coincides with an expected visitor. To eliminate any mistakes, The IVA will ask the visitor to confirm the identification before proceeding.
  • the Virtual Personal Secretary (VS) variant of the IVA provides the client with a service on a one-to-one basis.
  • the client's VS through its on-board Al or other adaptive learning program, becomes customised to his or her needs, learning and acquiring relevant knowledge with time. It effectively travels with its client, and can be used wherever a telephone socket can be reached.
  • the services which the VS provides are : A personal organiser with an address book, general notebook and diary built-in.
  • a booking secretary which can interact with online booking services to arrange travel, entertainment, and the like.
  • a meetings secretary which can arrange meetings through other VSs. That is, the task carried out by the VS in this embodiment is to contact a second VS, and act as the input for this second VS to arrange a meeting time for the humans involved.
  • a conference call arranger which is an extension of the meetings secretary.
  • the Assistant may offer up-to-the-minute hotel information and services including: billing details, restaurant information including current access, room service facilities, leisure facilities, location maps, external facilities including local attractions, and an information service for specific events such as conferences .
  • Advanced versions of the Assistant learn its client's preferences, and mould its behaviour to fit. Once the guest is registered, the system knows whether he or she is a regular with the hotel or its chain. If so, the history the Assistant has built up is accessed, and is continued from that point. Failing that, the Assistant defaults to a neutral set of parameters, and starts building a user profile from there.
  • the Guide is a virtual character which talks its client through a physical trip or tour.
  • the guide operates in various settings such as permanent buildings, temporary exhibitions, cruise liners, country parks, towns and so on.
  • Features of the Guide include:
  • Networked terminals through which the customer and the IVA can communicate .
  • Hand-held terminals through which the customer and the IVA can communicate.
  • Non-shortest route to avoid temporary ' traffic- jams', congestion data updated continuously.
  • An example is a large motor show.
  • the entire exhibition area including access foyer is modelled as a terrain map. (The model could also be extended to the various car parks) .
  • stands, stalls, bars, snack bars, first-aid stations and toilets are defined as region objects. These are linked by corridors.
  • Large stands have stations for vehicles, and may have exhibition areas, demonstration rooms and/or discussion rooms. These are all defined as part of the base model for the guide, and used for the routing required by some of the above features.
  • a variant or supplement is a hand-held device using 'Bluetooth' or a comparable wire-free technology in place of the fixed-location screens.
  • the essential properties do not change .
  • the Guide becomes mobile and travels with its client. With this variant, information is provided at any time instead of just at fixed locations, and the irritation of possible queuing is removed.
  • the IVA becomes an advisor on any of a number of topics. Top of its agenda in this role is helping in filling out forms which, however well-intentioned their designers, often contain ambiguities to the unfamiliar eye. A typical example is a tax advisor.
  • Input is from a combination of menu selection, 'yes' and 'no' to direct questions, and typed keywords.
  • Well-focussed input is also an implementation option.
  • Interactive language teaching is a well-established application of computers.
  • the employment of an IVA gives the process a more human face.
  • Basic versions simply prompt the student according to his or her requests according to a pre-scripted plan.
  • the Teacher is backed by an Al which adapts automatically to its assessment of the student's progress. It is able to make objective assessments of progress.
  • a householder is in direct communication with his house as a whole, or individual appliances within it. Such devices range from lights and curtains to washing machines, cooker, and the like. The range would also include security systems.
  • To comply with the IVA definition there needs to be a virtual character in the loop, and therefore some medium, probably a display screen although possibly holographic projection, to carry it. This could be a small screen attached to the domestic unit, a touch screen or computer in one or more key locations or a mobile device of the kind described for the 'Guide' earlier. In this case, and optionally for the fixed screens, a wireless link such as 'Bluetooth' would be used.
  • a side benefit of age-group-layered child-specific systems is that children become familiar with interactive computing from the earliest age. The child thus becomes comfortable with online working.
  • the virtual character embedded in the IVA need not be human. If it is not, the definition still applies. The only difference is that the face will be a non-human such as an alien, or animal, or any kind of anthropomorphic image . As long as the virtual character talks and synchronises its expressions with its words, it falls within the scope of the IVA. This is important since in many cases (education, for instance) young children may be more attentive to non-human characters .

Abstract

A method of providing an interactive virtual assistant (IVA) is described, the method including the steps of receiving input from a user; processing the input to determine a task to be performed; performing the task; and generating and displaying an animated virtual character in response to the user. Also described is an apparatus for implementing such a method. The type of task to be performed depends on the role which the IVA is to perform. A number of embodiments are described, ranging from a virtual receptionist to a museum guide. Possible tasks include alerting a third party to the presence of a visitor; guiding a user around an exhibition; or arranging meetings and conferences via IVAs belonging to third parties.

Description

INTERACTIVE VIRTUAL ASSISTANT
The present invention relates to apparatus for implementing an interactive virtual assistant, such as a virtual receptionist or a virtual museum guide; and to a method of implementing such an interactive virtual assistant. The invention further relates to a method of raising revenue for individuals by licensing an image of the individual for use as virtual assistants.
"Virtual assistant", as used herein, refers to a computer or other processing device generated and animated character which is able to serve a user in an assistant role.
A number of relatively low-level human-based roles and tasks exist in a modern environment which would be amenable to automation, if suitable technology were found. For example, an office receptionist, a museum or tour guide, or a call centre operative, all require to deal with frequently repetitive tasks requiring little human creativity. However, people are reluctant to replace humans in these roles with automated substitutes, primarily because people prefer to interact with humans, so providing a sense of engagement and interest. Until recently, automated substitutes were not able to provide the same sense of engagement, and users of automated systems may have found such systems user-unf iendly in terms of their interface. Further, interactivity with automated systems has been difficult, since the typical input and output modes found are unfamiliar and awkward to a proportion of users. In addition, to many people it is important to see a human face with which to interact .
The present applicants, in their co-pending US Patent Application 09/766,718 entitled "Character Animation System" (the contents of which are incorporated herein by reference) describe a system whereby an animated character may be generated and displayed in substantially real time (or close to real time) with improved lip synchronisation and facial movements to generated speech, and with the incorporation of a number of additional movements and expressions. This results in a much improved and more realistic virtual character over previously known systems . Important features of this system include the application of a text-to-speech engine to generate phonemes from text to be spoken, and the use of a phoneme-to-viseme converter to generate visemes and morph targets from output phonemes. Selection of a restricted set of generated visemes to express, and incorporation of random expressions and additional visemes, results in a more naturalistic-looking character.
The present invention makes use of this system in the generation and implementation of an improved interactive virtual assistant. The invention is, however, not restricted to implementation using the specific virtual character technology, but can make use of any suitable character generation system having similar characteristics.
According to a first aspect of the present invention, there is provided an apparatus for providing an interactive virtual assistant implemented by a virtual character, the apparatus comprising: display means for displaying said virtual character; user interaction means for receiving input from a user; processing means for processing received input; task execution means for performing a task in response to received input; and response generation means for responding to said user including character animation means for generating and animating said virtual character on said display means in response to received input .
Thus, the present invention allows a user to provide input (such as instructions, questions, or an identification of the user) to an interactive virtual assistant (hereinafter referred to as an IVA) . The IVA then performs a task in response to the input, and a character is animated to provide a response to the user. The response may be visual, audible, tactile, physical, or a combination of any of these. The animated character provides a human- like interface for the user, allowing an easier interaction than if no human were involved. The invention therefore allows an IVA to take over a number of roles which would normally otherwise be performed by a human.
The user interaction means may comprise any suitable device for providing an interface between the user and the IVA. For example, a touch sensitive display screen; a voice recognition device; a smart card or magnetic swipe card reader, which may for example identify a user as they swipe a security card in the reader; a keyboard or numeric keypad; an infra-red communications port, for communicating with, for example, a mobile communications device such as a laptop or palmtop computer, a personal digital assistant (PDA) , a mobile telephone, or the like; a wireless technology (such as Bluetooth) detection device for interacting with similarly-equipped mobile devices; a hardwired communications device, such as a telephone or videophone; sensors such as a pressure-sensitive footpad for detecting the presence of a user. Any or all of these devices may be used in combination: for example, a pressure- sensitive footpad may be used to alert the IVA of the presence of a user, while a voice recognition device may be used for the input of further data or commands.
Where the user interaction means is able to interact with mobile devices such as mobile telephones or PDAs, or with smart cards or the like, the apparatus may automatically identify the user based on some distinguishing characteristic encoded in the mobile device or smart card. This will enable the apparatus to provide a more efficient and personal service to the user.
Preferably the processing means comprises a computing device. The computing device may be of conventional type, and may be running known computer software for processing the received input. Alternatively, specific software may be written for the implementation of the IVA, depending on the type of input liable to be received. Hardware or firmware may of course be used in place of computer software.
The task execution means will typically comprise a computing device, or a computer program (in the form of hardware or software) . Tasks to be executed may be explicitly included in the user input (for example, a request to find a particular person's telephone number or the like) ,- or may be implicit
(for example, user input in the form of activation of a pressure sensitive pad may implicitly include the task to be executed of 'activate the IVA and greet the user' ) .
Particular tasks which may be performed depend on the particular implementation of the invention used; typically a restricted range of tasks are available, although in theory, a sufficiently powerful artificial intelligence system is able to deal with an unlimited range of tasks. Example tasks may include: determining the subsequent content of speech to be made by the IVA (for example, where the input includes the user's name or the like, the implicit task may be to generate a phoneme and viseme output to greet the user by name; or where the input is a selection of one of a range of options, the task may be to provide task-dependent sub-options for example to further specify details of the task) ; alerting a third party to the presence of the user (for example, where the IVA is acting as a virtual receptionist) , the third party may be alerted by email, telephone, or the like; retrieving information for the user, from for example a remote or local database, the internet, an expert system, or the like; contacting and interacting with an IVA belonging to a third party. This last task may be particularly useful in the implementation of a virtual PA' ; a user may instruct the IVA to arrange a meeting with a third party at a mutually convenient time, whereupon the IVA liaises with a similarly-enabled IVA belonging to the third party. Once both IVAs have performed their tasks of arranging a time and place for the meeting, the IVAs then report to their respective users.
As stated above, the particular tasks which are available depend on the purpose for which the IVA is available. The task execution means is implementable in software, such that the available tasks may be readily reconfigured to allow for different functionalities to be implemented. In particular, roles for an IVA according to the present invention include, but are not limited to, virtual receptionist; personal secretary; hotel room assistant; environment guide; virtual advisor; virtual trainer; language teacher; e-commerce host; domestic unit; children's assistant. Each of these will be described in more detail later; additional roles are of course possible.
Preferably the character animation means comprises a computing device together with a suitable computer program (whether hardware or software) . The computer program is preferably as described in our co-pending Patent Application "Character Animation System", referred to above, although any suitable program may be used. Preferably the character animation is generated in substantially real-time.
The virtual character may be any moving, talking virtual image, not necessarily human. For example, if used as a children's assistant, it may be preferable to use a zooform assistant, although animated with human-characteristic expressions and mannerisms. The apparatus may further comprise means for selecting a virtual character to be displayed. For example, the IVA may be selectable from a range of preset possibilities (for example, characters may be available in a range of ethnicities and genders, as well as species) . The character selection means may also or instead include means for adding new characters to the available options; for example, additional characters may be purchased or rented from the apparatus provider, or new characters may be created based on user-added data. For example, a user may wish to create a character based on themselves to act as a virtual PA or the like. In addition to animation of the character in response to received input, at least a portion of the animation may be dependent on the task performed; for example, where the task is to retrieve particular information from a database, the animation depends on the data thereby retrieved; or if the task is to alert a particular person to the ' presence of the user, the animation is dependent on whether the particular person can be alerted or not.
The animated character preferably includes facial movements and expressions corresponding to generated speech; the animation may in addition or alternatively include gestural animations (for example, to provide an output in sign language) , text output (as subtitles, or to present retrieved data which cannot conveniently be presented as speech), still pictures or images, and prerecorded video clips or the like.
Preferably the display means comprises a display screen. For example, a cathode-ray tube screen, an LCD screen, a projection screen, or the like. Alternatively the display means may comprise a holographic display, three-dimensional display means, or similar device. The apparatus may further include sound generation means, for generating speech sounds . The display means may be local to the character animation means, or may be remote. Where the display means is remote, the display may be physically connected to the character animation means (for example, fibre optic or other data communication cables) , or may be in remote data communication (for example, via radio, infra red, or other electromagnetic communication, such as is used in Bluetooth or similar wireless technology) . A remote display means may be incorporated in some other device, such as a personal digital assistant (PDA), mobile telephone, videophone, or the like. In certain embodiments of the invention, the remote display means may be portable; this may be particularly useful where the IVA is to be used as a museum guide or the like. Any other components of the apparatus may also be portable along with the display means; for example, a computer processor may be located at a central location, and a user interaction means and display means provided as a portable handset .
The apparatus may in certain embodiments comprise a plurality of display means, and may optionally also comprise a plurality of user interaction means. For example, a display screen may be located within each of a plurality of rooms in a building, with user interaction means arranged to alert the IVA to the presence and identity of a user adjacent each screen. Thus, the IVA may be aware of a user as they move about a building, and the IVA may be on hand in each room to assist the user. A plurality of distinct virtual characters may of course be generated in such an embodiment, each assisting a different user as they move about a building. According to a second aspect of the present invention, there is provided a method of providing an interactive virtual assistant to a user, the method comprising the steps of: receiving input from a user; processing the input to determine a task to be performed; performing said task in response to the input; responding to the user at least in part by generating and animating a virtual character in response to the input; and displaying the animated virtual character to the user.
According to a further aspect of the present invention, there is provided a computer program product for providing an interaction to a user, the product comprising a computer-readable medium having thereon computer program code for : receiving input from a user; processing the input to determine a task to be performed; performing said task in response to the input; responding to the user at least in part by generating and animating a virtual character in response to the input; and displaying the animated virtual character to the user.
According to a yet further aspect of the present invention, there is provided a method of generating licensing revenue for an individual, the method comprising the steps of: reproducing distinctive characteristics of the individual as a virtual character; charging third parties to implement the virtual character in any of the above methods according to the invention; and providing at least a portion of an amount charged to third parties to the individual or a representative thereof.
The distinctive characteristics may include the individual's appearance, speech patterns, or mannerisms. Which particular characteristics are reproduced will obviously depend on the individual, and may also depend on the role which the virtual character is to play (for example, if the character is in animal form, the degree of reproduction of the individual's appearance will be lessened, while the reproduction may focus on speech and mannerisms) .
According to a further aspect of the present invention, there is provided a method of generating revenue for an individual, the method comprising the step of charging a third party for reproducing distinctive characteristics of the individual as a virtual character.
According to a yet further aspect of the present invention, there is provided a method of marketing goods or services, the method comprising the steps of: generating and animating a virtual character having distinctive characteristics of an individual; and using the virtual character to promote said goods or services .
Preferably the individual is a celebrity.
These and other aspects of the present invention will now be described by way of example only and with reference to the accompanying drawings, in which :
Fig. 1 shows an outline flow chart illustrating the implementation of the method of providing an interaction to a user in accordance with one embodiment of the present invention;
Fig. 2 shows a sketch of an apparatus for implementing an interactive virtual assistant, in accordance with one embodiment of the present invention; and
Figs. 3a to 3d show various stages in the implementation of a virtual receptionist, in accordance with one embodiment of the present invention.
Referring now to Fig. 1, this shows an outline flow chart illustrating the implementation of a method of providing a virtual assistant, in accordance with one embodiment of the present invention. The figure shows a client or user 12 of the interactive virtual assistant (IVA) interacting with one of a number of possible input devices 1 . The user 12 interacts with an input device 14 to provide information referring to a task to be performed to a central processor 16. The input device 14 is the means by which the user 12 makes his or her requirements known to the IVA. The device 14 can be anything which can convey data, ranging, for example, from simple touch-screen push-buttons, through standard keyboard to speech input. The content of the user's information can range from the simplest λyes/no' button selection, through menu selection to text input which can be understood by the processor 16 to sophisticated speech recognition devices.
The processor 16 analyses the input received from the devices 14, and performs the requested task in an appropriate manner. The processor 16 carries out or sets in motion some kind of task for the user, based on the requirements conveyed by the user, or implied on behalf of the user. As an example of 'implied1, when a visitor walks up to an idling virtual image and steps on a pressure pad, the IVA infers that the visitor wants attention, even although this is not specifically indicated. The task can be any which results in a response to the client, for example, a spoken response by the IVA, or a physical event, or the like.
After the task has completed, or while the task is being performed, a virtual character 18 is generated and animated. The processor 16 determines what the character 18 will say, and with what mood or attitude, or emotional expressions. The animation itself is conducted by a self-contained program suite associated with the virtual character 18. The generation and animation of the virtual character is substantially as described in our co-pending patent application referred to above; although any computer-generated character technology may be used. The virtual character is any moving, talking virtual image, normally but not necessarily human, which reports a response to the task carried out by the IVA to the user. In most embodiments, only a face or head is used, but a full body image may be used provided some part generates expressions synchronised to the words to provide additional information to a user. Information is normally audio (spoken) , but may be supported by moving or still pictures or diagrams in windows inset in part of the screen. Optionally, the spoken information can be duplicated by printed text or sign language for the deaf. Dialogue with the client can take place in any language with which the IVA has been programmed.
Once the task has been completed, the virtual character 18 is animated to provide an output 20 to the user 12. The output 20 may be the results of the task, or may be a request for further information or clarification as to the nature of the task from the client; for example, if the client makes an ambiguous request to see an employee, the IVA may ask the client to specify the employee more precisely; or if a database search returns an excessive number of 'hits', the IVA may prompt the client to narrow the field of search by specifying more terms. Further implementation of the method consists of repeating the stages of the method until the user or IVA indicates that the exchange is at an end; for example, by leaving the vicinity of the IVA, or by saying 'goodbye' or the like. While the cycling through the method is being performed, the user has the impression that a sustained dialogue is being conducted with the virtual character, while their requests are being carried out.
Fig. 2 shows a sketch of a specific apparatus for providing an interactive virtual assistant . The apparatus 30 can be used to provide a 'virtual receptionist', as will be described below. The apparatus 30 comprises a processing device 32, which performs the interpretation of input, generation of animations, and execution of tasks; a display screen and loudspeaker 34, on which animations of the IVA are displayed to a user; and a touch sensitive portion of the screen 36, by which a user can enter data to the processor 32. The processor 32 is connected via a data communications network to one or more third party computers 38, which are not themselves part of the apparatus.
The use of the apparatus 30 in the provision of a virtual receptionist will now be described. Images of the receptionist are shown in Figs. 3a to 3d. The Virtual Receptionist
The 'Virtual Receptionist' is an animated character (for example, that illustrated in Fig. 3a) generated by a PC or other processing device 32. The 'public face' of the receptionist is seen on a screen 34 in the reception foyer of the organisation employing the IVA. A visitor arriving on the premises sees a screen 34 showing the image of the IVA shown in Fig. 3a. The visitor also sees an area on a touch sensitive screen 36 containing the message: 'Please touch the screen here for assistance'. Compliance activates a menu containing a list of company personnel names (Fig. 3b) . It also triggers a spoken and animated welcome from the IVA suggesting the visitor select a name from the list. Touching a name prompts the IVA to ask the visitor to type his or her name into a text box (Fig. 3c) . Typing may be conducted via the touch sensitive screen 36, or by means of a separate keyboard provided for the purpose. This done, the IVA sends an internal email or other message to the computer 38 of the required staff member, alerting the staff member to the arrival of the visitor. A preferred embodiment of the invention uses the company telephone network to place a telephone call to the required staff member, alerting them to the presence of the visitor by means of a computer-generated or prerecorded voice message. Use of a telephone call has the advantage that the staff member may provide the IVA with further instructions as to how to deal with the visitor immediately. Once the message has been sent, the IVA invites the visitor to take a seat until the contacted staff member can get there (Fig. 3d) , by speaking a phrase with the visitor's name, for example. If the required staff member cannot be reached; for example, if they do not acknowledge the message within a short time, the IVA goes through the staff list for an alternative, until someone deals with the visitor. This list is in the preferred order appropriate to the original required contact .
In alternative embodiments of the invention, the IVA can recognise that one or more visitors have arrived, and thus be activated. This can be done with any of a number of techniques including beam interruption, swipe-card use, camera visual interpretation etc. As a default, the touch screen will always be available.
In appropriate embodiments, the IVA is able to determine the identity of the visitor by associating the time of arrival with the company's day calendar. This is subject to ambiguity if two or more visitors have close appointments, or an unscheduled arrival coincides with an expected visitor. To eliminate any mistakes, The IVA will ask the visitor to confirm the identification before proceeding.
A number of other applications of the interactive virtual assistant will now be described. These applications all make use of the same underlying technology as the virtual receptionist just described, and would require little additional technology for implementation. The Personal Secretary
As the name implies, the Virtual Personal Secretary (VS) variant of the IVA provides the client with a service on a one-to-one basis. The client's VS, through its on-board Al or other adaptive learning program, becomes customised to his or her needs, learning and acquiring relevant knowledge with time. It effectively travels with its client, and can be used wherever a telephone socket can be reached. Among the services which the VS provides are : A personal organiser with an address book, general notebook and diary built-in.
A booking secretary which can interact with online booking services to arrange travel, entertainment, and the like. A meetings secretary which can arrange meetings through other VSs. That is, the task carried out by the VS in this embodiment is to contact a second VS, and act as the input for this second VS to arrange a meeting time for the humans involved.
A conference call arranger which is an extension of the meetings secretary.
The Hotel Room Assistant
As soon as a guest registers in a hotel, his identity is recorded with the hotel's computer system. When he enters his room, his television activates and a virtual character greets him personally. It then invites the guest to select further services from information to room service. For those who do not like or want the service, one option is to immediately deactivate it.
The Assistant may offer up-to-the-minute hotel information and services including: billing details, restaurant information including current access, room service facilities, leisure facilities, location maps, external facilities including local attractions, and an information service for specific events such as conferences .
Advanced versions of the Assistant learn its client's preferences, and mould its behaviour to fit. Once the guest is registered, the system knows whether he or she is a regular with the hotel or its chain. If so, the history the Assistant has built up is accessed, and is continued from that point. Failing that, the Assistant defaults to a neutral set of parameters, and starts building a user profile from there.
The Environment Guide
The Guide is a virtual character which talks its client through a physical trip or tour. The guide operates in various settings such as permanent buildings, temporary exhibitions, cruise liners, country parks, towns and so on. Features of the Guide include:
Understanding the intent of the customer being guided. Regularly updated information about locations in the target domain.
Networked terminals through which the customer and the IVA can communicate .
Hand-held terminals through which the customer and the IVA can communicate.
On-going routing to a required destination.
Specification of several destinations on tour route.
Non-shortest route to avoid temporary ' traffic- jams', congestion data updated continuously.
Omission of selected routing options, e.g., no lifts, stairs only (or vice-versa) .
An example is a large motor show. The entire exhibition area including access foyer is modelled as a terrain map. (The model could also be extended to the various car parks) . There may be about eight halls in use, each hall being defined as a region, and the doors between the halls, to the foyer and outside as accesses. Within each hall, stands, stalls, bars, snack bars, first-aid stations and toilets are defined as region objects. These are linked by corridors. Large stands have stations for vehicles, and may have exhibition areas, demonstration rooms and/or discussion rooms. These are all defined as part of the base model for the guide, and used for the routing required by some of the above features.
As part of the service, the system contains information on various locations. This could be the vehicle models on each of the stand stations, times of demonstrations, personnel present at a stand (in case a client is arriving for an appointment, for instance) and, importantly, how crowded the stand is and how long the queues at any particular model. This and other information is dependent on regular updating by stand personnel. Automating this depends on the exhibitors' willingness to invest in (buy or hire) the necessary equipment. The system would collate the required information and the Guide inform the user of it .
A variant or supplement is a hand-held device using 'Bluetooth' or a comparable wire-free technology in place of the fixed-location screens. The essential properties do not change . The Guide becomes mobile and travels with its client. With this variant, information is provided at any time instead of just at fixed locations, and the irritation of possible queuing is removed.
The Virtual Advisor
The IVA becomes an advisor on any of a number of topics. Top of its agenda in this role is helping in filling out forms which, however well-intentioned their designers, often contain ambiguities to the unfamiliar eye. A typical example is a tax advisor.
The Advisor provides a computer-based alternative to an inconvenient visit to the nearest tax office or studying obscure tax guides . The support software behind the IVA contains a knowledge-base of tax matters, and a highly targeted means of interpreting the client's questions. The client is interactively guided through the knowledge framework to the answers he or she requires .
The Virtual Trainer
The Trainer is an application area in which the IVA has the task of training and testing personnel in dealing with the public, and typical and emergency situations which they might face.
Staff training and assessment are vitally important in many areas, and often either neglected or minimally carried out by expensive lecturing and role-playing. The IVA trainer is a low-cost means of rapidly bringing new trainees up to speed, or early-culling unsuitable ones. A typical example is that, of the hotel receptionist, who is unlikely to become extinct even with the proliferation of the presently-described virtual receptionist. It could, however, be used for counter-staff, cabin-staff, waiters, maintenance engineers, room-service providers, demonstrators, salespeople and managers in hotels, airports, airlines, banks, exhibitions, supermarkets and so on. It needs to be stressed that the virtual trainer is not conceived as a replacement for current training and testing methods, but rather an addition to the trainer's armoury .
The ever-present problem of understanding the trainee's input is kept as simple as possible. Input is from a combination of menu selection, 'yes' and 'no' to direct questions, and typed keywords. Well-focussed input is also an implementation option.
A set of IVAs address the trainee with pre-scripted words and manners ranging from pleasant and apologetic to belligerent. As an example, during a run, an IVA simulating a good-natured old lady approaches the trainee receptionist and makes a polite demand. The receptionist responds in some way (possibly from the key options for facts and attitude) and the IVA continues the conversation. During this dialogue, a second IVA playing the part of a belligerent man arrives, interrupts and demands immediate attention in an unpleasant way. How does the receptionist respond?
Because the system knows both the trainee's and the IVA's responses, a record can be kept of each session, and the trainee's performance graded automatically. Furthermore, the session can be rerun with an IVA also playing the trainee's role showing both what the trainee did and what he or she should have done. The grading capability allows the system to be used for a certain level of performance assessment .
The Language Teacher
Interactive language teaching is a well-established application of computers. The employment of an IVA gives the process a more human face. Basic versions simply prompt the student according to his or her requests according to a pre-scripted plan. In more advanced versions, the Teacher is backed by an Al which adapts automatically to its assessment of the student's progress. It is able to make objective assessments of progress.
The e-Commerce Host
An appropriately presented IVA hosts sales sites for every kind of transaction from simple supermarket shopping to buying and contracting business services. As with language teaching, this on-line service is well-established, but when it becomes the task element of the IVA interactive cycle it is new.
The Domestic Unit
A householder is in direct communication with his house as a whole, or individual appliances within it. Such devices range from lights and curtains to washing machines, cooker, and the like. The range would also include security systems. To comply with the IVA definition, there needs to be a virtual character in the loop, and therefore some medium, probably a display screen although possibly holographic projection, to carry it. This could be a small screen attached to the domestic unit, a touch screen or computer in one or more key locations or a mobile device of the kind described for the 'Guide' earlier. In this case, and optionally for the fixed screens, a wireless link such as 'Bluetooth' would be used.
The Children's Assistant
This application is alternatively described as the Junior Interactive Virtual Assistant . It is a series of re-makes of the previous application areas geared for use by children in various age groups . Among the distinguishing features are: A particularly intuitive and age group friendly interface .
Lower touch screens for younger age groups .
Child-targeted detail within the application areas . Automatic filters denying children access to potentially damaging material or controls.
A side benefit of age-group-layered child-specific systems is that children become familiar with interactive computing from the earliest age. The child thus becomes comfortable with online working. It has been noted above that the virtual character embedded in the IVA need not be human. If it is not, the definition still applies. The only difference is that the face will be a non-human such as an alien, or animal, or any kind of anthropomorphic image . As long as the virtual character talks and synchronises its expressions with its words, it falls within the scope of the IVA. This is important since in many cases (education, for instance) young children may be more attentive to non-human characters .
It can be seen from the foregoing that the present invention provides a versatile and adaptable virtual assistant technology which may be used in a wide range of roles, for conducting a large variety of tasks. It will be understood that the above- described applications are not exhaustive, and that modifications and variations may be made to the virtual assistant without departing from the scope of the invention.

Claims

1. Apparatus for providing an interactive virtual assistant implemented by a virtual character, the apparatus comprising: display means for displaying said virtual character; user interaction means for receiving input from a user; processing means for processing received input; task execution means for performing a task in response to received input; and response generation means for responding to said user including character animation means for generating and animating said virtual character on said display means in response to received input .
2. The apparatus of claim 1 wherein the processing means comprises a computing device.
3. The apparatus of claim 1 or claim 2, wherein the task execution means comprises a computing device.
4. The apparatus of any one of the preceding claims, wherein the task comprises determining the subsequent content of speech to be made by the IVA.
5. The apparatus of any one of the preceding claims wherein the task comprises alerting a third party to the presence of the user.
6. The apparatus of any one of the preceding claims, wherein the task comprises contacting and interacting with an IVA belonging to a third party.
7. The apparatus of any one of the preceding claims, wherein the character animation means comprises a computing device together with a suitable computer program.
8. The apparatus of any one of the preceding claims wherein the character animation is generated in substantially real-time.
9. The apparatus of any one of the preceding claims further comprising means for selecting a virtual character to be displayed.
10. The apparatus of any one of the preceding claims, wherein the animated character includes facial movements and expressions corresponding to generated speech.
11. The apparatus of any one of the preceding claims wherein the animation includes gestural animations, text output, still pictures or images, or pre-recorded video clips.
12. The apparatus of any one of the preceding claims, wherein the display means comprises a display screen.
13. The apparatus of any one of the preceding claims wherein the display means is local to the character animation means.
14. The apparatus of any one of the preceding claims, wherein the display means is remote from the character animation means.
15. The apparatus of any one of the preceding claims comprising a plurality of display means, and optionally also comprising a plurality of user interaction means.
16. A method of providing an interactive virtual assistant to a user, the method comprising the steps of: receiving input from a user; processing the input to determine a task to be performed; performing said task in response to the input; responding to the user at least in part by generating and animating a virtual character in response to the input; and displaying the animated virtual character to the user.
17. A computer program, preferably on a data carrier or a computer-readable medium, the computer program having instructions or code for: receiving input from a user; processing the input to determine a task to be performed; performing said task in response to the input; responding to the user at least in part by generating and animating a virtual character in response to the input; and causing the animated virtual character to be displayed to the user.
18. The computer program of claim 17, wherein the task comprises determining the subsequent content of speech to be made by the IVA.
19. The computer program of claim 17 or claim 18, wherein the task comprises alerting a third party to the presence of the user.
20. The computer program of claim 17 or claim 18 or claim 19, wherein the task comprises contacting and interacting with an IVA belonging to a third party.
21. The computer program of any one of claims 17 to 20, wherein the character animation is generated in substantially real-time.
22. The computer program of any one of claims 17 to 21 further comprising computer program code for selecting a virtual character to be displayed.
23. The computer program of any one of claims 17 to 22, wherein the animated character includes facial movements and expressions corresponding to generated speech.
24. The computer program of any one of claims 17 to 23, wherein the animation includes gestural animations, text output, still pictures or images, or pre-recorded video clips.
25. A method of generating licensing revenue for an individual, the method comprising the steps of: reproducing distinctive characteristics of the individual as a virtual character; charging third parties to implement the virtual character in a method according to claim 16; and providing at least a portion of an amount charged to third parties to the individual or a representative thereof.
26. A method of generating revenue for an individual, the method comprising the step of charging a third party for reproducing distinctive characteristics of the individual as a virtual character.
27. A method of marketing goods or services, the method comprising the steps of: generating and animating a virtual character having distinctive characteristics of an individual; and using the virtual character to promote said goods or services.
PCT/GB2002/000202 2001-01-22 2002-01-18 Interactive virtual assistant WO2002057896A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2002225160A AU2002225160A1 (en) 2001-01-22 2002-01-18 Interactive virtual assistant

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US76724201A 2001-01-22 2001-01-22
US09/767,242 2001-01-22

Publications (2)

Publication Number Publication Date
WO2002057896A2 true WO2002057896A2 (en) 2002-07-25
WO2002057896A3 WO2002057896A3 (en) 2004-03-04

Family

ID=25078909

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2002/000202 WO2002057896A2 (en) 2001-01-22 2002-01-18 Interactive virtual assistant

Country Status (2)

Country Link
AU (1) AU2002225160A1 (en)
WO (1) WO2002057896A2 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004102369A1 (en) * 2003-05-14 2004-11-25 Ncr International, Inc Self-service terminal
GR20030100229A (en) * 2003-05-22 2005-01-25 Ινστιτουτοαπληροφορικησακαιατηλεματικης/Εθνικοακεντροαερευνασακαιατεχνολογικησααναπτυξησαα Object, method and proceddure for the development od advanceddddd information technology applications with the use of virtuaal reality
US6929176B2 (en) 2002-05-04 2005-08-16 Ncr Corporation Self-service terminal
WO2012106815A1 (en) * 2011-02-11 2012-08-16 4D Retail Technology Corp. System and method for virtual shopping display
CN102844738A (en) * 2010-02-02 2012-12-26 4D零售科技公司 Systems and methods for human intelligence personal assistance
CN103489210A (en) * 2013-09-06 2014-01-01 雷路德 Virtual character proactive contacting system and method thereof
CN103500244A (en) * 2013-09-06 2014-01-08 雷路德 Virtual friend conversational system and method thereof
US9361628B2 (en) 2010-04-13 2016-06-07 Dean Stark Interactive video shelving system
US9367869B2 (en) 2012-02-13 2016-06-14 Dean Stark System and method for virtual display
WO2016187477A1 (en) * 2015-05-20 2016-11-24 Daqri, Llc Virtual personification for augmented reality system
US9733699B2 (en) 2010-04-13 2017-08-15 Dean Stark Virtual anamorphic product display with viewer height detection
US10887161B1 (en) 2017-10-16 2021-01-05 Wells Fargo Bank, N.A. Multi-device interface application
CN112199002A (en) * 2020-09-30 2021-01-08 完美鲲鹏(北京)动漫科技有限公司 Interaction method and device based on virtual role, storage medium and computer equipment
US11113080B2 (en) 2017-02-06 2021-09-07 Tata Consultancy Services Limited Context based adaptive virtual reality (VR) assistant in VR environments
CN114115524A (en) * 2021-10-22 2022-03-01 青岛海尔科技有限公司 Interaction method of intelligent water cup, storage medium and electronic device
US11704501B2 (en) 2017-11-24 2023-07-18 Microsoft Technology Licensing, Llc Providing a response in a session

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0691609A1 (en) * 1994-07-08 1996-01-10 Microsoft Corporation Software platform having a real world interface with animated characters
WO1997005540A1 (en) * 1995-07-26 1997-02-13 Benjamin Badinter Method for displaying information on a computer screen, related animated information medium, and system therefor
US5657462A (en) * 1993-11-17 1997-08-12 Collegeview Partnership Method and apparatus for displaying animated characters upon a computer screen in which a composite video display is merged into a static background such that the border between the background and the video is indiscernible
KR20000063179A (en) * 2000-04-20 2000-11-06 이창대 Advertisement of goods, method of selling, system thereof, by utizing sound and motion image of famous men(talent, singer, sportsman, gag-man)
WO2000067111A1 (en) * 1999-05-03 2000-11-09 Screenfriends Corporation An animated, interactive computer interface system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5657462A (en) * 1993-11-17 1997-08-12 Collegeview Partnership Method and apparatus for displaying animated characters upon a computer screen in which a composite video display is merged into a static background such that the border between the background and the video is indiscernible
EP0691609A1 (en) * 1994-07-08 1996-01-10 Microsoft Corporation Software platform having a real world interface with animated characters
WO1997005540A1 (en) * 1995-07-26 1997-02-13 Benjamin Badinter Method for displaying information on a computer screen, related animated information medium, and system therefor
WO2000067111A1 (en) * 1999-05-03 2000-11-09 Screenfriends Corporation An animated, interactive computer interface system
KR20000063179A (en) * 2000-04-20 2000-11-06 이창대 Advertisement of goods, method of selling, system thereof, by utizing sound and motion image of famous men(talent, singer, sportsman, gag-man)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6929176B2 (en) 2002-05-04 2005-08-16 Ncr Corporation Self-service terminal
WO2004102369A1 (en) * 2003-05-14 2004-11-25 Ncr International, Inc Self-service terminal
GR20030100229A (en) * 2003-05-22 2005-01-25 Ινστιτουτοαπληροφορικησακαιατηλεματικης/Εθνικοακεντροαερευνασακαιατεχνολογικησααναπτυξησαα Object, method and proceddure for the development od advanceddddd information technology applications with the use of virtuaal reality
CN102844738A (en) * 2010-02-02 2012-12-26 4D零售科技公司 Systems and methods for human intelligence personal assistance
US9733699B2 (en) 2010-04-13 2017-08-15 Dean Stark Virtual anamorphic product display with viewer height detection
US9361628B2 (en) 2010-04-13 2016-06-07 Dean Stark Interactive video shelving system
WO2012106815A1 (en) * 2011-02-11 2012-08-16 4D Retail Technology Corp. System and method for virtual shopping display
US20120206449A1 (en) * 2011-02-11 2012-08-16 Dean Stark System and method for virtual shopping display
US9563906B2 (en) 2011-02-11 2017-02-07 4D Retail Technology Corp. System and method for virtual shopping display
CN103562982A (en) * 2011-02-11 2014-02-05 4D零售科技公司 System and method for virtual shopping display
US9367869B2 (en) 2012-02-13 2016-06-14 Dean Stark System and method for virtual display
CN103500244A (en) * 2013-09-06 2014-01-08 雷路德 Virtual friend conversational system and method thereof
CN103489210A (en) * 2013-09-06 2014-01-01 雷路德 Virtual character proactive contacting system and method thereof
WO2016187477A1 (en) * 2015-05-20 2016-11-24 Daqri, Llc Virtual personification for augmented reality system
US11113080B2 (en) 2017-02-06 2021-09-07 Tata Consultancy Services Limited Context based adaptive virtual reality (VR) assistant in VR environments
US10887161B1 (en) 2017-10-16 2021-01-05 Wells Fargo Bank, N.A. Multi-device interface application
US11855838B1 (en) 2017-10-16 2023-12-26 Wells Fargo Bank, N.A. Multi-device interface application
US11704501B2 (en) 2017-11-24 2023-07-18 Microsoft Technology Licensing, Llc Providing a response in a session
CN112199002A (en) * 2020-09-30 2021-01-08 完美鲲鹏(北京)动漫科技有限公司 Interaction method and device based on virtual role, storage medium and computer equipment
CN114115524A (en) * 2021-10-22 2022-03-01 青岛海尔科技有限公司 Interaction method of intelligent water cup, storage medium and electronic device
CN114115524B (en) * 2021-10-22 2023-08-18 青岛海尔科技有限公司 Interaction method of intelligent water cup, storage medium and electronic device

Also Published As

Publication number Publication date
AU2002225160A1 (en) 2002-07-30
WO2002057896A3 (en) 2004-03-04

Similar Documents

Publication Publication Date Title
US11823677B2 (en) Interaction with a portion of a content item through a virtual assistant
US10628635B1 (en) Artificially intelligent hologram
Kanda et al. An affective guide robot in a shopping mall
McTear Spoken dialogue technology: toward the conversational user interface
Kanda et al. A communication robot in a shopping mall
WO2002057896A2 (en) Interactive virtual assistant
US20110004481A1 (en) System and method for communicating and interfacing between real and virtual environments
US20040089709A1 (en) RFID activated information kiosk
Krüger et al. Adaptive mobile guides
Koay et al. Companion migration–initial participants’ feedback from a video-based prototyping study
Alves Lino et al. Responsive environments: User experiences for ambient intelligence
CN113641442A (en) Interaction method, electronic device and storage medium
CN114186045A (en) Artificial intelligence interactive exhibition system
JP2001195430A (en) System for expediting knowledge sharing
Lee Designing personalization in technology-based services
Budkov et al. Dialog model development of a mobile information and reference robot
Polak et al. Artificial Intelligence in Communication with Music Fans: An Example from South Korea
Zikky et al. Utilizing Virtual Humans as Campus Virtual Receptionists
Ravikumar et al. A proposal for introducing Virtual Reality Assistant at Dubai Women's College
Wahlster et al. The shopping experience of tomorrow: Human-centered and resource-adaptive
Nijholt Multimodality and ambient intelligence
Stewart-Halevy The Inductive Turn in Conceptual Art
Nijholt et al. Maps, agents and dialogue for exploring a virtual world
CN110998725B (en) Generating a response in a dialog
Nijholt From virtual environment to virtual community

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SL TJ TM TN TR TT TZ UA UG US UZ VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

WWE Wipo information: entry into national phase

Ref document number: 2002715511

Country of ref document: EP

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWW Wipo information: withdrawn in national office

Ref document number: 2002715511

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP