CA2346606A1 - Method and apparatus for coupling a visual browser to a voice browser - Google Patents

Method and apparatus for coupling a visual browser to a voice browser Download PDF

Info

Publication number
CA2346606A1
CA2346606A1 CA002346606A CA2346606A CA2346606A1 CA 2346606 A1 CA2346606 A1 CA 2346606A1 CA 002346606 A CA002346606 A CA 002346606A CA 2346606 A CA2346606 A CA 2346606A CA 2346606 A1 CA2346606 A1 CA 2346606A1
Authority
CA
Canada
Prior art keywords
browser
visual
network
formatted
voice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002346606A
Other languages
French (fr)
Other versions
CA2346606C (en
Inventor
Bruce D. Lucas
David A. Reich
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of CA2346606A1 publication Critical patent/CA2346606A1/en
Application granted granted Critical
Publication of CA2346606C publication Critical patent/CA2346606C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/12Use of codes for handling textual entities
    • G06F40/14Tree-structured documents
    • G06F40/143Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M3/00Automatic or semi-automatic exchanges
    • H04M3/42Systems providing special services or facilities to subscribers
    • H04M3/487Arrangements for providing information services, e.g. recorded voice services or time announcements
    • H04M3/493Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
    • H04M3/4938Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals comprising a voice browser which renders and interprets, e.g. VoiceXML
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M2201/00Electronic components, circuits, software, systems or apparatus used in telephone systems
    • H04M2201/40Electronic components, circuits, software, systems or apparatus used in telephone systems using speech recognition

Abstract

A method and apparatus for concurrently accessing network-based electronic content in a Voice Browser and a Visual Browser can include the steps of retrieving a network-based document formatted for display in the Visual Browser; identifying in the retrieved document a reference to the Voice Browser, the reference specifying electronic content formatted for audible presentation in the Voice Browser; and, transmitting the reference to the Voice Browser. The Voice Browser can retrieve the specified electronic content and audibly present the electronic content. Concurrently, the Visual Browser can visually present the network-based document formatted for visual presentation in the Visual Browser. Likewise, the method of the invention can include the steps of retrieving a network-based document formatted for audible presentation in the Voice Browser;
identifying in the retrieved document a reference to the Visual Browser, the reference specifying electronic content formatted for visual presentation in the Visual Browser;
and, transmitting the reference to the Visual Browser. The Visual Browser can retrieve the specified electronic content and visually present the specified electronic content. Concurrently, the Voice Browser can audibly present the network-based document formatted for audible presentation in the Voice Browser.

Claims (27)

1. A method for concurrently accessing network-based electronic content in a Voice Browser and a Visual Browser comprising the steps of:

retrieving a network-based document formatted for display in the Visual Browser;
identifying in the retrieved document a reference to the Voice Browser, said reference specifying electronic content formatted for audible presentation in the Voice Browser; and, transmitting said reference to the Voice Browser;
the Voice Browser retrieving said specified electronic content and audibly presenting said electronic content in the Voice Browser;
the Visual Browser visually presenting said network-based document concurrently with said audible presentation.
2. The method according to claim 1, wherein said network-based document formatted for visual presentation in the Visual Browser is an HTML formatted document and the Visual Browser is a Web Browser.
3. The method according to claim 1, wherein said electronic content formatted for audible presentation in the Voice Browser is VoiceXML formatted electronic content.
4. The method according to claim 2, wherein said reference is a coordination markup attribute.
5. The method according to claim 1, wherein said reference specifies a network-based document containing said electronic content formatted for audible presentation in the Voice Browser, whereby the Voice Browser can retrieve said specified network-based document through the computer communications network and audibly present said electronic content contained in said specified network-based document, while the Visual Browser can visually present said network-based document formatted for visual presentation.
6. A method for concurrently accessing network-based electronic content in a Voice Browser and a Visual Browser comprising:
retrieving a network-based document formatted for audible presentation in the Voice Browser;
identifying in the retrieved document a reference to the Visual Browser, said reference specifying electronic content formatted for visual presentation in the Visual Browser; and, transmitting said reference to the Visual Browser;
the Visual Browser retrieving said specified electronic content and visually presenting said electronic content in the Visual Browser;
the Voice Browser audibly presenting said network-based document concurrently with said visual presentation.
7. The method according to claim 6, wherein said network-based document formatted for audible presentation in the Voice Browser is a VoiceXML document.
8. The method according to claim 6, wherein said electronic content formatted for visual presentation in the Visual Browser is HTML formatted electronic content and the Visual Browser is a Web Browser.
9. The method according to claim 6, wherein said reference specifies a network-based document containing said electronic content formatted for visual presentation in the Visual Browser, whereby the Visual Browser can retrieve said specified network-based document through the computer communications network and visually present said electronic content contained in said network-based document, while the Voice Browser can audibly present said network-based document formatted for audible presentation.
10. A machine readable storage, having stored thereon a computer program for concurrently accessing network-based electronic content in a Visual Browser and a Voice Browser, said computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:
retrieving a network-based document formatted for display in the Visual Browser;
identifying in the retrieved document a reference to the Voice Browser, said reference specifying electronic content formatted for audible presentation in the Voice Browser; and, transmitting said reference to the Voice Browser;
the Voice Browser retrieving said specified electronic content and audibly presenting said electronic content in the Voice Browser;
the Visual Browser visually presenting said network-based document concurrently with said audible presentation.
11. The machine readable storage according to claim 10, wherein said network-based document formatted for visual presentation in the Visual Browser is an HTML document and the Visual Browser is a Web Browser.
12. The machine readable storage according to claim 10, wherein said electronic content formatted for audible presentation in the Voice Browser is VoiceXML formatted electronic content.
13. The machine readable storage according to claim 11, wherein said reference is a coordination markup attribute.
14. The machine readable storage according to claim 10, wherein said reference specifies a network-based document containing said electronic content formatted for audible presentation in the Voice Browser, whereby the Voice Browser can retrieve said specified network-based document through the computer communications network and audibly present said electronic content contained in said specified network-based document, while the Visual Browser can visually present said network-based document formatted for visual presentation.
15. A machine readable storage, having stored thereon a computer program for concurrently accessing network-based electronic content in a Visual Browser and a Voice Browser, said computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:
retrieving a network-based document formatted for audible presentation in the Voice Browser;
identifying in the retrieved document a reference to the Visual Browser, said reference specifying electronic content formatted for visual presentation in the Visual Browser; and, transmitting said reference to the Visual Browser;
the Visual Browser retrieving said specified electronic content and visually presenting said electronic content in the Visual Browser:
the Voice Browser audibly presenting said network-based document concurrently with said visual presentation.
16. The machine readable storage according to claim 15, wherein said network-based document formatted for audible presentation in the Voice Browser is a VoiceXML
document.
17. The machine readable storage according to claim 15, wherein said electronic content formatted for visual presentation in the Visual Browser is HTML formatted electronic content and the Visual Browser is a Web Browser.
18. The machine readable storage according to claim 15, wherein said reference specifies a network-based document containing said electronic content formatted for visual presentation in the Visual Browser, whereby the Visual Browser can retrieve said specified network-based document through the computer communications network and visually present said electronic content contained in said specified network-based document, while the Voice Browser can audibly present said network-based document formatted for audible presentation.
19. A multi-modal browser comprising:

a Visual Browser for visually presenting visual content retrieved from a computer communications network;
a Voice Browser for audibly presenting voice content retrieved from said computer communications network; and, a Coupling Interface for synchronizing said visual and audible presentation of said visual and voice content in each of said respective Visual and Voice Browsers.
20. The multi-modal browser of claim 19, wherein said visual content is HTML
formatted content and said voice content is VoiceXML formatted content.
21. The multi-modal browser of claim 19, further comprising:
a Time Critical Event Handler, said Time Critical Event Handler listening for time critical content, said Time Critical Event Handler identifying in said time critical content a content-type indicator;
said Time Critical Event Handler forwarding said time critical content to said Visual Browser if said content-type indicator indicates visual content, said Time Critical Event Handler forwarding said time critical content to said Voice Browser if said content-type indicator indicates voice content.
22. A method for modifying a network-based document for supporting concurrent access to network-based voice and visual content in a Voice Browser and a Visual Browser, comprising the steps of:
incorporating visual content in a network-based document;
formatting said network-based document for visual presentation in a Visual Browser; and, inserting at least one markup tag in said network-based document, said markup tag containing a coordination markup attribute specifying a network-based document formatted for audible presentation in a Voice Browser, whereby the Visual Browser when rendering said network-based document formatted for visual display can identify said coordination markup attribute in said markup tag and can transmit a reference to said specified network-based document to the Voice Browser causing the Voice Browser to retrieve said specified network-based document and audibly present said specified network-based document concurrently with the visual presentation of said network-based document formatted for visual presentation in the Visual Browser.
23. The method according to claim 22, wherein said network based document formatted for visual presentation in a Visual Browser is an HTML formatted document and said Visual Browser is a Web Browser.
24. The method according to claim 22, wherein said network based document formatted for audible presentation in a Voice Browser is a VoiceXML formatted document and said Voice Browser is a VoiceXML Browser.
25. A method for modifying a network-based document for supporting concurrent access to network-based voice and visual content in a Voice Browser and a Visual Browser, comprising the steps of:
incorporating voice content in a network-based document;
formatting said network-based document for audible presentation in a Voice Browser;
inserting at least one markup tag in said network-based document;
said markup tag containing a coordination markup attribute specifying a network-based document formatted for visual presentation in a Visual Browser, whereby the Voice Browser when rendering said network-based document formatted for audible display can identify said coordination markup attribute in said markup tag and can transmit a reference to said specified network-based document to the Visual Browser causing the Visual Browser to retrieve said specified network-based document and visually present said specified network-based document concurrently with the audible presentation of said network-based document formatted for audible presentation in the Voice Browser.
26. The method according to claim 25, wherein said network based document formatted for visual presentation in a Visual Browser is an HTML formatted document and said Visual Browser is a Web Browser.
27. The method according to claim 25, wherein said network based document formatted for audible presentation in a Voice Browser is a VoiceXML formatted document and said Voice Browser is a VoiceXML Browser.
CA2346606A 2000-06-28 2001-05-07 Method and apparatus for coupling a visual browser to a voice browser Expired - Fee Related CA2346606C (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/605,612 2000-06-28
US09/605,612 US7080315B1 (en) 2000-06-28 2000-06-28 Method and apparatus for coupling a visual browser to a voice browser

Publications (2)

Publication Number Publication Date
CA2346606A1 true CA2346606A1 (en) 2001-12-28
CA2346606C CA2346606C (en) 2012-07-10

Family

ID=24424435

Family Applications (1)

Application Number Title Priority Date Filing Date
CA2346606A Expired - Fee Related CA2346606C (en) 2000-06-28 2001-05-07 Method and apparatus for coupling a visual browser to a voice browser

Country Status (4)

Country Link
US (4) US7080315B1 (en)
CA (1) CA2346606C (en)
DE (1) DE10125406A1 (en)
TW (1) TWI233045B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7356472B2 (en) 2003-12-11 2008-04-08 International Business Machines Corporation Enabling speech within a multimodal program using markup
US9065911B2 (en) 2007-09-28 2015-06-23 Nuance Communications, Inc. System, method and architecture for control and multi-modal synchronization of speech browsers

Families Citing this family (172)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8645137B2 (en) 2000-03-16 2014-02-04 Apple Inc. Fast, language-independent method for user authentication by voice
US7406657B1 (en) * 2000-09-22 2008-07-29 International Business Machines Corporation Audible presentation and verbal interaction of HTML-like form constructs
US7054939B2 (en) * 2001-06-28 2006-05-30 Bellsouth Intellectual Property Corportion Simultaneous visual and telephonic access to interactive information delivery
US6983307B2 (en) * 2001-07-11 2006-01-03 Kirusa, Inc. Synchronization among plural browsers
US6876727B2 (en) * 2002-07-24 2005-04-05 Sbc Properties, Lp Voice over IP method for developing interactive voice response system
US7275217B2 (en) * 2002-09-09 2007-09-25 Vijay Anand Saraswat System and method for multi-modal browsing with integrated update feature
US8311835B2 (en) * 2003-08-29 2012-11-13 Microsoft Corporation Assisted multi-modal dialogue
KR100561228B1 (en) * 2003-12-23 2006-03-15 한국전자통신연구원 Method for VoiceXML to XHTML+Voice Conversion and Multimodal Service System using the same
US7551727B2 (en) * 2004-10-20 2009-06-23 Microsoft Corporation Unified messaging architecture
US7912186B2 (en) * 2004-10-20 2011-03-22 Microsoft Corporation Selectable state machine user interface system
US7924985B2 (en) * 2005-04-21 2011-04-12 The Invention Science Fund I, Llc Interaction history applied to structured voice interaction system
US8139725B2 (en) * 2005-04-22 2012-03-20 The Invention Science Fund I, Llc Associated information in structured voice interaction systems
US7813485B2 (en) * 2005-05-26 2010-10-12 International Business Machines Corporation System and method for seamlessly integrating an interactive visual menu with an voice menu provided in an interactive voice response system
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US20080065715A1 (en) * 2006-08-28 2008-03-13 Ko-Yu Hsu Client-Server-Based Communications System for the Synchronization of Multimodal data channels
WO2008026197A2 (en) * 2006-08-28 2008-03-06 Mark Heifets System, method and end-user device for vocal delivery of textual data
US9318108B2 (en) * 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US7881932B2 (en) * 2006-10-02 2011-02-01 Nuance Communications, Inc. VoiceXML language extension for natively supporting voice enrolled grammars
US8229080B2 (en) * 2006-11-10 2012-07-24 Verizon Patent And Licensing Inc. Testing and quality assurance of multimodal applications
US8009811B2 (en) 2006-11-10 2011-08-30 Verizon Patent And Licensing Inc. Testing and quality assurance of interactive voice response (IVR) applications
US8886540B2 (en) * 2007-03-07 2014-11-11 Vlingo Corporation Using speech recognition results based on an unstructured language model in a mobile communication facility application
US8949266B2 (en) 2007-03-07 2015-02-03 Vlingo Corporation Multiple web-based content category searching in mobile search application
US8886545B2 (en) 2007-03-07 2014-11-11 Vlingo Corporation Dealing with switch latency in speech recognition
US8838457B2 (en) * 2007-03-07 2014-09-16 Vlingo Corporation Using results of unstructured language model based speech recognition to control a system-level function of a mobile communications facility
US8949130B2 (en) * 2007-03-07 2015-02-03 Vlingo Corporation Internal and external speech recognition use with a mobile communication facility
US10056077B2 (en) 2007-03-07 2018-08-21 Nuance Communications, Inc. Using speech recognition results based on an unstructured language model with a music system
US8880405B2 (en) * 2007-03-07 2014-11-04 Vlingo Corporation Application text entry in a mobile environment using a speech processing facility
US8054310B2 (en) * 2007-06-18 2011-11-08 International Business Machines Corporation Recasting a legacy web page as a motion picture with audio
US20090006965A1 (en) * 2007-06-26 2009-01-01 Bodin William K Assisting A User In Editing A Motion Picture With Audio Recast Of A Legacy Web Page
US7945847B2 (en) * 2007-06-26 2011-05-17 International Business Machines Corporation Recasting search engine results as a motion picture with audio
US10002189B2 (en) 2007-12-20 2018-06-19 Apple Inc. Method and apparatus for searching using an active ontology
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US20120311585A1 (en) 2011-06-03 2012-12-06 Apple Inc. Organizing task items that represent tasks to perform
US9431006B2 (en) 2009-07-02 2016-08-30 Apple Inc. Methods and apparatuses for automatic speech recognition
US9811602B2 (en) * 2009-12-30 2017-11-07 International Business Machines Corporation Method and apparatus for defining screen reader functions within online electronic documents
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US9594845B2 (en) * 2010-09-24 2017-03-14 International Business Machines Corporation Automating web tasks based on web browsing histories and user actions
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9280610B2 (en) 2012-05-14 2016-03-08 Apple Inc. Crowd sourcing information to fulfill user requests
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
US9781262B2 (en) 2012-08-02 2017-10-03 Nuance Communications, Inc. Methods and apparatus for voice-enabling a web application
US10157612B2 (en) * 2012-08-02 2018-12-18 Nuance Communications, Inc. Methods and apparatus for voice-enabling a web application
US9292252B2 (en) 2012-08-02 2016-03-22 Nuance Communications, Inc. Methods and apparatus for voiced-enabling a web application
US9292253B2 (en) 2012-08-02 2016-03-22 Nuance Communications, Inc. Methods and apparatus for voiced-enabling a web application
US9400633B2 (en) 2012-08-02 2016-07-26 Nuance Communications, Inc. Methods and apparatus for voiced-enabling a web application
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
KR20230137475A (en) 2013-02-07 2023-10-04 애플 인크. Voice trigger for a digital assistant
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
WO2014197334A2 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
EP3937002A1 (en) 2013-06-09 2022-01-12 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
US9600227B2 (en) 2013-11-21 2017-03-21 Google Technology Holdings LLC System and method for speech-based navigation and interaction with a device's visible screen elements using a corresponding view hierarchy
US10296160B2 (en) 2013-12-06 2019-05-21 Apple Inc. Method for extracting salient dialog usage from live data
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
EP3480811A1 (en) 2014-05-30 2019-05-08 Apple Inc. Multi-command single utterance input method
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10291776B2 (en) * 2015-01-06 2019-05-14 Cyara Solutions Pty Ltd Interactive voice response system crawler
US11489962B2 (en) 2015-01-06 2022-11-01 Cyara Solutions Pty Ltd System and methods for automated customer response system mapping and duplication
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
WO2017108139A1 (en) * 2015-12-23 2017-06-29 Intel Corporation Changing information output modalities
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179309B1 (en) 2016-06-09 2018-04-23 Apple Inc Intelligent automated assistant in a home environment
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DK179049B1 (en) 2016-06-11 2017-09-18 Apple Inc Data driven natural language event detection and classification
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
US10474753B2 (en) 2016-09-07 2019-11-12 Apple Inc. Language identification using recurrent neural networks
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US11281993B2 (en) 2016-12-05 2022-03-22 Apple Inc. Model and ensemble compression for metric learning
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
US10417266B2 (en) 2017-05-09 2019-09-17 Apple Inc. Context-aware ranking of intelligent response suggestions
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. User interface for correcting recognition errors
US10395654B2 (en) 2017-05-11 2019-08-27 Apple Inc. Text normalization based on a data-driven learning network
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
US11301477B2 (en) 2017-05-12 2022-04-12 Apple Inc. Feedback analysis of a digital assistant
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. Low-latency intelligent automated assistant
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
US10403278B2 (en) 2017-05-16 2019-09-03 Apple Inc. Methods and systems for phonetic matching in digital assistant services
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. Far-field extension for digital assistant services
US20180336275A1 (en) 2017-05-16 2018-11-22 Apple Inc. Intelligent automated assistant for media exploration
US10311144B2 (en) 2017-05-16 2019-06-04 Apple Inc. Emoji word sense disambiguation
US10657328B2 (en) 2017-06-02 2020-05-19 Apple Inc. Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling
US11062497B2 (en) * 2017-07-17 2021-07-13 At&T Intellectual Property I, L.P. Structuralized creation and transmission of personalized audiovisual data
US10445429B2 (en) 2017-09-21 2019-10-15 Apple Inc. Natural language understanding using vocabularies with compressed serialized tries
US10755051B2 (en) 2017-09-29 2020-08-25 Apple Inc. Rule-based natural language processing
US10636424B2 (en) 2017-11-30 2020-04-28 Apple Inc. Multi-turn canned dialog
US10733982B2 (en) 2018-01-08 2020-08-04 Apple Inc. Multi-directional dialog
US10733375B2 (en) 2018-01-31 2020-08-04 Apple Inc. Knowledge-based framework for improving natural language understanding
US10789959B2 (en) 2018-03-02 2020-09-29 Apple Inc. Training speaker recognition models for digital assistants
US10592604B2 (en) 2018-03-12 2020-03-17 Apple Inc. Inverse text normalization for automatic speech recognition
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
US10909331B2 (en) 2018-03-30 2021-02-02 Apple Inc. Implicit identification of translation payload with neural machine translation
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
US10984780B2 (en) 2018-05-21 2021-04-20 Apple Inc. Global semantic word embeddings using bi-directional recurrent neural networks
US11386266B2 (en) 2018-06-01 2022-07-12 Apple Inc. Text correction
DK179822B1 (en) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. Virtual assistant operation in multi-device environments
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
US11076039B2 (en) 2018-06-03 2021-07-27 Apple Inc. Accelerated task performance
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11217251B2 (en) 2019-05-06 2022-01-04 Apple Inc. Spoken notifications
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. User activity shortcut suggestions
DK201970510A1 (en) 2019-05-31 2021-02-11 Apple Inc Voice identification in digital assistant systems
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
US10996827B2 (en) 2019-07-10 2021-05-04 Bank Of America Corporation System for rendering applications based on real time accessibility assessment
WO2021056255A1 (en) 2019-09-25 2021-04-01 Apple Inc. Text detection using global geometry estimators
US11134149B1 (en) * 2020-06-15 2021-09-28 Verizon Patent And Licensing Inc. Systems and methods for providing multi-modal interaction via user equipment

Family Cites Families (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5748186A (en) * 1995-10-02 1998-05-05 Digital Equipment Corporation Multimodal information presentation system
US6057856A (en) * 1996-09-30 2000-05-02 Sony Corporation 3D virtual reality multi-user interaction with superimposed positional information display for each user
US6088032A (en) * 1996-10-04 2000-07-11 Xerox Corporation Computer controlled display system for displaying a three-dimensional document workspace having a means for prefetching linked documents
US5983200A (en) * 1996-10-09 1999-11-09 Slotznick; Benjamin Intelligent agent for executing delegated tasks
US6282511B1 (en) * 1996-12-04 2001-08-28 At&T Voiced interface with hyperlinked information
US6018710A (en) * 1996-12-13 2000-01-25 Siemens Corporate Research, Inc. Web-based interactive radio environment: WIRE
US6006241A (en) * 1997-03-14 1999-12-21 Microsoft Corporation Production of a video stream with synchronized annotations over a computer network
US6449653B2 (en) * 1997-03-25 2002-09-10 Microsoft Corporation Interleaved multiple multimedia stream for synchronized transmission over a computer network
US6125376A (en) * 1997-04-10 2000-09-26 At&T Corp Method and apparatus for voice interaction over a network using parameterized interaction definitions
JPH1125119A (en) * 1997-06-30 1999-01-29 Canon Inc Hypertext reference system
US6223292B1 (en) * 1997-07-15 2001-04-24 Microsoft Corporation Authorization systems, methods, and computer program products
US20020002458A1 (en) * 1997-10-22 2002-01-03 David E. Owen System and method for representing complex information auditorially
US6145003A (en) * 1997-12-17 2000-11-07 Microsoft Corporation Method of web crawling utilizing address mapping
US6654931B1 (en) * 1998-01-27 2003-11-25 At&T Corp. Systems and methods for playing, browsing and interacting with MPEG-4 coded audio-visual objects
SG92628A1 (en) * 1999-02-13 2002-11-19 Newstakes Inc A method and apparatus for converting video to multiple mark-up-language presentations
US6606611B1 (en) * 1999-02-27 2003-08-12 Emdadur Khan System and method for audio-only internet browsing using a standard telephone
US7188353B1 (en) * 1999-04-06 2007-03-06 Sharp Laboratories Of America, Inc. System for presenting synchronized HTML documents in digital television receivers
US6636831B1 (en) * 1999-04-09 2003-10-21 Inroad, Inc. System and process for voice-controlled information retrieval
US6922733B1 (en) * 1999-06-30 2005-07-26 International Business Machines Corporation Method for coordinating visual and speech web browsers
US8448059B1 (en) * 1999-09-03 2013-05-21 Cisco Technology, Inc. Apparatus and method for providing browser audio control for voice enabled web applications
US6668273B1 (en) * 1999-11-18 2003-12-23 Raindance Communications, Inc. System and method for application viewing through collaborative web browsing session
US20070005428A1 (en) * 1999-12-08 2007-01-04 Jacobs Paul E Method for distributing advertisements to client devices having e-mail software installed including multiple advertisement operational modes
US6349132B1 (en) * 1999-12-16 2002-02-19 Talk2 Technology, Inc. Voice interface for electronic documents
US7116765B2 (en) * 1999-12-16 2006-10-03 Intellisync Corporation Mapping an internet document to be accessed over a telephone system
AU2758201A (en) * 2000-01-04 2001-07-16 Ma'at System and method for anonymous observation and use of premium content
US6496802B1 (en) * 2000-01-07 2002-12-17 Mp3.Com, Inc. System and method for providing access to electronic works
US6721781B1 (en) * 2000-01-25 2004-04-13 International Business Machines Corporation Method of providing an alternative audio format of a web page in response to a request for audible presentation of the same
US6732142B1 (en) * 2000-01-25 2004-05-04 International Business Machines Corporation Method and apparatus for audible presentation of web page content
US6711714B1 (en) * 2000-02-02 2004-03-23 Siemens Corporate Research, Inc. Linearization of framesets for audibly rendering frames
US20010034746A1 (en) * 2000-02-26 2001-10-25 Alex Tsakiris Methods and systems for creating user-defined personal web cards
US6697564B1 (en) * 2000-03-03 2004-02-24 Siemens Corporate Research, Inc. Method and system for video browsing and editing by employing audio
US7210093B1 (en) * 2000-03-09 2007-04-24 International Business Machines Corporation Method, system, and program for displaying pages downloaded from over a network in an application window
US20050021862A1 (en) * 2000-03-31 2005-01-27 Dickens Coal Llc Automatic selection of content-delivery provider using link mapping database
US7093129B1 (en) * 2000-06-19 2006-08-15 International Business Machines Corporation Secured encrypted communications in a voice browser
US7519902B1 (en) * 2000-06-30 2009-04-14 International Business Machines Corporation System and method for enhanced browser-based web crawling
JP3703080B2 (en) * 2000-07-27 2005-10-05 インターナショナル・ビジネス・マシーンズ・コーポレーション Method, system and medium for simplifying web content
US6745163B1 (en) * 2000-09-27 2004-06-01 International Business Machines Corporation Method and system for synchronizing audio and visual presentation in a multi-modal content renderer
US6934756B2 (en) * 2000-11-01 2005-08-23 International Business Machines Corporation Conversational networking via transport, coding and control conversational protocols
US7266767B2 (en) * 2000-11-27 2007-09-04 Parker Philip M Method and apparatus for automated authoring and marketing
GB0029025D0 (en) * 2000-11-29 2001-01-10 Hewlett Packard Co Enhancement of communication capabilities
US7062437B2 (en) * 2001-02-13 2006-06-13 International Business Machines Corporation Audio renderings for expressing non-audio nuances
US20030164848A1 (en) * 2001-03-01 2003-09-04 International Business Machines Corporation Method and apparatus for summarizing content of a document for a visually impaired user
US20020178182A1 (en) * 2001-05-04 2002-11-28 Kuansan Wang Markup language extensions for web enabled recognition
US6983307B2 (en) * 2001-07-11 2006-01-03 Kirusa, Inc. Synchronization among plural browsers
US7032169B2 (en) * 2002-05-22 2006-04-18 International Business Machines Corporation Method and system for distributed coordination of multiple modalities of computer-user interaction
US20050164153A1 (en) * 2004-01-28 2005-07-28 Beatty Alejandra L. Method and apparatus for providing presentation options during an on-line educational exam based upon a user's profile

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7356472B2 (en) 2003-12-11 2008-04-08 International Business Machines Corporation Enabling speech within a multimodal program using markup
US7599839B2 (en) 2003-12-11 2009-10-06 Nuance Communications, Inc. Enabling speech within a multimodal program using markup
US8027840B2 (en) 2003-12-11 2011-09-27 Nuance Communications, Inc. Enabling speech within a multimodal program using markup
US8359203B2 (en) 2003-12-11 2013-01-22 Nuance Communications, Inc. Enabling speech within a multimodal program using markup
US9065911B2 (en) 2007-09-28 2015-06-23 Nuance Communications, Inc. System, method and architecture for control and multi-modal synchronization of speech browsers

Also Published As

Publication number Publication date
DE10125406A1 (en) 2002-01-17
US20140089783A1 (en) 2014-03-27
US7657828B2 (en) 2010-02-02
TWI233045B (en) 2005-05-21
US7080315B1 (en) 2006-07-18
US20100293446A1 (en) 2010-11-18
CA2346606C (en) 2012-07-10
US8555151B2 (en) 2013-10-08
US20060206591A1 (en) 2006-09-14

Similar Documents

Publication Publication Date Title
CA2346606A1 (en) Method and apparatus for coupling a visual browser to a voice browser
US9703883B2 (en) Social bookmarking of resources exposed in web pages
US8055999B2 (en) Method and apparatus for repurposing formatted content
US6081815A (en) Method for processing a hyperlink formatted message to make it compatible with an alphanumeric messaging device
US7739589B2 (en) Extending markup of a browser using a plug-in framework
US7620908B2 (en) Managing a user interface
US20080288854A1 (en) Deriving Menu-Based Voice Markup from Visual Markup
US8571606B2 (en) System and method for providing multi-modal bookmarks
CA2436940A1 (en) A method and system for voice activating web pages
US6344851B1 (en) Method and system for website overview
US20020003547A1 (en) System and method for transcoding information for an audio or limited display user interface
JP2008171449A (en) Engine to present user interface based on logical structure such as one for customer relationship management system through web site
EP1344368A2 (en) Translation of digital contents based on receiving device capabilities
RU2009133463A (en) CONTENT DISTRIBUTION CONTROL DEVICE, DATA TRANSMISSION TERMINAL, CONTENT DISTRIBUTION PROGRAM AND SYSTEM
US20030131008A1 (en) System and method for transaction recording and playback
CN109446460B (en) HTML5 online video redirection system and implementation method thereof
CN110175026A (en) A kind of webpage visualizing editing method, device and storage medium
US8838673B2 (en) Method and apparatus to generate audio versions of web pages
US20030121002A1 (en) Method and system for exchanging information through speech via a packet-oriented network
WO2001035235A1 (en) System and method for accessing web content using limited display devices
JP2014531639A (en) Method and apparatus for extending page tag and computer storage medium
CN103186370B (en) Web page presentation method and system
CN116185407A (en) Data processing method and device, electronic equipment and computer storage medium
CN114115880A (en) Method for sucking top of page table operation column under screens of different sizes
MXPA00003342A (en) Method for processing hyperlink information

Legal Events

Date Code Title Description
EEER Examination request
MKLA Lapsed

Effective date: 20200831