WO2008079505B1 - Method and apparatus for hybrid audio-visual communication - Google Patents
Method and apparatus for hybrid audio-visual communicationInfo
- Publication number
- WO2008079505B1 WO2008079505B1 PCT/US2007/082598 US2007082598W WO2008079505B1 WO 2008079505 B1 WO2008079505 B1 WO 2008079505B1 US 2007082598 W US2007082598 W US 2007082598W WO 2008079505 B1 WO2008079505 B1 WO 2008079505B1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- stream
- video stream
- avatar control
- media content
- accordance
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities
- H04M3/567—Multimedia conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/22—Arrangements for supervision, monitoring or testing
- H04M3/2227—Quality of service monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/57—Arrangements for indicating or recording the number of the calling subscriber at the called subscriber's set
- H04M1/575—Means for retrieving and displaying personal data about calling party
- H04M1/576—Means for retrieving and displaying personal data about calling party associated with a pictorial or graphical representation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72427—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality for supporting games or graphical animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities
- H04M3/563—User guidance or feature selection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/16—Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
- H04W28/18—Negotiating wireless communication parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
- H04W84/04—Large scale networks; Deep hierarchical networks
- H04W84/042—Public Land Mobile systems, e.g. cellular systems
Abstract
A method and apparatus for providing communication between a sending terminal and one or more receiving terminals in a communication network. The media content of a signal transmitted by the sending terminal is detected and one or more of a voice stream, an avatar control parameter stream and a video stream are generated from the media content. At least one of the voice stream, the avatar control parameter stream and the video stream are selected as an output to be transmitted to the receiving terminal. The network server may be operable to generate synthetic video from the voice input, a natural video input and/or incoming avatar control parameters. Figure 7 is a flow chart of a method for providing hybrid audio visual communication consistent with some embodiments of the invention.
Claims
1. A method for providing communication between a sending terminal and at least one receiving terminal in a communication network, the method comprising: detecting media content of a signal transmitted by the sending terminal; generating, from the media content, a voice stream, an avatar control parameter stream and a video stream; selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream; and transmitting the selected output to the at least one receiving terminal.
2. A method in accordance with claim 1, wherein the media content comprises a voice stream and wherein generating an avatar control parameter stream from the media content comprises detecting features in the voice stream that correspond to visemes and generating avatar control parameters representative of the visemes.
3. A method in accordance with claim 2, wherein generating a video stream from the media content comprises: rendering images using the avatar control parameters; and encoding the rendered images as the video stream.
4. A method in accordance with claim 1, wherein the media content comprises a video stream and wherein generating an avatar control parameter stream from the media content comprises: detecting facial expressions in video images contained in the video stream; and encoding the facial expressions as avatar control parameters.
5. A method in accordance with claim I5 wherein the media content comprises a video stream and wherein generating an avatar control parameter stream from the media content comprises: detecting gestures in video images of the video stream; and encoding the gestures as avatar control parameters.
6. A method in accordance with claim 1 wherein the media content comprises a natural video stream, the method further comprising detecting facial expressions in video images of the natural video stream; encoding the facial expressions as avatar control parameters; rendering images using the avatar control parameters; encoding the rendered images as a synthetic video stream; and selecting, as output, at least one of the voice stream, the avatar control parameter stream, the natural video stream, and the synthetic video stream.
7. A method in accordance with claim 1 wherein the media content comprises a natural video stream, the method further comprising detecting gestures in video images of the natural video stream; encoding the gestures as avatar control parameters; rendering images using the avatar control parameters; encoding the rendered images as a synthetic video stream; and selecting, as output, at least one of the voice stream, the avatar control parameter stream, the natural video stream, and the synthetic video stream.
8. A method in accordance with claim 1, wherein the media content comprises an avatar parameter stream, and wherein generating a video stream from the media content comprises: rendering images using the avatar control parameter stream; and encoding the rendered images as a synthetic video stream.
9. A method in accordance with claim 1 , wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon a preference of the user of the sending terminal.
10. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon a preference of a user of the at least one receiving terminal.
11. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon capabilities of the at least one receiving terminal.
12. A method in accordance with claim 1, wherein the capabilities of the at least one receiving terminal are determined by a data exchange between the at least one receiving terminal and a network server performing the method.
13. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon a load status of a network server performing the method.
14. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon the available capacity of a communication channel between the at least one receiving terminal and a network server performing the method.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/614,560 | 2006-12-21 | ||
US11/614,560 US20080151786A1 (en) | 2006-12-21 | 2006-12-21 | Method and apparatus for hybrid audio-visual communication |
Publications (3)
Publication Number | Publication Date |
---|---|
WO2008079505A2 WO2008079505A2 (en) | 2008-07-03 |
WO2008079505A3 WO2008079505A3 (en) | 2008-10-09 |
WO2008079505B1 true WO2008079505B1 (en) | 2008-12-04 |
Family
ID=39542639
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2007/082598 WO2008079505A2 (en) | 2006-12-21 | 2007-10-26 | Method and apparatus for hybrid audio-visual communication |
Country Status (2)
Country | Link |
---|---|
US (1) | US20080151786A1 (en) |
WO (1) | WO2008079505A2 (en) |
Families Citing this family (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080256452A1 (en) * | 2007-04-14 | 2008-10-16 | Philipp Christian Berndt | Control of an object in a virtual representation by an audio-only device |
US8180029B2 (en) | 2007-06-28 | 2012-05-15 | Voxer Ip Llc | Telecommunication and multimedia management method and apparatus |
US11095583B2 (en) | 2007-06-28 | 2021-08-17 | Voxer Ip Llc | Real-time messaging method and apparatus |
US8346206B1 (en) * | 2007-07-23 | 2013-01-01 | At&T Mobility Ii Llc | Customizable media feedback software package and methods of generating and installing the package |
US8063905B2 (en) * | 2007-10-11 | 2011-11-22 | International Business Machines Corporation | Animating speech of an avatar representing a participant in a mobile communication |
KR101597286B1 (en) * | 2009-05-07 | 2016-02-25 | 삼성전자주식회사 | Apparatus for generating avatar image message and method thereof |
US8878773B1 (en) | 2010-05-24 | 2014-11-04 | Amazon Technologies, Inc. | Determining relative motion as input |
JP6392497B2 (en) * | 2012-05-22 | 2018-09-19 | コモンウェルス サイエンティフィック アンド インダストリアル リサーチ オーガニゼーション | System and method for generating video |
US8970656B2 (en) * | 2012-12-20 | 2015-03-03 | Verizon Patent And Licensing Inc. | Static and dynamic video calling avatars |
GB2509323B (en) * | 2012-12-28 | 2015-01-07 | Glide Talk Ltd | Reduced latency server-mediated audio-video communication |
US20140258419A1 (en) * | 2013-03-05 | 2014-09-11 | Motorola Mobility Llc | Sharing content across modalities |
US9094576B1 (en) * | 2013-03-12 | 2015-07-28 | Amazon Technologies, Inc. | Rendered audiovisual communication |
KR102169523B1 (en) * | 2013-05-31 | 2020-10-23 | 삼성전자 주식회사 | Display apparatus and control method thereof |
GB201315142D0 (en) * | 2013-08-23 | 2013-10-09 | Ucl Business Plc | Audio-Visual Dialogue System and Method |
US9152377B2 (en) * | 2013-08-29 | 2015-10-06 | Thomson Licensing | Dynamic event sounds |
US9307191B2 (en) | 2013-11-19 | 2016-04-05 | Microsoft Technology Licensing, Llc | Video transmission |
KR20150068609A (en) * | 2013-12-12 | 2015-06-22 | 삼성전자주식회사 | Method and apparatus for displaying image information |
US9614969B2 (en) * | 2014-05-27 | 2017-04-04 | Microsoft Technology Licensing, Llc | In-call translation |
JP7173249B2 (en) * | 2017-05-09 | 2022-11-16 | ソニーグループ株式会社 | CLIENT DEVICE, DISPLAY SYSTEM, CLIENT DEVICE PROCESSING METHOD AND PROGRAM |
JP6946724B2 (en) | 2017-05-09 | 2021-10-06 | ソニーグループ株式会社 | Client device, client device processing method, server and server processing method |
US10924710B1 (en) * | 2020-03-24 | 2021-02-16 | Htc Corporation | Method for managing avatars in virtual meeting, head-mounted display, and non-transitory computer readable storage medium |
US11218666B1 (en) * | 2020-12-11 | 2022-01-04 | Amazon Technologies, Inc. | Enhanced audio and video capture and presentation |
US11429835B1 (en) * | 2021-02-12 | 2022-08-30 | Microsoft Technology Licensing, Llc | Holodouble: systems and methods for low-bandwidth and high quality remote visual communication |
GB2606131A (en) * | 2021-03-12 | 2022-11-02 | Palringo Ltd | Communication platform |
US20230199147A1 (en) * | 2021-12-21 | 2023-06-22 | Snap Inc. | Avatar call platform |
US11831696B2 (en) | 2022-02-02 | 2023-11-28 | Microsoft Technology Licensing, Llc | Optimizing richness in a remote meeting |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6483513B1 (en) * | 1998-03-27 | 2002-11-19 | At&T Corp. | Method for defining MPEP 4 animation parameters for an animation definition interface |
US6307576B1 (en) * | 1997-10-02 | 2001-10-23 | Maury Rosenfeld | Method for automatically animating lip synchronization and facial expression of animated characters |
US6272231B1 (en) * | 1998-11-06 | 2001-08-07 | Eyematic Interfaces, Inc. | Wavelet-based facial motion capture for avatar animation |
US6081278A (en) * | 1998-06-11 | 2000-06-27 | Chen; Shenchang Eric | Animation object having multiple resolution format |
US7039676B1 (en) * | 2000-10-31 | 2006-05-02 | International Business Machines Corporation | Using video image analysis to automatically transmit gestures over a network in a chat or instant messaging session |
AUPR212600A0 (en) * | 2000-12-18 | 2001-01-25 | Canon Kabushiki Kaisha | Efficient video coding |
JP3385320B2 (en) * | 2001-03-06 | 2003-03-10 | シャープ株式会社 | Animation playback terminal, animation playback method, and program therefor |
US7663628B2 (en) * | 2002-01-22 | 2010-02-16 | Gizmoz Israel 2002 Ltd. | Apparatus and method for efficient animation of believable speaking 3D characters in real time |
US6873854B2 (en) * | 2002-02-14 | 2005-03-29 | Qualcomm Inc. | Method and an apparatus for adding a new member to an active group call in a group communication network |
US7640293B2 (en) * | 2002-07-17 | 2009-12-29 | Research In Motion Limited | Method, system and apparatus for messaging between wireless mobile terminals and networked computers |
US7130282B2 (en) * | 2002-09-20 | 2006-10-31 | Qualcomm Inc | Communication device for providing multimedia in a group communication network |
US8411594B2 (en) * | 2002-09-20 | 2013-04-02 | Qualcomm Incorporated | Communication manager for providing multimedia in a group communication network |
US6925438B2 (en) * | 2002-10-08 | 2005-08-02 | Motorola, Inc. | Method and apparatus for providing an animated display with translated speech |
KR100932483B1 (en) * | 2002-11-20 | 2009-12-17 | 엘지전자 주식회사 | Mobile communication terminal and avatar remote control method using the same |
US7283489B2 (en) * | 2003-03-31 | 2007-10-16 | Lucent Technologies Inc. | Multimedia half-duplex sessions with individual floor controls |
US20050030905A1 (en) * | 2003-08-07 | 2005-02-10 | Chih-Wei Luo | Wireless communication device with status display |
US20050041625A1 (en) * | 2003-08-22 | 2005-02-24 | Brewer Beth Ann | Method and apparatus for providing media communication setup strategy in a communication network |
US7308649B2 (en) * | 2003-09-30 | 2007-12-11 | International Business Machines Corporation | Providing scalable, alternative component-level views |
JP2005117141A (en) * | 2003-10-03 | 2005-04-28 | Nec Corp | Apparatus, system and method of half-duplex communication |
-
2006
- 2006-12-21 US US11/614,560 patent/US20080151786A1/en not_active Abandoned
-
2007
- 2007-10-26 WO PCT/US2007/082598 patent/WO2008079505A2/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
WO2008079505A3 (en) | 2008-10-09 |
WO2008079505A2 (en) | 2008-07-03 |
US20080151786A1 (en) | 2008-06-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2008079505B1 (en) | Method and apparatus for hybrid audio-visual communication | |
US7508413B2 (en) | Video conference data transmission device and data transmission method adapted for small display of mobile terminals | |
CN102981613B (en) | terminal and terminal control method | |
CN101594528A (en) | Information processing system, messaging device, information processing method and program | |
TW201021576A (en) | System and method for dynamic video encoding in multimedia streaming | |
CN101123730A (en) | Apparatus and method for transmitting moving picture stream using bluetooth | |
JP2008067203A (en) | Device, method and program for synthesizing video image | |
CN103109528A (en) | System and method for the control and management of multipoint conferences | |
US20070147367A1 (en) | VoIP communication remote control system and remote controller thereof | |
WO2017050067A1 (en) | Video communication method, apparatus, and system | |
US11914922B2 (en) | Audio mixing for teleconferencing | |
CN102025972A (en) | Mute indication method and device applied for video conference | |
CN102984496A (en) | Processing method, device and system of video and audio information in video conference | |
CN105247875A (en) | Distribution control system and distribution system | |
WO2019165960A1 (en) | Media data real time transmission control method, system and storage medium | |
US6928087B2 (en) | Method and apparatus for automatic cross-media selection and scaling | |
CN105392032A (en) | Method and apparatus for controlling multimedia playing | |
JP2009065696A (en) | Device, method and program for synthesizing video image | |
JP2006033743A5 (en) | ||
CN110730362A (en) | Low-flow video communication transmission system and method | |
US20070195962A1 (en) | Apparatus and method for outputting audio data using wireless terminal | |
WO2015117383A1 (en) | Method for call, terminal and computer storage medium | |
CN114257771A (en) | Video playback method and device for multi-channel audio and video, storage medium and electronic equipment | |
CN104378651A (en) | Dynamic encoding device and method based on bandwidth detection | |
EP3860151A1 (en) | Audio / video capturing using audio from remote device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07854435 Country of ref document: EP Kind code of ref document: A2 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 07854435 Country of ref document: EP Kind code of ref document: A2 |