WO2007133129A1

WO2007133129A1 - Method and arrangement relating to audio/video conferencing

Info

Publication number: WO2007133129A1
Application number: PCT/SE2006/000567
Authority: WO
Inventors: Johan Eckerstein; David Pagrotsky; Emmanuel Frecon
Original assignee: Myjoice Ab
Priority date: 2006-05-15
Filing date: 2006-05-15
Publication date: 2007-11-22
Also published as: EP2025160A1

Abstract

The present invention relates to an audio/video-call system for allowing communication between a first user and a second user, the communication includes data stream comprising control signals, and audio and video data, the system comprising: a first and second communication systems separated geographically, wherein the first user control both of the communication systems, and wherein each of the communication systems include a video camera, a microphone, an network connection for connecting the communication system to the network. The network connection is used for sending a data stream initiating a call from the first communication system to the second communication system, and for transmitting the data stream between the first and the second communication systems once the call is initiated.

Description

P17353PC.A01

METHOD AND ARRANGEMENT RELATING TO AUDIO/VIDEO CONFERENCING

TECHNICAL FIELD The present invention relates to an audio/video-call system for allowing communication between a first user and a second user, the communication including data stream comprising control signals, and audio and video data.

BACKGROUND OF THE INVENTION The number of Internet users increases rapidly. More and more households, homes, institutes etc. get connected using cheaper and faster broad band. Thus, the PC with Internet access is rapidly becoming a key consumer communication device. Broad band connection allows among others Internet communications and electronic conferencing, and offering both voice and video connectivity is of course an attractive improvement.

The steadily rise of population of house bound persons, i.e. elderly, disabled etc. implies for more caretaking measurements. However, the increased number of peoples in need of care has usually disadvantages, e.g. due to lack of resources, lack of communication with relatives etc .

US 2005/0240086 relates to an intelligent wearable monitoring system includes a wireless personal area network for extended monitoring of a patient's motor functions. The wireless personal area network includes an intelligent accelerometer unit, a personal server and a remote access unit. The intelligent accelerometer unit measures acceleration data of the patient, in real-time. The personal server processes the acceleration data, applying linear and non-linear analysis, such as fractal analysis, to generate motor function information from the acceleration data. Motor function information is transmitted to a remote access unit for statistical analysis and formatting into visual representations. A data management unit receives the formatted motor function information and displays the information, for example, for viewing by the patient's physician. This invention does not allow interaction with another user. US publication No. 2001054071 , relates to a conference system for electronic care giving. A one-on-one audio/video-call system allows communication between an elderly or disabled user and caregiver using audio, video and/or data. The system includes two communication systems-one for the user and one for the caregiver. Each communication system includes a console coupled to a television. A remote control issues commands to the console to initiate or accept calls to the other communication system. The phone line is used to carry the audio portion of the call while the network is used to carry any video and/or data portion of the call. In some embodiments, a passkey allows the caregiver's communication system to control the user's communication system without any interaction by the user. In other embodiments, more than one video camera is used for a communication system. A microphone can be coupled to the remote control to collect the audio channel for the call.

In US2004010464 a special purpose VideoPhone terminal combines a desk, telephone and display in a non-threatening manner. The desk may accommodate users may be confined to a wheelchair or need extra leg room. The telephone is, or either is disguised as a less-menacing-looking telephone. The VideoPhone terminal provides a local participant with a near real-time motion image of the remote participant on what appears to be a television screen but is actually a touch screen.

SUMMARY OF THE INVENTION

The present invention aims to provide a simple, but yet efficient arrangement and method which provides easy communication between two users, using audio and video conferencing ability. The arrangement of the invention also allows for efficient monitoring of an object (person) through the communication channel allowing one way control of the channel.

These and other advantageous are obtained through an audio/video-call system for allowing communication between a first user and a second user, the communication includes data stream comprising control signals, and audio and video data, the system comprising: a first and second communication systems separated geographically, wherein the first user control both of the communication systems. Each of the communication systems includes a video camera, a microphone, and a network connection for connecting the communication system to the network. The network connection is used for sending a data stream initiating a call from the first communication system to the second communication system, and for transmitting the data stream between the first and the second communication systems once the call is initiated. The system may further comprise a television socket for coupling the first communication system to a television (or monitor), wherein the video data and available messages is displayed on the television.

Most preferably, the application area of the system is when the first user is a caregiver and the second user is an elderly or disabled individual.

The system may be controlled by a remote control and thus comprises a remote control device comprising an alarm switch wirelessly coupled to the second communication system, for the second user to indicate distress and for the second communication system to respond by contacting help.

The base of the system is a control unit at said second communication system and comprises a microprocessor, input/output interfaces, memory, transceivers, antenna input, network connection, TV output and audio/video input. Preferably, the unit further comprises peripheral controller output/inputs. The control unit connects to regular TV set and takes over both the TV, set top boxes and other remote controls.

The system further comprises a remote control being a programmable remote control having ability to display on the remote control pictogrammes or icons.

The system further comprises a remote controller intended to be used as one or several of a remote control, communication and alarming. According to one embodiment the remote controller is formed as arm watch for easy wear of it. The remote controller comprises one or several of a processing unit, a transmitter circuit and a tilt and/or crash sensor, a loudspeaker, microphone and or earphone contacts for communication with the control unit. The remote controller comprises sensors for health monitoring: one or several of temperature, acceleration, and/or motion. The remote controller may also comprise voice recognition application, allowing making calls to known contacts through voice control.

The invention further relates to a control unit for connection to a communication network, a monitor and audio/video receiver. The unit further comprises: a processing unit for receiving data from a first user and initiating a communication channel to a second user, a processing unit for handling control of the initiated communication by said first user, receiver for receiving television broad cast and outputting to said monitor, means for interrupting said received television broad cast and outputting said initiated call in audio and video, and means for receiving control signals from said second user. The control unit may further comprise means for recording an interrupted program upon call initiation and outputting said recorded program after said call termination. The control unit may further comprise means for receiving text messages for output on said monitor. In one embodiment, the control unit further comprises means for receiving control signals from a remote controller in radiofrequency. The control unit may comprise means for detecting a remote control in vicinity and for initiating a call. Preferably, the control unit further comprises means for receiving updating and/or reconfiguration functionality through said network. Most preferably, the receiving updating and/or reconfiguration functionality is initiated by said user.

The invention also relates to a mobile remote control device for communication with a control unit as mentioned earlier and comprises a switch, a processing unit, a transmitter circuit and a sensor. Preferably, the switch is an alarm initiating switch, the sensor is a tilt and/or accelerometer sensor, and the radio transmitter is Bluetooth.

The invention also relates to a programmable remote control for communication with a aforementioned control unit comprising a control arrangement provided as icons. For easier handling and better visualisation, the programmable remote control comprises a touch-screen for said icons.

The invention also relates to a method for an audio/video-call allowing a first user to communicate with a second user, wherein the communication includes a data stream. The method comprises: providing a communication system controlled by the first user which includes a video camera, a microphone, a network socket for connecting the communication system to the network, sending a call request as a data stream over the network from the communication system to a geographically separated second communication system; converting the image of the user to a video signal with the video camera; sending the video signal as data flow over the network from the communication system to the second communication system once the call request is accepted by the second communication system and controlling the second system. Moreover, the invention relates to a computer program in a device for connection to a communication network, a monitor and audio/video receiver. The computer program further comprises: an instruction set for processing received data from an input connectable to a first user and initiating a communication channel to a second user, an instruction set for receiving control data by said first user and controlling said device, an instruction set for receiving a television broadcast and outputting said broadcast to said monitor, an instruction set for interrupting said received television broad cast and outputting said initiated call in audio and video, and an instruction set for receiving control signals from said second user. According to preferred embodiment, the computer program may further comprise an instruction set for receiving and handling configuration data for said device from said fist user.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following the invention will be described with reference to a number of non-limiting embodiments illustrated in the attached drawings, in which: Fig. 1 illustrates a block diagram of the invention concept,

Fig. 2 is a block diagram of a central controlling arrangement according to the present invention,

Fig. 3 is a block diagram of a device according to the present invention, and Fig. 4 is a flow diagram over functions of the system according to the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In the following, the invention will be described in more detail with reference to an example of a home system for purpose of taking care and mainly monitoring an elder or disabled person. However, it is appreciated by a person skilled in the art that the system of the invention may be used for other purposes as well.

Basically the system 100 according to the invention, as illustrated in Fig. 1 , comprises a "home" system 120, a monitoring system 150 and a centralised remote repository 170 for settings and control. In the following home system will refer to a monitored persons system and monitoring system to a person monitoring the home system. The home system 120 itself comprises a number of devices: a set-top box or control unit 121 connected to a TV set or monitor 122, a remote control 123, a video camera 124 with integrated microphone (or separate one) and optionally a wearable (mobile) bracelet 125. Additional peripheral devices 126 such as a video recorder, DVD (RW) player may also 5 be connected to the system.

The monitoring system 150 may comprise a computer unit 151 having audio-video receiving possibility and connection to a network (Internet). The computer may be an ordinary PC running the system specific program for putting calls (IP telephony) to the 10 home system and also remote controlling the home system.

The centralised remote repository 170 comprises a server and server applications allowing the monitoring system to place calls to the home system, uploading applications in the home system and monitoring system.

15

According to one preferred embodiment, the home system has three main functionalities: watching TV programmes as with regular TV sets, communicating with relatives and other people via a video-conference solution and reading multimedia messages sent by relatives (or others such as care providers) and monitoring from relatives and care

20 providers of elderly who are disabled.

The home system requires access to the Internet 160 (or any network having ability for transmitting audio and video data with good quality) for all of its functions, may be apart from watching TV. An extension may include access to a fixed telephone line for 25 emergency communication.

The home system may simplify the installation through the use of wireless communication whenever possible: e.g. between the remote control and the set-top box, between the set- top box and the Internet (and thus relatives and other users of the system), between the 30 set-top box and the wearable bracelet.

The set top box 1210 is illustrated in fig. 2. According to one preferred embodiment, the set top box comprises a microprocessor 1211, input/output interfaces (transmitter/receiver) 1212, memory (-es) 1213, and optional onboard display drivers 35 1214, IR and/or Bluetooth transceivers 1215, antenna input 1216, network connection 1217, TV output 1218, audio/video input 1219 and peripheral controller output/inputs 1220.

The microprocessor 1211 is arranged to process data and execute program instructions stored on a memory unit. The microprocessor controls the inputs and outputs and communication circuits.

Input/output interfaces (transmitter/receiver) 1212 of different types (illustrated as one unit for simplicity reasons), depending on application areas, are used for receiving/transmitting and converting: analogue and/or digital TV signals, network signals,

IR and/or Bluetooth (or any other wireless communication means), Audio/video inputs

One or several memories 1213 are arranged for storing data and program instructions. The memories may be volatile and/or non-volatile, for storing control software, statistics, updates, and so on as understood by the person skilled in the art. The memory units may comprise any suitable type including but not limited to: RAM, SRAM (Static RAM) or DRAM (Dynamic RAM) (e.g. DIMM RAM versions, RDRAM, Rambus), diskette, CD-ROM (Compact Disk-Read Only Memory), DVD (Digital Video Disk), flash or similar memory media (e.g. compactflash, SD secure digital, memorystick, miniSD, MMC multimediacard, smartmedia, transflash, XD), HD-DVD (High Definition DVD), or Bluray DVD, USB (Universal Serial Bus) based memory media, magnetic tape media, optical storage media, magneto-optical media, bubble memory etc.

The set top box may interact (display information) with a user using a display or diodes thus having optional onboard display drivers 1214.

The remote control and/or bracelet may communicate with the set top box using IR and/or Bluetooth, or any other suitable wireless transmission ability and uses for example IR and/or Bluetooth transceivers 1215 for receiving and transmitting data. Thus, the set top box is also provided with IR detector and/or Bluetooth antenna (not shown). Input 1216 is arranged as a TV antenna or dish connector. However, input 1216 may also be a digital box or a decoder input. Preferably, the set top box may contain circuits for decoding digital transmissions.

5 The network connection 1217 may be one or several of Ethernet, PSTN connector, WiFi (card) etc.

The TV output 1218 is used to transmit signals to the TV and display proper information. The output may be an antenna output. 10

The audio/video input 1219 connects the camera and optional microphones to the box.

The box may be arranged to control for example a video recorder, DVD (RW) player or any other peripheral devices through peripheral controller output/inputs 1220. 15

Preferably, the set-top box connects to regular TV set, through offering a broad range of input/output facilities, and takes over both the TV, current set top boxes and the old remote control.

20 The remote control may be an ordinary remote control used for controlling the TV. However, in preferred embodiment a programmable remote control having ability to display pictogrammes or icons, e.g. for different TV channels or functionalities. For example a remote control comprising a programmable touch-screen may be used. This also allows for easier use of the remote control, for example for elderly or partially-sighted

25 individuals. The remote control may comprise a programmable PDA.

An exemplary embodiment of a bracelet 1250 is illustrated in fig. 3. The bracelet is intended to be used as for example a remote control, communication and alarming. According to the illustrated embodiment, the bracelet 1250 is formed as arm watch having

30 two straps 1251 and a carrying structure 1252. On the carrying structure is arranged: switches 1253 for different functions, such as alarm button, remote control for controlling some functions of set-top box, etc., a processing unit 1254, a transmitter circuit 1255 and a tilt and/or crash sensor 1256. In this embodiment it is assumed that the bracelet comprises a radio transmitter, such as Bluetooth, and thus provided with an antenna

35 1257. It is clear that other types of transmitters such as infrared may also be used. Moreover, the bracelet may also be equipped with a loudspeaker, microphone and or earphone contacts for communication between the user and the set-top box.

The bracelet communicates wirelessly to the set-top box and may offer a restricted number of buttons for calling specific remote users. The bracelet relays to the audio-video conferencing system, letting only audio pass through its internal microphone and speaker.

The bracelet can be extended with extra sensors for health monitoring: temperature, acceleration, motion etc. The motion sensor may be used to alarm if the carrying person does not move within a specific time period.

The bracelet may also be equipped with voice recognition application, allowing making calls to known contacts through voice control.

The tilt and/or crash sensor 1256 can be used for generating automated alarm to the set top box (or other equipment) if the carrier of the bracelet, for example falls or collapses.

The system has users as its central concept, as opposed to their phone numbers or other identifiers within the audio-video conference system. Users are represented by lists of connection mechanisms (different phone numbers, video conference identity, etc) and the system implements escalation from one connection mechanism to the other whenever calling. The list will reflect various degrees of availability and quality of communication.

Preferably, but not exclusively, the users are represented by pictogrames (typically, a portrait picture) on various places such as the bracelet, the remote control or the TV screen.

The remote users of the system can communicate to a system either via a home system according to the invention, a regular Internet-based audio-video conferencing client, or a voice call on a cell phone.

It is also possible to use communication through cell phone video call, e.g. using protocols such as H.323. The H.323 standard provides a foundation for audio, video, and data communications across IP-based networks, including the Internet. H.323 is an umbrella recommendation from the International Telecommunications Union (ITU) that sets standards for multimedia communications over Local Area Networks (LANs).

In one preferred embodiment, the presence of the bracelet in the vicinity of the set-top box is detected and reflected as the presence of the identifier of the home user within the audio-video conferencing system. Consequently, remote users can always know if the home user is at home or not, and trigger appropriate response in emergency cases or initiate communication. It may also be used for positioning of the carrier.

In one embodiment, the system may be configured in such way that whenever a known remote user calls the home system, the current TV programme will be interrupted and replaced by the audio-video conferencing solution on the TV screen for the duration of the call.

The set top box may comprise a mass-storage device, such as a hard-disk, or by controlling a peripheral recorder, and record the current TV programme and the program will continue from the point it was interrupted whenever a call has finished.

Issuing a call via the remote control will not interrupt the current TV programme until the call is successfully placed to one of the numbers representing the user being called. From that point, the currently watched programme can automatically be recorded as described above.

It is also possible to use a picture in picture function to show the picture of the caller together with the TV program.

As mentioned earlier, the remote control may use pictogrammes for all of its functions for better understanding and access. Users are represented by portrait pictures, TV and news message channels by recognisable logos. For example, pressing on a news message channel button on the remote control will interrupt TV watching or a pending video conference call and will visualise the latest messages from that channel.

In a preferred embodiment, every home system is associated to a (news) message channel. Relatives (and others who have been granted access) can then send messages to that channel via the centralised repository. Thus, the system allows for pushing messages to the receiver through the TV. Currently, the Internet operators are not willing to let people offer services from their home networks. In order to better comply with this requirements, most of the information is moved outside the homes and as services within the associated company of the present system. Such a move provides other advantages, such as better security of data through regular backups, less dependency on the Internet operators, and possible disturbances in the Internet connections, centralised access and storage to data, etc.

Preferably, the remote control and the home system implement one function at a time only. Switching from one function to the other will automatically interrupt the previous function.

Incoming calls from unknown remote users can either be completely ignored, or be answered via the remote control or the bracelet. Incoming calls from pre-authorized senders may be initiated automatically without pressing a button. This is very useful when monitoring people.

Most preferably, the system settings are hosted within the centralised repository; they are automatically propagated (and copied) to all the devices composing the home system via the network (Internet). This means that authorized people can configure the home system without having to be physically present. This includes, for example who is allowed to automatically initiate a call, setup of escalations sequences etc.

The monitoring user is able to control the wet top box by initiating and reconfiguring it remotely, e.g. entering new authorised callers, blocking callers, adding some functionalities, initiating message reception etc.

The function of the system is exemplified in the following with reference to flow diagram of Fig. 4. The first user, i.e. a relative places a call 40 through his computer, e.g. by using an address book or pre-programmed function having an unique number corresponded to the specific set-top box. The call is routed 41 through the internet to correct addressee. The set top box checks 42 the incoming call and if the caller is in the authorised list, the call is let through otherwise the call is disregarded 43. The set top box checks 44 whether the TV is on or the bracelet is nearby; if so the call is put through. The TV may be turned on automatically. The call is put through 45 otherwise the caller is informed 46 that there is no user in nearby. The TV program is interrupted and the image of the caller is screened 47. Optionally, the set top box starts recording 48 the current TV program. From now on the caller has the total control over the system and can maintain the call and control the set top box. The call is terminated from the caller side and the called person cannot terminate the call. If the caller is not on the list, the receiver must accept the call, e.g. using the bracelet.

The invention is not limited to the described and illustrated embodiments. The invention may be modified and altered in a number of ways without departuring from the scope of the invention according to the attached claims.

Claims

1. An audio/video-call system for allowing communication between a first user and a second user, the communication includes data stream comprising control signals, and audio and video data, the system comprising: a first and second communication systems separated geographically, wherein the first user controls both of the communication systems upon an initiated call, and wherein each of the communication systems include a video camera, a microphone, an network connection for connecting the communication system to the network, wherein the network connection is used for sending a data stream initiating a call from the first communication system to the second communication system, and for transmitting the data stream between the first and the second communication systems once the call is initiated.

2. The system of claim 1 , further comprising a television socket for coupling the first communication system to a television, wherein the video data and available messages is displayed on the television.

3. The system of claim 1 , wherein the first user is a caregiver and the second user is an elderly or disabled individual.

4. The system of claim 1 , further comprising a remote control device comprising an alarm switch wirelessly coupled to the second communication system, for the second user to indicate distress and for the second communication system to respond by providing an alarm.

5. The system of claim 1 , further comprising a control unit (1210) at said second communication system and comprising a microprocessor (1211), input/output interfaces (1212), memory (1213), transceivers (1215), an antenna input (1216), a network connection (1217), a TV output (1218) and an audio/video input (1219).

6. The system of claim 5, said unit further comprising peripheral controller output/inputs (1220).

7. The system of claim 5, wherein the control unit connects to regular TV set and takes over both the TV, receiver set top boxes and remote controls.

8. The system of claim 5, comprising a remote control being a programmable remote control having ability to display pictogrammes or icons.

5 9. The system of claim 1 , further comprising a remote controller (1250) intended to be used as one or several of a remote control, communication and alarming.

10. The system of claim 9, wherein the remote controller (1250) is formed as arm watch.

10 11. The system of claim 9, wherein the remote controller (1250) comprises one or several of a processing unit (1254), a transmitter circuit (1255) and a tilt and/or crash sensor (1256), a loudspeaker, microphone and or earphone contacts for communication with the control unit .

15 12. The system of claim 9, wherein the remote controller (1250) comprises sensors for health monitoring being one or several of temperature, acceleration, motion etc.

13. The system of claim 9, wherein the remote controller (1250) comprises voice recognition application, allowing making calls to known contacts through voice control.

20

14. A control unit (121) for connection to a communication network, a monitor (122) and audio/video receiver (124), the unit further comprising: a processing unit (1211) for receiving data from a first user and initiating a communication channel to a second user, 25 - a processing unit (1211) for handling control of the initiated communication by said first user, receiver for receiving television broad cast and outputting to said monitor, means for interrupting said received television broad cast and outputting said initiated call in audio and video, and 30 - means for receiving control signals from said second user.

15. The control unit according to claim 14, further comprising means for recording an interrupted program upon call initiation and outputting said recorded program after said call termination.

35

16. The control unit according to claim 14, further comprising means for receiving text messages for output on said monitor.

17. The control unit according to claim 14, further comprising means for receiving control 5 signals from a remote controller in radiofrequency.

18. The control unit according to claim 14, further comprising means for detecting a remote control in vicinity.

10 19. The control unit according to claim 14, further comprising means for receiving updating and/or reconfiguration functionality through said network.

20. The control unit according to claim 19, wherein said receiving updating and/or reconfiguration functionality is initiated by said user.

15

21. A mobile remote control device (1250) for communication with a control unit according to any of claims 14-20 and comprising a switches (1253), a processing unit (1254), a transmitter circuit (1255) and a sensor (1256).

20 22. The mobile remote control of claim 21 , wherein said switch is an alarm initiating switch.

23. The mobile remote control of claim 21 , wherein said sensor is a tilt and/or accelerometer sensor.

25

24. The mobile remote control of claim 21 , wherein said radio transmitter is Bluetooth.

25. A programmable remote control for communication with a control unit according to any of claims 14-20 comprising a control arrangement provided as icons.

30

26. The programmable remote control of claim 25 comprising a touch-screen for said icons.

27. A method for an audio/video-call allowing a first user to communicate with a second 35 user, wherein the communication includes a data stream, the method comprising: providing a communication system controlled by the first user which includes a video camera, a microphone, a network socket for connecting the communication system to the network, sending a call request as a data stream over the network from the communication system to a geographically separated second communication system; 5 converting the image of the user to a video signal with the video camera; sending the video signal as data flow over the network from the communication system to the second communication system once the call request is accepted by the second communication system and controlling the second system.

10 28. A computer program in a device (121) for connection to a communication network, a monitor (122) and audio/video receiver (124), the computer program further comprising: an instruction set for processing received data from an input connectable to a first user and initiating a communication channel to a second user, 15 - an instruction set for receiving control data by said first user and controlling said device, an instruction set for receiving a television broadcast and outputting said broadcast to said monitor, an instruction set for interrupting said received television broad cast and 20 outputting said initiated call in audio and video, and an instruction set for receiving control signals from said second user.

29. The computer program of claim 28, further comprising an instruction set for receiving and handling configuration data for said device from said fist user. 25