WO2001006791A1

WO2001006791A1 - Wireless video surveillance system

Info

Publication number: WO2001006791A1
Application number: PCT/AU2000/000364
Authority: WO
Inventors: Ruben Gonzalez
Original assignee: Activesky Inc.
Priority date: 1999-04-22
Filing date: 2000-04-26
Publication date: 2001-01-25
Also published as: JP2003504985A; CN1347619A; KR20010112444A; AUPP990199A0; EP1172004A1

Abstract

A wireless video surveillance system incorporating a video recording means recording video images, an interface means converting the video images to transmittable data, and a portable monitor device for receiving the transmittable data, converting the data to video images and displaying the video images. The video recording means is suitably a commercially available video camera and the portable monitor device is suitably a commercially available personal digital assistant. In one form, the PDA is able to transmit control signals to the interface means to control the video recording means. Other devices, such as an external data storage, movement detectors, sound detectors and light switches, may also be interfaced with the interface module.

Description

WIRELESS VIDEO SURVEILLANCE SYSTEM

This invention relates to an apparatus and method for wireless video surveillance and communication. In particular, it relates to an apparatus and method employing commercially available, hand-held portable devices, such as personal information managers and personal digital assistants.

BACKGROUND TO THE INVENTION

The use of personal digital assistants (PDA's), such as the 3COM Palm Pilot^®, is becoming very widespread. Functionality of PDA's varies between manufacturers but all include a liquid crystal display, an input device, memory and a processor unit. Various improvements are constantly being made to increase available memory, improve processing power and extend software applications. Recently, PDA manufacturers have released devices having colour liquid crystal displays and wireless communication capability. This innovation has been driven by a desire to provide PDA's with internet access for web surfing and e-mail. Similar market pressures are driving the integration of processing capability into mobile phones to facility direct web access for the same purposes.

It is also known to integrate PDA's with mobile phones to provide an integrated system giving full voice and data transfer facilities. One such system is described in United States patent number 5625673, assigned to Lucent Technologies Inc. The patent describes a PDA that includes means for cordless connection to specialised accessories, such as a cellular telephone and a modem.

The capabilities of PDA's have not been extended for application in the video surveillance area. Existing surveillance systems, such as house security systems, do not normally offer a video surveillance capability. When such a capability is provided, it is usually linked to a base station that provides remote manual surveillance.

Substitute Sheet The majority of existing surveillance systems utilise a local processing centre that packets data for transmittal to the base station. Transmittal may be by dedicated land-line or may be via a dial-up connection.

OBJECT OF THE INVENTION

It is an object of the invention to provide an apparatus and method for wireless video surveillance.

SUMMARY OF THE INVENTION

In one form, although it need not be the only, or indeed the broadest form, the invention resides in a wireless video surveillance system comprising: a portable monitor device and first wireless communication means; one or more video recording means for recording video images of a scene; and at least one interface module converting said video images to transmittable data, said interface module incorporating a second wireless communication means for transmission of said transmittable data from said at least one interface module to said portable monitor device, said portable monitor device incorporating means for receiving said transmittable data, converting said transmittable data to said video images and displaying said video images.

In preference, the portable monitor device is a personal digital assistant or similar hand-held processing unit incorporating processor means, memory means and video display means.

The video recording means may be a digital camera or may be a commercially available analogue video camera, such as a Camcorder^®.

Suitably, the interface module includes input means for receiving video signals from said video recording means. A suitable input port is a USB port for digital video input. If an analog video recording means is employed, the input means suitably includes a video input port and

Substitute Sheet analog to digital conversion means.

The interface module preferably also includes processing means for converting said video images to transmittable data and one or more transmission buffers. The wireless transmission means is suitably a signal transmitting means but is preferably a signal receiving and transmitting means.

The processing means of the interface module is suitably programmed with video and audio compression algorithms. Corresponding audio and video decompression algorithms are suitably programmed in the processor means of the portable monitor device.

The portable monitor device suitably also includes input means for inputting signals for transmitting to said interface module. Preferably said first wireless communication means includes means for transmitting said signals and said second wireless communication means includes means for receiving said signals.

In a further form, the invention resides in a method of providing wireless video surveillance including the steps of: recording a video image of a scene; processing the recorded image to form data for wireless transmission; transmitting the data to a portable monitor device; and processing the data to display the image on the portable monitor device.

Processing the recorded image preferably includes the steps of compressing the image at the interface means and decompressing the image at the portable monitor device.

The method may further include the step of storing said data for later transmission.

BRIEF DESCRIPTION OF THE DRAWINGS Preferred embodiments of the invention are described with reference to the following figures in which: FIG 1 is a block diagram of a first embodiment of the invention;

Substitute Sheet FIG 2 is a block diagram of a first embodiment of the interface module; FIG 3 is a block diagram of a second embodiment of the interface module; FIG 4 is a block diagram of a third embodiment of the interface module;

FIG 5 is a block diagram of the portable video monitor;

FIG 6 is a block diagram of a second embodiment of the invention; and

FIG 7 is a block diagram of a third embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIG 1 there is shown a block diagram of a wireless video surveillance system consisting of a video recording means 1, in signal connection with an interface module 2 which is in signal connection with an antenna 3. Signals 4 are transmitted between the antenna 3 and an antenna 5 associated with a portable monitor device 6 which is a personal digital assistant or portable personal computer. Video images from the recorder 1 are displayed on a display 7 of the monitor 6.

Video and audio signals are transmitted from the interface module to the portable monitor device and control signals may be transmitted from the portable monitor device to the interface module.

In the video recording means 1 is a commercially available VHS video recorder having standard video and audio outputs. Other video formats may include PAL, NTSC, SVHS, BETACAM, RGB, SECAM and DVD.

A first preferred embodiment of the interface module 2 is shown in greater detail in FIG 2. In this embodiment the interface module is configured to transmit signals to the portable monitor device but is not configured to receive signals. Video and audio input from the recorder 1 is input through port 8 and converted to digital form in analogue to digital converter 9. The output from the ADC is buffered in dual frame buffer 10 for input to a digital signal processor 11.

Substitute Sheet Alternatively a digital camera could be employed as the video recording means 1. In this case, the digital output from the camera can be input directly to the dual frame buffer 10 through USB port 12. Although a USB port is described in the preferred embodiment, it will be appreciated that other interface protocols such as SCSI, IEEE 1394, etc, can be used.

The digital signals are processed for transmission in the digital signal processing chip 11. The required processing will depend on the transmission network being employed and known signal processing and compression algorithms can be used. Suitable algorithms include H.261 , H.263, MPEG1 , MPEG2, MPEG4, etc. Random access memory 13 is provided for on-board storage of data for signal processing.

Although a software implementation of the signal processing has been described, a hardware implementation is also suitable. In one embodiment, signal processing can be performed using an ASIC

(Application specific integrated circuit)/FPGA (Field programming gate array) solution. Alternatively, an ASIC front end can be used with a programmable DSP backend. The invention is not limited to any one of these possible solutions. If a DSP solution is used, the DSP is programmed with available compression software. The invention is not limited to any particular software solution, although the inventor has 8-bit and 24-bit coding methods are suitable. Different coding methods may be more suitable for certain colour video displays available in the wide variety of PDA's currently on the market. New coding solutions are regularly becoming available and the inventor recognises that the invention may employ new algorithms in the future.

The selected video compression method will implement three basic stages. The first stage performs temporal decorrelation, also termed interframe coding. This process consists of an optional block based motion compensation step followed by temporal prediction and replenishment. This determines which portions of the image frame have

Substitute Sheet changed and selectively sends update information to reproduce the changes. This eliminates the need to retransmit the entire image each time when there have not been extensive change over time. A side product of this stage is that the output can be used to trigger an alarm to be sent to the video monitor when any motion is detected.

Depending on the coding method used, quantisation may be performed at this point.

The next stage decomposes the resultant interframe images from the previous step into basic coding units. The nature of these units varies depending on the coding method being used. In the classic transform based coding methods these coding units are 2D blocks of spatial frequency coefficients. These may equally be image primitives of another form such as run length vectors or uniform colour blocks. The values of the coding units are then substantially quantised to eliminate unwanted information. This process is known as intraframe coding.

The final stage involves reordering the symbols or values to be coded so as to enhance compressability followed by some form of variable length or statistically based coding to efficiently create a compact bit stream for each frame. This final stage is almost always lossless. This resulting bit stream is then appropriately packaged in a frame packet for insertion into the output data stream.

Audio coding will also be implemented in the DSP stage. In one embodiment a standard ADPCM compression method is used. GSM speech coding will also be useful for use in conjunction with the integrated PDA/mobile phone scenario.

For each frame of video compressed and transmitted an audio frame of equivalent time duration will also be compressed and sent. In this way, excepting the event of system control frames in the data stream, the predominant structure of the transmitted data stream will consist of alternating video and audio frames in an interleaved manner.

The processed data is buffered in transmission buffer 14 before

Substitute Sheet being transmitted by transmission module 15. The transmission module 15 can be any suitable wireless transmission system such as CDMA (code division multiple acces), TDMA (time division multiple access), IP (wireless internet protocol), PHS (personal handyphone system), LMDS (local multipoint distribution service), wireless ATM, GSM, GPRS, HSCSD and other 3G wireless networks. Alternative embodiments are discussed in detail below.

It will be appreciated that the elements of the interface module need not be physical contained in a single device. For example, the transmission module may be separate fro the rest of the interface module. For a wireless WAN protocol, the transmission module may be a GSM or CDMA base station connected to the rest of the interface module by a modem or ISDN connection. If the communication protocol is a wireless LAN, the transmission module may be connected to a separate base station or wireless access point by an ethernet. The transmission module may also be a television frequency modulator to permit the device to transmit to a pocket television that may be in range.

A second embodiment of the interface module is shown in FIG 3. This embodiment is configured to transmit signals to the portable monitor device as well as receive control signals. The DSP is replaced with a general purpose CPU 16. As with the first embodiment, video and audio signals are received at input 8, digitised by ADC 9, and buffered in DFB 10. Signals from a digital source can be input directly on input 12.

As well as RAM 13 for local storage, an external storage device 17 is provided. The external storage device allows large volumes of data to be stored which can be accessed by signals from the portable monitor device. Two types of external storage devices may be provided. Uncompressed analogue video data must be stored in a VCR which is accessed by a sequential, read-only access. Compressed digital data can be stored on any of a number of suitable devices

Substitute Sheet (including magnetic disks drives, digital tape drives, CD ROMs, DVDs, WORM drives, etc) with read-write, random access. There may be multiple external storge devices.

The external storage devices permit a history of video and audio signals to be stored for review at a later time. One application of this arrangement is the removal and storage of recordings, such as video and audio tapes, for use in legal proceedings that could result from security breaches detected by the invention. Other applications are discussed below. Signals 4 are sent and received by a transmit/receive means 18.

A number of buffers 19 are provided for both transmission and reception. An output means 20 provides analogue or digital signals at port 21 for controlling various other devices in response to signals received from the portable monitor device. For example, a signal from the potable monitor device could be used to turn lights on or off, reset a motion detector or external alarm, or commence video recording. In one application, a signal from the portable monitor device could be used to commence playback of a prerecorded programme stored on the external storage device 17. A third embodiment of the interface module 2 is shown in FIG 4.

This embodiment differs from the second embodiment by implementing a multiple address data bus 22. This embodiment facilitates the use of multiple external storage devices and multiple output means. It is also possible to implement multiple input sources which may each have a unique address selectable from the portable monitor device. This embodiment provides maximum flexibility to the wireless video surveillance system. For example, the range of devices connected to the interface module may include a number of video cameras, one or more external movement detectors and multiple control lines (such as light switches). Detection of movement would generate a warning signal that is transmitted to the portable monitor device. The user would then transmit a signal to select the video camera closest to the

Substitute Sheet detected movement. The video is then visible in the LCD 7 of the monitor 6.

The important elements of the portable monitor device 6 are shown in greater detail in FIG 5. The monitor includes a receive/transmit module 23 for receiving the data transmitted by the interface module 2 and transmitting data to the interface module.

Buffers 24 are provided for both transmission and reception. Received signals are processed in the CPU 25. On-board RAM 26 stores data and parameters for the signal processing. The processed image is displayed on the liquid crystal display 7. Control signals input on line

27 are processed by the CPU 25 and transmitted to the interface module 2.

The portable monitor device can be any device capable of wireless network connection that has a graphic display and a general purpose processor. PDAs are preferred due to their availability but many mobile phones, such as WAP phones and l-NET phnes are also suitable. Naturally, purpose built devices will also be suitable. The inventor envisages that hybrid devices that interface a custom DSP with a standard PDA will be useful. In this form, the video decoding process may be performed in the custom DSP instead of the CPU of the PDA.

Fig 6 shows an embodiment in which a VHS video camera 28 is connected to an interface module 29 that provides a low power signal

30 to an antenna 31 for a local area network. This embodiment may have a typical range of 150m for signals 30. The signals 30 are typically radio frequency signals although infrared signals may also be suitable in point to point applications.

The monitor device 33 is a personal digital assistant with a wireless network card 34 and antenna 32 to receive signals 30. The wireless network card 34 interfaces to the personal digital assistant through a PCMCIA slot, compact flash port, or other expansion port. The embodiment of Fig 6 supports simplex video and full duplex

Substitute Sheet audio communication. It is most suitable for implementation in domestic applications, for child minding and security. For example, a parent may use the configuration to monitor a child sleeping in a separate room. Another application is to monitor an entryway. The full duplex audio allows communication with a person seeking entrance while the simplex video channel allows the person to be viewed.

The primary advantage of the embodiment of Fig 6 is that the video monitor is portable and therefore the user can move freely around the home. It will be appreciated that this is a significant improvement over known prior art security and child-minding systems. For longer range applications the embodiment of Fig 7 is preferable. A digital video camera 35 provides audio and video input to the interface module 36. The transmission module 37 is a standard telephone interface for transmitting signals 38 across a CDMA or GSM cellular phone network from antenna 39. The monitor device comprises a personal digital assistant 40 with a modem 41 connected to a phone 42. The PDA is programmed with algorithms to process the received data for display.

In a variation of this embodiment the PDA and phone are integrated into a single device. Suitable devices have recently become commercially available.

As mentioned above, the embodiments provide simplex video and duplex audio communication. The second embodiment provides duplex audio via the phone. Duplex audio communication for the first embodiment may be provided by built-in microphones provided in some PDA's or by an external microphone coupled to the network card.

These embodiments may be extended by providing local storage of audio and video footage which can be viewed on command from the monitor. Movement and sound detection functionality have also been provided in the interface module so that an alarm can be provided if movement or sound is detected in the viewed scene. Combined movement detection, sound detection and local video storage are

Substitute Sheet particularly useful for the second embodiment to facilitate dial-up operation.

In one method of operation, the interface module is configured to detect movement in the field of view. When movement is detected the video footage is stored locally and a call is placed to the portable monitor device. The phone touch tones or the duplex audio channel may then be used to send tone commands to trigger replay of the stored video footage. The great advantage of this systems is that security is provided without the need of a fixed base station with permanent monitoring.

The external storage may be used to store a fixed period of data in one or more FIFO buffers. The size of the FIFO buffers (and hence the length of recorded video) can be a user set variable. When movement is detected the user can choose to view a block of video from a fixed period before the alarm.

It will be appreciated that multiple cameras and interface modules can be programmed to a single or multiple video monitors. It would therefore be possible for a number of security guards to carry personal digital assistants providing mobile monitoring of multiple camera installations.

Although the above embodiments only offer simplex video communication, it will be appreciated that the invention is not limited to this implementation. The provision of simplex rather than duplex video is due to the processing power available in commercial PDA's. Improved processing power will allow the PDA's to run software that permits compression and transmission of video images. A video recording means is added to the portable monitor device for duplex video transmission.

It will be appreciated that suitable network protocols must be used to ensure that video data is reliably transmitted across the wireless link to the remote monitor. These may be connection oriented, such as TCP, or connectionless, such as UDP. The nature of the

Substitute Sheet protocol will hcnage depending on the nature of the wireless network being used, the bandwidth, and the channel characteristics. The protocol must perform the following functions: error control; flow control; packetisation; connection establishment; and link management. There are many existing protocols for these purposes that have been designed for use with data networks. However in the case of video, special attention may be required to handle errors since retransmission of corrupted data is inappropriate due to the real-time constraints imposed by the nature of video on the reception and processing of transmitted data.

To handle this situation the following error control scheme is provided.

(1 ) Frames of video data are individually sent to the receiver, each with a check sum or cyclic redundancy check appended to enable the receiver to assess if the frame has been received in error;

(2a) If there was no error then the frame is processed normally; (2b) If the frame is in error then the frame is discarded and a status message is sent to the transmitter indicating the number of the video frame that was in error; (3) The video transmitter upon receiving such an error status message stops sending all predicted frames and instead immediately sends the next available key frame to the receiver;

(4) After sending the key frame the transmitter resumes sending normal interframe coded video frames until another error status message is received.

A key frame is a video frame that has only been intraframe coded but not interframe coded. Interframe coding is where the prediction processes is performed and makes these frames dependent on all the preceding video frames after and including the last key frame. Key frames are only sent as the first frame and whenever an error occurs. The first frame needs to be a key frame because there is no previous frame to predict from to perform the interframe coding

Substitute Sheet process.

Although the primary application is envisaged as being point-to- point, a broadcast implementation is also possible. In this case, there may be multiple monitors for a single base station. This implementation may be particularly useful in a facility security application where a number of guards are monitoring the security of a facility from a variety of locations.

Authentication security may also be incorporated into the system to minimise the risk of unauthorised use if the portable monitor device is lost or stolen. Suitable security protocols will be known to persons skilled in this area.

Throughout the specification the aim has been to describe the invention without limiting the invention to any specific combination of features.

Substitute Sheet

Claims

1. A wireless video surveillance system comprising: at least one portable monitor device and first wireless communication means; one or more video recording means for recording video images of a scene; and at least one interface module converting said video images to transmittable data, a second wireless communication means associated with said interface module for transmission of said transmittable data from said at least one interface module to said portable monitor device, said portable monitor device incorporating means for receiving said transmittable data, converting said transmittable data to said video images and displaying said video images.

2. The wireless video surveillance system of claim 1 wherein the portable monitor device is a personal digital assistant or similar handheld processing unit incorporating processor means, memory means and display means.

3. The wireless video surveillance system of claim 1 wherein the interface module includes input means for receiving video signals from said video recording means.

4. The wireless video surveillance system of claim 1 wherein the video recording means is a digital camera

5. The wireless video surveillance system of claim 1 wherein the video recording means is an analogue video camera.

6. The wireless video surveillance system of claim 5 wherein the interface module includes a video input means and analog to digital conversion means.

7. The wireless video surveillance system of claim 1 wherein the interface module includes processing means for converting said video images to transmittable data and one or more transmission buffers.

8. The wireless video surveillance system of claim 7 wherein the

Substitute Sheet processing means is programmed with video and audio compression algorithms.

9. The wireless video surveillance system of claim 1 wherein the interface module includes processing means programmed with video and audio compression algorithms and wherein the portable monitor device incorporates a processor means programmed with corresponding audio and video decompression algorithms.

10. The wireless video surveillance system of claim 1 wherein the first wireless transmission means is a signal receiving means.

11. The wireless video surveillance system of claim 1 wherein the first wireless transmission means is a signal receiving and transmitting means.

12. The wireless video surveillance system of claim 1 wherein the second wireless transmission means is a signal transmitting means.

13. The wireless video surveillance system of claim 1 wherein the second wireless transmission means is a signal receiving and transmitting means.

14. The wireless video surveillance system of claim 1 wherein the second wireless communication means is integral with said interface module.

15. The wireless video surveillance system of claim 1 wherein the portable monitor device includes input means for inputting signals for transmitting to said interface module.

16. The wireless video surveillance system of claim 15 wherein said first wireless communication means includes means for transmitting said signals and said second wireless communication means includes means for receiving said signals.

17. A method of providing wireless video surveillance including the steps of: recording a video image of a scene; processing the recorded image to form data for wireless transmission; transmitting the data to a portable monitor device; and

Substitute Sheet processing the data to display the image on the portable monitor device.

18. The method of claim 17 wherein the step of processing the recorded image includes the further steps of compressing the image at the interface means and decompressing the image at the portable monitor device.

19. The method of claim 17 further including the step of storing said data for later transmission.

20. The method of claim 17 further including the transmission protocol steps of: transmitting the data as individual frames of video data, said individual frames including key frames and predicted frames, each frame including a check sum or cyclic redundancy check appended to enable a receiver at the portable monitor device to assess if the frame has been received in error; if the frame is in error, discarding the frame and transmitting a status message indicating an identifier of the frame that is in error; stopping the transmitting of predicted frames and transmitting the next available keyframe; and after transmitting the keyframe resuming transmitting predicted frames.

Substitute Sheet