WO2010002237A2

WO2010002237A2 - Intrusion detection using parallax information

Info

Publication number: WO2010002237A2
Application number: PCT/MY2009/000078
Authority: WO
Inventors: Yen San Yong; Hock Woon Hon
Original assignee: Mimos Berhad
Priority date: 2008-06-30
Filing date: 2009-06-25
Publication date: 2010-01-07
Also published as: WO2010002237A3

Abstract

There is disclosed a method and system for detecting the presence of an object within a restricted zone that comprises of a computerized means (1), an image capturing means (2) having connection to the computerized means and adapted to capture image pair, a software resides in the memory device that once executed, performs intrusion analysis based on images captured by the image capturing means; characterized in that the intrusion analysis is performed by processing the captured image pair to obtain parallax information (10), estimating the position of an object in the image in a depth plane (11), and determining whether the object position collides within the restricted zone (12). If the object position collided within the restrictive zone, entry to the restrictive zone is considered as intrusion. The parallax information is calculated by first shifting (20) one of the image pair toward the other such that the images are overlapping and thereafter recording the number (21) of the overlapping pixels. The shift that has the most overlapping pixels is selected to be the selected parallax value and will be compared against values in the parallax checklist and if the selected parallax value falls within the values in the restricted zone, it is an intrusion into the restrictive zone.

Description

INTRUSION DETECTION USING PARALLAX INFORMATION

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to the field of intrusion detection and more particularly to a computerized method and system of intrusion detection that utilizes parallax information in an object of interest.

2. BACKGROUND OF THE INVENTION

Security is one of basic necessity of life and business everywhere. In area where restriction applies for which sensitive information, equipments or the likes should be protected against unauthorized access, suitable detection system for security should be deployed together with notification mechanism to replace the sometime inconsistent performance of human security guard services .

Intrusion detection systems per se are generally known in the art. They are typically used as surveillance device to detect and guard against intrusion, theft and other criminal activities perpetrated by individual or individuals with criminal intent. In general, the known detection system includes sensors, controller and other components installed within a prescribed zone or zones. When unauthorized person enters a secured zone, the sensor that typically includes passive infra-red sensor, vibration sensor, magnetic contact or other similar sensor in that zone will be activated and proceeded to send an activation signal to the controller to trigger the alarm or other form of warning signal. Problems associated with false alarm are common with this type of detection system. Such false alarm is sometime due to the over sensitiveness of the sensors employed, or the failure of the sensor to distinguish between legitimate access and non-legitimate access.

One of the possible solutions to replace the conventional sensor-based intrusion detection system is image processing and image recognition. Close circuit television (CCTV) system has long been used to monitor the security situation of guarded premises. Still and video (moving) images are used as input for the intrusion detection system. However, such prior image-based monitoring system also has limited effectiveness due to the problem of image processing to timely and correctly analyze ensuing possible intrusion event. One of the possible reasons, the effectiveness of the video cameras used that are exposed to the environment is affected by heavy rain, strong or even low availability of light.

One particular prior art image-based detection system is disclosed in US-2008/0043106-Al . The document discloses a system and method for intrusion detection that comprises of a first camera to acquire first visual image of a monitored area, =nd a second camera to acquire second image of the monitored area. The intrusion detection works by comparing the first image with a background image of the monitored area. The system then marks differences between the first image and the background image as potential intruder. Three- dimensional characteristics associated with potential intruder are also evaluated. In the embodiment of this disclosure, the method comprises of acquiring first images of the monitored area from a first camera, acquiring second images of the monitored area from a second camera and generating a background image from the monitored area. Further, the method also includes correlating first pixels associated with the first images with second pixels associated with the background image of the monitored area, such that the first pixels are horizontally and vertically aligned with the second pixels. The three-dimensional characteristics are compared against at least one predetermined threshold and if such threshold is exceeded, alert to indicate presence of intruder is raised. Such three-dimensional characteristic includes size of the moving object, location and/or movement. Although meeting its intended objects, the disclosed solution is not without having it inherent disadvantages. As an example, due to the intensive analysis of the images, performance tends to degrade.

Typically, stereo camera technique is used in -olumetric intrusion detection to define the border in three-dimension correctly and provide video surveillance. However in standard parallax estimation process used in such volumetric intrusion detection system, object features are required as input for the matching process and such processes are taking longer time which affects the detection performance in real time. Additionally, the performance of such a method depends more on the quality of result from the prior stage, i.e. object of interest extraction.

It is therefore an object of the present invention to provide an alternative solution to an image-based intrusion detection system as known in the art. The present invention proposes an intrusion detection system that uses image capturing means in the form of a stereo camera to capture image pairs which is then processed to obtain its parallax to estimate the object position in terms of depth plane of the object. The proposed solution then determines whether the object collides with the restricted zone and then takes appropriate action. Specifically, the parallax estimation is executed using image shifting and blob overlapping information. Such a method reduces the processing time and the dependency of the prior stage processing output, therefore much faster real time intrusion detection because the depth information is estimated without going through stereo matching or object features extraction as known in the prior art.

3. SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a method for intrusion detection system that utilizes parallax informatⁱon in object of interest .

It is also another object of the present invention to provide a system for intrusion detection that utilizes parallax information in object of interest.

It is yet another object of the present invention to provide a computerized system for intrusion detection that detects unauthorized access to a restricted zone.

These and other objects of the present invention are accomplished by providing,

An intrusion detection system adapted for detecting the presence of an object within a restricted zone (5) comprising :-

a computerized means (1) having at least a processor, a memory device and a storing device;

an image capturing means (2) having connection to said computerized means and adapted to capture image pair;

a software residing in the memory device that once executed, performing intrusion analysis based on images captured by said image capturing means;

characterized in that :-

said intrusion analysis is performed by processing the captured image pair to obtain its parallax information (10);

estimating the position of object in the image in a depth plane; and

determining whether the object position collides within said restricted zone and if the position of the object collided within the restrictive zone (5) , entry to said restrictive zone is considered as an intrusion.

Preferably, the image capturing means is a stereo camera having fish-eye lens for wider monitoring zone.

Preferably, the method used for the parallax information calculation is performed by first shifting one of the image pair toward the other such that the images are overlapping and thereafter recording the number of the overlapping pixels within the overlapping region.

Also preferable, the shifting is performed at a rate of one pixel or more at a time and the recording of the number of the overlapping pixels and comparison are performed at each shifting.

The objects may be further accomplished by providing,

In a computerized means (1) having at least a processor, a memory device, a storing device, an image capturing means (2) having connection to said computerized means (1) and adapted to capture image pair; a method is provided for detecting the presence of an object within a restricted zone (5) , characterized in that said method comprises the step of:-

a. analyzing said captured image pair to obtain its parallax information (10) ;

b. estimating the position of object in the image in depth plane; and

c. determining whether the object position collides within the restricted zone and if the position of the object collided within the restrictive zone (5) , entry to the restrictive zone is considered as an intrusion.

4. BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments of the invention will now be described, by way of example only, with reference to the accompanying figure in which :

Figure 1 shows an illustration of an intrusion detection system for detecting the presence of an object within a restricted zone according to the present invention.

Figure 2 shows a flow chart of a method for intrusion detection system according to the embodiment of the present invention.

Figure 3 shows a flow chart of a method for intrusion detection system according to the next embodiment of the present invention. Figure 4 and 5 show parallax information calculation utilized for intrusion detection system according to the present invention.

5. DETAILED DESCRIPTION OF THE DRAWINGS

Referring first to Figure 1, there is shown an intrusion detection system that is used for detecting the presence of an object according to the embodiment of the present invention. In this illustration, the intrusion detection system is shown comprises of a computerized means (1), an image acquisition means that is configured in the form of stereo camera (2) connected to the computerized means and a customized volumetric intrusion detection software that resides in the computerized means. Preferably, the image capturing image is stereo camera with fish-eye lens. The computerized means generally includes a processor (not shown) , a memory device (not shown) , a storing device (not shown) and a display terminal (3) . The premise for which the intrusion detection system is installed may be demarcated into zones i.e. free zone (4) and restricted zone (5) . The monitored zone (6) of the premise appears in both cameras. In this particular implementation, the premise is an exhibition gallery. The stereo camera (2) is utilized for capturing video image pair to be used as input to the computerized means. When executed, the software that resi^des in the memory device will perform intrusion analysis based on the parallax information of the images. When the entry of a person into the zones has been analyzed, suitable warning will be activated if such entry is found to be an intrusion to the restricted zone.

In the related matter, parallax information in general is known in the art. Parallax refers to the change of angular position of two observations of a single object relative to each other. By observing parallax, the position and distance of a particular object could be determined. Stereo camera is typically used to capture stereo images (left and right individual images) where they will be sent to the computerized means for processing. For stereo camera that uses fish-eye lens for purpose of wider monitoring coverage, the circular images will need to be transformed into rectangular images for the process. In objects analysis for security surveillance, the objects are extracted from the background and calculation will be performed to determine whether or not the object is within the restricted zone. Alert signal will trigger if the object intersects with the restricted zone.

Generally there are two types of disparity presentation in moving image analysis, sparse disparity map and dense disparity map. Volumetric intrusion detection requires general depth information for objects of interest, as such sparse disparity map is typically enough to perform the task. Generally, sparse disparity map could be acquired through any of the following steps :-

1. Feature extraction, where object features are extracted from the left and right images. Such object features include edge point, objects' area, objects' average intensity value, objects' bounding box and others.

2. Stereo matching, where the process comprises of matching the objects in one image to corresponding objects in other image with the aid of extracted objects' features.

3. Parallax calculation, where for each matched object, parallax is extracted. In this respect, Parallax = P_L - PR where P_L is the feature point (edge point, centroid, etc.) in the left image while P_R is the correspondence feature in the right image .

In contrast to the parallax calculation mentioned above, the proposed invention utilizes sparse disparity map that also includes defining parallax checklist, image shifting and recording the number overlapping pixels and parallax selection. In the defining parallax checklist (40), all parallax values that fall within the zones (i.e. the restricted zone (5), free zone (4) and monitored zone (6)) are taken into account for image shifting. In image shifting and recording the number of overlapping pixels (20), the steps include overlying the right image with i pixel shift in x-axis, on the left image, recording the number of overlapping pixels for each blobs (overlapping region between the images) . The i value is the parallax values that are listed in the checklist. If the i value fall within the values that are within the restricted zone (5), appropriate warning will be triggered.

Among the advantages processing in real time could be significantly reduced. In standard method typical of the known system, image features extraction and stereo matching occupy the majority of the processing time. The present invention eliminates this step thus significantly reduces processing time. Further, the dependency to prior stage (i.e. object of interest extraction) is also reduced. Object features are typically extracted based on the shape and intensity of the identified object. If the prior stage does not provide accurate result, it would affect the feature results and thus increases the false matching in stereo matching and also the parallax calculation. In the present invention, it uses binary image therefore problems created by intensity are eliminated. In addition, the present invention uses comparison among blobs to obtain the number of overlapping pixel which gives better tolerance for the shape of the extracted objects. As a result, more robust result could be obtained.

Referring now to Figure 2 where the flow chart of the method of volumetric intrusion detection system according to the first embodiment of present invention is illustrated. Generally, the intrusion analysis is performed by processing the captured image pair to obtain parallax information (10) , estimating the positi^~n of an object in the image in a depth plane, and determining whether the object position collides within the restricted zone (5) . If the object position collided within the restrictive zone, entry to the restrictive zone is considered as intrusion. Stereo image by the camera consists of left image (18) and right image (19). Extraction of object of interest (26) is performed. The parallax information is then calculated (10) . If the parallax value falls within the restricted zone, appropriate action will taken including alerting security, warning and etc. Such action will be taken because the entry into the zone is considered as an intrusion.

Figure 3 shows another alternative embodiment of the present invention where a fish-eye lens stereo camera is used. The angular or circular image captured by the fish-eye lens stereo camera will first need to be transformed into rectangular images through geometric transformation process (25) and extraction of object of interest (26) follows. The parallax information is then calculated (10) through process similar to the first embodiment.

Prior or standard method (30) of parallax information is illustrated in the same figure. Such prior method includes features extraction (31) , stereo matching (32) and parallax calculation (33) . As mentioned earlier, such prior method does has it disadvantages including slower processing and low tolerance on image quality. The present invention offered much faster processing because it does not requires object extraction and image processing to obtain depth information. The intrusion detection works through the application of defining the parallax checklist (40) th^~t defined parallax values that are required to be applied in image shifting. Then it performs the image shifting of the image pair captured by the stereo camera and recording of the number (20) of overlapping pixels in the overlapping region. In parallax selection (41), parallax value (i) that met the required criteria, such as the object parallax value that has maximum overlapping pixels, will be selected. If such value falls within the values in the restricted zone (5), the object entry into the restricted zone is deemed to be an intrusion.

Reference is now made to Figure 4 and Figure 5 to illustrate in greater detail the principal of parallax information calculation according to the present invention. As illustrated in Figure 4, an object of interest within a region will have different change of angular position as seen by the two cameras. The parallax boundary (42) between the two images could be considered as unauthorized entry into the restricted zone. The parallax information calculation of the present invention is performed by first shifting (20) the right image (19) of the image pair to the left image (18) with i pixel shift in the x-direction such that the images are overlapping and forming a blob. Overlapping pixels found in the overlapping zone will be counted (21) at each i shift and the i value that have the maximum numbers of overlapping pixels will be selected. As mentioned earlier, parallax selection is made on the i value that meets selection rule, for example the value that has maximum number of overlapping pixels.

In operation, the intrusion detection system works as the following. Any object entering into the monitored zone will be evaluated in real time. At each position the parallax value will be calculated based on the number of overlapping pixels counted within the overlapping zone or blob. If the value falls within the predetermined values that deemed to be an intrusion into the restricted zone, appropriate action will be implemented including sounding of alert, alarm and other. Although the discussion of the proposed invention generally refers to moving object analysis and detection, static object analysis could also be made without difficulty. Further, the image capturing means could also be implemented through used of single camera but equipped with a suitably arranged mirror to capture image pair, or multiplicity of cameras arranged near to each other or scattered independently. In the later arrangement, depending on the circumstance, images from any of the two image capturing means could be selected and subjected to the method of the present invention.

It is envisaged that the proposed system and method of volumetric intrusion detection utilizing parallax information in an object of the present invention could be implemented together with other security surveillance system such as remote monitoring, zoning of particular area, central monitoring system (CMS) and other application.

While the preferred embodiments of the present invention have been described, it should be understood that various changes, adaptations and modifications may be made thereto. It should be understood, therefore, that the invention is not limited to details of the illustrated invention shown in the figures and that variations in such minor details will be apparent to one skilled in the art.

Claims

WHAT IS CLAIMED IS:

1. An intrusion detection system adapted for detecting the presence of an object within a restricted zone (5) comprising :-

5 a computerized means (1) having at least a processor, a memory device and a storing device;

an image capturing means (2) having connection to said computerized means and adapted to 10 capture image pair;

a software residing in the memory device that once executed, performing intrusion analysis based on images captured by said image capturing means; 5 characterized in that :-

estimating the position of object in the 0 image in a depth plane; and

determining whether the object position collides within said restricted zone and if the position of the object collided within the restrictive zone (5) , entry to said 5 restrictive zone is considered as an intrusion .

2. An intrusion detection system as claimed in Claim 1, further characterized in that said image capturing means (2) includes a stereo camera having fish-eye lens.

3. An intrusion detection system as claimed in Claim 1, further characterized in that said parallax information is calculated by first shifting (20) one of the image pair toward the other such that the images are overlapping and thereafter recording the number (21) of overlapping pixels within said overlapping region.

4. An intrusion detection system as claimed in Claim 3, further characterized in that said shifting is performed at a rate of one pixel or more at a time according to values defined in a parallax checklist, and said recording the number of overlapping pixels is performed at each shifting, and thereafter selecting the parallax value that has the most overlapping pixels .

5. An intrusion detection system as claimed in Claim 4, further characterized in that said parallax checklist includes set of values that are required to be applied in said image shifting, said values include values in the restricted zone and values in a free zone, and if the selected parallax value falls within the set in the restricted zone, said entry is considered as an intrusion into the restricted zone .

6. An intrusion detection system as claimed in any of the preceding claims, further characterized in that said image is a moving image.

7. In a computerized means (1) having at least a processor, a memory device, a storing device, an image capturing means (2) having connection to said computerized means (1) and adapted to capture image pair; a method is provided for detecting the presence of an object within a restricted zone (5) , characterized in that said method comprises the step of:-

a. analyzing said captured image pair to obtain its parallax information (10) ;

b. estimating the position of object in the image in depth plane; and

8. A method for detecting the presence of an object as claimed in Claim 7, further characterized in that said image capturing means (2) includes a stereo camera having fish-eye lens .

9. A method of detecting the presence of an object as claimed in Claim 7, further characterized in that said parallax information is calculated by first shifting (20) one of the image pair toward the other such that the images are overlapping and thereafter recording (21) the number of overlapping pixels within said overlapping region.

10. A method of detecting the presence of an object as claimed in Claim 9, further characterized in that said shifting is performed at a rate of one pixel or more at a time and said counting of overlapping pixels is performed at each shifting, and thereafter selecting the parallax value that has the most overlapping pixels .

11. A method of detecting the presence of an object as claimed in Claim 10, further characterized in that said parallax checklist includes set of values that are required to be applied in said image shifting, said values include values in the restricted zone and values in a free zone, and if the selected parallax value falls within the set in the restricted zone, said entry is considered as an intrusion into the restricted zone .