WO2009052574A1

WO2009052574A1 - Improvements in oudoor advertising metrics

Info

Publication number: WO2009052574A1
Application number: PCT/AU2008/001573
Authority: WO
Inventors: Andrew James Mathers
Original assignee: Andrew James Mathers
Priority date: 2007-10-25
Filing date: 2008-10-24
Publication date: 2009-04-30

Abstract

A monitoring device for advertising has a camera located to record images of multiple targets within viewing distance of an advertisement and a computer for receiving images of targets from the camera. The computer runs software for image processing on those images of targets. The software comprises modules for background subtraction and for determining confirmed impressions from targets in an audience of the advertisement.

Description

IMPROVEMENTS IN QUDOOR ADVERTISING METRICS

Field of the Invention

The present invention relates to outdoor advertising metrics and, in particular, to a method and a system for collecting advertising information from a camera or sensor. More particularly, the present invention relates to the collection of selected live audience metrics.

Background of the Invention

Although the background, objects and preferred embodiments of the invention will be hereinafter described with reference to outdoor advertising, it is to be understood that the invention is not limited thereto but has wider application. For example, the information gathered and processed in accordance with the invention may be used by government or non commercial institutions, or collected in the home.

It is to be understood that the terminology employed herein is for the purpose of description only and should not be regarded as limiting. For instance, the terms "comprising" or "comprises" are to be understood as non-limiting, unless otherwise stated. The term "machine vision" is to be understood as including cameras or other visual sensor devices that can, using software, identify features or attributes of people in the immediate vicinity of any advertising displays with which the machine vision are associated.

The traditional measurement of outdoor advertising effectiveness has been based on guessing the audience metrics by collating historical data from government bodies involved in town planning and development. A problem for advertisers has been that this is inaccurate and does not represent a true or realtime account of the audience demographics, nor does it offer accurate indication as to who looked at the advertising display, or when.

It has been found by the present inventor that the traditional format of out of home advertising measurement does not optimize the opportunity for advertisers to market their brand to viewers, and maintains the burden on the commercial networks to sell advertising space.

For the outdoor network owner, for example, the invention may provide a system for selectively delivering metric information from a periodic or continuous broadcast of metrics derived from machine vision to the owner of outdoor properties and means for providing data regarding the number of pedestrians and vehicles looking at and around the advertising. For Out of Home (00H) network owners, the system functions as an inventory optimization tool.

For the media planner, the invention may provide a system for selectively delivering metric information for aggregating the outdoor network traffic locations and demographics and means for providing visibility to the most valuable and relevant locations, in time and space, for media placement, with defined traffic metrics across a placement campaign. This enables offering a network's clients or media agencies a web interface, to enable the network to take their OOH media spend and key performance indicators (timing, reach, frequency etc) and enter these into the system, where a range of price vs benefit options will be generated and displayed.

For the brand or advertiser, this tool can be used as a contributor to the embracing of digital OOH by advertisers, because of the ease of the planning, buying and campaign accountability. There are additional features of the invention that will be described hereinafter. As such, those skilled in the art will appreciate that the conception, upon which the disclosure is based, may be readily utilized as the basis for designing other methods and systems for carrying out the objects of the present invention. It is important, therefore, that the broad outline of the invention described be regarded as including such equivalent constructions in so far as they do not depart from the spirit and scope of the present invention.

Summary of the Invention

It is, therefore, an object of the present invention to address the aforementioned shortcomings of the prior art.

It is another object of the present invention to provide a collection and processing of audience data passing an advertising display.

Accordingly, the invention provides a monitoring device for advertising having a camera located to record images of multiple targets within viewing distance of an advertisement and a computer for receiving images of targets from the camera. The computer runs software for image processing on those images of targets. The software comprises modules for background subtraction and for determining confirmed impressions from targets in an audience of the advertisement. There is also software for generating a report that includes an account of confirmed impressions.

In some embodiments, the software comprises modules for decomposing the images and for determining confirmed impressions and demographic data from targets in the audience of the advertisement together with software for generating a report that includes an account of confirmed impressions and demographic information about targets in the audience. Brief Description of the Drawing Figures

Figure i is a schematic diagram illustrating image capture of an audience having targets and image processing; Figure 2 is a schematic diagram illustrating image capture and software isolation of head portions of targets; Figure 3 is a schematic diagram illustraing the identification of target attributes, being objects associated with a person; Figure 4 is a schematic diagram illustrating the capture and processing of an advertising identifier; Figure 5 is a schametic digram illustrating the synchronizaion of a target with an advertising identifier; and Figure 6 is a schematic diagram illustrating aspects of the present invention.

Description of Preferred Embodiments

With reference now to the above summarized invention, a method and a system embodying the principles and concepts of the present invention will now be described. The description focuses on the measurement of all out of home advertising formats in relation to audiences using machine vision. By extension, the same tecnology may be applied in any type of location. The tecnology is specifically conducted with the use of machine vision (this is a general term for object recognition software used with a video or still camera feed having a fixed or moving field of view) to a computer processor. "Opportunity to see" refers to a characteristic of persons or vehicles ("targets" or "consumers") in the field of view and "confirmed impressions" refers to a target that is facing the advertising and at least potentially looking at it. Demographic or other descriptive attribute details of audiences may be based on the requests of the advertiser or the network owner, these requests being derived from their own understanding of their target consumer market. Demographics (at least in the context of this specification) can be as broad as make of vehicle, make of clothing or accessories to age, sex, ethnicity and gender. Also, the use of a mobile phone or baby pram may be appropriate as demographic identifiers. Any feature or attribute or object associated with a target or a person maybe useful. This use of machine vision is specifically conducted to acquire a cost per thousand ("CPM") of confirmed advertising audience impressions.

An out-of-home advertising, real time CPM may be achieved by monitoring audiences their volumes and their attributes (including the aforementioned demographics), passing an out-of-home advertising display and then delivering a definitive cost for the advertising based on a real time account of the audience volumes provided by the live video feed into a data system that captures and processes the image attributes.lt is also important that the system is capable of deleting images of targets or consumers upon processing thus protecting the privacy of the individual. Monitoring the audiences of the advertising displays and use of the information collected is primarily for advertising market research purposes. However additional uses are possible. For example, the system can provide information for other uses such as town planning, crowd control and politics.

As shown in Figure l, the invention provides methods and apparatus for counting and categorizing both "opportunity to see" and "confirmed impression" targets. A sensor such as a digital or still camera ioo is located on, adjacent to or within viewing distance of a visual advertisement 101. The advertisement maybe a printed media such as a billboard or a digital medium, either moving or still. In this embodiment, the camera ioo collects moving or still images of multiple individual people or vehicles ("targets") 102 comprising an audience. An image is preferably of an entire audience. The camera ioo transmits image data, preferably wirelessly 103, to a network 104 such as the internet. The "raw" or unprocessed image data 105 is received by a computer 106. The captured image, video segment or stream is processed by the computer 106 so as to isolate important aspects of the captured image 107. As will be further explained, the image isolation in this example 107 comprises a background subtraction as well as an isolation of skin tones. Skin tones are detected by colour changes over time and colour differences. In this way, skin can be readily distinguished from clothing, hair and the background. Where video or moving images are used, the background is substractable because it is static whereas the targets are in motion. In static images, the background remains relatively constant form frame to frame. The next step 108 is the counting of the isolated target images. In this example there are three targets. In the next step, the target images are classified using one or more of a variety of software algorithims 109. In this example, the software determines that one target HO is not facing or looking at the advertisement 101. This is because the head area is fully or almost fully comprised of hair 111. The other two targets 112, 113 are categorized as confirmed impressions because the software has, for example, either not detected a predominance of hair 111 or has e.g. detected one or both eyes 114 of the appropriate target. From the aforementioned data processing, the software is able to calculate and report on the total number of targets in the audience 115 and then determine the number of percentage or confirmed impressions by dividing the number of targets by the number of confirmed impressions in the audience 116.

In a second embodiment depicted in Figure 2, the image processing software isolates the head portions 200 of the targets 201 in the captured image data. Accordingly, the classification algorithim 202 is run only on the isolated head portions 203.

The camera 100 must be adapted to capture an audience in a field of view (that may be still or moving). A camera with a fish eye lens is particularly well adapted. The camera may also be adapted to simultaneously capture both the targets in the audience as well as the advertisement 101. This enables the camera 100 to detect (and correlate) either the advertising content itself or a coded or unencoded identifier 120 that appears within the advertisement. An advertisement 101 may also be associated with a time code or other data that can be compared to the images captured by the camera so that the captured identifier 120 can be correlated with particular advertisements, locations, time, etc. An enhancement of the above discussed methods and apparatus is depicted in Figure 3. The technology discussed with regard to Figures 1 and 2 for image capture and transmission to a computer are the same. However, the images are additionally used for detecting specific kinds of targets and for performing demographic analysis. As suggested by Figure 3, the captured image 300 comprises various different targets 301. The captured image is processed so as to isolate the targets 302. The isolated targets are ordered or sorted 303 to make the analysis easier for subsequent software operations to deal with. Software algorithims operate on the sorted images to identify specific and particular objects, colours, physical dimensions or other attributes. In this example, an isolated image of a target is further decomposed and analysed by software to reveal that the target person is holding an object 305. The target person is also resolved by software to reveal a dark vertical colour field that terminates below the head and extends to (about) the waist. In one example, the software futher isolates the held object 305 and the vertical colour field 306. Further software operations assist in more specifically identifying the target person 307, the object 305 in person's hand and the person's vertical colour field 306. In one method step or series of software operations 308, the target person 307 is compared with database or stored image data or analysed (as per the examples given in Figures 1 and 2) to determine, for example, if the target qualifies as "opportunity to see" or a "confirmed impression". Further analysis can reveal, for example, that the physical height of the target qualifies the target as an adult male. By comparing the isolated images of the held object 305 and vertical colour field 306, the software determines that the held object 305 is a mobile telephone and that the vertical colour field 306 is a neck tie. Subsequently, the target person 307 is associated, by the software, as in possession of a mobile telephone 309. The same target person 307 is associated or linked, by the software, with a neck tie 310. The software then compiles 311 the various attributes, properties, characteristics or other data about the target person 307 and compares these against pre-existing assumptions 312. For example, in some locations, an individual target person 307 in possession of a mobile telephone 309 and wearing a tie 310 can be categorized 313 as an employed male having particular income and spending attributes 314. Of course, the assumptions that are applied to the target's characteristics, attributes, possessions, clothing etc will result in different assumptions, depending on the location of the audience, the time that the image was captured, culture, season and other factors. These are all taken into account by the software environment in which the image processing occurs.

Another optional feature of the invention is depicted in Figure 4. Any of the above teachings, apparatus and methods may be combined with digital advertisement recognition. As suggested by Figure 4, an advertisement, particularly a digitally transmitted one 400 can carry an advertising identifier 401. The advertising identifier maybe visible to the human eye or displayed in a way that it is "detectable"by software but not to the human eye. The image capture device, camera or sensor 100 is adapted to capture the advertising identifier 401 at the same time that it captures images of the targets 102. Thus, at least a portion of the captured images comprises a sub-image 402 that includes at least a portion of the advertisement 400 and its advertising identifier 401. The receiving computer 403 is adapted, using software to isolate and recognize 404 that portion of the captured image that represents the advertising identifier 405. Having made an identification of the advertising identifier 401, the software can correlate the advertisement 400 with any other data including target data, time data or data regarding advertiser, network, brand etc.

An example of synchronizing a target with a particular advertisement is depicted in Figure 5. If a fairly specific target can be synchronized as a confirmed impression with a specific advertisement, then the effectiveness of the advertisement is fairly easy to confirm or deny. In this example, images are captured, simultaneously of an audience having targets 500 and of the advertising identifier 501. The targets are isolated 502 and the image identifier is isolated 503. The various target images and sub-images are matched and otherwise analysed 504. The advertising identifiers are also matched against database data so as to properly identify them 505. Then the various target images ans sub-images are categorized 506. The image identifiers may also be categorized 507 according to factors such as branch, market category, brand, network etc. The advertising identifier or attributes of it are then synchronized with the attributes, characteristics and data extracted from the target images and sub-images 5θ8. This allows analysis to be performed regarding the effectiveness of the advertising, the influence of the advertisement on the audience and its targets, and thus enables the suitability of effectiveness of the advertisement to be established.

In summary and as suggested in the flow chart of Figure 6, , the above described system is able to collect data 601 at advertising sites, on individuals etc. and process it with, for example, an IP camera system and suitable software to enable networks to account for OOH or other media assets. The image-processing unit (IPU) consists of an input device such as a camera 600 and one or more computer processors. The input device may be a video camera 600 such as a monochrome camera, a full colour camera, or a camera that is sensitive to non-visible portions of the spectrum. Images may be a set of captured frames, video, moving images etc. 601. The input device may also include a plurality of cameras for generating a three dimensional image or a single camera with different objectives or filters for recording an image or a single at different wavelengths, e.g. visible, IR or UV. Input may be visible to the end user or hidden. The generally remote (from advertising site) image-processing unit uses background subtraction techniques 602 to separate the individuals from the background. In order to recognise particulars in an image, (e.g. one or more humans, their gender, type of clothing, object association) the image must go through different processing steps that can be broadly categorised into five categories in the following example:

1. Face detection — each separated sprite goes through neural network based face detection 603. The detected face is then normalised to 20 x 20. The normalised face is passed on to the next modules for further processing.

2. Object detection — The sprite is converted to binary using an edge detection technique, all the edges inside the image are detected 604. The system has a reference set of predefined shape models such as glasses etc. A variable size search window scans the binary image to find matches.

3. Colour processing 605 — the colour that is dominant in the upper part of each individual is processed in this engine, from this it can determine the colour of the shirt worn by the individual. Each similarly coloured cluster may also be analysed for texture. Regions or clusters of pixels with similar colour classifications maybe determined and compared to identify and distinguish the texture of skin from clothing.

4. Counting and tracking — All detected faces are processed in this module to enable accurate counting of individuals.

5. Classification — A Support Vector machine (SVM) based pattern recognition algorithm is used that compares the search window including the detected faces with several reference sets (gender for faces and symbols for brand)

The processor and associated operator interface also allows a significant amount of flexibility where multiple targets, with a variety of associated physical characteristics and symbols are detected simultaneously. In addition the system can be connected to the internet to allow download of information and remote diagnostics.

The owner or manager of the system can then introduce the information into a database 606, then utilize any form of database processing such as OLAP (Online Analytical Processing) to interpret trends and patterns 607. Over time this development will enable networks to mine data across the network assets and create optimized pricing and planning models.

A network will then be able to create packages or reports 608 that include scaled benefits to clients based on their targets, also enable them to get close to 100% optimisation and still offer value for money based on more targeted audience impressions based on advertising packages that can be customised to clients requirements.

This permits networks to negotiate prices based on value and price, knowing at all times what their base inventory cost is.

The aforementioned technology facilitates the refinement and media planning and buying services based on OOH network audience data that has been collected over time. This data will reveal seasonal and geographic shifts in audience volumes and audience attributes. The benefit to this historical data is that trends can be identified. The trends recognized can then be commercially leveraged by advertisers using OOH displays. There is a lot of value in the international aggregation of OOH audience trends and this will be achieved in time to give local, regional, national and global advertisers the ability to identify key target audiences around the world at anyone one time and their migration patterns over time.

International media planning and buying agencies will for the first time have the ability to negotiate an international media plan with global media network companies. This will increase both their bargaining power during media cost negotiations, at the same time allowing them to derive more accountable metrics for their media effectiveness when reporting back to the marketing directors of the OOH advertisers.

It is predicted that the trend toward globalization will continue and the cultural barriers between contries will decrease, enabling a unified communication process that will be adopted and applied to capture the attention of a target consumer. This tool will enable the delivery of this media to the relevant tagets anywhere in the world.

It will also be readily apparent to persons skilled in the art that various modifications maybe made in the details of steps of the method, and features of the system, for selectively delivering advertising information from a continuous broadcast of media metrics as described above without departing from the scope or ambit of the present invention.

The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgement or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates before the filing date of this patent application.

Claims

What is claimed is:

1. A monitoring device for advertising, comprising: a camera located to record images of multiple targets within viewing distance of an advertisement; a computer for receiving images of targets from the camera and running software for image processing on those images of targets; the software comprising modules for background subtraction and for determining confirmed impressions from targets in an audience of the advertisement; and software for generating a report that includes an account of confirmed impressions.

2. The device of claim l, wherein: the software for image processing is adapted to distinguish confirmed impressions from "opportunity to see" targets.

3. The device of claim l, wherein: the software for image processing is adapted to identify objects associated with targets by comparing an isolated image of the object with stored information in a database.

4. The device of claim 1, wherein: the software for image processing is adapted to identify and categorise a target that is a person by isolating skin tones of the person.

5. The device of claim 1, wherein: the software for image processing is adapted to count to the total number of persons in an audience; and identify and categorise a target that is a person by isolating skin tones of a head portion of the person and use that information to count confirmed impressions; and then compare that count to the total number in the audience.

6. The device of claim l, wherein: the camera is located so as to, and adapted to record an advertising identifier on an advertisement and the software further comprises a module for recognising the advertising identifier.

7. A monitoring device for advertising, comprising: a camera located to record images of an audience comprising targets within viewing distance of an advertisement; a computer for receiving images of the audience from the camera and running software for image processing on those images; the software comprising modules for decomposing the images and for determining confirmed impressions and demographic data from targets in the audience of the advertisement; and software for generating a report that includes an account of confirmed impressions and demographic information about targets in the audience.

8. The device of claim 7, wherein: the software for image processing is adapted to distinguish confirmed impressions from "opportunity to see" targets.

9. The device of claim 7, wherein: the software for image processing is adapted to identify objects associated with targets by comparing an isolated sub-image of the object with stored information.

10. The device of claim 7, wherein: the software for image processing is adapted to identify and categorise a target that is a person by isolating skin tones of the person.

11. The device of claim 7, wherein: the software for image processing is adapted to count to the total number of persons in an audience; and identify and categorise a target that is a person by isolating skin tones of a head portion of the person and use that information to count confirmed impressions; and then compare that count to the total number in the audience.

12. The device of claim 7, wherein: the camera is located so as to, and adapted to record an advertising identifier on an advertisement and the software further comprises a module for recognising the advertising identifier.