System and method for measuring similarity between a set of known temporal ...

(54) SYSTEM AND METHOD FOR MEASURING SIMILARITY BETWEEN A SET OF KNOWN TEMPORAL MEDIA SEGMENTS AND A ONE OR MORE TEMPORAL MEDIA STREAMS

(75) Inventors: Rudolf Maarten Bolle, Bedford Hills, NY (US); Arun Hampapur, White Plains, NY (US)

(73) Assignee: International Business Machines Corp., Armonk, NY (US)

( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days.

(21) Appl. No.: 09/496,604

(22) Filed: Feb. 2, 2000

(51) Int. CI.7 G06F 17/30

(52) U.S. CI 707/104.1; 707/102

(58) Field of Search 707/100, 102,

707/104.1; 345/723

(56) References Cited

U.S. PATENT DOCUMENTS

4,677,466 A 6/1987 Lert, Jr. et al 725/22

5,151,788 A 9/1992 Blum 348/725

5,225,904 A * 7/1993 Golin et al 271/18.1

5,343,251 A 8/1994 Nafeh 348/571

5,485,611 A * 1/1996 Astle 707/1

5,504,518 A 4/1996 Ellis et al 725/22

5,557,330 A * 9/1996 Astle 348/394.1

5,655,117 A * 8/1997 Goldberg et al 707/102

5,696,866 A 12/1997 Iggulden et al 386/46

5,708,477 A 1/1998 Forbes et al 318/552

5,751,280 A * 5/1998 Abbott et al 707/500.1

5,956,716 A * 9/1999 Kenner et al 707/10

6,065,050 A * 5/2000 DeMoney 709/219

6,246,803 Bl 6/2001 Gauch 382/276

6,360,234 B2 * 3/2002 Jain et al 700/500.1

This invention is a scaleable system to perform exact matching or similarity matching between a large store of reference temporal media sequences and a query target temporal media sequence. The system is not limited to finding exact matching media segments but also can find media segments that are similar. One kind of similarity between media segments is the similarity between a long commercial and a short commercial, where the short commercial is formed by sub-segments of the longer commercial. Another kind of similarity of two media segment is when they depict three-dimensional actions that are similar and imaged from similar viewpoints. Given a reference media segment, a multitude of features are computed in a consistent way from either predetermined or media contentdependent key intervals. These features are stored in segment index tables along with identifiers of the corresponding reference media segments. At search time, again, a multitude of features is computed from each key interval and now the features are used to search the segment index table for identifiers of stored media segments. If the key intervals are similar, many if not all, features computed from the key intervals point to the same reference media identifier.

26 Claims, 23 Drawing Sheets