CN100573525C - Being used for coloured image represents and the method and apparatus of retrieving - Google Patents

Being used for coloured image represents and the method and apparatus of retrieving Download PDF

Info

Publication number
CN100573525C
CN100573525C CNB2007101008890A CN200710100889A CN100573525C CN 100573525 C CN100573525 C CN 100573525C CN B2007101008890 A CNB2007101008890 A CN B2007101008890A CN 200710100889 A CN200710100889 A CN 200710100889A CN 100573525 C CN100573525 C CN 100573525C
Authority
CN
China
Prior art keywords
image
color
colored
descriptor
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
CNB2007101008890A
Other languages
Chinese (zh)
Other versions
CN101038595A (en
Inventor
L·茨普林斯基
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rakuten Group Inc
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Publication of CN101038595A publication Critical patent/CN101038595A/en
Application granted granted Critical
Publication of CN100573525C publication Critical patent/CN100573525C/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/46—Measurement of colour; Colour measuring devices, e.g. colorimeters
    • G01J3/50—Measurement of colour; Colour measuring devices, e.g. colorimeters using electric radiation detectors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5862—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/46—Measurement of colour; Colour measuring devices, e.g. colorimeters
    • G01J3/463—Colour matching
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99931—Database or file accessing
    • Y10S707/99933—Query processing, i.e. searching
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99931—Database or file accessing
    • Y10S707/99933—Query processing, i.e. searching
    • Y10S707/99936—Pattern matching access
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99941—Database schema or data structure
    • Y10S707/99944—Object-oriented database structure
    • Y10S707/99945—Object-oriented database structure processing

Abstract

The present invention relates to a kind of method for expressing of coloured image, this method comprises the selection image area, select a kind of or more colors representative color as this district, and, for district, relate to the parameter of the color distribution that representative color is relevant separately and to utilize described parameter be this image area derivation descriptor for each representative color calculates at least two with two kinds or more representative colors.

Description

Being used for coloured image represents and the method and apparatus of retrieving
The application is that application number is dividing an application of 00809556.6 patented claim.
Technical field
The present invention relates to be used for being retrieved as the method and apparatus that coloured image or image area are represented that carries out of purpose, and the method and apparatus that is used to retrieve coloured image or image area.
Background technology
Knownly be used to recover retrieval technique from the rest image and the video image of for example multimedia database based on picture material.Many feature of image comprise color, texture, side information, shape and motion, are adopted by this technology.This The Application of Technology comprises internet search engine, interactive television, teletherapy and tele-shopping.
For the image from image data base is recovered, comprise descriptor presentation video or image area based on color in the image with descriptor.Known many dissimilar descriptors based on color, comprise the average color of image area, based on the statistical moment of change color in the image area scope, the color of representative color such as maximum region in the overlay image district, and color histogram, histogram wherein is by the pixel in each chromatic zones in one group of default color is counted derivation.
A kind of known content-based image recovery system is a QBIC (query by image content: query by image content) (see US 5579471, mpeg file MPEGM4582/P165: be used for the color descriptor of MPEG-7, IBM Corporation Almaden research centre).In the pattern in the operational mode of this system, each image in the database is divided into piece.Each piece is divided into subclass with similar color and the subclass of selecting this maximum.The average color of selected subclass is elected to be the representative color of piece separately.This representative color information is stored in the database.Can carry out inquiry in the database by selecting query image.Adopt above-mentioned identical mode to derive representative color information at this query image.Utilize a kind of algorithm that this Query Information and the information that is stored in the image in this database are compared to determine immediate coupling then.
Mpeg file M4582/P437 and US 5586197 disclose a kind of similar methods, but what adopt is a kind of method that more flexibly image is divided into piece and different movement images.In another modification described in the mpeg file M4582/P576: the color showing of sensation target, adopt a single value to each color in the two kinds of representative colors in every district.
Some graphical representation methods based on color histogram have obtained exploitation, such as mpeg file M4582/P76: a kind of color descriptor at MPEG-7: the color histogram of variable square.Other technology has adopted the descriptive statistics of color distribution in the image area.For example, mpeg file M4582/P549: utilize the color descriptor that the pictorial information in subarea is estimated in the video sequence, disclose a kind of technology, this technology is divided into image high entropy and low entropy district and every class one zone is calculated the color distribution characteristics.Mpeg file M4852/P319:MPEG-7 color descriptor suggestion is described as the descriptor of image area adopting a mean value and covariance value.
All said methods all have important disadvantages.Certain methods precision height in them specifically refers to the color histogram technology, but needs a large amount of relatively storage and processing time.Other method has high storage and counting yield such as employing methods a kind of or two kinds of representative colors, but precision is not high enough.These descriptive statistics symbols are traded off between this two classes technology, but they all lack dirigibility, pixel color is carried out wide variation in a district situation particularly ought occur.
Summary of the invention
The invention provides and a kind ofly be similar to the method that color distribution is come presentation video with the descriptor in deduced image district by utilizing some components to distribute, each this component distribution is corresponding to a kind of representative color in the image area.
The present invention also provides a kind of method of utilizing this descriptor retrieving images.
The present invention also provides a computer program to be used to implement the medium of described method and this computer program of computer-readable storage.This computer-readable medium can be separable medium such as floppy disk or CD-ROM or such as the storer of RAM.
Description of drawings
With reference to the following drawings one embodiment of the invention are described:
Fig. 1 is the block scheme according to the system of one embodiment of the invention;
Fig. 2 is the process flow diagram of first search method; With
Fig. 3 is the process flow diagram of second search method.
Embodiment
Fig. 1 shows the system according to one embodiment of the invention.This system comprises display unit 4 monitor and to the indicating device 6 of control module 2 input instructions mouse for example for example that control module 2 for example is used for the computing machine of control system operation, is connected with control module 2 and is used to show the output that comprises image and literal.This system also comprises the descriptor database 10 of the image data base 8 and the memory descriptor information of the digital version of storing a plurality of video sequences, and this will be hereinafter be described in more detail in the image that appears in the storing image data storehouse 8 each.Image data base 8 all is connected with control module 2 with each database in the descriptor database 10.This system also comprises a search engine 12, and search engine 12 is computer programs of controlled unit 2 controls and operates on the descriptor database 10.
In this embodiment, unit of system a single position for example image library be provided, system unit is by permalink on this position.
The descriptor of descriptor database 10 store storage all images in image data base.More specifically, in this embodiment, descriptor database 10 comprises at each the descriptor in a plurality of districts of each image.Derivation to this descriptor will be described hereinafter.
The rectangular block that each image in the database 8 is divided into some nonoverlapping pixels.Then at each piece, the color by selecting preset number and the number of pixels in the piece of each color counted derive color histogram.
The color histogram that is obtained has been showed the color distribution of this piece scope interior pixel.Usually this district has a kind of or more dominant color, and this histogram has the peak corresponding to these colors.
Descriptor to these pieces is based on the dominant color of confirming from this histogram.Descriptor at each piece comprises following element:
(1) number of dominant color, n is called as the degree of descriptor, wherein n 〉=1; And for each dominant color:
(2) (a) weighting of the relative conspicuousness of the separately dominant color of expression in the piece.Here, this weighting is the ratio of the sum of pixel in number and the piece of pixel in the piece of relevant colors.
(b) mean value, m = m x m y m z
X wherein, y and z are the indexed color components, for example the red, green and blue color component in the RGB color space.Here, this mean value is corresponding to the color component of dominant color separately
(c) covariance matrix C = c xx c xy c xz c yx c yy c yz c zx c zy c zz
C wherein IjVariable and the c of expression color component i IjCovariance between expression component i and the component j.This covariance matrix is symmetric matrix (c Ij=c Ji), so only need store 6 numbers.
In carrying out the above-mentioned process of obtaining this descriptor, this color distribution is used as n different sub-distribution process, and wherein n is the number of dominant color, and it is the center as on average that each height distributes with dominant color separately.The scope of this a little distribution can be overlapping well, and adopt scope that suitable algorithm determines each distribution to calculate weighting, average and covariance matrix as understood by one of ordinary skill in the art.The method of an estimation descriptor component is that the Gaussian function (Gaussian function) that will be positioned at the histogram peak center carries out match with histogram, and this match is to be undertaken by the actual histogram counting is reduced to minimum with the difference of the value of estimating from the mixing of Gaussian function.
This descriptor database 10 is for being stored in descriptor as hereinbefore defined of each piece storage of each image in the image data base 8.The expression of the color distribution in each piece scope of employing foregoing description symbol structure comprises a large amount of descriptors, needs less storage space but for example compare with full histogram information.
As an example, three peaks corresponding to three kinds of dominant color can be arranged for the color histogram of a concrete piece.Distribute the histogram color distribution analyzed and cause producing a descriptor as the son of three kinds of colors and comprise the number that indicates dominant color, three weightings, three mean vector of corresponding color vector at three peaks and the covariance matrix of three correspondences.
By being stored in the descriptor in the descriptor database, utilize native system in image data base, to carry out image retrieval.The present embodiment provides two kinds of search methods: based on the search method of solid color with based on the retrieval in zone.
With reference to process flow diagram shown in Figure 2 search method based on solid color is described.
In this retrieval based on solid color, the user uses indicating device 6 and is displayed on the color that will retrieve by selection such as the menu of toning wheel or palette on the display unit 4 and comes input inquiry (step 102).Control module 2 obtains corresponding color vector at this inquiry color then, and this color vector has the component at the color component separately of this inquiry color, i.e. red, green and blue component (step 104).
Control module 2 utilizes search engine 12 retrieving images in the image library 8 that comprises this inquiry color then.This search engine 12 utilizes this inquiry color vector and implements matching process (step 106) at the descriptor of image block in descriptor database 10.
Implement this matching process by adopting following formula to calculate matching value M.
M = exp [ - 1 2 ( q - m ) T C - 1 ( q - m ) ]
Wherein q is this query vector.Utilization is calculated matching value at each value of m in the descriptor of this piece and C for each dominant color in each piece.So,, obtained n matching value for the descriptor of a n degree.
This matching value can be considered to the probability function value corresponding to each color distribution in the piece at the some place that is defined by the inquiry color value, and this probability density function is modeled as a Gaussian function.
To a given descriptor, matching value M is big more, and then this corresponding blocks and selected color relation are approaching more.
When to each descriptor computation in the database 10 during matching value, this search engine 12 begins this result is sorted according to the size of the M M value from maximum, and any degree is only considered maximum M value (step 108) greater than 1 descriptor.
This control module 2 obtains the result of this matching process from search engine 12, and from this image data base the K of a preset number is had and tap into coupling most, promptly recovers corresponding to K the image with the highest M value.Then these images are presented on the display unit 4 (step 110).The device of control module 2 defines how many immediate couplings to be displayed on this display unit.The user can change this number.
Will be understood that from description above based on the retrieval of solid color the image from database 8 is recovered, these images have a piece that comprises a kind of dominant color, this dominant color is identical or approaching with the color that the user begins to select.
Will the retrieval based on the district be described with reference to Fig. 3.
In the retrieval based on the district, control module 2 shows one group of default retrieving images (step 202) from image data base on display unit 4.This retrieving images can be determined by the device of control module fully, perhaps depend on the requirement of user's input.For example, in the system of bigger support based on keyboard retrieval, the user may input word " leaves (leaf) " this can cause producing the image of one group of default description leaf as image based on the color retrieval.
Each image in this retrieving images becomes the form of piece to show image division by utilizing grid, and these pieces are corresponding to the piece of deriving descriptor.Then, the user utilizes indicating device 6 to select a piece (step 204) on the image of shown interested color distribution.
2 pairs of descriptors at the image of selecting from descriptor database 10 of control module recover and accord with (step 206) used as query specification then.Because from image data base 8, adopted retrieving images, so this descriptor can be obtained.This search engine utilizes adaptation function that query specification symbol and other descriptor that is stored in the descriptor database are relatively retrieved (step 208) then.
For having at a kind of mean value m in this dominant color aWith the square matrix C of association aQuery specification symbol and have at a kind of mean value m in this dominant color bWith the square matrix C of association bOther descriptor, adaptation function is defined as:
m s ( a , b ) = ∫ exp [ - 1 2 ( q - m a ) T C a - 1 ( q - m a ) ] exp [ - 1 2 ( q - m b ) T C b - 1 ( q - m b ) ] dq
Wherein q be one be similar to the trivector of color vector and integration wherein be carry out in from (0,0,0) to the scope of (255,255,255) wherein 255 are maximal values of a color component.This limit of integration will depend on color coordinates system and employed expression in other embodiment.
This is equivalent to, and corresponding color at this image block is distributed that formal model with Gaussian function turns into is the probability mass function, and determines their overlapping degree, perhaps in other words determines the similarity between them.Above-mentioned result of calculation is big more, and then these corresponding color distribution are just approaching more.In this case, this function is determined the distribution of color and the sub overlapping degree that distributes of the color in the image of being stored in the query image piece.
Be used for a descriptor and other full coupling descriptor that mates are defined as:
m f = Σ i , j v i w j m s ( i , j )
Wherein v and w are the weightings that son distributes, and this summation is to carry out in all sub-distribution ranges in two districts.
So,, calculate one about matching value from each dominant color in the descriptor of descriptor database 10 for each dominant color that in the descriptor of query image piece, is described.To the matching value that obtains be weighted and then the summation provide one corresponding to m fFinal matching value.
As mentioned above, to the full matching value of all descriptor computation in the database that accords with about query specification.As in retrieval based on solid color, this result is sorted (step 210), and promptly show K image of approaching coupling for the user will have the highest matching value, be presented at (step 212) on the display unit.
By selecting an image area to carry out another time retrieval iteration in the image of finding in the retrieval formerly.
Can utilize and be different from aforesaid other similar and estimate and mate.Provided the another one example below.
For a pair of descriptor F at two districts 1And F 2, the similar D of estimating is defined as follows:
D ( F 1 , F 2 ) = Σ j = 1 N 1 Σ i = 1 N 1 p 1 i p 1 j f 1 i 1 j + Σ i = 1 N 2 Σ j = 1 N 2 p 2 i p 2 j f 2 i 2 j - Σ i = 1 N 1 Σ j = 1 N 2 2 p 1 i p 2 j f 1 i 2 j ,
Wherein
f xiyj = 1 2 π v xiyjl v xiyju v xryjv exp [ - ( c xiyjl v xiyjl + c xiyju v xiyju + c xiyjv v xiyjv ) / 2 ]
With
c xryjl＝(c xil-c yjl) 2，v xiyjl＝(v xil+v yjl)，
c xiyju＝(c xiu-c yju) 2，v xiyju＝(v xiu+v yju)，
c xryjv＝(c xiv-c yjv) 2，v xiyjv＝(v xiv+v yjv).
Here, i and j indicate color separately;
X and y indication descriptor;
N 1It is the number of representative color in first descriptor;
N 2It is the number of representative color in second descriptor;
P 1iBe i weighting in first descriptor;
P 2jBe j weighting in second descriptor;
1, u and v represent color component, for example are the red, green and blue color component in this instantiation; And
C and v are respectively dominant color value (mean value) and color variance, so c XijBe the 1st component of the i kind representative color value of x descriptor, and v XijBe the 1st component or the like of this variable of the i kind representative color of x descriptor.
Opposite with previously described adaptation function, to descriptor F 1And F 2, the D value is more little, then corresponding to descriptor F 1And F 2The district between coupling just approaching more.Correspondingly, according to incremental order the D value that is produced by aforesaid retrieving is sorted from minimum D value beginning.Not so change is considered different similarly to estimate, and is undertaken retrieving and matching process by aforesaid process is suitably revised basic identically.It should be noted that: this is similar estimates and has adopted this variable and do not need this association's square matrix.So comprise this variable at the descriptor in a district but do not need this association's square matrix.Correspondingly, compare with previous described descriptor and reduced memory requirement.
Can be provided for according to system of the present invention, for example image library.Another kind of situation, the distance of this database and system control unit can be far, and this database is connected with control module by interim connection such as telephone wire or network a such as internet.Can permanent storage or portable data storage medium such as, image and descriptor database are provided among CD-ROM or the DVD.
In the superincumbent description, color showing is described according to the red, green and blue color component.Certainly, can adopt other expression such as adopting tone, saturation degree and intensity, perhaps YUV coordinate system perhaps in the subclass of the color component of other color space, is for example only used the color harmony saturation degree in HIS.
Aforesaid embodiment of the present invention have adopted the descriptor that the rectangular block of image is derived.The subarea of other image is used as the base of descriptor.For example, can adopt the district of difformity and size.Another kind of situation can derive descriptor for the district corresponding to the image of target, for example, and automobile, house or people.Under any therein situation, can be all images or parts of images derivation descriptor.
In this retrieving, it is not to be used for importing a simple color inquiry or to select an image block that the user for example can use indicating device, by rotate a district describing image around it, this control module is derived a descriptor and is utilized it to retrieve by above-mentioned similar fashion for this district thus.And not to utilize the image that is stored in the image data base to cause retrieval, but for example adopt image reading apparatus or digital camera that image is input in the system.In order to retrieve under such situation, this system can at first automatically or by the user be defined as this image or image area derivation descriptor again.
Can adopt hardware or software to implement suitable aspect of the present invention.
In embodiment above, adopt Gaussian function that the branch quantum at each representative color is distributed and be similar to, and these average values of a function and covariance are used as descriptor value.But, can adopt other function or parameter that component is distributed and be similar to, for example, as basis function, descriptor is based on these functions such as sine and cosine functions in employing.

Claims (17)

1. method of representing coloured image comprises:
Select the dominant color of two or more colored conducts for the zone of image or described image;
Calculate and colored variance parameter for each dominant color about the relevant expression colour variance of each dominant color of described image with two or more dominant color or image-region; And
For each dominant color,, calculate the weighting parameters of the relative importance of the dominant color of representative in image or image-region according to number corresponding to the pixel of the dominant color in image or image-region; And
Use described colored variance parameter and described weighting parameters to draw the descriptor that is used for described image or image-region, described descriptor comprises described colored variance parameter and described weighting parameters.
2. the method as requiring in the claim 1 wherein selects the step of dominant color to comprise the color histogram that draws for image or image-region.
3. the method as requiring in the claim 2 wherein selects the step of dominant color to comprise the local peaking that is identified in the color histogram and select corresponding colored as dominant color.
4. as the method for requirement in the claim 3, wherein local peaking treats as mean value, and with respect to described mean value calculation variance.
5. the method as requiring in each of claim 1 to 4 wherein obtains described descriptor for a target in the described image.
6. the method as requiring in each of claim 1 to 4 also comprises:
Select image or image-region.
7. the method as requiring in each of claim 1 to 4, wherein weighting parameters is based on corresponding to the number of pixels of dominant color and the ratio of the number of pixels in image or the image-region.
8. the method for claim 7, wherein each weighting parameters is defined in the percentage of the pixel that has relevant dominant color value in image or the image-region.
9. method that is used for searching at the coloured image of data storage device comprises:
Import the inquiry relevant with the colour of image,
More described inquiry with by use adaptation function according to as the descriptor of the method image that draw, that be used to store of requirement in the claim 1, and
Select and show to be illustrated at least one image that mates closely between at least a portion of inquiry and image for its adaptation function.
10. the method as requiring in the claim 9, wherein input inquiry comprises the zone of selecting inquiry image or described image and is retrieved as described image or the resulting descriptor of image-region, and wherein adaptation function is used to inquire and the descriptor of the image that is used for being stored.
11. as the method that requires in claim 9 or the claim 10, wherein adaptation function calculates matching value M, wherein M=exp[(-1/2) (q-m) TC -1(q-m)],
Wherein q is the color vectors corresponding to inquiry, and m and C are the descriptor numerical value of representative for first and second central moments of the color distribution of dominant color.
12. as the method that requires in claim 9 or 10, wherein adaptation function calculates full matching value m f, m wherein f=∑ I, jv iw jm s(i, j),
M wherein s(i j) is matching value by the dominant color of subscript value i and j indication, v iAnd w jBe the weighting that corresponding son distributes,
m s(a，b)＝∫exp[(-1/2)(q-m a) TC a -1(q-m a)]exp[(-1/2)(q-m b) TC b -1(q-m b)]dq
And m wherein a, m b, C a, C bBe the descriptor numerical value of first and second central moments of the color distribution of the dominant color represented by a and b of representative, q is the trivector of color vector, and wherein the gamut to q carries out integration in the respective color space coordinates.
13. as the method that requires in the claim 9, wherein the input inquiry comprises the single colored numerical value of selection.
14. an equipment that is used to implement represent the method for coloured image comprises:
Selecting arrangement is used to select the dominant color of two or more colored conducts for the zone of image or described image;
First calculation element is used for calculating and colored variance parameter about the relevant expression colour variance of each dominant color of described image with two or more dominant color or image-region for each dominant color; And
Second calculation element for each dominant color, according to the number corresponding to the pixel of the dominant color in image or image-region, calculates the weighting parameters of the relative importance of the dominant color of representative in image or image-region; And
Deriving means is used to use described colored variance parameter and described weighting parameters to draw the descriptor that is used for described image or image-region, and described descriptor comprises described colored variance parameter and described weighting parameters.
15. an equipment that is used for implementing being used to searching in the method for the coloured image of data storage device comprises:
Input media is used to import the inquiry relevant with the colour of image,
Comparison means, be used for more described inquiry with by use adaptation function according to as the descriptor of the method image that draw, that be used to store of claim 1 requirement, and
Select and display device, be used to select and show be illustrated at least one image that mates closely between at least a portion of inquiry and image for its adaptation function.
16. a system that is used for presentation video comprises:
Detecting device is used for determining that at least two for the zone of image or described image are leading colored;
Controller, be used for for each leading colored, according to corresponding to the ratio of the number of leading colored pixel with the number of pixel in image or image-region, determine the leading colored weighting parameters of colored variance and representative, so that generate color descriptors according to the pixel in described image or image-region with respect to other colored importance in image or the image-region; And
Storer is used for described color descriptors is stored in storer as the description for described image or image-region.
17. a system that is used for searching image comprises:
Input media is used to import at least one image;
Detecting device is used for determining that at least two for the zone of described input picture or described input picture are leading colored;
Controller, be used for according to corresponding to the number of leading colored pixel and ratio at the number of the pixel of image or image-region, determine to dominate colored colored variance and represent each leading colored weighting parameters, so that according to the generation of the pixel in described image or image-region color descriptors with respect to other colored importance in image or the image-region for each;
Comparer, be used for comparison for the described color descriptors of described input picture be stored in the color descriptors of the image of storer so that find out the image that at least one mates basically; And
Display is used to show described at least one image that mates basically.
CNB2007101008890A 1999-04-29 2000-04-28 Being used for coloured image represents and the method and apparatus of retrieving Expired - Lifetime CN100573525C (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB9909961.6 1999-04-29
GB9909961A GB2349460B (en) 1999-04-29 1999-04-29 Method of representing colour images

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CNB008095566A Division CN1322457C (en) 1999-04-29 2000-04-28 Method and apparatus for representing and searching for colour images

Publications (2)

Publication Number Publication Date
CN101038595A CN101038595A (en) 2007-09-19
CN100573525C true CN100573525C (en) 2009-12-23

Family

ID=10852533

Family Applications (3)

Application Number Title Priority Date Filing Date
CNB2007101008903A Expired - Lifetime CN100573526C (en) 1999-04-29 2000-04-28 Being used for coloured image represents and the method and apparatus of retrieving
CNB008095566A Expired - Lifetime CN1322457C (en) 1999-04-29 2000-04-28 Method and apparatus for representing and searching for colour images
CNB2007101008890A Expired - Lifetime CN100573525C (en) 1999-04-29 2000-04-28 Being used for coloured image represents and the method and apparatus of retrieving

Family Applications Before (2)

Application Number Title Priority Date Filing Date
CNB2007101008903A Expired - Lifetime CN100573526C (en) 1999-04-29 2000-04-28 Being used for coloured image represents and the method and apparatus of retrieving
CNB008095566A Expired - Lifetime CN1322457C (en) 1999-04-29 2000-04-28 Method and apparatus for representing and searching for colour images

Country Status (9)

Country Link
US (3) US6801657B1 (en)
EP (3) EP1173827B1 (en)
JP (5) JP2000348179A (en)
KR (3) KR100860902B1 (en)
CN (3) CN100573526C (en)
AU (1) AU4767100A (en)
DE (1) DE60011603T2 (en)
GB (1) GB2349460B (en)
WO (1) WO2000067203A1 (en)

Families Citing this family (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7417640B1 (en) 1999-01-29 2008-08-26 Lg Electronics Inc. Method for dominant color setting of video region and data structure and method of confidence measure extraction
GB2349460B (en) * 1999-04-29 2002-11-27 Mitsubishi Electric Inf Tech Method of representing colour images
US7212667B1 (en) * 1999-05-17 2007-05-01 Samsung Electronics Co., Ltd. Color image processing method for indexing an image using a lattice structure
JP2001338284A (en) * 2000-05-25 2001-12-07 Matsushita Electric Ind Co Ltd Method, device and system for image processing
US7899243B2 (en) 2000-11-06 2011-03-01 Evryx Technologies, Inc. Image capture and identification system and process
US8224078B2 (en) 2000-11-06 2012-07-17 Nant Holdings Ip, Llc Image capture and identification system and process
US7565008B2 (en) 2000-11-06 2009-07-21 Evryx Technologies, Inc. Data capture and identification system and process
US7680324B2 (en) 2000-11-06 2010-03-16 Evryx Technologies, Inc. Use of image-derived information as search criteria for internet and other search engines
US9310892B2 (en) 2000-11-06 2016-04-12 Nant Holdings Ip, Llc Object information derived from object images
KR100788643B1 (en) * 2001-01-09 2007-12-26 삼성전자주식회사 Searching method of image based on combination of color and texture
GB0103965D0 (en) * 2001-02-17 2001-04-04 Univ Nottingham Image and image content processing,representation and analysis for image matching,indexing or retrieval and database management
KR100451649B1 (en) * 2001-03-26 2004-10-08 엘지전자 주식회사 Image search system and method
EP1302865A1 (en) * 2001-10-10 2003-04-16 Mitsubishi Electric Information Technology Centre Europe B.V. Method and apparatus for searching for and retrieving colour images
FR2842979B1 (en) * 2002-07-24 2004-10-08 Thomson Licensing Sa METHOD AND DEVICE FOR PROCESSING DIGITAL DATA
EP1418546A1 (en) * 2002-11-07 2004-05-12 Mitsubishi Electric Information Technology Centre Europe B.V. Method and apparatus for representing moving objects in a sequence of images
US7292365B2 (en) * 2003-01-15 2007-11-06 Xerox Corporation Methods and systems for determining distribution mean level without histogram measurement
US7263220B2 (en) * 2003-02-28 2007-08-28 Eastman Kodak Company Method for detecting color objects in digital images
US20040179735A1 (en) * 2003-03-13 2004-09-16 Aruna Kumar Method and apparatus for characterizing objects within an image
US7184577B2 (en) * 2003-03-14 2007-02-27 Intelitrac, Inc. Image indexing search system and method
EP2293250B1 (en) * 2003-07-04 2012-05-09 Mitsubishi Electric Information Technology Centre Europe B.V. Method and apparatus for representing a group of images
US7624123B2 (en) * 2004-02-26 2009-11-24 Ati Technologies, Inc. Image processing system and method
US7983835B2 (en) 2004-11-03 2011-07-19 Lagassey Paul J Modular intelligent transportation system
US7813552B2 (en) 2004-09-23 2010-10-12 Mitsubishi Denki Kabushiki Kaisha Methods of representing and analysing images
US7840081B2 (en) 2004-09-23 2010-11-23 Mitsubishi Denki Kabushiki Kaisha Methods of representing and analysing images
GB2418555A (en) * 2004-09-23 2006-03-29 Mitsubishi Electric Inf Tech Representing an image using descriptors based on colour information
GB2418556A (en) * 2004-09-23 2006-03-29 Mitsubishi Electric Inf Tech Representing an image using descriptors based on colour information
GB0428406D0 (en) * 2004-12-24 2005-02-02 Oxford Instr Superconductivity Cryostat assembly
US8176414B1 (en) 2005-09-30 2012-05-08 Google Inc. Document division method and system
GB2431797B (en) * 2005-10-31 2011-02-23 Sony Uk Ltd Image processing
US20080002855A1 (en) * 2006-07-03 2008-01-03 Barinder Singh Rai Recognizing An Unidentified Object Using Average Frame Color
EP2074590A2 (en) 2006-09-28 2009-07-01 Koninklijke Philips Electronics N.V. Content detection of an image comprising pixels
JP2010505320A (en) * 2006-09-28 2010-02-18 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Image content detection
EP2084625B1 (en) * 2006-10-19 2011-01-12 Koninklijke Philips Electronics N.V. Dominant color descriptors
CN101334780A (en) * 2007-06-25 2008-12-31 英特维数位科技股份有限公司 Figure image searching method, system and recording media for storing image metadata
KR100925913B1 (en) 2007-08-28 2009-11-09 현대자동차주식회사 Non-aqueous Polymer Electrolyte Membrane by Coated compounds as anion binding agents and Polymer Electrolyte Membrane Fuel Cell using it
CN101911120A (en) * 2008-01-17 2010-12-08 皇家飞利浦电子股份有限公司 Extracting colors
US20090231327A1 (en) * 2008-03-12 2009-09-17 Harris Corporation Method for visualization of point cloud data
US20090232355A1 (en) * 2008-03-12 2009-09-17 Harris Corporation Registration of 3d point cloud data using eigenanalysis
CN102037489B (en) * 2008-05-21 2013-08-21 Tp视觉控股有限公司 Image resolution enhancement
US8290252B2 (en) * 2008-08-28 2012-10-16 Microsoft Corporation Image-based backgrounds for images
US8155452B2 (en) 2008-10-08 2012-04-10 Harris Corporation Image registration using rotation tolerant correlation method
CN101777064A (en) * 2009-01-12 2010-07-14 鸿富锦精密工业（深圳）有限公司 Image searching system and method
US8179393B2 (en) * 2009-02-13 2012-05-15 Harris Corporation Fusion of a 2D electro-optical image and 3D point cloud data for scene interpretation and registration performance assessment
US8290305B2 (en) * 2009-02-13 2012-10-16 Harris Corporation Registration of 3D point cloud data to 2D electro-optical image data
US20100208981A1 (en) * 2009-02-13 2010-08-19 Harris Corporation Method for visualization of point cloud data based on scene content
CN101877137B (en) * 2009-04-30 2013-01-02 国际商业机器公司 Method for highlighting thematic element and system thereof
US8630489B2 (en) * 2009-05-05 2014-01-14 Microsoft Corporation Efficient image matching
US20110115812A1 (en) * 2009-11-13 2011-05-19 Harris Corporation Method for colorization of point cloud data based on radiometric imagery
US20110200249A1 (en) * 2010-02-17 2011-08-18 Harris Corporation Surface detection in images based on spatial data
JP2011221606A (en) * 2010-04-05 2011-11-04 Sony Corp Information processing method and graphical user interface
US9053562B1 (en) 2010-06-24 2015-06-09 Gregory S. Rabin Two dimensional to three dimensional moving image converter
CN102129693B (en) * 2011-03-15 2012-07-25 清华大学 Image vision significance calculation method based on color histogram and global contrast
KR101767269B1 (en) * 2011-04-25 2017-08-10 한국전자통신연구원 Apparatus and method for searching image
CN102819582B (en) * 2012-07-26 2014-10-08 华数传媒网络有限公司 Quick searching method for mass images
KR101370718B1 (en) * 2012-10-26 2014-03-06 한국과학기술원 Method and apparatus for 2d to 3d conversion using panorama image
US9992021B1 (en) 2013-03-14 2018-06-05 GoTenna, Inc. System and method for private and point-to-point communication between computing devices
US9158988B2 (en) * 2013-06-12 2015-10-13 Symbol Technclogies, LLC Method for detecting a plurality of instances of an object
US9305368B2 (en) * 2013-06-21 2016-04-05 Intel Corporation Compression and decompression of graphics data using pixel region bit values
US9465995B2 (en) * 2013-10-23 2016-10-11 Gracenote, Inc. Identifying video content via color-based fingerprint matching
US20150127377A1 (en) * 2013-11-07 2015-05-07 A.T. Still University Color matching for health management
CN103593458A (en) * 2013-11-21 2014-02-19 电子科技大学 Mass image searching system based on color features and inverted indexes
CN106157334B (en) * 2015-04-14 2019-11-15 北京智谷睿拓技术服务有限公司 Information processing method and equipment
US10387991B2 (en) 2016-07-01 2019-08-20 Intel Corporation Method and apparatus for frame buffer compression
US10916333B1 (en) * 2017-06-26 2021-02-09 Amazon Technologies, Inc. Artificial intelligence system for enhancing data sets used for training machine learning-based classifiers
US11216505B2 (en) * 2019-09-05 2022-01-04 Adobe Inc. Multi-resolution color-based image search
US11341759B2 (en) * 2020-03-31 2022-05-24 Capital One Services, Llc Image classification using color profiles
US11887217B2 (en) 2020-10-26 2024-01-30 Adobe Inc. Text editing of digital images
CN112699259B (en) * 2021-03-25 2021-07-13 北京达佳互联信息技术有限公司 Information display method and device, electronic equipment and computer readable storage medium

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5579471A (en) * 1992-11-09 1996-11-26 International Business Machines Corporation Image query system and method
JP3311077B2 (en) * 1993-05-06 2002-08-05 松下電器産業株式会社 Image retrieval device
JP3444924B2 (en) * 1993-06-29 2003-09-08 株式会社東芝 Empty area detection device
JP3419415B2 (en) * 1993-07-07 2003-06-23 日本電信電話株式会社 Video feature processing method
JP3234064B2 (en) * 1993-09-02 2001-12-04 キヤノン株式会社 Image retrieval method and apparatus
DE59403478D1 (en) * 1993-09-27 1997-08-28 Siemens Ag METHOD FOR SEGMENTING DIGITAL COLOR IMAGES
JPH07146871A (en) * 1993-11-24 1995-06-06 Hitachi Ltd Still picture retrieval device and method therefor
DE69535098T2 (en) * 1994-11-15 2007-04-19 Canon K.K. Method and apparatus for searching images in a database
JP3703164B2 (en) * 1995-05-10 2005-10-05 キヤノン株式会社 Pattern recognition method and apparatus
JPH0981591A (en) * 1995-09-14 1997-03-28 Dainippon Screen Mfg Co Ltd Retrieval key registering method for picture data base
JPH09106458A (en) * 1995-10-12 1997-04-22 Dainippon Printing Co Ltd Image processing method
US5641596A (en) * 1995-12-05 1997-06-24 Eastman Kodak Company Adjusting film grain properties in digital images
US5873080A (en) * 1996-09-20 1999-02-16 International Business Machines Corporation Using multiple search engines to search multimedia data
US5899999A (en) * 1996-10-16 1999-05-04 Microsoft Corporation Iterative convolution filter particularly suited for use in an image classification and retrieval system
US5819288A (en) * 1996-10-16 1998-10-06 Microsoft Corporation Statistically based image group descriptor particularly suited for use in an image classification and retrieval system
US5852823A (en) * 1996-10-16 1998-12-22 Microsoft Image classification and retrieval system using a query-by-example paradigm
JP3198980B2 (en) * 1996-10-22 2001-08-13 松下電器産業株式会社 Image display device and moving image search system
US6081276A (en) * 1996-11-14 2000-06-27 International Business Machines Corporation Method and apparatus for creating a color name dictionary and for querying an image by color name
JPH10149441A (en) * 1996-11-20 1998-06-02 Casio Comput Co Ltd Picture processing method and device therefor
JP3459950B2 (en) * 1997-04-30 2003-10-27 学校法人立命館 Face detection and face tracking method and apparatus
KR100295225B1 (en) * 1997-07-31 2001-07-12 윤종용 Apparatus and method for checking video information in computer system
JP3747589B2 (en) * 1997-09-17 2006-02-22 コニカミノルタビジネステクノロジーズ株式会社 Image feature amount comparison device and recording medium storing image feature amount comparison program
WO1999017250A1 (en) * 1997-10-01 1999-04-08 Island Graphics Corporation Image comparing system
US6026411A (en) * 1997-11-06 2000-02-15 International Business Machines Corporation Method, apparatus, and computer program product for generating an image index and for internet searching and querying by image colors
US5949904A (en) * 1997-11-06 1999-09-07 International Business Machines Corporation Method, apparatus and computer program product for querying by image colors using JPEG image format
US6163622A (en) * 1997-12-18 2000-12-19 U.S. Philips Corporation Image retrieval system
US6335985B1 (en) * 1998-01-07 2002-01-01 Kabushiki Kaisha Toshiba Object extraction apparatus
DE69942901D1 (en) * 1998-04-02 2010-12-16 Canon Kk Device and method for searching images
US6345274B1 (en) * 1998-06-29 2002-02-05 Eastman Kodak Company Method and computer program product for subjective image content similarity-based retrieval
US6516100B1 (en) 1998-10-29 2003-02-04 Sharp Laboratories Of America, Inc. Method for image characterization using color and texture statistics with embedded spatial information
US6411953B1 (en) * 1999-01-25 2002-06-25 Lucent Technologies Inc. Retrieval and matching of color patterns based on a predetermined vocabulary and grammar
US6373979B1 (en) * 1999-01-29 2002-04-16 Lg Electronics, Inc. System and method for determining a level of similarity among more than one image and a segmented data structure for enabling such determination
US6593936B1 (en) * 1999-02-01 2003-07-15 At&T Corp. Synthetic audiovisual description scheme, method and system for MPEG-7
US6774917B1 (en) * 1999-03-11 2004-08-10 Fuji Xerox Co., Ltd. Methods and apparatuses for interactive similarity searching, retrieval, and browsing of video
US6526169B1 (en) * 1999-03-15 2003-02-25 Grass Valley (Us), Inc. Histogram-based segmentation of objects from a video signal via color moments
GB2349460B (en) * 1999-04-29 2002-11-27 Mitsubishi Electric Inf Tech Method of representing colour images
US6477272B1 (en) * 1999-06-18 2002-11-05 Microsoft Corporation Object recognition with co-occurrence histograms and false alarm probability analysis for choosing optimal object recognition process parameters

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Region-Based Image Querying. Chad，Carson，Serge，Belongie，，Hayit，Greenspan，Jitendra，，，，Malik.1997 IEEE. 1997
Region-Based Image Querying. Chad,Carson,Serge,Belongie,Hayit,Greenspan,Jitendra,Malik.1997 IEEE. 1997 *
Vector Angular Distance Measure for Indexing andRetrieval of Color. D.Androutsos,K.N.Plataniotis, A.N. Venetsanopoulos.Part of the IS&T/SPIE Conference on Storage and Retrieval for Image and Video Databases Ⅶ,Vol.3656 . 1999

Also Published As

Publication number Publication date
CN1322457C (en) 2007-06-20
EP1173827A1 (en) 2002-01-23
JP5123998B2 (en) 2013-01-23
JP5121972B2 (en) 2013-01-16
JP2002543539A (en) 2002-12-17
DE60011603T2 (en) 2005-07-14
US7636094B2 (en) 2009-12-22
KR20070086677A (en) 2007-08-27
DE60011603D1 (en) 2004-07-22
JP2000348179A (en) 2000-12-15
US6801657B1 (en) 2004-10-05
EP1173827B1 (en) 2004-06-16
EP1496473A3 (en) 2007-03-07
GB2349460B (en) 2002-11-27
EP1496473A2 (en) 2005-01-12
JP2011146078A (en) 2011-07-28
JP5236785B2 (en) 2013-07-17
EP1445733A2 (en) 2004-08-11
AU4767100A (en) 2000-11-17
KR20010113903A (en) 2001-12-28
EP1445733A3 (en) 2007-03-07
JP2011192316A (en) 2011-09-29
KR100860902B1 (en) 2008-09-29
CN101038595A (en) 2007-09-19
GB2349460A (en) 2000-11-01
CN1358296A (en) 2002-07-10
KR20070008729A (en) 2007-01-17
US7015931B1 (en) 2006-03-21
KR100863631B1 (en) 2008-10-15
JP2010262673A (en) 2010-11-18
GB9909961D0 (en) 1999-06-30
CN101042708A (en) 2007-09-26
US20060072829A1 (en) 2006-04-06
WO2000067203A1 (en) 2000-11-09
KR100809798B1 (en) 2008-03-04
CN100573526C (en) 2009-12-23

Similar Documents

Publication Publication Date Title
CN100573525C (en) Being used for coloured image represents and the method and apparatus of retrieving
Gupta et al. Visual information retrieval
US7392233B2 (en) Image searching system, image searching method, and a recording medium storing an image searching program
Yeung et al. Video browsing using clustering and scene transitions on compressed sequences
Schaefer A next generation browsing environment for large image repositories
US6522782B2 (en) Image and text searching techniques
US20020164078A1 (en) Information retrieving system and method
Park et al. Color image retrieval using hybrid graph representation
US6792434B2 (en) Content-based visualization and user-modeling for interactive browsing and retrieval in multimedia databases
Yoo et al. Expert system for color image retrieval
WO2004031991A1 (en) Reduction of search ambiguity with multiple media references
Ali Content-based image classification and retrieval: A rule-based system using rough sets framework
JPH10340272A (en) Simular picture retrieval device/method
Glotin et al. Efficient image concept indexing by harmonic & arithmetic profiles entropy
Pavithra et al. Retrieval of homogeneous images using appropriate color space selection
Bhatt et al. WAGBIR: Wavelet and Gabor Based Image Retrieval Technique for the Spatial-Color and Texture Feature Extraction Using BPN in Multimedia Database
JPH05242161A (en) Image retrieval device
Al-Oraiqat et al. A modified image comparison algorithm using histogram features
Lindley The FRAMES processing model for the synthesis of dynamic virtual video sequences
Zhao et al. Evaluation on color spaces and distance measures for color histogram-based image retrieval
Gevers et al. Interactive Retrieval of Color Images
Chang et al. Supporting subjective image queries without seeding requirements: proposing test queries for Benchathlon
Androutsos et al. Excluding specified colors from image queries using a multidimensional query space
Schaefer et al. Visualisation of large image databases

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20190604

Address after: Tokyo, Japan, Japan

Patentee after: Rakuten Inc.

Address before: Tokyo, Japan, Japan

Patentee before: Mitsubishi Electric Corporation

TR01 Transfer of patent right
CX01 Expiry of patent term

Granted publication date: 20091223

CX01 Expiry of patent term