CA2505904C - Method and system for ranking documents of a search result to improve diversity and information richness - Google Patents
Method and system for ranking documents of a search result to improve diversity and information richness Download PDFInfo
- Publication number
- CA2505904C CA2505904C CA2505904A CA2505904A CA2505904C CA 2505904 C CA2505904 C CA 2505904C CA 2505904 A CA2505904 A CA 2505904A CA 2505904 A CA2505904 A CA 2505904A CA 2505904 C CA2505904 C CA 2505904C
- Authority
- CA
- Canada
- Prior art keywords
- document
- documents
- affinity
- collection
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9536—Search customisation based on social or collaborative filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
Abstract
A method and system for ranking documents of search results based on information richness and diversity of topics. A ranking system determines the information richness of each document within a search result. The ranking system groups documents of a search result based on their relatedness, meaning that they are directed to similar topics. The ranking system ranks the documents to ensure that the highest ranking documents may include at least one document covering each topic, that is, one document from each of the groups. The ranking system selects the document from each group that has the highest information richness of the documents within the group. When the documents are presented to a user in rank order, the user will likely find on the first page of the search result documents that cover a variety of topics, rather than just a single popular topic.
Description
METHOD AND SYSTEM FOR RANKING DOCUMENTS OF A SEARCH RESULT
TO IMPROVE DIVERSITY AND INFORMATION RICHNESS
TECHNICAL FIELD
The described technology relates generally to ranking documents of a search result identified by a search request submitted to a search engine service.
BACKGROUND
Many search engine services, such as Google and Overture, provide for searching for information that is accessible via the Internet. These search engine services allow users to search for display pages, such as web pages, that may be of interest to users. After a user submits a search request that includes search terms, the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by "crawling" the web (i.e., the World Wide Web) to extract the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages. The keywords of any particular web page can be extracted using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on. The search engine service may calculate a relevance score that indicates how relevant each web page is to the search request based on closeness of each match, web page popularity (e.g., Google's PageRank), and so on. The search engine service then displays to the user the links to those web pages in an order that is based on their relevance. Search engines may more generally provide searching for information in any collection of documents.
For example, the collections of documents could include all U.S. patents, all federal court opinions, all archived documents of a company, and so on.
The highest ranking web pages of a search result provided by a web-based search engine service may be all directed to the same popular topic. For example, if a user submits a search request with the search term "Spielberg,"
then . 71570-1 the highest ranking web pages of the search result would likely be related to Steven Spielberg. If the user, however, was not interested in Steven Spielberg, but was instead interested in locating a home page for a mathematics professor with the same last name, then the ranking of the web pages would not be helpful to the user. Although the professor's home page may be included in the search result, the user may need to review several pages of links to the web pages of the search result to locate the link to the professor's home page. In general, it may be difficult for users to locate a desired document when it is not identified on the first page of a search result. Moreover, users can become frustrated when they have to page through multiple pages of a search result to find a document of interest.
It would be desirable to have a technique for ranking documents that would provide a greater diversity of topics within the highest ranking documents, and it would be further desirable to have each of such highest ranking documents be very rich in information content relating to its topic.
SUMMARY
According to one aspect of the present invention, there is provided a computing device for ranking documents of a search result, each document having words, comprising: a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising: receiving from a user a query; identifying documents as a search result for the received query; identifying keywords of the identified documents; for each pair of identified documents of the search result, calculating an affinity measurement indicating affinity that a document of the pair has to the other document of the pair, wherein affinity measurement is calculated based on the following:
aff(di,dj)=di=d, II
7/,11
TO IMPROVE DIVERSITY AND INFORMATION RICHNESS
TECHNICAL FIELD
The described technology relates generally to ranking documents of a search result identified by a search request submitted to a search engine service.
BACKGROUND
Many search engine services, such as Google and Overture, provide for searching for information that is accessible via the Internet. These search engine services allow users to search for display pages, such as web pages, that may be of interest to users. After a user submits a search request that includes search terms, the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by "crawling" the web (i.e., the World Wide Web) to extract the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages. The keywords of any particular web page can be extracted using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on. The search engine service may calculate a relevance score that indicates how relevant each web page is to the search request based on closeness of each match, web page popularity (e.g., Google's PageRank), and so on. The search engine service then displays to the user the links to those web pages in an order that is based on their relevance. Search engines may more generally provide searching for information in any collection of documents.
For example, the collections of documents could include all U.S. patents, all federal court opinions, all archived documents of a company, and so on.
The highest ranking web pages of a search result provided by a web-based search engine service may be all directed to the same popular topic. For example, if a user submits a search request with the search term "Spielberg,"
then . 71570-1 the highest ranking web pages of the search result would likely be related to Steven Spielberg. If the user, however, was not interested in Steven Spielberg, but was instead interested in locating a home page for a mathematics professor with the same last name, then the ranking of the web pages would not be helpful to the user. Although the professor's home page may be included in the search result, the user may need to review several pages of links to the web pages of the search result to locate the link to the professor's home page. In general, it may be difficult for users to locate a desired document when it is not identified on the first page of a search result. Moreover, users can become frustrated when they have to page through multiple pages of a search result to find a document of interest.
It would be desirable to have a technique for ranking documents that would provide a greater diversity of topics within the highest ranking documents, and it would be further desirable to have each of such highest ranking documents be very rich in information content relating to its topic.
SUMMARY
According to one aspect of the present invention, there is provided a computing device for ranking documents of a search result, each document having words, comprising: a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising: receiving from a user a query; identifying documents as a search result for the received query; identifying keywords of the identified documents; for each pair of identified documents of the search result, calculating an affinity measurement indicating affinity that a document of the pair has to the other document of the pair, wherein affinity measurement is calculated based on the following:
aff(di,dj)=di=d, II
7/,11
2 . 71570-1 where aff(di, di) is the affinity measurement of document di to document di, document di represents the document of the pair and document ¨
di represents the other document of the pair, d, represents a vector of document di, di represents a vector of document dj, and 11 ci,11 represents the length of vector di, each vector having at least one entry for at least one identified keyword of the corresponding document; for each identified document of the search result, calculating information richness for the identified document based on an element of a normalized affinity matrix that is derived from an affinity matrix, wherein the information richness is calculated based on the following:
InfoRich(d,)= E InfoRich(c 1 ) = ic ji au j#1 where InfoRich(di) is the information richness of document di, document di represents the identified document of the search result and document dj represents another document in the search result, and /14 is an element of the normalized affinity matrix, wherein the affinity matrix containing the calculated affinity measurement of the identified document and the another document in the search result; and displaying indications of documents of the search result ranked based on the calculated information richnesses of the documents of the search result.
According to another aspect of the present invention, there is provided a method in a computer system with a processor and a memory for calculating information richness of a document within a collection of documents, the documents in the collection having words, the method comprising:
identifying by the processor an affinity each document in the collection has to another document in the collection, wherein the affinity is identified for each pair of documents with one document of the pair being the document and the other document of the pair being another document in the collection of documents, affinity indicating to what extent the information of one document is subsumed by the information of another document; determining by the processor information 2a , 71570-1 richness for each document in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix; storing in the memory indications of the determined information richnesses of the documents in the collection; and ranking the documents based on the stored indications of information richnesses, wherein the determined information richness for each document is defined as InfoRich(d,)= E InfoRich(d ) = 1171 all J*1 where InfoRich(di) is the information richness of document di, document d, represents the document and document di represents another document of the collection of documents, and M is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the document to the another documents in the collection, wherein the documents in the collection are web pages.
According to still another aspect of the present invention, there is provided a computing device for calculating information richness of a document within a collection of web pages, the web pages having words, comprising: a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising: identifying an affinity of each web page in the collection has to the another web page in the collection, wherein the affinity is identified for each pair of web pages with one web page of the pair being the web page and the other web page of the pair being another web page in the collection of web pages, affinity indicating to what extent the information of one web page is subsumed by the information of another web page;
determining information richness for each web page in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix;
storing in the memory indications of the determined information richnesses of the web pages in the collection; and ranking the web pages based on the stored indications of information richnesses, wherein the determined information richness for each web page is defined as 2b InfoRich( d, )= InfoRich( d ) = ICI ji all j#1 where InfoRich(d) is the information richness of web page dõ web page di represents the web page and web page di represents another web page of the collection of web pages, and /17/,, is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the web page to the another web page in the collection.
A system ranks documents of search results based on information richness and diversity of topics. A ranking system groups documents of a search result based on their relatedness, meaning that they are directed to similar topics.
The ranking system ranks the documents to ensure that the highest ranking documents include at least one document covering each topic. The ranking system then selects the document from each group that has the highest information richness of the documents within the group as one of the highest ranking documents.
2c . 71570-1 BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a diagram that illustrates an affinity graph in one embodiment.
Figure 2 is a block diagram that illustrates components of the ranking system in one embodiment.
Figure 3 is a flow diagram illustrating the overall processing of the ranking system in one embodiment.
Figure 4 is a flow diagram that illustrates the processing of a construct affinity graph component in one embodiment.
Figure 5 is a flow diagram that illustrates the processing of a rank documents component in one embodiment.
2d DETAILED DESCRIPTION
A method and system for ranking documents of search results based on information richness and diversity of topics is provided. In one embodiment, a ranking system determines the information richness of each document within a search result. Information richness is a measure of how much information a document contains relating to its topics. A document (e.g., web page) with high information richness will likely contain information that subsumes information of documents that are related to the same topic but have lower information richness.
The ranking system groups documents of a search result based on their relatedness, meaning that they are directed to similar topics. The ranking system ranks the documents to ensure that the highest ranking documents may include at least one document covering each topic, that is, one document from each of the groups. The ranking system selects the document from each group that has the highest information richness of the documents within the group. When the documents are presented to a user in rank order, the user will likely find on the first page of the search result documents that cover a variety of topics, rather than just a single popular topic. For example, if the search request includes the search term "Spielberg," then one document on the first page of the search result may be related to Steven Spielberg, while another document on the first page may be related to a Professor Spielberg. In this way, a user will more likely be presented with documents covering a diversity of topics on the first page of the search result and will less likely become frustrated when the topic of interest is not the most popular topic relating to the search request. Moreover, because the ranking system ranks documents with higher information richness higher than documents with a lower information richness, the user is more likely to find the desired information within a document presented on the first page of the search result.
In one embodiment, the ranking system calculates the information richness of documents of a search result based on an affinity graph. Affinity is a measure of to what extent the information of one document is subsumed by the information of another document. For example, a document that describes one of Spielberg's movies superficially may have a high affinity to a document that describes all of Spielberg's movies in detail. Conversely, the document that describes all of Spielberg's movies in detail may have a relatively low affinity to the document that describes one of Spielberg's movies superficially. Documents that are related to
di represents the other document of the pair, d, represents a vector of document di, di represents a vector of document dj, and 11 ci,11 represents the length of vector di, each vector having at least one entry for at least one identified keyword of the corresponding document; for each identified document of the search result, calculating information richness for the identified document based on an element of a normalized affinity matrix that is derived from an affinity matrix, wherein the information richness is calculated based on the following:
InfoRich(d,)= E InfoRich(c 1 ) = ic ji au j#1 where InfoRich(di) is the information richness of document di, document di represents the identified document of the search result and document dj represents another document in the search result, and /14 is an element of the normalized affinity matrix, wherein the affinity matrix containing the calculated affinity measurement of the identified document and the another document in the search result; and displaying indications of documents of the search result ranked based on the calculated information richnesses of the documents of the search result.
According to another aspect of the present invention, there is provided a method in a computer system with a processor and a memory for calculating information richness of a document within a collection of documents, the documents in the collection having words, the method comprising:
identifying by the processor an affinity each document in the collection has to another document in the collection, wherein the affinity is identified for each pair of documents with one document of the pair being the document and the other document of the pair being another document in the collection of documents, affinity indicating to what extent the information of one document is subsumed by the information of another document; determining by the processor information 2a , 71570-1 richness for each document in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix; storing in the memory indications of the determined information richnesses of the documents in the collection; and ranking the documents based on the stored indications of information richnesses, wherein the determined information richness for each document is defined as InfoRich(d,)= E InfoRich(d ) = 1171 all J*1 where InfoRich(di) is the information richness of document di, document d, represents the document and document di represents another document of the collection of documents, and M is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the document to the another documents in the collection, wherein the documents in the collection are web pages.
According to still another aspect of the present invention, there is provided a computing device for calculating information richness of a document within a collection of web pages, the web pages having words, comprising: a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising: identifying an affinity of each web page in the collection has to the another web page in the collection, wherein the affinity is identified for each pair of web pages with one web page of the pair being the web page and the other web page of the pair being another web page in the collection of web pages, affinity indicating to what extent the information of one web page is subsumed by the information of another web page;
determining information richness for each web page in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix;
storing in the memory indications of the determined information richnesses of the web pages in the collection; and ranking the web pages based on the stored indications of information richnesses, wherein the determined information richness for each web page is defined as 2b InfoRich( d, )= InfoRich( d ) = ICI ji all j#1 where InfoRich(d) is the information richness of web page dõ web page di represents the web page and web page di represents another web page of the collection of web pages, and /17/,, is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the web page to the another web page in the collection.
A system ranks documents of search results based on information richness and diversity of topics. A ranking system groups documents of a search result based on their relatedness, meaning that they are directed to similar topics.
The ranking system ranks the documents to ensure that the highest ranking documents include at least one document covering each topic. The ranking system then selects the document from each group that has the highest information richness of the documents within the group as one of the highest ranking documents.
2c . 71570-1 BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a diagram that illustrates an affinity graph in one embodiment.
Figure 2 is a block diagram that illustrates components of the ranking system in one embodiment.
Figure 3 is a flow diagram illustrating the overall processing of the ranking system in one embodiment.
Figure 4 is a flow diagram that illustrates the processing of a construct affinity graph component in one embodiment.
Figure 5 is a flow diagram that illustrates the processing of a rank documents component in one embodiment.
2d DETAILED DESCRIPTION
A method and system for ranking documents of search results based on information richness and diversity of topics is provided. In one embodiment, a ranking system determines the information richness of each document within a search result. Information richness is a measure of how much information a document contains relating to its topics. A document (e.g., web page) with high information richness will likely contain information that subsumes information of documents that are related to the same topic but have lower information richness.
The ranking system groups documents of a search result based on their relatedness, meaning that they are directed to similar topics. The ranking system ranks the documents to ensure that the highest ranking documents may include at least one document covering each topic, that is, one document from each of the groups. The ranking system selects the document from each group that has the highest information richness of the documents within the group. When the documents are presented to a user in rank order, the user will likely find on the first page of the search result documents that cover a variety of topics, rather than just a single popular topic. For example, if the search request includes the search term "Spielberg," then one document on the first page of the search result may be related to Steven Spielberg, while another document on the first page may be related to a Professor Spielberg. In this way, a user will more likely be presented with documents covering a diversity of topics on the first page of the search result and will less likely become frustrated when the topic of interest is not the most popular topic relating to the search request. Moreover, because the ranking system ranks documents with higher information richness higher than documents with a lower information richness, the user is more likely to find the desired information within a document presented on the first page of the search result.
In one embodiment, the ranking system calculates the information richness of documents of a search result based on an affinity graph. Affinity is a measure of to what extent the information of one document is subsumed by the information of another document. For example, a document that describes one of Spielberg's movies superficially may have a high affinity to a document that describes all of Spielberg's movies in detail. Conversely, the document that describes all of Spielberg's movies in detail may have a relatively low affinity to the document that describes one of Spielberg's movies superficially. Documents that are related to
3 very different topics would have no affinity to each other. The collection of affinities of each document to every other document represents the affinity graph.
A document that has many other documents that have a high affinity to it will likely have high information richness because its information subsumes the information of many other documents. Moreover, if those other documents with the high affinity also have relatively high information richness themselves, then the information richness of the document is even higher.
In one embodiment, the ranking system helps ensure diversity of the high ranking documents of a search result also using an affinity graph. The ranking system may have an initial ranking score of the documents based on a conventional ranking technique (e.g., relevance), an information richness technique, or some other ranking technique. The ranking system initially selects the document with the highest initial ranking score as the document with the highest final ranking score. The ranking system then reduces the ranking score of each document that has a high affinity to the selected document. The ranking system reduces the ranking score because the content of those documents is likely to be subsumed by the selected document and would represent redundant information. The ranking system then selects that document of the remaining documents that has the then-highest ranking score. The ranking system reduces the ranking score of each document that has a high affinity to that newly selected document. The ranking system repeats this process until a desired number of documents have a final ranking score, all the documents have a final ranking score, or some other termination condition is met. In one embodiment, diversity represents the number of different topics in a collection of documents, and information richness of a document in a collection denotes the informative degree of the document with respect to the entire collection.
One skilled in the art will appreciate that the documents of search results can be ranked based on information richness alone or diversity alone, rather than on a combination of information richness and diversity. A search engine service may use information richness alone, for example, by identifying groups of documents related to similar topics and determining the information richness of each document within its group. The search engine service may then factor the determined information richness into the ranking of documents so that documents with the highest information richness of their group are likely to be ranked higher
A document that has many other documents that have a high affinity to it will likely have high information richness because its information subsumes the information of many other documents. Moreover, if those other documents with the high affinity also have relatively high information richness themselves, then the information richness of the document is even higher.
In one embodiment, the ranking system helps ensure diversity of the high ranking documents of a search result also using an affinity graph. The ranking system may have an initial ranking score of the documents based on a conventional ranking technique (e.g., relevance), an information richness technique, or some other ranking technique. The ranking system initially selects the document with the highest initial ranking score as the document with the highest final ranking score. The ranking system then reduces the ranking score of each document that has a high affinity to the selected document. The ranking system reduces the ranking score because the content of those documents is likely to be subsumed by the selected document and would represent redundant information. The ranking system then selects that document of the remaining documents that has the then-highest ranking score. The ranking system reduces the ranking score of each document that has a high affinity to that newly selected document. The ranking system repeats this process until a desired number of documents have a final ranking score, all the documents have a final ranking score, or some other termination condition is met. In one embodiment, diversity represents the number of different topics in a collection of documents, and information richness of a document in a collection denotes the informative degree of the document with respect to the entire collection.
One skilled in the art will appreciate that the documents of search results can be ranked based on information richness alone or diversity alone, rather than on a combination of information richness and diversity. A search engine service may use information richness alone, for example, by identifying groups of documents related to similar topics and determining the information richness of each document within its group. The search engine service may then factor the determined information richness into the ranking of documents so that documents with the highest information richness of their group are likely to be ranked higher
4 than other documents within their group. The search engine service may use diversity alone, for example, by identifying groups of documents relating to similar topics and ensuring that at least one document from each group is ranked high in the search results regardless of its information richness. For example, the search s engine service may select to display on the first page of the search result the document from each group with the highest relevance of the group.
An affinity graph represents documents as nodes and affinity values as the weight of directed edges between the nodes. The ranking system represents an affinity graph by a square matrix that maps each document to every other document in a collection of documents. The ranking system sets the value of elements of the matrix to the affinity of the corresponding documents. If M is the matrix, then Mei represents the affinity of document i to document j. The ranking system calculates the affinity of documents by representing each document as a vector. The vector represents the informational content of the document. For example, each vector may contain the 25 most important keywords of the document. The ranking system may calculate affinity according to the following:
(1) aff(di,d i)=cii = cif where aff(di,di) is the affinity of document d, to document di, di represents the vector of document di, di represents the vector of document di, and represents the length of vector di. Equation 1 sets the affinity to the length of the projection from di to di. One skilled in the art will appreciate that affinity can be defined in many different ways. For example, the affinity of one document to another could be defined based on the percentage of the keywords of the one document that are in the keywords of the other document. In set theory terms, the affinity of one document to another document can be expressed as the number of keywords in the intersection of two documents divided by the number of keywords in the other document. Each element of the matrix M represents a directed edge in the affinity graph from the node of one document to the node of the other document. In one embodiment, the ranking system sets an affinity value that is below an affinity threshold value (e.g., .2) to zero. Conceptually, this means that
An affinity graph represents documents as nodes and affinity values as the weight of directed edges between the nodes. The ranking system represents an affinity graph by a square matrix that maps each document to every other document in a collection of documents. The ranking system sets the value of elements of the matrix to the affinity of the corresponding documents. If M is the matrix, then Mei represents the affinity of document i to document j. The ranking system calculates the affinity of documents by representing each document as a vector. The vector represents the informational content of the document. For example, each vector may contain the 25 most important keywords of the document. The ranking system may calculate affinity according to the following:
(1) aff(di,d i)=cii = cif where aff(di,di) is the affinity of document d, to document di, di represents the vector of document di, di represents the vector of document di, and represents the length of vector di. Equation 1 sets the affinity to the length of the projection from di to di. One skilled in the art will appreciate that affinity can be defined in many different ways. For example, the affinity of one document to another could be defined based on the percentage of the keywords of the one document that are in the keywords of the other document. In set theory terms, the affinity of one document to another document can be expressed as the number of keywords in the intersection of two documents divided by the number of keywords in the other document. Each element of the matrix M represents a directed edge in the affinity graph from the node of one document to the node of the other document. In one embodiment, the ranking system sets an affinity value that is below an affinity threshold value (e.g., .2) to zero. Conceptually, this means that
5 there is no directed edge from the node of one document to the node of the other document in the affinity graph when the affinity is low. The affinity matrix is represented by the following:
f aff (d õ d j), if aff (d õ d j) ?.. all;
( 2 ) Mli =10 , otherwise where Mu is an element of the matrix and aft; is the affinity threshold value.
A
group of nodes with many edges between them may represent a single topic because many of the documents in the group have an affinity greater than a threshold affinity to each other. In contrast, nodes with no links between them represent documents directed to different topics.
The ranking system calculates information richness for each document by applying an edge analysis algorithm to the affinity graph. The ranking system normalizes the affinity matrix so that the values in each row add up to 1. The normalized affinity matrix is represented by the following:
{
(3) Ai. = Mi,j/i, Mu , if iiMu * 0 41 0 , otherwise where Ai is an element of the normalized affinity matrix. The ranking system calculates information richness according to the following:
InfoRich(d i) = E InfoRich(d j) = .1171 ii (4) all j*i where InfoRich(d i) is the information richness of document d.. Thus, information richness is defined recursively. Equation 4 can be represented in matrix form by the following:
.1=MTA, (5) [0032] where A = [InfoRich (d i)]n., is the eigenvector of the normalized affinity matrix lcir .
Since the normalized affinity matrix SI is typically a sparse matrix, all-zero rows could possibly appear in it, which means that some documents have no other documents with significant affinity to them. To compute a meaningful eigenvector, the ranking system uses a dumping factor (e.g., .85) that may be a document ranking based on popularity of the document. The information richness using a dumping factor is represented by the following:
-InfoRich(c1 ,) = c = E InfoRich(d j) = M i, + (1-0
f aff (d õ d j), if aff (d õ d j) ?.. all;
( 2 ) Mli =10 , otherwise where Mu is an element of the matrix and aft; is the affinity threshold value.
A
group of nodes with many edges between them may represent a single topic because many of the documents in the group have an affinity greater than a threshold affinity to each other. In contrast, nodes with no links between them represent documents directed to different topics.
The ranking system calculates information richness for each document by applying an edge analysis algorithm to the affinity graph. The ranking system normalizes the affinity matrix so that the values in each row add up to 1. The normalized affinity matrix is represented by the following:
{
(3) Ai. = Mi,j/i, Mu , if iiMu * 0 41 0 , otherwise where Ai is an element of the normalized affinity matrix. The ranking system calculates information richness according to the following:
InfoRich(d i) = E InfoRich(d j) = .1171 ii (4) all j*i where InfoRich(d i) is the information richness of document d.. Thus, information richness is defined recursively. Equation 4 can be represented in matrix form by the following:
.1=MTA, (5) [0032] where A = [InfoRich (d i)]n., is the eigenvector of the normalized affinity matrix lcir .
Since the normalized affinity matrix SI is typically a sparse matrix, all-zero rows could possibly appear in it, which means that some documents have no other documents with significant affinity to them. To compute a meaningful eigenvector, the ranking system uses a dumping factor (e.g., .85) that may be a document ranking based on popularity of the document. The information richness using a dumping factor is represented by the following:
-InfoRich(c1 ,) = c = E InfoRich(d j) = M i, + (1-0
(6) all .1; n where the c is the dumping factor and n is the number of documents in the collection. Equation 6 can be represented in matrix form as follows:
(7) n where.e. is a unit vector with all components equal to 1. The computation of information richness can be analogized to an information flow and sink model.
With this model, information flows among the nodes at each iteration. A
document d1 has a set of documents A(d1) to which it has an affinity as represented by the following:
A(d ;) = 41 i I Vj # i,aff (d i,d j) > aff,}
With this model, information flows among the nodes at each iteration. A
document d1 has a set of documents A(d1) to which it has an affinity as represented by the following:
A(d ;) = 41 i I Vj # i,aff (d i,d j) > aff,}
(8) In each iteration, the information can flow according to one of the following rules:
1. With a probability c (i.e., the dumping factor), the information will flow into one of the document in A(d i) , and the probability of flowing into the document di is proportional to aff (d i , d j) .
2. With a probability of 1- c , the information will randomly flow into any documents in the collection.
A Markov chain can be induced from the above process, where the states are given by the documents and the transition (or flow) matrix is given by cictir + u (a) n where U. [ -1- .
The stationary probability distribution of each state is given by n nxn the principal eigenvector of the transition matrix.
In one embodiment, the ranking system calculates an affinity rank by combining information richness with a similarity penalty so that multiple documents directed to the same topic are not all ranked highly to the exclusion of documents directed to other topics. The use of a similarity penalty results in an increase in the diversity of topics among the most highly ranked documents. The ranking system may use a iterative greedy algorithm to calculate the similarity penalties with the initial affinity rank of a document being set to its information richness. At each iteration, the algorithm selects the document with the next highest affinity rank and reduces the affinity rank of the documents directed to the same topic by a similarity penalty. Thus, once a document is selected, all other documents directed to the same topic will have their affinity rank reduced to improve the chance that the highest ranking documents represent diverse topics. The ranking system may reduce the affinity rank of documents according to the following:
AR; = ARi -/-4.0 = InfoRich(di) (10) where ARj represents the affinity rank of document j and i is the selected document. Because the similarity penalty is based on the affinity matrix, the more a document is similar to the selected document, the greater its similarity penalty.
The ranking system may combine an affinity ranking with a text-based ranking (e.g., conventional relevance) to generate an overall ranking in an embodiment.
The rankings can be combined based on scores or based on ranks. With combined scoring, the text-based score is combined with the affinity rank to give an overall score representing the final score of the document. The combined scoring may be based on a linear combination of the text-based score and the affinity rank. Because the scores may have different orders of magnitude, the ranking system normalizes the scores. The combined scoring may be represented by the following:
Sim(q,d,) log ARe (11) Score(q,d,)= a ¨ +fl. , Vd, E
Sime (q) log AR
where a+ /1=1, e represents the search results for search request q, Sim(q,di) represents the similarity of document di to the search request q, and (12) Sime(q)=Maxvd,EoSim(q,di) (13) ARe = Maxvd,eeARI
With combined ranking, the text-based ranking is combined with the affinity ranking to give a final ranking of the documents. The combined ranking may be based on a linear combination of the text-based ranking and the affinity ranking.
The combined ranking may be represented by the following:
Score(q,di) = a = Ranksi,õ(q40+ fi = Rank,,, Vd E
(14) where Score represents the final ranking for document di for the search request q, Ranksiõ,(q.d) represents the text-based ranking, and Rank represents the affinity ranking. The a and fl in both combining algorithms are parameters that can be tuned. When a =1 and /3=0, no re-ranking is performed, and the search result is ranked based on the text-based search. When /3> a, more weight is put on the affinity ranking when re-ranking. When /3=1 and a= 0, the re-ranking is performed based solely on the affinity ranking.
Figure 1 is a diagram that illustrates an affinity graph in one embodiment.
The affinity graph 100 includes nodes 111-115, nodes 121-124, and node 131, which each represent a document. The directed edges between the nodes indicate the affinity of one node to another. For example, node 111 has an affinity to node 115, but node 115 does not have an affinity (or has an affinity below a threshold level) to node 111. In this example, the node group 110 comprises nodes 111-115 that are directed to the same topic, because there are many edges between the nodes in that node group. Similarly, the node group 120 comprises nodes 121-124 that are directed to the same topic. Node group 130 has only one node because that node does not have an affinity to any other node and no node has an affinity to it. Node 115 is likely to have the highest information richness of all the nodes in node group 110 and node 124 is likely to have the highest information richness of all the nodes in node group 120 because each node has the greatest number of nodes who have an affinity to it.
Figure 2 is a block diagram that illustrates components of the ranking system in one embodiment. The ranking system 200 includes data stores 201-204 and components 211-216. The document store 201 contains the collection of documents and may represent all web pages available via the Internet. The generate affinity graph component 211 generates an affinity graph based on the documents of the document store. The generate affinity graph component stores the affinities in the affinity graph store 202. The calculate information richness component 212 inputs the affinity graph from the affinity graph store and calculates an information richness score for each document. The component stores the calculated information richness scores in the information richness store 203. In one embodiment, the generate affinity graph component and the calculate information richness component can execute off-line to generate the affinity graph and the information richness scores prior to the conducting of a search. The conduct search component 213 receives a search request from a user and identifies the search result from the documents in the document store. The
1. With a probability c (i.e., the dumping factor), the information will flow into one of the document in A(d i) , and the probability of flowing into the document di is proportional to aff (d i , d j) .
2. With a probability of 1- c , the information will randomly flow into any documents in the collection.
A Markov chain can be induced from the above process, where the states are given by the documents and the transition (or flow) matrix is given by cictir + u (a) n where U. [ -1- .
The stationary probability distribution of each state is given by n nxn the principal eigenvector of the transition matrix.
In one embodiment, the ranking system calculates an affinity rank by combining information richness with a similarity penalty so that multiple documents directed to the same topic are not all ranked highly to the exclusion of documents directed to other topics. The use of a similarity penalty results in an increase in the diversity of topics among the most highly ranked documents. The ranking system may use a iterative greedy algorithm to calculate the similarity penalties with the initial affinity rank of a document being set to its information richness. At each iteration, the algorithm selects the document with the next highest affinity rank and reduces the affinity rank of the documents directed to the same topic by a similarity penalty. Thus, once a document is selected, all other documents directed to the same topic will have their affinity rank reduced to improve the chance that the highest ranking documents represent diverse topics. The ranking system may reduce the affinity rank of documents according to the following:
AR; = ARi -/-4.0 = InfoRich(di) (10) where ARj represents the affinity rank of document j and i is the selected document. Because the similarity penalty is based on the affinity matrix, the more a document is similar to the selected document, the greater its similarity penalty.
The ranking system may combine an affinity ranking with a text-based ranking (e.g., conventional relevance) to generate an overall ranking in an embodiment.
The rankings can be combined based on scores or based on ranks. With combined scoring, the text-based score is combined with the affinity rank to give an overall score representing the final score of the document. The combined scoring may be based on a linear combination of the text-based score and the affinity rank. Because the scores may have different orders of magnitude, the ranking system normalizes the scores. The combined scoring may be represented by the following:
Sim(q,d,) log ARe (11) Score(q,d,)= a ¨ +fl. , Vd, E
Sime (q) log AR
where a+ /1=1, e represents the search results for search request q, Sim(q,di) represents the similarity of document di to the search request q, and (12) Sime(q)=Maxvd,EoSim(q,di) (13) ARe = Maxvd,eeARI
With combined ranking, the text-based ranking is combined with the affinity ranking to give a final ranking of the documents. The combined ranking may be based on a linear combination of the text-based ranking and the affinity ranking.
The combined ranking may be represented by the following:
Score(q,di) = a = Ranksi,õ(q40+ fi = Rank,,, Vd E
(14) where Score represents the final ranking for document di for the search request q, Ranksiõ,(q.d) represents the text-based ranking, and Rank represents the affinity ranking. The a and fl in both combining algorithms are parameters that can be tuned. When a =1 and /3=0, no re-ranking is performed, and the search result is ranked based on the text-based search. When /3> a, more weight is put on the affinity ranking when re-ranking. When /3=1 and a= 0, the re-ranking is performed based solely on the affinity ranking.
Figure 1 is a diagram that illustrates an affinity graph in one embodiment.
The affinity graph 100 includes nodes 111-115, nodes 121-124, and node 131, which each represent a document. The directed edges between the nodes indicate the affinity of one node to another. For example, node 111 has an affinity to node 115, but node 115 does not have an affinity (or has an affinity below a threshold level) to node 111. In this example, the node group 110 comprises nodes 111-115 that are directed to the same topic, because there are many edges between the nodes in that node group. Similarly, the node group 120 comprises nodes 121-124 that are directed to the same topic. Node group 130 has only one node because that node does not have an affinity to any other node and no node has an affinity to it. Node 115 is likely to have the highest information richness of all the nodes in node group 110 and node 124 is likely to have the highest information richness of all the nodes in node group 120 because each node has the greatest number of nodes who have an affinity to it.
Figure 2 is a block diagram that illustrates components of the ranking system in one embodiment. The ranking system 200 includes data stores 201-204 and components 211-216. The document store 201 contains the collection of documents and may represent all web pages available via the Internet. The generate affinity graph component 211 generates an affinity graph based on the documents of the document store. The generate affinity graph component stores the affinities in the affinity graph store 202. The calculate information richness component 212 inputs the affinity graph from the affinity graph store and calculates an information richness score for each document. The component stores the calculated information richness scores in the information richness store 203. In one embodiment, the generate affinity graph component and the calculate information richness component can execute off-line to generate the affinity graph and the information richness scores prior to the conducting of a search. The conduct search component 213 receives a search request from a user and identifies the search result from the documents in the document store. The
9 conduct search component stores the search result in the search result store along with an indication of the relevance of each document of the search result to the search request. The calculate similarity penalty component 214 calculates a similarity penalty to apply to the affinity rank based on the information of the search result store, the affinity graph store, and the information richness store.
The calculate affinity rank component 215 generates an affinity rank for each document in the search result. The calculate affinity rank component factors in the information richness of the document, the affinity graph store, and the search result. The calculate final score component 216 combines the affinity rank and relevance score to calculate a final score.
The computing device on which the ranking system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). The memory and storage devices are computer-readable media that may contain instructions that implement the ranking system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communications link.
Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection.
The ranking system may be implemented in various operating environments.
Various well-known computing systems, environments, and configurations that may be suitable for use include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The ranking system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
Figure 3 is a flow diagram illustrating the overall processing of the ranking system in one embodiment. The ranking system is provided with a collection of documents that may represent a search result. In block 301, the component constructs an affinity graph for the collection of documents. The component may s construct an affinity graph covering all documents in a corpus of documents (e.g., all web pages) off-line or covering only the documents of the collection in real time. In block 302, the component calculates the information richness of each document of the collection. In block 303, the component ranks the documents of the collection and then completes.
Figure 4 is a flow diagram that illustrates the processing of a construct affinity graph component in one embodiment. The component is passed a collection of documents and constructs an affinity graph for those documents. In blocks 401-403, the component loops generating a document vector for each document in a collection of documents. In block 401, the component selects the next document in the collection. In decision block 402, if all the documents in the collection have already been selected, then the component continues at block 404, else the component continues at block 403. In block 403, the component generates the document vector for the selected document and then loops to block 401 to select the next document in the collection. In blocks 404-408, the component calculates the affinity for each pair of documents in the collection. In block 404, the component selects the next document in the collection starting with the first document. In decision block 405, if all the documents have already been selected, then the component returns the affinity graph, else the component continues at block 406. In blocks 406-408, the component loops choosing each document of the collection. In block 406, the component chooses the next document in the collection starting with the first document. In decision block 407, if all the documents in the collection have already been chosen, then the component loops to block 404 to select the next document in the collection, else the component continues at block 408. In block 408, the component calculates the affinity of the selected document to the chosen document according to Equation 1 and then loops to block 406 to choose the next document in the collection.
Figure 5 is a flow diagram that illustrates the processing of a rank documents component in one embodiment. The component is passed a collection of documents that has had its affinity graph generated and the information richness of each document calculated. In blocks 501-503, the component loops initializing the affinity rank of each document in the collection to its information richness. In block 501, the component selects the next document in the collection. In decision block 502, if all the documents have already been selected, then the component continues at block 504, else the component continues at block 503. In block 503, the component sets the affinity rank of the selected document to the information richness of the selected document and then loops to block 501 to select the next document in the collection. In blocks 504-508, the component loops identifying o pairs of documents and adjusting affinity ranks by a similarity penalty. In block 504, the component selects the next document with the highest affinity rank.
In decision block 505, if a termination condition has been reached, then the component returns the ranked documents, else the component continues at block 506. In blocks 506-508, the component loops choosing documents and adjusting is the affinity rank by the similarity penalty. In block 506, the component chooses the next document that has an affinity to the selected document as indicated by a non-zero value in the affinity graph for the affinity from the chosen document to the selected document. In decision block 507, if all those documents have already been chosen, then the component loops to block 504 to select the next document 20 with the highest affinity rank. In block 508, the component adjusts the affinity rank for the chosen document by a similarity penalty according to Equation 10. The component then loops to block 506 to choose the next document with an affinity to the selected document.
One skilled in the art will appreciate that although specific embodiments of the ranking system have been described herein for purposes of illustration, various modifications may be made without deviating from the scope of the claims. In one embodiment, the ranking system may calculate the affinity and information richness on a block-by-block basis rather than a document-by-document basis. A block represents information of a web page that is generally related to a single topic. The ranking of the web page may be based in part on the importance of a block to its web page. The importance of blocks is described in U.S. Patent No. 7,363,279 entitled "Method and System for Calculating Importance of a Block Within a Display Page"
and issued on April 22, 2008. Accordingly, the invention is not limited except by the appended claims.
The calculate affinity rank component 215 generates an affinity rank for each document in the search result. The calculate affinity rank component factors in the information richness of the document, the affinity graph store, and the search result. The calculate final score component 216 combines the affinity rank and relevance score to calculate a final score.
The computing device on which the ranking system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). The memory and storage devices are computer-readable media that may contain instructions that implement the ranking system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communications link.
Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection.
The ranking system may be implemented in various operating environments.
Various well-known computing systems, environments, and configurations that may be suitable for use include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The ranking system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
Figure 3 is a flow diagram illustrating the overall processing of the ranking system in one embodiment. The ranking system is provided with a collection of documents that may represent a search result. In block 301, the component constructs an affinity graph for the collection of documents. The component may s construct an affinity graph covering all documents in a corpus of documents (e.g., all web pages) off-line or covering only the documents of the collection in real time. In block 302, the component calculates the information richness of each document of the collection. In block 303, the component ranks the documents of the collection and then completes.
Figure 4 is a flow diagram that illustrates the processing of a construct affinity graph component in one embodiment. The component is passed a collection of documents and constructs an affinity graph for those documents. In blocks 401-403, the component loops generating a document vector for each document in a collection of documents. In block 401, the component selects the next document in the collection. In decision block 402, if all the documents in the collection have already been selected, then the component continues at block 404, else the component continues at block 403. In block 403, the component generates the document vector for the selected document and then loops to block 401 to select the next document in the collection. In blocks 404-408, the component calculates the affinity for each pair of documents in the collection. In block 404, the component selects the next document in the collection starting with the first document. In decision block 405, if all the documents have already been selected, then the component returns the affinity graph, else the component continues at block 406. In blocks 406-408, the component loops choosing each document of the collection. In block 406, the component chooses the next document in the collection starting with the first document. In decision block 407, if all the documents in the collection have already been chosen, then the component loops to block 404 to select the next document in the collection, else the component continues at block 408. In block 408, the component calculates the affinity of the selected document to the chosen document according to Equation 1 and then loops to block 406 to choose the next document in the collection.
Figure 5 is a flow diagram that illustrates the processing of a rank documents component in one embodiment. The component is passed a collection of documents that has had its affinity graph generated and the information richness of each document calculated. In blocks 501-503, the component loops initializing the affinity rank of each document in the collection to its information richness. In block 501, the component selects the next document in the collection. In decision block 502, if all the documents have already been selected, then the component continues at block 504, else the component continues at block 503. In block 503, the component sets the affinity rank of the selected document to the information richness of the selected document and then loops to block 501 to select the next document in the collection. In blocks 504-508, the component loops identifying o pairs of documents and adjusting affinity ranks by a similarity penalty. In block 504, the component selects the next document with the highest affinity rank.
In decision block 505, if a termination condition has been reached, then the component returns the ranked documents, else the component continues at block 506. In blocks 506-508, the component loops choosing documents and adjusting is the affinity rank by the similarity penalty. In block 506, the component chooses the next document that has an affinity to the selected document as indicated by a non-zero value in the affinity graph for the affinity from the chosen document to the selected document. In decision block 507, if all those documents have already been chosen, then the component loops to block 504 to select the next document 20 with the highest affinity rank. In block 508, the component adjusts the affinity rank for the chosen document by a similarity penalty according to Equation 10. The component then loops to block 506 to choose the next document with an affinity to the selected document.
One skilled in the art will appreciate that although specific embodiments of the ranking system have been described herein for purposes of illustration, various modifications may be made without deviating from the scope of the claims. In one embodiment, the ranking system may calculate the affinity and information richness on a block-by-block basis rather than a document-by-document basis. A block represents information of a web page that is generally related to a single topic. The ranking of the web page may be based in part on the importance of a block to its web page. The importance of blocks is described in U.S. Patent No. 7,363,279 entitled "Method and System for Calculating Importance of a Block Within a Display Page"
and issued on April 22, 2008. Accordingly, the invention is not limited except by the appended claims.
Claims (8)
1. A computing device for ranking documents of a search result, each document having words, comprising:
a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising:
receiving from a user a query;
identifying documents as a search result for the received query;
identifying keywords of the identified documents;
for each pair of identified documents of the search result, calculating an affinity measurement indicating affinity that a document of the pair has to the other document of the pair, wherein affinity measurement is calculated based on the following:
where aff(d i, d j) is the affinity measurement of document d i to document d j, document d i represents the document of the pair and document d j represents the other document of the pair, ~ represents a vector of document d i, ~
represents a vector of document d j, and ¦¦ ~i¦¦ represents the length of vector d i, each vector having at least one entry for at least one identified keyword of the corresponding document;
for each identified document of the search result, calculating information richness for the identified document based on an element of a normalized affinity matrix that is derived from an affinity matrix, wherein the information richness is calculated based on the following:
where InfoRich(d i) is the information richness of document d i, document d i represents the identified document of the search result and document d j represents another document in the search result, and ~ ji is an element of the normalized affinity matrix, wherein the affinity matrix containing the calculated affinity measurement of the identified document and the another document in the search result; and displaying indications of documents of the search result ranked based on the calculated information richnesses of the documents of the search result.
a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising:
receiving from a user a query;
identifying documents as a search result for the received query;
identifying keywords of the identified documents;
for each pair of identified documents of the search result, calculating an affinity measurement indicating affinity that a document of the pair has to the other document of the pair, wherein affinity measurement is calculated based on the following:
where aff(d i, d j) is the affinity measurement of document d i to document d j, document d i represents the document of the pair and document d j represents the other document of the pair, ~ represents a vector of document d i, ~
represents a vector of document d j, and ¦¦ ~i¦¦ represents the length of vector d i, each vector having at least one entry for at least one identified keyword of the corresponding document;
for each identified document of the search result, calculating information richness for the identified document based on an element of a normalized affinity matrix that is derived from an affinity matrix, wherein the information richness is calculated based on the following:
where InfoRich(d i) is the information richness of document d i, document d i represents the identified document of the search result and document d j represents another document in the search result, and ~ ji is an element of the normalized affinity matrix, wherein the affinity matrix containing the calculated affinity measurement of the identified document and the another document in the search result; and displaying indications of documents of the search result ranked based on the calculated information richnesses of the documents of the search result.
2. The computing device of claim 1 wherein the identifying of the documents including submitting the query to a search engine service and receiving indications of the documents as the search result.
3. The computing device of claim 1 wherein the documents are web pages.
4. A method in a computer system with a processor and a memory for calculating information richness of a document within a collection of documents, the documents in the collection having words, the method comprising:
identifying by the processor an affinity each document in the collection has to another document in the collection, wherein the affinity is identified for each pair of documents with one document of the pair being the document and the other document of the pair being another document in the collection of documents, affinity indicating to what extent the information of one document is subsumed by the information of another document;
determining by the processor information richness for each document in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix;
storing in the memory indications of the determined information richnesses of the documents in the collection; and ranking the documents based on the stored indications of information richnesses, wherein the determined information richness for each document is defined as where lnfoRich(d i) is the information richness of document d i, document di represents the document and document di represents another document of the collection of documents, and ~ ij is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the document to the another documents in the collection, wherein the documents in the collection are web pages.
identifying by the processor an affinity each document in the collection has to another document in the collection, wherein the affinity is identified for each pair of documents with one document of the pair being the document and the other document of the pair being another document in the collection of documents, affinity indicating to what extent the information of one document is subsumed by the information of another document;
determining by the processor information richness for each document in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix;
storing in the memory indications of the determined information richnesses of the documents in the collection; and ranking the documents based on the stored indications of information richnesses, wherein the determined information richness for each document is defined as where lnfoRich(d i) is the information richness of document d i, document di represents the document and document di represents another document of the collection of documents, and ~ ij is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the document to the another documents in the collection, wherein the documents in the collection are web pages.
5. A
computing device for calculating information richness of a document within a collection of web pages, the web pages having words, comprising:
a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising:
identifying an affinity of each web page in the collection has to the another web page in the collection, wherein the affinity is identified for each pair of web pages with one web page of the pair being the web page and the other web page of the pair being another web page in the collection of web pages, affinity indicating to what extent the information of one web page is subsumed by the information of another web page;
determining information richness for each web page in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix;
storing in the memory indications of the determined information richnesses of the web pages in the collection; and ranking the web pages based on the stored indications of information richnesses, wherein the determined information richness for each web page is defined as where lnfoRich(d i) is the information richness of web page d i, web page di represents the web page and web page di represents another web page of the collection of web pages, and M~ ij is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the web page to the another web page in the collection.
computing device for calculating information richness of a document within a collection of web pages, the web pages having words, comprising:
a memory having computer-executable instructions and a processor for executing the computer-executable instructions stored in the memory, the executed computer-executable instructions perform steps comprising:
identifying an affinity of each web page in the collection has to the another web page in the collection, wherein the affinity is identified for each pair of web pages with one web page of the pair being the web page and the other web page of the pair being another web page in the collection of web pages, affinity indicating to what extent the information of one web page is subsumed by the information of another web page;
determining information richness for each web page in the collection based on an element of a normalized affinity matrix that is derived from an affinity matrix;
storing in the memory indications of the determined information richnesses of the web pages in the collection; and ranking the web pages based on the stored indications of information richnesses, wherein the determined information richness for each web page is defined as where lnfoRich(d i) is the information richness of web page d i, web page di represents the web page and web page di represents another web page of the collection of web pages, and M~ ij is an element of the normalized affinity matrix, wherein the affinity matrix containing the identified affinity of the web page to the another web page in the collection.
6. The computing device of claim 5 wherein the identifying of the affinity of each web page includes generating an affinity graph.
7. The computing device of claim 5 wherein the ranking of web pages includes submitting a query to a search engine service and receiving indications of the web pages as the search result.
8. The computing device of claim 7 wherein the ranking is further based on a search-based relevance.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US10/837,540 US7664735B2 (en) | 2004-04-30 | 2004-04-30 | Method and system for ranking documents of a search result to improve diversity and information richness |
| US10/837,540 | 2004-04-30 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2505904A1 CA2505904A1 (en) | 2005-10-30 |
| CA2505904C true CA2505904C (en) | 2013-09-03 |
Family
ID=34939598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2505904A Expired - Fee Related CA2505904C (en) | 2004-04-30 | 2005-04-29 | Method and system for ranking documents of a search result to improve diversity and information richness |
Country Status (10)
| Country | Link |
|---|---|
| US (1) | US7664735B2 (en) |
| EP (1) | EP1591923A1 (en) |
| JP (1) | JP4845420B2 (en) |
| KR (1) | KR101130535B1 (en) |
| CN (1) | CN100573513C (en) |
| AU (1) | AU2005201824A1 (en) |
| BR (1) | BRPI0502189A (en) |
| CA (1) | CA2505904C (en) |
| MX (1) | MXPA05004681A (en) |
| RU (1) | RU2383922C2 (en) |
Families Citing this family (68)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6560600B1 (en) * | 2000-10-25 | 2003-05-06 | Alta Vista Company | Method and apparatus for ranking Web page search results |
| US7809548B2 (en) * | 2004-06-14 | 2010-10-05 | University Of North Texas | Graph-based ranking algorithms for text processing |
| US20070073708A1 (en) * | 2005-09-28 | 2007-03-29 | Smith Adam D | Generation of topical subjects from alert search terms |
| US20070094242A1 (en) * | 2005-10-26 | 2007-04-26 | John Dove | System and method for returning search results |
| US20070112867A1 (en) * | 2005-11-15 | 2007-05-17 | Clairvoyance Corporation | Methods and apparatus for rank-based response set clustering |
| US20070112898A1 (en) * | 2005-11-15 | 2007-05-17 | Clairvoyance Corporation | Methods and apparatus for probe-based clustering |
| US7827208B2 (en) | 2006-08-11 | 2010-11-02 | Facebook, Inc. | Generating a feed of stories personalized for members of a social network |
| US8171128B2 (en) | 2006-08-11 | 2012-05-01 | Facebook, Inc. | Communicating a newsfeed of media content based on a member's interactions in a social network environment |
| US7644074B2 (en) * | 2005-12-22 | 2010-01-05 | Microsoft Corporation | Search by document type and relevance |
| US7814099B2 (en) * | 2006-01-31 | 2010-10-12 | Louis S. Wang | Method for ranking and sorting electronic documents in a search result list based on relevance |
| US7818315B2 (en) * | 2006-03-13 | 2010-10-19 | Microsoft Corporation | Re-ranking search results based on query log |
| US20080005137A1 (en) * | 2006-06-29 | 2008-01-03 | Microsoft Corporation | Incrementally building aspect models |
| US9779441B1 (en) * | 2006-08-04 | 2017-10-03 | Facebook, Inc. | Method for relevancy ranking of products in online shopping |
| US20080109435A1 (en) * | 2006-11-07 | 2008-05-08 | Bellsouth Intellectual Property Corporation | Determining Sort Order by Traffic Volume |
| US8156112B2 (en) | 2006-11-07 | 2012-04-10 | At&T Intellectual Property I, L.P. | Determining sort order by distance |
| US8301621B2 (en) * | 2006-11-07 | 2012-10-30 | At&T Intellectual Property I, L.P. | Topic map for navigational control |
| US20080114750A1 (en) * | 2006-11-14 | 2008-05-15 | Microsoft Corporation | Retrieval and ranking of items utilizing similarity |
| US7958126B2 (en) * | 2006-12-19 | 2011-06-07 | Yahoo! Inc. | Techniques for including collection items in search results |
| US20080154878A1 (en) * | 2006-12-20 | 2008-06-26 | Rose Daniel E | Diversifying a set of items |
| US20080215571A1 (en) * | 2007-03-01 | 2008-09-04 | Microsoft Corporation | Product review search |
| US8117137B2 (en) * | 2007-04-19 | 2012-02-14 | Microsoft Corporation | Field-programmable gate array based accelerator system |
| US8024327B2 (en) * | 2007-06-26 | 2011-09-20 | Endeca Technologies, Inc. | System and method for measuring the quality of document sets |
| US8935249B2 (en) | 2007-06-26 | 2015-01-13 | Oracle Otc Subsidiary Llc | Visualization of concepts within a collection of information |
| US8543380B2 (en) * | 2007-10-05 | 2013-09-24 | Fujitsu Limited | Determining a document specificity |
| US20090094209A1 (en) * | 2007-10-05 | 2009-04-09 | Fujitsu Limited | Determining The Depths Of Words And Documents |
| CN101855631B (en) * | 2007-11-08 | 2016-06-29 | 上海惠普有限公司 | Navigational ranking for focused crawling |
| US8321406B2 (en) * | 2008-03-31 | 2012-11-27 | Google Inc. | Media object query submission and response |
| KR100926876B1 (en) * | 2008-04-01 | 2009-11-16 | 엔에이치엔(주) | Rank Learning Model Creation Method and Rank Learning Model Creation System Using Rank Occurrence Probability |
| US20090287668A1 (en) * | 2008-05-16 | 2009-11-19 | Justsystems Evans Research, Inc. | Methods and apparatus for interactive document clustering |
| JP5146108B2 (en) * | 2008-05-27 | 2013-02-20 | 日本電気株式会社 | Document importance calculation system, document importance calculation method, and program |
| CN101625680B (en) * | 2008-07-09 | 2012-08-29 | 东北大学 | Document retrieval method in patent field |
| US8301638B2 (en) * | 2008-09-25 | 2012-10-30 | Microsoft Corporation | Automated feature selection based on rankboost for ranking |
| US8131659B2 (en) * | 2008-09-25 | 2012-03-06 | Microsoft Corporation | Field-programmable gate array based accelerator system |
| US9135396B1 (en) * | 2008-12-22 | 2015-09-15 | Amazon Technologies, Inc. | Method and system for determining sets of variant items |
| US8458171B2 (en) * | 2009-01-30 | 2013-06-04 | Google Inc. | Identifying query aspects |
| US8533202B2 (en) | 2009-07-07 | 2013-09-10 | Yahoo! Inc. | Entropy-based mixing and personalization |
| US8245135B2 (en) * | 2009-09-08 | 2012-08-14 | International Business Machines Corporation | Producing a visual summarization of text documents |
| CN101650746B (en) * | 2009-09-27 | 2011-06-29 | 中国电信股份有限公司 | Method and system for verifying sequencing results |
| US8849807B2 (en) | 2010-05-25 | 2014-09-30 | Mark F. McLellan | Active search results page ranking technology |
| US9240020B2 (en) | 2010-08-24 | 2016-01-19 | Yahoo! Inc. | Method of recommending content via social signals |
| EP2568396A1 (en) | 2011-09-08 | 2013-03-13 | Axel Springer Digital TV Guide GmbH | Method and apparatus for generating a sorted list of items |
| US8838583B1 (en) | 2011-10-05 | 2014-09-16 | Amazon Technologies, Inc | Diversity within search results |
| US9075498B1 (en) * | 2011-12-22 | 2015-07-07 | Symantec Corporation | User interface for finding similar documents |
| US9501566B1 (en) | 2012-01-17 | 2016-11-22 | Veritas Technologies Llc | User interface for transparent concept search |
| JP6149434B2 (en) * | 2012-04-10 | 2017-06-21 | 株式会社リコー | Information processing apparatus, document management server, program, file system |
| US20140075282A1 (en) * | 2012-06-26 | 2014-03-13 | Rediff.Com India Limited | Method and apparatus for composing a representative description for a cluster of digital documents |
| US9400789B2 (en) * | 2012-07-20 | 2016-07-26 | Google Inc. | Associating resources with entities |
| US9536001B2 (en) * | 2012-11-13 | 2017-01-03 | Microsoft Technology Licensing, Llc | Intent-based presentation of search results |
| US9129020B2 (en) | 2012-12-21 | 2015-09-08 | Microsoft Technology Licensing, Llc | Search results through interest circles |
| CN103927545B (en) * | 2014-03-14 | 2017-10-17 | 小米科技有限责任公司 | Clustering method and relevant apparatus |
| US9355227B2 (en) | 2014-06-30 | 2016-05-31 | Konica Minolta Laboratory U.S.A., Inc. | Dynamic document display personalization implemented in a digital rights management system |
| US9992262B2 (en) * | 2014-07-29 | 2018-06-05 | Konica Minolta Laboratory U.S.A., Inc. | Personalized document content aggregation and document association implemented in a digital rights management system |
| US9858251B2 (en) | 2014-08-14 | 2018-01-02 | Rakuten Kobo Inc. | Automatically generating customized annotation document from query search results and user interface thereof |
| KR102243286B1 (en) * | 2014-09-18 | 2021-04-22 | 경북대학교 산학협력단 | Method for constructing database, recording medium for performing the method |
| CN104881798A (en) * | 2015-06-05 | 2015-09-02 | 北京京东尚科信息技术有限公司 | Device and method for personalized search based on commodity image features |
| US11281639B2 (en) | 2015-06-23 | 2022-03-22 | Microsoft Technology Licensing, Llc | Match fix-up to remove matching documents |
| US10242071B2 (en) | 2015-06-23 | 2019-03-26 | Microsoft Technology Licensing, Llc | Preliminary ranker for scoring matching documents |
| US10467215B2 (en) * | 2015-06-23 | 2019-11-05 | Microsoft Technology Licensing, Llc | Matching documents using a bit vector search index |
| US11392568B2 (en) | 2015-06-23 | 2022-07-19 | Microsoft Technology Licensing, Llc | Reducing matching documents for a search query |
| US10685029B2 (en) | 2015-11-23 | 2020-06-16 | Google Llc | Information ranking based on properties of a computing device |
| GB2545931A (en) * | 2015-12-31 | 2017-07-05 | Francis Murphy Dominic | Defining edges and their weights between nodes in a network |
| CN105955990A (en) * | 2016-04-15 | 2016-09-21 | 北京理工大学 | Method for sequencing and screening of comments with consideration of diversity and effectiveness |
| RU2630427C2 (en) * | 2016-08-12 | 2017-09-07 | Дмитрий Владимирович Мительков | Method and system of semantic processing text documents |
| US10733359B2 (en) * | 2016-08-26 | 2020-08-04 | Adobe Inc. | Expanding input content utilizing previously-generated content |
| GB2570447A (en) * | 2018-01-23 | 2019-07-31 | Canon Kk | Method and system for improving construction of regions of interest |
| US11699094B2 (en) * | 2018-10-31 | 2023-07-11 | Salesforce, Inc. | Automatic feature selection and model generation for linear models |
| US11328238B2 (en) * | 2019-04-01 | 2022-05-10 | Microsoft Technology Licensing, Llc | Preemptively surfacing relevant content within email |
| CN110516062B (en) * | 2019-08-26 | 2022-11-04 | 腾讯科技（深圳）有限公司 | Method and device for searching and processing document |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5020019A (en) * | 1989-05-29 | 1991-05-28 | Ricoh Company, Ltd. | Document retrieval system |
| US5598557A (en) * | 1992-09-22 | 1997-01-28 | Caere Corporation | Apparatus and method for retrieving and grouping images representing text files based on the relevance of key words extracted from a selected file to the text files |
| US5576954A (en) * | 1993-11-05 | 1996-11-19 | University Of Central Florida | Process for determination of text relevancy |
| US5642502A (en) * | 1994-12-06 | 1997-06-24 | University Of Central Florida | Method and system for searching for relevant documents from a text database collection, using statistical ranking, relevancy feedback and small pieces of text |
| US5870740A (en) * | 1996-09-30 | 1999-02-09 | Apple Computer, Inc. | System and method for improving the ranking of information retrieval results for short queries |
| US6601075B1 (en) * | 2000-07-27 | 2003-07-29 | International Business Machines Corporation | System and method of ranking and retrieving documents based on authority scores of schemas and documents |
| US20020194161A1 (en) * | 2001-04-12 | 2002-12-19 | Mcnamee J. Paul | Directed web crawler with machine learning |
| WO2004025490A1 (en) | 2002-09-16 | 2004-03-25 | The Trustees Of Columbia University In The City Of New York | System and method for document collection, grouping and summarization |
| JP4356347B2 (en) * | 2003-04-16 | 2009-11-04 | セイコーエプソン株式会社 | Document extraction system |
-
2004
- 2004-04-30 US US10/837,540 patent/US7664735B2/en not_active Expired - Fee Related
-
2005
- 2005-04-28 BR BR0502189-8A patent/BRPI0502189A/en not_active IP Right Cessation
- 2005-04-29 RU RU2005113189/09A patent/RU2383922C2/en not_active IP Right Cessation
- 2005-04-29 MX MXPA05004681A patent/MXPA05004681A/en not_active Application Discontinuation
- 2005-04-29 KR KR1020050036407A patent/KR101130535B1/en not_active IP Right Cessation
- 2005-04-29 EP EP05103553A patent/EP1591923A1/en not_active Withdrawn
- 2005-04-29 AU AU2005201824A patent/AU2005201824A1/en not_active Abandoned
- 2005-04-29 CA CA2505904A patent/CA2505904C/en not_active Expired - Fee Related
- 2005-04-30 CN CNB2005100896477A patent/CN100573513C/en not_active Expired - Fee Related
- 2005-05-02 JP JP2005134488A patent/JP4845420B2/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| US7664735B2 (en) | 2010-02-16 |
| KR20060047664A (en) | 2006-05-18 |
| CN100573513C (en) | 2009-12-23 |
| CA2505904A1 (en) | 2005-10-30 |
| KR101130535B1 (en) | 2012-04-12 |
| CN1758244A (en) | 2006-04-12 |
| RU2383922C2 (en) | 2010-03-10 |
| MXPA05004681A (en) | 2006-03-08 |
| JP4845420B2 (en) | 2011-12-28 |
| US20050246328A1 (en) | 2005-11-03 |
| RU2005113189A (en) | 2006-11-10 |
| AU2005201824A1 (en) | 2005-11-17 |
| BRPI0502189A (en) | 2006-01-10 |
| JP2005322244A (en) | 2005-11-17 |
| EP1591923A1 (en) | 2005-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2505904C (en) | Method and system for ranking documents of a search result to improve diversity and information richness | |
| US7577650B2 (en) | Method and system for ranking objects of different object types | |
| US7529735B2 (en) | Method and system for mining information based on relationships | |
| CA2507309C (en) | Method and system for schema matching of web databases | |
| US7676520B2 (en) | Calculating importance of documents factoring historical importance | |
| US20070005588A1 (en) | Determining relevance using queries as surrogate content | |
| US20050210006A1 (en) | Field weighting in text searching | |
| EP1999654A2 (en) | Propagating relevance from labeled documents to unlabeled documents | |
| CA2640230A1 (en) | Training a ranking function using propagated document relevance | |
| US20080222138A1 (en) | Method and Apparatus for Constructing a Link Structure Between Documents | |
| EP2174246A1 (en) | Ranking documents based on a series of document graphs | |
| US7376643B2 (en) | Method and system for determining similarity of objects based on heterogeneous relationships | |
| US20090282032A1 (en) | Topic distillation via subsite retrieval | |
| CA2682051A1 (en) | Look-ahead document ranking system | |
| US7890502B2 (en) | Hierarchy-based propagation of contribution of documents | |
| US7774340B2 (en) | Method and system for calculating document importance using document classifications | |
| Yu et al. | Web search technology | |
| Kim et al. | C-rank: A contribution-based Web page ranking approach | |
| Hijikata et al. | HITS algorithm improvement using anchor-related text extracted by DOM structure analysis | |
| Xu et al. | A unified model of literal mining and link analysis for ranking web resources | |
| Xu et al. | Literal-matching-biased link analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |
Effective date: 20150429 |