WO2008061133A1

WO2008061133A1 - Search result ranking based on attributes of search listing collections

Info

Publication number: WO2008061133A1
Application number: PCT/US2007/084659
Authority: WO
Inventors: Joshua Rosen
Original assignee: Notebookz.Com
Priority date: 2006-11-15
Filing date: 2007-11-14
Publication date: 2008-05-22

Abstract

The relevance of search results to a search query is adjusted based on various attributes of search listing collections to which the search results have been added. The attributes include one or more of: the relevance of the title of the search result collection to the search query, the number of times a search result has been added to a search listing collection, the popularity of the search listing collection, and the relevance of the search listing collection to the search query. The popularity of the search listing collection is dependent on one or more of: the number of subscribers to the search listing collection, the reputation of the users who have subscribed to the search listing collection, and usage metric of the search listing collection, such as page view and click-through rates.

Description

SEARCH RESULT RANKING BASED ON ATTRIBUTES OF SEARCH LISTING COLLECTIONS

BACKGROUND OF THE INVENTION

Field of the Invention

[0001] The present invention relates generally to search engines and, more particularly, to search engines that rank search results based on attributes of search listing collections that contain the search results.

Description of the Related Art

[0002] Search engines have become popular tools to identify and locate specific information on the Internet. A search engine is a computer program that, when queried for information, retrieves either related information or pointers to the location of related information, or both, by evaluating content stored in its search database.

[0003] The search engines provided by Internet search services, such as Yahoo!, Google and MSN, accept a search query and returns search results ranked in accordance with their respective relevance algorithm. These search results are displayed through a graphical user interface (GUI) that includes an editable text field that displays the search query, a listing of search results, and a listing of text advertisements known in the art as sponsored links. A click on any of the search results hyperlinks the user to the web site associated with that search result and a click on any of the sponsored links hyperlinks the user to the web site of the sponsor.

[0004] One of the more popular ways to rank search results involves analyzing the location and frequency of keywords on a web page. Another frequently used technique is analyzing how web pages are hyperlinked to each other. A web page gets a ranking boost based on the number of other web pages that are hyperlinked to it. This technique is described in U.S. Pat. No. 6,285,999, the entire contents of which are incorporated by reference herein. Click-through rates of search results are analyzed in some search engines. The general rule is: the higher the click-through rate, the higher the ranking. A search engine known as Alexa tracks a user's actions while browsing and derives the relevance of search results based on the browsing histories of its users.

SUMMARY OF THE INVENTION

[0005] The present invention provides another method for ranking search results. In this method, the relevance of search results to a search query is determined or adjusted based on various attributes of search listing collections to which the search results have been added. The attributes include one or more of: the relevance of the title of the search result collection to the search query, the number of times a search result has been added to a search listing collection, the popularity of the search listing collection, and the relevance of the search listing collection to the search query. The popularity of the search listing collection is dependent on one or more of: the number of subscribers to the search listing collection, the reputation of the users who have subscribed to the search listing collection, and usage metric of the search listing collection, such as page view and click-through rates.

[0006] The present invention also provides a method for maintaining a search database. In this method, each time a search result is added to a search listing collection, the search database is updated to reflect this association. The relevance scores of the search results that are contained in the search listing collection are then determined or adjusted based on various attributes of the search listing collection.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments. [0008] FIG. 1 illustrates a networked computer environment in which the present invention may be practiced.

[0009] FIGS. 2 and 3 are schematic representations of two different embodiments of a search result GUI.

[0010] FIG. 4 is a flow diagram that illustrates the method for updating a search result database when a user adds a search result to a search listing collection.

[0011] FIG. 5 illustrates a process for generating the search result GUI of FIG. 2 or FIG. 3.

[0012] FIG. 6 is a flow diagram that illustrates one method for generating search results.

[0013] FIGS. 7A-7E are flow diagrams that illustrate the method for adjusting the relevance score of a search result according to embodiments of the present invention.

[0014] FIG. 8 is a flow diagram that illustrates another method for generating search results.

DETAILED DESCRIPTION

[0015] FIG. 1 illustrates a networked computer environment in which the present invention may be practiced. As shown, the networked computer environment includes a plurality of client computers 110 (only two of which are shown), a plurality of web servers 120 with associated content storage units 125 (only two of which are shown), a search interface web server 130 with an associated content storage unit 135, and a search engine web server 140 with an associated content storage unit 145. The client computers 110, the web server computers 120, the search interface web server 130, and the search engine web server 140 are connected over a computer network 150, e.g., the Internet.

[0016] The search engine web server 140 may be operated by the same party that is operating the search interface web server 130. In some implementations, the functions of both the search interface web server 130 and the search engine web server 140 can be hosted on a single web server. The search engine web server 140 may also be operated by a different party, e.g., an Internet search services provider. As another alternative, the search engine functions may be provided by both the party that is operating the search interface web server 130 and an Internet search services provider.

[0017] Each client computer 110 includes conventional components of a computing device, e.g., a processor, system memory, a hard disk drive, input devices such as a mouse and a keyboard, and output devices such as a monitor (not shown). Each web server (which may be any of the web servers 120, the search interface web server 130, and the search engine web server 140) includes a processor and a system memory (not shown), and manages the contents stored in its respective content storage unit using a relational database software. The web server is programmed to communicate with the client computers 110 and other web servers using the TCP/IP protocol. The client computers 110 are programmed to execute web browser programs and access the web pages managed by the web servers by specifying a uniform resource locator (URL) for the web server into the browser.

[0018] In the embodiments of the present invention described below, users are respectively operating the client computers 110 that are connected to the search interface web server 130 over the Internet. The web pages that are displayed to a user are transmitted from the search interface web server 130 to that user's client computer 110 and processed by the web browser program stored in that user's client computer 110 for display through the monitor of that user's client computer 110.

[0019] FIG. 2 is a schematic representation of a search engine graphical user interface (GUI). This GUI is generated by the client computer 110 based on content transmitted by the search interface web server 130. The GUI schematically represented in FIG. 2 has four different sections. The first section 210 is a section for collecting search results into user-defined groups, referred to herein as "search listing collections" or "notebooks" and managing the notebooks. A list 201 of three such notebooks is shown in FIG. 2 inside the first section 210. A notebook may be shared with another user using the E-mail hyperlink or by specifying viewing and editing privileges for the other users using the Share hyperlink. A user who has viewing privileges to a notebook may import the notebook into his or her personal collection of notebooks. A user "subscribes" to a notebook by importing the notebook into his or her personal collection of notebooks. If the user also has editing privileges, the user can edit the notebook contents as if the notebook were his or her own. The second section 220 is a section for inputting a search query. The third section 230 is a section for displaying search results that are responsive to the search query. The fourth section 240 is a section for displaying text advertisements that are responsive to the search query.

[0020] Each of the four sections comprises at least one inline frame (also known as iframe). The first section 210 comprises one inline frame. The second section 220 comprises one inline frame. The third section 230 comprises multiple inline frames. Each search result (231a, 231b, 231c, 231 d) in the third section 230 is displayed in a separate inline frame. Some search results like the search result 231b and the search result 231c are notebooks that contain multiple pages. Notebooks are distinguished in the GUI from search results containing just an individual search listing using graphics that depict a stack of pages. Also, the displays for notebooks have navigation arrows 232, 233. The search listings from any one notebook are displayed one at a time. Other search listings within the same notebook can be accessed and displayed by clicking on the left arrow 232 to navigate backward through the notebook and on the right arrow 233 to navigate forward through the notebook. The fourth section 240 comprises one inline frame.

[0021] A GUI having inline frames permits any one of the inline frame to be updated with new content without disrupting the rest of the GUI. For example, a click on any of the following hyperlinks: Images, News, Blogs, and Y. Pages, within an inline frame of one of the search results 231 , which is interpreted as a request to search the Internet for images, news, blogs, or yellow pages, respectively, based on search query and the content of the search result currently displayed in the inline frame, causes only the inline frame to update without disrupting the rest of the GUI. Also, when the user clicks on the GUI controls 232, 233 appearing in the inline frame corresponding to search result 231b (or 231c), the display contents of this inline frame are updated without disrupting the rest of the GUI. [0022] FIG. 3 is a schematic representation of another embodiment of a search engine GUI. This GUI is generated by the client computer 110 based on content transmitted by the search interface web server 130. The GUI schematically represented in FIG. 3 has three different sections including a first section 310, a second section 320, and a third section 330. The first section 310, which shows one notebook title 301 , operates in the same manner as the first section 210 of FIG. 2, except that the first section 310 can be collapsed (or expanded after it has been collapsed) by clicking on the arrow 313. The drop region of the first section 310, identified with a darker background in FIG. 3, is active in either the expanded state or the collapsed state, so that search results can be added to a notebook by selecting them, dragging them to this region and then dropping them in this region. The second section 320 is a section for inputting a search query. The third section 330 is a section for displaying search results 331 that are responsive to the search query, and text advertisements (e.g., sponsored link 340) that are responsive to the search query. In the embodiment of the GUI shown in FIG. 3, a text advertisement appears directly underneath each notebook, such as under search result 331b.

[0023] Each of the three sections 310, 320, 330 comprises at least one inline frame. The first section 310 comprises one inline frame. The second section 320 comprises one inline frame. The third section 330 comprises multiple inline frames. Each search result 331 in the third section 330 is displayed in a separate inline frame, and each sponsored link 340 in the third section 330 is displayed in a separate inline frame. Some search results, such as search result 331b, may comprise notebooks. For these, the pages from the notebooks are displayed one at a time. Other pages of the notebooks can be accessed by clicking on GUI control for the previous page number (e.g., GUI control 332) or the next page number (e.g., GUI control 334). The page number of the current page is also displayed.

[0024] The GUI shown in FIG. 2 or FIG. 3 permits a user to create or add to a notebook by grabbing a search result using an input device and dropping it into input section 210 or 310, which functions as a virtual basket for collecting search results under various notebook titles. To keep track of these actions, the search interface web server 130 maintains a search result database. The configuration of this search result database is represented in a simplified form in Table 1. Each row of the search result database stores data about a particular search result and, at a minimum, the following data are stored in each row of this database: (1) the URL of the search result, (2) a running count of how many different notebooks contain this search result, and (3) a pointer to an array that stores the notebook IDs of the different notebooks that contain this search result.

Table 1

[0025] FIG. 4 is a flow diagram that illustrates the steps carried out by the search interface web server 130 to update the search results database when a user grabs a search result and drags and drops it into input section 210 or 310. In step 410, the URL of the search result and the notebook ID of the notebook that is currently opened in input section 210 or 310 are retrieved. A check is first made to see if there is any record of the retrieved URL in the search result database (step 412). This is done by accessing the array that stores IDs of notebooks that contain the search result using the pointer stored in the search result database and comparing the retrieved notebook ID with the array of notebook IDs. If there is, steps 414 to 418 are carried out. If there is none, the retrieved URL is added as a new entry in the search result database (step 420). in addition, the corresponding counter value is set to be 1, and an array is created in which the retrieved notebook ID is stored as the first entry (step 422). The pointer to the array is stored in the search result database.

[0026] Step 414 is executed when the retrieved URL has a record in the search result database. In this step, the retrieved notebook ID is compared against the notebook IDs of the different notebooks that contain this search result. If there is a match, the counter of this search result is not updated. If there is no match, the counter of this search result is incremented by one and the retrieved notebook ID is added to the array of notebook IDs of the different notebooks that contain this search result (step 416).

[0027] The search interface web server 130 also maintains popularity data for the different notebooks in a notebook database and reputation data for its users in a user database. The configuration of the notebook database is represented in a simplified form in Table 2 and the configuration of the user database is represented in a simplified form in Table 3.

Table 2

Table 3

[0028] Each row of the notebook database stores data about a particular notebook and, at a minimum, the following data are stored in each row of this database: (1) the notebook ID; (2) the notebook title; (3) a running count of the number of different users who are subscribed to this notebook; (4) a pointer to an array that stores the user IDs of the different subscribers; (5) number of pages in this notebook; (6) number of page views; and (7) number of click-throughs. A user is subscribed to a notebook if the user is the creator of the notebook or imports a notebook created by another user into his or her library of notebooks. [0029] Each row of the user database stores data about a user and, at a minimum, the following data are stored in each row of this database: (1) the user ID; (2) a running count of the number of notebooks that the user has created; and (3) a pointer to an array that stores the notebook IDs of the notebooks that the user has created.

[0030] FIG. 5 illustrates one process for generating the GUI shown schematically in FIG. 2 or FIG. 3. In the process illustrated in FIG. 5, the display contents for GUI 510, exclusive of the search results, are represented as HTML_bg, and the display contents of each of the search results provided in an inline frame are represented as HTMLn. The user input is represented as user_input.

[0031] The components of the client computer 110 shown in FIG. 5 include a processor 111 and a memory unit 112. A web browser program is stored in the memory unit 112 and executed by the processor 111 to manage the interactions between it and the search interface web server 130 over the Internet 150. The components of the search interface web server 130 shown in FIG. 5 include a processor 131 and a memory unit 132. A web server program is stored in the memory unit 132 and executed by the processor 131 to manage the interactions between it and the client computer 110 over the Internet 150. The memory unit 132 also stores a program executable by the processor 131 for interacting with the search engine web server 140, e.g., transmitting a search request and receiving search results in response thereto. The components of the search engine web server 140 shown in FIG. 5 include a processor 141 and a memory unit 142. A search engine program is stored in the memory unit 142 and executed by the processor 141 to carry out searches requested by the search interface web server 130.

[0032] FIG. 5 illustrates a GUI 510 at two different points in time. Initially, the GUI 510 has no search results displayed. Then, the user submits a search request, represented as userjnput, through the GUI 510. The client computer 110 passes on this search request to the search interface web server 130 over the Internet 150. The search interface web server 130 passes on this search request to the search engine web server 140 and receives search results in response thereto. The search results are retrieved in accordance with the relevance algorithm used by the search engine web server 140. The search interface web server 130 receives these search results and ranks the search results in accordance with embodiments of the present invention. Upon receiving this information, the client computer 110 generates the GUI 510 with new display contents for HTML_bg, and the search results displayed in separate inline frames, which are represented in FIG. 5 as HTML1 , HTML2, HTML3, and HTML4. In the implementation where the functions of the search interface web server 130 and the search engine web server 140 are carried out by a single web server, the relevance algorithm of the search engine is modified to incorporate the ranking methods described below.

[0033] FIG. 6 is a flow diagram that illustrates the steps carried out by the search interface web server 130 to generate search results according to an embodiment of the present invention. In step 610, the search interface web server 130 receives a search query from a client computer 110 over the Internet 150. Then, in step 612, the search interface web server 130 performs the requested search through the search engine web server 140. The search engine web server 140 generates the search results in a particular order and transmits them to the search interface web server 130, which reorders and stores them (step 614). The search interface web server 130 then generates a web page with the search results that have been re-ordered in step 614 (step 616), and transmits the web page to the client computer 110 over the Internet 150. The client computer 110 then generates a GUI that includes the search results received from the search interface web server 130, such as the one shown schematically in FIG. 2 or FIG. 3.

[0034] Embodiments of the present invention uses various data that are stored in the search result database, the notebook database and the user database to rank a set of search results (SR1 , SR2, SR3, ..., SRn) that have been determined, in accordance with various conventional methods, to be relevant to a search query that has been entered by a user. The ranking is carried out based on various adjustments in relevance scores of the search results (SR1, SR2, SR3, ..., SRn). If relevance scores of the search results (SR1 , SR2, SR3, ..., SRn) are not provided, the rank of the search results initially returned by the search engine web server 140 may be used as the relevance score. If the search results (SR1 , SR2, SR3, ..., SRn) are provided unranked or from a non-reputable source, all of the search results are assigned the same relevance score, e.g., 1.

[0035] According to a first embodiment, the relevance scores of search results (SR1 , SR2, SR3, ..., SRn) are adjusted based on any user-defined classifications of the search results. In one implementation, the user-defined classification of a search result may be a tag a user attaches to the search result. In another implementation, the user- defined classification of a search result may be the title of a notebook in which a user collects the search result. FIG. 7 A illustrates the steps carried out when adjusting the relevance score of each search result, SRi, in the set of search results (SR1 , SR2, SR3, ..., SRn) based on any user-defined classifications of the search result. In step 710, the search result database is examined to see if the search result, SRi, has any user-defined classifications. For each such user-defined classification, steps 712, 714 and 716 are executed. In step 712, the similarity between the user-defined classification and the search query is determined and scored. If the similarity is beyond a minimum threshold (step 714), the relevance score of the search result is increased (step 716).

[0036] According to a second embodiment, the relevance scores of search results (SR1 , SR2, SR3, ..., SRn) are adjusted based on the number of times they have been added to notebooks. FIG. 7B illustrates the steps carried out when adjusting the relevance score of each search result, SRi, in the set of search results (SR1 , SR2, SR3, ..., SRn) based on the number of times they have been added to notebooks. In step 720, the search result database is examined and the value corresponding to the number of different notebooks containing the search result, SRi, is retrieved. In step 722, the relevance score of the search result is increased in proportion to this value.

[0037] According to a third embodiment, the relevance scores of search results (SR1 ,

SR2, SR3 SRn) are adjusted based on the popularity of the notebooks that contain the search results. FIG. 7C illustrates the steps carried out when adjusting the relevance score of each search result, SRi, in the set of search results (SR1 , SR2, SR3, ..., SRn) based on the popularity of the notebooks that contain the search result. In step 730, the search result database is examined to see if the search result, SRi, is contained in any notebooks. For each such notebook, steps 732 and 734 are executed. The popularity score associated with the notebook is retrieved in step 732 and the relevance score of the search result is increased in proportion to the notebook's popularity score in step 734.

[0038] According to a fourth embodiment, the relevance scores of search results (SR1 , SR2, SR3, ..., SRn) are adjusted based on the popularity of the notebooks that contain the search results and the relevance of such notebooks to the search query. FIG. 7D illustrates the steps carried out when adjusting the relevance score of each search result, SRi, in the set of search results (SR1 , SR2, SR3, ..., SRn) based on the popularity of the notebooks that contain the search result and the relevance of such notebooks to the search query. In step 740, the search result database is examined to see if the search result, SRi, is contained in any notebooks. For each such notebook, steps 742, 744 and 746 are executed. In step 742, the popularity score associated with the notebook is retrieved. Then, in step 744, the relevance of the notebook to the search query is determined. In this step, the contextual relevance of the pages in the notebook is examined with respect to the search query in accordance with conventional methods. In step 746, the relevance score of the search result is increased in proportion to the notebook's popularity score and relevance.

[0039] According to a fifth embodiment, the relevance scores of search results (SR1 , SR2, SR3, .... SRn) are adjusted based on the popularity of the notebooks that contain the search results and the reputation of the subscribers to such notebooks. FIG. 7D illustrates the steps carried out when adjusting the relevance score of each search result, SRi, in the set of search results (SR1 , SR2, SR3, ..., SRn) based on the popularity of the notebooks that contain the search result and the reputation of the subscribers to such notebooks. In step 750, the search result database is examined to see if the search result, SRi, is contained in any notebooks. For each such notebook, steps 752, 754 and 756 are executed. In step 752, the popularity score associated with the notebook is retrieved. Then, in step 754, the popularity score is adjusted based on the reputation scores of subscribers to the notebook. In step 756, the relevance score of the search result is increased in proportion to the notebook's popularity score as may be adjusted in step 754.

[0040] In the embodiments of the present invention described above, the popularity score of a notebook is determined periodically and stored with the notebook in the notebook database, and the reputation score of a user is determined periodically and stored along with the user in the user database. The popularity score of a notebook is computed as a function of at least one of the following: the number of subscribers to the notebook, the reputation scores of the subscribers to the notebook, the number of pages in the notebook, the number of page views of the pages in the notebook, and the click-through rates of the pages in the notebook. The reputation score of a subscriber is computed as a function of the popularity scores of the notebooks that have been created by the subscriber.

[0041] FIG. 8 is a flow diagram that illustrates the steps carried out by the search interface web server 130 to generate search results. In step 810, the search interface web server 130 receives a search query from a client computer 110 over the Internet 150. Then, in step 812, the search interface web server 130 performs the requested search through the search engine web server 140. The search engine web server 140 generates the search results in a particular order and transmits them to the search interface web server 130, which stores them (step 814). The search interface web server 130 then searches for all notebooks that include any of the search results (step 816). If more than one notebook includes a search result, the notebook with the highest popularity score is selected for display in place of the search result (step 818). When this notebook is displayed, the notebook page corresponding to the search result will be displayed initially. In step 820, the web page having the display contents as modified in step 818 is transmitted to the client computer 110 over the Internet 150. The client computer 110 then generates a GUI that includes the display contents as modified in step 818, such as the one shown schematically in FIG. 2 or FIG. 3.

[0042] In another embodiment, a golden notebook may be generated in response to a search query and displayed to the user as the topmost search result. The golden notebook is a collection of relevant pages from notebooks that are determined to be the top N most relevant, where N can be any integer greater than 1 , e.g., 5, 10, or 20.

[0043] While particular embodiments according to the invention have been illustrated and described above, those skilled in the art understand that the invention can take a variety of forms and embodiments within the scope of the appended claims.

Claims

What is claimed is:

1. A method for generating a list of search results in response to a search query, comprising the steps of: receiving a search query; retrieving search results that are relevant to the search query; ranking the search results in part based on relevance of user-defined classifications of the search results to the search query; and transmitting the ranked search results for display.

2. The method according to claim 1 , wherein each search result has a relevance score and the relevance score of said each search result is adjusted based on the relevance of user-defined classifications of said each search result to the search query.

3. The method according to claim 2, wherein the relevance score of said each search result is further adjusted based on the number of times said each search result has been assigned a user-defined classification.

4. The method according to claim 3, wherein the user-defined classification comprises tags.

5. The method according to claim 3, wherein the user-defined classifications comprise titles of search listing collections to which one or more of the search results have been added.

6. A method for generating a list of search results in response to a search query, comprising the steps of: receiving a search query; retrieving search results that are relevant to the search query; ranking the search results in part based on the number of times the search results have been added to various search listing collections; and transmitting the ranked search results for display.

7. The method according to claim 6, wherein popularity scores are maintained for search listing collections and the step of ranking is based in part on the number of times the search results have been added to various search listing collections and the popularity scores of the search listing collections to which the search results have been added.

8. The method according to claim 7, further comprising the step of determining the relevance of search listing collections that contain the search results to the search query, wherein the step of ranking is based in part on the number of times the search results has been added to a search listing collection and the popularity scores and relevance of the search listing collections to which the search results have been added.

9. The method according to claim 7, wherein the popularity score of a search listing collection is correlated to the number of different subscribers to the search listing collection.

10. The method according to claim 9, wherein the popularity score of a search listing collection is further correlated to the number of pages in the search listing collection.

11. The method according to claim 10, wherein the popularity score of a search listing collection is further correlated to the number of page views or click-through rates of the search results within the search listing collection.

12. The method according to claim 7, wherein reputation scores are maintained for subscribers to search listing collections and the popularity score of a search listing collection is a function of the reputation scores of the subscribers to the search listing collection.

13. The method according to claim 12, wherein the reputation score of a subscriber is based on the popularity scores of the search listing collections created by the subscriber.

14. A method for maintaining a search database, comprising the steps of: associating a first search listing collection with a search result when the search result is added to the first search listing collection; associating a second search listing collection with a search result when the search result is added to the second search listing collection; and computing and storing a score of the first search listing collection and the second search listing collection, wherein the relevance of a search result to a search query is based in part on the computed scores of search listing collections that are associated with the search result and the relevance of the associated search listing collections to the search query.

15. The method according to claim 14, wherein the score comprises a popularity score.

16. The method according to claim 15, wherein the popularity score of a search listing collection is correlated to the number of different subscribers to the search listing collection.

17. The method according to claim 16, wherein the popularity score of a search listing collection is further correlated to the number of pages in the search listing collection.

18. The method according to claim 17, wherein the popularity score of a search listing collection is further correlated to the number of page views or click-through rates of the search results within the search listing collection.

19. The method according to claim 14, wherein reputation scores are maintained for subscribers to search listing collections and the popularity score of a search listing collection is a function of the reputation scores of the subscribers to the search listing collection.

20. The method according to claim 19, wherein the reputation score of a subscriber is based on the popularity scores of the search listing collections created by the subscriber.