US20100191744A1 - Ranking functions using document usage statistics - Google Patents
Ranking functions using document usage statistics Download PDFInfo
- Publication number
- US20100191744A1 US20100191744A1 US12/359,939 US35993909A US2010191744A1 US 20100191744 A1 US20100191744 A1 US 20100191744A1 US 35993909 A US35993909 A US 35993909A US 2010191744 A1 US2010191744 A1 US 2010191744A1
- Authority
- US
- United States
- Prior art keywords
- document
- documents
- actual
- usage
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99932—Access augmentation or optimizing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99937—Sorting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99938—Concurrency, e.g. lock management in shared database
Definitions
- Ranking functions that rank documents according to their relevance to a given search query are known. Efforts continue in the art to develop ranking functions that provide better search results for a given search query compared to search results generated by search engines using known ranking functions.
- the document relevance score is generated via a ranking function that comprises one or more query-independent components, wherein at least one query-independent component includes a usage parameter that takes into account actual document usage data maintained and stored on a web server for one or more documents on the network.
- the ranking functions may be used by a search engine to rank multiple documents in order (typically, in descending order) based on the document relevance scores of the multiple documents.
- FIG. 1 represents an exemplary logic flow diagram showing exemplary steps in a method of producing ranked search results in response to a search query inputted by a user;
- FIG. 2 is a block diagram of some of the primary components of an exemplary operating environment for implementation of the methods and processes disclosed herein;
- FIG. 3 represents a logic flow diagram showing exemplary steps in an exemplary method of determining document relevance scores for documents on a network
- FIG. 4 represents a logic flow diagram showing exemplary steps in a method of ranking search results generated using a ranking function containing a document usage parameter.
- Each document relevance score is calculated using a ranking function that desirably contains one or more query-independent components (e.g., a function component that that does not depend on a given search query or search query term), one or more query-dependent components (e.g., a function component that depends on the specifics of a given search query or search query term), or a combination thereof.
- the document relevance scores determined by the ranking function may be used to rank documents within a network space (e.g., a corporate intranet space) according to each document relevance score.
- An exemplary search process in which the disclosed methods may be used is shown as exemplary process 10 in FIG. 1 .
- FIG. 1 depicts exemplary search process 10 , which starts with process step 80 , wherein a user inputs a search query.
- exemplary search process 10 proceeds to step 200 , wherein a search engine searches all documents within a network space for one or more terms of the search query.
- exemplary search process 10 proceeds to step 300 , wherein a ranking function of the search engine ranks the documents within the network space based on the relevance score of each document, the document relevance score being based on one or more query-independent components, one or more query-dependent components, or a combination thereof.
- exemplary search process 10 proceeds to step 400 , wherein ranked search results are presented to the user, typically in descending order, identifying documents within the network space that are most relevant to the search query.
- At least one query-independent component of a ranking function used to determine a document relevance score takes into account “document usage data” or “document usage statistics” related to actual usage of one or more documents within a network space by one or more users.
- the document usage data and/or statistics is generated and stored by application code on a web server, which is separate from a given search engine.
- document usage data may be maintained by a web site so that each time a user requests a URL, the server updates a usage counter.
- the usage counter may maintain document-related data obtained for a given time interval, such as last week, last month, last year, or the lifetime of a given document or set of documents.
- Application code may be used to obtain the usage data from the web site via (i) a special application programming interface (API), (ii) a web service request, or (iii) by requesting an administration web page that returns usage data for every URL on the web site.
- API application programming interface
- Specific web sites may be used to generate and maintain usage data within a network space, as well as store the usage data in a local or remote storage system.
- Suitable web sites for generating, maintaining and storing usage data of documents within a network space include, but are not limited to, WINDOWS® SHAREPOINT® Services sites.
- the disclosed methods of determining a document relevance score may further utilize a ranking function that comprises one or more additional query-independent components.
- Suitable additional query-independent components include, but are not limited to, a query-independent component that takes into account a click distance for each document within a network space as described in U.S. patent application Ser. No. 10/955,983 entitled “SYSTEM AND METHOD FOR RANKING SEARCH RESULTS USING CLICK DISTANCE” filed on Aug. 30, 2004, a query-independent component that takes into account a biased click distance for each document within a network space as described in U.S. patent application Ser. No.
- the disclosed methods of determining a document relevance score utilizes a ranking function that comprises at least one query-independent component, which includes both the above-described document usage parameter and one or more of the above-described additional query-independent components.
- the document relevance score may be used to rank documents within a network space.
- a method of ranking documents on a network may comprise the steps of determining a document relevance score for each document on the network using the above-described method; and ranking the documents in a desired order (typically, in descending order) based on the document relevance scores of each document.
- the document relevance score may also be used to rank search results of a search query.
- a method of ranking search results of a search query may comprise the steps of determining a document relevance score for each document in the search results of a search query using the above-described method, and ranking the documents in a desired order (typically, in descending order) based on the document relevance scores of each document.
- Application programs using the methods disclosed herein may be loaded and executed on a variety of computer systems comprising a variety of hardware components.
- An exemplary computer system and exemplary operating environment for practicing the methods disclosed herein is described below.
- FIG. 2 illustrates an example of a suitable computing system environment 100 on which the methods disclosed herein may be implemented.
- the computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the methods disclosed herein. Neither should the computing environment 100 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment 100 .
- the methods disclosed herein are operational with numerous other general purpose or special purpose computing system environments or configurations.
- Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the methods disclosed herein include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
- program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
- the methods and processes disclosed herein may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both local and remote computer storage media including memory storage devices.
- an exemplary system for implementing the methods and processes disclosed herein includes a general purpose computing device in the form of a computer 110 .
- Components of computer 110 may include, but are not limited to, a processing unit 120 , a system memory 130 , and a system bus 121 that couples various system components including, but not limited to, system memory 130 to processing unit 120 .
- System bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.
- ISA Industry Standard Architecture
- MCA Micro Channel Architecture
- EISA Enhanced ISA
- VESA Video Electronics Standards Association
- PCI Peripheral Component Interconnect
- Computer 110 typically includes a variety of computer readable media.
- Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media.
- Computer readable media may comprise computer storage media and communication media.
- Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
- Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium, which can be used to store the desired information and which can be accessed by computer 110 .
- Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
- modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
- communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media as used herein.
- System memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132 .
- ROM read only memory
- RAM random access memory
- BIOS basic input/output system 133
- RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120 .
- FIG. 2 illustrates operating system 134 , application programs 135 , other program modules 136 , and program data 137 .
- Computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media.
- FIG. 2 illustrates a hard disk drive 140 that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 151 that reads from or writes to a removable, nonvolatile magnetic disk 152 , and an optical disk drive 155 that reads from or writes to a removable, nonvolatile optical disk 156 such as a CD ROM or other optical media.
- Hard disk drive 141 is typically connected to system bus 121 through a non-removable memory interface such as interface 140
- magnetic disk drive 151 and optical disk drive 155 are typically connected to system bus 121 by a removable memory interface, such as interface 150 .
- hard disk drive 141 is illustrated as storing operating system 144 , application programs 145 , other program modules 146 , and program data 147 . Note that these components can either be the same as or different from operating system 134 , application programs 135 , other program modules 136 , and program data 137 . Operating system 144 , application programs 145 , other program modules 146 , and program data 147 are given different numbers here to illustrate that, at a minimum, they are different copies.
- a user may enter commands and information into computer 110 through input devices such as a keyboard 162 and pointing device 161 , commonly referred to as a mouse, trackball or touch pad.
- Other input devices may include a microphone, joystick, game pad, satellite dish, scanner, or the like.
- processing unit 120 may be connected by a user input interface 160 that is coupled to system bus 121 , but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB).
- a monitor 191 or other type of display device is also connected to system bus 121 via an interface, such as a video interface 190 .
- computer 110 may also include other peripheral output devices such as speakers 197 and printer 196 , which may be connected through an output peripheral interface 195 .
- Computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180 .
- Remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer 110 , although only a memory storage device 181 has been illustrated in FIG. 2 .
- the logical connections depicted in FIG. 2 include a local area network (LAN) 171 and a wide area network (WAN) 173 , but may also include other networks.
- LAN local area network
- WAN wide area network
- Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
- computer 110 When used in a LAN networking environment, computer 110 is connected to LAN 171 through a network interface or adapter 170 .
- computer 110 When used in a WAN networking environment, computer 110 typically includes a modem 172 or other means for establishing communications over WAN 173 , such as the Internet.
- Modem 172 which may be internal or external, may be connected to system bus 121 via user input interface 160 , or other appropriate mechanism.
- program modules depicted relative to computer 110 may be stored in the remote memory storage device.
- FIG. 2 illustrates remote application programs 185 as residing on memory device 181 . It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
- Methods and processes disclosed herein may be implemented using one or more application programs including, but not limited to, a server system software application (e.g., WINDOWS SERVER SYSTEMTM software application), a search ranking application, and an application for generating, maintaining and storing usage data of documents within a network space (e.g., WINDOWS® SHAREPOINT® Services application), any one of which could be one of numerous application programs designated as application programs 135 , application programs 145 and remote application programs 185 in exemplary system 100 .
- a server system software application e.g., WINDOWS SERVER SYSTEMTM software application
- search ranking application e.g., a search ranking application
- an application for generating, maintaining and storing usage data of documents within a network space e.g., WINDOWS® SHAREPOINT® Services application
- the disclosed methods of generating a document relevance score for a given document may be implemented in other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, networked personal computers, minicomputers, mainframe computers, and the like.
- the disclosed methods of generating a document relevance score for a given document may also be practiced in distributed computing environments, where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both local and remote memory storage devices.
- the disclosed methods may rank a document on a network utilizing a ranking function that takes into account a document usage value of each document on the network.
- the disclosed methods of determining a document relevance score for a document on a network may comprise a number of steps.
- the method of determining a document relevance score for a document on a network comprises the steps of assigning an actual usage value (U A ) to one or more documents on a network comprising N documents, wherein the actual usage value (U A ) is based on actual usage data maintained and stored on a server; if less than N documents are assigned an actual usage value (U A ), assigning a default usage value (U D ) to the documents that do not have actual usage data associated therewith; and using the usage value (i.e., U A or U D ) for each document to determine the document relevance score of a given document on the network.
- actual usage data represents one or more types of data associated with the “usage” of the document by one or more users.
- Types of actual usage data for a given document or set of documents may include, but are not limited to, the number of document views by all users within a given period of time, the average number of document views per user within a given period of time, total time spent on a particular document within a given period of time, average time spent on a particular document within a given period of time, etc.
- the given period of time may be, for example, last week, last month, last year, the lifetime of the document, or any other desired period of time.
- the steps of generating, maintaining and storing document usage data or statistics for documents within a network space may be performed by application code commonly found on computing systems.
- Document usage data is generated, maintained and stored independently of a given search query or search engine, and is typically generated, maintained and stored by application code on the server that maintains the document (or page) and makes the document (or page) available to a user.
- Suitable application programs for generating, maintaining and storing document usage data or statistics include, but are not limited to, WINDOWS® SHAREPOINT® Services and other similar application programs.
- Document usage data stored and maintained on these service sites, as well as other web sites performing a similar function, may be accessed using application code as discussed above.
- document usage data may be accessed from a given web site (e.g., a WINDOWS® SHAREPOINT® Services site) via (i) a special application programming interface (API), (ii) a web service request, or (iii) by requesting an administration web page that returns usage data for every URL on the web site.
- a given web site e.g., a WINDOWS® SHAREPOINT® Services site
- API application programming interface
- the disclosed methods of determining a document relevance score for a document on a network may comprise a number of additional steps including, but not limited to, monitoring one or more documents within a network space for actual document usage; storing actual document usage data for one or more documents in a local or remote data storage file; calculating an actual usage value (U A ) for a document based on actual usage data for the document or a folder containing the document; storing actual usage values (U A ) for one or more documents in a local or remote data storage file; requesting stored document usage data or actual usage values (U A ) from a local or remote data storage file (e.g., a request for such data from a search engine after a specific search query by a user); retrieving actual document usage data or an actual usage value (U A ) for one or more documents from a local or remote data storage file; and optionally, merging a document usage value (i.e., actual or default) with one or more additional document properties to determine a document relevance score for a document.
- FIG. 3 represents a logic flow diagram showing exemplary steps in an exemplary method of providing actual or default usage values for documents on a network followed by an optional downgrading/upgrading procedure by a system administrator.
- exemplary method 401 starts at block 402 and proceeds to step 403 .
- step 403 a first document on the network is crawled for actual usage data.
- the step of crawling a first document for actual usage data may be performed using a crawler application capable of determining whether the first document has any actual usage data associated therewith, and if the first document has actual usage data associated therewith, retrieving the actual usage data.
- Suitable crawler applications for use in the disclosed methods of providing actual or default usage values for documents on a network include, but are not limited to, crawler applications described in U.S. Pat. Nos. 6,463,455 and 6,631,369, the subject matter of both of which is hereby incorporated in its entirety by reference.
- the actual usage data may be obtained from one or more files that store actual usage data for one or more documents on a network.
- the actual usage data may be stored, along with the document, as a document component, or may be stored in a data storage file separate from the actual document.
- Suitable remote storage systems include, but are not limited to, WINDOWS® SHAREPOINT® Services (WSS) products commercially available from Microsoft Corporation (Redmond, Wash.), as well as any other similar remote storage system.
- WSS remote storage system records actual usage data including, for example, the number of requests to every document on a given network across all users, and produces statistics of number of clicks per document during the last week, the last month, the last year, or the overall lifetime of the document, or any other period of time.
- the methods disclosed herein are not limited to a WSS remote storage system, but may utilize a WSS remote storage system or any other similar document data system in the disclosed methods.
- exemplary method 401 proceeds to decision block 404 .
- decision block 404 a determination is made by application code as to whether the document has actual usage data associated therewith. If a decision is made that the document has actual usage data associated therewith, exemplary method 401 proceeds to step 405 , wherein a usage value based on actual usage (U A ) is assigned to the document.
- the actual usage value (U A ) may be determined using one or more components of the actual usage data associated with the document. For example, in some embodiments, the actual usage value (U A ) assigned to the document may be related only to the number of users viewing the document.
- the actual usage value (U A ) assigned to the document may be related to the number of document views by all users within a given period of time, the average number of document views per user within a given period of time, total time spent on a particular document within a given period of time, average time spent on a particular document within a given period of time, or a combination of any of the above criteria, wherein the given period of time comprises last week, last month, last year, the lifetime of the document, or any other desired period of time.
- the actual usage data associated with a given document suggests that the document was not used or viewed during a given time period.
- the document could be assigned a usage value (U A ) equal to zero to indicate no usage during the time period; however, typically, usage values (U A ) based on actual use or no actual use are assigned a number other than zero.
- actual usage data may be associated with a set of documents as oppose to individual documents.
- a folder may contain a set of documents, and an associated server may only track usage data related to accessing (i.e., use of) the folder, and not the individual documents within the folder.
- a usage value (U A ) may be provided for each document within the folder based on the actual usage data of the folder.
- each usage value (U A ) will be the same for each document within the folder; however, different usage values (U A ) may be assigned to different documents within a folder if so desired.
- exemplary method 401 proceeds to decision block 406 described below.
- exemplary method 401 proceeds to step 407 , wherein a default usage value (U D ) is assigned to the document.
- a default usage value (U D ) may be assigned to a document that is part of a web site that does not maintain document usage data.
- the default usage value (U D ) assigned to the document may be used to provide an initial importance to the document relative to documents having actual usage data. For example, if a higher usage value for a given document indicates relative importance of the document within the network, assigning a lower default usage value (U D ) to the document downgrades the importance of the document relative to other documents on the network.
- the default usage value (U D ) assigned to the document may be relative to actual usage values (U A ) assigned to other documents on the network. For example, in order to lower the relative importance of the document, a default usage value (U D ) may be assigned to the document, wherein the default usage value (U D ) is less than any actual usage value (U A ) assigned to other documents on the network as described above.
- a default usage value (U D ) may be assigned to the document, wherein the default usage value (U D ) is greater than any actual usage value (U A ) assigned to other documents on the network or greater than some of the actual usage values (U A ) assigned to some of the other documents on the network.
- a default usage value (U D ) may be assigned to a document without actual usage data so that the document is given an average relative importance compared to documents having an assigned actual usage value (U A ).
- default usage values (U D ) for documents without actual usage data may range from a minimum assigned actual usage value (U Amin ) to a maximum assigned actual usage value (U Amax ), or be within a specific range between the minimum assigned actual usage value (U Amin ) and the maximum assigned actual usage value (U Amax ).
- documents without actual usage data are provided with an average relative importance, suggesting medium usage, compared to documents that have actual usage data associated therewith.
- exemplary method 401 proceeds to decision block 406 .
- decision block 406 a determination is made by application code as to whether all of the documents on a network have an actual (U A ) or default (U D ) usage value. If a decision is made that all of the documents on a network do not have an actual (U A ) or default (U D ) usage value, exemplary method 401 proceeds to step 408 , wherein the next document is crawled for actual usage data. From step 408 , exemplary method 401 returns to decision block 404 and proceeds as discussed above.
- exemplary method 401 proceeds to decision block 409 .
- decision block 409 a determination is made by a system administrator whether to downgrade any actual (U A ) or default (U D ) usage values in order to more closely represent the importance of a given document within a network space.
- exemplary method 401 proceeds to step 410 , wherein the actual (U A ) or default (U D ) usage value of one or more documents (or URLs) are adjusted either negatively or positively. From step 410 , exemplary method 401 proceeds to step 411 described below.
- step 411 the actual (U A ) and default (U D ) usage values are utilized in a ranking function to determine an overall document relevance score for each document within a network space. From step 411 , exemplary method 401 proceeds to end block 412 .
- the actual (U A ) or default (U D ) usage value for each document may be used as a parameter in a ranking function to provide a document relevance score for each document.
- a document relevance score may be used to rank search results of a search query.
- An exemplary method of ranking search results generated using a ranking function containing a document usage value parameter is shown in FIG. 4 .
- FIG. 4 provides a logic flow diagram showing exemplary steps in exemplary method 20 , wherein exemplary method 20 comprises a method of ranking search results generating using a ranking function containing a usage value parameter.
- exemplary method 20 starts at block 201 and proceeds to step 202 .
- step 202 a user requests a search by inputting a search query.
- step 202 actual or default usage values for each of the documents on the network have previously been calculated.
- exemplary method 20 proceeds to step 203 .
- step 203 the actual or default usage value for each document on a network is merged with any other document statistics (e.g., other query-independent statistics) for each document stored in the index. Merging the actual or default usage values with other document statistics allows for a faster query response time since all the information related to ranking is clustered together. Accordingly, each document listed in the index has an associated actual or default usage value after the merge.
- exemplary method 20 proceeds to step 204 .
- query-independent document statistics for a given document are provided as a component of a ranking function.
- Query-dependent data is also provided for the given document, typically as a separate component of the ranking function.
- the query-dependent data or content-related portion of the ranking function depends on the actual search terms and the content of the given document.
- the ranking function comprises at least one query-independent (QID) component comprises a usage parameter.
- the query-independent (QID) component may be represented by the following equation:
- the query-independent (QID) component may be represented by the following equation:
- the query-independent (QID) component may be represented by the following equation:
- U represents an actual usage value or a default usage value
- w u , k u , B and C represent tuning parameters (i.e., scalar constants) for the usage value.
- the ranking function comprises a sum of the above-described query-independent (QID) component and at least one query-dependent (QD) component, such as
- the QD component can be any document scoring function.
- the QD component corresponds to a field weighted scoring function described in U.S. patent application Ser. No. 10/804,326 entitled “FIELD WEIGHTING IN TEXT DOCUMENT SEARCHING,” filed on Mar. 18, 2004, the subject matter of which is hereby incorporated in its entirety by reference.
- U.S. patent application Ser. No. 10/804,326 one equation that may be used as a representation of the field weighted scoring function is as follows:
- wtf′ represents a weighted term frequency or sum of term frequencies of given terms in the search query multiplied by weights across all fields (e.g., the title, the body, etc. of the document) and normalized according to the length of each field and the corresponding average length
- N represents a number of documents on the network
- n represents a number of documents containing a query term
- k 1 is a tunable constant.
- the ranking function may further comprise a QID component that takes into account (i) a click distance value as determined by the methods disclosed in U.S. patent application Ser. No. 10/955,983 entitled “SYSTEM AND METHOD FOR RANKING SEARCH RESULTS USING CLICK DISTANCE” filed on Aug. 30, 2004, (ii) a biased click distance value as determined by the methods disclosed in U.S. patent application Ser. No. 11/206,286 entitled “RANKING FUNCTIONS USING A BIASED CLICK DISTANCE OF A DOCUMENT ON A NETWORK” filed on Aug.
- this optional additional QID component may comprise a function as follows:
- W cd represents a weight of a query-independent component such as a component containing a click distance or biased click distance parameter
- b cd represents a weight of a click distance or biased click distance relative to the URL depth
- b ud represents a weight of a URL depth
- CD represents a computed or assigned click distance or biased click distance for a document
- k ew represents a tuning constant that is determined by optimizing the precision of the ranking function, similar to other tuning parameters (i.e., k ew may represent the edge weight value when all edges have the same edge weight value, or k ew may represent the average or mean edge value when edge weight values differ from one another)
- UD represents a URL depth
- k cd is the click distance saturation constant.
- the weighted terms assist in defining the importance of each of their related terms (i.e., the component containing a click distance or biased click distance parameter, the click distance or biased click distance value for a given document, and the URL depth of the given document respectively) and ultimately the outcome of the scoring function.
- the URL depth is an optional addition to the above-referenced query-independent component to smooth the effect that the click distance or biased click distance value may have on the scoring function.
- a document that is not very important i.e., has a large URL depth
- the URL depth is represented by the number of slashes in a document's URL. For example, www.example.com ⁇ d1 ⁇ d2 ⁇ d3 ⁇ d4.htm includes four slashes and would therefore have a URL depth of 4. This document however, may have a link directly from the main page www.example.com giving it a relatively low click distance or biased click distance value.
- Including the URL depth term in the above-referenced function and weighting the URL depth term against the click distance or biased click distance value compensates for a relatively high click distance or biased click distance value to more accurately reflect the document's importance within the network.
- a URL depth of 3 or more may be considered a deep link.
- the ranking function used to determine a document relevance score for a given document comprises a function as follows:
- the URL depth may be removed from the ranking function or other components may be added to the ranking function to improve the accuracy of the query-dependent component, the query-independent component, or both.
- the above-described query-independent component containing a usage parameter may be incorporated into other ranking functions (not shown) to improve ranking of search results.
- step 205 a document relevance score is determined for a given document, stored in memory, and associated with the given document. From step 205 , exemplary method 20 proceeds to decision block 206 .
- step 207 the search results of the query comprising numerous documents are ranked according to their associated document relevance scores.
- the resulting document relevance scores take into account the actual or default usage value of each of the documents within the network.
- An exemplary computing system contains at least one application module usable on the computing system, wherein the at least one application module comprises application code loaded thereon, wherein the application code performs a method of generating a document relevance score for documents within a network.
- the application code may be loaded onto the computing system using any of the above-described computer readable medium having thereon computer-executable instructions for generating a document relevance score for documents within a network and using document relevance scores to rank search results of a search query as described above.
Abstract
Description
- This application claims priority to U.S. patent application Ser. No., filed, and incorporated herein by reference thereto. See also paragraph [0016] below for additional related applications, to which no priority is claimed.
- Ranking functions that rank documents according to their relevance to a given search query are known. Efforts continue in the art to develop ranking functions that provide better search results for a given search query compared to search results generated by search engines using known ranking functions.
- The following summary is included only to introduce some concepts discussed in the Detailed Description below. This summary is not comprehensive and is not intended to delineate the scope of the claimed subject matter, which is set forth by the claims presented at the end.
- Described herein are, among other things, various technologies for determining a document relevance score for a given document on a network. The document relevance score is generated via a ranking function that comprises one or more query-independent components, wherein at least one query-independent component includes a usage parameter that takes into account actual document usage data maintained and stored on a web server for one or more documents on the network. The ranking functions may be used by a search engine to rank multiple documents in order (typically, in descending order) based on the document relevance scores of the multiple documents.
- Many of the attendant features will be explained below with reference to the following detailed description considered in connection with the accompanying drawings.
- The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein like reference numerals are used to designate like parts in the accompanying description.
-
FIG. 1 represents an exemplary logic flow diagram showing exemplary steps in a method of producing ranked search results in response to a search query inputted by a user; -
FIG. 2 is a block diagram of some of the primary components of an exemplary operating environment for implementation of the methods and processes disclosed herein; -
FIG. 3 represents a logic flow diagram showing exemplary steps in an exemplary method of determining document relevance scores for documents on a network; and -
FIG. 4 represents a logic flow diagram showing exemplary steps in a method of ranking search results generated using a ranking function containing a document usage parameter. - To promote an understanding of the principles of the methods and processes disclosed herein, descriptions of specific embodiments follow and specific language is used to describe the specific embodiments. It will nevertheless be understood that no limitation of the scope of the disclosed methods and processes is intended by the use of specific language. Alterations, further modifications, and such further applications of the principles of the disclosed methods and processes discussed are contemplated as would normally occur to one ordinarily skilled in the art to which the disclosed methods and processes pertains.
- Methods of determining a document relevance score for documents on a network are disclosed. Each document relevance score is calculated using a ranking function that desirably contains one or more query-independent components (e.g., a function component that that does not depend on a given search query or search query term), one or more query-dependent components (e.g., a function component that depends on the specifics of a given search query or search query term), or a combination thereof. The document relevance scores determined by the ranking function may be used to rank documents within a network space (e.g., a corporate intranet space) according to each document relevance score. An exemplary search process in which the disclosed methods may be used is shown as
exemplary process 10 inFIG. 1 . -
FIG. 1 depictsexemplary search process 10, which starts withprocess step 80, wherein a user inputs a search query. Fromstep 80,exemplary search process 10 proceeds tostep 200, wherein a search engine searches all documents within a network space for one or more terms of the search query. Fromstep 200,exemplary search process 10 proceeds tostep 300, wherein a ranking function of the search engine ranks the documents within the network space based on the relevance score of each document, the document relevance score being based on one or more query-independent components, one or more query-dependent components, or a combination thereof. Fromstep 300,exemplary search process 10 proceeds tostep 400, wherein ranked search results are presented to the user, typically in descending order, identifying documents within the network space that are most relevant to the search query. - As discussed in more detail below, in some exemplary methods of determining a document relevance score, at least one query-independent component of a ranking function used to determine a document relevance score takes into account “document usage data” or “document usage statistics” related to actual usage of one or more documents within a network space by one or more users. The document usage data and/or statistics is generated and stored by application code on a web server, which is separate from a given search engine. For example, document usage data may be maintained by a web site so that each time a user requests a URL, the server updates a usage counter. The usage counter may maintain document-related data obtained for a given time interval, such as last week, last month, last year, or the lifetime of a given document or set of documents. Application code may be used to obtain the usage data from the web site via (i) a special application programming interface (API), (ii) a web service request, or (iii) by requesting an administration web page that returns usage data for every URL on the web site.
- Specific web sites may be used to generate and maintain usage data within a network space, as well as store the usage data in a local or remote storage system. Suitable web sites for generating, maintaining and storing usage data of documents within a network space include, but are not limited to, WINDOWS® SHAREPOINT® Services sites.
- The disclosed methods of determining a document relevance score may further utilize a ranking function that comprises one or more additional query-independent components. Suitable additional query-independent components include, but are not limited to, a query-independent component that takes into account a click distance for each document within a network space as described in U.S. patent application Ser. No. 10/955,983 entitled “SYSTEM AND METHOD FOR RANKING SEARCH RESULTS USING CLICK DISTANCE” filed on Aug. 30, 2004, a query-independent component that takes into account a biased click distance for each document within a network space as described in U.S. patent application Ser. No. 11/206,286 entitled “RANKING FUNCTIONS USING A BIASED CLICK DISTANCE OF A DOCUMENT ON A NETWORK” filed on Aug. 15, 2005, and a query-independent component that takes into account the URL for each document within a network space as described in U.S. patent application Ser. No. 10/955,983 entitled “SYSTEM AND METHOD FOR RANKING SEARCH RESULTS USING CLICK DISTANCE” filed on Aug. 30, 2004. The subject matter of each of the above-mentioned U.S. patent applications, which are assigned to the assignee of the present patent application, is hereby incorporated by reference in its entirety.
- In yet a further exemplary embodiment, the disclosed methods of determining a document relevance score utilizes a ranking function that comprises at least one query-independent component, which includes both the above-described document usage parameter and one or more of the above-described additional query-independent components.
- The document relevance score may be used to rank documents within a network space. For example, a method of ranking documents on a network may comprise the steps of determining a document relevance score for each document on the network using the above-described method; and ranking the documents in a desired order (typically, in descending order) based on the document relevance scores of each document.
- The document relevance score may also be used to rank search results of a search query. For example, a method of ranking search results of a search query may comprise the steps of determining a document relevance score for each document in the search results of a search query using the above-described method, and ranking the documents in a desired order (typically, in descending order) based on the document relevance scores of each document.
- Application programs using the methods disclosed herein may be loaded and executed on a variety of computer systems comprising a variety of hardware components. An exemplary computer system and exemplary operating environment for practicing the methods disclosed herein is described below.
- Exemplary Operating Environment
-
FIG. 2 illustrates an example of a suitablecomputing system environment 100 on which the methods disclosed herein may be implemented. Thecomputing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the methods disclosed herein. Neither should thecomputing environment 100 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in theexemplary operating environment 100. - The methods disclosed herein are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the methods disclosed herein include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
- The methods and processes disclosed herein may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The methods and processes disclosed herein may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
- With reference to
FIG. 2 , an exemplary system for implementing the methods and processes disclosed herein includes a general purpose computing device in the form of acomputer 110. Components ofcomputer 110 may include, but are not limited to, aprocessing unit 120, asystem memory 130, and asystem bus 121 that couples various system components including, but not limited to,system memory 130 toprocessing unit 120.System bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus. -
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed bycomputer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium, which can be used to store the desired information and which can be accessed bycomputer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media as used herein. -
System memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS) containing the basic routines that help to transfer information between elements withincomputer 110, such as during start-up, is typically stored inROM 131.RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processingunit 120. By way of example, and not limitation,FIG. 2 illustratesoperating system 134,application programs 135,other program modules 136, andprogram data 137. -
Computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,FIG. 2 illustrates ahard disk drive 140 that reads from or writes to non-removable, nonvolatile magnetic media, amagnetic disk drive 151 that reads from or writes to a removable, nonvolatilemagnetic disk 152, and anoptical disk drive 155 that reads from or writes to a removable, nonvolatileoptical disk 156 such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like.Hard disk drive 141 is typically connected tosystem bus 121 through a non-removable memory interface such asinterface 140, andmagnetic disk drive 151 andoptical disk drive 155 are typically connected tosystem bus 121 by a removable memory interface, such asinterface 150. - The drives and their associated computer storage media discussed above and illustrated in
FIG. 2 provide storage of computer readable instructions, data structures, program modules and other data forcomputer 110. InFIG. 2 , for example,hard disk drive 141 is illustrated as storingoperating system 144,application programs 145,other program modules 146, andprogram data 147. Note that these components can either be the same as or different fromoperating system 134,application programs 135,other program modules 136, andprogram data 137.Operating system 144,application programs 145,other program modules 146, andprogram data 147 are given different numbers here to illustrate that, at a minimum, they are different copies. - A user may enter commands and information into
computer 110 through input devices such as akeyboard 162 andpointing device 161, commonly referred to as a mouse, trackball or touch pad. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected toprocessing unit 120 through auser input interface 160 that is coupled tosystem bus 121, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). Amonitor 191 or other type of display device is also connected tosystem bus 121 via an interface, such as avideo interface 190. In addition to monitor 191,computer 110 may also include other peripheral output devices such asspeakers 197 andprinter 196, which may be connected through an outputperipheral interface 195. -
Computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as aremote computer 180.Remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative tocomputer 110, although only amemory storage device 181 has been illustrated inFIG. 2 . The logical connections depicted inFIG. 2 include a local area network (LAN) 171 and a wide area network (WAN) 173, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet. - When used in a LAN networking environment,
computer 110 is connected toLAN 171 through a network interface oradapter 170. When used in a WAN networking environment,computer 110 typically includes amodem 172 or other means for establishing communications over WAN 173, such as the Internet.Modem 172, which may be internal or external, may be connected tosystem bus 121 viauser input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative tocomputer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,FIG. 2 illustratesremote application programs 185 as residing onmemory device 181. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used. - Methods and processes disclosed herein may be implemented using one or more application programs including, but not limited to, a server system software application (e.g., WINDOWS SERVER SYSTEM™ software application), a search ranking application, and an application for generating, maintaining and storing usage data of documents within a network space (e.g., WINDOWS® SHAREPOINT® Services application), any one of which could be one of numerous application programs designated as
application programs 135,application programs 145 andremote application programs 185 inexemplary system 100. - As mentioned above, those skilled in the art will appreciate that the disclosed methods of generating a document relevance score for a given document may be implemented in other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, networked personal computers, minicomputers, mainframe computers, and the like. The disclosed methods of generating a document relevance score for a given document may also be practiced in distributed computing environments, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
- As discussed above, methods of determining a document relevance score for a document on a network are provided. The disclosed methods may rank a document on a network utilizing a ranking function that takes into account a document usage value of each document on the network.
- The disclosed methods of determining a document relevance score for a document on a network may comprise a number of steps. In one exemplary embodiment, the method of determining a document relevance score for a document on a network comprises the steps of assigning an actual usage value (UA) to one or more documents on a network comprising N documents, wherein the actual usage value (UA) is based on actual usage data maintained and stored on a server; if less than N documents are assigned an actual usage value (UA), assigning a default usage value (UD) to the documents that do not have actual usage data associated therewith; and using the usage value (i.e., UA or UD) for each document to determine the document relevance score of a given document on the network.
- As used herein, the term “actual usage data” represents one or more types of data associated with the “usage” of the document by one or more users. Types of actual usage data for a given document or set of documents may include, but are not limited to, the number of document views by all users within a given period of time, the average number of document views per user within a given period of time, total time spent on a particular document within a given period of time, average time spent on a particular document within a given period of time, etc. The given period of time may be, for example, last week, last month, last year, the lifetime of the document, or any other desired period of time.
- The steps of generating, maintaining and storing document usage data or statistics for documents within a network space may be performed by application code commonly found on computing systems. Document usage data is generated, maintained and stored independently of a given search query or search engine, and is typically generated, maintained and stored by application code on the server that maintains the document (or page) and makes the document (or page) available to a user. Suitable application programs for generating, maintaining and storing document usage data or statistics include, but are not limited to, WINDOWS® SHAREPOINT® Services and other similar application programs.
- Document usage data stored and maintained on these service sites, as well as other web sites performing a similar function, may be accessed using application code as discussed above. For example, document usage data may be accessed from a given web site (e.g., a WINDOWS® SHAREPOINT® Services site) via (i) a special application programming interface (API), (ii) a web service request, or (iii) by requesting an administration web page that returns usage data for every URL on the web site.
- The disclosed methods of determining a document relevance score for a document on a network may comprise a number of additional steps including, but not limited to, monitoring one or more documents within a network space for actual document usage; storing actual document usage data for one or more documents in a local or remote data storage file; calculating an actual usage value (UA) for a document based on actual usage data for the document or a folder containing the document; storing actual usage values (UA) for one or more documents in a local or remote data storage file; requesting stored document usage data or actual usage values (UA) from a local or remote data storage file (e.g., a request for such data from a search engine after a specific search query by a user); retrieving actual document usage data or an actual usage value (UA) for one or more documents from a local or remote data storage file; and optionally, merging a document usage value (i.e., actual or default) with one or more additional document properties to determine a document relevance score for a document.
-
FIG. 3 represents a logic flow diagram showing exemplary steps in an exemplary method of providing actual or default usage values for documents on a network followed by an optional downgrading/upgrading procedure by a system administrator. As shown inFIG. 3 ,exemplary method 401 starts atblock 402 and proceeds to step 403. Instep 403, a first document on the network is crawled for actual usage data. - The step of crawling a first document for actual usage data (step 403) may be performed using a crawler application capable of determining whether the first document has any actual usage data associated therewith, and if the first document has actual usage data associated therewith, retrieving the actual usage data. Suitable crawler applications for use in the disclosed methods of providing actual or default usage values for documents on a network include, but are not limited to, crawler applications described in U.S. Pat. Nos. 6,463,455 and 6,631,369, the subject matter of both of which is hereby incorporated in its entirety by reference.
- As discussed above, the actual usage data may be obtained from one or more files that store actual usage data for one or more documents on a network. The actual usage data may be stored, along with the document, as a document component, or may be stored in a data storage file separate from the actual document. Suitable remote storage systems include, but are not limited to, WINDOWS® SHAREPOINT® Services (WSS) products commercially available from Microsoft Corporation (Redmond, Wash.), as well as any other similar remote storage system. For example, the WSS remote storage system records actual usage data including, for example, the number of requests to every document on a given network across all users, and produces statistics of number of clicks per document during the last week, the last month, the last year, or the overall lifetime of the document, or any other period of time. Further, as noted above, it should be understood that the methods disclosed herein are not limited to a WSS remote storage system, but may utilize a WSS remote storage system or any other similar document data system in the disclosed methods.
- Once the document is crawled,
exemplary method 401 proceeds todecision block 404. Atdecision block 404, a determination is made by application code as to whether the document has actual usage data associated therewith. If a decision is made that the document has actual usage data associated therewith,exemplary method 401 proceeds to step 405, wherein a usage value based on actual usage (UA) is assigned to the document. The actual usage value (UA) may be determined using one or more components of the actual usage data associated with the document. For example, in some embodiments, the actual usage value (UA) assigned to the document may be related only to the number of users viewing the document. In other embodiments, the actual usage value (UA) assigned to the document may be related to the number of document views by all users within a given period of time, the average number of document views per user within a given period of time, total time spent on a particular document within a given period of time, average time spent on a particular document within a given period of time, or a combination of any of the above criteria, wherein the given period of time comprises last week, last month, last year, the lifetime of the document, or any other desired period of time. - In some cases, the actual usage data associated with a given document suggests that the document was not used or viewed during a given time period. In such a case, the document could be assigned a usage value (UA) equal to zero to indicate no usage during the time period; however, typically, usage values (UA) based on actual use or no actual use are assigned a number other than zero.
- Further, in some cases, actual usage data may be associated with a set of documents as oppose to individual documents. For example, a folder may contain a set of documents, and an associated server may only track usage data related to accessing (i.e., use of) the folder, and not the individual documents within the folder. In this embodiment, if there is actual usage data associated with a folder, a usage value (UA) may be provided for each document within the folder based on the actual usage data of the folder. Typically, each usage value (UA) will be the same for each document within the folder; however, different usage values (UA) may be assigned to different documents within a folder if so desired.
- From
step 405,exemplary method 401 proceeds to decision block 406 described below. - Returning to decision block 404, if a decision is made that the document does not have actual usage data associated therewith,
exemplary method 401 proceeds to step 407, wherein a default usage value (UD) is assigned to the document. For example, a default usage value (UD) may be assigned to a document that is part of a web site that does not maintain document usage data. The default usage value (UD) assigned to the document may be used to provide an initial importance to the document relative to documents having actual usage data. For example, if a higher usage value for a given document indicates relative importance of the document within the network, assigning a lower default usage value (UD) to the document downgrades the importance of the document relative to other documents on the network. - In one exemplary embodiment wherein a higher usage value for a given document indicates relative importance of the document within the network, the default usage value (UD) assigned to the document may be relative to actual usage values (UA) assigned to other documents on the network. For example, in order to lower the relative importance of the document, a default usage value (UD) may be assigned to the document, wherein the default usage value (UD) is less than any actual usage value (UA) assigned to other documents on the network as described above. If it is desired to increase the relative importance of the document, a default usage value (UD) may be assigned to the document, wherein the default usage value (UD) is greater than any actual usage value (UA) assigned to other documents on the network or greater than some of the actual usage values (UA) assigned to some of the other documents on the network.
- In other embodiments, a default usage value (UD) may be assigned to a document without actual usage data so that the document is given an average relative importance compared to documents having an assigned actual usage value (UA). For example, in this embodiment, default usage values (UD) for documents without actual usage data may range from a minimum assigned actual usage value (UAmin) to a maximum assigned actual usage value (UAmax), or be within a specific range between the minimum assigned actual usage value (UAmin) and the maximum assigned actual usage value (UAmax). In this embodiment, documents without actual usage data are provided with an average relative importance, suggesting medium usage, compared to documents that have actual usage data associated therewith.
- From
step 407,exemplary method 401 proceeds todecision block 406. Atdecision block 406, a determination is made by application code as to whether all of the documents on a network have an actual (UA) or default (UD) usage value. If a decision is made that all of the documents on a network do not have an actual (UA) or default (UD) usage value,exemplary method 401 proceeds to step 408, wherein the next document is crawled for actual usage data. Fromstep 408,exemplary method 401 returns to decision block 404 and proceeds as discussed above. - Returning to decision block 406, if a determination is made by application code that all documents on the network have an actual (UA) or default (UD) usage value,
exemplary method 401 proceeds todecision block 409. Atdecision block 409, a determination is made by a system administrator whether to downgrade any actual (UA) or default (UD) usage values in order to more closely represent the importance of a given document within a network space. If a decision is made to downgrade one or more actual (UA) or default (UD) usage values in order to more closely represent the importance of one or more documents within a network space,exemplary method 401 proceeds to step 410, wherein the actual (UA) or default (UD) usage value of one or more documents (or URLs) are adjusted either negatively or positively. Fromstep 410,exemplary method 401 proceeds to step 411 described below. - Returning to decision block 409, if a decision is made not to downgrade (or upgrade) one or more actual (UA) or default (UD) usage values,
exemplary method 401 proceeds directly to step 411. Instep 411, the actual (UA) and default (UD) usage values are utilized in a ranking function to determine an overall document relevance score for each document within a network space. Fromstep 411,exemplary method 401 proceeds to endblock 412. - Once all actual (UA) and default (UD) usage values have been determined and optionally downgraded (or optionally upgraded), if so desired, the actual (UA) or default (UD) usage value for each document may be used as a parameter in a ranking function to provide a document relevance score for each document. Such a document relevance score may be used to rank search results of a search query. An exemplary method of ranking search results generated using a ranking function containing a document usage value parameter is shown in
FIG. 4 . -
FIG. 4 provides a logic flow diagram showing exemplary steps inexemplary method 20, whereinexemplary method 20 comprises a method of ranking search results generating using a ranking function containing a usage value parameter. As shown inFIG. 4 ,exemplary method 20 starts atblock 201 and proceeds to step 202. Instep 202, a user requests a search by inputting a search query. Prior to step 202, actual or default usage values for each of the documents on the network have previously been calculated. Fromstep 202,exemplary method 20 proceeds to step 203. - In
step 203, the actual or default usage value for each document on a network is merged with any other document statistics (e.g., other query-independent statistics) for each document stored in the index. Merging the actual or default usage values with other document statistics allows for a faster query response time since all the information related to ranking is clustered together. Accordingly, each document listed in the index has an associated actual or default usage value after the merge. Once the merge is complete,exemplary method 20 proceeds to step 204. - In
step 204, query-independent document statistics for a given document, including a usage parameter, are provided as a component of a ranking function. Query-dependent data is also provided for the given document, typically as a separate component of the ranking function. The query-dependent data or content-related portion of the ranking function depends on the actual search terms and the content of the given document. - In one embodiment, the ranking function comprises at least one query-independent (QID) component comprises a usage parameter. In one embodiment, the query-independent (QID) component may be represented by the following equation:
-
- wherein:U represents an actual usage value or a default usage value; and wu and ku represent tuning parameters for the usage value. In a further embodiment, the query-independent (QID) component may be represented by the following equation:
-
QID(doc)=w u U+k u (2) - wherein:U represents an actual usage value or a default usage value; and wu and ku represent tuning parameters for the usage value. In yet a further embodiment, the query-independent (QID) component may be represented by the following equation:
-
QID(doc)=w u[1+exp(−k u U−B]+C (3) - wherein: U represents an actual usage value or a default usage value; and wu, ku, B and C represent tuning parameters (i.e., scalar constants) for the usage value.
- In a further embodiment, the ranking function comprises a sum of the above-described query-independent (QID) component and at least one query-dependent (QD) component, such as
-
Score=QD(doc,query)+QID(doc). - The QD component can be any document scoring function. In one embodiment, the QD component corresponds to a field weighted scoring function described in U.S. patent application Ser. No. 10/804,326 entitled “FIELD WEIGHTING IN TEXT DOCUMENT SEARCHING,” filed on Mar. 18, 2004, the subject matter of which is hereby incorporated in its entirety by reference. As provided in U.S. patent application Ser. No. 10/804,326, one equation that may be used as a representation of the field weighted scoring function is as follows:
-
- wherein: wtf′ represents a weighted term frequency or sum of term frequencies of given terms in the search query multiplied by weights across all fields (e.g., the title, the body, etc. of the document) and normalized according to the length of each field and the corresponding average length, N represents a number of documents on the network, n represents a number of documents containing a query term, and k1 is a tunable constant.
- The above terms and equation are further described in detail in U.S. patent application Ser. No. 10/804,326, the subject matter of which is hereby incorporated in its entirety by reference.
- In some embodiments, the ranking function may further comprise a QID component that takes into account (i) a click distance value as determined by the methods disclosed in U.S. patent application Ser. No. 10/955,983 entitled “SYSTEM AND METHOD FOR RANKING SEARCH RESULTS USING CLICK DISTANCE” filed on Aug. 30, 2004, (ii) a biased click distance value as determined by the methods disclosed in U.S. patent application Ser. No. 11/206,286 entitled “RANKING FUNCTIONS USING A BIASED CLICK DISTANCE OF A DOCUMENT ON A NETWORK” filed on Aug. 15, 2005, the subject matter of both of which is incorporated herein by reference in its entirety, (iii) a URL depth of a document, or (iv) a combination of (i) or (ii) and (iii). For example, this optional additional QID component may comprise a function as follows:
-
- wherein: Wcd represents a weight of a query-independent component such as a component containing a click distance or biased click distance parameter, bcd represents a weight of a click distance or biased click distance relative to the URL depth, bud represents a weight of a URL depth, CD represents a computed or assigned click distance or biased click distance for a document, kew represents a tuning constant that is determined by optimizing the precision of the ranking function, similar to other tuning parameters (i.e., kew may represent the edge weight value when all edges have the same edge weight value, or kew may represent the average or mean edge value when edge weight values differ from one another), UD represents a URL depth, and kcd is the click distance saturation constant.
- The weighted terms (Wcd, bcd, and bud) assist in defining the importance of each of their related terms (i.e., the component containing a click distance or biased click distance parameter, the click distance or biased click distance value for a given document, and the URL depth of the given document respectively) and ultimately the outcome of the scoring function.
- The URL depth (UD) is an optional addition to the above-referenced query-independent component to smooth the effect that the click distance or biased click distance value may have on the scoring function. For example, in some cases, a document that is not very important (i.e., has a large URL depth) may have a short click distance or biased click distance value. The URL depth is represented by the number of slashes in a document's URL. For example, www.example.com\d1\d2\d3\d4.htm includes four slashes and would therefore have a URL depth of 4. This document however, may have a link directly from the main page www.example.com giving it a relatively low click distance or biased click distance value. Including the URL depth term in the above-referenced function and weighting the URL depth term against the click distance or biased click distance value compensates for a relatively high click distance or biased click distance value to more accurately reflect the document's importance within the network. Depending on the network, a URL depth of 3 or more may be considered a deep link.
- In one embodiment, the ranking function used to determine a document relevance score for a given document comprises a function as follows:
-
- wherein the terms are as described above.
- In other embodiments, the URL depth may be removed from the ranking function or other components may be added to the ranking function to improve the accuracy of the query-dependent component, the query-independent component, or both. Furthermore, the above-described query-independent component containing a usage parameter may be incorporated into other ranking functions (not shown) to improve ranking of search results.
- Once document statistics for a given document are provided to a ranking function in
step 204,exemplary method 20 proceeds to step 205. Instep 205, a document relevance score is determined for a given document, stored in memory, and associated with the given document. Fromstep 205,exemplary method 20 proceeds todecision block 206. - At
decision block 206, a determination is made by application code whether a document relevance score has been calculated for each document within a network. If a determination is made that a document relevance score has not been calculated for each document within a network,exemplary method 20 returns to step 204 and continues as described above. If a determination is made that a document relevance score has been calculated for each document within a network,exemplary method 20 proceeds to step 207. - In
step 207, the search results of the query comprising numerous documents are ranked according to their associated document relevance scores. The resulting document relevance scores take into account the actual or default usage value of each of the documents within the network. Once the search results are ranked,exemplary method 20 proceeds to step 208 where ranked results are displayed to a user. Fromstep 208,exemplary method 20 proceeds to step 209 where highest ranked results are selected and viewed by the user. Fromstep 209,exemplary method 20 proceeds to step 210 whereexemplary method 20 ends. - In addition to the above-described methods of generating a document relevance score for documents within a network and using document relevance scores to rank search results of a search query, computer readable medium having stored thereon computer-executable instructions for performing the above-described methods are also disclosed herein.
- Computing systems are also disclosed herein. An exemplary computing system contains at least one application module usable on the computing system, wherein the at least one application module comprises application code loaded thereon, wherein the application code performs a method of generating a document relevance score for documents within a network. The application code may be loaded onto the computing system using any of the above-described computer readable medium having thereon computer-executable instructions for generating a document relevance score for documents within a network and using document relevance scores to rank search results of a search query as described above.
- While the specification has been described in detail with respect to specific embodiments thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily conceive of alterations to, variations of, and equivalents to these embodiments. Accordingly, the scope of the disclosed methods, computer readable medium, and computing systems should be assessed as that of the appended claims and any equivalents thereto.
Claims (9)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/359,939 US20120041960A9 (en) | 2005-09-21 | 2009-01-26 | Ranking functions using document usage statistics |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/231,955 US7499919B2 (en) | 2005-09-21 | 2005-09-21 | Ranking functions using document usage statistics |
US12/359,939 US20120041960A9 (en) | 2005-09-21 | 2009-01-26 | Ranking functions using document usage statistics |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/231,955 Division US7499919B2 (en) | 2005-09-21 | 2005-09-21 | Ranking functions using document usage statistics |
Publications (2)
Publication Number | Publication Date |
---|---|
US20100191744A1 true US20100191744A1 (en) | 2010-07-29 |
US20120041960A9 US20120041960A9 (en) | 2012-02-16 |
Family
ID=37885403
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/231,955 Active 2026-08-12 US7499919B2 (en) | 2005-09-21 | 2005-09-21 | Ranking functions using document usage statistics |
US12/359,939 Abandoned US20120041960A9 (en) | 2005-09-21 | 2009-01-26 | Ranking functions using document usage statistics |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/231,955 Active 2026-08-12 US7499919B2 (en) | 2005-09-21 | 2005-09-21 | Ranking functions using document usage statistics |
Country Status (15)
Country | Link |
---|---|
US (2) | US7499919B2 (en) |
EP (1) | EP1952277B1 (en) |
JP (2) | JP4698737B2 (en) |
KR (1) | KR101311050B1 (en) |
CN (1) | CN101268464B (en) |
AU (1) | AU2006292103A1 (en) |
BR (1) | BRPI0616233A2 (en) |
CA (1) | CA2620045A1 (en) |
IL (1) | IL189495A (en) |
MY (1) | MY149197A (en) |
NO (1) | NO20080934L (en) |
NZ (1) | NZ566532A (en) |
RU (1) | RU2419861C2 (en) |
WO (1) | WO2007035919A1 (en) |
ZA (1) | ZA200802250B (en) |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110302168A1 (en) * | 2010-06-08 | 2011-12-08 | International Business Machines Corporation | Graphical models for representing text documents for computer analysis |
US8452797B1 (en) * | 2011-03-09 | 2013-05-28 | Amazon Technologies, Inc. | Personalized recommendations based on item usage |
US8738635B2 (en) | 2010-06-01 | 2014-05-27 | Microsoft Corporation | Detection of junk in search result ranking |
US8812493B2 (en) | 2008-04-11 | 2014-08-19 | Microsoft Corporation | Search results ranking using editing distance and document information |
US8843486B2 (en) | 2004-09-27 | 2014-09-23 | Microsoft Corporation | System and method for scoping searches using index keys |
US8914399B1 (en) | 2011-03-09 | 2014-12-16 | Amazon Technologies, Inc. | Personalized recommendations based on item usage |
US9286271B2 (en) | 2010-05-26 | 2016-03-15 | Google Inc. | Providing an electronic document collection |
US9348912B2 (en) | 2007-10-18 | 2016-05-24 | Microsoft Technology Licensing, Llc | Document length as a static relevance feature for ranking search results |
US9384285B1 (en) * | 2012-12-18 | 2016-07-05 | Google Inc. | Methods for identifying related documents |
US9495341B1 (en) | 2012-12-18 | 2016-11-15 | Google Inc. | Fact correction and completion during document drafting |
US9495462B2 (en) | 2012-01-27 | 2016-11-15 | Microsoft Technology Licensing, Llc | Re-ranking search results |
US9514113B1 (en) | 2013-07-29 | 2016-12-06 | Google Inc. | Methods for automatic footnote generation |
US9529791B1 (en) | 2013-12-12 | 2016-12-27 | Google Inc. | Template and content aware document and template editing |
US9529916B1 (en) | 2012-10-30 | 2016-12-27 | Google Inc. | Managing documents based on access context |
US9542374B1 (en) | 2012-01-20 | 2017-01-10 | Google Inc. | Method and apparatus for applying revision specific electronic signatures to an electronically stored document |
US9703763B1 (en) | 2014-08-14 | 2017-07-11 | Google Inc. | Automatic document citations by utilizing copied content for candidate sources |
US9842113B1 (en) | 2013-08-27 | 2017-12-12 | Google Inc. | Context-based file selection |
US11308037B2 (en) | 2012-10-30 | 2022-04-19 | Google Llc | Automatic collaboration |
Families Citing this family (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060064411A1 (en) * | 2004-09-22 | 2006-03-23 | William Gross | Search engine using user intent |
US7499919B2 (en) * | 2005-09-21 | 2009-03-03 | Microsoft Corporation | Ranking functions using document usage statistics |
US8661029B1 (en) | 2006-11-02 | 2014-02-25 | Google Inc. | Modifying search result ranking based on implicit user feedback |
US9092510B1 (en) | 2007-04-30 | 2015-07-28 | Google Inc. | Modifying search result ranking based on a temporal element of user feedback |
JP2008299655A (en) * | 2007-05-31 | 2008-12-11 | Ntt Resonant Inc | Information retrieval device |
US20090106221A1 (en) * | 2007-10-18 | 2009-04-23 | Microsoft Corporation | Ranking and Providing Search Results Based In Part On A Number Of Click-Through Features |
US8510299B2 (en) * | 2007-10-23 | 2013-08-13 | At&T Intellectual Property I, L.P. | Method and apparatus for providing a user traffic weighted search |
US7958136B1 (en) * | 2008-03-18 | 2011-06-07 | Google Inc. | Systems and methods for identifying similar documents |
JP5384884B2 (en) * | 2008-09-03 | 2014-01-08 | 日本電信電話株式会社 | Information retrieval apparatus and information retrieval program |
KR101065091B1 (en) | 2009-02-17 | 2011-09-16 | 엔에이치엔(주) | System and method for determining rank of document based on contribution score |
US9009146B1 (en) | 2009-04-08 | 2015-04-14 | Google Inc. | Ranking search results based on similar queries |
US8447760B1 (en) | 2009-07-20 | 2013-05-21 | Google Inc. | Generating a related set of documents for an initial set of documents |
US8498974B1 (en) | 2009-08-31 | 2013-07-30 | Google Inc. | Refining search results |
US8972391B1 (en) | 2009-10-02 | 2015-03-03 | Google Inc. | Recent interest based relevance scoring |
JP2011091522A (en) * | 2009-10-21 | 2011-05-06 | Fujifilm Corp | Server and program for generating document image, and document browsing system |
US20110282869A1 (en) * | 2010-05-11 | 2011-11-17 | Maxim Zhilyaev | Access to information by quantitative analysis of enterprise web access traffic |
US9623119B1 (en) | 2010-06-29 | 2017-04-18 | Google Inc. | Accentuating search results |
US9002867B1 (en) | 2010-12-30 | 2015-04-07 | Google Inc. | Modifying ranking data based on document changes |
US8909637B2 (en) | 2011-06-03 | 2014-12-09 | Facebook, Inc. | Context-based ranking of search results |
US9268857B2 (en) | 2011-06-03 | 2016-02-23 | Facebook, Inc. | Suggesting search results to users before receiving any search query from the users |
US9110992B2 (en) | 2011-06-03 | 2015-08-18 | Facebook, Inc. | Context-based selection of calls-to-action associated with search results |
CN103577413B (en) * | 2012-07-20 | 2017-11-17 | 阿里巴巴集团控股有限公司 | Search result ordering method and system, search results ranking optimization method and system |
US9779140B2 (en) * | 2012-11-16 | 2017-10-03 | Google Inc. | Ranking signals for sparse corpora |
US9201929B1 (en) * | 2013-08-09 | 2015-12-01 | Google, Inc. | Ranking a search result document based on data usage to load the search result document |
PL406196A1 (en) * | 2013-11-22 | 2015-05-25 | Eo Networks Spółka Akcyjna | Method for delivering to the user of the results of the document retrieval in the form of Internet page |
US9830392B1 (en) * | 2013-12-18 | 2017-11-28 | BloomReach Inc. | Query-dependent and content-class based ranking |
US10127240B2 (en) * | 2014-10-17 | 2018-11-13 | Zestfinance, Inc. | API for implementing scoring functions |
US9720953B2 (en) | 2015-07-01 | 2017-08-01 | Zestfinance, Inc. | Systems and methods for type coercion |
US10061817B1 (en) | 2015-07-29 | 2018-08-28 | Google Llc | Social ranking for apps |
US11106705B2 (en) | 2016-04-20 | 2021-08-31 | Zestfinance, Inc. | Systems and methods for parsing opaque data |
US11941650B2 (en) | 2017-08-02 | 2024-03-26 | Zestfinance, Inc. | Explainable machine learning financial credit approval model for protected classes of borrowers |
US11847574B2 (en) | 2018-05-04 | 2023-12-19 | Zestfinance, Inc. | Systems and methods for enriching modeling tools and infrastructure with semantics |
US11816541B2 (en) | 2019-02-15 | 2023-11-14 | Zestfinance, Inc. | Systems and methods for decomposition of differentiable and non-differentiable models |
JP7276757B2 (en) | 2019-03-18 | 2023-05-18 | ゼストファイナンス,インコーポレーテッド | Systems and methods for model fairness |
KR102245386B1 (en) | 2019-07-16 | 2021-04-27 | 서울과학기술대학교 산학협력단 | Software function analysis system using social network and data envelopment and its method |
US11720962B2 (en) | 2020-11-24 | 2023-08-08 | Zestfinance, Inc. | Systems and methods for generating gradient-boosted models with improved fairness |
Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6182067B1 (en) * | 1997-06-02 | 2001-01-30 | Knowledge Horizons Pty Ltd. | Methods and systems for knowledge management |
US6272507B1 (en) * | 1997-04-09 | 2001-08-07 | Xerox Corporation | System for ranking search results from a collection of documents using spreading activation techniques |
US20020026390A1 (en) * | 2000-08-25 | 2002-02-28 | Jonas Ulenas | Method and apparatus for obtaining consumer product preferences through product selection and evaluation |
US20020032772A1 (en) * | 2000-09-14 | 2002-03-14 | Bjorn Olstad | Method for searching and analysing information in data networks |
US20020062323A1 (en) * | 2000-11-20 | 2002-05-23 | Yozan Inc | Browser apparatus, server apparatus, computer-readable medium, search system and search method |
US20030046389A1 (en) * | 2001-09-04 | 2003-03-06 | Thieme Laura M. | Method for monitoring a web site's keyword visibility in search engines and directories and resulting traffic from such keyword visibility |
US20030167252A1 (en) * | 2002-02-26 | 2003-09-04 | Pliant Technologies, Inc. | Topic identification and use thereof in information retrieval systems |
US20040260695A1 (en) * | 2003-06-20 | 2004-12-23 | Brill Eric D. | Systems and methods to tune a general-purpose search engine for a search entry point |
US20050165718A1 (en) * | 2004-01-26 | 2005-07-28 | Fontoura Marcus F. | Pipelined architecture for global analysis and index building |
US20050210079A1 (en) * | 2004-03-17 | 2005-09-22 | Edlund Stefan B | Method for synchronizing documents for disconnected operation |
US20060059144A1 (en) * | 2004-09-16 | 2006-03-16 | Telenor Asa | Method, system, and computer program product for searching for, navigating among, and ranking of documents in a personal web |
US20070067284A1 (en) * | 2005-09-21 | 2007-03-22 | Microsoft Corporation | Ranking functions using document usage statistics |
US7260573B1 (en) * | 2004-05-17 | 2007-08-21 | Google Inc. | Personalizing anchor text scores in a search engine |
US7716226B2 (en) * | 2005-09-27 | 2010-05-11 | Patentratings, Llc | Method and system for probabilistically quantifying and visualizing relevance between two or more citationally or contextually related data objects |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4025379B2 (en) * | 1996-09-17 | 2007-12-19 | 株式会社ニューズウオッチ | Search system |
JP2002202992A (en) * | 2000-12-28 | 2002-07-19 | Speed System:Kk | Homepage retrieval system |
US8001118B2 (en) * | 2001-03-02 | 2011-08-16 | Google Inc. | Methods and apparatus for employing usage statistics in document retrieval |
US6947920B2 (en) * | 2001-06-20 | 2005-09-20 | Oracle International Corporation | Method and system for response time optimization of data query rankings and retrieval |
KR100509276B1 (en) * | 2001-08-20 | 2005-08-22 | 엔에이치엔(주) | Method for searching web page on popularity of visiting web pages and apparatus thereof |
US7565367B2 (en) | 2002-01-15 | 2009-07-21 | Iac Search & Media, Inc. | Enhanced popularity ranking |
JP3922693B2 (en) * | 2002-06-17 | 2007-05-30 | Necシステムテクノロジー株式会社 | Internet information retrieval system |
US7346839B2 (en) * | 2003-09-30 | 2008-03-18 | Google Inc. | Information retrieval based on historical data |
-
2005
- 2005-09-21 US US11/231,955 patent/US7499919B2/en active Active
-
2006
- 2006-09-20 JP JP2008532469A patent/JP4698737B2/en active Active
- 2006-09-20 AU AU2006292103A patent/AU2006292103A1/en not_active Abandoned
- 2006-09-20 CN CN2006800345316A patent/CN101268464B/en active Active
- 2006-09-20 MY MYPI20080638A patent/MY149197A/en unknown
- 2006-09-20 EP EP06804098.9A patent/EP1952277B1/en active Active
- 2006-09-20 ZA ZA200802250A patent/ZA200802250B/en unknown
- 2006-09-20 NZ NZ566532A patent/NZ566532A/en not_active IP Right Cessation
- 2006-09-20 KR KR1020087006775A patent/KR101311050B1/en active IP Right Grant
- 2006-09-20 RU RU2008110731/08A patent/RU2419861C2/en not_active IP Right Cessation
- 2006-09-20 WO PCT/US2006/037206 patent/WO2007035919A1/en active Application Filing
- 2006-09-20 BR BRPI0616233-9A patent/BRPI0616233A2/en not_active Application Discontinuation
- 2006-09-20 CA CA002620045A patent/CA2620045A1/en not_active Abandoned
-
2008
- 2008-02-13 IL IL189495A patent/IL189495A/en active IP Right Grant
- 2008-02-25 NO NO20080934A patent/NO20080934L/en not_active Application Discontinuation
-
2009
- 2009-01-26 US US12/359,939 patent/US20120041960A9/en not_active Abandoned
-
2011
- 2011-02-03 JP JP2011021985A patent/JP5174931B2/en active Active
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6272507B1 (en) * | 1997-04-09 | 2001-08-07 | Xerox Corporation | System for ranking search results from a collection of documents using spreading activation techniques |
US6182067B1 (en) * | 1997-06-02 | 2001-01-30 | Knowledge Horizons Pty Ltd. | Methods and systems for knowledge management |
US20020026390A1 (en) * | 2000-08-25 | 2002-02-28 | Jonas Ulenas | Method and apparatus for obtaining consumer product preferences through product selection and evaluation |
US20020032772A1 (en) * | 2000-09-14 | 2002-03-14 | Bjorn Olstad | Method for searching and analysing information in data networks |
US20020062323A1 (en) * | 2000-11-20 | 2002-05-23 | Yozan Inc | Browser apparatus, server apparatus, computer-readable medium, search system and search method |
US20030046389A1 (en) * | 2001-09-04 | 2003-03-06 | Thieme Laura M. | Method for monitoring a web site's keyword visibility in search engines and directories and resulting traffic from such keyword visibility |
US20030167252A1 (en) * | 2002-02-26 | 2003-09-04 | Pliant Technologies, Inc. | Topic identification and use thereof in information retrieval systems |
US20040260695A1 (en) * | 2003-06-20 | 2004-12-23 | Brill Eric D. | Systems and methods to tune a general-purpose search engine for a search entry point |
US20050165718A1 (en) * | 2004-01-26 | 2005-07-28 | Fontoura Marcus F. | Pipelined architecture for global analysis and index building |
US20050210079A1 (en) * | 2004-03-17 | 2005-09-22 | Edlund Stefan B | Method for synchronizing documents for disconnected operation |
US7260573B1 (en) * | 2004-05-17 | 2007-08-21 | Google Inc. | Personalizing anchor text scores in a search engine |
US20060059144A1 (en) * | 2004-09-16 | 2006-03-16 | Telenor Asa | Method, system, and computer program product for searching for, navigating among, and ranking of documents in a personal web |
US20070067284A1 (en) * | 2005-09-21 | 2007-03-22 | Microsoft Corporation | Ranking functions using document usage statistics |
US7716226B2 (en) * | 2005-09-27 | 2010-05-11 | Patentratings, Llc | Method and system for probabilistically quantifying and visualizing relevance between two or more citationally or contextually related data objects |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8843486B2 (en) | 2004-09-27 | 2014-09-23 | Microsoft Corporation | System and method for scoping searches using index keys |
US9348912B2 (en) | 2007-10-18 | 2016-05-24 | Microsoft Technology Licensing, Llc | Document length as a static relevance feature for ranking search results |
US8812493B2 (en) | 2008-04-11 | 2014-08-19 | Microsoft Corporation | Search results ranking using editing distance and document information |
US9286271B2 (en) | 2010-05-26 | 2016-03-15 | Google Inc. | Providing an electronic document collection |
US9292479B2 (en) | 2010-05-26 | 2016-03-22 | Google Inc. | Providing an electronic document collection |
US8738635B2 (en) | 2010-06-01 | 2014-05-27 | Microsoft Corporation | Detection of junk in search result ranking |
US8375061B2 (en) * | 2010-06-08 | 2013-02-12 | International Business Machines Corporation | Graphical models for representing text documents for computer analysis |
US20110302168A1 (en) * | 2010-06-08 | 2011-12-08 | International Business Machines Corporation | Graphical models for representing text documents for computer analysis |
US8452797B1 (en) * | 2011-03-09 | 2013-05-28 | Amazon Technologies, Inc. | Personalized recommendations based on item usage |
US8914399B1 (en) | 2011-03-09 | 2014-12-16 | Amazon Technologies, Inc. | Personalized recommendations based on item usage |
US9542374B1 (en) | 2012-01-20 | 2017-01-10 | Google Inc. | Method and apparatus for applying revision specific electronic signatures to an electronically stored document |
US9495462B2 (en) | 2012-01-27 | 2016-11-15 | Microsoft Technology Licensing, Llc | Re-ranking search results |
US9529916B1 (en) | 2012-10-30 | 2016-12-27 | Google Inc. | Managing documents based on access context |
US11308037B2 (en) | 2012-10-30 | 2022-04-19 | Google Llc | Automatic collaboration |
US11748311B1 (en) | 2012-10-30 | 2023-09-05 | Google Llc | Automatic collaboration |
US9495341B1 (en) | 2012-12-18 | 2016-11-15 | Google Inc. | Fact correction and completion during document drafting |
US9384285B1 (en) * | 2012-12-18 | 2016-07-05 | Google Inc. | Methods for identifying related documents |
US9514113B1 (en) | 2013-07-29 | 2016-12-06 | Google Inc. | Methods for automatic footnote generation |
US9842113B1 (en) | 2013-08-27 | 2017-12-12 | Google Inc. | Context-based file selection |
US11681654B2 (en) | 2013-08-27 | 2023-06-20 | Google Llc | Context-based file selection |
US9529791B1 (en) | 2013-12-12 | 2016-12-27 | Google Inc. | Template and content aware document and template editing |
US9703763B1 (en) | 2014-08-14 | 2017-07-11 | Google Inc. | Automatic document citations by utilizing copied content for candidate sources |
Also Published As
Publication number | Publication date |
---|---|
MY149197A (en) | 2013-07-31 |
NO20080934L (en) | 2008-03-14 |
US7499919B2 (en) | 2009-03-03 |
BRPI0616233A2 (en) | 2011-06-14 |
WO2007035919A1 (en) | 2007-03-29 |
EP1952277A4 (en) | 2012-01-18 |
RU2008110731A (en) | 2009-09-27 |
RU2419861C2 (en) | 2011-05-27 |
IL189495A (en) | 2012-05-31 |
CA2620045A1 (en) | 2007-03-29 |
ZA200802250B (en) | 2009-10-28 |
KR20080046670A (en) | 2008-05-27 |
US20070067284A1 (en) | 2007-03-22 |
JP2009509275A (en) | 2009-03-05 |
NZ566532A (en) | 2010-03-26 |
JP2011108268A (en) | 2011-06-02 |
JP4698737B2 (en) | 2011-06-08 |
KR101311050B1 (en) | 2013-09-24 |
JP5174931B2 (en) | 2013-04-03 |
CN101268464A (en) | 2008-09-17 |
IL189495A0 (en) | 2008-06-05 |
AU2006292103A1 (en) | 2007-03-29 |
EP1952277B1 (en) | 2018-05-16 |
US20120041960A9 (en) | 2012-02-16 |
EP1952277A1 (en) | 2008-08-06 |
CN101268464B (en) | 2011-03-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7499919B2 (en) | Ranking functions using document usage statistics | |
US7599917B2 (en) | Ranking search results using biased click distance | |
US8346759B2 (en) | Searching documents for ranges of numeric values | |
US9348912B2 (en) | Document length as a static relevance feature for ranking search results | |
US7620634B2 (en) | Ranking functions using an incrementally-updatable, modified naïve bayesian query classifier | |
US8099417B2 (en) | Semi-supervised part-of-speech tagging | |
US7840569B2 (en) | Enterprise relevancy ranking using a neural network | |
US20090106221A1 (en) | Ranking and Providing Search Results Based In Part On A Number Of Click-Through Features | |
US20110161311A1 (en) | Search suggestion clustering and presentation | |
US20090187550A1 (en) | Specifying relevance ranking preferences utilizing search scopes | |
US8719276B1 (en) | Ranking nodes in a linked database based on node independence | |
US8775443B2 (en) | Ranking of business objects for search engines | |
EP1505520A2 (en) | Ranking of business objects for search engines |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034747/0417 Effective date: 20141014 Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:039025/0454 Effective date: 20141014 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |