US20140074545A1 - Human workflow aware recommendation engine - Google Patents
Human workflow aware recommendation engine Download PDFInfo
- Publication number
- US20140074545A1 US20140074545A1 US13/844,114 US201313844114A US2014074545A1 US 20140074545 A1 US20140074545 A1 US 20140074545A1 US 201313844114 A US201313844114 A US 201313844114A US 2014074545 A1 US2014074545 A1 US 2014074545A1
- Authority
- US
- United States
- Prior art keywords
- items
- user
- recommended
- determining
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/01—Social networking
Definitions
- the present disclosure relates to providing recommendations and, in one particular example, to providing recommendations for user workflows.
- recommendation engines have been developed.
- existing recommendation engines are based on very narrow views of a user's context and do not take into consideration social and workflow information.
- conventional recommendation engines may provide product recommendations for a specific user searching for a specific type of product.
- these recommendation engines do not consider the task the user is trying to accomplish or the relationships between the user and other users that may have searched for similar products or that may have performed similar tasks.
- conventional recommendation engines' limited view of the context within which the user is performing the search they cannot provide sufficiently targeted information.
- a recommendation system that provides customized collections of information that relate both to the context in which the user is operating and the time at which the user requires the information.
- the process may include receiving a request for a recommendation from a computing device of a user.
- the process may further include determining user similarity scores between the user and other users as well as contextual similarity scores between a context of the user and contexts of a plurality of items.
- a first set of recommended items may be generated based on the user similarity scores and a second set of recommended items may be generated based on the contextual similarity scores.
- a weighted average of scores associated with the items in the first and second sets of recommended items may be determined to generate one or more recommendations for the user.
- the one or more recommendations may then be transmitted to the computing device of the user.
- FIG. 1 illustrates an exemplary system for generating recommendations according to various embodiments.
- FIG. 2 illustrates an exemplary process for generating recommendations according to various embodiments.
- FIG. 3 illustrates an exemplary process for determining user and contextual similarity according to various embodiments.
- FIG. 4 illustrates an exemplary social graph according to various embodiments.
- FIG. 5 illustrates an exemplary organization graph according to various embodiments.
- FIG. 6 illustrates an exemplary process for extracting and aggregating recommendation data according to various embodiments.
- FIG. 7 illustrates an exemplary user interface for a recommendation engine according to various embodiments.
- FIG. 8 illustrates another exemplary user interface for a recommendation engine according to various embodiments.
- FIG. 9 illustrates an exemplary computing system that may be used to carry out the various embodiments described herein.
- the process may include receiving a request for a recommendation from a computing device of a user.
- the process may further include determining user similarity scores between the user and other users as well as contextual similarity scores between a context of the user and contexts of a plurality of items.
- a first set of recommended items may be generated based on the user similarity scores and a second set of recommended items may be generated based on the contextual similarity scores.
- a weighted average of scores associated with the items in the first and second sets of recommended items may be determined to generate one or more recommendations for the user.
- the one or more recommendations may then be transmitted to the computing device of the user.
- FIG. 1 illustrates an exemplary system 100 for generating recommendations according to various embodiments.
- system 100 may include computing devices 102 , 104 , and 106 that may communicate with each other and/or server 110 via network 108 .
- Server 110 may include recommendation logic 112 for generating recommendations and a local and/or remote database 114 .
- Server 110 and computing devices 102 , 104 , and 106 may include any one of various types of computing devices having, for example, a processing unit, a memory (including a permanent storage device), and a communication interface, as well as other conventional computer components (e.g., an input device, such as a keyboard and mouse, and an output device, such as a display).
- an input device such as a keyboard and mouse
- an output device such as a display
- computing devices 102 , 104 , and 106 may include any type of computing device, such as a mobile phone, laptop, tablet, desktop computer, or the like. While three computing devices are shown, it should be appreciated that system 100 may include any number of computing devices.
- Server 110 and computing devices 102 , 104 , and 106 may communicate, for example, using suitable communication interfaces via network 108 , such as the Internet, a LAN, a WAN, or the like.
- Server 110 and computing devices 102 , 104 , and 106 may communicate, in part or in whole, via wireless or hardwired communications, such as Ethernet, IEEE 802.11a/b/g/n/ac wireless, or the like.
- communication between computing devices 102 , 104 , and 106 and server 110 may include various servers, such as a mobile server or the like.
- Each computing device 102 , 104 , and 106 may be associated with one or more users. Through these devices, associated users may create content, create tasks, and be assigned tasks. In some examples, users may be registered with the system such that information relating to their activity within the system may be collected to drive recommendations. For example, users may create a profile to be stored by server 110 in database 114 .
- computing devices 102 , 104 , and 106 may be configured to monitor or determine the mode of a user.
- a mode may include the context of the user at a point in time at which an action was performed.
- the context of the user may include a location of the user, activity of the user, time of day/week/month/year, type of computing device used by the user, or any other data describing the context (e.g., environment or setting) associated with the user. These may be application dependent, but typically may be a collection of weighted categories. For example, at a given point in time, a user may be 100% mobile, 25% work, and 75% social. This may be interpreted as the user traveling with friends, but checking work email occasionally.
- the computing devices 102 , 104 , and 106 may determine the mode of a user in a variety of ways. For example, actions of the user, a location of the user, time of day/week/month/year, applications running on the device, and the like may be monitored by the device to determine a mode of the user.
- the mode determined by computing devices 102 , 104 , and 106 may be transmitted to server 110 via network 108 to be stored in database 114 and accessed by recommendation logic 112 .
- Server 110 may include or access recommendation logic 112 and database 114 .
- database 114 may include a local and/or remote storage device for storing various types of items, such as user data, content data, workspace data, workflow data, task data, and the like. This data may be provided to server 110 from users associated with computing devices 102 , 104 , and 106 or may be entered into database 114 by an administrator or owner of system 100 .
- Content generally includes any information that is entered into the system, such as documents, user profiles, text documents, images, forms, blogs, comments, polls, invitations, calendar entries, and the like. There are many sub-types of content and users may create new types.
- server 110 may extract key phrases from the content for use by recommendation logic 112 .
- the content may be structured (e.g., RDBMS tables/rows) or unstructured (e.g., text documents).
- a workflow may generally include a description or template that describes work that needs to be performed to accomplish a goal.
- a workflow may be separated into two categories: complex and simple.
- a complex workflow may include a description of a multi-step process that may include branching and decision logic that may be followed to accomplish a goal.
- required users and content to perform the steps may be identified abstractly by the roles they fulfill within the workflow. For example, a required user may be identified by a title or a task that a user is capable of performing. Similarly, required content may include an identifier describing a type of document required.
- a simple workflow may include a single unit of work that may be used to accomplish a sub-goal in a complex workflow. In a simple workflow, required users and content may also be identified abstractly by role and may be inherited from the complex workflow that it is included within.
- a task may include a populated manifestation of a workflow and the status of the work required by the workflow.
- a task is the per-instance manifestation of a workflow.
- a task may include references to preceding tasks, identities of the users and contents selected (e.g., selected by a user) to fulfill each role in the workflow, text associated with the task (e.g., a text description of the task to be completed), the identity of an issuer of the task, identifiers of associated workflows, target user(s) assigned to the task, or the like.
- a workflow may describe a process for reviewing a document.
- the workflow may abstractly identify a user issuing the task, a user assigned to perform the task, a document to be reviewed, and the steps to be performed to review the document.
- the task may include an identification of the actual issuer of the task, an identification of the actual user assigned to perform the task, an identification of the actual document to be reviewed, and the steps to be performed to review the document.
- the social graph may include interconnected nodes, where each node represents a user and the connecting edges represent relationships between these users.
- the relationship may be defined by different relationship types, such as friend, co-worker, classmate, etc., and may be defined by the associated edge within the social graph.
- an organization graph may include interconnected nodes, where each node represents a user and the connecting edges represent relationships between these users.
- the users may be users within a particular organization and the edges may represent structured relationships between these users within the organization.
- an organization graph for a company may include nodes representing employees and edges representing manager/subordinate/peer relationships between the employees.
- a collaboration graph may include interconnected nodes, where each node represents a user and the connecting edges represent relationships between these users. However, the collaboration graph may instead track and document interactions between users of system 100 as they collaborate to accomplish shared goals. In some examples, the collaboration graph may be generated based on users being members of a workspace and/or based on users being assigned roles or otherwise participating in a workflow task.
- system 100 may collect, on a per-user basis, information about user interactions with other users and with content, including frequency and time of the interaction. Each recorded interaction may include the user's mode at the time of the interaction.
- database 114 Another type of data that may be stored in database 114 is a rating of a user, content, or workflow. These ratings may be entered by users and stored on a per-user basis. Other ratings may be collected and stored without user input. For example, a task may be rated according to success criteria, such as the time required for completion.
- server 110 may create and track relationships between items, such as users, tasks, content, and the like. This essentially represents tracking the creation and utilization of content. Server 110 may further generate workspaces for users. These workspaces include logical meeting places where users may share information that applies to a shared task.
- Server 110 may further track tasks and workflows assigned to users associated with computing devices 102 , 104 , and 106 .
- a user associated with computing device 102 may want to assign a task to a user associated with computing device 104 .
- the user of computing device 102 may send task data associated with the task to server 110 via network 108 .
- Server 110 may process the task assignment by storing the task data within database 114 and forwarding the task data to computing device 104 via network 108 .
- server 110 may be used to receive workflow data associated with a particular user.
- a user of computing device 106 may want to send a request to server 110 via network 108 to receive workflow data associated with the user.
- Server 110 may access database 114 to retrieve workflow data associated with the user (e.g., by using a username/password) and may transmit the retrieved workflow data to computing device 106 via network 108 .
- Server 110 may be further programmed to format data, accessed from local or remote databases or other sources of data, for presentation to users of computing devices 102 , 104 , and 106 , preferably in the format discussed in detail herein.
- Server 110 may utilize various Web data interface techniques such as Common Gateway Interface (CGI) protocol and associated applications (or “scripts”), Java® “servlets” (i.e., Java applications running on the Web server), an application that utilizes Software Development Kit Application Programming Interfaces (“SDK APIs”), or the like to present information and receive input from computing devices 102 , 104 , and 106 .
- Server 110 although described herein in the singular, may actually include multiple computers, devices, backends, and the like, communicating (wired and/or wirelessly) and cooperating to perform the functions described herein.
- individually shown devices may comprise multiple devices and be distributed over multiple locations.
- additional servers and devices may be included such as Web servers, media servers, mail servers, mobile servers, advertisement servers, and the like as will be appreciated by those of ordinary skill in the art.
- FIG. 2 illustrates an exemplary process 200 that may be performed to generate recommendations.
- process 200 may be performed by a computing device, such as server 110 , programmed with recommendation logic, such as recommendation logic 112 , within a computing environment similar or identical to system 100 .
- a request for a recommendation may be received.
- the recommendation may be received by a computing device (e.g., server 110 of FIG. 1 ) via a wired or wireless network (e.g., network 108 of FIG. 1 ) from a computing device associated with a user (e.g., computing device 102 , 104 , or 106 of FIG. 1 ).
- the request may include an identification of the user making the request, the context of the user, a requested item (e.g., user, content, workspace, workflow, task, etc.), other search parameters (e.g., search strings, etc.), or the like.
- the request need not be explicitly made by the user. For example, a user may access a document, and a request for a recommendation based on the requested document may automatically be made.
- user and contextual similarity may be determined.
- a computing device such as server 110 , programmed with recommendation logic, such as recommendation logic 112 , may determine the user and contextual similarity based on information received from the requesting user at block 201 and data associated with items, such as users, content, workspaces, workflows, tasks, and the like, stored in a local or remote database (e.g., database 114 ).
- an exemplary process 300 may be used to determine the user and contextual similarity.
- Process 300 may be performed by a computing device, such as server 110 , programmed with recommendation logic, such as recommendation logic 112 , within a computing environment similar or identical to system 100 .
- workflow data may be accessed.
- the data may be accessed from a local or remote workflow database similar or identical to database 114 of FIG. 1 .
- the workflow data may include social graphs, organization graphs, collaboration graphs, content data, utilization data, ratings data, workflow data, task data, goal data, and the like, associated with users tracked by the system.
- user similarity may be determined between the user requesting a recommendation at block 201 and users associated with the workflow data. For example, if a recommendation is to be made for user A, user similarities may be determined between user A and other users being tracked by the recommendation system 100 whose data is included within the workflow data. The user similarity may be determined using one or more of social similarity, organizational similarity, contextual similarity, and preference similarity.
- Social similarity is based on the concept that a user is likely to have needs and preferences similar to those of friends. Additionally, a user is more likely to have similar needs and preferences to a closely related user (e.g., a friend) than a more distantly related user (e.g., a friend of a friend).
- distances between users in a social graph accessed from a database at block 201 may be determined.
- FIG. 4 shows a social graph 400 for user A.
- social graph 400 includes nodes corresponding to users A, B, C, D, and E and edges indicating the type of relationships existing between the users and user A.
- social graph 400 indicates that users D and B are friends of user A, while users C and E are friends of friends of user A.
- the inverse of social distance may be used.
- the social distance between two users represents the number of edges that must be traversed along the shortest path to move from the node of one user to another.
- users A and D are more similar than users A and E.
- Organizational similarity may be based on the concept that a user is likely to have needs and preferences similar to others in the same group or those that are organizationally near. Similar to determining social similarity, distances between users in an organization graph accessed from a database at block 301 may be determined. However, since organization graphs take the form of a tree, different algorithms for computing similarity may be used. The result of these algorithms may indicate that users that are organizationally near have a higher value for this metric. For example, FIG. 5 illustrates an organization graph 500 for user A. Organization graph 500 includes nodes corresponding to users A, B, C, D, E, F, and G and edges indicating relationships between the users and user A. To determine organizational similarity, the inverse of organizational distance may be used.
- the distance between user A and user A's subordinates is 2 because a vertical traversal extending away from the level of A is needed to reach users C and D, likewise indicating that user A is fairly similar to users C and D.
- user A and user A's peer have a distance value of 1 since a vertical traversal away from user A's level (+2) toward user B, followed by a vertical traversal from user B toward user A's level at E ( ⁇ 1) are needed to get from nodes A to E.
- Contextual similarity may be based on the concept that another user that fulfilled the same roles in tasks and workflows is likely to have similar needs and preferences, that users that have generated or consumed similar content are likely to have similar needs and preferences, and the like.
- To determine contextual similarity it may be determined if the users have collaborated in the same workspace or on the same workflow, utilized the same content, and the like.
- a user's profile may include content, users, workflows, workspaces, and the like that the user has interacted with as well as roles that the user may have filled in the workflows.
- IR information retrieval
- known key phrase extraction techniques such as the Keyphrase Extraction Algorithm (KEA) (described at http://www.nzdl.org/Kea/) or Apache Tika (described at http://tika.apache.org/), and IR techniques, such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index), may be used.
- KAA Keyphrase Extraction Algorithm
- Apache Tika described at http://tika.apache.org/
- IR techniques such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index)
- KAA Keyphrase Extraction Algorithm
- Apache Tika described at http://tika.apache.org/
- IR techniques such as the Vector Space Model (described at
- the contextual similarity may take into consideration how recently and/or frequently users have interacted in similar workspaces/workflows or interacted with similar content.
- the frequency and age of collaboration may be factored into the resulting similarity score using a configurable half-life period, as discussed below with respect to preference similarity (e.g., equations 1-5, discussed below).
- Preference similarity may be based on the concept that users that have expressed preference for similar items, such as users, content, or workflows, are likely to have similar preferences and needs.
- collected utilization and ratings information accessed at block 301 from a database similar or identical to database 114 may be used. If a user has expressed a preference for an item, that information may be used.
- These expressions of preference may take the form of a score (e.g., rating from 1-10) or Boolean value (e.g., like/dislike).
- these ratings may be converted to a normalized value (e.g., real number in the range [ ⁇ 1 . . . 1] or other similarly scaled values).
- the user's preference may be derived from comments made about an item in the system.
- sentiment analysis e.g., described at http://en.wikipedia.org/wiki/Sentiment_analysis
- a rating in the range [ ⁇ 1 . . . 1] (or other similarly scaled values).
- a user's preference may be inferred from the successful completion of workflows and tasks.
- the level of preference may be derived for a configurable metric related to the task or workflow. A few such examples are the time required for task completion, final task status, or something derived from the content associated with a task, such as the value of a deal. Note that workflows with a negative outcome may affect the preferences negatively.
- the frequency and age of utilization and ratings may be taken into account.
- more recent utilization and/or high-frequency utilization may increase a user's effective preference for an item while old and/or low-frequency utilization may reduce the effective preference.
- a number of mechanisms for computing this are possible. For example, a configurable half-life period may be used.
- the algorithm for generating a single preference value may configurable within the system, but one example for calculating a preference is provided by equations 1-5, shown below.
- “p” represents the final preference value in the range [ ⁇ 1 . . . 1] (or other similarly scaled values)
- “rp” represents the normalized user preference after the value has been decayed
- “rw” represents the rating preference weight coefficient
- “up” represents the normalized utilization preference after the value has been decayed
- “uw” represents the utilization preference weight coefficient
- “r” represents the rating value provided by the user in the range [ ⁇ 1 . . .
- “rd” represents the calculated user rating decay coefficient
- “ra” represents the age of the most recent user rating (units may be configurable)
- “rh” represents the configured constant half-life of user ratings
- “u” represents the user utilization count for a given item in the range [1 . . . n] (or other similarly scaled values)
- “ud” represents the calculated utilization decay coefficient
- “ua” represents the age of the most recent user utilization (units may be configurable)
- “uh” represents the configured constant half-life of user utilization ratings.
- the above or other algorithms may be used to calculate a preference score for a user.
- the preference score may be calculated for a user's preference for other users, content, workflows, and the like. Preferences may be computed on a per-user/per-type basis. This may result in an ordered list of the top preference items and a preference value being identified for each user. Each preference may be represented as a real number preference value in the range [ ⁇ 1 . . . 1] (or other similarly scaled values). In addition, mode information associated with each utilization may be stored with the computed preference. The number of top preferences retained per user may be configurable for performance reasons.
- Each individual mechanism used to calculate the various user similarities may return a user-to-user similarity matrix containing a real number similarity rating in the range [0 . . . 1] (or other similarly scaled values).
- the results from each mechanism may be combined into a single user/user similarity matrix with a similarity rating as a real number value in the range [0 . . . 1] (or other similarly scaled values).
- Similarity metrics may be combined using configurable weights.
- block 303 may be performed after receiving a request for a recommendation at block 201 , or may be pre-computed (e.g., using process 600 , described below) at some other designated time (e.g., when a user is added, periodically, when contextual data changes, etc.).
- an ordered list of the most similar users may be stored (e.g., in database 114 ) for each user. The size of this list may be configurable to any desired size.
- task similarity may be determined between a task to be completed by a user and tasks associated with the workflow data.
- the request for a recommendation received at block 201 may be received from a user attempting to complete a particular task.
- This task may be compared to task data included within the workflow data accessed at block 301 .
- Task similarity may be based on the concept that a task may be compared for similarity along a number of axes.
- the task data may include data identifying a workflow from which the task is derived, users that the task is assigned to, users issuing the task, content associated with the task, and the like. Tasks derived from the same workflow, initiated by similar users, assigned to similar users, having similar content, may be determined to be similar.
- KA Keyphrase Extraction Algorithm
- Apache Tika described at http://tika.apache.org/
- IR techniques such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index), may be used on the task data.
- similarity scores returned by these techniques for each axis of similarity may be combined (e.g., a weighted average) into a single notion of similarity between two tasks.
- block 305 may be performed after receiving a request for a recommendation at block 201 , or may be pre-computed at some other designated time (e.g., when a task is added, a task is modified, content changes, periodically, etc.).
- an ordered list of the most similar tasks may be stored (e.g., in database 114 ) for each task.
- the size of this list may be configurable to any desired size.
- process 600 of FIG. 6 may be performed to extract and aggregate preference data.
- new key phrases may be extracted from user profile data, user digital artifacts, and user workflows.
- new user roles may be extracted from user workflows.
- user interaction records with each item may be updated. This may include updating the frequency and age of the interactions. This may be merged with user ratings (preferences) of the items at block 607 and used to calculate preference values using equations 1-5 at block 609 . Based on blocks 601 , 603 , 605 , 607 , and 609 , the user context data, the preferred users, the preferred digital artifacts, and the preferred workflows of the user may be updated.
- goal similarity may be determined between a goal of a user and goals associated with the workflow data.
- Goal similarity may be determined, for example, by comparison between the keywords associated with pairs of tasks to be completed.
- the request for a recommendation received at block 201 may be received from a user attempting to accomplish a particular goal. This goal may be compared to goal data included within the workflow data accessed at block 301 .
- Goal similarity may include, for example, the concept that a goal may be compared for similarity along a number of axes.
- KA Keyphrase Extraction Algorithm
- Apache Tika described at http://tika.apache.org/
- IR techniques such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index)
- similarity scores returned by these techniques for each axis of similarity may be combined (e.g., a weighted average) into a single notion of similarity between two goals.
- block 307 may be performed after receiving a request for a recommendation at block 201 , or may be pre-computed (e.g., using process 600 ) at some other designated time (e.g., when a goal is added, a goal is modified, periodically, etc.).
- an ordered list of the most similar goals may be stored (e.g., in database 114 ) for each goal.
- the size of this list may be configurable to any desired size.
- process 200 of FIG. 2 after determining user and contextual similarity at block 203 (or after block 201 if block 203 was pre-computed), the process may proceed to block 205 .
- the n most similar users may be identified. This may be based on the user similarity determined at block 303 of process 300 . For example, block 303 of process 300 may generate an ordered list of users based on their similarity to the user requesting the recommendation at block 201 . Based on this list, the n most similar users may be identified.
- the value n represents a configurable value that may be any value. In some examples, n may default to 20.
- preferred items of the n most similar users identified at block 205 may be determined.
- the preferred items may include any type of item, such as workflows, users, contacts, tasks, documents, forms, calendar entries, conference rooms, etc.
- the possible set types may or may not be predetermined.
- the preferred items may be limited to a subset of item types based on input from the user or may be provided on behalf of the user without the user's knowledge based on the context of the user at the time the request is made.
- an application on the user's computing device uses the recommendation system to recommend users to assign a task to, then the application may request a recommendation for the type “user.”
- the preferred items may be taken from the ordered list of each similar user's list of preferred items that may be stored in database 114 .
- the lists of preferred items determined at block 207 may be merged into a single ordered list by merge-sorting each similar user's list of preferred items based on the preference values. Duplicates may optionally be removed, retaining only the most preferred items.
- additional items may be determined based on context. This may be based on the task similarity determined at block 305 and the goal similarity determined at block 307 of process 300 .
- the requesting user's context may be retrieved, and that context, along with any per-request context (e.g., search criteria), may be used to search items in database 114 .
- the result of block 211 may include one or more ordered lists of items matching or similar to the search criteria (if provided) derived from the user. For example, a list of similar tasks and a list of similar goals determined at blocks 305 and 307 , respectively, may be produced by block 211 .
- items that are related to or similar to these items may also be returned. Each item in the lists may include a similarity score in the range [0 . . . 1] (or other similarly scaled values).
- the lists of items determined at block 211 may be merged into a single ordered list by merge-sorting each list of similar context items. Duplicates may optionally be removed, retaining only the most similar items.
- the merged and sorted lists from blocks 213 and 209 may be merged into a single ordered list by merge-sorting each list of similar context items. Duplicates may optionally be removed, retaining only the most similar.
- the final score of a recommended item may include a weighted average of the scores from similar users and the contextual search from blocks 209 and 213 . If a given item in one input list is not represented in the other input list, then that score may be assumed to be 0.
- the weighting between the two mechanisms may be a configuration option of the system.
- a set of recommendations may be generated and returned to the user.
- the set of recommendations may include the merge and sorted recommended items generated at block 215 .
- a computing device e.g., server 110
- each item in the list may include an identifier, name, and score.
- the identifier may include the system identifier for the recommended item. This may generally be hidden from the user and used by the application when the item is selected.
- the name may include a user-visible name of an item that may be displayed in a user interface.
- the score may include a numerical representation of the strength of the recommendation. For example, items with higher scores may be more highly recommended.
- Recommendation scores may be computed for each recommendation request and may only have meaning as a relative value within the result list.
- server 110 may begin performing process 200 .
- processes 200 and 300 may be performed to identify other users similar to User A that have previously requested the same form.
- Server 110 may find that other users most similar to User A requesting the same form all submitted it to the person in the organizational chart who is their supervisor or boss.
- Server 110 may thus recommend that User A submit the form to his boss (e.g., as shown in FIG. 8 ).
- User A is presented with document A and document B but does not know which one to select.
- User A may request a recommendation from system 100 .
- equations 1-5 may be performed.
- the half-life of a document may be configured to be 24 months (e.g., after 24 months, half of its value is lost).
- the unit of time may be a month (considered to be 30 days).
- the variable “rh” may be equal to 24 (the configured constant half-life of user ratings) and the variable “uh” may be equal to 24 (the configured constant half-life of user utilization ratings).
- the decay coefficient may be, in half-life terminology, calculated as the natural log of 2 divided by the half-life (in this example, 24). This may provide the calculated quantities “rd” equal to 0.029 and “ud” equal to 0.029.
- the final preference value may be determined using equation 1.
- User A may have the same problem and knowledge discussed above, but instead of clicking “Suggest Next Step,” User A may select “Suggest a Workflow” in the interface of FIG. 7 .
- server 110 may begin performing process 200 .
- processes 200 and 300 may be performed to identify other users similar to User A that have previously requested the same form.
- Server 110 may further evaluate other workflows and, based on similarities, display several potential workflows that represent the paths others took in the same situation. These workflows may include steps and concepts that User A was unaware of, including:
- recommendations may be generated based on user similarities and contextual similarities.
- collaborative filtering, key phrase extraction, and IR techniques may be used. This advantageously allows the system to make recommendations based on various types of data, resulting in the production of recommendations when certain types of data are unavailable.
- the system may provide recommendations for items already known to the user. This allows the system to provide a recommendation for an item that the user interacted with before but may be unaware could be useful in a particular context.
- FIG. 9 depicts an exemplary computing system 900 configured to perform any one of the above-described processes.
- computing system 900 may include, for example, a processor, memory, storage, and input/output devices (e.g., monitor, keyboard, disk drive, Internet connection, etc.).
- computing system 900 may include circuitry or other specialized hardware for carrying out some or all aspects of the processes.
- computing system 900 may be configured as a system that includes one or more units, each of which is configured to carry out some aspects of the processes either in software, hardware, or some combination thereof.
- FIG. 9 depicts computing system 900 with a number of components that may be used to perform the above-described processes.
- the main system 902 includes a motherboard 904 having an input/output (“I/O”) section 906 , one or more central processing units (CPUs) 908 , and a memory section 910 , which may have a flash memory card 912 related to it.
- the I/O section 906 is connected to a display 924 , a keyboard 914 , a disk storage unit 916 , and a media drive unit 918 .
- the media drive unit 918 may read/write a computer-readable medium 920 , which may contain programs 922 and/or data.
- a non-transitory computer-readable medium may be used to store (e.g., tangibly embody) one or more computer programs for performing any one of the above-described processes by means of a computer.
- the computer program may be written, for example, in a general-purpose programming language (e.g., Pascal, C, C++, Java) or some specialized application-specific language.
Abstract
Recommendation systems and processes for generating recommendations within the context of a socially-enabled human workflow system are provided. The processes may include accessing workflow data, such as social graphs, organization graphs, collaboration graphs, content data, utilization data, ratings data, and the like, associated with a user requesting a recommendation. The process may further include determining one or more of a user similarity score, task similarity score, goal similarity score, and content similarity score. The process may further include generating one or more recommendations based at least in part on one or more of the user similarity score, task similarity score, goal similarity score, and content similarity score.
Description
- This application claims priority to U.S. Provisional Patent Application Ser. No. 61/698,514, filed Sep. 7, 2012, the entire disclosure of which is hereby incorporated by reference in its entirety for all purposes as if put forth in full below.
- 1. Field
- The present disclosure relates to providing recommendations and, in one particular example, to providing recommendations for user workflows.
- 2. Related Art
- In the information age, people often struggle with an overload of information. For example, in the context of user workflows, a user is often required to manually filter through irrelevant or low-value information in order to determine what is required to complete a task. This may reduce a user's ability to make decisions and perform tasks efficiently.
- To help users navigate the large volumes of information, recommendation engines have been developed. However, existing recommendation engines are based on very narrow views of a user's context and do not take into consideration social and workflow information. For example, conventional recommendation engines may provide product recommendations for a specific user searching for a specific type of product. However, these recommendation engines do not consider the task the user is trying to accomplish or the relationships between the user and other users that may have searched for similar products or that may have performed similar tasks. As a result of conventional recommendation engines' limited view of the context within which the user is performing the search, they cannot provide sufficiently targeted information.
- Thus, what is desired is a recommendation system that provides customized collections of information that relate both to the context in which the user is operating and the time at which the user requires the information.
- Recommendation systems and processes for generating recommendations within the context of a socially enabled human workflow system are disclosed. The process may include receiving a request for a recommendation from a computing device of a user. The process may further include determining user similarity scores between the user and other users as well as contextual similarity scores between a context of the user and contexts of a plurality of items. A first set of recommended items may be generated based on the user similarity scores and a second set of recommended items may be generated based on the contextual similarity scores. A weighted average of scores associated with the items in the first and second sets of recommended items may be determined to generate one or more recommendations for the user. The one or more recommendations may then be transmitted to the computing device of the user.
- The present application may be best understood by reference to the following description taken in conjunction with the accompanying drawing figures, in which like parts may be referred to by like numerals.
-
FIG. 1 illustrates an exemplary system for generating recommendations according to various embodiments. -
FIG. 2 illustrates an exemplary process for generating recommendations according to various embodiments. -
FIG. 3 illustrates an exemplary process for determining user and contextual similarity according to various embodiments. -
FIG. 4 illustrates an exemplary social graph according to various embodiments. -
FIG. 5 illustrates an exemplary organization graph according to various embodiments. -
FIG. 6 illustrates an exemplary process for extracting and aggregating recommendation data according to various embodiments. -
FIG. 7 illustrates an exemplary user interface for a recommendation engine according to various embodiments. -
FIG. 8 illustrates another exemplary user interface for a recommendation engine according to various embodiments. -
FIG. 9 illustrates an exemplary computing system that may be used to carry out the various embodiments described herein. - The following description is presented to enable a person of ordinary skill in the art to make and use the various embodiments. Descriptions of specific devices, techniques, and applications are provided only as examples. Various modifications to the examples described herein will be readily apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the present technology. Thus, the disclosed technology is not intended to be limited to the examples described herein and shown, but is to be accorded the scope consistent with the claims.
- Various examples are described below relating to recommendation engines and processes for generating recommendations within the context of a socially enabled human workflow system. The process may include receiving a request for a recommendation from a computing device of a user. The process may further include determining user similarity scores between the user and other users as well as contextual similarity scores between a context of the user and contexts of a plurality of items. A first set of recommended items may be generated based on the user similarity scores and a second set of recommended items may be generated based on the contextual similarity scores. A weighted average of scores associated with the items in the first and second sets of recommended items may be determined to generate one or more recommendations for the user. The one or more recommendations may then be transmitted to the computing device of the user.
-
FIG. 1 illustrates anexemplary system 100 for generating recommendations according to various embodiments. Generally,system 100 may includecomputing devices server 110 vianetwork 108.Server 110 may includerecommendation logic 112 for generating recommendations and a local and/orremote database 114.Server 110 andcomputing devices computing devices system 100 may include any number of computing devices. -
Server 110 andcomputing devices network 108, such as the Internet, a LAN, a WAN, or the like.Server 110 andcomputing devices computing devices server 110 may include various servers, such as a mobile server or the like. - Each
computing device server 110 indatabase 114. - In some examples,
computing devices computing devices devices server 110 vianetwork 108 to be stored indatabase 114 and accessed byrecommendation logic 112. -
Server 110 may include oraccess recommendation logic 112 anddatabase 114. In some examples,database 114 may include a local and/or remote storage device for storing various types of items, such as user data, content data, workspace data, workflow data, task data, and the like. This data may be provided toserver 110 from users associated withcomputing devices database 114 by an administrator or owner ofsystem 100. - One type of data that may be stored in
database 114 is content. Content generally includes any information that is entered into the system, such as documents, user profiles, text documents, images, forms, blogs, comments, polls, invitations, calendar entries, and the like. There are many sub-types of content and users may create new types. In some examples,server 110 may extract key phrases from the content for use byrecommendation logic 112. The content may be structured (e.g., RDBMS tables/rows) or unstructured (e.g., text documents). - Another type of data that may be stored in
database 114 is a workflow. A workflow may generally include a description or template that describes work that needs to be performed to accomplish a goal. A workflow may be separated into two categories: complex and simple. A complex workflow may include a description of a multi-step process that may include branching and decision logic that may be followed to accomplish a goal. In a complex workflow, required users and content to perform the steps may be identified abstractly by the roles they fulfill within the workflow. For example, a required user may be identified by a title or a task that a user is capable of performing. Similarly, required content may include an identifier describing a type of document required. A simple workflow may include a single unit of work that may be used to accomplish a sub-goal in a complex workflow. In a simple workflow, required users and content may also be identified abstractly by role and may be inherited from the complex workflow that it is included within. - Another type of data that may be stored in
database 114 is a task. A task may include a populated manifestation of a workflow and the status of the work required by the workflow. In other words, a task is the per-instance manifestation of a workflow. Thus, a task may include references to preceding tasks, identities of the users and contents selected (e.g., selected by a user) to fulfill each role in the workflow, text associated with the task (e.g., a text description of the task to be completed), the identity of an issuer of the task, identifiers of associated workflows, target user(s) assigned to the task, or the like. For example, a workflow may describe a process for reviewing a document. The workflow may abstractly identify a user issuing the task, a user assigned to perform the task, a document to be reviewed, and the steps to be performed to review the document. The task may include an identification of the actual issuer of the task, an identification of the actual user assigned to perform the task, an identification of the actual document to be reviewed, and the steps to be performed to review the document. - Another type of data that may be stored in
database 114 is a social graph. The social graph may include interconnected nodes, where each node represents a user and the connecting edges represent relationships between these users. The relationship may be defined by different relationship types, such as friend, co-worker, classmate, etc., and may be defined by the associated edge within the social graph. - Another type of data that may be stored in
database 114 is an organization graph. Similar to the social graph, an organization graph may include interconnected nodes, where each node represents a user and the connecting edges represent relationships between these users. However, in an organization graph, the users may be users within a particular organization and the edges may represent structured relationships between these users within the organization. For example, an organization graph for a company may include nodes representing employees and edges representing manager/subordinate/peer relationships between the employees. - Another type of data that may be stored in
database 114 is a collaboration graph. Similar to the social and organization graphs, a collaboration graph may include interconnected nodes, where each node represents a user and the connecting edges represent relationships between these users. However, the collaboration graph may instead track and document interactions between users ofsystem 100 as they collaborate to accomplish shared goals. In some examples, the collaboration graph may be generated based on users being members of a workspace and/or based on users being assigned roles or otherwise participating in a workflow task. - Another type of data that may be stored in
database 114 is the utilization of a user or content. In particular,system 100 may collect, on a per-user basis, information about user interactions with other users and with content, including frequency and time of the interaction. Each recorded interaction may include the user's mode at the time of the interaction. - Another type of data that may be stored in
database 114 is a rating of a user, content, or workflow. These ratings may be entered by users and stored on a per-user basis. Other ratings may be collected and stored without user input. For example, a task may be rated according to success criteria, such as the time required for completion. - Using the data stored in
database 114 described above,server 110 may create and track relationships between items, such as users, tasks, content, and the like. This essentially represents tracking the creation and utilization of content.Server 110 may further generate workspaces for users. These workspaces include logical meeting places where users may share information that applies to a shared task. -
Server 110 may further track tasks and workflows assigned to users associated withcomputing devices computing device 102 may want to assign a task to a user associated withcomputing device 104. To do so, the user ofcomputing device 102 may send task data associated with the task toserver 110 vianetwork 108.Server 110 may process the task assignment by storing the task data withindatabase 114 and forwarding the task data tocomputing device 104 vianetwork 108. Additionally,server 110 may be used to receive workflow data associated with a particular user. For example, a user ofcomputing device 106 may want to send a request toserver 110 vianetwork 108 to receive workflow data associated with the user.Server 110 may accessdatabase 114 to retrieve workflow data associated with the user (e.g., by using a username/password) and may transmit the retrieved workflow data tocomputing device 106 vianetwork 108. -
Server 110 may be further programmed to format data, accessed from local or remote databases or other sources of data, for presentation to users ofcomputing devices Server 110 may utilize various Web data interface techniques such as Common Gateway Interface (CGI) protocol and associated applications (or “scripts”), Java® “servlets” (i.e., Java applications running on the Web server), an application that utilizes Software Development Kit Application Programming Interfaces (“SDK APIs”), or the like to present information and receive input from computingdevices Server 110, although described herein in the singular, may actually include multiple computers, devices, backends, and the like, communicating (wired and/or wirelessly) and cooperating to perform the functions described herein. - It will be recognized that, in some examples, individually shown devices may comprise multiple devices and be distributed over multiple locations. Further, various additional servers and devices may be included such as Web servers, media servers, mail servers, mobile servers, advertisement servers, and the like as will be appreciated by those of ordinary skill in the art.
-
FIG. 2 illustrates anexemplary process 200 that may be performed to generate recommendations. In some examples,process 200 may be performed by a computing device, such asserver 110, programmed with recommendation logic, such asrecommendation logic 112, within a computing environment similar or identical tosystem 100. - At
block 201, a request for a recommendation may be received. The recommendation may be received by a computing device (e.g.,server 110 ofFIG. 1 ) via a wired or wireless network (e.g.,network 108 ofFIG. 1 ) from a computing device associated with a user (e.g.,computing device FIG. 1 ). The request may include an identification of the user making the request, the context of the user, a requested item (e.g., user, content, workspace, workflow, task, etc.), other search parameters (e.g., search strings, etc.), or the like. It should be appreciated that the request need not be explicitly made by the user. For example, a user may access a document, and a request for a recommendation based on the requested document may automatically be made. - At
block 203, user and contextual similarity may be determined. A computing device, such asserver 110, programmed with recommendation logic, such asrecommendation logic 112, may determine the user and contextual similarity based on information received from the requesting user atblock 201 and data associated with items, such as users, content, workspaces, workflows, tasks, and the like, stored in a local or remote database (e.g., database 114). - In some examples, an
exemplary process 300, shown inFIG. 3 , may be used to determine the user and contextual similarity.Process 300 may be performed by a computing device, such asserver 110, programmed with recommendation logic, such asrecommendation logic 112, within a computing environment similar or identical tosystem 100. - At
block 301, workflow data may be accessed. In some examples, the data may be accessed from a local or remote workflow database similar or identical todatabase 114 ofFIG. 1 . The workflow data may include social graphs, organization graphs, collaboration graphs, content data, utilization data, ratings data, workflow data, task data, goal data, and the like, associated with users tracked by the system. - At
block 303, user similarity may be determined between the user requesting a recommendation atblock 201 and users associated with the workflow data. For example, if a recommendation is to be made for user A, user similarities may be determined between user A and other users being tracked by therecommendation system 100 whose data is included within the workflow data. The user similarity may be determined using one or more of social similarity, organizational similarity, contextual similarity, and preference similarity. - Social similarity is based on the concept that a user is likely to have needs and preferences similar to those of friends. Additionally, a user is more likely to have similar needs and preferences to a closely related user (e.g., a friend) than a more distantly related user (e.g., a friend of a friend). To determine social similarity, distances between users in a social graph accessed from a database at
block 201 may be determined. For example,FIG. 4 shows asocial graph 400 for user A. As shown,social graph 400 includes nodes corresponding to users A, B, C, D, and E and edges indicating the type of relationships existing between the users and user A. In particular,social graph 400 indicates that users D and B are friends of user A, while users C and E are friends of friends of user A. In one example, to determine social similarity, the inverse of social distance may be used. The social distance between two users represents the number of edges that must be traversed along the shortest path to move from the node of one user to another. For example, the social distance between users A and D is one, since only a single edge must be traversed to get from node A to node D. This equates to a social similarity value of 1 (similarity=1/1). In contrast, the social distance between users A and E is two, since two edges (A to D and D to E) must be traversed to get from node A to node E. This equates to a social similarity value of 0.5 (similarity=1/2). Thus, users A and D are more similar than users A and E. - Organizational similarity may be based on the concept that a user is likely to have needs and preferences similar to others in the same group or those that are organizationally near. Similar to determining social similarity, distances between users in an organization graph accessed from a database at
block 301 may be determined. However, since organization graphs take the form of a tree, different algorithms for computing similarity may be used. The result of these algorithms may indicate that users that are organizationally near have a higher value for this metric. For example,FIG. 5 illustrates anorganization graph 500 for userA. Organization graph 500 includes nodes corresponding to users A, B, C, D, E, F, and G and edges indicating relationships between the users and user A. To determine organizational similarity, the inverse of organizational distance may be used. Organizational distance may be determined by starting with an organization distance of zero and increasing the distance value by 2 for each vertical traversal extending away from the level of the source user and reducing the distance value by 1 for each vertical traversal that moves toward the level of the source user. For example, if user A is the source user, the distance between user A and user A's manager (user B) is two because a vertical traversal extending away from the level of user A is needed to reach user B, indicating that user A and user B are fairly similar. This equates to an organizational similarity value of 0.5 (similarity=1/2). Similarly, the distance between user A and user A's subordinates (users C and D) is 2 because a vertical traversal extending away from the level of A is needed to reach users C and D, likewise indicating that user A is fairly similar to users C and D. In contrast, user A and user A's peer (user E) have a distance value of 1 since a vertical traversal away from user A's level (+2) toward user B, followed by a vertical traversal from user B toward user A's level at E (−1) are needed to get from nodes A to E. This indicates that users A and E are very similar with a summed distance value of 1 (+2−1=1). This equates to an organizational similarity value of 1 (similarity=1/1). - Contextual similarity may be based on the concept that another user that fulfilled the same roles in tasks and workflows is likely to have similar needs and preferences, that users that have generated or consumed similar content are likely to have similar needs and preferences, and the like. To determine contextual similarity, it may be determined if the users have collaborated in the same workspace or on the same workflow, utilized the same content, and the like. For example, a user's profile may include content, users, workflows, workspaces, and the like that the user has interacted with as well as roles that the user may have filled in the workflows. Using key phrase extraction and information retrieval (IR) techniques, comparisons may be made between profiles of different users. For example, known key phrase extraction techniques, such as the Keyphrase Extraction Algorithm (KEA) (described at http://www.nzdl.org/Kea/) or Apache Tika (described at http://tika.apache.org/), and IR techniques, such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index), may be used.
- In some examples, the contextual similarity may take into consideration how recently and/or frequently users have interacted in similar workspaces/workflows or interacted with similar content. The frequency and age of collaboration may be factored into the resulting similarity score using a configurable half-life period, as discussed below with respect to preference similarity (e.g., equations 1-5, discussed below).
- Preference similarity may be based on the concept that users that have expressed preference for similar items, such as users, content, or workflows, are likely to have similar preferences and needs. To determine a user's preference for an item, collected utilization and ratings information accessed at
block 301 from a database similar or identical todatabase 114 may be used. If a user has expressed a preference for an item, that information may be used. These expressions of preference may take the form of a score (e.g., rating from 1-10) or Boolean value (e.g., like/dislike). To account for the different ways preference may be expressed, these ratings may be converted to a normalized value (e.g., real number in the range [−1 . . . 1] or other similarly scaled values). If, however, a user does not specifically rate an item, the user's preference may be derived from comments made about an item in the system. In this case, sentiment analysis (e.g., described at http://en.wikipedia.org/wiki/Sentiment_analysis) may be used to derive a rating in the range [−1 . . . 1] (or other similarly scaled values). Alternatively or in addition, a user's preference may be inferred from the successful completion of workflows and tasks. The level of preference may be derived for a configurable metric related to the task or workflow. A few such examples are the time required for task completion, final task status, or something derived from the content associated with a task, such as the value of a deal. Note that workflows with a negative outcome may affect the preferences negatively. - In some examples, whether the preference is provided by the user or derived/inferred, the frequency and age of utilization and ratings may be taken into account. In these examples, more recent utilization and/or high-frequency utilization may increase a user's effective preference for an item while old and/or low-frequency utilization may reduce the effective preference. A number of mechanisms for computing this are possible. For example, a configurable half-life period may be used. The algorithm for generating a single preference value may configurable within the system, but one example for calculating a preference is provided by equations 1-5, shown below.
-
p=((rp*rw)+(up*uw))/2 (1) -
rp=r*rd (2) -
rd=0.5̂(ra*rh) (3) -
up=(1/(u+1))*ud (4) -
ud=0.5̂(ua*uh) (5) - In the above equation, “p” represents the final preference value in the range [−1 . . . 1] (or other similarly scaled values), “rp” represents the normalized user preference after the value has been decayed, “rw” represents the rating preference weight coefficient, “up” represents the normalized utilization preference after the value has been decayed, “uw” represents the utilization preference weight coefficient, “r” represents the rating value provided by the user in the range [−1 . . . 1] (or other similarly scaled values), “rd” represents the calculated user rating decay coefficient, “ra” represents the age of the most recent user rating (units may be configurable), “rh” represents the configured constant half-life of user ratings, “u” represents the user utilization count for a given item in the range [1 . . . n] (or other similarly scaled values), “ud” represents the calculated utilization decay coefficient, “ua” represents the age of the most recent user utilization (units may be configurable), and “uh” represents the configured constant half-life of user utilization ratings.
- In some examples, the above or other algorithms may be used to calculate a preference score for a user. The preference score may be calculated for a user's preference for other users, content, workflows, and the like. Preferences may be computed on a per-user/per-type basis. This may result in an ordered list of the top preference items and a preference value being identified for each user. Each preference may be represented as a real number preference value in the range [−1 . . . 1] (or other similarly scaled values). In addition, mode information associated with each utilization may be stored with the computed preference. The number of top preferences retained per user may be configurable for performance reasons.
- It should be appreciated that there are a number of different mechanisms that may be used to compute the similarity between two users. Each individual mechanism used to calculate the various user similarities (e.g., social similarity, organizational similarity, contextual similarity, and preference similarity) may return a user-to-user similarity matrix containing a real number similarity rating in the range [0 . . . 1] (or other similarly scaled values). The results from each mechanism may be combined into a single user/user similarity matrix with a similarity rating as a real number value in the range [0 . . . 1] (or other similarly scaled values). Similarity metrics may be combined using configurable weights.
- In some examples, block 303 may be performed after receiving a request for a recommendation at
block 201, or may be pre-computed (e.g., usingprocess 600, described below) at some other designated time (e.g., when a user is added, periodically, when contextual data changes, etc.). Using the determined user similarities, an ordered list of the most similar users may be stored (e.g., in database 114) for each user. The size of this list may be configurable to any desired size. - At
block 305, task similarity may be determined between a task to be completed by a user and tasks associated with the workflow data. For example, the request for a recommendation received atblock 201 may be received from a user attempting to complete a particular task. This task may be compared to task data included within the workflow data accessed atblock 301. Task similarity may be based on the concept that a task may be compared for similarity along a number of axes. The task data may include data identifying a workflow from which the task is derived, users that the task is assigned to, users issuing the task, content associated with the task, and the like. Tasks derived from the same workflow, initiated by similar users, assigned to similar users, having similar content, may be determined to be similar. Thus, to determine task similarity, known key phrase extraction techniques, such as the Keyphrase Extraction Algorithm (KEA) (described at http://www.nzdl.org/Kea/) or Apache Tika (described at http://tika.apache.org/), and IR techniques, such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index), may be used on the task data. In some examples, similarity scores returned by these techniques for each axis of similarity (e.g., workflow, issuing user, assigned users, content, etc.) may be combined (e.g., a weighted average) into a single notion of similarity between two tasks. - In some examples, similar to block 303, block 305 may be performed after receiving a request for a recommendation at
block 201, or may be pre-computed at some other designated time (e.g., when a task is added, a task is modified, content changes, periodically, etc.). Using the determined task similarities, an ordered list of the most similar tasks may be stored (e.g., in database 114) for each task. The size of this list may be configurable to any desired size. For example,process 600 ofFIG. 6 may be performed to extract and aggregate preference data. Atblock 601, new key phrases may be extracted from user profile data, user digital artifacts, and user workflows. Atblock 603, new user roles may be extracted from user workflows. Atblock 605, user interaction records with each item may be updated. This may include updating the frequency and age of the interactions. This may be merged with user ratings (preferences) of the items atblock 607 and used to calculate preference values using equations 1-5 atblock 609. Based onblocks - At
block 307, goal similarity may be determined between a goal of a user and goals associated with the workflow data. Goal similarity may be determined, for example, by comparison between the keywords associated with pairs of tasks to be completed. For example, the request for a recommendation received atblock 201 may be received from a user attempting to accomplish a particular goal. This goal may be compared to goal data included within the workflow data accessed atblock 301. Goal similarity may include, for example, the concept that a goal may be compared for similarity along a number of axes. Thus, to determine goal similarity, known key phrase extraction techniques, such as the Keyphrase Extraction Algorithm (KEA) (described at http://www.nzdl.org/Kea/) or Apache Tika (described at http://tika.apache.org/), and IR techniques, such as the Vector Space Model (described at http://en.wikipedia.org/wiki/Vector_space_model) or Jaccard Index (described at http://en.wikipedia.org/wiki/Jaccard_index), may be used on the goal data. In some examples, similarity scores returned by these techniques for each axis of similarity may be combined (e.g., a weighted average) into a single notion of similarity between two goals. - In some examples, block 307 may be performed after receiving a request for a recommendation at
block 201, or may be pre-computed (e.g., using process 600) at some other designated time (e.g., when a goal is added, a goal is modified, periodically, etc.). Using the determined goal similarities, an ordered list of the most similar goals may be stored (e.g., in database 114) for each goal. The size of this list may be configurable to any desired size. - While blocks of
process 300 are shown and described in a particular order, it should be appreciated that the blocks may be performed in any order and not all blocks need be performed. - Returning to process 200 of
FIG. 2 , after determining user and contextual similarity at block 203 (or afterblock 201 ifblock 203 was pre-computed), the process may proceed to block 205. - At
block 205, the n most similar users may be identified. This may be based on the user similarity determined atblock 303 ofprocess 300. For example, block 303 ofprocess 300 may generate an ordered list of users based on their similarity to the user requesting the recommendation atblock 201. Based on this list, the n most similar users may be identified. The value n represents a configurable value that may be any value. In some examples, n may default to 20. - At
block 207, preferred items of the n most similar users identified atblock 205 may be determined. In some examples, the preferred items may include any type of item, such as workflows, users, contacts, tasks, documents, forms, calendar entries, conference rooms, etc. The possible set types may or may not be predetermined. In other examples, the preferred items may be limited to a subset of item types based on input from the user or may be provided on behalf of the user without the user's knowledge based on the context of the user at the time the request is made. For example, if an application on the user's computing device uses the recommendation system to recommend users to assign a task to, then the application may request a recommendation for the type “user.” The preferred items may be taken from the ordered list of each similar user's list of preferred items that may be stored indatabase 114. - At
block 209, the lists of preferred items determined atblock 207 may be merged into a single ordered list by merge-sorting each similar user's list of preferred items based on the preference values. Duplicates may optionally be removed, retaining only the most preferred items. - At
block 211, additional items may be determined based on context. This may be based on the task similarity determined atblock 305 and the goal similarity determined atblock 307 ofprocess 300. The requesting user's context may be retrieved, and that context, along with any per-request context (e.g., search criteria), may be used to search items indatabase 114. The result ofblock 211 may include one or more ordered lists of items matching or similar to the search criteria (if provided) derived from the user. For example, a list of similar tasks and a list of similar goals determined atblocks block 211. In some examples, items that are related to or similar to these items may also be returned. Each item in the lists may include a similarity score in the range [0 . . . 1] (or other similarly scaled values). - At
block 213, the lists of items determined atblock 211 may be merged into a single ordered list by merge-sorting each list of similar context items. Duplicates may optionally be removed, retaining only the most similar items. - At
block 215, the merged and sorted lists fromblocks blocks - At
block 217, a set of recommendations may be generated and returned to the user. The set of recommendations may include the merge and sorted recommended items generated atblock 215. For example, a computing device (e.g., server 110) may transmit some or all of the set of recommended items to a computing device associated with the user (e.g.,computing device - The following examples are provided to illustrate the operation of
processes - In the first example, User A has been injured at the workplace and wants to get paid worker's compensation. In this example, User A is the user and obtaining worker's compensation is the goal. To accomplish this goal, User A has a vague idea that he needs to get at least one form approved, but does not know which forms to get, where to find them, or who needs to sign them. Among the list of documents available in the Human Resources portal accessible by a workflow management application on his computing device may be one labeled “Worker's Compensation.” In response to User A requesting the document, the workflow application may cause a display of a workflow (e.g., as shown in
FIG. 7 ). - In this example, User A found the correct document, but now he needs to know where to send it. To determine the destination of the document, User A may click on the workflow application's “Suggest Next Step” button shown in
FIG. 7 . In response to a selection of the “Suggest Next Step” button,server 110 may begin performingprocess 200. In particular, processes 200 and 300 may be performed to identify other users similar to User A that have previously requested the same form.Server 110 may find that other users most similar to User A requesting the same form all submitted it to the person in the organizational chart who is their supervisor or boss.Server 110 may thus recommend that User A submit the form to his boss (e.g., as shown inFIG. 8 ). - In another example, instead of selecting the “Worker's Compensation” form, User A is presented with document A and document B but does not know which one to select. In this example, User A may request a recommendation from
system 100. To generate thisrecommendation using processes - Further, in this example, it may have been determined that four months ago, User B rated document A at 8 out of 10 stars, and six months ago, User C rated document B at 6 out of 10 stars. To use these values in equations 1-5, they may be normalized to a value within the range of [−1 . . . 1] (or other similarly scaled values), where zero stars is equal to −1 and 10 stars is equal to 1.0. Thus, the 6 stars may be normalized to 0.2 while the 8 stars becomes normalized to 0.6. Additionally, document A may have been accessed three times and document B may have been accessed two times. All three accesses of document A may have been four months ago. One access of document B may have been six months ago while the other occurred 2 months ago. Given this activity, the following may be quantified for documents A and B:
-
A B r 0.6 0.2 Rating value provided by the user in the range [−1 . . . 1] (or other similarly scaled values). ra 4 6 Age of the most recent user rating. Units are configurable. u 3 2 User utilization count for a given item in the range [1 . . . n] (or other similarly scaled values). ua 4 2 Age of the most recent user utilization. Units are configurable. - Using equations 2-5, these variables may be calculated to be:
-
A B rp 0.017 0.066 Normalized user preference after the value has been decayed. rw 0.891 0.841 Rating preference weight coefficient. up 0.224 0.298 Normalized utilization preference after the value has been decayed. uw 0.891 0.944 Utilization preference weight coefficient. - The final preference value may be determined using
equation 1. The resulting preferences based on the values above are A=0.107 and B=0.009. Thus, document A may be preferred over document B. - In another example, User A may have the same problem and knowledge discussed above, but instead of clicking “Suggest Next Step,” User A may select “Suggest a Workflow” in the interface of
FIG. 7 . In response to the selection,server 110 may begin performingprocess 200. In particular, processes 200 and 300 may be performed to identify other users similar to User A that have previously requested the same form.Server 110 may further evaluate other workflows and, based on similarities, display several potential workflows that represent the paths others took in the same situation. These workflows may include steps and concepts that User A was unaware of, including: -
- requesting the Patient's Bill of Rights document
- that his supervisor needs to notify the company's insurer
- that the insurer must accept the claim
- that the insurer will instruct User A to see a doctor
- that User A schedule this appointment
- that User A attend this appointment
- that the doctor may recommend treatment from a therapist
- that User A schedule this appointment
- that User A get this therapy
- that User A schedule a follow-up appointment with the doctor
- that User A attend this appointment
- that the doctor notify the insurer that the patient has been discharged
- User A may now have a better idea of what to do next, what future steps he will need to take, what unexpected events may occur along the way (e.g., therapy), how long it all may take, etc.
- Using the processes provided above, recommendations may be generated based on user similarities and contextual similarities. In particular, collaborative filtering, key phrase extraction, and IR techniques may be used. This advantageously allows the system to make recommendations based on various types of data, resulting in the production of recommendations when certain types of data are unavailable. Additionally, the system may provide recommendations for items already known to the user. This allows the system to provide a recommendation for an item that the user interacted with before but may be unaware could be useful in a particular context.
-
FIG. 9 depicts anexemplary computing system 900 configured to perform any one of the above-described processes. In this context,computing system 900 may include, for example, a processor, memory, storage, and input/output devices (e.g., monitor, keyboard, disk drive, Internet connection, etc.). However,computing system 900 may include circuitry or other specialized hardware for carrying out some or all aspects of the processes. In some operational settings,computing system 900 may be configured as a system that includes one or more units, each of which is configured to carry out some aspects of the processes either in software, hardware, or some combination thereof. -
FIG. 9 depictscomputing system 900 with a number of components that may be used to perform the above-described processes. Themain system 902 includes amotherboard 904 having an input/output (“I/O”)section 906, one or more central processing units (CPUs) 908, and amemory section 910, which may have aflash memory card 912 related to it. The I/O section 906 is connected to adisplay 924, akeyboard 914, adisk storage unit 916, and amedia drive unit 918. Themedia drive unit 918 may read/write a computer-readable medium 920, which may containprograms 922 and/or data. - At least some values based on the results of the above-described processes may be saved for subsequent use. Additionally, a non-transitory computer-readable medium may be used to store (e.g., tangibly embody) one or more computer programs for performing any one of the above-described processes by means of a computer. The computer program may be written, for example, in a general-purpose programming language (e.g., Pascal, C, C++, Java) or some specialized application-specific language.
- Although only certain exemplary embodiments have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of the present disclosure. For example, aspects of embodiments disclosed above may be combined in other combinations to form additional embodiments. Accordingly, all such modifications are intended to be included within the scope of the present disclosure.
Claims (30)
1. A computer-implemented method for generating workflow recommendations for a user, the method comprising:
receiving, from a computing device of a user, a request for a recommendation;
determining a plurality of user similarity scores between the user and a plurality of users;
determining a plurality of contextual similarity scores between a context of the user and a context of a plurality of items;
determining a first set of recommended items based on the plurality of user similarity scores;
determining a second set of recommended items based on the plurality of contextual similarity scores;
generating an aggregated set of recommended items based on the first set of recommended items and the second set of recommended items; and
transmitting, to the computing device of the user, the set of aggregated recommended items.
2. The computer-implemented method of claim 1 , wherein determining the plurality of user similarity scores is based on one or more of a social graph and an organization graph.
3. The computer-implemented method of claim 1 , wherein determining the first set of recommended items based on the plurality of user similarity scores comprises:
generating a ranked list of the plurality of users based on the plurality of user similarity scores;
identifying a subset of similar users based on the ranked list of the plurality of users;
for each user of the subset of similar users, identifying a list of preferred items for that user;
merging and ranking items in the lists of preferred items into a first combined list of preferred items; and
determining the first set of recommended items based on the first combined list of preferred items.
4. The computer-implemented method of claim 1 , wherein determining the plurality of contextual similarity scores between the context of the user and the context of the plurality of items comprises:
determining a task similarity score between a task to be completed by the user and each task of the plurality of items; and
determining a goal similarity score between a goal of the user and each goal of the plurality of items.
5. The computer-implemented method of claim 4 , wherein determining the task similarity score comprises comparing an associated workflow, an initiating user, an assignment, or an associated content of the task to be completed by the user with an associated workflow, an initiating user, an assignment, and an associated content of each task of the plurality of items.
6. The computer-implemented method of claim 4 , wherein determining the goal similarity score comprises performing an information retrieval operation on the goal of the user and the goal associated with each goal of the plurality of items.
7. The computer-implemented method of claim 4 , wherein determining the second set of recommended items based on the plurality of contextual similarity scores comprises:
generating a ranked list of the plurality of items based on the task similarity scores and the goal similarity scores; and
determining the second set of recommended items based on the ranked list of the plurality of items.
8. The computer-implemented method of claim 1 , wherein generating the aggregated set of recommended items based on the first set of recommended items and the second set of recommended items comprises:
merging and ranking items in the first set of recommended items and the second set of recommended items into a second combined list of preferred items; and
generating the aggregated set of recommended items based on the second combined list of preferred items.
9. The computer-implemented method of claim 1 , wherein merging and ranking items in the first set of recommended items and the second set of recommended items into the second combined list of preferred items comprises:
determining a weighted average of scores of the items in the first set of recommended items and scores of the items in the second set of recommended items; and
ranking the items of the first set of recommended items and the second set of recommended items based on the determined weighted average scores.
10. The computer-implemented method of claim 1 , wherein the aggregated set of recommended items comprises a recommended document, a recommended task, a recommended workflow, or an identification of a recommended user.
11. A non-transitory computer-readable storage medium comprising computer-executable instructions for generating workflow recommendations for a user, the computer-executable instructions comprising instructions for:
receiving, from a computing device of a user, a request for a recommendation;
determining a plurality of user similarity scores between the user and a plurality of users;
determining a plurality of contextual similarity scores between a context of the user and a context of a plurality of items;
determining a first set of recommended items based on the plurality of user similarity scores;
determining a second set of recommended items based on the plurality of contextual similarity scores;
generating an aggregated set of recommended items based on the first set of recommended items and the second set of recommended items; and
transmitting, to the computing device of the user, the set of aggregated recommended items.
12. The non-transitory computer-readable storage medium of claim 11 , wherein determining the plurality of user similarity scores is based on one or more of a social graph and an organization graph.
13. The non-transitory computer-readable storage medium of claim 11 , wherein determining the first set of recommended items based on the plurality of user similarity scores comprises:
generating a ranked list of the plurality of users based on the plurality of user similarity scores;
identifying a subset of similar users based on the ranked list of the plurality of users;
for each user of the subset of similar users, identifying a list of preferred items for that user;
merging and ranking items in the lists of preferred items into a first combined list of preferred items; and
determining the first set of recommended items based on the first combined list of preferred items.
14. The non-transitory computer-readable storage medium of claim 11 , wherein determining the plurality of contextual similarity scores between the context of the user and the context of the plurality of items comprises:
determining a task similarity score between a task to be completed by the user and each task of the plurality of items; and
determining a goal similarity score between a goal of the user and each goal of the plurality of items.
15. The non-transitory computer-readable storage medium of claim 14 , wherein determining the task similarity score comprises comparing an associated workflow, an initiating user, an assignment, or an associated content of the task to be completed by the user with an associated workflow, an initiating user, an assignment, and an associated content of each task of the plurality of items.
16. The non-transitory computer-readable storage medium of claim 14 , wherein determining the goal similarity score comprises performing an information retrieval operation on the goal of the user and the goal associated with each goal of the plurality of items.
17. The non-transitory computer-readable storage medium of claim 14 , wherein determining the second set of recommended items based on the plurality of contextual similarity scores comprises:
generating a ranked list of the plurality of items based on the task similarity scores and the goal similarity scores; and
determining the second set of recommended items based on the ranked list of the plurality of items.
18. The non-transitory computer-readable storage medium of claim 11 , wherein generating the aggregated set of recommended items based on the first set of recommended items and the second set of recommended items comprises:
merging and ranking items in the first set of recommended items and the second set of recommended items into a second combined list of preferred items; and
generating the aggregated set of recommended items based on the second combined list of preferred items.
19. The non-transitory computer-readable storage medium of claim 11 , wherein merging and ranking items in the first set of recommended items and the second set of recommended items into the second combined list of preferred items comprises:
determining a weighted average of scores of the items in the first set of recommended items and scores of the items in the second set of recommended items; and
ranking the items of the first set of recommended items and the second set of recommended items based on the determined weighted average scores.
20. The non-transitory computer-readable storage medium of claim 11 , wherein the aggregated set of recommended items comprises a recommended document, a recommended task, a recommended workflow, or an identification of a recommended user.
21. An apparatus for generating workflow recommendations for a user, the apparatus comprising:
a memory comprising computer-executable instructions for:
receiving, from a computing device of a user, a request for a recommendation;
determining a plurality of user similarity scores between the user and a plurality of users;
determining a plurality of contextual similarity scores between a context of the user and a context of a plurality of items;
determining a first set of recommended items based on the plurality of user similarity scores;
determining a second set of recommended items based on the plurality of contextual similarity scores;
generating an aggregated set of recommended items based on the first set of recommended items and the second set of recommended items; and
transmitting, to the computing device of the user, the set of aggregated recommended items; and
a processor for executing the computer-executable instructions.
22. The apparatus of claim 21 , wherein determining the plurality of user similarity scores is based on one or more of a social graph and an organization graph.
23. The apparatus of claim 21 , wherein determining the first set of recommended items based on the plurality of user similarity scores comprises:
generating a ranked list of the plurality of users based on the plurality of user similarity scores;
identifying a subset of similar users based on the ranked list of the plurality of users;
for each user of the subset of similar users, identifying a list of preferred items for that user;
merging and ranking items in the lists of preferred items into a first combined list of preferred items; and
determining the first set of recommended items based on the first combined list of preferred items.
24. The apparatus of claim 21 , wherein determining the plurality of contextual similarity scores between the context of the user and the context of the plurality of items comprises:
determining a task similarity score between a task to be completed by the user and each task of the plurality of items; and
determining a goal similarity score between a goal of the user and each goal of the plurality of items.
25. The apparatus of claim 24 , wherein determining the task similarity score comprises comparing an associated workflow, an initiating user, an assignment, or an associated content of the task to be completed by the user with an associated workflow, an initiating user, an assignment, and an associated content of each task of the plurality of items.
26. The apparatus of claim 24 , wherein determining the goal similarity score comprises performing an information retrieval operation on the goal of the user and the goal associated with each goal of the plurality of items.
27. The apparatus of claim 24 , wherein determining the second set of recommended items based on the plurality of contextual similarity scores comprises:
generating a ranked list of the plurality of items based on the task similarity scores and the goal similarity scores; and
determining the second set of recommended items based on the ranked list of the plurality of items.
28. The apparatus of claim 21 , wherein generating the aggregated set of recommended items based on the first set of recommended items and the second set of recommended items comprises:
merging and ranking items in the first set of recommended items and the second set of recommended items into a second combined list of preferred items; and
generating the aggregated set of recommended items based on the second combined list of preferred items.
29. The apparatus of claim 21 , wherein merging and ranking items in the first set of recommended items and the second set of recommended items into the second combined list of preferred items comprises:
determining a weighted average of scores of the items in the first set of recommended items and scores of the items in the second set of recommended items; and
ranking the items of the first set of recommended items and the second set of recommended items based on the determined weighted average scores.
30. The apparatus of claim 21 , wherein the aggregated set of recommended items comprises a recommended document, a recommended task, a recommended workflow, or an identification of a recommended user.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/844,114 US20140074545A1 (en) | 2012-09-07 | 2013-03-15 | Human workflow aware recommendation engine |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201261698514P | 2012-09-07 | 2012-09-07 | |
US13/844,114 US20140074545A1 (en) | 2012-09-07 | 2013-03-15 | Human workflow aware recommendation engine |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140074545A1 true US20140074545A1 (en) | 2014-03-13 |
Family
ID=50234246
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/844,114 Abandoned US20140074545A1 (en) | 2012-09-07 | 2013-03-15 | Human workflow aware recommendation engine |
Country Status (2)
Country | Link |
---|---|
US (1) | US20140074545A1 (en) |
WO (1) | WO2014039898A2 (en) |
Cited By (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140207506A1 (en) * | 2013-01-21 | 2014-07-24 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending a workflow |
US20140207801A1 (en) * | 2013-01-21 | 2014-07-24 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending events |
CN104166732A (en) * | 2014-08-29 | 2014-11-26 | 合肥工业大学 | Project collaboration filtering recommendation method based on global scoring information |
US20150019640A1 (en) * | 2013-07-15 | 2015-01-15 | Facebook, Inc. | Large Scale Page Recommendations on Online Social Networks |
US20160110363A1 (en) * | 2014-10-21 | 2016-04-21 | Anatoliy TKACH | Method and system for measuring and matching individual cultural preferences and for targeting of culture related content and advertising to the most relevant audience |
US9396236B1 (en) * | 2013-12-31 | 2016-07-19 | Google Inc. | Ranking users based on contextual factors |
US20160328406A1 (en) * | 2015-05-08 | 2016-11-10 | Informatica Llc | Interactive recommendation of data sets for data analysis |
US20170004434A1 (en) * | 2015-06-30 | 2017-01-05 | International Business Machines Corporation | Determining Individual Performance Dynamics Using Federated Interaction Graph Analytics |
US20170200112A1 (en) * | 2016-01-13 | 2017-07-13 | International Business Machines Corporation | Managing a set of shared tasks using biometric data |
WO2017116931A3 (en) * | 2015-12-29 | 2017-08-31 | Crowd Computing Systems, Inc. | Task similarity clusters for worker assessment |
US9910487B1 (en) * | 2013-08-16 | 2018-03-06 | Ca, Inc. | Methods, systems and computer program products for guiding users through task flow paths |
US9928241B2 (en) * | 2014-03-18 | 2018-03-27 | Smartsheet Inc. | Systems and methods for analyzing electronic communications to dynamically improve efficiency and visualization of collaborative work environments |
US20180103118A1 (en) * | 2016-10-11 | 2018-04-12 | Synergex Group | Methods, systems, and media for pairing devices to complete a task using an application request |
WO2018075334A1 (en) * | 2016-10-20 | 2018-04-26 | Microsoft Technology Licensing, Llc | Collaborator recommendation using collaboration graphs |
KR101854912B1 (en) * | 2018-03-07 | 2018-05-04 | 주식회사 텐디 | Method of analyzing correlation between applications and apparatus for analyzing correlation between applications |
US20190147404A1 (en) * | 2017-11-16 | 2019-05-16 | Salesforce.Com, Inc. | Email streaming records |
US10311384B2 (en) | 2015-07-29 | 2019-06-04 | Microsoft Technology Licensing, Llc | Automatic creation and maintenance of a taskline |
US20190207946A1 (en) * | 2016-12-20 | 2019-07-04 | Google Inc. | Conditional provision of access by interactive assistant modules |
US20200065736A1 (en) * | 2018-08-23 | 2020-02-27 | Capital One Sercices, LLC | Task management platform |
US10685187B2 (en) | 2017-05-15 | 2020-06-16 | Google Llc | Providing access to user-controlled resources by automated assistants |
US10699105B2 (en) | 2017-06-02 | 2020-06-30 | International Business Machines Corporation | Workflow creation by image analysis |
CN111400616A (en) * | 2020-03-31 | 2020-07-10 | 北京达佳互联信息技术有限公司 | Account recommendation method and device |
US11087023B2 (en) | 2018-08-07 | 2021-08-10 | Google Llc | Threshold-based assembly of automated assistant responses |
US11277452B2 (en) | 2020-05-01 | 2022-03-15 | Monday.com Ltd. | Digital processing systems and methods for multi-board mirroring of consolidated information in collaborative work systems |
US11277361B2 (en) | 2020-05-03 | 2022-03-15 | Monday.com Ltd. | Digital processing systems and methods for variable hang-time for social layer messages in collaborative work systems |
US11301623B2 (en) | 2020-02-12 | 2022-04-12 | Monday.com Ltd | Digital processing systems and methods for hybrid scaling/snap zoom function in table views of collaborative work systems |
US11307753B2 (en) | 2019-11-18 | 2022-04-19 | Monday.Com | Systems and methods for automating tablature in collaborative work systems |
US11361156B2 (en) | 2019-11-18 | 2022-06-14 | Monday.Com | Digital processing systems and methods for real-time status aggregation in collaborative work systems |
US11392556B1 (en) | 2021-01-14 | 2022-07-19 | Monday.com Ltd. | Digital processing systems and methods for draft and time slider for presentations in collaborative work systems |
US11410129B2 (en) | 2010-05-01 | 2022-08-09 | Monday.com Ltd. | Digital processing systems and methods for two-way syncing with third party applications in collaborative work systems |
US11423500B2 (en) | 2019-12-12 | 2022-08-23 | Netspective Communications Llc | Computer-controlled precision education and training |
US11436359B2 (en) | 2018-07-04 | 2022-09-06 | Monday.com Ltd. | System and method for managing permissions of users for a single data type column-oriented data structure |
US11436417B2 (en) | 2017-05-15 | 2022-09-06 | Google Llc | Providing access to user-controlled resources by automated assistants |
US20220286362A1 (en) * | 2021-03-05 | 2022-09-08 | Dell Products, L.P. | Data Driven Chaos Engineering Based on Service Mesh and Organizational Chart |
US20230176831A1 (en) * | 2021-12-06 | 2023-06-08 | International Business Machines Corporation | Microservices recommendation framework |
US11698890B2 (en) | 2018-07-04 | 2023-07-11 | Monday.com Ltd. | System and method for generating a column-oriented data structure repository for columns of single data types |
US11741071B1 (en) | 2022-12-28 | 2023-08-29 | Monday.com Ltd. | Digital processing systems and methods for navigating and viewing displayed content |
US11803771B2 (en) | 2018-03-24 | 2023-10-31 | Autodesk, Inc. | Techniques for classifying and recommending software workflows |
US11829953B1 (en) | 2020-05-01 | 2023-11-28 | Monday.com Ltd. | Digital processing systems and methods for managing sprints using linked electronic boards |
US11886683B1 (en) | 2022-12-30 | 2024-01-30 | Monday.com Ltd | Digital processing systems and methods for presenting board graphics |
US11893381B1 (en) | 2023-02-21 | 2024-02-06 | Monday.com Ltd | Digital processing systems and methods for reducing file bundle sizes |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11030542B2 (en) | 2016-04-29 | 2021-06-08 | Microsoft Technology Licensing, Llc | Contextually-aware selection of event forums |
CN111368211B (en) * | 2020-02-20 | 2023-05-16 | 腾讯科技(深圳)有限公司 | Relation chain determining method, device and storage medium |
CN114282976A (en) * | 2021-12-27 | 2022-04-05 | 赛尔网络有限公司 | Supplier recommendation method and device, electronic equipment and medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5446891A (en) * | 1992-02-26 | 1995-08-29 | International Business Machines Corporation | System for adjusting hypertext links with weighed user goals and activities |
US6850895B2 (en) * | 1998-11-30 | 2005-02-01 | Siebel Systems, Inc. | Assignment manager |
US7761393B2 (en) * | 2006-06-27 | 2010-07-20 | Microsoft Corporation | Creating and managing activity-centric workflow |
US20100306016A1 (en) * | 2009-05-27 | 2010-12-02 | Microsoft Corporation | Personalized task recommendations |
US20110179025A1 (en) * | 2010-01-21 | 2011-07-21 | Kryptonite Systems Inc | Social and contextual searching for enterprise business applications |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8386406B2 (en) * | 2009-07-08 | 2013-02-26 | Ebay Inc. | Systems and methods for making contextual recommendations |
-
2013
- 2013-03-15 US US13/844,114 patent/US20140074545A1/en not_active Abandoned
- 2013-09-06 WO PCT/US2013/058613 patent/WO2014039898A2/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5446891A (en) * | 1992-02-26 | 1995-08-29 | International Business Machines Corporation | System for adjusting hypertext links with weighed user goals and activities |
US6850895B2 (en) * | 1998-11-30 | 2005-02-01 | Siebel Systems, Inc. | Assignment manager |
US7761393B2 (en) * | 2006-06-27 | 2010-07-20 | Microsoft Corporation | Creating and managing activity-centric workflow |
US20100306016A1 (en) * | 2009-05-27 | 2010-12-02 | Microsoft Corporation | Personalized task recommendations |
US20110179025A1 (en) * | 2010-01-21 | 2011-07-21 | Kryptonite Systems Inc | Social and contextual searching for enterprise business applications |
Cited By (107)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11410129B2 (en) | 2010-05-01 | 2022-08-09 | Monday.com Ltd. | Digital processing systems and methods for two-way syncing with third party applications in collaborative work systems |
US20140207801A1 (en) * | 2013-01-21 | 2014-07-24 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending events |
US20190272267A1 (en) * | 2013-01-21 | 2019-09-05 | Salesforce.Com, Inc. | Generating or updating sets of events using a database system |
US10565217B2 (en) | 2013-01-21 | 2020-02-18 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending events |
US9892170B2 (en) | 2013-01-21 | 2018-02-13 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending events |
US9607090B2 (en) * | 2013-01-21 | 2017-03-28 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending events |
US20140207506A1 (en) * | 2013-01-21 | 2014-07-24 | Salesforce.Com, Inc. | Computer implemented methods and apparatus for recommending a workflow |
US10367862B2 (en) * | 2013-07-15 | 2019-07-30 | Facebook, Inc. | Large-scale page recommendations on online social networks |
US20150019640A1 (en) * | 2013-07-15 | 2015-01-15 | Facebook, Inc. | Large Scale Page Recommendations on Online Social Networks |
US9552055B2 (en) * | 2013-07-15 | 2017-01-24 | Facebook, Inc. | Large scale page recommendations on online social networks |
US20170085607A1 (en) * | 2013-07-15 | 2017-03-23 | Facebook, Inc. | Large-Scale Page Recommendations on Online Social Networks |
US9910487B1 (en) * | 2013-08-16 | 2018-03-06 | Ca, Inc. | Methods, systems and computer program products for guiding users through task flow paths |
US9396236B1 (en) * | 2013-12-31 | 2016-07-19 | Google Inc. | Ranking users based on contextual factors |
US10133790B1 (en) | 2013-12-31 | 2018-11-20 | Google Llc | Ranking users based on contextual factors |
US9928241B2 (en) * | 2014-03-18 | 2018-03-27 | Smartsheet Inc. | Systems and methods for analyzing electronic communications to dynamically improve efficiency and visualization of collaborative work environments |
CN104166732A (en) * | 2014-08-29 | 2014-11-26 | 合肥工业大学 | Project collaboration filtering recommendation method based on global scoring information |
US20160110363A1 (en) * | 2014-10-21 | 2016-04-21 | Anatoliy TKACH | Method and system for measuring and matching individual cultural preferences and for targeting of culture related content and advertising to the most relevant audience |
US20160328406A1 (en) * | 2015-05-08 | 2016-11-10 | Informatica Llc | Interactive recommendation of data sets for data analysis |
US20170004434A1 (en) * | 2015-06-30 | 2017-01-05 | International Business Machines Corporation | Determining Individual Performance Dynamics Using Federated Interaction Graph Analytics |
US10311384B2 (en) | 2015-07-29 | 2019-06-04 | Microsoft Technology Licensing, Llc | Automatic creation and maintenance of a taskline |
US10755221B2 (en) | 2015-12-29 | 2020-08-25 | Workfusion, Inc. | Worker answer confidence estimation for worker assessment |
US11074537B2 (en) | 2015-12-29 | 2021-07-27 | Workfusion, Inc. | Candidate answer fraud for worker assessment |
US11074536B2 (en) | 2015-12-29 | 2021-07-27 | Workfusion, Inc. | Worker similarity clusters for worker assessment |
US10878360B2 (en) | 2015-12-29 | 2020-12-29 | Workfusion, Inc. | Candidate answer fraud for worker assessment |
US11074535B2 (en) | 2015-12-29 | 2021-07-27 | Workfusion, Inc. | Best worker available for worker assessment |
US10776741B2 (en) | 2015-12-29 | 2020-09-15 | Workfusion, Inc. | Task-level answer confidence estimation for worker assessment |
WO2017116931A3 (en) * | 2015-12-29 | 2017-08-31 | Crowd Computing Systems, Inc. | Task similarity clusters for worker assessment |
US10726377B2 (en) | 2015-12-29 | 2020-07-28 | Workfusion, Inc. | Task similarity clusters for worker assessment |
US20170200112A1 (en) * | 2016-01-13 | 2017-07-13 | International Business Machines Corporation | Managing a set of shared tasks using biometric data |
US20180103118A1 (en) * | 2016-10-11 | 2018-04-12 | Synergex Group | Methods, systems, and media for pairing devices to complete a task using an application request |
US10834231B2 (en) * | 2016-10-11 | 2020-11-10 | Synergex Group | Methods, systems, and media for pairing devices to complete a task using an application request |
WO2018075334A1 (en) * | 2016-10-20 | 2018-04-26 | Microsoft Technology Licensing, Llc | Collaborator recommendation using collaboration graphs |
US20190207946A1 (en) * | 2016-12-20 | 2019-07-04 | Google Inc. | Conditional provision of access by interactive assistant modules |
US10685187B2 (en) | 2017-05-15 | 2020-06-16 | Google Llc | Providing access to user-controlled resources by automated assistants |
US11436417B2 (en) | 2017-05-15 | 2022-09-06 | Google Llc | Providing access to user-controlled resources by automated assistants |
US10699105B2 (en) | 2017-06-02 | 2020-06-30 | International Business Machines Corporation | Workflow creation by image analysis |
US20190147404A1 (en) * | 2017-11-16 | 2019-05-16 | Salesforce.Com, Inc. | Email streaming records |
KR101854912B1 (en) * | 2018-03-07 | 2018-05-04 | 주식회사 텐디 | Method of analyzing correlation between applications and apparatus for analyzing correlation between applications |
US11823015B2 (en) * | 2018-03-24 | 2023-11-21 | Autodesk, Inc. | Techniques for classifying and recommending software workflows |
US11803771B2 (en) | 2018-03-24 | 2023-10-31 | Autodesk, Inc. | Techniques for classifying and recommending software workflows |
US11698890B2 (en) | 2018-07-04 | 2023-07-11 | Monday.com Ltd. | System and method for generating a column-oriented data structure repository for columns of single data types |
US11436359B2 (en) | 2018-07-04 | 2022-09-06 | Monday.com Ltd. | System and method for managing permissions of users for a single data type column-oriented data structure |
US11966494B2 (en) | 2018-08-07 | 2024-04-23 | Google Llc | Threshold-based assembly of remote automated assistant responses |
US11314890B2 (en) | 2018-08-07 | 2022-04-26 | Google Llc | Threshold-based assembly of remote automated assistant responses |
US11790114B2 (en) | 2018-08-07 | 2023-10-17 | Google Llc | Threshold-based assembly of automated assistant responses |
US11455418B2 (en) | 2018-08-07 | 2022-09-27 | Google Llc | Assembling and evaluating automated assistant responses for privacy concerns |
US20220083687A1 (en) | 2018-08-07 | 2022-03-17 | Google Llc | Threshold-based assembly of remote automated assistant responses |
US11087023B2 (en) | 2018-08-07 | 2021-08-10 | Google Llc | Threshold-based assembly of automated assistant responses |
US11822695B2 (en) | 2018-08-07 | 2023-11-21 | Google Llc | Assembling and evaluating automated assistant responses for privacy concerns |
US20200065736A1 (en) * | 2018-08-23 | 2020-02-27 | Capital One Sercices, LLC | Task management platform |
US10891571B2 (en) * | 2018-08-23 | 2021-01-12 | Capital One Services, Llc | Task management platform |
US11507738B2 (en) | 2019-11-18 | 2022-11-22 | Monday.Com | Digital processing systems and methods for automatic updates in collaborative work systems |
US11727323B2 (en) | 2019-11-18 | 2023-08-15 | Monday.Com | Digital processing systems and methods for dual permission access in tables of collaborative work systems |
US11526661B2 (en) | 2019-11-18 | 2022-12-13 | Monday.com Ltd. | Digital processing systems and methods for integrated communications module in tables of collaborative work systems |
US11361156B2 (en) | 2019-11-18 | 2022-06-14 | Monday.Com | Digital processing systems and methods for real-time status aggregation in collaborative work systems |
US11775890B2 (en) | 2019-11-18 | 2023-10-03 | Monday.Com | Digital processing systems and methods for map-based data organization in collaborative work systems |
US11307753B2 (en) | 2019-11-18 | 2022-04-19 | Monday.Com | Systems and methods for automating tablature in collaborative work systems |
US11651459B2 (en) | 2019-12-12 | 2023-05-16 | Netspective Communications Llc | Computer-controlled precision education and training |
US11423500B2 (en) | 2019-12-12 | 2022-08-23 | Netspective Communications Llc | Computer-controlled precision education and training |
US11301623B2 (en) | 2020-02-12 | 2022-04-12 | Monday.com Ltd | Digital processing systems and methods for hybrid scaling/snap zoom function in table views of collaborative work systems |
CN111400616A (en) * | 2020-03-31 | 2020-07-10 | 北京达佳互联信息技术有限公司 | Account recommendation method and device |
US11348070B2 (en) | 2020-05-01 | 2022-05-31 | Monday.com Ltd. | Digital processing systems and methods for context based analysis during generation of sub-board templates in collaborative work systems |
US11687706B2 (en) | 2020-05-01 | 2023-06-27 | Monday.com Ltd. | Digital processing systems and methods for automatic display of value types based on custom heading in collaborative work systems |
US11410128B2 (en) | 2020-05-01 | 2022-08-09 | Monday.com Ltd. | Digital processing systems and methods for recommendation engine for automations in collaborative work systems |
US11277452B2 (en) | 2020-05-01 | 2022-03-15 | Monday.com Ltd. | Digital processing systems and methods for multi-board mirroring of consolidated information in collaborative work systems |
US11397922B2 (en) | 2020-05-01 | 2022-07-26 | Monday.Com, Ltd. | Digital processing systems and methods for multi-board automation triggers in collaborative work systems |
US11954428B2 (en) | 2020-05-01 | 2024-04-09 | Monday.com Ltd. | Digital processing systems and methods for accessing another's display via social layer interactions in collaborative work systems |
US11907653B2 (en) * | 2020-05-01 | 2024-02-20 | Monday.com Ltd. | Digital processing systems and methods for network map visualizations of team interactions in collaborative work systems |
US11886804B2 (en) | 2020-05-01 | 2024-01-30 | Monday.com Ltd. | Digital processing systems and methods for self-configuring automation packages in collaborative work systems |
US11475408B2 (en) | 2020-05-01 | 2022-10-18 | Monday.com Ltd. | Digital processing systems and methods for automation troubleshooting tool in collaborative work systems |
US11829953B1 (en) | 2020-05-01 | 2023-11-28 | Monday.com Ltd. | Digital processing systems and methods for managing sprints using linked electronic boards |
US11275742B2 (en) | 2020-05-01 | 2022-03-15 | Monday.com Ltd. | Digital processing systems and methods for smart table filter with embedded boolean logic in collaborative work systems |
US11501256B2 (en) | 2020-05-01 | 2022-11-15 | Monday.com Ltd. | Digital processing systems and methods for data visualization extrapolation engine for item extraction and mapping in collaborative work systems |
US11501255B2 (en) | 2020-05-01 | 2022-11-15 | Monday.com Ltd. | Digital processing systems and methods for virtual file-based electronic white board in collaborative work systems |
US11367050B2 (en) | 2020-05-01 | 2022-06-21 | Monday.Com, Ltd. | Digital processing systems and methods for customized chart generation based on table data selection in collaborative work systems |
US11354624B2 (en) | 2020-05-01 | 2022-06-07 | Monday.com Ltd. | Digital processing systems and methods for dynamic customized user experience that changes over time in collaborative work systems |
US11531966B2 (en) | 2020-05-01 | 2022-12-20 | Monday.com Ltd. | Digital processing systems and methods for digital sound simulation system |
US11282037B2 (en) | 2020-05-01 | 2022-03-22 | Monday.com Ltd. | Digital processing systems and methods for graphical interface for aggregating and dissociating data from multiple tables in collaborative work systems |
US11537991B2 (en) | 2020-05-01 | 2022-12-27 | Monday.com Ltd. | Digital processing systems and methods for pre-populating templates in a tablature system |
US11587039B2 (en) | 2020-05-01 | 2023-02-21 | Monday.com Ltd. | Digital processing systems and methods for communications triggering table entries in collaborative work systems |
US11347721B2 (en) | 2020-05-01 | 2022-05-31 | Monday.com Ltd. | Digital processing systems and methods for automatic application of sub-board templates in collaborative work systems |
US11301814B2 (en) | 2020-05-01 | 2022-04-12 | Monday.com Ltd. | Digital processing systems and methods for column automation recommendation engine in collaborative work systems |
US11675972B2 (en) | 2020-05-01 | 2023-06-13 | Monday.com Ltd. | Digital processing systems and methods for digital workflow system dispensing physical reward in collaborative work systems |
US11301811B2 (en) | 2020-05-01 | 2022-04-12 | Monday.com Ltd. | Digital processing systems and methods for self-monitoring software recommending more efficient tool usage in collaborative work systems |
US11416820B2 (en) | 2020-05-01 | 2022-08-16 | Monday.com Ltd. | Digital processing systems and methods for third party blocks in automations in collaborative work systems |
US11301813B2 (en) | 2020-05-01 | 2022-04-12 | Monday.com Ltd. | Digital processing systems and methods for hierarchical table structure with conditional linking rules in collaborative work systems |
US11301812B2 (en) | 2020-05-01 | 2022-04-12 | Monday.com Ltd. | Digital processing systems and methods for data visualization extrapolation engine for widget 360 in collaborative work systems |
US11755827B2 (en) | 2020-05-01 | 2023-09-12 | Monday.com Ltd. | Digital processing systems and methods for stripping data from workflows to create generic templates in collaborative work systems |
US11277361B2 (en) | 2020-05-03 | 2022-03-15 | Monday.com Ltd. | Digital processing systems and methods for variable hang-time for social layer messages in collaborative work systems |
US11481288B2 (en) | 2021-01-14 | 2022-10-25 | Monday.com Ltd. | Digital processing systems and methods for historical review of specific document edits in collaborative work systems |
US11687216B2 (en) | 2021-01-14 | 2023-06-27 | Monday.com Ltd. | Digital processing systems and methods for dynamically updating documents with data from linked files in collaborative work systems |
US11475215B2 (en) | 2021-01-14 | 2022-10-18 | Monday.com Ltd. | Digital processing systems and methods for dynamic work document updates using embedded in-line links in collaborative work systems |
US11928315B2 (en) | 2021-01-14 | 2024-03-12 | Monday.com Ltd. | Digital processing systems and methods for tagging extraction engine for generating new documents in collaborative work systems |
US11449668B2 (en) | 2021-01-14 | 2022-09-20 | Monday.com Ltd. | Digital processing systems and methods for embedding a functioning application in a word processing document in collaborative work systems |
US11531452B2 (en) | 2021-01-14 | 2022-12-20 | Monday.com Ltd. | Digital processing systems and methods for group-based document edit tracking in collaborative work systems |
US11726640B2 (en) | 2021-01-14 | 2023-08-15 | Monday.com Ltd. | Digital processing systems and methods for granular permission system for electronic documents in collaborative work systems |
US11893213B2 (en) | 2021-01-14 | 2024-02-06 | Monday.com Ltd. | Digital processing systems and methods for embedded live application in-line in a word processing document in collaborative work systems |
US11397847B1 (en) | 2021-01-14 | 2022-07-26 | Monday.com Ltd. | Digital processing systems and methods for display pane scroll locking during collaborative document editing in collaborative work systems |
US11392556B1 (en) | 2021-01-14 | 2022-07-19 | Monday.com Ltd. | Digital processing systems and methods for draft and time slider for presentations in collaborative work systems |
US11782582B2 (en) | 2021-01-14 | 2023-10-10 | Monday.com Ltd. | Digital processing systems and methods for detectable codes in presentation enabling targeted feedback in collaborative work systems |
US11765043B2 (en) * | 2021-03-05 | 2023-09-19 | Dell Products, L.P. | Data driven chaos engineering based on service mesh and organizational chart |
US20220286362A1 (en) * | 2021-03-05 | 2022-09-08 | Dell Products, L.P. | Data Driven Chaos Engineering Based on Service Mesh and Organizational Chart |
US20230176831A1 (en) * | 2021-12-06 | 2023-06-08 | International Business Machines Corporation | Microservices recommendation framework |
US11853725B2 (en) * | 2021-12-06 | 2023-12-26 | International Business Machines Corporation | Microservices recommendation framework |
US11741071B1 (en) | 2022-12-28 | 2023-08-29 | Monday.com Ltd. | Digital processing systems and methods for navigating and viewing displayed content |
US11886683B1 (en) | 2022-12-30 | 2024-01-30 | Monday.com Ltd | Digital processing systems and methods for presenting board graphics |
US11893381B1 (en) | 2023-02-21 | 2024-02-06 | Monday.com Ltd | Digital processing systems and methods for reducing file bundle sizes |
Also Published As
Publication number | Publication date |
---|---|
WO2014039898A3 (en) | 2014-05-08 |
WO2014039898A2 (en) | 2014-03-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20140074545A1 (en) | Human workflow aware recommendation engine | |
JP6688913B2 (en) | Access control for data resources | |
US10380703B2 (en) | Calculating a trust score | |
US11853343B2 (en) | Method, apparatus, and computer program product for user-specific contextual integration for a searchable enterprise platform | |
US9152969B2 (en) | Recommendation ranking system with distrust | |
US10311106B2 (en) | Social graph visualization and user interface | |
US20170235788A1 (en) | Machine learned query generation on inverted indices | |
US20220067665A1 (en) | Three-party recruiting and matching process involving a candidate, referrer, and hiring entity | |
CN116911650A (en) | Extrapolating trends in trust scores | |
US10387840B2 (en) | Model generator for historical hiring patterns | |
US20150120718A1 (en) | Smart question routing and answerer growing for online community question-answer services | |
WO2016140697A1 (en) | Increasing interaction between clusters with low connectivity in a network | |
AU2014200389B2 (en) | Behavior management and expense insight system | |
US20160217427A1 (en) | Systems, methods, and devices for implementing a referral processing engine | |
US20180025322A1 (en) | Skill-based recommendation of events to users | |
US20230037222A1 (en) | Method, apparatus and computer program product for generating tiered search index fields in a group-based communication platform | |
US20240028979A1 (en) | Systems and Methods for Enhancing and Facilitating Access to Specialized Data | |
US11429877B2 (en) | Unified logging of actions for labeling | |
US20190042950A1 (en) | Learning computing activities and relationships using graphs | |
US20190042951A1 (en) | Analysis of computing activities using graph data structures | |
US11205155B2 (en) | Data selection based on career transition embeddings | |
US20170004456A1 (en) | Search by applicant ranker scores | |
US11386365B2 (en) | Efficient percentile estimation for applicant rankings | |
US20220101159A1 (en) | Recommending network connections by optimizing for two-sided implicit value of an edge | |
US20210319386A1 (en) | Determination of same-group connectivity |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MAGNET SYSTEMS, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MINDER, KEVIN A.;CHAN, ROBYN J.;KIM, HANJU;SIGNING DATES FROM 20130613 TO 20130710;REEL/FRAME:030842/0377 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |