EP1934767A2 - Determining event causality in a wavefront environment - Google Patents
Determining event causality in a wavefront environmentInfo
- Publication number
- EP1934767A2 EP1934767A2 EP06789595A EP06789595A EP1934767A2 EP 1934767 A2 EP1934767 A2 EP 1934767A2 EP 06789595 A EP06789595 A EP 06789595A EP 06789595 A EP06789595 A EP 06789595A EP 1934767 A2 EP1934767 A2 EP 1934767A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- event
- events
- causality
- successor
- predecessor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
Definitions
- Embodiments of the present invention relate to the fields of data processing and data communication. More specifically, embodiments of the present invention are related to methods and apparatuses for determining (or allowing the determining) of event causality in a networking environment, and/or the usage (or allowing the usage) of the determination.
- Figure 1 illustrates a block diagram of a computing environment suitable for use in accordance with various embodiments of the present invention
- Figure 2 illustrates a block diagram of an exemplary computing system in accordance with various embodiments of the present invention
- Figure 3 illustrates a graphical representation of causally related events, in accordance with various embodiments of the present invention
- Figure 4 illustrates a graphical representation of potentially causally related events, in accordance with various embodiments of the present invention
- Figures 5 A-D illustrate a linear causality search progression to determine event causality relationships, in accordance with various embodiments of the present invention
- Figure 6 illustrates a graphical representation of causality showing events along transitive edges, in accordance with various embodiments of the present invention
- Figure 7 illustrates the graphical representation of causality of Figure 3 partitioned into causal chains, in accordance with various embodiments of the present invention
- Figure 8 illustrates the partitioned graphical representation of causality of Figure 7 showing arrays to represent causal tables, in accordance with various embodiments of the present invention
- Figure 9 illustrates the partitioned graphical representation of causality of Figure 7 showing linked lists to represent causal tables, in accordance with various embodiments of the present invention
- Figure 10 illustrates the partitioned graphical representation of causality of Figure 7 showing a packed representation for causal tables, in accordance with various embodiments of the present invention
- Figure 11 illustrates the partitioned graphical representation of causality of Figure 7 showing an event table where each causal chain maintains a master index of all other causal chains, in accordance with various embodiments of the present invention
- Figure 12 illustrates a partitioned graphical representation of causality showing two event types where both event types include binary counters, in accordance with various embodiments of the present invention
- Figure 13 illustrates a partitioned graphical representation of causality of Figure 7 further showing a causal graph partitioned into subspaces or cells, in accordance with various embodiments of the present invention
- Figure 14 illustrates the partitioned graphical representation of causality of Figure 13 showing boundary events at subspace or cell boundaries, in accordance with various embodiments of the present invention
- Figure 15 illustrates multi-phase storage of events within an event subspace, in accordance with various embodiments of the present invention
- Figure 16 illustrates two phase boundary event subspace causality determination, in accordance with various embodiments of the present invention
- Figure 17 illustrates an inserted node, and the effect on wavefront tables in the successor portion of the graph, in accordance with various embodiments of the present invention
- Figure 18 illustrates how the invalidation zone can be represented through a successor wavefront, in accordance with various embodiments of the present invention.
- Figure 19 illustrates how the invalidation wavefront is represented as a single value attached to the chain with insertion, in accordance with various embodiments of the present invention.
- Figures 20 A and B illustrate a filtered directed acyclic graph (DAG) using a non- minimal set of connecting edges and a minimal set, respectively;
- Figure 21 illustrates a filtered DAG with an event to be inserted at the insertion point, the filtered DAG including a DAG minimal set of events having no predecessor events and DAG maximal set of events having no successor events;
- Figure 22 illustrates one progression of the explicit search algorithm for deriving the maximal predecessor set for the inserted event in accordance with one embodiment of the present invention;
- Figure 23 illustrates one progression of the explicit search algorithm for deriving the minimal successor set for the inserted event in accordance with one embodiment of the present invention
- Figure 24 illustrates a filtered DAG with the event inserted
- Figure 25 illustrates the DAG from Figure 18 broken down into filtered chain lists in accordance with one embodiment of the present invention
- Figure 26 illustrates using a filtered chain list procedure to find the maximal predecessor set in accordance with one embodiment of the present invention
- Figure 27 illustrates using the filtered chain list procedure to find the minimal successor set in accordance with one embodiment of the present invention
- Figure 28 illustrates a filtered chain list and DAG with the event inserted.
- Embodiments of the present invention include, but are not limited to, methods related to filtering directed acyclic graphs (DAGs) derived from tracking event causality in wavefiont tables following at least one node insertion as derived from a plurality of events, while maintaining transitive event relationships; in particular, in a network environment, employing filtered chain lists and apparatus or systems equipped to practice the method or aspects thereof.
- DAGs directed acyclic graphs
- a and/or B means "(A), (B), or (A and B)".
- the phrase “at least one of A, B, and C” means “(A), (B), (C), (A and B), (A and C), (B and C) or (A, B and C)”.
- the phrase “(A) B” means "(A B) or (B)", that is "A” is optional.
- the use of any of these phrases does not imply or indicate that the particular feature, structure, or characteristic being described is a necessary component for every embodiment for which such a description is included.
- computing environment 100 includes a private network 102 coupled to a public network 114. More specifically, private network 102 includes a number of application servers 104, user computers 106 and gateways 108 coupled to each other and public network 114 as shown. Additionally, private network 102 includes sensors 110 and
- public network 114 may include the Internet.
- Sensors 110 are employed to monitor network traffic and to detect and report occurrences of a wide range of events, whereas management servers 112 are deployed to manage private network 102 based at least in part on events detected and reported by sensors 110.
- at least one of management servers 112 is equipped with a communication interface and a causality module.
- the communication interface is configured to receive data associated with the occurrence of events of interrelated chains of events.
- the causality module is configured to determine causality and associate the detected/reported events, to maintain a record of its predecessor events with embodiments of the causality logic of the present invention, and to facilitate management of private network 102.
- event as used in this context broadly includes virtually all occurrences and happenings that may be sensed, monitored, and/or reported upon.
- embodiments of the present invention are particularly suitable for managing large networks.
- embodiments of the present invention are also suitable for and may be deployed to manage medium to small networks.
- one or more sensors 110 may be used to detect and to report occurrences of events.
- management server 112 may be employed to manage the network.
- sensors 110 may be combined with management servers 112.
- a management server 112 could be used to manage multiple networks.
- application servers 104 may also be combined with management servers 112.
- user computers may be combined with the application servers and/or management servers 104/112.
- the enumerated elements of Figure 1 otherwise represent a broad range of the corresponding elements known in the art.
- the computing environment 100 may include any number of application servers, sensors, management servers, user computers, gateways, and the like.
- Embodiments of the present invention may use a plurality of network device elements, provided the elements are properly endowed with the resources to handle the resulting number of users and level of usage the elements are to collectively support.
- Network Management Server Figure 2 illustrates a block diagram of computing device 200, which is suitable for use as a management server 112, in accordance with various embodiments.
- computing device 200 includes one or more processors 202, system memory 204, mass storage devices 206, input/output devices 208 and communication interfaces 210.
- Exemplary mass storage devices 206 include diskettes, hard drives, CDROMs, DVDs and the like;
- exemplary input/output devices 208 include keyboards, cursor controls and the like;
- exemplary communication interfaces 210 include network interface cards, modems and the like.
- the elements 202-210 are coupled to each other via system bus 212, which may represent one or more buses. In the case of multiple buses, the buses may be bridged by one or more bus bridges (not shown).
- System memory 204 and mass storage device 206 are employed to maintain and/or to store a working copy and a permanent copy (not shown) of the programming instructions implementing network management software 222 including event causal relationship analysis logic/module(s) 224.
- the permanent copy of the programming instructions may be loaded into mass storage device 206 in the factory or in the field through, e.g., a distribution medium (not shown) or through communication interfaces 210 from a distribution server (not shown).
- network management software 222 is adapted to be able to compute and to track the causal relationship of occurrences of events in private network 102 through analysis of network traffic.
- the causal relationship analysis logic in causality module(s) 224 allows network management software 222 to efficiently perform analysis on all or selected traffic occurring in private network 102 even though it may be constrained in computation power and/or storage space.
- network management software 222 is able to establish causal relationships between noticeably odd behaviors and to detect subtleties that would have been hidden otherwise. What constitutes "odd behavior" and/or “subtleties” is application dependent. As will be apparent from the description to follow, the nature of "odd behavior" and/or “subtleties” of a particular application may be reflected through the configuration of the analysis and/or usage of the analysis.
- one or more sensors 110 may be allocated to track connections between computers 106 and/or application servers 104, their connection types, and the quantity of data transferred. Then, assume that the one or more of sensors 110 are able to detect a connection from a first computer 106 to a finance file application server 104 transferring a large quantity of data; and, some time later, another connection between the first computer 106 and second computer 106 performs another transfer of a large quantity of data. Finally, a connection is detected between the second computer 106 and an Internet based disgruntled employee website, performing a similarly large data transfer. From these reported detections, management server 112 may infer that one or more employees may have transferred some amount of financial data to a disgruntled employee web site. While management server 112 may not have immediate insight into the actual data transferred, the events justify issuing an alert for a deeper investigation.
- the immediate causal relationship which may be of value to subsequent computations, may be selectively determined using different levels of granularity.
- the causal relationship may be selectively determined depending on the information available to a particular management server 112, e.g. , with respect to a particular computer and/or communications on the network. For example, in one embodiment, if a management server 112 has no information on how the processes executing on a particular server modify the file system or interact through shared memory, the management server 112 may assume that all events that occur on that server are causally related.
- a management server 112 may use causal chains and associate the causal chains with one or more recognizable entities.
- recognizable entities include a work station, a process, a server, and the like.
- several entities may be excluded from being considered, because some events they produce may be considered locally causally independent. Some examples of these excluded entities include firewalls, routers, switches, hubs, and the like.
- a management server 112 does not automatically consider two events on a firewall to be necessarily causally independent, rather management server 112 further determines the causality by observing the effects both observed events have on the rest of the network.
- each management server 112 used to compute the causal relationship is endowed with a relatively very large but relatively slow hard-drive, and a relatively small but relatively very fast bank of physical memory.
- the arrangement facilitates a two phase approach to the causal relationship analysis wherein one phase of the analysis exploits the large size of one type of storage and the other phase of the analysis exploits the speed of the other type of storage.
- a management server 112 may use the memory for first phase storage due to its limited size, and use the hard-drive for second phase storage because of its limited speed.
- the causal relationship analysis may employ the concept of event subspaces or cells, and involve looking up events in a small number of subspaces or cells. Thus, for speed, events of the subspaces or cells may be maintained and/or stored in physical memory. In one embodiment, the causal relationship analysis may involve a lookup event in a large number of infrequently referenced subspaces or cells. Thus, the data will be archived for a long period of time and events in these subspaces or cells are stored in the storage space of the hard-drive. However, in one embodiment, a caching mechanism is employed whereby events associated with subspaces or cells which are not frequently needed are flushed out of physical memory and events associated with subspaces or cells that are required for new computations are imported from the hard-drive.
- the embodiment designates the disk as first phase storage and doubly optimizes events associated with subspaces or cells stored there for space and designates physical memory as second phase storage and doubly optimize events associated with subspaces or cells stored there for speed.
- a causal relationship is a relationship between two events, a and b, which states that a meaningfully preceded b.
- the causal relationship is transitive and anti-reflexive.
- a ⁇ > b and b ⁇ > c implies that ⁇ ⁇ > c and that b ⁇ /> a.
- the causal relationship is also the transitive closure of the immediately causal relationship ( ⁇ ) between two events, a and b, which states that not only did a precede b, but there were no intermediate events between a and b.
- maintenance of the immediate causal relationship is effectuated by each event having pointers back to immediate causal predecessors.
- a summary of the transitive closure of this relationship is maintained. In general, this can be both time consuming (O(n 3 ), where O is the number of operations to be performed and n is the number of events tracked), and space consuming (O(n 2 ), in addition to the space required for storing events).
- the alternative to maintaining a transitive summary is to search through the immediate causal relationships to find whether one event transitively leads to another (O(n) per query, with n 2 possible queries).
- a causality module constitute operational programs performed by computing devices or computer-controlled network devices, such as management server 112. Describing the operational methods of the causality module by reference to a graphical representation enables one skilled in the art to develop such operational programs including such instructions to carry out the methods on suitably configured network devices (causality management server, user computers, servers, gateways, sensors, and the like). The operations may be performed in a computer controlled device or may be embodied in a network device.
- the causality problem of determining the causal relationship of the events and event associations may be represented by a causality graph representing the immediate causal relationship of all events.
- the causality problem is to determine whether two events are causally related. In other words, for two events, determine whether there is a path between them in the immediate causality graph.
- Figure 3 shows an example with two events, a and e, that are causally related through intermediate events b, c, and d. Accordingly, there is a path between events a and e in this graph of immediate causal relationships.
- determining whether two arbitrary events, a and e, are causally related requires clarifying the cloud of events that exist between events a and e.
- exemplary approaches include a linear search, an explicit transitive closure, variations on table-based approaches, and mixed counter methods.
- a linear search approach investigates a potentially large number of events to determine whether two arbitrarily chosen events are causally related.
- the explicit transitive closure methods could require a large amount of space to store the relationship information for a relatively small number of events.
- Variations of table methods produce rapid results with significantly less storage than the explicit transitive closure methods.
- Mixed counter methods produce results that take slightly more time than the various table methods, but also require significantly less space.
- the linear search approach begins with one of the two events, and searches outward on all immediate causal links, seeking the other event. In the worst case, it may be necessary to investigate all known events to determine that there is no causal link between the two.
- Figure 5 demonstrates the linear search approach to determine a ⁇ > b.
- the term "outward" is employed for ease of understanding.
- the directional characterization is not to be limiting on the present invention.
- a partial causality test, where possible, can restrict the direction of search. In our example, we may be able to rule out e ⁇ > ⁇ , so we need only search events that succeed a and precede e.
- the explicit transitive closure approach adds annotations to the graphical representation of causality to help reduce the time it takes to determine whether there is a path between a particular pair of events.
- the annotations are the edges that specifically represent the full causal relationship or the transitive closure of the immediate causal relationship.
- Figure 6 shows part of an example graph with the transitive edges as dashed lines, a ⁇ > e is represented as a single transitive edge.
- a causality annotation method that allows us to find a ⁇ > b in substantially constant time is employed. This methodology may also require significantly less space than storing one based on the explicit transitive closure method. Some practical issues with respect to data structure are discussed in greater detail at the end of the section.
- the first operation of a table-based approach is to partition the immediate causal graph into causal chains, where a causal chain is an incomplete sequence of events such that each event in the chain is causally related to all other events in the chain.
- Figure 7 shows the graphical representation of causality from Figure 3 partitioned into causal chains.
- This partition is not unique — there may be other, and possibly better, ways to partition this causality graph; however, in various embodiments, this is a full partition, which means that none of the causal chains can be merged.
- this is a full partition, which means that none of the causal chains can be merged.
- Figure 7 shows, many edges are not part of any causal chain. In fact, each chain can be as small as selected - down to a single event. But as explained below, it is beneficial to keep the largest possible causal chains.
- the next operation is to annotate the graphical representation of causality with the position of each event relative to the causal chains.
- event a is the first event in chain cl ⁇ event b is the first in chain c3, but it follows the first in chain c2; event c is the second in chain c3, but it follows the first in chain c2; etc.
- These annotations can take the form of tables, where each chain with a causal predecessor has an entry, and each entry contains the position of the latest predecessor in that chain, plus 1. If an event is the first in its chain, the table entry for its own chain contains the value 1. In the example, the table for a contains c2:l; the table for b contains c2:2 and c3:l; the table for event c contains c2:2 and c3:2; etc.
- each causal table contains data representative of a predecessor wavefront, where all preceding events for a particular event are either on or behind its wavefront.
- each causal table identifies the events that are on the predecessor wavefront (hereinafter, may also simply be referred to as wavefront), and as such, these causal tables may also be known as predecessor wavefront tables (or simply, wavefront tables).
- predecessor wavefront tables or simply, wavefront tables.
- the complexity of this algorithm with n causal chains and a maximum of p predecessors per event is O(np).
- the methodology for evaluating the causal relationship between two events, a, b, given the predecessor wavefront (PW) tables for both is as follows:
- the PW tables are implemented with arrays whose size are equal to the total number of causal chains recognized by the system, as illustrated in Figure 8. Each causal chain is then assigned a number and that number is the index to the value corresponding to that chain. This may tend to produce a very large representation if there are a large number of causal chains and much of the array may be either unused or contain redundant information.
- each causal chain may be assigned an index, as illustrated in Figure 9.
- the array is allocated as necessary one element at a time.
- Each element containing useful data also contains a pointer to the next element that contains useful data. Since the index is no longer implicit based on an offset into the array, in various embodiments, the index of each element is stored with the element as well. To find the entry for a particular index, in various embodiments, a linear search of all elements is performed, comparing the index to our desired index until the desired element is encountered. In a worst case scenario, this technique could require up to three times the memory of the native approach.
- the maximum size of a PW table is known and the size is considered to be satisfactorily below the total number of causal chains, this may justify use of this sparse method where all of the memory required for each PW table may be allocated at once. In this case, since all elements are packed within a single chunk of memory with no spaces between them, it is not necessary for any element to include a pointer to the next element.
- Finding a particular element is implemented with a binary search to first read the index of the middle element. Second, if the middle element index is equal to the index sought, that element is returned as a result of the search. Third, if the middle element index is greater than the index sought, perform the first operation on the lower half of the block. Fourth, if the middle element index is less than the index sought, perform the first operation on the upper half of the block.
- PW tables may be stored as hash tables. Depending on the quality of the hash function and hash table loading, this can provide effectively the substantial constant time lookup of elements as the case with explicit arrays, and much of the storage efficiency of the linked list or chunk allocated sparse arrays. However, in order to ensure fast lookups, the hash table often needs to be much larger than the amount of data stored.
- the offset of a particular key may differ from one PW table to another, even within the same causal chain. This is why it is necessary to perform a binary search for each lookup.
- the reason the position of a key may differ from one PW table to another is because each event may require a PW table with one or more cells than the previous event in the chain and there is implicit requirement that all elements must occur in array order.
- the chain indexed method is employed to relax this requirement. As shown in Figure 11, the size of each PW table is sized to the number of entries required, but entries are ordered based on when each of the other causal chains becomes known to the chain containing the PW tables. Thus, for each causal chain, each index appears in the same place in every array that contains that index.
- a mixed counter method is employed.
- the mixed counter method keeps two or more types of counters instead of uniformly assigning a PW table to each event. This approach works well when there are a large number of events within a causal chain whose immediate causal relations are in the chain as well.
- a binary mixed counter method is employed. For these embodiments, it is assumed there are at least two types of events; a type e event has no immediate predecessors outside of its own causal chain; and a type/event has at least one immediate predecessor outside of its own causal chain.
- the events with dashed outlines are of type e
- the solid outline events are of type/ Type e events are then assigned scalar counters that maintain a count back to the most recent type/event, and only type/events require full PW tables. Determining causality between two events; however, becomes a little more complicated. Specifically, the causality of events, a and b, then becomes a determination of whether a and b are type e or/ For type/ events, look up the values for ⁇ 's and b's causal chain in the respective PW table.
- For type e events look up the values for ⁇ 's and ⁇ 's causal chain in the most recent type/event and add the scalar counter to the value from its own chain. Compare the resultant values as above. If event ⁇ 's value for ⁇ 's causal chain is larger than ⁇ 's own value, than b ⁇ > a. Likewise, if event b's value for ⁇ 's causal chain is larger than ⁇ 's own value, than a ⁇ > b. Otherwise, there is no relationship.
- the ternary mixed counter method is employed.
- the types of causal chain are Type 1 and Type 2.
- a Type 1 causal chain has direct intersections with a vast number of other causal chains.
- a Type 2 causal chain has direct intersections with only a few other causal chains, and most of those are type 1 causal chains.
- causality chain types are based in part on factors that help reduce the storage space necessary for each application. These factors provide some flexibility in assigning causal chains to one type or the other.
- Type e is an event on either type of chain that has no immediate predecessors outside of its own chain.
- Type / is an event on a type 1 chain that has predecessors outside of its own causal chain.
- Type i is an event on a type 2 chain that has predecessors outside of its own causal chain.
- type e events are assigned scalar counters which count back to the most recent type/or type i event.
- Type/events get full PW tables, but type i events can either have full PW tables or pointers to a type e or type/event on a type 1 chain.
- the event space is partitioned into subspaces so that the simple formulation may be applied to each subspace.
- the scope of an event space is application dependent.
- An example is a networked computing environment in support of one or more mission critical applications for an organization/entity.
- this configuration allows both the number of causal chains and the number of events per causal chain to be controlled.
- the partition is causally consistent, completion of causality comparisons between events in the same subspace may stay relatively constant.
- Causality comparisons made between events from different subspaces require additional computation time based in part on the number of subspaces and the number of boundary events.
- Figure 13 shows an event space partitioned into four subspaces or cells.
- a subspace or cell is effectively a container for a number of events. Connections between events in different subspaces or cells are facilitated by adding boundary events.
- Figure 14 shows the partitioned causality graph from Figure 13 with appropriate boundary events. In order to compute the causal relationship between events in two different subspaces, the causal relationship between each event and the boundary event or events for its subspace are first computed. Thereafter, the causal relationship between the boundary events on the different subspaces are computed.
- the system would first determine that a ⁇ > ab ⁇ , a ⁇ > ac ⁇ , cd ⁇ ⁇ > e, and Cd 2 ⁇ > e. Next, the system would determine that ac ⁇ ⁇ > cd ⁇ , and ac ⁇ ⁇ > cJ 2 .
- the divided event space technique works better if the subspace partitions are substantially consistent, i.e., if one arrow points from subspace a to subspace b, then no arrows will point from subspace b to subspace a and, more restrictive, if one can trace a path from subspace a to subspace b by traveling arrows from tail to head, then there is no similar path between subspace b and subspace a.
- the subspace partition is not substantially consistent, it becomes necessary to evaluate boundary events when determining the causality between events in the same subspace.
- the boundary events must each maintain a separate PW table for each subspace they belong to. These boundary events may also maintain a higher order PW table to establish the relationships among boundary events directly.
- a hierarchical divided event space method is employed.
- subspaces can contain either events or other subspaces. This gives us a way to quickly determine between boundary events.
- the approach provides a method for quickly (O(lg ⁇ )) solving the causality problem without PW tables.
- each event comprises a subspace, and these subspaces are grouped into higher subspaces, etc. until all events are contained within a single very high order subspace. Determining the relationships between events relies on first finding a hierarchical level where each event is in a separate subspace, but the subspaces are neighbors. On each side of the split, perform the same algorithm between the query event and the boundary events. The process is repeated until there is an answer.
- a multi-phase approach is employed.
- the approach requires different amounts of storage depending on storage phases.
- Storage phases account for different types of storage media and optimize for space, access time, and other factors.
- Subspaces can move from one phase to another over time.
- Multi-phase storage can demonstrate a great deal of benefit when applied heterogeneously to a network of storage subspaces; that is, different cells in the system are in different storage phases.
- Figure 15 illustrates an extreme case of multi-phase storage. In one phase, the subspace is represented without any causal markings; and, in the second phase, the entire subspace is fully marked. In this extreme case, there may be two storage phases.
- Phase one has no causal optimization tags at all, and phase two has all events fully tagged (with PW table, or full transitive closure). This extreme approach does not work well in a heterogeneous environment because moving a subspace from phase one to phase two may require moving a number of other subspaces as well. In order to compute the relations for the boundary subspaces, the other side of the boundary needs to be computed.
- a mix approach has phase one identify boundary events, and phase two reconstruct the subspace with a binary or ternary count.
- phase two reconstruct the subspace with a binary or ternary count.
- the causal tags may be rebuilt based on purely local information and the storage space required for the second phase is also reduced.
- events not having a boundary event as a predecessor event may include a counter to the next MIy tagged event.
- all wavefront tables of existing events are recomputed whenever an event with existing successors is inserted into the graph. While this approach is adequate, it may require excessive computation for each insertion. Alternatively, one embodiment only re-computes those tables whose events succeed the inserted node. Unfortunately, the data structures and algorithms described do not contain enough information to make this determination a priori. Fortunately, it is possible, given a small amount of additional information per causality chain, to determine whether a particular table is still valid, or whether the table has been invalidated by predecessor insertion. Given this, it is then possible to re-compute an event's wavefront table only when it is needed, and only if it has been invalidated by one or more predecessor insertions.
- Figure 17 shows the effect of an insertion, illustrating the successor nodes whose tables are invalidated by the insertion in accordance with at least one embodiment of the present invention.
- each event with successors can have its entire set of successors represented by a successor wavefront.
- an event is an element of the successor wavefront if either 1) it is an immediate successor of the inserted event or 2) it is an indirect successor of the inserted event but it has at least one immediate predecessor that is not a successor of the inserted event. All events, x, that succeed a particular event, e, must either be on the successor wavefront or in front of the successor wavefront.
- Figure 18 illustrates the nodes that make up the successor wavefront of the graph in Figure 17.
- the successor wavefront could be tracked by tables (as is the predecessor wavefront), but successor wavefronts often change more rapidly than predecessor wavefronts, and, as such, would incur more frequent invalidation.
- the causality module determines whether that event is on the edge or in front of the successor wavefront of any inserted events. This can be determined by adding a value to each causality chain that represents the count of the highest valid table entry for that chain. To determine whether a particular wavefront table is valid, its entries are compared with the count value for each chain.
- Figure 19 illustrates the valid number derived from the insertion point of the inserted node in accordance with one embodiment of the present invention.
- the table should be checked for validity. Since the table is only checked when data in the table is accessed, a table may be invalidated many times but is only rebuilt once. Although this process does add a computation cost to checking tables, the embodiment can also leverage the cellular nature of some of the storage techniques described above. In many cases, all tables in a particular cell are valid and, if so, the detailed check described above may be unnecessary.
- tone embodiment associates a Boolean state with each cell that is set to "true” when all tables are rebuilt and “false” when any changes are made to a cell that would either invalidate one or more tables or that add events that have no tables yet.
- the filtered graph will be constructed in preparation for display to a user.
- the inclusion edges for all relationships would include a number of edges that could easily be inferred by the user and which could significantly add to the visual clutter. For this reason, one embodiment actually only includes a minimal set of edges (i.e., the smallest set of edges that characterizes all relationships in the filtered graph) in the filtered graph. As such for each event, the focus is placed on finding the set of "relatively maximal" predecessors and the set of "relatively minimal” successors around which explicit edges are to be added.
- Figure 20 shows graphs containing edges for causal relationships between events on the graph.
- Figure 20 a) depicts a graph that contains edges (X, Y, and Z) for all causal relationships present in the graph and
- Figure 20 b) shows the same graph with only maximal predecessors and minimal successors.
- edge Z is removed because it represents a relationship between event c's non-maximal predecessor a, and c (or in other words, a relationship between event a and its non-minimal successor, c).
- One embodiment employs a "Forward/Reverse Search Algorithm" to compute the maximal predecessors and minimal successors.
- this search algorithm maintains a set of relatively minimal events and a set of relatively maximal events, where the relatively minimal set includes all events that have no predecessors in the filtered graph and the relatively maximal set includes all events that have no successors in the filtered graph.
- To add a node representing an event to a filtered graph one embodiment creates two search sets: one search set is initialized with the members of the overall graph minimal set that are also predecessors of the inserted node (called the forward search set) and the other search set is initialized with the members of the overall graph maximal set that are also successors of the inserted node (called the reverse search set).
- the search moves the forward search set forward by replacing the immediate forward search set with the set of successors of the previous forward search set that are also predecessors of the inserted event.
- the events in the forward search set that have no such successors are added to a maximal predecessors set for the inserted node/event.
- the search replaces the reverse search set with the set of predecessors of the previous reverse search set that are also successors of the inserted node/event.
- the events in the reverse search set that have no such predecessors are added to a minimal successors set for the inserted node/event until the reverse search set is empty.
- edges are added between the inserted node/event and the nodes/events in the minimal successors set.
- the set of maximal predecessors is empty, the inserted node/event is added to the overall graph minimal set.
- the set of minimal successors is empty, the inserted node/event is added to the overall graph maximal set. If appropriate, an inserted node/event may be added to both the overall graph minimal set and the overall graph maximal set.
- Figures 21 through 24 demonstrate the insertion of an event using the previously described procedure.
- Figure 21 shows an initial directed acyclic graph with the projected insertion point highlighted and Figures 22 and 23 illustrate the forward and reverse searches from the graph minimal and maximal sets to find the insertion point.
- Figure 22 a) shows the forward search set, initialized with events from the graph minimal set that precede the insertion point
- Figure 22 b) shows the set containing the step 0 set successors that are also predecessors of the insertion point
- Figure 22 c) shows the set containing the step 1 set successors that are also predecessors of the insertion point. Since the step 2 set contains no successors that are also predecessors of the insertion point, this set also represents the maximal predecessor set.
- filtered causal chains allows insertions of events into a filtered causal graph that only require a logarithmic number of comparisons per insertion.
- filtered events are first arranged into filtered chain lists, such that each causal chain in the original graph has, at most, one filtered chain list and each filtered chain list contains all filtered events which are in the corresponding chain in the non-filtered graph. These filtered chain lists preserve the order from the original chain.
- the process for inserting an event into a filtered graph starts with obtaining a chain relative maximal predecessor set (which contains the highest order predecessors for the inserted event in each causal chain for which any predecessors exist) and a chain relative minimal successor set for each inserted event (which contains the lowest order successors for the inserted event in each causal chain for which any successors exist).
- the maximal predecessor set may be generated by performing a binary search on each filtered chain list for the predecessor of the inserted event using the wavefront values that correspond to the chain of the given filtered chain list.
- the minimal successor set may be generated by performing a binary search on each filtered chain list for the successor of the inserted event using the wavefront values that correspond to the inserted event's own chain.
- the process proceeds to reduce the chain relative maximal predecessor set to the inserted event's maximal predecessor set (note: the latter is a subset of the former), and to reduce the chain relative minimal successor set to the inserted event's minimal successor set (again, the latter is a subset of the former).
- the first part can be performed by checking the filtered graph successors of each event in the chain relative maximal predecessor set to determine whether any of the successors of each event precedes the inserted event. If not, then the starting event being checked is part of the maximal predecessors set. Additionally, if a successor of the starting event being checked precedes the inserted event, the starting event is not part of the maximal predecessor set.
- the second part can be performed by checking the filtered graph predecessors of each event in the chain relative minimal successor set to determine whether any of the predecessors succeed the inserted event. If not, then the starting event being checked is part of the minimal successors set.
- This algorithm is illustrated in Figures 25 through 27.
- Figure 25 shows the filtered chain list breakdown of the graph from Figure 21.
- Figure 26 a) shows the first step of finding the relative maximal predecessor set, which is performing a binary search on each filtered chain list to find the chain relative maximal predecessor set.
- Figure 26 b) shows the next step of reducing the chain relative maximal predecessor set to the maximal predecessor set by removing those events that have successors that precede the insertion point, hi this case, event /is a successor of both d and e and, since /precedes the insertion point, both d and e must be eliminated. On the other hand, /has no successors that precede the insertion point so it remains in the maximal predecessor set. The process of finding the minimal successors is similar.
- Figure 27 a) shows the result of finding the chain relative minimal successor set through binary search (g andy).
- Figure 27 b) shows the result of the second step, reducing chain relative minimal successor set to the minimal successor set by eliminating all events that have predecessors that succeed the insertion point. Since g precedes 7,7 must be eliminated.
- Figure 28 adds the new event to the filtered graph. As shown in Figure 28, to do so, edges are added from each event in the maximal predecessor set of Figure 26 b) to the inserted node and edges from the inserted node to each event in the minimal successor set of Figure 27 b).
- the complexity of building a filtered graph in accordance with this embodiment is approximately O(n ⁇ gn), which can be determined as follows: each filtered chain binary search takes O(lgn) time, and there are at most 2m of these where m is the number of chains.
Abstract
Description
Claims
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US70597905P | 2005-08-05 | 2005-08-05 | |
US70597805P | 2005-08-05 | 2005-08-05 | |
US11/499,619 US7487241B2 (en) | 2005-08-05 | 2006-08-03 | Performing efficient insertions in wavefront table based causal graphs |
US11/499,306 US20070032986A1 (en) | 2005-08-05 | 2006-08-03 | Efficient filtered causal graph edge detection in a causal wavefront environment |
PCT/US2006/030932 WO2007019524A2 (en) | 2005-08-05 | 2006-08-04 | Determining event causality in a wavefront environment |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1934767A2 true EP1934767A2 (en) | 2008-06-25 |
EP1934767A4 EP1934767A4 (en) | 2010-04-28 |
Family
ID=37728007
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP06789595A Withdrawn EP1934767A4 (en) | 2005-08-05 | 2006-08-04 | Determining event causality in a wavefront environment |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP1934767A4 (en) |
CA (1) | CA2620369A1 (en) |
WO (1) | WO2007019524A2 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5493729A (en) * | 1990-03-14 | 1996-02-20 | Hitachi, Ltd. | Knowledge data base processing system and expert system |
US6249755B1 (en) * | 1994-05-25 | 2001-06-19 | System Management Arts, Inc. | Apparatus and method for event correlation and problem reporting |
GB2362536A (en) * | 2000-05-17 | 2001-11-21 | 3Com Corp | Network management apparatus and method for identifying events on a network |
US20020194393A1 (en) * | 1997-09-24 | 2002-12-19 | Curtis Hrischuk | Method of determining causal connections between events recorded during process execution |
-
2006
- 2006-08-04 EP EP06789595A patent/EP1934767A4/en not_active Withdrawn
- 2006-08-04 CA CA002620369A patent/CA2620369A1/en not_active Abandoned
- 2006-08-04 WO PCT/US2006/030932 patent/WO2007019524A2/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5493729A (en) * | 1990-03-14 | 1996-02-20 | Hitachi, Ltd. | Knowledge data base processing system and expert system |
US6249755B1 (en) * | 1994-05-25 | 2001-06-19 | System Management Arts, Inc. | Apparatus and method for event correlation and problem reporting |
US20020194393A1 (en) * | 1997-09-24 | 2002-12-19 | Curtis Hrischuk | Method of determining causal connections between events recorded during process execution |
GB2362536A (en) * | 2000-05-17 | 2001-11-21 | 3Com Corp | Network management apparatus and method for identifying events on a network |
Non-Patent Citations (1)
Title |
---|
See also references of WO2007019524A2 * |
Also Published As
Publication number | Publication date |
---|---|
WO2007019524A2 (en) | 2007-02-15 |
CA2620369A1 (en) | 2007-02-15 |
WO2007019524A3 (en) | 2009-06-18 |
EP1934767A4 (en) | 2010-04-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8825581B2 (en) | Simplifying a graph of correlation rules while preserving semantic coverage | |
Balbo | Introduction to stochastic Petri nets | |
Hartig | SPARQL for a Web of Linked Data: Semantics and computability | |
US7363203B2 (en) | Determining event causality including employment of partitioned event space | |
WO2012138319A1 (en) | Creating a correlation rule defining a relationship between event types | |
Sengupta et al. | Arrow: Approximating reachability using random walks over web-scale graphs | |
US20050288915A1 (en) | Determining event causality including employment of causal chains | |
US20070032986A1 (en) | Efficient filtered causal graph edge detection in a causal wavefront environment | |
Kuijpers et al. | An experimental study of context-free path query evaluation methods | |
US7487241B2 (en) | Performing efficient insertions in wavefront table based causal graphs | |
Bai et al. | Efficient temporal core maintenance of massive graphs | |
Skandar et al. | An efficient duplication record detection algorithm for data cleansing | |
EP2710493A1 (en) | System and method for configuration policy extraction | |
CN111694860A (en) | Safety detection time sequence data real-time abnormity discovery method and electronic device | |
EP1934767A2 (en) | Determining event causality in a wavefront environment | |
WO2007001312A2 (en) | Determining event causality | |
Czumaj et al. | Contention resolution in hashing based shared memory simulations | |
Mullangi et al. | SCISSOR: scalable and efficient reachability query processing in time-evolving hierarchies | |
Ma et al. | Graph pattern matching for dynamic team formation | |
EP1897004A2 (en) | Determining event causality | |
Kellou-Menouer et al. | On-line versioned schema inference for large semantic web data sources | |
CN112307272A (en) | Method and device for determining relation information between objects, computing equipment and storage medium | |
JP2008547313A (en) | Determine event causality | |
Zhang et al. | Dolha-an efficient and exact data structure for streaming graphs | |
Dobra et al. | Multi-query optimization for sketch-based estimation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20080227 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR MK RS |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: VANTOS, INC.. |
|
R17D | Deferred search report published (corrected) |
Effective date: 20090618 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 13/00 20060101AFI20090911BHEP |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20100330 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04L 12/24 20060101AFI20100324BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20100302 |