WO2003096129A1 - An automated performance monitoring and adaptation system - Google Patents
An automated performance monitoring and adaptation system Download PDFInfo
- Publication number
- WO2003096129A1 WO2003096129A1 PCT/AU2003/000577 AU0300577W WO03096129A1 WO 2003096129 A1 WO2003096129 A1 WO 2003096129A1 AU 0300577 W AU0300577 W AU 0300577W WO 03096129 A1 WO03096129 A1 WO 03096129A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- rate
- actions
- alerts
- threshold
- alert
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/04—Arrangements for maintaining operational condition
Definitions
- the present invention relates to an automatic performance monitoring and adaptation system for adapting an event detection system to improve system performance.
- Fraud is a serious problem in modern telecommunications systems, and can result in revenue loss by the telecommunications service provider, reduced operational efficiency, and increased subscriber churn.
- any provider that can reduce revenue loss resulting from fraud - either by its prevention or early detection - has a significant advantage over its competitors. . ' ,' ,
- the growth of the Internet has led to a gradual increase in the number of long calls made by domestic subscribers to telecommunications services.
- These changes cause the performance of automated fraud detection systems to degrade with time, with increasingly large number of false alarms being generated, and increasingly large numbers of frauds being missed. This degradation is frequently ignored, or, according to present best practice, avoided by regular modifications to the fraud detection engine's configuration.
- Such reconfiguration is time consuming and expensive, however, and increases the risk of introducing errors.
- Most fraud detection systems consist of at least two subsystems - a fraud detection engine (FDE), which analyses incoming data for evidence of fraudulent behaviour (in response to which it generates alerts), and an alert investigation team (AIT), which investigates the causes of the alerts to determine whether they were caused by an actual fraud.
- FDE fraud detection engine
- AIT alert investigation team
- the data that the fraud detection engine monitors would typically be a call data record (CDR) stream within which descriptions of the characteristics of calls made on a telecommunications network appear shortly after their termination.
- CDR call data record
- the fields contained in the call data record are (from top to bottom) A-number (the number of the phone from which the call was made), B-number (the number to which the call was made), B-number type (whether the call was local, national, international, etc. encoded as a number), the call's cost, its duration, and the date and time at which it started. Note that the four rightmost digits of the A- and B-numbers have been masked with 'X's to conceal the identities of the calling and called parties.
- the stream may also contain additional information, such as customer data (which can provide a customer's address, payment history, etc.).
- the fraud detection engine usually contains many components, including change detection algorithms (which search for the changes in behaviour that occur during periods of fraudulent activity), rules (which look for known characteristics of fraudulent behaviour), and data-driven classifiers such as neural networks (which can be trained using examples of real frauds to provide an indication of the likelihood that a fraud is in progress).
- change detection algorithms which search for the changes in behaviour that occur during periods of fraudulent activity
- rules which look for known characteristics of fraudulent behaviour
- data-driven classifiers such as neural networks (which can be trained using examples of real frauds to provide an indication of the likelihood that a fraud is in progress).
- a performance monitoring and adaptation system comprising at least: a performance assessor configured to monitor the rate at which alerts are generated by an event detection system and to perform a first set of actions if the rate crosses a threshold.
- an event detection system comprising at least: an event detection engine that generates an alert if the event is suspected; and a performance assessor configured to monitor the rate at which alerts are generated by the event detection engine and to perform a first set of actions if the rate crosses a threshold.
- the threshold is an end of a configurable range, wherein the first set of actions is triggered if the rate falls outside of the range.
- a configurable number of thresholds may be provided, each of which trigger a respective set of actions if the rate of alerts crosses the respective threshold.
- the set of actions includes one or more actions.
- the action of the first set of actions performed is determined by the direction in which the rate of alerts crosses the threshold.
- the system further comprises a second performance assessor configured to monitor the rate at which false alerts are generated by the event detection system to perform a second set of actions if the rate of false alerts crosses a second threshold. False alerts are false positives, false negatives or both.
- the second threshold is an end of a second configurable range, wherein the second set of actions is triggered if the rate of false alerts falls outside the second configurable range.
- a configurable number of thresholds may be provided, each of which trigger a respective set of actions if the rate of false alerts crosses the respective threshold.
- the action of the second set of actions performed is determined by the direction in which the rate of false alerts crosses the second threshold.
- the first set of actions includes a first alert flood action conducted when the rate of alerts crosses above a configurable first upper trigger rate.
- the first set of actions includes a first alert drought action which occurs when the rate of alerts crosses below a first configurable lower trigger rate.
- a lower reset threshold is built into the first lower trigger rate, such that the rate of alerts must rise above the first lower trigger rate added to a first lower threshold amount before the lower trigger will re-activate the first alert drought action after a previous activation.
- an upper reset threshold is built into the first upper trigger rate, such that the rate of alerts must fall below the first upper trigger rate less a first upper reset threshold amount before the upper trigger will re-activate the first alert flood action after a previous activation.
- the second set of actions includes a second alert flood action which is triggered when a function of the false alert rate rises above a configurable second upper trigger rate.
- the second set of actions includes a second alert drought action which is triggered when a function of the rate of false alerts are under a second configurable lower trigger rate.
- the function is a moving average function.
- a lower reset threshold is built into the range of rate of false alerts, such that the moving average of the rate of false alerts must rise above the second lower trigger rate added to a second lower reset threshold amount before the lower trigger will re-activate the second drought alert action.
- an upper reset threshold is built into the range or rates of false alerts, such that the moving average of the rate of false alerts must fall below the second upper trigger rate less a second upper reset threshold amount before the second upper trigger will re-activate the second alert flood action.
- the actions modify the event detection engine.
- the actions modify a respective parameter of the event detection engine.
- the event detection engine is comprised of a plurality of components, wherein each component uses a different method to detect possible occurrences of the specified event.
- the performance assessor maintains a configurable number of configurable alert thresholds for each component.
- the actions are conducted by execution of a respective script.
- each script can send signals to the event detection engine to modify the configuration of the event detection engine so as to produce a change in the rate of generation of alerts or false alerts.
- each action includes sending a message to a configuration/administration team.
- a positive transition script is associated with the first upper trigger rate and a negative transaction script is associated with the lower trigger rate.
- the positive transition script disables the associated event detection engine component and sends a message to the configuration/administration team.
- the negative transition script sends a message to the configuration/administration team.
- the second performance assessor obtains false alert information from an alert investigation team that investigates whether each alert is real or false.
- the false alert information includes or is used to derive false art rates.
- the moving average is calculated by taking the average of the false negative or false positive rates over a configurable number of configurable periods.
- the second performance assessor identifies components within the event detection engine that are generating too many false alerts in response to normal activity or generating too few alerts in response to actual instances of the event.
- the event detection engine detects events by inference.
- the event detective engine is a fraud detection engine.
- a performance monitoring and adaptation system for an event detection system comprising at least: a performance assessor configured to monitor a function of the rate at which false alerts are generated by an event detection system and to perform a second set of actions if the function of the rate crosses a threshold.
- an event detection system comprising at least: an event detection engine that generates an alert if the event is suspected; and a performance assessor configured to monitor a function of the rate at which false alerts are generated by the specified event detection engine and to perform a second set of actions if the function of the rate crosses a threshold.
- a method of detecting an event from data comprising the steps of: providing an event detection engine for analysing data for an indication of the event; generating an alert if the event is suspected; monitoring the rate at which alerts are generated by the event detection engine; determining whether the rates crosses a threshold; and if the rates crosses the threshold performing a first set of actions.
- a method of detecting an event from data comprising the steps of: providing an event detection engine for analysing data for an indication of the event; generating an alert if the event is suspected; investigating whether the alert is real or false; monitoring the rate at which false alerts are generated by the event detection engine; determining whether the rate of false alerts crosses a threshold; and if the rate of false alerts crosses the threshold performing a second set of actions.
- Figure 1 is a schematic representation of an indirect event detection system having an automatic performance monitoring and adaptation system according to the present invention.
- Figure 2 is an example showing hysteresis based threshold triggering based on rates of alert generated by the system of Figure 1.
- an automatic performance monitoring and adaptation system incorporated into an event detection system 10 which includes an event detection engine 11, an alert investigation team 12, a configuration and administration team 13, an unsupervised performance assessor 14 and a supervised performance assessor 15.
- the event detection engine 1 1 is a fraud detection engine used, for example, to indirectly detect fraud, (such as by inference), in a telecommunication network. It provides fraud alert messages to the alert investigation team 12. The alerts are also provided to the unsupervised performance assessor 14 to determine over time the rate of generation of alerts.
- the unsupervised performance assessor 14 provides feedback to the fraud detection engine 11 based on the rates of alerts; and provides feedback messages to the configuration and administration team 13, alerting the team 13 of the feedback provided to the engine 11.
- the alert investigation team 12 investigates fraud alerts and provides feedback based on the outcome of that investigation to the fraud detection engine 11 and the supervised performance assessor 15.
- the supervised performance assessor 15 uses the investigation outcome feedback to determine rates of generation of false alerts. Based on the assessment of the rates of generation of false alert further feedback is provided by the supervised performance assessor 15 to the fraud detection engine 11. Feedback messages are also provided to the configuration and investigation team 13. Based on the alerts from the unsupervised performance assessor 14 and supervised performance assessor 15, the configuration and administration team 13 provides further manual configuration to the fraud detection engine 11 and components thereof.
- the unsupervised performance assessor 14 and the supervised performance assessor 15 may be in the form of a programmed computer or a network of computers that may be independent from or form part of the overall fraud detection system.
- the unsupervised performance assessor 14 and supervised performance assessor 15 both automatically monitor the performance of individual components within the fraud detection engine 11 and according to the method described above provide so that the feedback is used to modify the behaviour of components of the fraud detection engine 11 to maximise fraud detection performance.
- the unsupervised performance assessor 14 monitors the rates at which individual fraud detection engine components generate alerts, and execute scripts to provide the feedback to the fraud detection engine 11 should the rates fall below or rise above acceptable levels set by the configuration and administration team 13.
- the unsupervised performance assessor 14 estimates the alert rate for each component within the fraud detection engine 11 by counting the number of alerts generated by each component over a configurable period of time. The period should be as long as possible to minimise the random variation in the measured alert rate (which results from the finite size of the sample of alert instances), but as short as possible to minimise the response time of the unsupervised performance assessor 14. In practice a time period of one hour has been found to provide a good trade off between these requirements in systems that monitor call data records in telecommunications networks.
- the unsupervised performance assessor maintains a configurable number of configurable alert rate thresholds. Associated with each threshold is a hysteresis, and a pair of scripts, which control the action taken by the UPA 14 when each threshold is passed as a component's alerts rate either increases or decreases.
- the script executes when a component's alert rate passes the threshold as it decreases is referred to as the negative transition script.
- the script executed when the components alert rate passes the other threshold as it increases is referred to as the positive transition script.
- the hysteresis is provided to reset the triggering of the respective script to stop the positive and negative transition scripts being executed in rapid succession as a result of random variation in a component's alert rate when it lies close to one of the thresholds.
- a threshold of 0.001 percent could be defined with a hysteresis of 0.001 percent.
- a component of the fraud detection engine 11 that starts off with an alert rate of 0.1 percent would not cause either of the scripts associated with the threshold to be executed. If its alert rate fell below the 0.001 percent, however, the negative transition script associated with the threshold would be executed. If the alert rate repeatedly crossed the threshold, the negative transition script would not be re-executed unless the alert rate first rose above the threshold plus the hysteresis (i.e. rose above 0.002 percent), causing the positive transition script to be executed.
- the scripts can send signals to the fraud detection engine 11 components, and the signals may be used to modify the configurations of these components.
- Different fraud detection engine 11 components can accept different signals from the scripts, depending on their design and implementation.
- a change detection algorithm within the fraud detection engine 11 may be able to accept signals instructing it to reduce its sensitivity by a specific amount (for example, by increasing an internal threshold), whereas a neural network may only be able to accept a signal instructing it to disable itself.
- a change detection algorithm adjusting its sensitivity in response to a signal generated by a script, its sensitivity could be specified explicitly in the algorithm's configuration, and modified directly by the script without any signal being sent to the algorithm itself.
- Scripts can also send messages to the configuration and administration team 13 to inform them that alert thresholds have been passed. This provides the team 13 with important information about the performance of individual fraud detection engine 11 components that is useful for maintaining the system's configuration. For example, when the configuration is reviewed by the configuration and administration team 13, the messages sent by the scripts tell the team 13 which components in the original configuration generated too many or too few alerts, and hence need to be modified.
- a typical application of the unsupervised performance assessor 14 is to define two thresholds: 1) the 'flood' threshold, which identifies fraud detection engine 11 components that generate too many alerts, and 2) the 'drought' threshold, which identifies fraud detection engine 11 components that generate too few.
- the flood threshold would be defined to be around 5 percent or so (depending on the rate at which the alert investigation team 12 can process alerts), and the drought threshold to be around 0.001 percent. Hystereses associated with each of 4 and 0.001 percent have been found to work well in practice.
- the positive transition script associated with the flood threshold is set to disable the associated fraud detection engine 11 component and send a message to the configuration and administration team 13, as shown below.
- the negative transition script associated with the drought threshold is set to send a message to the configuration and administration team 13 but to leave the fraud detection engine 11 component enabled below.
- the functions 'OnPositiveTransitionOfFloodThreshold' and OnNegativeTransitionOfDroughtThreshold' are passed to identifiers of the fraud detection engine 11 components responsible for the scripts being invoked.
- the identifiers are numeric, alphanumeric, or alphabetic strings that are associated with, and unique to, each fraud detection engine 11 component. For example, a change detection component within the fraud detection engine 11 that monitors the cost of calls may be given the identifier 'ChangeDetector UniversalCallCost'.
- the argument of the 'SendMessage' function is the string that is to be sent to the configuration and administration team 13.
- the identifier responsible for the script's execution is inserted into that string in the pseudo-code so that, for example, if the aforementioned change detection algorithm caused the positive flood transition script to be executed, the message 'Warning: FDE component ChangeDetector_CallCost is in flood and has been disabled' would be sent to the configuration and administration team.
- the negative and positive transition scripts associated with the flood and drought thresholds respectively may be empty (i.e. they do nothing).
- the unsupervised performance assessor 14 be configured to disable fraud detection engine 11 components that generate unexpectedly large numbers of alerts, which would swamp the alert investigation team 12 if they were allowed to continue, but only warns the configuration and administration team 13 if a component generates too few alerts so that its configuration can be modified at the next configuration review.
- An alternative arrangement could add an additional 'flood warning' threshold at around 3 percent, with a hysteresis of 2 percent.
- the team 13 can be issued with a warning that a fraud detection engine 11 component is at risk of being disabled by the positive transition flood threshold script, allowing time for the team 13 to modify the component's configuration to reduce its alert rate before this occurs.
- monitoring the alert rate of fraud detection engine 11 components with the unsupervised performance assessor 14 is of great practical importance because it allows components that are generating too few or too many alerts to be identified. For example, if a component generates too many alerts, the throughput of the system is reduced by the overhead of processing the alerts and transferring them to the alert investigation team 12. This can cause the fraud detection system to lag behind its input, producing a backlog and robbing the system of its ability to search for fraud in real time. This increases the amount of time that frauds can persist before they are detected and stopped, increasing the revenue lost by the network operator.
- Any component that generates a large number of alerts is also likely to be generating many more alerts in response to events that are not frauds than those that are, and is thus a poor fraud detector.
- the overall fraud detection performance of the system could therefore be improved by modifying the configuration of the component or removing it altogether.
- a fraud detection engine 11 component that generates too few alerts is also problematic, because the resources it uses within the system may not be justified by its fraud detection abilities. (For example, this is certainly the case for a component that never generates alerts.) Such components can usually operate at higher sensitivities without generating an excessive number of alerts, while also offering increased speed and strength of response to actual fraud events. Alternatively, the performance of the system can sometimes be improved if these components are removed completely because the increase in throughput that results can increase the speed at which frauds are detected, thus reducing the revenue lost by the network operator before the fraud is stopped.
- the assessor 14 can respond to changes in the alert rates of individual fraud detection engine 11 components far faster than can the configuration and administration team 13.
- a fraud detection system with a UPA-type mechanism is thus able to respond to changes in its environment, far more quickly than one without.
- the supervised performance assessor (SPA) 15 is similar to the unsupervised performance assessor 14, except that the supervised performance assessor 15 uses feedback provided by the alert investigation team 12 to maintain statistics on, and apply thresholds to, a function of the false positive and false negative rates of fraud detection engine 11 components.
- a false positive occurs when a fraud detection engine 11 component generates an alert that, upon investigation by the alert investigation team 12 turns out not to be associated with a real fraud.
- a false negative occurs when a fraud detection engine 11 component fails to generate an alert for an event that was part of a fraud.
- Thresholds within the supervised performance assessor 15 are defined on the function of the false negative and false positive rates of fraud detection engine 11 components, and trigger the execution of scripts in the same way as scripts are triggered within the unsupervised performance assessor 14.
- the function of the false negative and false positive rates of fraud detection engine 11 components are moving averages of their false negative and false positive rates over a configurable number of configurable periods. For example, a period of one day is often chosen as the configurable period, and the moving average is taken over a fourteen day window of such periods.
- the supervised performance assessor 15 has an important role to play in maintaining good fraud detection performance within the system by identifying components within the fraud detection engine 11 that are generating too many fraud alerts in response to normal activity, or generating too few alerts in response to fraud.
- the former are problematic because they use system resources - particularly those of the alert investigation team 13 - to search for fraudulent activity that does not exist. This increases the amount of time that the team 12 takes to identify the real frauds, and hence increases the revenue lost by the network operator to the fraudsters before the fraud is stopped.
- a fraud detection engine 11 component generates too few alerts in response to real frauds, it is likely that its sensitivity could be increased, with the result that it responds more rapidly to real fraud events.
- the SPA's ability to automatically execute scripts in response to false positive and false negative alert rate moving averages crossing thresholds means that it can adapt the fraud detection engine 11 components far more rapidly to changing conditions than can a fraud detection system that relies on human intervention.
- the skilled addressee will realise that the present invention provides advantages over existing fraud detection systems that do not have a performance assessor automatically monitoring the performance of the fraud detection engine.
- the overall systems performance in terms of fraud detection sensitivity, and throughput, may be maximised as well as minimising the number of false alerts sent to the alert investigation team.
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP03718556A EP1540429A1 (en) | 2002-05-13 | 2003-05-13 | An automated performance monitoring and adaptation system |
AU2003222680A AU2003222680A1 (en) | 2002-05-13 | 2003-05-13 | An automated performance monitoring and adaptation system |
US10/987,451 US20050154688A1 (en) | 2002-05-13 | 2004-11-12 | Automated performance monitoring and adaptation system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB0210938.7 | 2002-05-13 | ||
GB0210938A GB0210938D0 (en) | 2002-05-13 | 2002-05-13 | An automatic performance monitoring and adaptation system |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/987,451 Continuation US20050154688A1 (en) | 2002-05-13 | 2004-11-12 | Automated performance monitoring and adaptation system |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2003096129A1 true WO2003096129A1 (en) | 2003-11-20 |
Family
ID=9936574
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/AU2003/000577 WO2003096129A1 (en) | 2002-05-13 | 2003-05-13 | An automated performance monitoring and adaptation system |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP1540429A1 (en) |
AU (1) | AU2003222680A1 (en) |
GB (1) | GB0210938D0 (en) |
WO (1) | WO2003096129A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10756955B2 (en) | 2015-11-24 | 2020-08-25 | International Business Machines Corporation | Dynamic thresholds for computer system alerts |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4213127A (en) * | 1979-01-31 | 1980-07-15 | The United States Of America As Represented By The Secretary Of The Air Force | Doubly adaptive CFAR apparatus |
US5627886A (en) * | 1994-09-22 | 1997-05-06 | Electronic Data Systems Corporation | System and method for detecting fraudulent network usage patterns using real-time network monitoring |
US5819226A (en) * | 1992-09-08 | 1998-10-06 | Hnc Software Inc. | Fraud detection using predictive modeling |
US5966650A (en) * | 1995-07-13 | 1999-10-12 | Northern Telecom Limited | Detecting mobile telephone misuse |
WO2000047006A1 (en) * | 1999-02-01 | 2000-08-10 | Nokia Networks Oy | Adaptation of codec operating modes in a telecommunication network |
WO2000064193A2 (en) * | 1999-04-20 | 2000-10-26 | Amdocs Software Systems Limited | Telecommunications system for generating a three-level customer behavior profile and for detecting deviation from the profile. |
US6327352B1 (en) * | 1997-02-24 | 2001-12-04 | Ameritech Corporation | System and method for real-time fraud detection within a telecommunications system |
-
2002
- 2002-05-13 GB GB0210938A patent/GB0210938D0/en not_active Ceased
-
2003
- 2003-05-13 WO PCT/AU2003/000577 patent/WO2003096129A1/en not_active Application Discontinuation
- 2003-05-13 AU AU2003222680A patent/AU2003222680A1/en not_active Abandoned
- 2003-05-13 EP EP03718556A patent/EP1540429A1/en not_active Withdrawn
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4213127A (en) * | 1979-01-31 | 1980-07-15 | The United States Of America As Represented By The Secretary Of The Air Force | Doubly adaptive CFAR apparatus |
US5819226A (en) * | 1992-09-08 | 1998-10-06 | Hnc Software Inc. | Fraud detection using predictive modeling |
US5627886A (en) * | 1994-09-22 | 1997-05-06 | Electronic Data Systems Corporation | System and method for detecting fraudulent network usage patterns using real-time network monitoring |
US5966650A (en) * | 1995-07-13 | 1999-10-12 | Northern Telecom Limited | Detecting mobile telephone misuse |
US6327352B1 (en) * | 1997-02-24 | 2001-12-04 | Ameritech Corporation | System and method for real-time fraud detection within a telecommunications system |
WO2000047006A1 (en) * | 1999-02-01 | 2000-08-10 | Nokia Networks Oy | Adaptation of codec operating modes in a telecommunication network |
WO2000064193A2 (en) * | 1999-04-20 | 2000-10-26 | Amdocs Software Systems Limited | Telecommunications system for generating a three-level customer behavior profile and for detecting deviation from the profile. |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10756955B2 (en) | 2015-11-24 | 2020-08-25 | International Business Machines Corporation | Dynamic thresholds for computer system alerts |
Also Published As
Publication number | Publication date |
---|---|
AU2003222680A1 (en) | 2003-11-11 |
EP1540429A1 (en) | 2005-06-15 |
GB0210938D0 (en) | 2002-06-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7570751B2 (en) | System and method for real-time fraud detection within a telecommunication network | |
EP0897566B1 (en) | Monitoring and retraining neural network | |
EP0894378B1 (en) | Signature based fraud detection system | |
JP5547289B2 (en) | Method and apparatus for detecting fraud in a telecommunications network | |
KR20010072141A (en) | System for intrusion detection and vulnerability analysis in a telecommunications signaling network | |
US20100128860A1 (en) | Methods, computer program products, and systems for managing voice over internet protocol (voip) network elements | |
US20150004964A1 (en) | Method and apparatus for telecommunications network performance anomaly events detection and notification | |
JP2009022042A (en) | Automated fraud management in transaction-based networks | |
WO1998032085A1 (en) | Generic processing capability | |
US7505567B1 (en) | Method for providing detection of fault location for defect calls in a VoIP network | |
US20050154688A1 (en) | Automated performance monitoring and adaptation system | |
CN106385339A (en) | Monitoring method and monitoring system for access performance of enterprise network | |
EP0890255B1 (en) | Fraud monitoring in a telecommunications network | |
US6570968B1 (en) | Alert suppression in a telecommunications fraud control system | |
CN106452941A (en) | Network anomaly detection method and device | |
EP1540429A1 (en) | An automated performance monitoring and adaptation system | |
US7367055B2 (en) | Communication systems automated security detection based on protocol cause codes | |
TWI812491B (en) | System and method for cybersecurity threat detection and early warning | |
CN117834163A (en) | System and method for detecting and early warning threat of security | |
US20230344933A1 (en) | Systems and methods for use in blocking of robocall and scam call phone numbers | |
JPH10229395A (en) | Method and device for managing fault information | |
CN113157652A (en) | User line image and abnormal behavior detection method based on user operation audit | |
KR20050049028A (en) | Hooters |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PH PL PT RO RU SC SD SE SG SK SL TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 10987451 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 3933/DELNP/2004 Country of ref document: IN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2003222680 Country of ref document: AU |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2003718556 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 2003718556 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: JP |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: JP |