CA2393919C - Methods for optimizing and evaluating network access techniques - Google Patents
Methods for optimizing and evaluating network access techniques Download PDFInfo
- Publication number
- CA2393919C CA2393919C CA002393919A CA2393919A CA2393919C CA 2393919 C CA2393919 C CA 2393919C CA 002393919 A CA002393919 A CA 002393919A CA 2393919 A CA2393919 A CA 2393919A CA 2393919 C CA2393919 C CA 2393919C
- Authority
- CA
- Canada
- Prior art keywords
- vpn
- access
- test
- network
- method recited
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/24—Testing correct operation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
Abstract
Methods of evaluating access to Virtual Private Networks (VPNs) from alternate access connections and for improving access to VPNs from alternate access connections such as ISPs. The methods include downloading a set of tests through a network backbone to simulate access to the VPN from the alternate access connection and comparing the results of the test to benchmarks for the test of access to the VPN from leased lines and modems. By obtaining comparisons of the tests and the benchmarks, it is possible to determine the throughput of data through the VPN and tell the customer how its access connections are affecting, among other things, the raw data throughput. This allows the customer and/or the network service provider to adjust the access connections to improve communications with the VPN.
Description
METHODS FOR OPTIIViIZING AND EVALUATING
NETWORK ACCESS TECHNIQUES
BACKGROUND OF THE INVENTION
Field of the Invention This invention relates to methods for evaluating different network access arrangements by testing access connections and comparing the test results against benchmarks for the tested connections that would be implemented on various types or leased lines. More specifically, the invention relates to methods of improving network access when there is a mixture of access technologies available to users of the network.
Descriution of the Related Art The demand for telecommunications services has mushroomed over the last few years. Many types of networks now exist through which data, voice, video and other multimedia applications can be routed and switched. Many forms of data and applications may be downloaded through such networks, such as Internet and Intranet web page fetches, form processing, mail fetches, supply chain and HR transactions, for example Service Access Point (SAP) transactions and PeopleSoft transactions, queries to Internet servers, voice over IP (VoIP), and other applications.
Virtual private networks (VPNs) have typically been created which allow customers or users of the network to obtain efficient use of the network resources, and cheaper service costs for data and other services. A VPN is typically set up with a series of leased lines or permanent virtual circuits (PVCs) that interconnect with a local public switched telephone network (PSTN) to provide the aforementioned cost savings, efficiency and enhanced usability of services. The backbone for typical VPNs is a wide area network (WAN), local area network (LAN), or other forms of private or shared-private networks.
Typically, data customers of a VPN have connected to the VPN through a leased line or a modem.
The future development of VPN access technology will require that network owners and service providers provide access to the VPN by whatever connection the user or customer has at its disposal. Thus, connecting to the VPN from the customers' Internet Service Provider (ISP) or through the Internet itself will be a primary modality by which a VPN is accessed. Customers will therefore connect to their VPNs using IPsec tunneling. In point of fact, the VPN may be accessed by the customer through a mix of access technologies such as IPsec (IP security), which is a collection of security measures for IP
tunneling, digital subscriber lines (DSL), modems and leased lines. Thus, the VPNs of the future will require access management and evaluation that cuts across a variety of access technologies. This will provide new and unforeseen problems with network management since DSL, IPsec tunneling and other forms of private access to networks are not in the control of the network or service providers and therefore cannot be easily monitored or characterized.
Currently, there exists diagnostic equipment for networks that can report the network data carrying characteristics such as throughput, for example. However, these current diagnostic tools require that the VPN be established in the network and running before any information can be gathered with the diagnostic tools. This does not provide any predictive capability to the network and cannot tell network designers or service providers which access connections will be the best performers for the particular services that the customer wishes the VPN to perform. These current tools therefore do not provide adequate methods for assessing VPN access concerns.
There thus exists a long-felt need in the art for methods of characterizing, evaluating and/or optimizing network access from a variety of sources. These methods should be designed to folly understand the issues of data management and throughput when customers access their VPNs from a variety of access points and technologies. Moreover, the methods should allow the network and service providers to be able to make recommendations to customers concerning the access technology to utilize, and to modify the access technologies and the access to the VPN in order to improve and/or optimize access thereto.
Such results have not heretofore been achieved in the art.
2a SUMMARY OF THE INVENTION
Certain exemplary embodiments can provide a method for evaluating the performance of a customer's virtual private network (VPN) network access arrangements from an alternate access connection, comprising the steps of downloading a test through a network backbone to a network that will produce the VPN for a user to simulate interactions with the VPN and the alternate access connection; testing the data-carrying performance of the VPN with respect to the alternate access connection; comparing results of the test that has been downloaded to benchmarks for the test that could be run on a set of access connections to the 1o backbone from other than the alternate access connection; and reporting the results of the comparison to providers of the VPN to customers.
Certain exemplary embodiments can provide a method for improving access to a virtual private network (VPN), comprising the steps of: simulating interactions with the VPN from access by an alternate access connection by downloading a test through a network that will produce the VPN, testing the data-carrying performance of the VPN with the alternate access connection, and comparing results of the test with a benchmark for the test that would have connected to the VPN through an access connection other than from the alternate access connection; and adjusting access to the VPN as a result of the comparisons made.
2o The aforementioned problems are solved, and long-felt needs met, by methods of evaluating and improving network access from alternate access connections to VPNs provided in accordance with the present invention. The methods preferably comprise downloading a test through a network backbone to a network that will produce a VPN for a user to simulate interactions with the VPN and the alternate access connection. Results of the test that has been downloaded are compared to benchmarks for the test that could be run through a set of access connections to the backbone from other than an alternate access connection. In a preferred embodiment results of the comparison are then reported.
Even more preferably, the present methods improve access to a VPN by simulating interactions with the VPN and adjusting access to the VPN as a result of the simulations.
Advantageously, the inventive methods allow VPN managers and service providers to assess the performance of a customer's VPN before the VPN is created, thereby allowing the VPN
provider to predict the VPN's performance given the network connections that will come into play, and to adjust the network connections. This also allows the VPN
provider to advise the customer of the better or best access connections to use.
The methods of the present invention thus allow network designers and providers to improve data throughput in a VPN and to greatly increase the efficiency of the VPN for a customer. Moreover, the present methods allow the VPN provider to inform customers regarding the performance to their VPN and to advise customers of the type of services that should be accessed through a private service to the VPN as opposed to access to the VPN
through some other type of access connection, for example, a leased line (T1, T3, etc.), or a modem. Such results have not heretofore been achieved in the art.
These and other features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings which are first described briefly below. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the invention, for which reference should be made to the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings wherein like reference numerals refer to like elements throughout the several views thereof:
Figure 1 depicts a prior art network having a backbone through which VPNs have been established through leased lines and/or modems;
Figure 2 depicts a netwark having a backbone through which VPNs may be established with access connections that include alternate access connections and servers such as Internet Service Providers (ISPs), as well as leased lines, modems, DSL connections and the Iike;Figure 3 is a block diagram of a system for testing VPNs in accordance with the methods of the present invention;
Figure 4 depicts a table that embodies a report of tests or simulations provided in accordance with the present invention;
Figure 5 depicts a table that embodies a report of retesting of alternate access connections to a VPN provided in accordance with the present invention; and Figure 6 is a flow diagram of a preferred embodiment of methods of the present invention that may be programmed in software.
DETAILED DESCRIPTION OF THE PREFEIZR~D EMBODI1VIENTS
Refernng now to the drawings, Figure 1 depicts a prior art network for establishing VPNs that contains a backbone 10, for example a WAN, LAN or other appropriate media to establish connectivity for customers. A plurality of customer premises equipment (CPE) 20 exist at locations remote from each other and are operable to establish connections with the VPN so that services can be provided to the customers having access from the different CPEs 20. In the past, the connectivity has been established exclusively through leased lines (LL) 30 and/or modem connections 40.
Figure 2 depicts a new configuration for establishing VPNs which will become prevalent in the art as ISPs more and more take over data and voice service provisioning.
For example, CPEs 20 may communicate with the backbone 10 through a LL 30 or a modem as has been done in the past. Moreover, CPE 20 may communicate with the backbone over broadband or cable connections 40 or DSL connections 50. In this case, the CPE 20 will likely first encounter an ISP 60 which introduces a new dynamic to the provisioning process as well as the interaction and set-up of the customer's VPN. This arrangement will produce the need for the ISP 60 to establish a peered connection 70 with the backbone 10 in order for the VPN to be established, remain active and function properly.
Such a situation introduces new issues to the network's performance which heretofore have not been quantified or well understood. This is particularly the case since the ISP 60 is out of the control of the network provider.
Referring now to Figure 3, a system to aid in quantifying the network characteristics for the hybrid access system of Figure 2 is depicted. A computer 80 is interfaced to the network and is operable to download to the network a script or client that contains one or more tests that will simulate access to a customer's VPN. It will be appreciated by those skilled in the art that computer 80 could be a server, personal computer, laptop computer, personal digital assistant (PDA) or any other computing device that is adapted or adaptable 5 to interface through the appropriate communication protocol with the network.
The tests downloaded by computer 80 will simulate interactions with private servers or ISPs 60 connected to the VPN backbone 10 and a server 90 associated with VPN
backbone 10. Many types of tests may be downloaded. For example, but not intending to limit the invention in' any way, the tests may include and simulate one or a combination of Intranet web page fetches, Intranet form processing, Mail fetches, supply chain and HR
transactions, for example SAP transactions and PeopleSoft transactions, queries to Internet servers (for example the server 100 shown in Figure 3) and VoIP, among others.
It will be further appreciated by those with skill in the art that the set of tests is expandable and can include any other network enabled software processes that the customer might be interested in running on their network.
In accordance with the invention, at the same time that the tests) are being downloaded and run through the network, a set of benchmarks for the same set of queries embodied by the tests) run on leased lines is preferably accessed for comparisons to the results of the access to the VPN achieved by the test(s). This results in a substantive comparison of access to the VPN through network backbone 10 from an alternate access server or server by the tests) against the access achieved purely through the benchmarks associated with the well-understood and defined leased lines. This information might then be provided to customers of the VPNs so that they may understand the relative benefits of different types of access connections to their VPNs. Moreover, since oftentimes a customer will complain to the network provider regarding slow or otherwise inadequate access to the VPN, the results of such comparisons will give the network provider the ammunition it needs to explain the problems to the customers and to perhaps elucidate that it is the customer's alternate server access connections that are causing the issues extant with current access speeds. These options have not heretofore been available to network providers in the past.
Since it may be desirable to provide reports of the comparisons made, formatting and presentation of the reports will be important in understanding them. Figure 4 is an example of a hypothetical report or table of the comparisons made in accordance with certain of the methods of the present invention. As can be seen, the alternate access (private server access) connection speeds to the VPN are compared to 56 Kbps, 512 Kbps and T1 access speeds for various applications. In the current example of Figure 4, raw throughput gives the maximum sustained flows into and out of the network. In some cases, transactions could not be completed at all; these are denoted with an *. As will be observed, different tests produce different times to comglete the application for multiple attempts, usually three in all.
The results of the tests may lead to several conclusions for the customer. For example, the customer may have purchased a 1 Mbps SDSL line, but testing as described above shows that the behavior of the line in the VPN will be closer to a 56 kbps link than any other connection. The customer may be very unhappy with the performance of their expensive, high bandwidth line and so testing in accordance with the present invention will show the customer why the high-speed line performs so poorly, and will create a paradigm for devising possible remedies to the problem. These possibilities have not heretofore existed or been available in the art.
Another use of the methods of the present invention is periodic retesting of alternate access to the system, particularly from private servers such as ISPs. One way to focus on retesting for any type of access connection is to observe raw throughput data to determine the performance of the VPN with different access connections. Figure 5 is a table of raw throughput data to judge performance at different times of the year. By examining the table of Figure 5, it can be observed that something occurred at the end of September or the beginning of October that had a serious impact on the throughput, that is, the maximum and minimum throughputs decreased significantly. Why this occurred is unclear, and may be unknowable without further analysis. However, one might infer that if other customers entering their VPNs at the same access point to the backbone show no significant changes to the throughput, then the problem probably lies further out and the VPN
provider or customer can initiate action with other networks or providers to attempt to solve this problem. This capability in and of itself provides a tremendous diagnostic advantage over current methods of diagnosing network throughput problems, and greatly enhances the efficiencies of the network employing the inventive methods. Moreover, this allows the network owner and/or VPN provider to serve its customers better, thereby greatly increasing the probability that these customers will be satisfied and retained. Again, these salutary effects have not heretofore been achieved in the art.
Yet other aspects of the present invention allow adjustments to network access connections to be made by a customer in light of the tests and comparisons made and described herein. These methods will allow network providers and/or customers to adjust access to the VPN through several methods, including, but not limited to, choosing a new access connection or adjusting hardware and/or software functionality within existing connections to improve access to the VPN. This may allow for a measurable increase in performance of the VPN which the customer and/or the network provider could measure to make the adjustments. Such methods may be programmed in software with an appropriate software language such as C++, UNIX, VISUALBASIC, JAVA, HTML, XML, WML, or any other high level, preferably object oriented language.
Whichever language is chosen in which to write and design the software, Figure 6 is a flow chart of an illustrative embodiment of the inventive methods. The methods begin at step 110, and at step 120 at least one test is downloaded through the network to simulate access to the VPN. At step 130, the network is accessed by the test to simulate interactions with the VPN through a private server, and at step 140 comparisons are made of the results of the test with a benchmark for the interactions over leased lines.
Optionally, at step 150 the results of the comparison may be reported in the form of the tables of Figures 4 and 5, or otherwise, to the customers, network providers, or other appropriate individuals or entities that would have an interest in the results.
It is then preferably determined at step 160 whether adjustments of the access to the VPNs should be made as a result of the aforementioned comparisons. If so, then at step 170 the adjustments are made, for example, a new access connection is chosen or the access connection software and/or hardware is modified to improve the connection. It will be appreciated by those skilled in the art that other adjustments may be made depending on the types of connections available to access the VPN and the hardware and/or software used by the private server, ISP or other providers of the leased lines. If it is not desired to make adjustments, then at step 180 it is preferably determined whether there are other or additional tests that should be run to fiuther characterize and/or improve the access connections to the VPN. If so, then the method reverts to step 130. If not, the method stops at step 190.
Thus the methods of the present invention provide network designers and providers with the ability to improve data throughput in a VPN and to greatly increase the efficiency of the VPN for a customer. Moreover, these methods allow the VPN provider to inform customers regarding the performance to their VPN and to advise customers of the type of services that should be accessed through a private provider to the VPN as opposed to access to the VPN through some other type of access connection, for example, a leased line (T1, T3, etc.), or a modem. These results have not heretofore been achieved in the art.
While there have been shown and described and pointed out certain novel features of the present invention. as applied to preferred embodiments thereof, it will be understood by those skilled in the art that various omissions and substitutions and changes in the methods and apparatus described herein, and in their operation, may be made by those skilled in the art without departing from the spirit and scope of the invention. It is expressly intended that all combinations of those elements and/or method steps which perform substantially the same function in substantially the same way to achieve the same results are within the scope of the invention. Substitutions of method steps and elements from one described embodiment to another are also fully intended and contemplated. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.
NETWORK ACCESS TECHNIQUES
BACKGROUND OF THE INVENTION
Field of the Invention This invention relates to methods for evaluating different network access arrangements by testing access connections and comparing the test results against benchmarks for the tested connections that would be implemented on various types or leased lines. More specifically, the invention relates to methods of improving network access when there is a mixture of access technologies available to users of the network.
Descriution of the Related Art The demand for telecommunications services has mushroomed over the last few years. Many types of networks now exist through which data, voice, video and other multimedia applications can be routed and switched. Many forms of data and applications may be downloaded through such networks, such as Internet and Intranet web page fetches, form processing, mail fetches, supply chain and HR transactions, for example Service Access Point (SAP) transactions and PeopleSoft transactions, queries to Internet servers, voice over IP (VoIP), and other applications.
Virtual private networks (VPNs) have typically been created which allow customers or users of the network to obtain efficient use of the network resources, and cheaper service costs for data and other services. A VPN is typically set up with a series of leased lines or permanent virtual circuits (PVCs) that interconnect with a local public switched telephone network (PSTN) to provide the aforementioned cost savings, efficiency and enhanced usability of services. The backbone for typical VPNs is a wide area network (WAN), local area network (LAN), or other forms of private or shared-private networks.
Typically, data customers of a VPN have connected to the VPN through a leased line or a modem.
The future development of VPN access technology will require that network owners and service providers provide access to the VPN by whatever connection the user or customer has at its disposal. Thus, connecting to the VPN from the customers' Internet Service Provider (ISP) or through the Internet itself will be a primary modality by which a VPN is accessed. Customers will therefore connect to their VPNs using IPsec tunneling. In point of fact, the VPN may be accessed by the customer through a mix of access technologies such as IPsec (IP security), which is a collection of security measures for IP
tunneling, digital subscriber lines (DSL), modems and leased lines. Thus, the VPNs of the future will require access management and evaluation that cuts across a variety of access technologies. This will provide new and unforeseen problems with network management since DSL, IPsec tunneling and other forms of private access to networks are not in the control of the network or service providers and therefore cannot be easily monitored or characterized.
Currently, there exists diagnostic equipment for networks that can report the network data carrying characteristics such as throughput, for example. However, these current diagnostic tools require that the VPN be established in the network and running before any information can be gathered with the diagnostic tools. This does not provide any predictive capability to the network and cannot tell network designers or service providers which access connections will be the best performers for the particular services that the customer wishes the VPN to perform. These current tools therefore do not provide adequate methods for assessing VPN access concerns.
There thus exists a long-felt need in the art for methods of characterizing, evaluating and/or optimizing network access from a variety of sources. These methods should be designed to folly understand the issues of data management and throughput when customers access their VPNs from a variety of access points and technologies. Moreover, the methods should allow the network and service providers to be able to make recommendations to customers concerning the access technology to utilize, and to modify the access technologies and the access to the VPN in order to improve and/or optimize access thereto.
Such results have not heretofore been achieved in the art.
2a SUMMARY OF THE INVENTION
Certain exemplary embodiments can provide a method for evaluating the performance of a customer's virtual private network (VPN) network access arrangements from an alternate access connection, comprising the steps of downloading a test through a network backbone to a network that will produce the VPN for a user to simulate interactions with the VPN and the alternate access connection; testing the data-carrying performance of the VPN with respect to the alternate access connection; comparing results of the test that has been downloaded to benchmarks for the test that could be run on a set of access connections to the 1o backbone from other than the alternate access connection; and reporting the results of the comparison to providers of the VPN to customers.
Certain exemplary embodiments can provide a method for improving access to a virtual private network (VPN), comprising the steps of: simulating interactions with the VPN from access by an alternate access connection by downloading a test through a network that will produce the VPN, testing the data-carrying performance of the VPN with the alternate access connection, and comparing results of the test with a benchmark for the test that would have connected to the VPN through an access connection other than from the alternate access connection; and adjusting access to the VPN as a result of the comparisons made.
2o The aforementioned problems are solved, and long-felt needs met, by methods of evaluating and improving network access from alternate access connections to VPNs provided in accordance with the present invention. The methods preferably comprise downloading a test through a network backbone to a network that will produce a VPN for a user to simulate interactions with the VPN and the alternate access connection. Results of the test that has been downloaded are compared to benchmarks for the test that could be run through a set of access connections to the backbone from other than an alternate access connection. In a preferred embodiment results of the comparison are then reported.
Even more preferably, the present methods improve access to a VPN by simulating interactions with the VPN and adjusting access to the VPN as a result of the simulations.
Advantageously, the inventive methods allow VPN managers and service providers to assess the performance of a customer's VPN before the VPN is created, thereby allowing the VPN
provider to predict the VPN's performance given the network connections that will come into play, and to adjust the network connections. This also allows the VPN
provider to advise the customer of the better or best access connections to use.
The methods of the present invention thus allow network designers and providers to improve data throughput in a VPN and to greatly increase the efficiency of the VPN for a customer. Moreover, the present methods allow the VPN provider to inform customers regarding the performance to their VPN and to advise customers of the type of services that should be accessed through a private service to the VPN as opposed to access to the VPN
through some other type of access connection, for example, a leased line (T1, T3, etc.), or a modem. Such results have not heretofore been achieved in the art.
These and other features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings which are first described briefly below. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the invention, for which reference should be made to the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings wherein like reference numerals refer to like elements throughout the several views thereof:
Figure 1 depicts a prior art network having a backbone through which VPNs have been established through leased lines and/or modems;
Figure 2 depicts a netwark having a backbone through which VPNs may be established with access connections that include alternate access connections and servers such as Internet Service Providers (ISPs), as well as leased lines, modems, DSL connections and the Iike;Figure 3 is a block diagram of a system for testing VPNs in accordance with the methods of the present invention;
Figure 4 depicts a table that embodies a report of tests or simulations provided in accordance with the present invention;
Figure 5 depicts a table that embodies a report of retesting of alternate access connections to a VPN provided in accordance with the present invention; and Figure 6 is a flow diagram of a preferred embodiment of methods of the present invention that may be programmed in software.
DETAILED DESCRIPTION OF THE PREFEIZR~D EMBODI1VIENTS
Refernng now to the drawings, Figure 1 depicts a prior art network for establishing VPNs that contains a backbone 10, for example a WAN, LAN or other appropriate media to establish connectivity for customers. A plurality of customer premises equipment (CPE) 20 exist at locations remote from each other and are operable to establish connections with the VPN so that services can be provided to the customers having access from the different CPEs 20. In the past, the connectivity has been established exclusively through leased lines (LL) 30 and/or modem connections 40.
Figure 2 depicts a new configuration for establishing VPNs which will become prevalent in the art as ISPs more and more take over data and voice service provisioning.
For example, CPEs 20 may communicate with the backbone 10 through a LL 30 or a modem as has been done in the past. Moreover, CPE 20 may communicate with the backbone over broadband or cable connections 40 or DSL connections 50. In this case, the CPE 20 will likely first encounter an ISP 60 which introduces a new dynamic to the provisioning process as well as the interaction and set-up of the customer's VPN. This arrangement will produce the need for the ISP 60 to establish a peered connection 70 with the backbone 10 in order for the VPN to be established, remain active and function properly.
Such a situation introduces new issues to the network's performance which heretofore have not been quantified or well understood. This is particularly the case since the ISP 60 is out of the control of the network provider.
Referring now to Figure 3, a system to aid in quantifying the network characteristics for the hybrid access system of Figure 2 is depicted. A computer 80 is interfaced to the network and is operable to download to the network a script or client that contains one or more tests that will simulate access to a customer's VPN. It will be appreciated by those skilled in the art that computer 80 could be a server, personal computer, laptop computer, personal digital assistant (PDA) or any other computing device that is adapted or adaptable 5 to interface through the appropriate communication protocol with the network.
The tests downloaded by computer 80 will simulate interactions with private servers or ISPs 60 connected to the VPN backbone 10 and a server 90 associated with VPN
backbone 10. Many types of tests may be downloaded. For example, but not intending to limit the invention in' any way, the tests may include and simulate one or a combination of Intranet web page fetches, Intranet form processing, Mail fetches, supply chain and HR
transactions, for example SAP transactions and PeopleSoft transactions, queries to Internet servers (for example the server 100 shown in Figure 3) and VoIP, among others.
It will be further appreciated by those with skill in the art that the set of tests is expandable and can include any other network enabled software processes that the customer might be interested in running on their network.
In accordance with the invention, at the same time that the tests) are being downloaded and run through the network, a set of benchmarks for the same set of queries embodied by the tests) run on leased lines is preferably accessed for comparisons to the results of the access to the VPN achieved by the test(s). This results in a substantive comparison of access to the VPN through network backbone 10 from an alternate access server or server by the tests) against the access achieved purely through the benchmarks associated with the well-understood and defined leased lines. This information might then be provided to customers of the VPNs so that they may understand the relative benefits of different types of access connections to their VPNs. Moreover, since oftentimes a customer will complain to the network provider regarding slow or otherwise inadequate access to the VPN, the results of such comparisons will give the network provider the ammunition it needs to explain the problems to the customers and to perhaps elucidate that it is the customer's alternate server access connections that are causing the issues extant with current access speeds. These options have not heretofore been available to network providers in the past.
Since it may be desirable to provide reports of the comparisons made, formatting and presentation of the reports will be important in understanding them. Figure 4 is an example of a hypothetical report or table of the comparisons made in accordance with certain of the methods of the present invention. As can be seen, the alternate access (private server access) connection speeds to the VPN are compared to 56 Kbps, 512 Kbps and T1 access speeds for various applications. In the current example of Figure 4, raw throughput gives the maximum sustained flows into and out of the network. In some cases, transactions could not be completed at all; these are denoted with an *. As will be observed, different tests produce different times to comglete the application for multiple attempts, usually three in all.
The results of the tests may lead to several conclusions for the customer. For example, the customer may have purchased a 1 Mbps SDSL line, but testing as described above shows that the behavior of the line in the VPN will be closer to a 56 kbps link than any other connection. The customer may be very unhappy with the performance of their expensive, high bandwidth line and so testing in accordance with the present invention will show the customer why the high-speed line performs so poorly, and will create a paradigm for devising possible remedies to the problem. These possibilities have not heretofore existed or been available in the art.
Another use of the methods of the present invention is periodic retesting of alternate access to the system, particularly from private servers such as ISPs. One way to focus on retesting for any type of access connection is to observe raw throughput data to determine the performance of the VPN with different access connections. Figure 5 is a table of raw throughput data to judge performance at different times of the year. By examining the table of Figure 5, it can be observed that something occurred at the end of September or the beginning of October that had a serious impact on the throughput, that is, the maximum and minimum throughputs decreased significantly. Why this occurred is unclear, and may be unknowable without further analysis. However, one might infer that if other customers entering their VPNs at the same access point to the backbone show no significant changes to the throughput, then the problem probably lies further out and the VPN
provider or customer can initiate action with other networks or providers to attempt to solve this problem. This capability in and of itself provides a tremendous diagnostic advantage over current methods of diagnosing network throughput problems, and greatly enhances the efficiencies of the network employing the inventive methods. Moreover, this allows the network owner and/or VPN provider to serve its customers better, thereby greatly increasing the probability that these customers will be satisfied and retained. Again, these salutary effects have not heretofore been achieved in the art.
Yet other aspects of the present invention allow adjustments to network access connections to be made by a customer in light of the tests and comparisons made and described herein. These methods will allow network providers and/or customers to adjust access to the VPN through several methods, including, but not limited to, choosing a new access connection or adjusting hardware and/or software functionality within existing connections to improve access to the VPN. This may allow for a measurable increase in performance of the VPN which the customer and/or the network provider could measure to make the adjustments. Such methods may be programmed in software with an appropriate software language such as C++, UNIX, VISUALBASIC, JAVA, HTML, XML, WML, or any other high level, preferably object oriented language.
Whichever language is chosen in which to write and design the software, Figure 6 is a flow chart of an illustrative embodiment of the inventive methods. The methods begin at step 110, and at step 120 at least one test is downloaded through the network to simulate access to the VPN. At step 130, the network is accessed by the test to simulate interactions with the VPN through a private server, and at step 140 comparisons are made of the results of the test with a benchmark for the interactions over leased lines.
Optionally, at step 150 the results of the comparison may be reported in the form of the tables of Figures 4 and 5, or otherwise, to the customers, network providers, or other appropriate individuals or entities that would have an interest in the results.
It is then preferably determined at step 160 whether adjustments of the access to the VPNs should be made as a result of the aforementioned comparisons. If so, then at step 170 the adjustments are made, for example, a new access connection is chosen or the access connection software and/or hardware is modified to improve the connection. It will be appreciated by those skilled in the art that other adjustments may be made depending on the types of connections available to access the VPN and the hardware and/or software used by the private server, ISP or other providers of the leased lines. If it is not desired to make adjustments, then at step 180 it is preferably determined whether there are other or additional tests that should be run to fiuther characterize and/or improve the access connections to the VPN. If so, then the method reverts to step 130. If not, the method stops at step 190.
Thus the methods of the present invention provide network designers and providers with the ability to improve data throughput in a VPN and to greatly increase the efficiency of the VPN for a customer. Moreover, these methods allow the VPN provider to inform customers regarding the performance to their VPN and to advise customers of the type of services that should be accessed through a private provider to the VPN as opposed to access to the VPN through some other type of access connection, for example, a leased line (T1, T3, etc.), or a modem. These results have not heretofore been achieved in the art.
While there have been shown and described and pointed out certain novel features of the present invention. as applied to preferred embodiments thereof, it will be understood by those skilled in the art that various omissions and substitutions and changes in the methods and apparatus described herein, and in their operation, may be made by those skilled in the art without departing from the spirit and scope of the invention. It is expressly intended that all combinations of those elements and/or method steps which perform substantially the same function in substantially the same way to achieve the same results are within the scope of the invention. Substitutions of method steps and elements from one described embodiment to another are also fully intended and contemplated. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.
Claims (14)
1. A method for evaluating the performance of a customer's virtual private network (VPN) network access arrangements from an alternate access connection, comprising the steps of:
downloading a test through a network backbone to a network that will produce the VPN for a user to simulate interactions with the VPN and the alternate access connection;
testing the data-carrying performance of the VPN with respect to the alternate access connection;
comparing results of the test that has been downloaded to benchmarks for the test that could be run on a set of access connections to the backbone from other than the alternate access connection; and reporting the results of the comparison to providers of the VPN to customers.
downloading a test through a network backbone to a network that will produce the VPN for a user to simulate interactions with the VPN and the alternate access connection;
testing the data-carrying performance of the VPN with respect to the alternate access connection;
comparing results of the test that has been downloaded to benchmarks for the test that could be run on a set of access connections to the backbone from other than the alternate access connection; and reporting the results of the comparison to providers of the VPN to customers.
2. The method recited in claim 1, wherein the benchmarked test comprises a member of the group consisting essentially of Internet web page fetches, Intranet web page fetches, form processing, mail fetches, Service Access Point (SAP) transactions, PeopleSoft transactions, queries to Internet servers, and voice over IP
(VoIP).
(VoIP).
3. The method recited in claim 2, further comprising the step of providing the results of the comparison to customers of the VPN.
4. The method recited in claim 3, wherein the VPN has been created and is being used by customers and further comprising the step of retesting the VPN
to characterize raw throughput of data.
to characterize raw throughput of data.
5. The method recited in claim 1, wherein the comparing step comprises the step of comparing the results of the test to benchmarks for access connections that are leased lines.
6. The method recited in claim 1, wherein the VPN that is produced is a simulation of a customer's VPN before the VPN is created.
7. The method recited in claim 3, further comprising the step of improving access to the VPN for customers of the VPN based on the comparison.
8. A method for improving access to a virtual private network (VPN), comprising the steps of:
simulating interactions with the VPN from access by an alternate access connection by downloading a test through a network that will produce the VPN, testing the data-carrying performance of the VPN with the alternate access connection, and comparing results of the test with a benchmark for the test that would have connected to the VPN through an access connection other than from the alternate access connection;
and adjusting access to the VPN as a result of the comparisons made.
simulating interactions with the VPN from access by an alternate access connection by downloading a test through a network that will produce the VPN, testing the data-carrying performance of the VPN with the alternate access connection, and comparing results of the test with a benchmark for the test that would have connected to the VPN through an access connection other than from the alternate access connection;
and adjusting access to the VPN as a result of the comparisons made.
9. The method recited in claim 8, wherein the simulating step comprises the steps of:
downloading the test through a network backbone to the network; and comparing results of the test that has been downloaded to a plurality of benchmarks for the test that could be run on a set of access connections to the backbone from other than the alternate access connection.
downloading the test through a network backbone to the network; and comparing results of the test that has been downloaded to a plurality of benchmarks for the test that could be run on a set of access connections to the backbone from other than the alternate access connection.
10. The method recited in claim 9, wherein the benchmarked test comprises a member of the group consisting essentially of Internet web page fetches, Intranet web page fetches, form processing, mail fetches, Service Access Point (SAP) transactions, PeopleSoft transactions, queries to Internet servers, and voice over IP
(VoIP).
(VoIP).
11. The method recited in claim 10, wherein the VPN has been created and is being used by customers and further comprising the step of providing the results of the comparison to customers of the VPN.
12. The method recited in claim 11, further comprising the step of retesting the VPN to characterize raw throughput of data.
13. The method recited in claim 12, wherein the comparing step comprises the step of comparing the results of the test to benchmarks for access connections that are leased lines.
14. The method recited in claim 12, wherein the VPN that is produced is a simulation of a customer's VPN before the VPN is created.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US09/909,123 | 2001-07-19 | ||
| US09/909,123 US7035222B2 (en) | 2001-07-19 | 2001-07-19 | Methods for optimizing and evaluating network access techniques |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2393919A1 CA2393919A1 (en) | 2003-01-19 |
| CA2393919C true CA2393919C (en) | 2006-06-13 |
Family
ID=25426667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002393919A Expired - Fee Related CA2393919C (en) | 2001-07-19 | 2002-07-18 | Methods for optimizing and evaluating network access techniques |
Country Status (2)
| Country | Link |
|---|---|
| US (2) | US7035222B2 (en) |
| CA (1) | CA2393919C (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7801158B2 (en) * | 2000-10-16 | 2010-09-21 | Verizon Communications Inc. | Congestion and thru-put visibility and isolation |
| US7035222B2 (en) * | 2001-07-19 | 2006-04-25 | At&T Corp. | Methods for optimizing and evaluating network access techniques |
| US7953424B2 (en) * | 2005-02-22 | 2011-05-31 | Qualcomm Incorporated | Apparatus and method for improving paging performance while simultaneously operating in different types of communication networks |
| US7355981B2 (en) * | 2001-11-23 | 2008-04-08 | Apparent Networks, Inc. | Signature matching methods and apparatus for performing network diagnostics |
| ATE323380T1 (en) * | 2001-12-28 | 2006-04-15 | Koninkl Philips Electronics Nv | FREE ACCESS TO IP VIDEO CONTENT FOR MHP DIGITAL APPLICATIONS |
| US7535847B1 (en) * | 2004-12-30 | 2009-05-19 | Sprint Communications Company Lp | Remote testing for service provider networks |
| US7440407B2 (en) * | 2005-02-07 | 2008-10-21 | At&T Corp. | Method and apparatus for centralized monitoring and analysis of virtual private networks |
| US20070140135A1 (en) * | 2005-12-15 | 2007-06-21 | Bellsouth Intellectual Property Corporation | Methods and systems for providing performance testing for private networks |
| US20070140133A1 (en) * | 2005-12-15 | 2007-06-21 | Bellsouth Intellectual Property Corporation | Methods and systems for providing outage notification for private networks |
| US7856574B2 (en) * | 2007-09-27 | 2010-12-21 | Microsoft Corporation | Internet connectivity evaluation |
| US8023418B2 (en) * | 2009-05-15 | 2011-09-20 | Epitiro Limited | Method and apparatus for testing a broadband internet connection |
| US10334461B2 (en) | 2014-03-07 | 2019-06-25 | Epitiro Holdings, Inc. | System, device and method for testing app performance |
| CN112448860B (en) * | 2019-08-30 | 2023-04-07 | 北京奇艺世纪科技有限公司 | Application program testing method and related equipment |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5303166A (en) * | 1992-04-14 | 1994-04-12 | International Business Machines Corporation | Method and system for automated network benchmark performance analysis |
| US6397359B1 (en) * | 1999-01-19 | 2002-05-28 | Netiq Corporation | Methods, systems and computer program products for scheduled network performance testing |
| US6654347B1 (en) * | 1999-10-22 | 2003-11-25 | Dell Usa L.P. | Site-to-site dynamic virtual local area network |
| GB0028113D0 (en) * | 2000-05-15 | 2001-01-03 | Band X Ltd | Communication system and method |
| US6636585B2 (en) * | 2000-06-26 | 2003-10-21 | Bearingpoint, Inc. | Metrics-related testing of an operational support system (OSS) of an incumbent provider for compliance with a regulatory scheme |
| US20030018513A1 (en) * | 2001-04-13 | 2003-01-23 | Hoffman George Harry | System, method and computer program product for benchmarking in a supply chain management framework |
| US9077760B2 (en) * | 2001-05-22 | 2015-07-07 | Accenture Global Services Limited | Broadband communications |
| US7035222B2 (en) * | 2001-07-19 | 2006-04-25 | At&T Corp. | Methods for optimizing and evaluating network access techniques |
-
2001
- 2001-07-19 US US09/909,123 patent/US7035222B2/en not_active Expired - Fee Related
-
2002
- 2002-07-18 CA CA002393919A patent/CA2393919C/en not_active Expired - Fee Related
-
2006
- 2006-02-17 US US11/357,677 patent/US7558211B2/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| CA2393919A1 (en) | 2003-01-19 |
| US7035222B2 (en) | 2006-04-25 |
| US20060140131A1 (en) | 2006-06-29 |
| US20030218983A1 (en) | 2003-11-27 |
| US7558211B2 (en) | 2009-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7925281B2 (en) | Methods for optimizing and evaluating network access techniques | |
| US7558211B2 (en) | Methods for optimizing and evaluating network access techniques | |
| EP1808000B1 (en) | Method for testing dsl capability of telephone lines | |
| US9215312B2 (en) | Telecommunication diagnostic information management | |
| CN100454852C (en) | Method for network analysis and network analyser module | |
| EP2328333B1 (en) | Method for assessing dsl capability of telephone lines | |
| US20060221942A1 (en) | Intelligent voice network monitoring | |
| WO2006039379A9 (en) | Test system for assessing dsl capability of telephone lines | |
| US20080175228A1 (en) | Proactive quality assessment of voice over IP calls systems | |
| Lewis et al. | Service level management definition, architecture, and research challenges | |
| US7876775B2 (en) | Connection management for data networks | |
| Hofmann et al. | Evaluation of architectures for QoS analysis of applications in Internet environment | |
| Mongi | A conceptual framework for QoE measurement and management in networked systems | |
| Tian | A Framework For Managing Voice Quality In Networks With IP Phones | |
| Hashim et al. | Power consumption in the DSL access network system | |
| Borzemski | Testing, Measuring, and Diagnosing Web Sites from the Users' Perspective | |
| Dischinger | Making broadband access networks transparent to researchers, developers, and users | |
| Yoon et al. | Web-Based Home Gateway Management System using SNMP | |
| Martin | A New Approach to Managing a Best Effort IP WAN Service | |
| Ramanathan et al. | Towards a resilience benchmarking for home gateways | |
| Azran et al. | The Investigation of Perceptual Speech 5G Wireless Communication Networks Check for updates | |
| Telecom et al. | Customer View of Internet Service Performance: Measurement Methodology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |