WO2014105095A2

WO2014105095A2 - Use of device monitoring and network topology data to improve functional verification and screening of cpe devices

Info

Publication number: WO2014105095A2
Application number: PCT/US2013/000284
Authority: WO
Inventors: Foad Towfiq
Original assignee: Promptlink Communications, Inc.
Priority date: 2012-12-28
Filing date: 2013-12-26
Publication date: 2014-07-03
Also published as: WO2014105096A1; WO2014105095A3

Abstract

A customer premise equipment (CPE) device may be functionally verified and screened using information that comes from a network in which the CPE device has been deployed and a device management system that monitored the device when it was operated in the network. Using operational monitoring information for a returned device as well as the network topology and the location of the device in the network, a front-end filtering process can inform device screening. Filtering may select configuration options for the screening test process and the impact of selected tests on answering fault diagnosis questions regarding the returned device.

Description

USE OF DEVICE MONITORING AND NETWORK TOPOLOGY DATA TO IMPROVE

FUNCTIONAL VERIFICATION AND SCREENING OF CPE DEVICES

TECHNICAL FIELD

[0001] This disclosure relates generally to electronic device testing. More

particularly, the disclosure relates to interfaces for electronic device testing. The present application claims priority from U.S. Provisional Patent Application Serial No. 61/848,254 filed on 12/28/2012, entitled "Use Of Device Monitoring And Network Topology Data To Improve Functional Verification And Screening Of CPE Devices Returned From The Field," the entire disclosure of which is herein incorporated by reference. The present application also claims priority from U.S. Provisional Patent Application Serial No. 61/848,255 filed on 12/28/2012, entitled "Method For Video Quality Analysis And Detection of Blockiness, Artifacts and Color Variation for High-Volume Testing of Devices Using Automated Video Testing System," the entire disclosure of which is herein incorporated by reference. The present application also claims priority from U.S. Provisional Patent Application Serial No. 61/848,256 filed on 12/28/2012, entitled "Enhanced Network Management And Customer Premise Equipment (CPE) Monitoring By Using Behavior Measured From A Device

Functional Verification And Screening System (Test Platform) In A Controlled Network Environment," the entire disclosure of which is herein incorporated by reference.

BACKGROUND

[0002] Service providers, including but not limited to communications, connectivity, and content service providers, may provide multimedia services, such as video, audio, telephony, data communications, wireless networking, and related services to their subscribers by deploying equipment or devices at their customers' premises, and then connecting this equipment to the service provider's network and infrastructure. The deployed devices are generally called Customer Premise Equipment (CPE). For example, cable companies, more commonly known as multiple system operators (MSOs), deliver their service to customers by connecting a device, such as a set-top box or a cable modem, to their network. This CPE device may act as the service delivery mechanism for the subscriber.

[0003] The logistics of managing deployment of CPE devices may be a key

, operational business process for some service providers (for example, ISPs, Wi-Fi network providers, MSOs, satellite TV companies, DSL vendors, telecommunications companies, etc.). Providers may select CPE devices based on the technical qualifications of the CPE devices, and providers purchase millions of CPE devices each year to use for delivering their subscription services to their customers. CPE devices may include TV set-top boxes, cable modems, Embedded Multimedia Terminal Adapters (EMTAs), wireless gateways, DSL modems, and/or devices that combine any of these capabilities, and possibly other capabilities, in an integrated package. Large quantities of CPE devices are in use, and the devices are growing more complex as they aim to deliver more valuable services.

[0004] Service providers may screen CPE devices after purchase from

manufacturers and prior to deployment to customers. They may also screen returned devices as they attempt to re-use them. The screening process may include visual inspection and/or functional testing by a screening system or test platform. Upon passing the screening tests, the devices may be repackaged and deployed, usually with the addition of asset management tags and other identification so that the service provider can keep track of their assets and manage the asset life-cycle costs. This asset identification is important, since CPE devices are often returned to the service provider as customers change or cancel their subscriptions, move, or if the customer reports problems with the CPE devices.

[0005] While device deployment is occurring, network operators and network service providers may create the infrastructure to deliver services and operate their business by deploying large networks of cabling, communications equipment, and the devices located at their subscriber's locations. The network is the backbone of the business and may be monitored and managed continuously for the service provider to profit from their work. However, the deployed CPE devices may serve as the point-of-service for service providers to deliver services to their subscribers, and active device monitoring and management may be a part of service provider's day-to-day operation of their network. [0006] Current network management technology seeks to simply monitor the currently deployed network equipment and devices in the network. With such tools, device status polling may be used to generate a composite view of the network status. The polling may use SNMP and MIB methods. This device status polling e.g., of current status, device operation, or performance values) may lead to graphical presentation of the polling results, as the whole and complete "current status" of the network. Service providers may face an operational issue of managing their stock of new and used CPE devices, and to maximize the investment in these CPE devices. Most cable operators own these devices and use them through a life-cycle of acquisition, deployment, operational use, one or more cycles of deployment and use, repair when broken, and re-deployment, and continued use until the end of the device's life.

SUMMARY OF THE DISCLOSURE

[0007] According to the disclosed subject matter, a customer premise equipment

(CPE) device may be functionally verified and screened using information that comes from a network in which the CPE device has been deployed and a device management system that monitored the device when it was operated in the network. Using operational monitoring information for a returned device as well as the network topology and the location of the device in the network, a front-end filtering process can inform device screening. Filtering may select configuration options for the screening test process and the impact of selected tests on answering fault diagnosis questions regarding the returned device.

[0008] A method of screening a customer premise equipment (CPE) device may involve performing a functional verification test of the CPE device using a test platform. A database may be used to receive test data from the functional verification test and operational data from an operational network management system. The operational data may have been collected while the CPE device was operated in a network. A screening result may be determined as a function of the test data and the operational data.

[0009] A system for screening a customer premise equipment (CPE) device may comprise a database comprising a storage device and a computing device comprising a processor and a memory. The memory may store processor-executable instructions for performing a functional verification test of the CPE device using a test platform. The database may receive test data from the functional verification test and operational data from an operational network management system. The operational data may have been collected while the CPE device was operated in a network. A screening result may be determined as a function of the test data and the operational data.

BRIEF DESCRIPTION OF THE DRAWING

[0010] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawing, in which:

[0011] Figure 1 is a diagram illustrating an example shared device database in which a test platform may screen a device and an operational platform may process the device.

DETAILED DESCRIPTION

[0012] The disclosed subject matter is described with specificity to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, it is contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies.

[0013] According to the disclosed subject matter, CPE devices may be tested and/or screened. Service providers, such as cable operators, may benefit by saving time and/or money when processing field-returned CPE devices, such as cable modems and set-top boxes [STBs). Fieldtreturned CPE devices may be returned for any of a variety of reasons, e.g., a subscriber moving, subscriber dissatisfaction with the service, device upgrade, and/or faults or problems with device operation. Regardless of the reason for the device being returned, the service provider has an asset value associated with the device and may seek to preserve that value by reusing the device if possible. Testing a field-returned CPE device with a device screening system, or test platform, in a controlled network

environment, such as a cable network environment, may facilitate reusing the CPE device.

[0014] CPE device screening may be enhanced by supplying decision-making information that may accompany the field-returned device. This information may come from device behavior that may have been monitored and/or recorded while the device has been deployed and operated in the service provider's network. This information may be used to manage the screening test process for returned devices and/or to lower the cost of the related redeployment of tested devices after screening.

[0015] The management of online and/or deployed devices may be enhanced by integrating the results of offline warehouse screening to help resolve operational cable plant network issues, for example, by validating that the returned device contributed to or was responsible for creating a network problem or, when lacking such validation from the screening system, by proving that the returned device was not the cause of subscriber- reported or technician-diagnosed problems that caused the device to be returned.

[0016] CPE device functional verification and screening may be improved by using information that comes from the network and device management system that monitored the device when it was operated in the network, e.g., a cable plant. Through the use of operational cable plant monitoring information for a returned device as well as the network topology and the location of the device in the network, a front-end filtering process can inform device screening. Filtering may select configuration options for the screening test process and the impact of selected tests on answering fault diagnosis questions regarding returned devices. Further, the device screening may^'detect possible network problems that could have been the cause of the reported device failure, rather than any specific device malfunction.

[0017] Information that may be obtained from online network management device polling may be shared with an offline device screening test platform associated with a Returned Material Authorization (RMA) process. The observed device behavior may be applied to the returned device screening process to make it easier to associate specific devices with reports of network noise or other operational problems. Screening tests may be performed that may help determine whether or not the device is independently responsible for an operational network issue. An RMA result may be determined. For example, if the device is independently responsible for the operational network issue, the device may be repaired or replaced or, if appropriate, discarded. If not, the device may be assigned for return to service. [0018] Software logic may correlate the device screening test process with measured network level characteristics, such as, for example, power levels or signal-to- noise ratios (SNRs) associated with the device when it was deployed in the network. This software logic may combine recent operations data with the device screening process of the test platform. The software logic may identify repeat offenders, e.g., devices that are repeatedly deployed and returned, and may relate historical test data and historical operational operations data. This may lead to better decision-making for the screening process.

[0019] Device screening tests may be enhanced through the use of operational device behavior data collected when the devices are deployed in a network. For example, better performance, more accurate results, the ability to use test results to correlate to other costly operational issues, and/or overall lower costs and higher asset value may be realized. CPE device screening may be improved by using information that comes from the network and a device management system that monitors the device when it is operated in a network, such as a cable plant. A history of device status polling information may be shared with device screening test data. Through the use of operational cable plant monitoring information for a returned device, a front-end filtering process can inform device screening. Filtering may select configuration options for the screening test process and the impact of selected tests on answering fault diagnosis questions regarding returned devices. Correlation of network and device management information with device screening test data may allow screening results to better choose the proper disposition of returned devices, e.g., to pass and be assigned for redeployment, or to fail and be assigned for repair or return to the original manufacturer for replacement or discard.

[0020] Information may be migrated from a device polling function of network management into a database that is shared with a device screening test process. Data that is migrated to the shared database may be used to identify faulty devices based on captured polling data and/or customer-reported claims of problems. The shared database may move the information out of the real-time network management system and into a position for making decisions about the screening test process. This kind of information may enable the RMA process to evaluate more subtle device performance characteristics, such as the qualitative aspects of the device's impact on network performance for sensitive network applications, such as Voice-over-Cable or streaming data applications, or for root cause analysis of long-term subtle operational problems in the cable plant, e.g., related to noise injected into an HFC plant that causes intermittent outages in various segments of the cable plant that may be difficult to locate.

[0021] Due to the complex interaction of devices in a network, it may be difficult to monitor and manage dynamic network properties or characteristics. CPE devices may be returned because the customer may claim a problem that may exist with their service and may assume that the problem is attributable to the device deployed to their premises. The observed device behavior may be applied to the returned device screening process to make it easier to associate specific devices with reports of network noise or other operational problems and to perform screening tests that may help determine whether or not the device is independently responsible for the operational network issues. If so, the device may be repaired or replaced. If not, the device may be assigned for return to service.

[0022] Software logic may correlate the device screening test process with measured network level characteristics, such as power levels or SNRs, associated with the device when it was deployed in the network. The software logic may be associated with a front-end screening process that may retrieve the shared data and may select actions to take and/or specific screening tests to perform on a returned device. The software logic may combine recent operations data with the device screening process of the test platform, for example, examining signal-to-noise values from operational polling and comparing those values to active test results from the test platform. The software logic may identify repeat offenders, e.g., devices that are repeatedly deployed and returned, and may relate historical test data and historical operations data. This may lead to better decision-making for the screening process. When this repeat offender designation is combined with operational and/or test polling data, the software logic may have a more complete picture of the device, enabling an improved choice of the proper disposition of returned devices.

[0023] An operational network management system may be used to establish baseline data about CPE or deployed devices that have been returned for RMA processing and/or screening. Device screening may be conducted offline, for example, at a warehouse or other facility that may be separate from the operational network. Integration of the operational network management system baseline data may inform the screening process with background and context as to why the device was returned and/or how to test the device to obtain accurate screening results.

[0024] Baseline device data, e.g., from the operational network management system, can be synthesized with the tests of the screening process. This synthesis of historical and current data may provide more information and more valuable information for use in troubleshooting and diagnosing dynamic problems that may be causing the problems associated with a returned device. This synthesis may be performed for specific devices that have been screened and then subsequently deployed or redeployed into the

operational network. In effect, each such device may become a point of enhanced

monitoring or a focal point for viewing network performance and network operations health and/or status.

[0025] Device polling may be used to read and present the operational status of the cable plant. Polled CPM data may be stored and shared with a cable operator's warehouse organization so as to inform the warehouse with historical data about the device, a reported fault with the device, the subscriber, the neighborhood, the segment of the HFC plant to which the device connected, and/or the like. Historic data may be provided so that filtering of devices can occur to place emphasis on, for example, focused testing linked to failure data of the device, direct RMA processing for devices under warranty, exploratory testing for the device to attempt to correlate its tested behavior with other network elements and observed performance characteristics, and/or nominal testing, screening and reentry into the pool of devices for redeployment, for example, if the device is returned without any specified fault or subscriber issue, e.g., the subscriber dropped the service because they are moving.

[0026] A software and testing platform may supply repeatable high-volume testing of cable devices. Such software may run operational tests without preprocessing to determine which tests to run or why to run such tests. Front end filtering may be performed so that testing performed by the software and testing platform system may help answer questions about, for example, possible health and/or performance issues in an operational cable plant and/or subtle performance characteristics of the device that may adversely affect other network elements. These characteristics may include, for example, high SNR values that may indicate that a specific device may not work effectively with other connected devices in a cable plant.

[0027] Referring now to the drawing, Figure 1 illustrates an example system 100 including a database 102. The database 102 may be shared between a test platform 104 and an operational network management system 106. The operational network

management system 106 may be connected to an operational network 108, for example, via a communication network such as the Internet.

[0028] The database 102 may be implemented using any storage technology, including but not limited to magnetic and/or optical storage, and may store device data. The device data may include device data from the screening process that may represent operational parameters for devices in the controlled environment of the test platform 104. The device data may also include network topology information, which may represent operational device parameters in the live networks. The operational network 108 may comprise a collection of network devices deployed at subscriber and/or customer locations, also known as customer premises. Customer Premise Equipment (CPE) may comprise network equipment that resides at the customer location to present the operator or service provider's service to its customers.

[0029] The system 100 may comprise a computing device, which may be integral with the database 102, the test platform 104, and/or the operational network management system 106, or may be implemented as a standalone computing device 110. The computing device may comprise a processor and a memory and may be configured to perform the functional verification process, the screening process, or other processes disclosed herein via processor-executable instructions.

[0030] As demonstrated by the foregoing discussion, the disclosed subject matter may provide certain advantages, particularly in the context of functional verification and screening of CPE devices. By using operational cable plant monitoring information for a returned device in combination with the network topology and the device in the network, systems, methods, and instrumentalities embodying the disclosed subject matter may inform device screening. Filtering may be used to select configuration options for the screening test process, as well as the impact of selected tests on answering fault diagnosis questions regarding the returned devices. Device screening can also identify network problems that may have caused reported device failures that may have been erroneously attributed to device malfunctions.

[0031] It will be understood by those who practice the disclosed subject matter and those skilled in the art that various modifications and improvements may be made without departing from the spirit and scope of the disclosed subject matter. The scope of protection afforded is to be determined solely by the claims and by the breadth of interpretation allowed by law.

Claims

Claims What is claimed is:

1. A method of screening a customer premise equipment (CPE) device, the method comprising:

performing a functional verification test of the CPE device using a test platform; receiving, in a database, test data from the functional verification test;

receiving, in the database, operational data from an operational network

management system, the operational data collected while the CPE device was operated in a network; and

determining a screening result as a function of the test data and the operational data.

2. The method of claim 1, further comprising:

receiving, in the database, network topology information for the network in which the CPE device has been deployed; and

determining the screening result as a function of the network topology information.

3. The method of claim 1, further comprising:

receiving, in the database, device location information identifying a location of the device relative to the network in which the CPE device has been deployed; and

determining the screening result as a function of the device location information.

4. The method of claim 1, further comprising determining, as a function of the screening result, whether the CPE device is responsible for a network operational issue.

5. The method of claim 1, further comprising determining, as a function of the screening result, a Returned Material Authorization (RMA) result.

6. The method of claim 5, wherein the RMA result is selected from the group consisting of return to service, repair, warranty action, or discard.

7. A system for screening a customer premise equipment (CPE) device, the system comprising:

a database comprising a storage device; and

a computing device comprising a processor and a memory storing processor- executable instructions for:

performing a functional verification test of the CPE device using a test platform;

receiving, in the database, test data from the functional verification test; receiving, in the database, operational data from an operational network management system, the operational data collected while the CPE device was operated in a network; and

8. The system of claim 7, wherein the memory stores further processor-executable instructions for:

9. The system of claim 7, wherein the memory stores further processor-executable instructions for:

10. The system of claim 7, wherein the memory stores further processor-executable instructions for determining, as a function of the screening result, whether the CPE device is responsible for a network operational issue.

11. The system of claim 7, wherein the memory stores further processor-executable instructions for determining, as a function of the screening result, a Returned Material Authorization (RMA] result.

12. The system of claim 11, wherein the RMA result is selected from the group consisting of return to service, repair, warranty action, or discard.

13. The system of claim 7, wherein the computing device comprises a test platform.

14. The system of claim 7, wherein the computing device comprises the database.

15. The system of claim 7, wherein the computing device comprises an operational network management system configured to collect the operational data while the CPE device was operated in the network.