Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberCN103297285 A
Publication typeApplication
Application numberCN 201210043849
Publication date11 Sep 2013
Filing date23 Feb 2012
Priority date23 Feb 2012
Also published asCN103297285B
Publication number201210043849.8, CN 103297285 A, CN 103297285A, CN 201210043849, CN-A-103297285, CN103297285 A, CN103297285A, CN201210043849, CN201210043849.8
Inventors金敬亭, 胡星, 乔敏
Applicant百度在线网络技术(北京)有限公司
Export CitationBiBTeX, EndNote, RefMan
External Links: SIPO, Espacenet
Distributed cluster performance test system, method and device
CN 103297285 A
Abstract
The invention provides a distributed cluster performance test system, method and device. The system comprises an upstream cluster, a tested module, a back end cluster, a network bridge, a pressure tool and a monitoring module. The network bridge is used for receiving a first pressure request sent by the upstream cluster, sending the first pressure request to the tested module, receiving a second pressure request generated by the tested module according to the first pressure request, forwarding the second pressure request to the back end cluster and storing the first pressure request and communication data packs returned by the back end cluster according to the second pressure request. The pressure tool is used for sending preset pressure to the tested module according to the first pressure request, and the tested module forwards the preset pressure to the network bridge, so that corresponding packs in the stored communication data packs are checked and are returned to tested module. The monitoring module and the tested module are placed on the same server. The monitoring module is used for monitoring performance data of the server and carrying out performance analysis according to the performance data. The network bridge is used for simulating a real cluster, performance test accuracy and test efficiency are improved, and cost is saved.
Claims(17)  translated from Chinese
1.一种分布式集群的性能测试系统,其特征在于,包括: 上游集群,所述上游集群用于发送第一压力请求; 被测模块; 后端集群,所述后端集群包括多个下游集群; 网桥,所述网桥分别与所述上游集群、被测模块和后端集群相连,所述网桥用于接收所述第一压力请求并将所述第一压力请求发送至所述被测模块,接收所述被测模块根据所述第一压力请求生成的第二压力请求并将所述第二压力请求转发至所述多个下游集群,以及保存所述第一压力请求以及所述多个下游集群根据所述第二压力请求返回的通信数据包; 压力工具,所述压力工具用于根据所述网桥保存的所述第一压力请求向所述被测模块发送预定压力,其中所述被测模块将所述预定压力转发至所述网桥,所述网桥根据所述预定压力在所述保存的返回的通信数据包中查找所述预定压力相应的数据包并返回至所述被测模块;以及监控模块,所述监控模块与所述被测模块位于同一个服务器上,用于监控所述服务器的性能数据,并根据所述性能数据进行性能分析。 A distributed cluster of performance test system, characterized by comprising: an upstream cluster, the cluster is used to send a first pressure upstream request; test module; rear end of the cluster, the cluster includes a plurality of downstream rear end clusters; bridge, said bridge with said upstream cluster, the cluster is connected to the test module and a rear end, said bridge means for receiving said first pressure and said first pressure request to the request the test module, said test modules second pressure receiving requests generated based on said first pressure and said second pressure request is forwarded to the requesting downstream plurality of clusters, and saving the request as well as a first pressure downstream of said plurality of cluster communication packets based on the second request to return the pressure; pressure tool, the tool is used in accordance with the pressure of the stored first bridge request a predetermined pressure to the pressure of the test module, wherein said test module forwarding said predetermined pressure to the bridge, the bridge to find the data packets corresponding to a predetermined pressure and returns to the communication of the data packet stored in accordance with the predetermined return pressure The test module; and a control module, the monitor module and the test modules are located on the same server, the server for performance data monitoring and performance analysis based on the performance data.
2.如权利要求1所述的分布式集群的性能测试系统,其特征在于,所述通信数据包包括: 魔鬼数字,用于标识协议包头; Dump ID,用于标识所保存的通信数据包的编号; key标识,用于标识压力请求及其对应的返回数据包; 版本号,用于区别不同的被测模块; 保留字段,用于扩展所述通信数据包; 包长度,用于标识所述通信数据包的实际长度;和Dump的数据包体,用于存放所述通信数据包。 2. The test system performance distributed cluster of claim 1, wherein said communication packet includes: Devil number that identifies the protocol header; Dump ID, for identifying the stored communication packets Number; key identifier is used to identify the corresponding pressure request and return packets; the version number, used to distinguish the different test modules; a reserved field for extending the communication packets; packet length, for identifying The actual length of said communication packet; and Dump packet body for the communication data packet storage.
3.如权利要求2所述的分布式集群的性能测试系统,其特征在于,所述网桥按照key标识&value对的形式建立词典以保存所述第一压力请求和所述返回的通信数据包,其中value为所述返回的通信数据包,key标识为所述第一压力请求包中的key标识。 3. distributed cluster 2, wherein the performance test system of claim wherein the bridge set up in accordance with the dictionary identifies key & value pairs to save the first pressure request and the return of communication packets , where value is the return of communication packets, key pressure identified as the first request packet key identifier.
4.如权利要求1所述的分布式集群的性能测试系统,其特征在于,所述监控模块具体包括: 总控单元,所述总控单元用于配置监控项、控制监控的启动和停止以及根据所述性能数据进行性能分析;以及多个分控单元,每个分控单元均与所述总控单元相连,用于根据所述总控单元的控制周期性地收集所述被测模块所在的服务器的性能数据,并对所述性能数据进行格式化。 The distributed cluster of performance test system as claimed in claim 1, characterized in that the monitoring module specifically includes: Master Control Unit, the master control unit for configuration monitoring item, control and stop and start monitoring performance analysis based on the performance data; and a plurality of sub-control units, each sub-control units are connected to the master control unit for collecting the periodically based on the control of the master control unit test module resides Performance data of the server, the performance and data formatting.
5.如权利要求1或4所述的分布式集群的性能测试系统,其特征在于,所述性能数据包括: vmstat 数据; CPU、内存和文件句柄/socket句柄等占用情况; iostat 数据; 被测模块运行的日志;以及网络数据。 5. distributed cluster performance test system or as claimed in claim 4, wherein the performance data includes: vmstat data; CPU, memory and file handles / socket handles, etc. occupancy; iostat data; test log module running; and network data.
6.如权利要求4所述的分布式集群的性能测试系统,其特征在于,所述总控单元具体包括: 建立所述总控单元与各个所述分控单元之间的双向信任关系; 将监控脚本部署到各个所述分控单元上; 批量启动或停止所述多个分控单元; 获取各个所述分控单元的机器物理参数;以及根据所格式化后的性能数据计算其在不同时间段内的平均值、最小值、最大值以及某一列数据在不同区间的比例。 6. A distributed cluster 4, wherein the performance test system of claim wherein the master control unit including: a two-way trust relationship with the master control unit between each of the sub-control unit; and will monitor script deployed to each of the points on the control unit; batch start or stop of the plurality of sub-control unit; obtaining each of the sub-control unit of the machine physical parameters; and calculated performance data after formatting it at different times segment mean, minimum, maximum, and the proportion of a column of data in different intervals.
7.如权利要求4所述的分布式集群的性能测试系统,其特征在于,所述监控模块还包括: 日志处理单元,所述日志处理单元与所述总控单元相连,用于对所述性能数据进行解析和统计。 7. A distributed cluster 4, wherein the performance test system as claimed in claim wherein said monitoring module further comprises: a log processing unit, a log processing unit is connected with the master control unit, for the analytical and statistical performance data.
8.一种网桥,其特征在于,包括: 模式选择模块,所述模式选择模块用于接收用户输入的录制模式或测试模式; 通信模块,所述通信模块用于在所述录制模式下接收上游集群发送的第一压力请求并将所述第一压力请求发送至被测`模块,以及接收所述被测模块根据所述第一压力请求生成的第二压力请求并将所述第二压力请求转发至后端集群,以及接收所述后端集群根据所述第二压力请求返回的通信数据包;所述通信模块还用于在所述测试模式下接收所述被测模块转发的预定压力; 录制模块,所述录制模块与所述通信模块相连,用于保存所述第一压力请求和所述返回的通信数据包;以及模拟模块,所述模拟模块分别与所述通信模块和所述录制模块相连,用于根据所述预定的压力在所述录制模块保存的通信数据包中查找所述预定的压力对应的返回数据包,其中所述通信模块还用于将所述预定压力对应的返回数据包发送至所述被测模块。 A bridge, characterized in that, comprising: a mode selection module, the mode selection module for receiving a user input recording mode or the test mode; communication module, a communication module for receiving in said recording mode a first pressure upstream of the cluster and a request to send the request to the first pressure to a second pressure generating test `transmission module, and the module receiving the request according to the measured first pressure and said second pressure request forwards the request to the rear end of the cluster, and the receiving communication packets returned according to the request of the rear end of the second cluster pressure; the communications module is further configured to receive a predetermined pressure in the test mode the test module forwarding ; recording module, connected to the recording module and the communication module for storing the first request and the pressure of the return communication data packet; and a simulation module, the analog module with said communication module and said recording module is connected, for finding the predetermined pressure in accordance with said predetermined pressure in said recording module stored communication packets corresponding return data packets, wherein the communication module is further configured to correspond to the predetermined pressure return packet to the test module.
9.如权利要求8所述的网桥,其特征在于,所述通信数据包包括: 魔鬼数字,用于标识协议包头; Dump ID,用于标识所保存的通信数据包的编号; key标识,用于标识压力请求及其对应的返回数据包; 版本号,用于区别不同的被测模块; 保留字段,用于扩展所述通信数据包; 包长度,用于标识所述通信数据包的实际长度;和Dump的数据包体,用于存放所述通信数据包。 Dump ID, for identifying the communication packet stored number;; devil numbers, protocol header that identifies key identifier: 9. The bridge as claimed in claim 8 wherein, wherein the communication packets comprises Pressure is used to identify the request and returns the corresponding data packet; actual packet length, data for identifying the communication packet; version number for distinguishing the different test modules; reserved field for extending the communication packet length; and Dump packet body for the communication data packet storage.
10.如权利要求9所述的网桥,其特征在于,所述模拟模块还用于按照key标识&value对的形式建立词典以保存所述第一压力请求和所述返回的通信数据包,其中value为所述返回的通信数据包,key标识为所述第一压力请求包中的key标识。 10. A bridge according to claim 9, characterized in that said analog module is further configured to establish key identification dictionary in accordance with the form of the & value of the request to hold the first pressure and the return packet communication, wherein the value of communication packets returned, key identified as the first pressure request packet key identifier.
11.一种分布式集群的性能测试方法,其特征在于,包括以下步骤: (a)在录制模式下,网桥接收上游集群发送的第一压力请求,并将所述第一压力请求发送至被测模块,以及保存所述第一压力请求;(b)所述被测模块接收所述第一压力请求,并根据所述第一压力请求生成第二压力请求,以及将所述第二压力请求发送至所述网桥; (c)所述网桥接收所述第二压力请求并将所述第二压力请求发送至后端集群,其中所述后端集群包括多个下游集群; (d)所述多个下游集群根据所述第二压力请求返回相应的数据包至所述网桥; (e)所述网桥接收所述并保存所述返回的数据包; (f)在测试模式下,通过压力工具根据所述网桥保存的第一压力请求发送预定压力至所述被测模块; (g)所述被测模块将所述预定压力转发至所述网桥,并接收所述网桥根据所述预定压力在所述保存的通信数据包中查找到的相应的返回数据包; (h)在测试过程中,监控所述被测模块所在的服务器的性能数据,并根据所述性能数据进行性能分析。 11. A method for distributed cluster performance test, characterized in that it comprises the following steps: (a) in the recording mode, the bridge receives a first pressure upstream of the cluster to send a request and the request is sent to the first pressure the test module, and saving the first pressure request; (b) the test module receives the request for the first pressure and a second pressure generating Request according to the first pressure and the second pressure request to the bridge; (c) the bridge request and the second pressure receiving said second pressure request is sent to the rear end of the cluster, wherein the rear end of the cluster includes a plurality of downstream clusters; (d ) downstream of said plurality of clusters according to the second pressure request to return the corresponding data packet to the bridge; (e) the bridge receives the returned data and storing the packet; (f) in the test mode under pressure tool by transmitting a predetermined pressure to the pressure of the test module according to the first bridge to save the request; (g) of the test module forwarding the predetermined pressure to the bridge, and receiving the Find bridge the communication packets stored in accordance with the predetermined pressure to the corresponding return packets; (h) during testing, monitoring performance data server where the test module, and according to the Performance data for performance analysis.
12.如权利要求11所述的分布式集群的性能测试方法,其特征在于,所述通信数据包包括: 魔鬼数字,用于标识协议包头; Dump ID,用于标识所保存的通信数据包的编号; key标识,用于标识压力请求及其对应的返回数据包; 版本号,用于区别不同的被测模块; 保留字段,用于扩展所述通信数据包; 包长度,用于标识所述通信数据包的实际长度;和Dump的数据包体,用于存放所述通信数据包。 Dump ID, for identifying the stored communication packets; devil numbers, used to identify the protocol header: 12. The performance test method according to Claim distributed cluster, wherein said communication packet comprises Number; key identifier is used to identify the pressure request and the corresponding return packets; the version number, used to distinguish the different test modules; a reserved field for extending the communication packets; packet length, for identifying the The actual length of the communication packets; and Dump packet body for the communication data packet storage.
13.如权利要求12所述的分布式集群的性能测试方法,其特征在于,还包括步骤: 所述网桥按照key标识&value对的形式建立词典以保存所述第一压力请求和所述返回的通信数据包,其中value为所述返回的通信数据包,key标识为所述第一压力请求包中的key标识。 13. Methods for measuring performance distributed cluster of claim 12, wherein, characterized by further comprising the step of: establishing a bridge dictionary identifies key & value in accordance with the form of pressure to hold the first request and the return communication packets, in which the value of communication packets returned, key is identified as the first pressure request packet key identifier.
14.如权利要求11所述的分布式集群的性能测试方法,其特征在于,所述步骤(h)通过通用监控工具实现,其中所述通用监控工具包括: 总控单元,所述总控单元用于配置监控项、控制监控的启动和停止以及根据所述性能数据进行性能分析;以及多个分控单元,每个分控单元均与所述总控单元相连,用于根据所述总控单元的控制周期性地收集所述被测模块所在的服务器的性能数据,并对所述性能数据进行格式化。 14. The method of performance testing distributed cluster of claim 11, characterized in that said step (h) by monitoring tools to achieve universal, wherein said common monitoring tool comprising: master control unit, the master control unit for configuration monitoring item, control and monitoring of start and stop performance analysis based on the performance data; and a plurality of sub-control units, each sub-control units are connected to the master control unit for the total control The control unit periodically collect data on the performance of the tested server module is located, and the performance data is formatted.
15.如权利要求14所述的分布式集群的性能测试方法,其特征在于,所述性能数据包括: vmstat 数据; CPU、内存和文件句柄的占用情况; iostat 数据; 被测模块的运行日志;以及网络数据。 Iostat data;; running log of the test module vmstat data; CPU, memory and file handles occupancy;: 15. The method of claim 14, wherein the performance testing requirements of distributed clusters, wherein the performance data includes and network data.
16.如权利要求14所述的分布式集群的性能测试方法,其特征在于,所述总控单元具体用于: 建立所述总控单元与各个所述分控单元之间的双向信任关系; 将监控脚本部署到各个所述分控单元上; 批量启动或停止所述多个分控单元; 获取各个所述分控单元的机器物理参数;以及根据所格式化后的性能数据计算其在不同时间段内的平均值、最小值、最大值以及某一列数据在不同区间的比例。 16. The method according to performance tests distributed cluster of claim wherein the master control unit is configured to: establish a two-way trust relationship between the master control unit and each of the sub-control unit between; Deploying monitor script to each of the points on the control unit; batch start or stop of the plurality of sub-control unit; obtaining each of the sub-control unit of the machine physical parameters; and calculated performance data after formatting it in different the average period of time, minimum, maximum, and the proportion of a column of data in different intervals.
17.如权利要求14所述的分布式集群的性能测试方法,其特征在于,所述通用监控工具还包括: 日志处理单元, 所述日志处理单元与所述总控单元相连,用于对所述性能数据进行解析和统计。 17. Methods for measuring performance distributed cluster of claim 14, wherein wherein said generic monitoring tools include: log processing unit, the log processing unit connected to the master control unit for the said performance data parsing and statistics.
Description  translated from Chinese

分布式集群的性能测试系统、方法和装置 Distributed cluster performance test system, method and apparatus

技术领域 Technical Field

[0001] 本发明涉及网络搜索技术领域,尤其涉及一种分布式集群的性能测试系统、方法和装置。 [0001] The present invention relates to the field of Web search technology, particularly to a distributed cluster of performance test systems, methods and devices.

背景技术 Background

[0002] 对于类似网络搜索引擎的分布式集群系统,现有的性能测试在真实环境中进行,需要将大量机器资源搭建后端集群环境。 [0002] For a similar network search engines distributed cluster system, the existing performance testing in a real environment, requiring a large amount of machine resources to build the back-end cluster environment. 这种方法存在的问题是:(1)浪费机器资源,人力维护成本高;(2)线下无法搭建和线上一样数据量规模的环境,数据量和线上相差很远; Problems with this approach are: (1) a waste of machine resources, human maintenance costs; (2) line can not build the same amount of data size and online environments, the amount of data and on-line very far;

[3]线下搭建的后端集群的网络拓扑和线上不一致。 [3] The rear end of the line to build a cluster of network topology and the line is inconsistent.

[0003] 而且,现有技术中需要根据不同的被测模块,准备不同的压力输入源、压力工具和监控工具,通用性差,需要较多的开发和维护成本。 [0003] Moreover, the prior art test modules depending on the needs, prepare different pressure input source, pressure tools and monitoring tools, general poor, need more development and maintenance costs.

发明内容 DISCLOSURE

[0004] 本发明旨在至少解决上述技术问题之一。 [0004] The present invention aims to solve at least one of the above-mentioned technical problems.

[0005] 为此,本发明的第一个目的在于提出一种可以提高测试效率和测试性能准确性、通用性好、成本低的分布式集群的性能测试系统。 [0005] To this end, a first object of the present invention is to provide a way to improve test efficiency and test the accuracy of performance, versatility and low cost of distributed cluster performance test systems.

[0006] 本发明的第二个目的在于提出一种网桥。 [0006] A second object of the present invention is to propose a bridge.

[0007] 本发明的第三个目的在于提出一种分布式集群的性能测试方法。 [0007] A third object of the present invention to provide a method for testing the performance of distributed clusters.

[0008] 为了实现上述目的,根据本发明的第一方面实施例的分布式集群的性能测试系统包括:上游集群,所述上游集群用于发送第一压力请求;被测模块;后端集群,所述后端集群包括多个下游集群;网桥,所述网桥分别与所述上游集群、被测模块和后端集群相连,所述网桥用于接收所述第一压力请求并将所述第一压力请求发送至所述被测模块,接收所述被测模块根据所述第一压力请求生成的第二压力请求并将所述第二压力请求转发至所述多个下游集群,以及保存所述第一压力请求以及所述多个下游集群根据所述第二压力请求返回的通信数据包;压力工具,所述压力工具用于根据所述网桥保存的所述第一压力请求向所述被测模块发送预定压力,其中所述被测模块将所述预定压力转发至所述网桥,所述网桥根据所述预定压力在所述保存的通信数据包中查找所述预定压力相应的数据包并返回至所述被测模块;以及监控模块,所述监控模块与所述被测模块位于同一个服务器上,用于监控所述服务器的性能数据,并根据所述性能数据进行性能分析。 Distributed cluster system performance test [0008] To achieve the above object, according to a first aspect of the present embodiment of the invention comprises: an upstream cluster, the cluster is used to send a first pressure upstream request; test module; the rear end of the cluster, The rear end of the cluster includes a plurality of downstream clusters; bridge, said bridge with said upstream cluster, the cluster is connected to the test module and a rear end, said bridge for receiving the request and the first pressure said first request to said measured pressure module, the second pressure receiving test request generation module according to the first pressure request and forwards the request to the second pressure downstream of the plurality of clusters, and save the requests and the plurality of first pressure downstream cluster communication request packet is returned from the second pressure; pressure tool, said pressure tools for requesting to save the bridge according to the first pressure The test module sends a predetermined pressure, wherein said test module forwarding said predetermined pressure to the bridge, the bridge to find the predetermined pressure is stored in the communication data packets in accordance with said predetermined pressure the corresponding data packet and return to the test modules; and monitoring module, the monitor module and the test modules are located on the same server for performance data to monitor the server and based on the performance data performance analysis.

[0009] 根据本发明实施例的分布式集群的性能测试系统,网桥分别与上游集群、被测模块和多个下游集群(后端集群)相连,通过一个网桥可以真实模拟线上拓扑和数据量,无需搭建真实的集群环境即可对被测模块的性能进行测试,提高测试准确性和测试效率,节约成本。 [0009] distributed cluster performance test system embodiment of the present invention, respectively upstream bridge cluster, test module and a plurality of downstream clusters (back-end clusters) connected by a bridge topology and can simulate online the amount of data without a clustered environment to build real performance test module for testing, improve test accuracy and test efficiency and cost savings.

[0010] 为了实现上述目的,根据本发明的第二方面实施例的网桥包括:模式选择模块,所述模式选择模块用于接收用户输入的录制模式或测试模式;通信模块,所述通信模块用于在所述录制模式下接收上游集群发送的第一压力请求并将所述第一压力请求发送至被测模块,以及接收所述被测模块根据所述第一压力请求生成的第二压力请求并将所述第二压力请求转发至后端集群,以及接收所述后端集群根据所述第二压力请求返回的通信数据包;所述通信模块还用于在所述测试模式下接收所述被测模块转发的预定压力;录制模块,所述录制模块与所述通信模块相连,用于保存所述第一压力请求和所述返回的通信数据包;以及模拟模块,所述模拟模块分别与所述通信模块和所述录制模块相连,用于根据所述预定的压力在所述录制模块保存的通信数据包中查找所述预定的压力对应的返回数据包,其中所述通信模块还用于将所述预定压力对应的返回数据包发送至所述被测模块。 Bridge [0010] To achieve the above object, according to a second aspect of the present embodiment of the invention comprises: mode selection module, the mode selection module for receiving a user input recording mode or the test mode; communication module, the communication module means for receiving a first pressure upstream of the cluster transmission request in the recording mode and the first pressure request to the module under test, and a second pressure receiving said request generating test module according to the first pressure request and forwards the request to the second backend cluster pressure, and receiving a communication packet returned in accordance with the rear end of said second pressure request cluster; the communications module further for receiving the test mode said predetermined pressure test module forwarding; recording module, the recording module and the communication module is connected to the first pressure request for storing and communicating data packets of the return; and analog modules, the analog modules are connected to the communication module and the recording module, for finding the predetermined pressure in accordance with said predetermined pressure in said recording module stored communication packets corresponding return packet, wherein said communication module further use at the predetermined pressure corresponding return packet to the test module.

[0011] 根据本发明实施例的网桥,通过与上游集群、被测模块和后端集群相连,可以真实地模拟线上拓扑和数量,无需搭建真实的集群环境即可对被测模块的性能进行测试,提高测试准确性和测试效率,节约成本。 [0011] According to the measured performance of the module bridge example embodiment of the invention, through, and back-end cluster test module is connected to the upper reaches of the cluster, can realistically simulate line topology and the number of cluster environments without the need to build a real test, improve test accuracy and test efficiency and cost savings.

[0012] 为了实现上述目的,根据本发明的第三方面实施例的分布式集群的性能测试方法,包括以下步骤:(a)在录制模式下,网桥接收上游集群发送的第一压力请求,并将所述第一压力请求发送至被测模块,以及保存所述第一压力请求;(b)所述被测模块接收所述第一压力请求,并根据所述第一压力请求生成第二压力请求,以及将所述第二压力请求发送至所述网桥;(c)所述网桥接收所述第二压力请求并将所述第二压力请求发送至后端集群,其中所述后端集群包括多个下游集群;(d)所述多个下游集群根据所述第二压力请求返回相应的数据包至所述网桥;(e)所述网桥接收所述并保存所述返回的数据包;(f)在测试模式下,通过压力工具根据所述网桥保存的第一压力请求发送预定压力至所述被测模块;(g)所述被测模块将所述预定压力转发至所述网桥,并接收所述网桥根据所述预定压力在所述保存的通信数据包中查找到的相应的数据包;(h)在测试过程中,监控所述被测模块所在的服务器的性能数据,并根据所述性能数据进行性能分析。 [0012] To achieve the above object, according to a third aspect of the present invention, the performance of the test method according to an embodiment of distributed clusters, comprising the steps of: (a) in the recording mode, a first pressure upstream of the bridge cluster send the request is received, and the first request to the measured pressure module, and saving the first pressure request; (b) the test module receives the request for the first pressure and the first pressure is generated in accordance with the request a second pressure request, and the request is sent to the second pressure bridge; (c) the bridge request and the second pressure receiving said second pressure request is sent to the rear end of the cluster, wherein the rear end cluster includes a plurality of downstream clusters; (d) the request to return more downstream clusters corresponding data packet to the bridge according to the second pressure; (e) the bridge receives the return and save the packets; (f) in the test mode, transmitting a predetermined pressure through a pressure tool to the test module according to the first pressure of the bridge to save the request; (g) of the test module forwarding the predetermined pressure to the bridge, and the bridge looks in the receiving communication packets stored according to said predetermined pressure corresponding data packet; (h) during testing, monitoring said test module is located server performance data and performance analysis based on the performance data.

[0013] 根据本发明实施例的分布式集群的性能测试方法,通过网桥抓取上游集群、被测模块以及后端集群之间的通信数据包,从而通过一个网桥可以真实模拟线上拓扑和数据量,无需搭建真实的集群环境即可对被测模块进行性能测试,提高测试准确性和测试效率,节约成本。 [0013] The method according to performance tests distributed cluster of embodiments of the present invention, crawling through the bridge upstream cluster communication packets between the test module and the rear end of the cluster, which can simulate through a bridge line topology and the amount of data without the need to build a real cluster environment allows the test module performance testing, improve test accuracy and test efficiency and cost savings.

[0014] 本发明附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本发明的实践了解到。 Additional aspects and advantages of the invention [0014] This part is given in the following description, the following description will become apparent in part, or learned by practice of the invention.

附图说明 Brief Description

[0015] 本发明上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中, [0015] The present invention described above and / or additional aspects and advantages of the drawings from the following description of embodiments will become apparent and readily appreciated,

[0016] 图1是根据本发明一个实施例的分布式集群的性能测试系统的录制模式结构框图; [0016] FIG. 1 is a block diagram of one embodiment according to the recording mode of the present invention distributed cluster performance test systems;

[0017] 图2是根据本发明一个实施例的分布式集群的性能测试系统的测试模式的结构框图; [0017] FIG. 2 is a block diagram in accordance with one embodiment of the invention a distributed cluster of Performance Testing System test pattern;

[0018] 图3是根据本发明一个实施例的监控模块的结构框图; [0018] FIG. 3 is a block diagram of one embodiment of a monitoring module of the present invention;

[0019] 图4是根据本发明另一个实施例的监控模块的结构框图; [0019] FIG. 4 is a block diagram of another embodiment according to the present invention, an example of the monitoring module;

[0020] 图5是根据本发明一个实施例的网桥的结构框图;以及 [0020] FIG. 5 is a block diagram of a bridge embodiment of the present invention; and

[0021] 图6是根据本发明一个实施例的分布式集群的性能测试方法的流程图。 [0021] FIG. 6 is a flowchart of a distributed cluster performance test method according to an embodiment of the present invention. 具体实施方式 DETAILED DESCRIPTION

[0022] 下面详细描述本发明的实施例,所述实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。 [0022] Embodiments of the present invention will be described in detail below, an exemplary embodiment of the embodiment shown in the accompanying drawings, throughout which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. 下面通过参考附图描述的实施例是示例性的,仅用于解释本发明,而不能理解为对本发明的限制。 The following examples with reference to the accompanying drawings are exemplary for explaining the present invention only, and not be construed as limiting the present invention. 相反,本发明的实施例包括落入所附加权利要求书的精神和内涵范围内的所有变化、修改和等同物。 In contrast, embodiments of the present invention includes all changes that fall within the appended claims the spirit and content scope, modifications and equivalents.

[0023] 在本发明的描述中,需要理解的是,术语“第一”、“第二”等仅用于描述目的,而不能理解为指示或暗示相对重要性。 [0023] In the description of the present invention, it should be understood that the term "first", "second" and used for descriptive purposes only, and can not be understood as indicating or implying relative importance. 在本发明的描述中,需要说明的是,除非另有明确的规定和限定,术语“相连”、“连接”应做广义理解,例如,可以是固定连接,也可以是可拆卸连接,或一体地连接;可以是机械连接,也可以是电连接;可以是直接相连,也可以通过中间媒介间接相连。 In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the term "coupled", "connected" to be broadly understood, for example, can be a fixed connection, it can be removably connected, or one connected; it can be mechanically connected to be electrically connected; can be directly connected, or may be connected indirectly through an intermediary. 对于本领域的普通技术人员而言,可以具体情况理解上述术语在本发明中的具体含义。 Those of ordinary skill in the art, it can be understood that the above-described circumstances specific meaning in terms of the present invention. 此外,在本发明的描述中,除非另有说明,“多个”的含义是两个或两个以上。 Further, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0024] 流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本发明的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本发明的实施例所属技术领域的技术人员所理解。 [0024] In the flow chart, or any other process or method in the manner described in this description may be understood to indicate that includes one or more steps for implementing specific logical functions or processes of the instruction code executable modules, fragment portion, and the scope or preferred embodiments of the present invention comprises a further realization, which may not be in the order shown or discussed, including based on the functionality involved by substantially simultaneous manner or in reverse order, to perform the functions It should be understood that the embodiment of ordinary skill in the art of the present invention.

[0025] 下面参考说明书附图描述根据本发明实施例的分布式集群的可靠性的测试系统。 [0025] The following description with reference to the accompanying drawings reliability test system according to the present invention, according to an embodiment of distributed cluster.

[0026] 一种分布式集群的性能测试系统,包括:上游集群,用于发送第一压力请求;被测模块;后端集群,包括多个下游集群;网桥,分别与上游集群、被测模块和后端集群相连,用于接收第一压力请求并将第一压力请求发送至被测模块,接收被测模块根据第一压力请求生成的第二压力请求并将第二压力请求转发至多个下游集群,以及保存第一压力请求以及多个下游集群根据第二压力请求返回的通信数据包;压力工具,用于根据网桥保存的第一压力请求向被测模块发送预定压力,其中被测模块将预定压力转发至网桥,网桥根据预定压力在保存的通信数据包中查找相应的数据包并返回至被测模块;以及监控模块,与被测模块位于同一个服务器上,用于监控该服务器的性能数据,并根据该性能数据进行性能分析。 [0026] A distributed cluster performance testing system, comprising: an upstream cluster, a request for transmitting a first pressure; test module; the rear end of the cluster, comprising a plurality of downstream clusters; bridge, respectively upstream cluster, measured and a rear end connected to the cluster module, for receiving a first request and a first pressure to the measured pressure request module, the module receives the request and the measured second pressure according to a second pressure of the first pressure forwards the request to a request generated by a plurality of communication data packets back downstream clusters, and saving a first pressure request and request multiple downstream clusters according to a second pressure; pressure tool for pressure bridge saved according to the first request to a predetermined pressure test modules, test Modules will be forwarded to a predetermined pressure bridge, the bridge to find according to a predetermined pressure in the preservation of the communication packets corresponding data packet and return to the test module; and a control module, and test modules are located on the same server, for monitoring The server's performance data and performance analysis based on the performance data.

[0027] 图1为本发明一个实施例的分布式集群的性能测试系统的录制过程的结构框图。 [0027] FIG. 1 is a block diagram of the recording process of the invention, an example of a distributed cluster performance test system in one embodiment. 如图1所示,根据本发明实施例的分布式集群的性能测试系统在录制过程中包括上游集群110、被测模块120、网桥130和后端集群140,其中,后端集群140包括多个下游集群141,网桥130分别与上游集群110、被测模块120和多个下游集群141相连。 As shown, according to the performance tests distributed cluster system embodiment of the present invention comprises an upstream cluster 110, the test module 120, the bridge 130 and the rear end of the recording cluster 140, wherein a rear end of the cluster 140 includes a plurality of Downstream clusters 141, 130, respectively, upstream bridge cluster 110, 120 and a plurality of the test module 141 is connected downstream clusters.

[0028] 具体地,上游集群110用于发送第一压力请求。 [0028] Specifically, the upstream pressure of the cluster 110 for transmitting a first request. 网桥130接收该第一压力请求并将该第一压力请求(二进制包)发送至被测模块120。 Bridge 130 receives the request and the first pressure of the first pressure request (binary packages) to the test module 120. 被测模块120接收该第一压力请求,并根据该第一压力请求生成第二压力请求,例如第一压力请求为搜索“百度、专利、审查意见”,生成的第二压力请求为搜索“专利、审查意见”。 Test module 120 receives the first pressure request, and the request based on the first pressure to generate a second pressure request, for example, the first pressure request for the search, "Baidu, patent, review opinions", the second pressure generated request for the search "patent review comments. " 然后,被测模块120将第二压力请求发送至多个下游集群141。 Then, the second pressure test module 120 requests to multiple downstream clusters 141. 多个下游集群141根据第二压力请求返回相应的通信数据包至网桥130。 A plurality of clusters 141 downstream pressure according to a second request returns a communication packet corresponding to the bridge 130. 在上述过程中,网桥130保存上游集群110发送的第一压力请求以及多个下游集群141返回的通信数据包,这个过程即为数据的录制过程。 In the process, saving 130 bridge upstream of the first pressure request cluster 110 and the communication packets sent more downstream clusters 141 returns, a process that is recording data. [0029] 图2为本发明一个实施例的分布式集群的性能测试系统的测试过程的结构框图。 [0029] FIG. 2 is a schematic block diagram of the test procedure distributed cluster of cases of the performance test system in one embodiment. 如图2所示,在测试过程中该分布式集群的性能测试系统仅使用被测模块120和网桥130,此外还包括压力工具150和监控模块160。 2, during the test, the distributed cluster performance test system uses only the test module 120 and bridge 130, 150 also includes a pressure tool and monitoring module 160. 利用录制过程保存下来的数据包进行数据回放,模拟被测模块整个后端集群。 Use of the recording process preserved packet data playback, analog test modules entire backend cluster.

[0030] 具体地,首先,压力工具150根据录制过程保存下来的第一压力请求数据包向被测模块120发送预定压力。 [0030] Specifically, first, a pressure tool 150 request packet according to the first pressure recording preserved transmits a predetermined pressure to the test module 120. 被测模块120再将该预定压力转发至网桥130,网桥130根据该预定压力在录制过程中保存下来的通信数据包中查找到该预定压力对应的数据包并将其返回至被测模块120。 The test module 120 then forwards a predetermined pressure to the bridge 130, the bridge 130 according to the predetermined pressure test module preserved in the recording process communication packets to find the predetermined pressure corresponding packet and return it to 120. 监控模块160与被测模块120布置在同一个服务器170上,用于监控该服务器170的性能数据,并根据该性能数据进行性能分析。 The monitoring module 160 and the measured module 120 is arranged on the same server 170 to monitor the server performance data 170, and performance analysis based on the performance data.

[0031] 应理解,对于压力工具150来说,不同模块的通信协议不一样,进行保存下来的二进制数据包的结构不一样。 [0031] It should be understood that for a pressure tool 150, the communication protocol is not the same in different modules, structural preserved binary data packet is not the same. 压力工具150要根据模块特点去包含很多请求包的二进制数据包里找出每一个请求包。 Pressure tool 150 to include the many requests for a packet based on the characteristics of binary data module package to find each request packet. 由此,在本发明的一个实施例中,在录制数据包时,每次通信包都具有如下的记录格式: Thus, in one embodiment of the present invention, when recording packet, each communication packet has the following recording formats:

[0032] [0032]

魔鬼数字Dump Id I Key 版本号保留字段&长度 n,誦AU自政 Devil Digital Dump Id I Key version number reserved field & length n, chanting AU from politics

4字节8字节8字节2字节2字节4字节 Dump的数据包体 4 bytes 8 bytes 8 bytes 2 bytes 2 bytes 4-byte packet body Dump

[0033] 其中,魔鬼数字用于标识一个协议包的头,避免二进制数据错乱;Dump ID用于标识所保存的通信数据包的编号;key标识为上文描述的保证回放真实性存在的key ;版本号用于区别不同模块;保留字段用于后续扩展使用,例如存放dump的时间戳,方便后续按照时间戳回放;包长度用于标识存取的数据包的实际长度;Dump的数据包体用于实际存放通信数据包,如压力请求数据包或返回的通信数据包。 [0033] where the devil figure for the first identifies a protocol packet, the binary data to avoid confusion; Dump ID for identifying the communication packets stored number; key identification to ensure the authenticity of the existence of the playback key is above; The version number used to distinguish between different modules; a reserved field for subsequent extensions, for example storage dump timestamps, facilitating future playback according to the time stamp; packet length is used to identify the actual length of the data packet access; Dump packet body with the actual storage of the communication packets, such as pressure request packet or communication packets returned.

[0034] 由此,压力工具150根据上述固定的记录格式进行解析,可以很容易的解析每一条记录,进而读取每次请求包,然后通过压力控制模块发送给被测模块120。 [0034] Thus, the pressure means 150 according to the recording format of the fixed resolve, can easily parse each record, and then read every request packet, then to the test module 120 is sent through a pressure control module.

[0035] 此外,在本发明的一个实施例中,网桥按照key标识&value对的形式建立词典来保存第一压力请求和返回的通信数据包,其中value为返回的通信数据包,key标识即为上述的通信包中包含的key标识。 [0035] Furthermore, in one embodiment of the present invention, the bridge set up in accordance with the dictionary identifies key & value pairs to save the first pressure request and return the communication packets, which value the communication packet is returned, key identification that is the key communication package contains the above logo. 由此,通过解析压力请求包或者对压力请求包处理后做签名而得到一个key,然后通过这个唯一的key在词典中查找捆绑的返回数据包,以此达到流量回放和线上运行流量一致。 Thus, doing autograph request packet obtained by parsing the pressure or pressure request packet processing after a key, and then returned to find the bundled package in the dictionary by the only key, in order to achieve a consistent flow running playback and online traffic.

[0036] 图3为本发明一个实施例的监控模块的结构框图。 [0036] FIG. 3 is a block diagram of an example of the invention is an embodiment of the monitoring module. 如图3所示,监控模块160包括总控单元161和多个分控单元162。 As shown in Figure 3, the monitoring module 160 includes a master control unit 161 and a plurality of sub-control unit 162. 总控单元161用于配置监控项、控制监控的启动和停止以及根据监控到的性能数据进行性能分析。 Master control unit 161 is used to configure the monitoring items, start and stop control and monitoring of performance analysis based on the monitored performance data. 每个分控单元162均与总控单元161相连,用于根据总控单元161的控制周期性地收集被测模块120的性能数据,并对性能数据进行格式化。 Each sub-control unit 162 are connected to the master control unit 161 for periodically collect performance data measured module 120 according to the control of the master control unit 161, and the performance data is formatted. 在本发明的一个实施例中,总控单元161为一台总控机器,多个分控单元162为多台性能测试机器。 In one embodiment of the present invention, the master control unit 161 as a master control machine, a plurality of sub-control unit 162 for multiple performance testing machine.

[0037] 具体地,分控单元162包括如下的功能函数: [0037] In particular, the sub-control unit 162 includes a function of a function:

[0038] (l)monitor_vmstat (fiIePath, period) [0038] (l) monitor_vmstat (fiIePath, period)

[0039] 功能:周期性地收集vmstat数据; [0039] Function: vmstat data collected periodically;

[0040] (2)monitor_module (fiIePath, period, module_name)[0041] 功能:周期性地收集某个模块的cpu、内存、文件句柄/socket句柄等占用情况; [0040] (2) monitor_module (fiIePath, period, module_name) [0041] function: a module periodically collected cpu, memory, file handles / socket handles, etc. occupancy;

[0042] (3)monitor_iostat (filePath, period, module_name) [0042] (3) monitor_iostat (filePath, period, module_name)

[0043] 功能:周期性地收集iostat数据; [0043] Function: iostat periodically collect data;

[0044] (4)monitor_net (filePath, period, module_name) [0044] (4) monitor_net (filePath, period, module_name)

[0045] 功能:周期性地收集网卡数据,采用sar命令; [0045] Function: Periodically collect card data, use the sar command;

[0046] (5) format_vmstat (filePathln, tablePath) [0046] (5) format_vmstat (filePathln, tablePath)

[0047] 功能:对vmstat数据进行格式化; [0047] Function: vmstat data format;

[0048] (6)format_module(filePathln, tablePath); [0048] (6) format_module (filePathln, tablePath);

[0049] 功能:对模块的性能数据进行格式化; [0049] Function: The module performance data format;

[0050] (7) format_iostat (filePathln, tablePath) [0050] (7) format_iostat (filePathln, tablePath)

[0051] 功能:对模块的iostat数据进行格式化; [0051] Function: iostat data module is formatted;

[0052] (8) format_net (filePathln, tablePath) [0052] (8) format_net (filePathln, tablePath)

[0053] 功能:对网络数据进行格式化。 [0053] Function: The network data is formatted.

[0054] 总控单元161包括如下的功能函数: [0054] The master control unit 161 includes a function of a function:

[0055] (I)install_trust(machineList) [0055] (I) install_trust (machineList)

[0056] 功能:建立总控单元和各个分控单元之间的双向信任关系,方便工具进行ssh和SCP操作; [0056] Function: Create a two-way trust relationship master control unit and between the various sub-control unit, convenient tools for ssh and SCP operations;

[0057] (2) deploy (machineList, dirList) [0057] (2) deploy (machineList, dirList)

[0058] 功能:将监控角本部署到各个分控单元上; [0058] Function: This deployment will monitor angle to the respective sub-control unit;

[0059] (3) start_monitor (machineList, dirList) [0059] (3) start_monitor (machineList, dirList)

[0060] 功能:批量启动各个分控单元; [0060] Function: Batch start of each sub-control unit;

[0061] (4) stop_monitor (machineList, dirList) [0061] (4) stop_monitor (machineList, dirList)

[0062] 功能:批量停止各个分控单元; [0062] Function: Batch stop each sub-control unit;

[0063] (5)get_machine_info(machineList) [0063] (5) get_machine_info (machineList)

[0064] 功能:获取各个分控单元上的机器物理参数; [0064] Function: Get the machine on the physical parameters of each sub-control unit;

[0065] (6)get_average(tablePathln, secsj tcibIePcithOut) [0065] (6) get_average (tablePathln, secsj tcibIePcithOut)

[0066] 功能:对格式化后的数据求其在不同时间段内的平均值; [0066] Function: The data formatted find the average in different time periods;

[0067] (7) get_min (tablePathln, secs, tablePathOut) [0067] (7) get_min (tablePathln, secs, tablePathOut)

[0068] 功能:对格式化后的数据求其在不同时间段内的最小值; [0068] Function: The data formatted seeking the minimum in different time periods;

[0069] (8) get_max (tablePathln, secs, tablePathOut) [0069] (8) get_max (tablePathln, secs, tablePathOut)

[0070] 功能:对格式化后的数据求其在不同时间段内的最大值; [0070] Function: The data formatted seeking the maximum in different time periods;

[0071] (9) get_rate (tablePathln, secs, tablePathOut) [0071] (9) get_rate (tablePathln, secs, tablePathOut)

[0072] 功能:对格式化后的数据求某一列数据在不同区间的比例。 [0072] Function: The data formatted seek a column of data in a different section of proportion.

[0073]图4为本发明另一个实施例的监控模块的结构框图。 [0073] Figure 4 is a block diagram of the invention of another embodiment of the monitoring module. 如图4所示,在图3所示的实施例的基础上,该监控模块160还包括日志处理单元163。 4, on the basis of the embodiment shown in FIG. 3, the monitoring module 160 of the processing unit 163 further includes a log. 日志处理单元163与总控单元161相连,用于对获取到的性能数据进行解析和统计。 Log processing unit 163 is connected with the master control unit 161, it is used to obtain performance data parsing and statistics. 具体地可包括如下的功能函数: Specifically, the following functions may include a function of:

[0074] (I) get_average_press(logPath, secs) [0074] (I) get_average_press (logPath, secs)

[0075] 功能:获取不同时间段内压力的平均值; [0075] Function: Get different time pressures mean;

[0076] (2) get_max_press (logPath, secs)[0077] 功能:获取不同时间段内压力的最大值; [0076] (2) get_max_press (logPath, secs) [0077] Function: Get the maximum pressure of different time periods;

[0078] (3) get_min_press (logPath, secs) [0078] (3) get_min_press (logPath, secs)

[0079] 功能:获取不同时间段内压力的最小值; [0079] Function: Get a different period of minimum pressure;

[0080] (4)get_average_parm(logPath, secs, key) [0080] (4) get_average_parm (logPath, secs, key)

[0081] 功能:获取不同时间段内某个参数的平均值; [0081] Function: Get the average value of a parameter over different time periods;

[0082] (5) get_max_parm(logPath, secs, key) [0082] (5) get_max_parm (logPath, secs, key)

[0083] 功能:获取不同时间段内某个参数的最大值; [0083] Function: Get the maximum value of a parameter over different time periods;

[0084] (6)get_min_parm(logPath, secs, key) [0084] (6) get_min_parm (logPath, secs, key)

[0085] 功能:获取不同时间段内某个参数的最小值; [0085] Function: Gets the minimum value of a parameter of different time periods;

[0086] (7) get_rate_parm(logPath, secs, key) [0086] (7) get_rate_parm (logPath, secs, key)

[0087] 功能:获取某个参数在不同区间的比例。 [0087] Function: Gets a parameter ratio in different intervals.

[0088] 应理解,上述总控单元、分控单元以及日志处理单元的功能函数仅为示意性的例子。 [0088] It should be understood, it said total performance function control unit, sub-control unit and a log processing unit is only illustrative examples. 本领域的技术人员可根据实际需要配置不同的监控项,也就是使用不同的功能函数,这些修改和变化均应包含在本发明的保护范围内。 Those skilled in the art can be configured according to the actual needs of different monitoring items, which is a function of the use of different functions, such modifications and variations should be included within the scope of the present invention.

[0089] 根据本发明实施例的分布式集群的性能测试系统,网桥分别与上游集群、被测模块和多个下游集群(后端集群)相连,通过一个网桥可以真实模拟线上拓扑和数据量,无需搭建真实的集群环境即可对被测模块的性能进行测试,提高测试准确性和测试效率,节约成本。 [0089] distributed cluster performance test system embodiment of the present invention, respectively upstream bridge cluster, test module and a plurality of downstream clusters (back-end clusters) connected by a bridge topology and can simulate online the amount of data without a clustered environment to build real performance test module for testing, improve test accuracy and test efficiency and cost savings.

[0090] 下面参考说明书附图描述根据本发明实施例的网桥。 [0090] The following description with reference to the accompanying drawings bridge embodiment according to the present invention.

[0091] 图5为本发明一个实施例的网桥的结构框图。 [0091] FIG. 5 is a block diagram of the invention according to one embodiment of the bridge. 如图5所示,根据本发明实施例的网桥包括模式选择模块310、通信模块320、录制模块330和模拟模块340。 As shown in Figure 5, selection module 310, communication module 320 according to an embodiment of the present invention, the bridge includes a mode, the recording module 330 and simulation module 340.

[0092] 具体地,模式选择模块310用于接收用户输入的录制模式或测试模式。 [0092] Specifically, the mode selection module 310 for receiving user input recording mode or test mode. 在录制模式下,通信模块320接收上游集群发送的第一压力请求并将该第一压力请求发送至被测模块,以及接收被测模块根据第一压力请求生成的第二压力请求并将该第二压力请求转发至后端集群,以及接收后端集群根据第二压力请求返回的通信数据包;录制模块330与通信模块320相连,保存第一压力请求和返回的通信数据包。 In the recording mode, the communication module 320 receives a first pressure upstream cluster requests sent and the request is sent to the first pressure test module, and a second pressure receiving test module requests generated from the first pressure and the first request second pressure request is forwarded to the back-end cluster, and receive back-end cluster communication packets according to a second pressure request returns; recording module 330 is connected with the communication module 320 to save a first pressure request and communication packets returned. 在测试模式下,通信模块320接收被测模块转发的预定压力。 In test mode, the communication module 320 receives a predetermined pressure test module forwarding. 模拟模块340分别与通信模块320和录制模块330相连,根据预定压力在录制模块330中保存的通信数据包中查找到该预定压力对应的返回数据包。 Analog module 340 and communications module 320, respectively, and recording module 330 is connected, according to a predetermined pressure to find the recording module 330 in communication packets stored in the predetermined pressure corresponding return packets. 然后,通信模块320将该数据包发送至被测模块。 Then, the communication module 320 transmits the packet to the module under test.

[0093] 应理解,不同模块的通信协议不一样,进行保存下来的二进制数据包的结构不一样。 [0093] It should be understood, the communication protocol different modules are not the same, down to save binary data packet structure is different. 在进行性能测试时,要根据模块特点去包含很多请求包的二进制数据包里找出每一个请求包。 During performance testing, to request packet to contain a lot of binary data package to find each request packet according to the module characteristics. 由此,在本发明的一个实施例中,在录制数据包时,每次通信包都具有如下的记录格式: Thus, in one embodiment of the present invention, when recording packet, each communication packet has the following recording formats:

[0094] [0094]

魔鬼数字Dump IdI Key 版本号保留字段&长度4字节8字节8字节2字节2字节4字节 Devil Digital Dump IdI Key version number reserved field & length 4 bytes 8 bytes 8 bytes 2 bytes 2 bytes 4 bytes

[0095] 其中,魔鬼数字用于标识一个协议包的头,避免二进制数据错乱;Dump ID用于标识所保存的通信数据包的编号;key标识为上文描述的保证回放真实性存在的key ;版本号用于区别不同模块;保留字段用于后续扩展使用,例如存放dump的时间戳,方便后续按照时间戳回放;包长度用于标识存取的数据包的实际长度;Dump的数据包体用于实际存放通信数据包,如压力请求数据包或返回的通信数据包。 [0095] where the devil figure for the first identifies a protocol packet, the binary data to avoid confusion; Dump ID for identifying the communication packets stored number; key identification to ensure the authenticity of the existence of the playback key is above; The version number used to distinguish between different modules; a reserved field for subsequent extensions, for example storage dump timestamps, facilitating future playback according to the time stamp; packet length is used to identify the actual length of the data packet access; Dump packet body with the actual storage of the communication packets, such as pressure request packet or communication packets returned. 由此,在进行性能测试时,根据上述固定的记录格式进行解析,可以很容易的解析每一条记录,进而读取每次请求包,然后发送预定压力给被测模块。 Accordingly, during the performance test, based on the fixed record format parsing, you can easily parse each record, and then read every request packet, and then transmits a predetermined pressure to the test module.

[0096] 此外,在本发明的一个实施例中,模拟模块240按照key标识&value对的形式建立词典来保存第一压力请求和返回的通信数据包,其中value为返回的通信数据包,key标识即为上述的通信包中包含的key标识。 [0096] Furthermore, in one embodiment of the present invention, the simulation module 240 in accordance with the establishment dictionary identifies key & value pairs to save the first pressure request and return the communication packets, which value the communication packet is returned, key identification Communication is the key identified above included in the package. 由此,可以先通过解析压力请求包或者对压力请求包处理后做签名而得到一个key,然后模拟模块240通过这个唯一的key在词典中查找捆绑的返回数据包,以此达到流量回放和线上运行流量一致。 Thus, you can start by analyzing the pressure request packet or request packet processing pressure signatures do get a key, and then simulate module 240 through this unique key lookup bundled return packets in the dictionary in order to achieve flow playback and line agreement on the run flow.

[0097] 根据本发明实施例的网桥,通过与上游集群、被测模块和后端集群相连,可以真实地模拟线上拓扑和数量,无需搭建真实的集群环境即可对被测模块的性能进行测试,提高测试准确性和测试效率,节约成本。 [0097] According to the measured performance of the module bridge example embodiment of the invention, through, and back-end cluster test module is connected to the upper reaches of the cluster, can realistically simulate line topology and the number of cluster environments without the need to build a real test, improve test accuracy and test efficiency and cost savings.

`[0098] 下面参考说明书附图描述根据本发明实施例的分布式集群的性能测试方法。 `[0098] described below with reference to the accompanying drawings distributed cluster performance test method according to an embodiment of the present invention.

[0099] 图6为本发明一个实施例的分布式集群的性能测试方法的流程图。 [0099] FIG. 6 is a flowchart of distributed cluster of cases of the invention of a performance test methods implemented.

[0100] 如图6所示,根据本发明实施例的分布式集群的性能测试方法,包括下述步骤。 [0100] As shown in Figure 6, according to the performance test method distributed cluster of embodiments of the present invention comprises the following steps.

[0101] 步骤S101,在录制模式下,网桥接收上游集群发送的第一压力请求,并将该第一压力请求发送至被测模块,以及保存该第一压力请求。 [0101] step S101, the in the recording mode, the bridge receives a first pressure upstream of the cluster to send a request, and the request is sent to the first pressure test module, and saving the first pressure request.

[0102] 步骤S102,被测模块接收第一压力请求,并根据该第一压力请求生成第二压力请求,以及将第二压力请求发送至网桥。 [0102] step S102, the measured first pressure receiving module request, and the request based on the first pressure generating a second pressure request, and the request is sent to the second pressure bridge.

[0103] 例如,第一压力请求为搜索“百度、专利、审查意见”,生成的第二压力请求为搜索“专利、审查意见”。 [0103] For example, the first pressure request for the search, "Baidu, patent, review opinions", the second pressure generated request for the search "patent, review comments." ` `

[0104] 步骤S103,网桥接收第二压力请求并将第二压力请求发送至后端集群,其中后端集群包括多个下游集群。 [0104] step S103, the bridge receives a second pressure request and sends the request to the rear end of the second pressure clusters, where the back-end cluster includes a plurality of downstream clusters.

[0105] 步骤S104,多个下游集群根据第二压力请求返回相应的数据包至网桥。 [0105] step S104, the multiple downstream clusters request to return the corresponding data packet according to a second pressure to the bridge.

[0106] 步骤S105,网桥接收并保存多个下游集群返回的数据包。 [0106] step S105, the bridge receives and stores a plurality of downstream clusters returned packets.

[0107] 上述步骤SlOl至步骤S105即为数据的录制过程。 [0107] the above steps SlOl to step S105 is recording data. 通过录制过程保存的数据包,即可模拟被测模块整个后端集群,实现后续的被测模块的性能测试。 By recording saved data packet, the entire rear end of the module under test to simulate the cluster, subsequent testing for performance test module.

[0108] 步骤S106,在测试模式下,通过压力工具根据网桥保存的第一压力请求发送预定压力至被测模块。 [0108] step S106, the in test mode, according to a first pressure by a pressure tool to save the bridge sends a predetermined pressure to the test module requests.

[0109] 应理解,对于压力工具来说,不同模块的通信协议不一样,进行保存下来的二进制数据包的结构不一样。 [0109] It should be understood that for pressure tool, the communication protocol is not the same in different modules, structural preserved binary data packet is not the same. 压力工具要根据模块特点去包含很多请求包的二进制数据包里找出每一个请求包。 Tools contains a lot of pressure to go to the request packet bag module according to the characteristics of the binary data to identify each request packet. 由此,在本发明的一个实施例中,在录制数据包时,每次通信包都具有如下的记录格式: Thus, in one embodiment of the present invention, when recording packet, each communication packet has the following recording formats:

[0110] [0110]

Figure CN103297285AD00121

[0111] 其中,魔鬼数字用于标识一个协议包的头,避免二进制数据错乱;Dump ID用于标识所保存的通信数据包的编号;key标识为上文描述的保证回放真实性存在的key ;版本号用于区别不同模块;保留字段用于后续扩展使用,例如存放dump的时间戳,方便后续按照时间戳回放;包长度用于标识存取的数据包的实际长度;Dump的数据包体用于实际存放通信数据包,如压力请求数据包或返回的通信数据包。 [0111] where the devil figure for the first identifies a protocol packet, the binary data to avoid confusion; Dump ID for identifying the communication packets stored number; key identification to ensure the authenticity of the existence of the playback key is above; The version number used to distinguish between different modules; a reserved field for subsequent extensions, for example storage dump timestamps, facilitating future playback according to the time stamp; packet length is used to identify the actual length of the data packet access; Dump packet body with the actual storage of the communication packets, such as pressure request packet or communication packets returned. 由此,压力工具根据上述固定的记录格式进行解析,可以很容易的解析每一条记录,进而读取每次请求包,然后通过压力发送控制模块发送预定压力给被测模块。 Thus, the pressure tool based on the fixed record format parsing, can easily parse each record, and then read every request packet, then the transmission control module sends a predetermined pressure by the pressure to be measured module.

[0112] 步骤S107,被测模块将预定压力转发至网桥。 [0112] step S107, a predetermined pressure test module will be forwarded to the bridge.

[0113] 步骤S108,网桥根据预定压力在所保存的通信数据包中查找到相应的返回数据包。 [0113] step S108, the bridge to find the pressure in accordance with a predetermined communication packets stored in the corresponding return packets.

[0114] 在本发明的一个实施例中,按照key标识&value对的形式建立词典来保存第一压力请求和返回的通信数据包,其中value为返回的通信数据包,key标识即为上述的通信包中包含的key标识。 [0114] In one embodiment of the present invention, in accordance with the key identification & value established in the form of a dictionary to hold the first pressure request and return the communication packets, which value the communication packet is returned, key identification is the above-described communication key identifier included in the package. 由此,可以先通过解析压力请求包或者对压力请求包处理后做签名而得到一个key,然后通过这个唯一的key在词典中查找捆绑的返回数据包,以此达到流量回放和线上运行流量一致。 Thus, it can start by analyzing the pressure request packet or request packet processing pressure signatures do get a key, and then returned to find the bundled package in the dictionary by the only key, in order to achieve traffic and online playback operation flow rate agreement.

[0115] 步骤S109,在测试过程中,监控被测模块所在服务器的性能数据,并根据该性能数据进行性能分析。 [0115] step S109, the testing process, monitor the performance of the server where the data of an object module, and performance analysis based on the performance data.

[0116] 应理解,不同模块需要监控的性能指标不一样,在本发明的一个实施例中提供通用的监控工具,用户只需要配置监控项,监控工具自动监控相应的数据,性能测试结束后,自动地根据要求对这些监控数据进行分析。 [0116] It should be understood that different modules need to monitor the performance indicators are not the same, in one embodiment of the present invention to provide a common monitoring tool, users only need to configure the monitoring items, monitoring tool automatically monitors the appropriate data, after the performance test, automatically according to the requirements of these monitoring data for analysis.

[0117] 具体地,通用监控工具分为两个单元,总控单元和多个分控单元。 [0117] In particular, the common monitoring tools are divided into two units, the master control unit and a plurality of sub-control units. 总控单元负责监控的部署、启动、停止以及数据的分析工作。 Master Control Unit is responsible for monitoring the deployment, start, stop, and analysis of data. 每个分控单元均与总控单元相连,根据总控单元的控制指令周期性地收集数据,并对收集到的数据进行格式化。 Each sub-control units are connected to the master control unit, periodically collect data in accordance with a control command master control unit, and the data collected will be formatted.

[0118] 在本发明的一个实施例中,分控单元收集的数据可包括:vmstat数据;CPU、内存和文件句柄的占用情况;iostat数据;被测模块的日志;以及网络数据等。 [01] In one embodiment of the present invention, the data collected by the sub-control unit may include: vmstat data; CPU, memory and file handles occupancy; iostat data; log test modules; and network data.

[0119] 总控单元的具体功能可包括:建立所述总控单元与各个所述分控单元之间的双向信任关系;将监控脚本部署到各个所述分控单元上;批量启动或停止所述多个分控单元;获取各个所述分控单元的机器物理参数;以及根据所格式化后的性能数据计算其在不同时间段内的平均值、最小值、最大值以及某一列数据在不同区间的比例等。 Specific functions [0119] The total control unit may include: a two-way trust relationship with the master control unit between each of the sub-control unit; and will be deployed to monitor the script on each of the sub-control unit; batch can be started or stopped by said plurality of sub-control unit; obtaining each of the sub-control unit of the machine physical parameters; and calculated performance data after formatting it in different time periods of the average, minimum, maximum, and a column of data in different proportion range.

[0120] 在本发明的另一个实施例中,通用监控工具还可包括日志处理单元。 [0120] In another embodiment of the present invention, the universal monitoring tool also includes a log processing unit. 该日志处理单元与总控单元相连,用于对获取到的性能数据进行解析和统计。 The log processing unit is connected with the master control unit for the performance of the acquired data is parsed and statistics. 例如,获取不同时间段内压力的平均值、最大值和最小值等。 For example, to obtain different time pressures of the average, maximum and minimum values and the like.

[0121] 根据本发明实施例的分布式集群的性能测试方法,通过网桥抓取上游集群、被测模块以及后端集群之间的通信数据包,从而通过一个网桥可以真实模拟线上拓扑和数据量,无需搭建真实的集群环境即可对被测模块进行性能测试,提高测试准确性和测试效率,节约成本。 [0121] The method according to performance tests distributed cluster of embodiments of the present invention, crawling through the bridge upstream cluster communication packets between the test module and the rear end of the cluster, which can simulate through a bridge line topology and the amount of data without the need to build a real cluster environment allows the test module performance testing, improve test accuracy and test efficiency and cost savings.

[0122] 综上,根据本发明实施例的分布式集群的性能测试的系统及方法和网桥,至少具有以下有益效果: [0122] In summary, according to systems and methods and embodiments of the present invention bridges distributed cluster of performance testing, at least have the following beneficial effects:

[0123] 1、拓扑结构和线上一致;[0124] 根据线上集群各个模块的连接关系拓扑结构,构建线下测试集群,保证被测模块所处的上下游环境和线上一致。 [0123] 1, topology and consistent line; [0124] According to the topology of the line connection between the various modules of the cluster, the cluster to build the line test, to ensure consistent test module is located on the downstream environment and online.

[0125] 2、数据量和线上相当; [0125] 2, a considerable amount of data and online;

[0126] 模拟集群使用的数据包来源于被测模块运行态dump下来或者通过网桥在调研环境中dump下来,数据量和数据内容与线上基本一致。 [0126] Packet simulation test module used in the cluster from running down state dump or dump down by the bridge in the research environment, basically the same amount of data and data content and online.

[0127] 3、数据流回放和线上一致; [0127] 3, the data stream playback and online the same;

[0128] 模拟集群使用的数据包在dump下来时对每个数据包都做了标记。 [0128] Packet simulated clusters used when dump down for each packet labeled. 每个请求和返回结果用一个唯一的标识建立词典,回放时,网桥根据请求包获得唯一的标识,然后通过这个唯一的标识在词典中查找捆绑的返回数据包,以此达到流量回放和线上运行流量一致。 Each request and returns the result set up with a unique identification dictionary, playback, bridges, according to the request packet to obtain a unique identity, and then look through the unique identifier in the dictionary bundled return packets, in order to achieve flow playback and line agreement on the run flow.

[0129] 4、提高性能测试准确性; [0129] 4, to improve the accuracy of the performance test;

[0130] 使用模拟线上拓扑和数据量的真实集群,提高性能测试准确性。 [0130] Using an analog line topology and real data clusters, improve performance test accuracy.

[0131] 5、提高测试效率; [0131] 5, to improve testing efficiency;

[0132] 无需搭建真实的集群环境,提高测试效率。 [0132] do not need to build a real cluster environment, improve test efficiency.

[0133] 6、节约成本; [0133] 6, cost savings;

[0134] 无需使用真实集群环境,使用一台机器模拟整个后端集群,大量节省机器资源,节约成本。 [0134] without using a real cluster environment, use a machine to simulate the entire rear end of the cluster, save a lot of machine resources and cost savings.

[0135] 应当理解,本发明的各部分可以用硬件、软件、固件或它们的组合来实现。 [0135] It should be understood that the various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. 在上述实施方式中,多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。 In the above embodiment, and a plurality of steps or methods may be implemented in software or firmware executed by the system to perform the appropriate instructions stored in a memory with. 例如,如果用硬件来实现,和在另一实施方式中一样,可用本领域公知的下列技术中的任一项或他们的组合来实现:具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路,具有合适的组合逻辑门电路的专用集成电路,可编程门阵列(PGA),现场可编程门阵列(FPGA)等。 For example, if implemented in hardware, as in another embodiment, it can be known in the art of any one of the following techniques or their combination to achieve: a data signal for implementing logic functions of logic gates discrete logic circuit having appropriate combinational logic gates of ASIC, programmable gate array (PGA), a field programmable gate array (FPGA) and so on.

[0136] 在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。 [0136] In the description of the present specification, reference to the term "one embodiment," "some embodiments", "an example", "concrete example", or "some examples" and the description is meant in connection with the embodiment or example described The particular feature, structure, material, or characteristic included in the present invention, at least one embodiment or example. 在本说明书中,对上述术语的示意性表述不一定指的是相同的实施例或示例。 In the present specification, the term of the above schematic representation is not necessarily referring to the same embodiment or example. 而且,描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。 Furthermore, the particular features, structures, materials, or characteristics described in any one or more embodiments or examples in proper manner.

[0137] 尽管已经示出和描述了本发明的实施例,对于本领域的普通技术人员而言,可以理解在不脱离本发明的原理和精神的情况下可以对这些实施例进行多种变化、修改、替换和变型,本发明的范围由所附权利要求及其等同限定。 [0137] Although there have been shown and described embodiments of the present invention, those of ordinary skill in the art, it can be understood without departing from the principles and spirit of the present invention may be made in these embodiments, various changes, modifications, substitutions and modifications, the scope of the invention being indicated by the appended claims and their equivalents defined.

Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
CN101127801A *29 Aug 200720 Feb 2008中兴通讯股份有限公司A method for testing relay performance of group call
CN202033748U *22 Apr 20119 Nov 2011阿里巴巴集团控股有限公司Search engine performance test system
US8037109 *30 Jun 200311 Oct 2011Microsoft CorporationGeneration of repeatable synthetic data
US20070118633 *10 Nov 200524 May 2007Kumar C P VCluster command testing
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
CN104461856A *22 Sep 201325 Mar 2015阿里巴巴集团控股有限公司Performance test method, device and system based on cloud computing platform
Classifications
International ClassificationH04L12/26
Legal Events
DateCodeEventDescription
11 Sep 2013C06Publication
16 Oct 2013C10Entry into substantive examination
16 Mar 2016C14Grant of patent or utility model