CN103297285A - Distributed cluster performance test system, method and device - Google Patents
Distributed cluster performance test system, method and device Download PDFInfo
- Publication number
- CN103297285A CN103297285A CN2012100438498A CN201210043849A CN103297285A CN 103297285 A CN103297285 A CN 103297285A CN 2012100438498 A CN2012100438498 A CN 2012100438498A CN 201210043849 A CN201210043849 A CN 201210043849A CN 103297285 A CN103297285 A CN 103297285A
- Authority
- CN
- China
- Prior art keywords
- module
- pressure request
- cluster
- pressure
- data packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a distributed cluster performance test system, method and device. The system comprises an upstream cluster, a tested module, a back end cluster, a network bridge, a pressure tool and a monitoring module. The network bridge is used for receiving a first pressure request sent by the upstream cluster, sending the first pressure request to the tested module, receiving a second pressure request generated by the tested module according to the first pressure request, forwarding the second pressure request to the back end cluster and storing the first pressure request and communication data packs returned by the back end cluster according to the second pressure request. The pressure tool is used for sending preset pressure to the tested module according to the first pressure request, and the tested module forwards the preset pressure to the network bridge, so that corresponding packs in the stored communication data packs are checked and are returned to tested module. The monitoring module and the tested module are placed on the same server. The monitoring module is used for monitoring performance data of the server and carrying out performance analysis according to the performance data. The network bridge is used for simulating a real cluster, performance test accuracy and test efficiency are improved, and cost is saved.
Description
Technical field
The present invention relates to the web search technical field, relate in particular to a kind of Performance Test System, method and apparatus of distributed type assemblies.
Background technology
For the distributed cluster system of similar network search engine, existing performance test is carried out in true environment, a large amount of machine resources need be built the rear end cluster environment.The problem that this method exists is: (1) waste machine resources, manpower maintenance cost height; (2) can't build under the line with line on the environment of the same data volume scale, fall far short on data volume and the line; Inconsistent on the network topology of the rear end cluster of (3) building under the line and the line.
And, need in the prior art according to different tested modules, prepare different pressure input source, compression tools and monitoring tools, versatility is poor, needs more development and maintenance cost.
Summary of the invention
The present invention is intended to one of solve the problems of the technologies described above at least.
For this reason, first purpose of the present invention is to propose a kind of Performance Test System that can improve testing efficiency and test performance accuracy, versatility is good, cost is low distributed type assemblies.
Second purpose of the present invention is to propose a kind of bridge.
The 3rd purpose of the present invention is to propose a kind of performance test methods of distributed type assemblies.
To achieve these goals, the Performance Test System of the distributed type assemblies of embodiment comprises according to a first aspect of the invention: the upstream cluster, and described upstream cluster is used for the first pressure request that sends; Tested module; The rear end cluster, described rear end cluster comprises a plurality of downstreams cluster; Bridge, described bridge links to each other with described upstream cluster, tested module and rear end cluster respectively, described bridge is used for receiving the described first pressure request and the described first pressure request being sent to described tested module, receive the second pressure request that described tested module generates according to the described first pressure request and the described second pressure request is forwarded to described a plurality of downstreams cluster, and preserve the communication data packet that the described first pressure request and described a plurality of downstreams cluster return according to the described second pressure request; Compression tools, the described first pressure request that described compression tools is used for preserving according to described bridge sends predetermined pressure to described tested module, wherein said tested module is forwarded to described bridge with described predetermined pressure, and described bridge is searched the corresponding packet of described predetermined pressure and is back to described tested module according to described predetermined pressure in the communication data packet of described preservation; And monitoring module, described monitoring module and described tested module are positioned on the same server, are used for the performance data of the described server of monitoring, and carry out performance evaluation according to described performance data.
Performance Test System according to the distributed type assemblies of the embodiment of the invention, bridge links to each other with upstream cluster, tested module and a plurality of downstreams cluster (rear end cluster) respectively, can real simulation line upper topology and data volume by a bridge, need not to build real cluster environment can test the performance of tested module, improve test accuracy and testing efficiency, save cost.
To achieve these goals, the bridge of embodiment comprises according to a second aspect of the invention: mode selection module, and described mode selection module is used for receiving recording mode or the test pattern of user's input; Communication module, described communication module is used for receiving the first pressure request of upstream cluster transmission and the described first pressure request is sent to tested module under described recording mode, and receive described tested module according to the second pressure request of described first pressure request generation and the described second pressure request is forwarded to the rear end cluster, and receive the communication data packet that described rear end cluster returns according to the described second pressure request; Described communication module also is used for receiving the predetermined pressure that described tested module is transmitted under described test pattern; Record module, the described module of recording links to each other with described communication module, is used for preserving the described first pressure request and the described communication data packet of returning; And analog module, described analog module links to each other with the described module of recording with described communication module respectively, be used for recording the return data bag that communication data packet that module preserves is searched described predetermined pressure correspondence according to described predetermined pressure described, wherein said communication module also is used for the return data bag of described predetermined pressure correspondence is sent to described tested module.
Bridge according to the embodiment of the invention, by linking to each other with upstream cluster, tested module and rear end cluster, artificial line upper topology and quantity truly, need not to build real cluster environment can test the performance of tested module, improve test accuracy and testing efficiency, save cost.
To achieve these goals, the performance test methods of the distributed type assemblies of embodiment according to a third aspect of the invention we, may further comprise the steps: (a) under recording mode, bridge receives the first pressure request that the upstream cluster sends, and the described first pressure request is sent to tested module, and preserve the described first pressure request; (b) described tested module receives the described first pressure request, and generates the second pressure request according to the described first pressure request, and the described second pressure request is sent to described bridge; (c) described bridge receives the described second pressure request and the described second pressure request is sent to the rear end cluster, and wherein said rear end cluster comprises a plurality of downstreams cluster; (d) described a plurality of downstreams cluster returns corresponding packet to described bridge according to the described second pressure request; (e) described bridge receives described and preserves the described packet that returns; (f) under test pattern, the first pressure request of preserving according to described bridge by compression tools sends predetermined pressure to described tested module; (g) described tested module is forwarded to described bridge with described predetermined pressure, and receives the corresponding packet that described bridge finds in the communication data packet of described preservation according to described predetermined pressure; (h) in test process, monitor the performance data of the server at described tested module place, and carry out performance evaluation according to described performance data.
Performance test methods according to the distributed type assemblies of the embodiment of the invention, by the communication data packet between bridge extracting upstream cluster, tested module and the rear end cluster, thereby can real simulation line upper topology and data volume by a bridge, need not to build real cluster environment and can carry out performance test to tested module, improve test accuracy and testing efficiency, save cost.
The aspect that the present invention adds and advantage part in the following description provide, and part will become obviously from the following description, or recognize by practice of the present invention.
Description of drawings
Above-mentioned and/or the additional aspect of the present invention and advantage be from obviously and easily understanding becoming the description of embodiment below in conjunction with accompanying drawing, wherein,
Fig. 1 is the recording mode structured flowchart of the Performance Test System of distributed type assemblies according to an embodiment of the invention;
Fig. 2 is the structured flowchart of the test pattern of the Performance Test System of distributed type assemblies according to an embodiment of the invention;
Fig. 3 is the structured flowchart of monitoring module according to an embodiment of the invention;
Fig. 4 is the structured flowchart of monitoring module in accordance with another embodiment of the present invention;
Fig. 5 is the structured flowchart of bridge according to an embodiment of the invention; And
Fig. 6 is the flow chart of the performance test methods of distributed type assemblies according to an embodiment of the invention.
Embodiment
Describe embodiments of the invention below in detail, the example of described embodiment is shown in the drawings, and wherein identical or similar label is represented identical or similar elements or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment that is described with reference to the drawings, only be used for explaining the present invention, and can not be interpreted as limitation of the present invention.On the contrary, embodiments of the invention comprise spirit and interior all changes, modification and the equivalent of intension scope that falls into institute's additional claims.
In description of the invention, it will be appreciated that term " first ", " second " etc. only are used for describing purpose, and can not be interpreted as indication or hint relative importance.In description of the invention, need to prove that unless clear and definite regulation and restriction are arranged in addition, term " links to each other ", " connection " should do broad understanding, for example, can be fixedly connected, also can be to removably connect, or connect integratedly; Can be mechanical connection, also can be to be electrically connected; Can be directly to link to each other, also can link to each other indirectly by intermediary.For the ordinary skill in the art, can concrete condition understand above-mentioned term concrete implication in the present invention.In addition, in description of the invention, except as otherwise noted, the implication of " a plurality of " is two or more.
Describe and to be understood that in the flow chart or in this any process of otherwise describing or method, expression comprises module, fragment or the part of code of the executable instruction of the step that one or more is used to realize specific logical function or process, and the scope of preferred implementation of the present invention comprises other realization, wherein can be not according to order shown or that discuss, comprise according to related function by the mode of basic while or by opposite order, carry out function, this should be understood by the embodiments of the invention person of ordinary skill in the field.
Below with reference to the test macro of Figure of description description according to the reliability of the distributed type assemblies of the embodiment of the invention.
A kind of Performance Test System of distributed type assemblies comprises: the upstream cluster is used for the first pressure request that sends; Tested module; The rear end cluster comprises a plurality of downstreams cluster; Bridge, link to each other with upstream cluster, tested module and rear end cluster respectively, be used for the first pressure request that receives and the first pressure request is sent to tested module, receive the second pressure request that tested module generates according to the first pressure request and the second pressure request is forwarded to a plurality of downstreams cluster, and preserve the communication data packet that the first pressure request and a plurality of downstreams cluster return according to the second pressure request; Compression tools, be used for sending predetermined pressure according to the first pressure request that bridge is preserved to tested module, wherein tested module is forwarded to bridge with predetermined pressure, and bridge is searched corresponding packet and is back to tested module in the communication data packet of preserving according to predetermined pressure; And monitoring module, be positioned on the same server with tested module, be used for the performance data of this server of monitoring, and carry out performance evaluation according to this performance data.
Fig. 1 is the structured flowchart of recording process of Performance Test System of the distributed type assemblies of one embodiment of the invention.As shown in Figure 1, Performance Test System according to the distributed type assemblies of the embodiment of the invention comprises upstream cluster 110, tested module 120, bridge 130 and rear end cluster 140 in recording process, wherein, rear end cluster 140 comprises a plurality of downstreams cluster 141, and bridge 130 links to each other with upstream cluster 110, tested module 120 and a plurality of downstreams cluster 141 respectively.
Particularly, upstream cluster 110 is used for the first pressure request that sends.Bridge 130 receives this first pressure request and this first pressure request (binary system bag) is sent to tested module 120.Tested module 120 receives this first pressure request, and generates the second pressure request according to this first pressure request, and for example the first pressure request is search " Baidu, patent, examination reports ", and the second pressure request of generation is search " patent, examination reports ".Then, tested module 120 is sent to a plurality of downstreams cluster 141 with the second pressure request.A plurality of downstreams cluster 141 returns corresponding communication data packet to bridge 130 according to the second pressure request.In said process, bridge 130 is preserved the first pressure request of upstream cluster 110 transmissions and the communication data packet that a plurality of downstreams cluster 141 returns, and this process is the recording process of data.
Fig. 2 is the structured flowchart of test process of Performance Test System of the distributed type assemblies of one embodiment of the invention.As shown in Figure 2, the Performance Test System of this distributed type assemblies only uses tested module 120 and bridge 130 in test process, comprises compression tools 150 and monitoring module 160 in addition.The packet that utilizes recording process to preserve to get off carries out data readback, the whole rear end of simulation tested module cluster.
Particularly, at first, compression tools 150 is preserved the first pressure request data package of getting off and is sent predetermined pressure to tested module 120 according to recording process.Tested module 120 is forwarded to bridge 130 with this predetermined pressure again, finds the packet of this predetermined pressure correspondence and it is back to tested module 120 in the communication data packet under bridge 130 is preserved in recording process according to this predetermined pressure.Monitoring module 160 is arranged on the same server 170 with tested module 120, is used for the performance data of this server 170 of monitoring, and carries out performance evaluation according to this performance data.
Should be understood that for compression tools 150, the communication protocol of disparate modules is different, the structure of preserving the binary data packets of getting off is different.Compression tools 150 will go to comprise in the binary data packets of a lot of request bags according to module characteristic finds out each request bag.Thus, in one embodiment of the invention, when the data recording bag, each communications packets all has following record format:
Wherein, devil's numeral is used for the head of a protocol package of sign, avoids the binary data entanglement; Dump ID is used for identifying the numbering of the communication data packet of preserving; Key is designated the key that above-described assurance playback authenticity exists; Version number is used for the difference disparate modules; Reserved field is used for subsequent expansion to be used, and for example deposits the timestamp of dump, convenient follow-up according to the timestamp playback; Packet length is used for the physical length of the packet of sign access; The packet body of Dump is used for the actual communication data packet of depositing, as pressure request data package or the communication data packet returned.
Thus, compression tools 150 is resolved according to the record format of said fixing, can resolve each bar record easily, and then reads each request bag, sends to tested module 120 by the pressure control module then.
In addition, in one embodiment of the invention, bridge is according to key according to ﹠amp; The form that value is right is set up the communication data packet that dictionary is preserved the first pressure request and returned, and wherein value is the communication data packet of returning, and the key sign is the key sign that comprises in the above-mentioned communications packets.Thus, by resolving pressure request bag or obtaining a key to doing signature after the processing of pressure request bag, search the return data bag of binding then by this unique key in dictionary, it is consistent with operation flow on the line to reach the flow playback with this.
Fig. 3 is the structured flowchart of the monitoring module of one embodiment of the invention.As shown in Figure 3, monitoring module 160 comprises Master Control Unit 161 and a plurality of branch controls unit 162.Master Control Unit 161 is used for the startup of configuration monitoring item, control monitoring and stops and carrying out performance evaluation according to the performance data that monitors.Each divides control unit 162 all to link to each other with Master Control Unit 161, is used for collecting according to the control cycle ground of Master Control Unit 161 performance data of tested module 120, and performance data is formatd.In one embodiment of the invention, Master Control Unit 161 is a master control machine, and a plurality of branch controls unit 162 is many performance test machines.
Particularly, divide control unit 162 to comprise following power function:
(1)monitor_vmstat(filePath，period)
Function: periodically collect the vmstat data;
(2)monitor_module(filePath，period，module_name)
Function: cpu, internal memory, the file handle/socket handle etc. of periodically collecting certain module take situation;
(3)monitor_iostat(filePath，period，module_name)
Function: periodically collect the iostat data;
(4)monitor_net(filePath，period，module_name)
Function: periodically collect network card data, adopt the sar order;
(5)format_vmstat(filePathIn，tablePath)
Function: the vmstat data are formatd;
(6)format_module(filePathIn，tablePath)；
Function: the performance data to module formats;
(7)format_iostat(filePathIn，tablePath)
Function: the iostat data to module format;
(8)format_net(filePathIn，tablePath)
Function: network data is formatd.
Master Control Unit 161 comprises following power function:
(1)install_trust(machineList)
Function: set up the two-way trust relationship between Master Control Unit and each branch control unit, make things convenient for instrument to carry out ssh and scp operation;
(2)deploy(machineList，dirList)
Function: will monitor the angle and originally be deployed on each branch control unit;
(3)start_monitor(machineList，dirList)
Function: start each in batches and divide the control unit;
(4)stop_monitor(machineList，dirList)
Function: stop each in batches and divide the control unit;
(5)get_machine_info(machineList)
Function: obtain each and divide the machine physical parameter of controlling on the unit;
(6)get_average(tablePathIn，secs，tablePathOut)
Function: the data after the format are asked its mean value in different time sections;
(7)get_min(tablePathIn，secs，tablePathOut)
Function: the data after the format are asked its minimum value in different time sections;
(8)get_max(tablePathIn，secs，tablePathOut)
Function: the data after the format are asked its maximum in different time sections;
(9)get_rate(tablePathIn，secs，tablePathOut)
Function: ask a certain columns according to the ratios in different intervals to the data after the format.
Fig. 4 is the structured flowchart of the monitoring module of another embodiment of the present invention.As shown in Figure 4, on the basis of embodiment shown in Figure 3, this monitoring module 160 also comprises journal processing unit 163.Journal processing unit 163 links to each other with Master Control Unit 161, is used for the performance data that gets access to is resolved and added up.Can comprise following power function particularly:
(1)get_average_press(logPath，secs)
Function: the mean value that obtains the different time sections internal pressure;
(2)get_max_press(logPath，secs)
Function: the maximum of obtaining the different time sections internal pressure;
(3)get_min_press(logPath，secs)
Function: the minimum value of obtaining the different time sections internal pressure;
(4)get_average_parm(logPath，secs，key)
Function: the mean value that obtains certain parameter in the different time sections;
(5)get_max_parm(logPath，secs，key)
Function: the maximum of obtaining certain parameter in the different time sections;
(6)get_min_parm(logPath，secs，key)
Function: the minimum value of obtaining certain parameter in the different time sections;
(7)get_rate_parm(logPath，secs，key)
Function: obtain certain parameter in different interval ratios.
The power function that should be understood that above-mentioned Master Control Unit, branch control unit and journal processing unit only is schematic example.Those skilled in the art can dispose different monitored item according to actual needs, just uses different power functions, and these modifications and variations all should be included in protection scope of the present invention.
Performance Test System according to the distributed type assemblies of the embodiment of the invention, bridge links to each other with upstream cluster, tested module and a plurality of downstreams cluster (rear end cluster) respectively, can real simulation line upper topology and data volume by a bridge, need not to build real cluster environment can test the performance of tested module, improve test accuracy and testing efficiency, save cost.
Below with reference to the bridge of Figure of description description according to the embodiment of the invention.
Fig. 5 is the structured flowchart of the bridge of one embodiment of the invention.As shown in Figure 5, the bridge according to the embodiment of the invention comprises mode selection module 310, communication module 320, records module 330 and analog module 340.
Particularly, mode selection module 310 is used for receiving recording mode or the test pattern of user's input.Under recording mode, communication module 320 receives the first pressure request of upstream cluster transmission and this first pressure request is sent to tested module, and receive tested module according to the second pressure request of first pressure request generation and this second pressure request is forwarded to the rear end cluster, and receive the communication data packet that the rear end cluster returns according to the second pressure request; Record module 330 and link to each other with communication module 320, the communication data packet of preserving the first pressure request and returning.Under test pattern, communication module 320 receives the predetermined pressure that tested module is transmitted.Analog module 340 respectively with communication module 320 with record module 330 and link to each other, find the return data bag of this predetermined pressure correspondence in the communication data packet of in recording module 330, preserving according to predetermined pressure.Then, communication module 320 is sent to tested module with this packet.
The communication protocol that should be understood that disparate modules is different, and the structure of preserving the binary data packets of getting off is different.When carrying out performance test, go to comprise in the binary data packets of a lot of request bags according to module characteristic and find out each request bag.Thus, in one embodiment of the invention, when the data recording bag, each communications packets all has following record format:
Wherein, devil's numeral is used for the head of a protocol package of sign, avoids the binary data entanglement; Dump ID is used for identifying the numbering of the communication data packet of preserving; Key is designated the key that above-described assurance playback authenticity exists; Version number is used for the difference disparate modules; Reserved field is used for subsequent expansion to be used, and for example deposits the timestamp of dump, convenient follow-up according to the timestamp playback; Packet length is used for the physical length of the packet of sign access; The packet body of Dump is used for the actual communication data packet of depositing, as pressure request data package or the communication data packet returned.Thus, when carrying out performance test, resolve according to the record format of said fixing, can resolve each bar record easily, and then read each request bag, send predetermined pressure then to tested module.
In addition, in one embodiment of the invention, analog module 240 is according to key sign ﹠amp; The form that value is right is set up the communication data packet that dictionary is preserved the first pressure request and returned, and wherein value is the communication data packet of returning, and the key sign is the key sign that comprises in the above-mentioned communications packets.Thus, can be earlier by resolving pressure request bag or obtaining a key to doing signature after the processing of pressure request bag, analog module 240 is searched the return data bag of binding by this unique key in dictionary then, and it is consistent with operation flow on the line to reach the flow playback with this.
Bridge according to the embodiment of the invention, by linking to each other with upstream cluster, tested module and rear end cluster, artificial line upper topology and quantity truly, need not to build real cluster environment can test the performance of tested module, improve test accuracy and testing efficiency, save cost.
Below with reference to the performance test methods of Figure of description description according to the distributed type assemblies of the embodiment of the invention.
Fig. 6 is the flow chart of performance test methods of the distributed type assemblies of one embodiment of the invention.
As shown in Figure 6, the performance test methods according to the distributed type assemblies of the embodiment of the invention comprises the steps.
Step S101, under recording mode, bridge receives the first pressure request that the upstream cluster sends, and this first pressure request is sent to tested module, and preserves this first pressure request.
Step S102, tested module receive the first pressure request, and generate the second pressure request according to this first pressure request, and the second pressure request is sent to bridge.
For example, the first pressure request is search " Baidu, patent, examination reports ", and the second pressure request of generation is search " patent, examination reports ".
Step S103, bridge receive the second pressure request and the second pressure request are sent to the rear end cluster, and wherein the rear end cluster comprises a plurality of downstreams cluster.
Step S104, a plurality of downstreams cluster returns corresponding packet to bridge according to the second pressure request.
Step S105, bridge receives and preserves the packet that a plurality of downstreams cluster returns.
Above-mentioned steps S101 is the recording process of data to step S105.By the packet that recording process is preserved, can simulate the whole rear end of tested module cluster, realize the performance test of follow-up tested module.
Step S106, under test pattern, the first pressure request of preserving according to bridge by compression tools sends predetermined pressure to tested module.
Should be understood that for compression tools the communication protocol of disparate modules is different, the structure of preserving the binary data packets of getting off is different.Compression tools will go to comprise in the binary data packets of a lot of request bags according to module characteristic finds out each request bag.Thus, in one embodiment of the invention, when the data recording bag, each communications packets all has following record format:
Wherein, devil's numeral is used for the head of a protocol package of sign, avoids the binary data entanglement; Dump ID is used for identifying the numbering of the communication data packet of preserving; Key is designated the key that above-described assurance playback authenticity exists; Version number is used for the difference disparate modules; Reserved field is used for subsequent expansion to be used, and for example deposits the timestamp of dump, convenient follow-up according to the timestamp playback; Packet length is used for the physical length of the packet of sign access; The packet body of Dump is used for the actual communication data packet of depositing, as pressure request data package or the communication data packet returned.Thus, compression tools is resolved according to the record format of said fixing, can resolve each bar record easily, and then reads each request bag, sends control module by pressure then and sends predetermined pressure to tested module.
Step S107, tested module is forwarded to bridge with predetermined pressure.
Step S108, bridge finds corresponding return data bag according to predetermined pressure in the communication data packet of preserving.
In one embodiment of the invention, according to key sign ﹠amp; The form that value is right is set up the communication data packet that dictionary is preserved the first pressure request and returned, and wherein value is the communication data packet of returning, and the key sign is the key sign that comprises in the above-mentioned communications packets.Thus, can search the return data bag of binding then by this unique key in dictionary earlier by resolving pressure request bag or obtaining a key to doing signature after the processing of pressure request bag, it is consistent with operation flow on the line to reach the flow playback with this.
Step S109 in test process, monitors the performance data of tested module place server, and carries out performance evaluation according to this performance data.
Should understand, the performance index that disparate modules need be monitored are different, general monitoring tools is provided in one embodiment of the invention, the user only needs the configuration monitoring item, monitoring tools is the corresponding data of monitoring automatically, performance test is automatically analyzed these monitor datas after finishing as requested.
Particularly, the universal monitor instrument is divided into two unit, Master Control Unit and a plurality of branch controls unit.The deployment that Master Control Unit is responsible for monitoring, start, stop and the analytical work of data.Each divides the control unit all to link to each other with Master Control Unit, periodically collects data according to the control command of Master Control Unit, and the data of collecting are formatd.
In one embodiment of the invention, divide the control data that the unit is collected to comprise: the vmstat data; The situation that takies of CPU, internal memory and file handle; The iostat data; The daily record of tested module; And network data etc.
The concrete function of Master Control Unit can comprise: set up the two-way trust relationship between described Master Control Unit and each the described branch control unit; Monitoring script is deployed on each described branch control unit; Start or stop described a plurality of branch controls unit in batches; Obtain the machine physical parameter of each described branch control unit; And according to the performance data after formaing calculate its mean value, minimum value, maximum and a certain columns in different time sections according in different interval ratios etc.
In another embodiment of the present invention, the universal monitor instrument also can comprise journal processing unit.This journal processing unit links to each other with Master Control Unit, is used for the performance data that gets access to is resolved and added up.For example, obtain mean value, maximum and the minimum value etc. of different time sections internal pressure.
Performance test methods according to the distributed type assemblies of the embodiment of the invention, by the communication data packet between bridge extracting upstream cluster, tested module and the rear end cluster, thereby can real simulation line upper topology and data volume by a bridge, need not to build real cluster environment and can carry out performance test to tested module, improve test accuracy and testing efficiency, save cost.
To sum up, system and method and bridge according to the performance test of the distributed type assemblies of the embodiment of the invention have following beneficial effect at least:
1, consistent on topological structure and the line;
According to the annexation topological structure of each module of cluster on the line, make up test cluster under the line, guarantee on the residing upstream and downstream environment of tested module and the line consistent.
2, suitable on data volume and the line;
The packet that the simulation cluster uses derive from that tested module run mode dump gets off or by bridge in the investigation environment under the dump, basically identical on data volume and data content and the line.
3, consistent on playback data stream and the line;
The packet that the simulation cluster uses has been done mark in dump following time to each packet.Each request and return results are set up dictionary with a unique sign, during playback, bridge obtains unique sign according to request bag, and by this unique return data bag of searching binding in the dictionary that is identified at, it is consistent with operation flow on the line to reach the flow playback with this then.
4, improve the performance test accuracy;
Use the true cluster of artificial line upper topology and data volume, improve the performance test accuracy.
5, improve testing efficiency;
Need not to build real cluster environment, improve testing efficiency.
6, save cost;
Need not to use true cluster environment, use the whole rear end of a machine simulation cluster, save machine resources in a large number, save cost.
Should be appreciated that each several part of the present invention can realize with hardware, software, firmware or their combination.In the above-described embodiment, a plurality of steps or method can realize with being stored in the memory and by software or firmware that suitable instruction execution system is carried out.For example, if realize with hardware, the same in another embodiment, in the available following technology well known in the art each or their combination realize: have for the discrete logic of data-signal being realized the logic gates of logic function, application-specific integrated circuit (ASIC) with suitable combinational logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) etc.
In the description of this specification, concrete feature, structure, material or characteristics that the description of reference term " embodiment ", " some embodiment ", " example ", " concrete example " or " some examples " etc. means in conjunction with this embodiment or example description are contained at least one embodiment of the present invention or the example.In this manual, the schematic statement to above-mentioned term not necessarily refers to identical embodiment or example.And concrete feature, structure, material or the characteristics of description can be with the suitable manner combination in any one or more embodiment or example.
Although illustrated and described embodiments of the invention, for the ordinary skill in the art, be appreciated that without departing from the principles and spirit of the present invention and can carry out multiple variation, modification, replacement and modification to these embodiment that scope of the present invention is by claims and be equal to and limit.
Claims (17)
1. the Performance Test System of a distributed type assemblies is characterized in that, comprising:
The upstream cluster, described upstream cluster is used for the first pressure request that sends;
Tested module;
The rear end cluster, described rear end cluster comprises a plurality of downstreams cluster;
Bridge, described bridge links to each other with described upstream cluster, tested module and rear end cluster respectively, described bridge is used for receiving the described first pressure request and the described first pressure request being sent to described tested module, receive the second pressure request that described tested module generates according to the described first pressure request and the described second pressure request is forwarded to described a plurality of downstreams cluster, and preserve the communication data packet that the described first pressure request and described a plurality of downstreams cluster return according to the described second pressure request;
Compression tools, the described first pressure request that described compression tools is used for preserving according to described bridge sends predetermined pressure to described tested module, wherein said tested module is forwarded to described bridge with described predetermined pressure, and described bridge is searched the corresponding packet of described predetermined pressure and is back to described tested module according to described predetermined pressure in the communication data packet of returning of described preservation; And
Monitoring module, described monitoring module and described tested module are positioned on the same server, are used for the performance data of the described server of monitoring, and carry out performance evaluation according to described performance data.
2. the Performance Test System of distributed type assemblies as claimed in claim 1 is characterized in that, described communication data packet comprises:
Devil's numeral is used for identity protocol packet header;
Dump ID is for the numbering that identifies the communication data packet of preserving;
The key sign is used for sign pressure request and corresponding return data bag thereof;
Version number is used for the different tested module of difference;
Reserved field is used for the described communication data packet of expansion;
Packet length is for the physical length of the described communication data packet of sign; With
The data inclusion of Dump is used for depositing described communication data packet.
3. the Performance Test System of distributed type assemblies as claimed in claim 2 is characterized in that, described bridge is according to key according to ﹠amp; The form that value is right is set up dictionary to preserve the described first pressure request and the described communication data packet of returning, and wherein value is the described communication data packet of returning, and key is designated the key sign in the described first pressure request bag.
4. the Performance Test System of distributed type assemblies as claimed in claim 1 is characterized in that, described monitoring module specifically comprises:
Master Control Unit, described Master Control Unit are used for the startup of configuration monitoring item, control monitoring and stop and carrying out performance evaluation according to described performance data; And
A plurality of branch controls unit, each divides the control unit all to link to each other with described Master Control Unit, is used for collecting according to the control cycle ground of described Master Control Unit the performance data of the server at described tested module place, and described performance data is formatd.
5. as the Performance Test System of claim 1 or 4 described distributed type assemblies, it is characterized in that described performance data comprises:
The vmstat data;
CPU, internal memory and file handle/socket handle etc. takies situation;
The iostat data;
The daily record of tested module operation; And
Network data.
6. the Performance Test System of distributed type assemblies as claimed in claim 4 is characterized in that, described Master Control Unit specifically comprises:
Set up the two-way trust relationship between described Master Control Unit and each the described branch control unit;
Monitoring script is deployed on each described branch control unit;
Start or stop described a plurality of branch controls unit in batches;
Obtain the machine physical parameter of each described branch control unit; And
According to the performance data after formaing calculate its mean value, minimum value, maximum and a certain columns in different time sections according in different interval ratios.
7. the Performance Test System of distributed type assemblies as claimed in claim 4 is characterized in that, described monitoring module also comprises:
Journal processing unit, described journal processing unit links to each other with described Master Control Unit, is used for described performance data is resolved and added up.
8. a bridge is characterized in that, comprising:
Mode selection module, described mode selection module is used for receiving recording mode or the test pattern of user's input;
Communication module, described communication module is used for receiving the first pressure request of upstream cluster transmission and the described first pressure request is sent to tested module under described recording mode, and receive described tested module according to the second pressure request of described first pressure request generation and the described second pressure request is forwarded to the rear end cluster, and receive the communication data packet that described rear end cluster returns according to the described second pressure request; Described communication module also is used for receiving the predetermined pressure that described tested module is transmitted under described test pattern;
Record module, the described module of recording links to each other with described communication module, is used for preserving the described first pressure request and the described communication data packet of returning; And
Analog module, described analog module link to each other with the described module of recording with described communication module respectively, be used for recording the return data bag that communication data packet that module preserves is searched described predetermined pressure correspondence according to described predetermined pressure described,
Wherein said communication module also is used for the return data bag of described predetermined pressure correspondence is sent to described tested module.
9. bridge as claimed in claim 8 is characterized in that, described communication data packet comprises:
Devil's numeral is used for identity protocol packet header;
Dump ID is for the numbering that identifies the communication data packet of preserving;
The key sign is used for sign pressure request and corresponding return data bag thereof;
Version number is used for the different tested module of difference;
Reserved field is used for the described communication data packet of expansion;
Packet length is for the physical length of the described communication data packet of sign; With
The data inclusion of Dump is used for depositing described communication data packet.
10. bridge as claimed in claim 9 is characterized in that, described analog module also is used for according to key sign ﹠amp; The form that value is right is set up dictionary to preserve the described first pressure request and the described communication data packet of returning, and wherein value is the described communication data packet of returning, and key is designated the key sign in the described first pressure request bag.
11. the performance test methods of a distributed type assemblies is characterized in that, may further comprise the steps:
(a) under recording mode, bridge receives the first pressure request that the upstream cluster sends, and the described first pressure request is sent to tested module, and preserves the described first pressure request;
(b) described tested module receives the described first pressure request, and generates the second pressure request according to the described first pressure request, and the described second pressure request is sent to described bridge;
(c) described bridge receives the described second pressure request and the described second pressure request is sent to the rear end cluster, and wherein said rear end cluster comprises a plurality of downstreams cluster;
(d) described a plurality of downstreams cluster returns corresponding packet to described bridge according to the described second pressure request;
(e) described bridge receives described and preserves the described packet that returns;
(f) under test pattern, the first pressure request of preserving according to described bridge by compression tools sends predetermined pressure to described tested module;
(g) described tested module is forwarded to described bridge with described predetermined pressure, and receives the corresponding return data bag that described bridge finds in the communication data packet of described preservation according to described predetermined pressure;
(h) in test process, monitor the performance data of the server at described tested module place, and carry out performance evaluation according to described performance data.
12. the performance test methods of distributed type assemblies as claimed in claim 11 is characterized in that, described communication data packet comprises:
Devil's numeral is used for identity protocol packet header;
Dump ID is for the numbering that identifies the communication data packet of preserving;
The key sign is used for sign pressure request and corresponding return data bag thereof;
Version number is used for the different tested module of difference;
Reserved field is used for the described communication data packet of expansion;
Packet length is for the physical length of the described communication data packet of sign; With
The data inclusion of Dump is used for depositing described communication data packet.
13. the performance test methods of distributed type assemblies as claimed in claim 12 is characterized in that, also comprises step:
Described bridge is according to key according to ﹠amp; The form that value is right is set up dictionary to preserve the described first pressure request and the described communication data packet of returning, and wherein value is the described communication data packet of returning, and key is designated the key sign in the described first pressure request bag.
14. the performance test methods of distributed type assemblies as claimed in claim 11 is characterized in that, described step (h) realizes that by the universal monitor instrument wherein said universal monitor instrument comprises:
Master Control Unit, described Master Control Unit are used for the startup of configuration monitoring item, control monitoring and stop and carrying out performance evaluation according to described performance data; And
A plurality of branch controls unit, each divides the control unit all to link to each other with described Master Control Unit, is used for collecting according to the control cycle ground of described Master Control Unit the performance data of the server at described tested module place, and described performance data is formatd.
15. the performance test methods of distributed type assemblies as claimed in claim 14 is characterized in that, described performance data comprises:
The vmstat data;
The situation that takies of CPU, internal memory and file handle;
The iostat data;
The running log of tested module; And
Network data.
16. the performance test methods of distributed type assemblies as claimed in claim 14 is characterized in that, described Master Control Unit specifically is used for:
Set up the two-way trust relationship between described Master Control Unit and each the described branch control unit;
Monitoring script is deployed on each described branch control unit;
Start or stop described a plurality of branch controls unit in batches;
Obtain the machine physical parameter of each described branch control unit; And
According to the performance data after formaing calculate its mean value, minimum value, maximum and a certain columns in different time sections according in different interval ratios.
17. the performance test methods of distributed type assemblies as claimed in claim 14 is characterized in that, described universal monitor instrument also comprises:
Journal processing unit, described journal processing unit links to each other with described Master Control Unit, is used for described performance data is resolved and added up.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210043849.8A CN103297285B (en) | 2012-02-23 | 2012-02-23 | The Performance Test System of distributed type assemblies, method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210043849.8A CN103297285B (en) | 2012-02-23 | 2012-02-23 | The Performance Test System of distributed type assemblies, method and apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103297285A true CN103297285A (en) | 2013-09-11 |
| CN103297285B CN103297285B (en) | 2016-03-16 |
Family
ID=49097633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210043849.8A Active CN103297285B (en) | 2012-02-23 | 2012-02-23 | The Performance Test System of distributed type assemblies, method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103297285B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461856A (en) * | 2013-09-22 | 2015-03-25 | 阿里巴巴集团控股有限公司 | Performance test method, device and system based on cloud computing platform |
| CN105515883A (en) * | 2014-09-22 | 2016-04-20 | 阿里巴巴集团控股有限公司 | Test method and system |
| CN106055486A (en) * | 2016-08-19 | 2016-10-26 | 浪潮(北京)电子信息产业有限公司 | Automatic operation maintenance method and platform of distributed file system |
| CN107302475A (en) * | 2017-07-06 | 2017-10-27 | 郑州云海信息技术有限公司 | The method of testing and device of a kind of Based on Distributed storage cluster |
| CN108494613A (en) * | 2018-01-22 | 2018-09-04 | 北京交通大学 | A kind of test system and method to interconnect between police digital cluster system |
| CN110166511A (en) * | 2018-02-13 | 2019-08-23 | 北京嘀嘀无限科技发展有限公司 | Data processing method and system |
| CN110808882A (en) * | 2019-11-06 | 2020-02-18 | 武汉联影医疗科技有限公司 | Test system and method |
| CN113656268A (en) * | 2021-07-30 | 2021-11-16 | 北京百度网讯科技有限公司 | Performance test method and device of business service, electronic equipment and storage medium |
| CN114691707A (en) * | 2022-05-27 | 2022-07-01 | 云账户技术(天津)有限公司 | Online cluster service configuration method, system, network equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070118633A1 (en) * | 2005-11-10 | 2007-05-24 | Kumar C P V | Cluster command testing |
| CN101127801A (en) * | 2007-08-29 | 2008-02-20 | 中兴通讯股份有限公司 | A method for testing relay performance of group call |
| US8037109B2 (en) * | 2003-06-30 | 2011-10-11 | Microsoft Corporation | Generation of repeatable synthetic data |
| CN202033748U (en) * | 2011-04-22 | 2011-11-09 | 阿里巴巴集团控股有限公司 | Search engine performance test system |
-
2012
- 2012-02-23 CN CN201210043849.8A patent/CN103297285B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8037109B2 (en) * | 2003-06-30 | 2011-10-11 | Microsoft Corporation | Generation of repeatable synthetic data |
| US20070118633A1 (en) * | 2005-11-10 | 2007-05-24 | Kumar C P V | Cluster command testing |
| CN101127801A (en) * | 2007-08-29 | 2008-02-20 | 中兴通讯股份有限公司 | A method for testing relay performance of group call |
| CN202033748U (en) * | 2011-04-22 | 2011-11-09 | 阿里巴巴集团控股有限公司 | Search engine performance test system |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461856A (en) * | 2013-09-22 | 2015-03-25 | 阿里巴巴集团控股有限公司 | Performance test method, device and system based on cloud computing platform |
| CN105515883A (en) * | 2014-09-22 | 2016-04-20 | 阿里巴巴集团控股有限公司 | Test method and system |
| CN106055486A (en) * | 2016-08-19 | 2016-10-26 | 浪潮(北京)电子信息产业有限公司 | Automatic operation maintenance method and platform of distributed file system |
| CN107302475A (en) * | 2017-07-06 | 2017-10-27 | 郑州云海信息技术有限公司 | The method of testing and device of a kind of Based on Distributed storage cluster |
| CN108494613A (en) * | 2018-01-22 | 2018-09-04 | 北京交通大学 | A kind of test system and method to interconnect between police digital cluster system |
| CN110166511A (en) * | 2018-02-13 | 2019-08-23 | 北京嘀嘀无限科技发展有限公司 | Data processing method and system |
| CN110808882A (en) * | 2019-11-06 | 2020-02-18 | 武汉联影医疗科技有限公司 | Test system and method |
| CN113656268A (en) * | 2021-07-30 | 2021-11-16 | 北京百度网讯科技有限公司 | Performance test method and device of business service, electronic equipment and storage medium |
| CN113656268B (en) * | 2021-07-30 | 2024-02-06 | 北京百度网讯科技有限公司 | Performance test method and device for business service, electronic equipment and storage medium |
| CN114691707A (en) * | 2022-05-27 | 2022-07-01 | 云账户技术(天津)有限公司 | Online cluster service configuration method, system, network equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103297285B (en) | 2016-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103297285A (en) | Distributed cluster performance test system, method and device | |
| CN103051500B (en) | A kind of method of testing of service network system and system | |
| CN102946616B (en) | Internet of things middleware performance testing system and testing method | |
| CN106292626B (en) | Data acquisition device and method for new-energy automobile test evaluation | |
| CN112180897B (en) | Relay protection/measurement and control device automatic test system and method based on universal test template | |
| CN107423211A (en) | A kind of SDNS interfaces automatization test system and method | |
| CN106874187A (en) | code coverage collection method and device | |
| CN107479540B (en) | Method for diagnosing faults and system | |
| CN103188719B (en) | Medium-high speed sensor network test macro | |
| CN108195597A (en) | A kind of reconfigurable vehicle field dynamic Multi parameters Test Macro | |
| CN110932887A (en) | BMC debugging method, system and device | |
| CN104793612A (en) | Unmanned aerial vehicle ground control station testing and data acquiring method and system thereof | |
| Smarsly et al. | IFC-based modeling of cyber-physical systems in civil engineering | |
| CN109802842B (en) | Application topology generation method and related equipment | |
| CN102055621A (en) | Method and apparatus for the efficient correlation of network traffic to related packets | |
| CN112255548B (en) | Battery management system test method and system thereof | |
| CN102710459B (en) | Site joint debugging system of automated system | |
| CN113012421A (en) | Method and system for remote operation and maintenance of HPLC communication module | |
| CN109450103A (en) | Condition detection method, device and the intelligent terminal of pressing plate | |
| CN100517243C (en) | Open type system for collecting testing data and method thereof | |
| WO2012009901A1 (en) | Data acquisition method in network resource estimation and system thereof | |
| KR101116878B1 (en) | Information changing system of ECU | |
| CN115801828A (en) | Intelligent substation secondary data analysis system based on platform and terminal | |
| CN108563664A (en) | A kind of real-time data processing method at industrial equipment end | |
| CN112866047A (en) | Wireless data detection device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |