CN102291449A - Method for testing and adjusting cluster storage system performance based on synchronous strategy - Google Patents
Method for testing and adjusting cluster storage system performance based on synchronous strategy Download PDFInfo
- Publication number
- CN102291449A CN102291449A CN2011102258122A CN201110225812A CN102291449A CN 102291449 A CN102291449 A CN 102291449A CN 2011102258122 A CN2011102258122 A CN 2011102258122A CN 201110225812 A CN201110225812 A CN 201110225812A CN 102291449 A CN102291449 A CN 102291449A
- Authority
- CN
- China
- Prior art keywords
- module
- control module
- client node
- performance
- module group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
- Test And Diagnosis Of Digital Computers (AREA)
Abstract
The invention provides a method for testing and adjusting cluster storage system performance based on a synchronous strategy. The method comprises a data node module group, a control module group and a client node module group, wherein the three module groups cooperate and communicate with each other, and take the control module group as the core to finish the system test and monitoring tasks together; the control module group is used for scheduling load and balancing all modules, and collecting the test result and synchronizing the time of all modules; the data node module group is used for collecting the data node monitoring information and realizing time synchronization; and the client node module group is used for simulating the read/write operation, collecting the monitored data of the client node and realizing time synchronization.
Description
Technical field
The present invention relates to a kind of Computer Applied Technology field, specifically a kind of based on the test of synchronization policy and the method for adjustment cluster storage system performance, the present invention is applicable in the popular cluster storage system of present industry.
Background technology
Computer cluster be called for short cluster be a kind of computer system it couple together highly closely cooperation by one group of loose integrated computer software and hardware and finish evaluation work.At present, because the dramatic growth of individual and business data, the cluster storage system development rapidly, but be to test the aspect at present for cluster storage system, substantially still prolong the method according to the individual node performance test, the method that the The performance test results of each node is added up is come the performance of gauge cluster storage system then.This method, though simple, rigorous inadequately and science.
At present, popular performance testing tool such as iozone, iometer etc., though can multithreading, multi-process test readwrite bandwidth also provides many test options, can test with machine-readable, write at random, the performance index of sequential read, sequential write, but these instruments are just done test at the readwrite bandwidth of individual node.Test the performance of whole group system if desired, add up if just simply the The performance test results of a plurality of nodes is done one, because the asynchronism that the process of the asynchronism of time and each node finishes, test result is often not accurate enough.Therefore, based on cluster storage system architecture design testing tool, just extremely urgent.
Summary of the invention
The purpose of this invention is to provide a kind of based on the test of synchronization policy and the method for adjustment cluster storage system performance.
Purpose method of the present invention realizes that in the following manner comprise back end module group, control module group and client node module group three big module groups: three big module groups cooperatively interact and communicate by letter,
WithThe control module group is a core, finish system testing and monitor task jointly, wherein, the control module group is responsible for load dispatch and balanced all modules, collect the time of test result and synchronous all modules, back end module group is used to collect back end monitor message and time synchronized, and client node module group is used to simulate the monitor data of read-write operation, collection client node and realize time synchronized, wherein:
Back end module group comprises data performance monitor module, time synchronized module and three submodules of communication control module, wherein, the data performance monitor module is used to collect offered load, the cpu load information of back end, the time synchronized module is used for and control module group lock in time, and the information interaction of data performance monitor module and time synchronized module and control module group is to finish by the communication control module of communication control module and control module group;
The control module group comprises the load dispatch module, and the information gathering module, computing module, time synchronized control module, communication control module 1 and communication control module 2 are formed as a result; The control module group comprises load dispatch module and time synchronized control module, can make balanced more and the better consistent and synchronism of read-write operation on each node, the load dispatch module receive computing module as a result as a result the time, the read-write analog module of each client will be dispatched and adjust to this module, make each client node read-write operation balanced more, guarantee the synchronism of each node time and read-write operation;
Client node module group comprises read-write analog module, performance monitoring module, time synchronized module and communication module, the read-write analog module is responsible for simulating read-write operation on client node, and accept the scheduling of load dispatch module, adjust read-write operation, the performance monitoring module is responsible for collecting performance, cpu load, the network load information of client node.
Excellent effect of the present invention is, this method adapts to the framework and the testing requirement of cluster storage system, can monitor simultaneously the readwrite performance of each node, be easy to find the performance bottleneck of group system, and owing to introduced the function of time synchronized and load dispatch, make the test on each node to finish synchronously, the result who tests out is more reliable, more science.Under this pattern,,, be easy to find bottleneck, thereby realize quick trouble-shooting, the purpose of fixing a breakdown if when therefore the performance of system goes wrong because testing tool all has traffic monitoring to all nodes of whole system.Debugging and failure diagnosis for system play a key effect.
During this Software deployment, need be in the back end deploy back end module group of cluster storage system, in client node deploy client node module group.The control module group can be deployed on the interior arbitrary node of cluster as required.
Description of drawings
Fig. 1 cluster memory property testing software Organization Chart;
Fig. 2 cluster storage system Organization Chart.
Embodiment
Explain below with reference to Figure of description method of the present invention being done.
The inventive method comprises three big module groups: back end module group, control module group and client node module group.Three big module groups cooperatively interact and communicate by letter, and are core with pool and control module group, finish system testing and monitor task jointly.Wherein, the control module group is responsible for load dispatch and balanced all modules, collects the time of test result and synchronous all modules.Back end module group is used to collect back end monitor message and time synchronized.Client node module group is used to simulate read-write operation, collects the monitor data of client node and realizes time synchronized.
Back end module group comprises three submodules: data performance monitor module, time synchronized module and communication control module.Wherein, the data performance monitor module is used to collect information such as the offered load, cpu load of back end, and the time synchronized module is used for and control module group lock in time.The information interaction of data performance monitor module and time synchronized module and control module group is to finish by the communication control module 2 of communication control module and control module group.
The control module group comprises the load dispatch module, the information gathering module, computing module, time synchronized control module, communication control module 1 and communication control module 2 are formed as a result, wherein, the information gathering module is responsible for collecting the monitor message of each module, feed back to computing module as a result then, the performance that performance adds up and all back end monitor that calculates all client nodes respectively by computing module as a result adds up then.Simultaneously, computing module will calculate the performance difference between each client node as a result, at this moment, if each client node performance difference surpasses certain value, just call the read-write analog module of load dispatch module schedules and balanced each client node, make each client node balancing performance.Wherein, communication control module 1 is responsible for and the communicating by letter of all client node module groups.Communication control module 2 is responsible for and all back end module group communications.The time synchronized control module is responsible for the time of synchronous whole system, guarantees the consistency and the synchronism of test.The core of this method that Here it is, utilize the time synchronized control module guarantee each node time synchronously, utilize the load dispatch module to guarantee the synchronism of the read-write operation of each node.
Client node module group comprises read-write analog module, performance monitoring module, time synchronized module and communication control module.The read-write analog module is responsible for simulating read-write operation on client node, and accepts the scheduling of load dispatch module, adjusts read-write operation.The performance monitoring module is responsible for collecting information such as the performance, cpu load, offered load of client node.
Method of the present invention adapts to the framework and the testing requirement of cluster storage system, can monitor simultaneously the readwrite performance of each node, be easy to find the performance bottleneck of group system, and owing to introduced the function of time synchronized and load dispatch, make the test on each node to finish synchronously, the result who tests out is more reliable, more science.
Except that the described technical characterictic of specification, be the known technology of those skilled in the art.
Claims (1)
1. one kind based on the test of synchronization policy with adjust the method for cluster storage system performance, and it is characterized in that comprising back end module group, control module group and client node module group three big module groups: three big module groups cooperatively interact and communicate by letter,
WithThe control module group is a core, finish system testing and monitor task jointly, wherein, the control module group is responsible for load dispatch and balanced all modules, collect the time of test result and synchronous all modules, back end module group is used to collect back end monitor message and time synchronized, and client node module group is used to simulate the monitor data of read-write operation, collection client node and realize time synchronized, wherein:
Back end module group comprises data performance monitor module, time synchronized module and three submodules of communication control module, wherein, the data performance monitor module is used to collect offered load, the cpu load information of back end, the time synchronized module is used for and control module group lock in time, and the information interaction of data performance monitor module and time synchronized module and control module group is to finish by the communication control module of communication control module and control module group;
The control module group comprises the load dispatch module, the information gathering module, computing module as a result, the time synchronized control module, communication control module 1 and communication control module 2 are formed, wherein, the information gathering module is responsible for collecting the monitor message of each module, feed back to computing module as a result, the performance that performance adds up and all back end monitor that calculates all client nodes respectively by computing module as a result adds up then, simultaneously, computing module will calculate the performance difference between each client node as a result, at this moment, if each client node performance difference surpasses certain value, just call the read-write analog module of load dispatch module schedules and balanced each client node, make each client node balancing performance, wherein, communication control module 1 is responsible for and the communicating by letter of all client node module groups, communication control module 2 is responsible for and all back end module group communications, the time synchronized control module is responsible for the time of synchronous whole system, guarantee the consistency and the synchronism of test, utilize the time synchronized control module guarantee each node time synchronously, utilize the load dispatch module to guarantee the synchronism of the read-write operation of each node;
Client node module group comprises read-write analog module, performance monitoring module, time synchronized module and communication module, the read-write analog module is responsible for simulating read-write operation on client node, and accept the scheduling of load dispatch module, adjust read-write operation, the performance monitoring module is responsible for collecting performance, cpu load, the network load information of client node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110225812.2A CN102291449B (en) | 2011-08-08 | 2011-08-08 | Method for testing and adjusting cluster storage system performance based on synchronous strategy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110225812.2A CN102291449B (en) | 2011-08-08 | 2011-08-08 | Method for testing and adjusting cluster storage system performance based on synchronous strategy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102291449A true CN102291449A (en) | 2011-12-21 |
| CN102291449B CN102291449B (en) | 2014-04-02 |
Family
ID=45337550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110225812.2A Active CN102291449B (en) | 2011-08-08 | 2011-08-08 | Method for testing and adjusting cluster storage system performance based on synchronous strategy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102291449B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710446A (en) * | 2012-06-07 | 2012-10-03 | 浪潮电子信息产业股份有限公司 | Performance tuning method of colony memory system based on synchronous strategy |
| CN103856517A (en) * | 2012-11-30 | 2014-06-11 | 英业达科技有限公司 | Synchronous equalization system and method thereof |
| CN103914452A (en) * | 2012-12-30 | 2014-07-09 | 航天信息股份有限公司 | Distributed file system time error correction method and system |
| CN104239026A (en) * | 2013-06-21 | 2014-12-24 | 伊姆西公司 | Method and device for measuring performance of storage system |
| CN106776166A (en) * | 2016-12-06 | 2017-05-31 | 郑州云海信息技术有限公司 | Ceph storage clusters method of testing and system under a kind of RDMA patterns |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030110172A1 (en) * | 2001-10-24 | 2003-06-12 | Daniel Selman | Data synchronization |
| CN101079896A (en) * | 2007-06-22 | 2007-11-28 | 西安交通大学 | A multi-availability mechanism coexistence framework of concurrent storage system |
| US20090106179A1 (en) * | 2007-10-18 | 2009-04-23 | Friedlander Robert R | System and method for the longitudinal analysis of education outcomes using cohort life cycles, cluster analytics-based cohort analysis, and probablistic data schemas |
| CN101783755A (en) * | 2010-02-24 | 2010-07-21 | 浪潮(北京)电子信息产业有限公司 | Method and device thereof for synchronically testing performance of cluster parallel file system |
-
2011
- 2011-08-08 CN CN201110225812.2A patent/CN102291449B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030110172A1 (en) * | 2001-10-24 | 2003-06-12 | Daniel Selman | Data synchronization |
| CN101079896A (en) * | 2007-06-22 | 2007-11-28 | 西安交通大学 | A multi-availability mechanism coexistence framework of concurrent storage system |
| US20090106179A1 (en) * | 2007-10-18 | 2009-04-23 | Friedlander Robert R | System and method for the longitudinal analysis of education outcomes using cohort life cycles, cluster analytics-based cohort analysis, and probablistic data schemas |
| CN101783755A (en) * | 2010-02-24 | 2010-07-21 | 浪潮(北京)电子信息产业有限公司 | Method and device thereof for synchronically testing performance of cluster parallel file system |
Non-Patent Citations (3)
| Title |
|---|
| FRANK SCHMUCK ET.AL: "GPFS:A Shared-Disk File System for Large Computing Clusters", 《PROCEEDINGS OF THE CONFERENCE ON FILE AND STORAGE TECHNOLOGIES》, 30 January 2002 (2002-01-30), pages 231 - 244 * |
| 汪璐等: "基于Lustre的BES集群存储系统", 《核电子学与探测技术》, vol. 30, no. 12, 31 December 2010 (2010-12-31), pages 1574 - 1577 * |
| 熊安萍: "基于对象存储文件系统的文件级RAID机制研究", 《湖南师范大学自然科学学报》, vol. 33, no. 4, 31 December 2010 (2010-12-31), pages 41 - 45 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710446A (en) * | 2012-06-07 | 2012-10-03 | 浪潮电子信息产业股份有限公司 | Performance tuning method of colony memory system based on synchronous strategy |
| CN103856517A (en) * | 2012-11-30 | 2014-06-11 | 英业达科技有限公司 | Synchronous equalization system and method thereof |
| CN103914452A (en) * | 2012-12-30 | 2014-07-09 | 航天信息股份有限公司 | Distributed file system time error correction method and system |
| CN103914452B (en) * | 2012-12-30 | 2017-03-29 | 航天信息股份有限公司 | Time error correction scheme and system in a kind of distributed file system |
| CN104239026A (en) * | 2013-06-21 | 2014-12-24 | 伊姆西公司 | Method and device for measuring performance of storage system |
| CN104239026B (en) * | 2013-06-21 | 2019-12-27 | 伊姆西公司 | Method and apparatus for measuring performance of storage system |
| CN106776166A (en) * | 2016-12-06 | 2017-05-31 | 郑州云海信息技术有限公司 | Ceph storage clusters method of testing and system under a kind of RDMA patterns |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102291449B (en) | 2014-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102291449B (en) | Method for testing and adjusting cluster storage system performance based on synchronous strategy | |
| CN108469989A (en) | A kind of reaction type based on clustering performance scalable appearance method and system automatically | |
| CN108845954B (en) | Pressure testing method, system and storage medium | |
| CN106059701A (en) | Device for testing clock synchronization correction value of time-triggered Ethernet by capturing protocol control frame | |
| CN106372221A (en) | File synchronization method, equipment and system | |
| CN108337303A (en) | A kind of method of data synchronization and distributed system | |
| CN108508477B (en) | Seismic data acquisition system and method | |
| CN206224231U (en) | A kind of HIL test systems of entire car controller VCU | |
| CN110750458A (en) | Big data platform testing method and device, readable storage medium and electronic equipment | |
| EP3486780B1 (en) | Instruction processing alignment system | |
| CN102967815A (en) | Chip testing method, automated testing equipment and system | |
| CN111581137B (en) | Data record recharge system and method | |
| CN108804241B (en) | Cross-platform task scheduling method, system, computer equipment and storage medium | |
| CN102710446A (en) | Performance tuning method of colony memory system based on synchronous strategy | |
| CN102710458A (en) | Monitoring method of testing process, test management server and test server | |
| CN109542602A (en) | It is a kind of based on the distributed task scheduling processing method of block chain, apparatus and system | |
| CN109117244A (en) | A kind of implementation method of resources of virtual machine application queuing mechanism | |
| CN108334428B (en) | Parallel testing method for system functions | |
| CN103778005A (en) | Automatic task allocation method and device | |
| CN103532786A (en) | Server synchronization detector as well as synchronous detection method and system | |
| CN109586996A (en) | The cloud platform browsing real-time data system and method compared based on network message time delay | |
| CN106776236A (en) | The method and apparatus of the execution of monitoring program | |
| CN114020635A (en) | Data testing method, device, equipment and medium based on scene message queue | |
| CN109656267B (en) | Parallel cooperative test method for flight control system | |
| CN111294250B (en) | Pressure testing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |