Summary of the invention
In order to address the above problem, the invention provides a kind of electric reliability index quick calculation method of processing based on data cached multithreading, the technological means of utilizing data reading speed in internal memory and multi-threaded parallel to process, under the condition of magnanimity basic data, improved the efficiency that the electric reliability index is calculated.
Technical scheme of the present invention is as follows:
A kind of electric reliability index quick calculation method of processing based on data cached multithreading, is characterized in that, the method comprises the following steps:
(1) create data buffer area in internal memory, divide by traffic data type, take and occur on management unit, facility registration days or have a power failure to set up data block, the buffer memory basic data as major key days;
(2) set up in each data block and change buffer zone, the data of change occur in record, and regularly will change basic data merging in buffer data and data block, empty the change buffer zone;
(3) condition that the reliability index computation requests of submitting to according to the user is inputted, in data buffer area, retrieval participates in the data block of calculating;
(4) according to the maximum thread of the current setting of system, by retrieving data in the data block that participates in calculating, on average cut into many groups, and create a plurality of base values computational threads and a base values merging thread;
(5), after all base values computational threads complete base values calculating, the base values result each base values computational threads calculated by base values merging thread is merged, and calculates management units at different levels Reliability Synthesis index.
Above-mentioned steps (1) creates data buffer area in internal memory, and the buffer memory basic data comprises the following steps:
(1a) divide by traffic data type, create business memory object initialization in data buffer area; Initialized information comprises data sorting field, extracted data filtercondition used in appointed object Data Source, the required field read, setting data piece;
(1b) read basic data from external data source, sorted by management unit, registration date or power off time simultaneously, form the original data record collection in internal memory;
(1c) the concentrated data of original data record that form in traversal step (1b), according to management unit, facility registration days or the generation newdata piece on days that has a power failure, or write existing data block by data;
(1d) set up by management unit, facility registration days or have a power failure the major key that forms days and the data block list of data block address pointer occur.
In above-mentioned steps (1), in described data block, basic data district internal data sortord is: the facility registration data are pressed the sequence of facility registration date, and equipment operating data is by the start time sequence that has a power failure.
Above-mentioned steps (2) is set up the data change buffer zone in each data block, empties buffer zone and comprises the following steps:
(2a) set up the data change buffer zone in each data block, create three lists and be respectively newly-increased list, modification list and delete list in the change buffer zone;
(2b) the basic data modification information regularly provided according to external data source, read to respectively newly-increased list, modification list and delete list by the basic data that change occurs by change type;
(2c) regularly will increase basic data in data in list, modification list and delete list and data block newly is merged;
(2d) empty newly-increased list, modification list and delete list.
In above-mentioned steps (3), comprise the steps:
(3a) obtain the reliability index computation requests that the user submits to from database, and convert thereof into the memory object of screening basic data condition;
(3b) travel through the data block of all business memory objects, according to data block major key information and index, calculate related zero-time garbled data piece;
(3c) data block of the participation filtered out being calculated is added in the list of computational data piece by the mode of address pointer.
In above-mentioned steps (4), comprise the following steps:
(4a) according to the number of the basic data total number of records specified data group that participates in the maximum thread of the current setting of system and the list of computational data piece calculating, and the par of the contained basic data of each data group after cutting;
(4b) all data blocks in the list of traversal computational data piece, create the data group that the basic data capacity is par described in step (4a) in the time of the ergodic data piece, when if the basic data amount comprised in current data block is greater than current data group residual capacity, the basic data split in data block is first filled the current data group completely, more remaining basic data is added in new data group; When if the basic data amount comprised in current data block is less than current data group residual capacity do not split, directly it is added in the current data group;
(4c) according to the quantity of data group, create a plurality of base values computational threads, and the list of basis of formation index calculation task; Every group of data are committed to created base values computational threads, and base values calculation task state corresponding to thread is set in calculating;
(4d) create a base values and merge thread.
In above-mentioned steps (5), comprise the following steps:
(5a) each base values computational threads travels through each basic data in handled data group, and the data in the newly-increased list of the change buffer zone in respective data blocks, and judge according to the memory object of screening basic data condition whether this basic data participates in calculating, if participate in calculating submit to the base values computing unit to be calculated these data, result of calculation deposits in base values the results list, if do not participate in process next basic data, until these group data are all finished dealing with;
(5b) after each base values computational threads completes, the calculation task state that this thread is corresponding has been set to;
(5c) task in the list of traversal base values calculation task, judge that whether all tasks all complete, and if all do not complete 1 second of dormancy execution step (5b) afterwards again, if all complete perform step (5d);
(5d) each outcome record in traversal base values the results list, merged the base values in the outcome record of same unit, calculates the Reliability Synthesis index of this management unit, and whole index results are write in reliability index the results list;
(5e) by the write into Databasce as a result of the index in reliability index the results list, feed back to the user.
The beneficial effect that the present invention reaches:
Method of the present invention is carried out buffer memory by many data blocks to basic data, according to the electric reliability business characteristic, has designed data cached structure, has improved the efficiency of data retrieval when the electric reliability index is calculated; And set up basic data change synchronization mechanism, and the change buffer zone is set, improved the synchronous efficiency of change data; Adopt the basic data grouping when index is calculated, utilize multithreading to complete the calculating of base values, finally a plurality of base values results are merged, calculate and meet the reliability index result of wanting, guaranteed the speed that the electric reliability index is calculated.
The present invention adopts most technological means of processing according to block cache business datum, data change buffer zone and multithreading, has improved electric reliability service basic data retrieval and counting yield, thereby has realized the quick calculating of electric reliability index.
Embodiment
Below in conjunction with accompanying drawing, the invention will be further described.Following examples are only for technical scheme of the present invention more clearly is described, and can not limit the scope of the invention with this.
The data structure of data buffer area as shown in Figure 1, is divided by traffic data type in data buffer area, creates the business memory object.Each business memory object comprises: the object Back ground Information be comprised of attribute list, source data table, ordering attribute, unique identification information; By management unit, facility registration days or power failure, occurring days is major key, the data block list that the key-value pair that the data block address index is value forms; A plurality of data blocks that formed by basic data district and change buffer zone.
Wherein: in the basic data district in data block, the data sorting mode is: the facility registration data are pressed the sequence of facility registration date, and equipment operating data is by the start time sequence that has a power failure; Change buffer zone in data block comprises newly-increased list, modification list and delete list, is respectively used to record newly-increased basic data, the basic data of attribute generation modification and deleted basic data.
Except the data buffer area, also comprise that data load-on module, data update module, index calculation task administration module and index computing module form in the technical program, each module cooperation relation as shown in Figure 2.Radix data in data block basic data district load by the data load-on module is unified when starting, the data of change in the buffer zone data update module corresponding by each data block be the data change concordance list record in reading database regularly, according to basic data major key read data information from base data table that change occurs, and according to the data change type, basic data is write to newly-increased list, modification list or delete list.Data update module is start by set date change buffer data and basic data district data merging task also, upgrades in the basic data district, makes it as up-to-date basic data, and empties the change buffer zone.
Index calculation task administration module is responsible for obtaining from database the reliability index computation requests that the user submits to, the design conditions that request is inputted convert the memory object of screening basic data condition to, travel through the data block of all business datum memory objects, calculate related zero-time garbled data piece according to data block major key information and index, and qualified data block is added in the list of computational data piece; Then, number according to the basic data total number of records specified data group that participates in the maximum thread of the current setting of system and the list of computational data piece calculating, and the par of the rear contained basic data of each data group of cutting, and the data block in the list of cutting calculations data block, form a plurality of data groups; Finally, according to the quantity of data group, create a plurality of base values computational threads, and the list of basis of formation index calculation task.Every group of data are committed to created base values computational threads, and, during base values calculation task state corresponding to thread be set to calculate, create a base values simultaneously and merge thread.
The index computing module is responsible for providing base values to calculate and base values merges service logic, and a plurality of base values computational threads that create according to index calculation task administration module and base values merge thread and complete reliability index calculating, and Output rusults.
To Fig. 6, technical scheme of the present invention is carried out to clear, complete description below in conjunction with accompanying drawing 3.Fig. 3 has illustrated to comprise the steps: the data cached loading flow process in the embodiment of the present invention
Step 301: according to configuration file, generation need to be carried out the traffic data type list of data buffer storage;
Step 302: obtain traffic data type list iterator;
Step 303: judge in traffic data type list iterator whether comprise next traffic data type, if having perform step 304, if do not have data cached loading end;
Step 304: create the business memory object, set up object Back ground Information district and empty data block list;
Step 305: initialization business memory object Back ground Information comprises: the attribute list of business memory object, source data table, for attribute and the business memory object unique identification of basic data sequence in data block;
Step 306: read basic data from the source data table of business memory object, the basis of formation data set;
Step 307: traversal basic data concentrated data according to the information creating such as the unit of basic data or add in existing data block, and are added into new data block address information in the data block list;
Step 308: return to step 303.
In order to keep the basic data in data block to be always up-to-date business datum, with source data, be consistent, added data update module in the present invention, this module starts two regularly carrying out of tasks, completes respectively in the change buffer zone that the data that change will occur in the source data table read to data block and the data that will change buffer zone are incorporated in the basic data district.
Fig. 4 has illustrated to comprise the steps: the synchronous flow process of change data in the embodiment of the present invention
Step 401: reading out data in the data change concordance list from database forms the change record collection;
Step 402: traversal change record collection, read basic data according to the change index information from base data table, and according to by type of service, basic data grouping after changing being preserved;
Step 403: the basic data in the traversal above-mentioned steps in each grouping is added into respectively in newly-increased list, modification list or the delete list of respective data blocks change buffer zone according to the change type of these data.
Fig. 5 has illustrated that the change buffered data in the embodiment of the present invention merges flow process, comprises the steps:
Step 501: the business memory object list of obtaining the change buffer zone;
Step 502: judge whether to exist next business memory object, if exist perform step 503, if there is no finish;
Step 503: the data block list of obtaining above-mentioned business memory object;
Step 504: judge whether to exist next data block, if exist perform step 505, if there is no perform step 502;
Step 505: obtain the newly-increased list in data block change buffer zone;
Step 506: travel through above-mentioned newly-increased list, the data in list are added into to the basic data district of existing or new data block;
Step 507: obtain the modification list in data block change buffer zone;
Step 508: travel through above-mentioned modification list, revise the basic data in the basic data district in this data block;
Step 509: obtain the delete list in data block change buffer zone;
Step 510: travel through above-mentioned delete list, delete in this data block basic data corresponding in the basic data district.
Above-mentioned three treatment schemees, data cached loading, change data are synchronous, the change buffered data merges the basic data that jointly realized in all data blocks of data buffer area can be in time and the purpose that is consistent of the data in the source data table, integrality and the accuracy of the basic data while having guaranteed index calculating.The arranging to have solved in the basic data district that directly variation of data is updated to data block of change buffer zone needs to carry out more business logic processing, and efficiency is lower, can't meet a large amount of data variation.Simultaneously, in the process of calculating in index, the data in the change buffer zone also participate in corresponding index and calculate, the accuracy of the index result of calculation of assurance.
Fig. 6 has illustrated to comprise the steps: the index computation requests treatment scheme in the embodiment of the present invention
Step 601: obtain the index computation requests from database;
Step 602: according to the conditional information comprised in the index computation requests, generate for screening the memory object of basic data;
Step 603: the screening conditions memory object generated according to above-mentioned steps 602 filters out and participates in the data block of calculating, and forms the list of computational data piece from data buffer area;
Step 604: according to the basic data amount size in the computational data piece list generated in the maximum thread of the current setting of system and above-mentioned steps 603, the basic data quantity that the number of specified data group and each data group comprise;
Step 605: according to the number of established data group and the quantity of contained basic data, the data block in the list of computational data piece is cut, form corresponding data group;
Step 606: according to the quantity of data group, create a plurality of base values computational threads, and the list of basis of formation index calculation task.Each data group is committed to created base values computational threads, and, during base values calculation task state corresponding to thread be set to calculate, create a base values simultaneously and merge thread;
Step 607: the basic data in each base values computational threads traversal corresponding data group completes base values and calculates, and, according to implementation status, upgrades the executing state of the performed base values calculation task of this base values computational threads;
Step 608: base values merges the executing state of each base values computational threads calculation task of thread timer access;
Step 609: judge whether the corresponding calculation task of all base values computational threads all completes, if all do not complete, performs step 610, if all complete, performs step 611;
Step 610: dormancy 1s, return to step 609;
Step 611: the base values result that a plurality of threads are returned is merged, and generates units at different levels Reliability Synthesis index, writes reliability index the results list;
Step 612: according to the data of reliability index the results list by units at different levels Reliability Synthesis index write into Databasce, to feed back to the user.
Fig. 7 has illustrated to comprise the steps: the index calculation process in the embodiment of the present invention
Step 701: obtain basic data iterator in the data group, this iterator has comprised basic data in each data block basic data district in this data group, and the iteration that increases basic data in list in the change buffer zone newly;
Step 702: whether have next data in judgement basis data iterator, if exist, perform step 703, if there is no, finish;
Step 703: read this basic data;
Step 704: judge whether this basic data exists in the modification list of buffer zone, if exist, perform step 705, if there is no, perform step 706;
Step 705: read the basic data in change buffer zone modification list, the data in alternative steps 703;
Step 706: whether this basic data exists in the delete list of buffer zone, if exist, performs step 702, if there is no, performs step 707;
Step 707: judge whether this basic data meets the user and submit design conditions to, if meet, performs step 708, if do not meet, performs step 702.
The embodiment of the present invention is carried out buffer memory by many data blocks to basic data, according to the electric reliability business characteristic, has designed data cached structure, has improved the efficiency of data retrieval when the electric reliability index is calculated; And set up basic data change synchronization mechanism, and the change buffer zone is set, improved the synchronous efficiency of change data; Adopt the basic data grouping when index is calculated, utilize multithreading to complete the calculating of base values, finally a plurality of base values results are merged, calculate and meet the reliability index result of wanting, guaranteed the speed that the electric reliability index is calculated.
The above has been described in detail purpose of the present invention, technical scheme and embodiment; institute is understood that; the above is not in order to limit the present invention; within the spirit and principles in the present invention all; any modification of making, be equal to replacement, improvement etc., within protection scope of the present invention all should be included in.