CN104077373A - Data consistency checking method - Google Patents
Data consistency checking method Download PDFInfo
- Publication number
- CN104077373A CN104077373A CN201410286228.1A CN201410286228A CN104077373A CN 104077373 A CN104077373 A CN 104077373A CN 201410286228 A CN201410286228 A CN 201410286228A CN 104077373 A CN104077373 A CN 104077373A
- Authority
- CN
- China
- Prior art keywords
- data
- check code
- record
- copied
- records
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
Abstract
The invention discloses a data consistency checking method. The data consistency checking method includes that when a data table is stored in a database system, a check code is figured out by a preset encryption method and stored according to contents of records of each line in the data table; when the database system receives an order which requires to copy data of the data table, the copied records and the corresponding check code are copied together and stored in a target storage device; as for each of currently stored records, the target storage device figures out a check code by the preset encryption method according to contents of the corresponding record and compares the figured out check code with the check code of the corresponding current stored record, if consistency exists, the copy of the records is determined to be consistent, and otherwise, errors of the copy of the records are determined to exist. By means of the data consistency checking method, data consistency can be guaranteed during copying of mass data, the efficiency is high, and expenses of transmission resources are low.
Description
Technical field
The present invention relates to database maintenance technology, particularly relate to a kind of data consistency method of calibration.
Background technology
In the application of database technology, carry out data Replica between multiple memory devices time, normally directly from the memory device of source, after reading out data, pass through intermediate storage medium or the network storage to object memory device.In such scheme, object memory device does not carry out school inspection to the consistance of stored data.But, in actual applications, in the copy procedure of data, may exist data by situations such as malice falsification or loss.Therefore, in the time that the data in database are copied, need to verify the integrality of data, to guarantee the security of data.
At present, the consistance of copy data being verified, is by respectively the data in source memory device and object memory device being compared to realize completing after data Replica.For example data are copied to B from A, traditional scheme need to check out A and B to compare.Owing to first data being read out respectively relatively before needing, therefore, under the application scenarios of mass data processing, can there is the problems such as data transfer overhead is large, verification time delay is large in this consistency comparison method, be not suitable for the processing of carrying out mass data.
Summary of the invention
In view of this, fundamental purpose of the present invention is to provide a kind of data consistency method of calibration, and the method can be in the consistance of carrying out guaranteeing when mass data copies data, and treatment effeciency is high, transfer resource expense is little.
In order to achieve the above object, the technical scheme that the present invention proposes is:
A kind of data consistency method of calibration, comprising:
During to Database Systems, for the every line item in tables of data, according to the content of this line item, adopting default encryption method calculate a check code and preserve data table stores;
In the time that described Database Systems receive the instruction that the data of described tables of data are copied, copied record and corresponding described check code are together copied, and be stored in object memory device;
For each record of current storage, described object memory device adopts default encryption method to calculate a check code according to the content of this record, and the described check code of this record of calculated check code and current storage is compared, if consistent, judge that copying of this record has consistance, otherwise, judge that copying of this record exists mistake.
In sum, the data consistency method of calibration that the present invention proposes, by generating respectively a check code for each record in tables of data, make carrying out in data Replica process, can realize accordingly the verification of data consistency, avoid problem in the reproduction process of traditional data table, the problem includes: therefore the problem such as data transfer overhead is large, verification time delay is large, is applicable to copying of mass data.
Brief description of the drawings
Fig. 1 is the schematic flow sheet of the embodiment of the present invention one.
Embodiment
For making the object, technical solutions and advantages of the present invention clearer, the present invention is described in further detail below in conjunction with the accompanying drawings and the specific embodiments.
Core concept of the present invention is: in the time of data table memory, for this tables of data increases by a check field, namely for each record in table generates a check code according to recording particular content respectively, in the time that this certain record is replicated, this record and its check code are together copied, like this, object memory device can carry out consistency checking to the data of this record according to this check code.
Fig. 1 is the schematic flow sheet of the embodiment of the present invention one, and as shown in Figure 1, this embodiment mainly comprises:
Step 101, during to Database Systems, for the every line item in tables of data, according to the content of this line item, adopting default encryption method calculate a check code and preserve data table stores.
This step is used to every record to generate a check code, carries out accordingly consistance judgement to copy.
Described encryption method will be saved in the each data storage device of system in advance, specifically can adopt existing encryption method.
Preferably, in order to obtain the check code of less byte, described encryption method can be cyclic redundancy check 32 (CRC32) method.Like this, only need additionally to increase by 4 bytes for every line item and preserve check code.
Preferably, the mode of the implicit field of described check code employing system is preserved.Like this, check code is sightless to user, only in the time that internal system is processed, uses.
Step 102, in the time that described Database Systems receive the instruction that the data of described tables of data are copied, copied record and corresponding described check code are together copied, and are stored in object memory device.
Step 103, for each record of current storage, described object memory device adopts default encryption method to calculate a check code according to the content of this record, and the described check code of this record of calculated check code and current storage is compared, if consistent, judge that copying of this record has consistance, otherwise, judge the copy error of this record.
Further, in order to strengthen the maintainability of system, in the time that the arbitrary line item in described tables of data is upgraded, according to the content that records after upgrading, adopt described encryption method to upgrade the check code of described record.
In the time that the record in described tables of data is increased, according to the content of increase record, adopt described encryption method calculate a check code and preserve.
In sum, these are only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.Within the spirit and principles in the present invention all, any amendment of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.
Claims (5)
1. a data consistency method of calibration, is characterized in that, comprising:
During to Database Systems, for the every line item in tables of data, according to the content of this line item, adopting default encryption method calculate a check code and preserve data table stores;
In the time that described Database Systems receive the instruction that the data of described tables of data are copied, copied record and corresponding described check code are together copied, and be stored in object memory device;
For each record of current storage, described object memory device adopts default encryption method to calculate a check code according to the content of this record, and the described check code of this record of calculated check code and current storage is compared, if consistent, judge that copying of this record has consistance, otherwise, judge that copying of this record exists mistake.
2. method according to claim 1, is characterized in that, described encryption method is CRC32 method.
3. method according to claim 1, is characterized in that, the mode of the implicit field of described check code employing system is preserved.
4. method according to claim 1, is characterized in that, described method further comprises:
In the time that the arbitrary line item in described tables of data is upgraded, according to the content that records after upgrading, adopt described encryption method to upgrade the check code of described record.
5. method according to claim 1, is characterized in that, described method further comprises:
In the time that the record in described tables of data is increased, according to the content of increase record, adopt described encryption method calculate a check code and preserve.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410286228.1A CN104077373B (en) | 2014-06-24 | 2014-06-24 | A kind of data consistency verification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410286228.1A CN104077373B (en) | 2014-06-24 | 2014-06-24 | A kind of data consistency verification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104077373A true CN104077373A (en) | 2014-10-01 |
| CN104077373B CN104077373B (en) | 2018-12-04 |
Family
ID=51598627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410286228.1A Active CN104077373B (en) | 2014-06-24 | 2014-06-24 | A kind of data consistency verification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104077373B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104820710A (en) * | 2015-05-19 | 2015-08-05 | 福建新大陆电脑股份有限公司 | Data check value generation method and system |
| CN106874270A (en) * | 2015-12-10 | 2017-06-20 | 中国移动通信集团云南有限公司 | A kind of method and apparatus of data consistency certification |
| CN107070645A (en) * | 2016-12-30 | 2017-08-18 | 华为技术有限公司 | Compare the method and system of the data of tables of data |
| CN107145803A (en) * | 2017-06-05 | 2017-09-08 | 上海联影医疗科技有限公司 | A kind of method of inspection, equipment and the computer-readable medium of peration data uniformity |
| CN108573007A (en) * | 2017-06-08 | 2018-09-25 | 北京金山云网络技术有限公司 | Method, apparatus, electronic equipment and the storage medium of data consistency detection |
| CN108614742A (en) * | 2016-12-12 | 2018-10-02 | 北京京东尚科信息技术有限公司 | Method of calibration, system and the device of report data |
| CN110912766A (en) * | 2019-10-18 | 2020-03-24 | 国家计算机网络与信息安全管理中心 | Communication network multi-plane data consistency checking method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5592618A (en) * | 1994-10-03 | 1997-01-07 | International Business Machines Corporation | Remote copy secondary data copy validation-audit function |
| CN101145117A (en) * | 2007-10-30 | 2008-03-19 | 华为技术有限公司 | Method and apparatus for treating error detection |
| CN101452410A (en) * | 2007-12-06 | 2009-06-10 | 中兴通讯股份有限公司 | Data backup system for embedded database, and data backup and recovery method |
| CN101783955A (en) * | 2010-03-24 | 2010-07-21 | 杭州华三通信技术有限公司 | Data recovering method when data is abnormal and equipment thereof |
-
2014
- 2014-06-24 CN CN201410286228.1A patent/CN104077373B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5592618A (en) * | 1994-10-03 | 1997-01-07 | International Business Machines Corporation | Remote copy secondary data copy validation-audit function |
| CN101145117A (en) * | 2007-10-30 | 2008-03-19 | 华为技术有限公司 | Method and apparatus for treating error detection |
| CN101452410A (en) * | 2007-12-06 | 2009-06-10 | 中兴通讯股份有限公司 | Data backup system for embedded database, and data backup and recovery method |
| CN101783955A (en) * | 2010-03-24 | 2010-07-21 | 杭州华三通信技术有限公司 | Data recovering method when data is abnormal and equipment thereof |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104820710A (en) * | 2015-05-19 | 2015-08-05 | 福建新大陆电脑股份有限公司 | Data check value generation method and system |
| CN104820710B (en) * | 2015-05-19 | 2018-05-01 | 福建新大陆电脑股份有限公司 | A kind of data check value generation method and system |
| CN106874270A (en) * | 2015-12-10 | 2017-06-20 | 中国移动通信集团云南有限公司 | A kind of method and apparatus of data consistency certification |
| CN106874270B (en) * | 2015-12-10 | 2020-10-16 | 中国移动通信集团云南有限公司 | Data consistency authentication method and device |
| CN108614742A (en) * | 2016-12-12 | 2018-10-02 | 北京京东尚科信息技术有限公司 | Method of calibration, system and the device of report data |
| CN108614742B (en) * | 2016-12-12 | 2021-09-14 | 北京京东尚科信息技术有限公司 | Report data verification method, system and device |
| CN107070645A (en) * | 2016-12-30 | 2017-08-18 | 华为技术有限公司 | Compare the method and system of the data of tables of data |
| CN107145803A (en) * | 2017-06-05 | 2017-09-08 | 上海联影医疗科技有限公司 | A kind of method of inspection, equipment and the computer-readable medium of peration data uniformity |
| CN108573007A (en) * | 2017-06-08 | 2018-09-25 | 北京金山云网络技术有限公司 | Method, apparatus, electronic equipment and the storage medium of data consistency detection |
| CN110912766A (en) * | 2019-10-18 | 2020-03-24 | 国家计算机网络与信息安全管理中心 | Communication network multi-plane data consistency checking method |
| CN110912766B (en) * | 2019-10-18 | 2021-04-20 | 国家计算机网络与信息安全管理中心 | Communication network multi-plane data consistency checking method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104077373B (en) | 2018-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104077373A (en) | Data consistency checking method | |
| US9588712B2 (en) | Systems and methods for migrating data | |
| US7921237B1 (en) | Preserving data integrity of DMA descriptors | |
| US9436722B1 (en) | Parallel checksumming of data chunks of a shared data object using a log-structured file system | |
| US8463746B2 (en) | Method and system for replicating data | |
| US20170289151A1 (en) | Technologies for dynamic loading of integrity protected modules into secure enclaves | |
| US20170052884A1 (en) | Generic test automation for restful web services applications | |
| CN107870727B (en) | Method and apparatus for storing data | |
| US9298592B2 (en) | Testing a software interface for a streaming hardware device | |
| TW201520813A (en) | Anti-malware mobile content data management apparatus and method | |
| WO2019095576A1 (en) | Data migration method, apparatus, computer device and storage medium | |
| CN112087530B (en) | Method, device, equipment and medium for uploading data to block chain system | |
| CN106843934A (en) | The treating method and apparatus of upgrade file | |
| CN102902732B (en) | A kind of method of adding verification in XML configuration file | |
| CN107169000A (en) | Static resource dissemination method and device | |
| CN103984633B (en) | A kind of bank main passes down the automatization test system of operation | |
| CN102402470A (en) | Method and device for automatically correcting error of program of flash memory and repairing program of flash memory | |
| CN104090948A (en) | Method, device and system for processing mass data of nuclear power station | |
| CN105868127A (en) | Data storage method and device and data reading method and device | |
| CN106027638A (en) | Hadoop data distribution method based on hybrid coding | |
| US9633062B1 (en) | Document fingerprints and templates | |
| CN109542860B (en) | Service data management method based on HDFS and terminal equipment | |
| CN104077315B (en) | Distributed File System Data introduction method and device | |
| US9936015B2 (en) | Method for building up a content management system | |
| JP5537668B2 (en) | Bus connection program and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |