CA2018932A1 - Method of removing uncommitted changes made to stored data by a database management system - Google Patents
Method of removing uncommitted changes made to stored data by a database management systemInfo
- Publication number
- CA2018932A1 CA2018932A1 CA002018932A CA2018932A CA2018932A1 CA 2018932 A1 CA2018932 A1 CA 2018932A1 CA 002018932 A CA002018932 A CA 002018932A CA 2018932 A CA2018932 A CA 2018932A CA 2018932 A1 CA2018932 A1 CA 2018932A1
- Authority
- CA
- Canada
- Prior art keywords
- backout
- failure
- log record
- backout failure
- changes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 101001074449 Crotalus durissus terrificus Phospholipase A2 inhibitor CNF Proteins 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000007792 addition Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2205—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/87—Monitoring of transactions
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11C—STATIC STORES
- G11C29/00—Checking stores for correct operation ; Subsequent repair; Testing stores during standby or offline operation
- G11C29/70—Masking faults in memories by using spares or by reconfiguring
- G11C29/74—Masking faults in memories by using spares or by reconfiguring using duplex memories, i.e. using dual copies
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99953—Recoverability
Abstract
METHOD OF REMOVING UNCOMMITTED CHANGES MADE TO
STORED DATA BY A DATABASE MANAGEMENT SYSTEM
ABSTRACT
A method of removing uncommitted changes made to stored data by a database management system, wherein the system stores a duplicate copy of data prior to effecting changes to the original copy, and upon failure of the system, or a transaction or transactions effecting the changes replaces the copy containing uncommited changes by the duplicate copy, characterised in that, in the case of failure of the system to replace the original copy (ie in the case of a backout failure), the system prevents subsequent attempts by the system to further change the data until the uncommitted changes have been removed by a process of backout failure processing, and wherein a batch backout utility program uses a set of system log records to bracket the duration of the backout failure for a particular file to track:
whether the system completed its backout failure processing;
which transactions had failed backout; and which transaction was the first to have a backout failure.
STORED DATA BY A DATABASE MANAGEMENT SYSTEM
ABSTRACT
A method of removing uncommitted changes made to stored data by a database management system, wherein the system stores a duplicate copy of data prior to effecting changes to the original copy, and upon failure of the system, or a transaction or transactions effecting the changes replaces the copy containing uncommited changes by the duplicate copy, characterised in that, in the case of failure of the system to replace the original copy (ie in the case of a backout failure), the system prevents subsequent attempts by the system to further change the data until the uncommitted changes have been removed by a process of backout failure processing, and wherein a batch backout utility program uses a set of system log records to bracket the duration of the backout failure for a particular file to track:
whether the system completed its backout failure processing;
which transactions had failed backout; and which transaction was the first to have a backout failure.
Description
9 3 ~
MET~OD VF REMOVING UNCOMMITTED CHANGES MADE TO
STORED DArrA sY A DATABASE MANAGEMENT SYSTE~
The present invention relates to a me-thod of removing uncommitted changes made to stored data by a database management system such as, for example, a transaction processing system, and in particular to a transaction processing system in which a copy is made of stored data prior to making changes to that stored data; failure of the system during the processing of a transaction, ie making changes to stored data, is dealt with such that integrity of tha data is not lost.
A transaction in an on-line transaction processing system, such as for example the IBM (IBM is a registered trade mark of International Business Machines Corporation) Customer Information Control System, CICS, is a change or a number of changes made to data held in storage in the transaction processing system. In order to ensure integrity of data a transaction must be "all-or-nothing", meaning that either the complete change to the data is made or none of the changes must be made. Usually the change or changes making up the transaction are made sequentially and upon completion of all the changes a signal is sent to the system to instruct the system to "commit" the changes.
Should the system fail or a transaction terminate abnormally before all changes making up the transaction have been made and committed, the system must remove or "backout" all those changes which have not been committed by the system in order to preserve data integrity. It does this by replacing the changed data with a copy of the original data (a "before-image") which it saved just be~ore the transaction performed its changes, thus reinstating the original data, and removing the unwanted changes.
It is possible for backout processing to fail during the course of replacement of the uncommitted changes (i.e.
a backout failure occurs), for example because the storage device on which the data containing the uncommitted changes has malfunctioned. Failure of the system to replace 2 ~
UXg-~9-018 2 uncommitted changes to the data has previously meant that, in order to maintain integrity of the data, action was required of an operator to prevent further changes being made to the da-ta. In previous transaction processing systems this was done by shutting down the whole system, correcting the problem, and then re-starting the system as an "emergency restart"; the operator either had to shut down the system leading to disruption of transaction processing, or accept that some data contained uncommitted changes. Shutdown of the system may have an effect on any "warm standby" systems, where another transaction processing system is standing-by ready to take over processing should the main system fail in any way. The standby system performs an emergency restart when it takes over, and any backout performed during this emergency restart will immediately fail again.
The present invention provides a transaction processing system able to continue operating while isolating that part of the data, generally referred to as a file, containing uncommitted changes until it has been replaced by the original data or before-image.
The present invention also provides a method which improves earlier backout methods by closing only the file on which the backout failure has occurred, and running a program to retry the backout.
Accordingly the present invention provides a method of removing uncommitted changes made to stored data by a database management system, wherein the system stores a duplicate copy of data prior to effecting changes to the original copy, and upon failure of the system, or a transaction or transactions effecting the changes replaces the copy containing uncommited changes by the duplicate copy, wherein, in the case of failure of the system to replace the original copy (ie in the case of a backout failure), the system prevents subsequent attempts by the system to further change the data until the uncommitted changes have been removed by a process of backout failure processing.
MET~OD VF REMOVING UNCOMMITTED CHANGES MADE TO
STORED DArrA sY A DATABASE MANAGEMENT SYSTE~
The present invention relates to a me-thod of removing uncommitted changes made to stored data by a database management system such as, for example, a transaction processing system, and in particular to a transaction processing system in which a copy is made of stored data prior to making changes to that stored data; failure of the system during the processing of a transaction, ie making changes to stored data, is dealt with such that integrity of tha data is not lost.
A transaction in an on-line transaction processing system, such as for example the IBM (IBM is a registered trade mark of International Business Machines Corporation) Customer Information Control System, CICS, is a change or a number of changes made to data held in storage in the transaction processing system. In order to ensure integrity of data a transaction must be "all-or-nothing", meaning that either the complete change to the data is made or none of the changes must be made. Usually the change or changes making up the transaction are made sequentially and upon completion of all the changes a signal is sent to the system to instruct the system to "commit" the changes.
Should the system fail or a transaction terminate abnormally before all changes making up the transaction have been made and committed, the system must remove or "backout" all those changes which have not been committed by the system in order to preserve data integrity. It does this by replacing the changed data with a copy of the original data (a "before-image") which it saved just be~ore the transaction performed its changes, thus reinstating the original data, and removing the unwanted changes.
It is possible for backout processing to fail during the course of replacement of the uncommitted changes (i.e.
a backout failure occurs), for example because the storage device on which the data containing the uncommitted changes has malfunctioned. Failure of the system to replace 2 ~
UXg-~9-018 2 uncommitted changes to the data has previously meant that, in order to maintain integrity of the data, action was required of an operator to prevent further changes being made to the da-ta. In previous transaction processing systems this was done by shutting down the whole system, correcting the problem, and then re-starting the system as an "emergency restart"; the operator either had to shut down the system leading to disruption of transaction processing, or accept that some data contained uncommitted changes. Shutdown of the system may have an effect on any "warm standby" systems, where another transaction processing system is standing-by ready to take over processing should the main system fail in any way. The standby system performs an emergency restart when it takes over, and any backout performed during this emergency restart will immediately fail again.
The present invention provides a transaction processing system able to continue operating while isolating that part of the data, generally referred to as a file, containing uncommitted changes until it has been replaced by the original data or before-image.
The present invention also provides a method which improves earlier backout methods by closing only the file on which the backout failure has occurred, and running a program to retry the backout.
Accordingly the present invention provides a method of removing uncommitted changes made to stored data by a database management system, wherein the system stores a duplicate copy of data prior to effecting changes to the original copy, and upon failure of the system, or a transaction or transactions effecting the changes replaces the copy containing uncommited changes by the duplicate copy, wherein, in the case of failure of the system to replace the original copy (ie in the case of a backout failure), the system prevents subsequent attempts by the system to further change the data until the uncommitted changes have been removed by a process of backout failure processing.
Preferably, in the case of failure of the system to replaca the original copy, the sys-tem displays notice of the failure and prevents subsequent attempts by the system to further change the data until the uncommitted changes have been removed. Suitably the system displays notice of failure to an operator using a visual display unit.
Suitably the database management system is a transaction processing system that does before-image logging.
In a preferred embodiment of -the present invention removal of unwanted data updates because they contain uncommitted changes is effected using a batch backout utility program.
A batch backout utility is a program which can be run when a transaction processing system such as CICS
encounters a problem which prevents it from removing the unwanted data updates that may be left behind by a transaction which does not complete its processing normally. The batch backout utility is a program which backs out the unwanted update by selecting the relevant before-image from the system log or a copy thereof and using it to replace the update. The system log is a sequential dataset on which the system records various data for use in system restart, performance monitoring, auditing, debugging, etc. (i.e. the system keeps a "log" of its actions on this datase-t). Before-images are included in this data.
The batch backout utility reads the system log and must decide which before-images to reinstate. This may not be a simple task. There may be many transactions executing concurrently on the system, and each transaction may be updating several files. A backout failure on one file can have a domino effect, causing a string of subsequent backout failures on the same file or others. Also, if the problem that caused the backout failure concerned a whole storage element such as, for example, a magnetic disk, then 2~118~
many files on that magnetic disk could be affected and many backout failures could occur in rapid succession.
When a backout failure occurs in a transaction processing syste~, data with uncomnlitted changes, ie. any unwanted data update, must be isolated so that other transactions cannot access it. In one embodiment of the present invention this is done by closing the file in which the data containing the uncommitted data is stored.
However, it may not be possible to close the file immediately because other transactions may be in the process of updating it, and it is preferable to allow these transactions to run to completion if possible. Of course, some of these transactions may not be able to complete (possibly because of the same problem) and they may terminate abnormally, and backout of their updates may fail. Thus subsequent backout failures on the same file could occur between the first backout failure and the point when the file can be closed and the unwanted updates isolated.
Additional complexity can arise if the whole system itself fails during a backout failure, for instance because of a power cut, system error, operating system fault etc.
-If a transaction processing system such as, forexample the CICS system fails (i.e. terminates abnormally) then it is restarted in emergency restart mode. In this mode, the system tries to get things back to the state they were in when it failed, before continuing. The system can work out which transactions were in progress (i.e.
executing) at the time of failure, and will treat these transactions as having terminated abnormally, and it will attempt to backout: their uncommitted updates. Obviously these attempts could end in yet more backout failures.
In the CICS system and other related systems several files may actually represent the same collection of data (i.e. dataset), but be accessed via different paths, and so two backout failures for different files may be for the same dataset. All this complexity is reflected in the records that appear on the system log, and so the batch backout utility may have a very complex task, trying to unravel this interlaced stream of records and keep track of them. Preferably the batch backout utility should be able to perform backout for multiple files in one run, to backout an update as soon as it has found the relevant before-image (so that it does not have to store the before-image) and to do this with only one scan through the system log. Also, the batch backout utility must be able to decide whether it should backout a particular update, or if it should leave it to emergency restart. Likewise, emergency restart must be able to decide which backouts to attempt, and which to leave to the batch backout utility.
The present invention provides a set of system log records to, in effect, bracket the duration (ie mark the beginning and end) of a backout failure for a particular file. Using these records, the batch backout utility can track:
whether the system completed its backout failure processing (ie. if the system failed before the backout failure processing was completed, then the batch backout utility should not attempt backout, because the emergency restart of the system will retry it);
which transactions had failed backout against which files ~and therefore which changes to backout);
which transaction was the first to have a backout failure (and so when to stop looking for any others).
The three types of backout failure log record of the invention show the start, middle and end of each backout failure process, and, although the processes themselves may be occurring concurrently, and hence the log records may appear interlaced on the system log, it is relatively easy for the batch backout utility to keep track of each individual backout failure process.
In a preferred embodiment of the present invention, whenever a backout failure occurs, the system calls a user-written program to allow the user to handle the 2~89~2 UK9-89-Ot8 6 situation if possible, but would change transaction processing on re~urn from the call to the user to execute the following "backout failure control" code, rather than just i~noring the backout failure. Whenever a backout failure occurs, a "backout failed" flag in a file status control block is examined. If it is not set, then this backout failure must be the first for this file, and the file is put into a "backout faile~" status, by switching the flag on. The system will check this status whenever a transaction requests access to the file. If it is found to be set, then the system will deny access to the file. The file is also put into a "closing" state. This means that the file will be closed as soon as the last transaction that has been accessing the file is finished with i-t.
(There may be more than one transaction at a time updating the file). A "backout failed" record is written to the log, containing information such as file name, transaction identifier, etc., and a "irst" flag to show that this backout failure is the first for this file. If the transaction processing system finds that the "backout failed" status flag has already been set when it examines it at the occurrence of a backout failure, then it knows that a previous backout failure has occurred on this file, and it will not try to set the flag again or close the file, but will just write out a backout failed record to the system log containing the similar information to that before except that the flag will be set to "subsequent"
rather than "first".
When the file eventually closes ~i.e. when all transactions accessing the file before the first backout failure have finished), then the unwanted updates on the file have now become isolated because the system will not allow any attempt to open a file which has the "backout failed'7 flag set. ~t this point another type of backout failed log record is written to signify that the backout failure process for this file is complete and that it is the responsibility of the batch backout utility to backout updates to this f:ile. The flag on this log record is set to t'end" to signify this9 and also extra inormation, which informs the batch backout utility of the filename to 2~ 3'-~J
dataset name connections -that it neecls, is included on the log record.
The batch backout utility need only read backwards through the system log and use the backout failed log records it sees to create and maintain an internal table which tells it exactly what it needs to know (i.e. which before-images to reinstate, whether to go ahead with a backout or to leave it to the emergency restart procedure, and also when it has finished all the backout processing that is needed).
In a further preferred embodiment of the invention the batch backout utility only needs to be given the names of the datasets that the user wants to backout. It reads backwards through (a copy of~ the system log looking for an "end" type of backout failed record containing a matching dataset name. If it finds one, then it stores the dataset name-to-filename connections from the record in its table.
It will now also look for "subsequent" or "first" backout failed log records which contain the stored filenames. It will add a transaction identifier from these records to its table and use the stored filename-transaction identifier combinations to select the correct before-images to reinstate. (-As it reads each before-image on the log, it will compare the filename and transaction identifier on the before-image with those stored in its table. If there is a table entry with a matching filename and transaction identifier then the batch backout utility knows that a backout failure must have occurred for this file and this transaction, and also that the file has been closed. I-t therefore knows that it should reinstate this before-image and backout an uncommitted update). It will also mark its table to remember if a "first" type record has been seen, so that it will ignore any more backout failed log records for this file (because they would be from a set of backout failure records earlier in time but later in sequence since the batch backout utility is reading backwards through the system log).
2~932 The batch backout u-tility also looks at the "start-of-task" flag in the records it reads. This flag shows the first record writ-ten for a transaction, and the batch backout utility will remove any matching transaction identifier from its table when it comes across the start of a transaction. In addi-tion, if the table shows that a "first" backout failed log record has been seen for a fil~
and the batch backout utility has just removed the last remaining transaction that had a backout failure on this file, then it can also remove this file entry since no more transaction entries will be made for this file. When there are no more file entries in the table, then the batch backout utility knows that it has backed-out all the uncommitted updates and it can stop.
According to the preferred embodiment of the invention the transaction processing system contains a backout failed log record. The backout failed log record contains the information that the batch backout utility will need, to calculate if it is safe to attempt backout, where to start, where to stop and which original records it must re-apply in order to back out uncommitted changes. Fields in tha backout failure log record include: FIRST flag (This is set if it is the first backout failure log record for a given dataset); dataset name; and transaction identifier.
There are three different types of backout failure log record:
(i) backout failure log record FIRST, this indicates which task was the first to have a backout failure for a given dataset.
(ii) backout failure log record SUBSEQUENT, this indicates which tasks had backout failures subsequently for a given dataset.
(iii) backou-t failure log record END, this indicates that backout failure has completed for a given dataset.
The batch backout utility will read the system log backwards looking for "backout failure" log records. When it finds one, it will check the dataset name in it, to see if it matches one that the user has specified as one which ' ~
9'~ 3 UKs-89-018 9 should be backe~ out. If so, then the batch backout utility will store the extra information in the "backout failure" log record and use this stored information to control which original records it should re-apply and w~en to stop processing.
The following pseudo code examples show how backout failure log records are used. The term "database" is used to represent the entity to be backed out. The backout failure code is called whenever the backout process fails to apply the before-image to the database. The following pseudo code shows an outline of suitable backout failure logic:
Example 1.
If this data base has already had a backout failure then Write a backout failure log record S~BSEQUENT
Else Write a backout failure log record FIRST
Record that thi~ data base has had a backout failure Ensure subsequent attempts to update the data base are disallowed ~nd if When all tasks using this data base have completed Close the data base Write a backout failure log record END
The batch backout utility reads the system log backwards searching for backout failure log records. A
table is held, within the batch backout utility, containing information from the previously processed backout failure log records. When the batch backout utility finds a backout failure log record it updates this table, depending on the type of backout failure log record read.
The following pseudo code shows an example of suitable batch backout utility logic:
.
~Q ~8~
Example 2.
Read system log backwards If backout failure log record then If for a database that the user has asked to be backed out then If first backout failure log record read backwards for this database then ¦ ¦ If backout failure log record END then Set flag to remember backout failure log rscord END
read Else ERROR. The system did not complete backout, backout for this clata base is the responsibility of the system.
End if Else ¦ If backout failure log record FIRST has been read Eor this database then ¦ Ignore this backout failure log record. ~ sst of backout failure log records has been read for this ¦ database, and this backout failure log record is for a previous backout failure.
Else ¦ ~ Record the transaction identifier in this backout failure log record, against the database If backout failure log record FIRST then Record this against data base End if End if Else ¦ Ignore End if End if If before-image then If before-image for a database that the user has asked to be backed out then --¦ If transaction identifier of before-imags has been found in a backout failure log record then ¦ Apply the before-image Else Ignore as there has not been a backout failure log record for this transaction identifier, this task did NOT
9 ¦ fail backout.
¦ End if Else ¦ Ignore End if End if If first log record for a -transaction then If this transaction identifier is recorded against any databases then Remove this record If there are any date bases that had the transaction identifier and a backout failure log record FIRST has been read for the database then Removs the database from the list of databases to be backed out If there are no databases to be backed out left t.hsn ¦ End of job End if End if End if End if .
Suitably the database management system is a transaction processing system that does before-image logging.
In a preferred embodiment of -the present invention removal of unwanted data updates because they contain uncommitted changes is effected using a batch backout utility program.
A batch backout utility is a program which can be run when a transaction processing system such as CICS
encounters a problem which prevents it from removing the unwanted data updates that may be left behind by a transaction which does not complete its processing normally. The batch backout utility is a program which backs out the unwanted update by selecting the relevant before-image from the system log or a copy thereof and using it to replace the update. The system log is a sequential dataset on which the system records various data for use in system restart, performance monitoring, auditing, debugging, etc. (i.e. the system keeps a "log" of its actions on this datase-t). Before-images are included in this data.
The batch backout utility reads the system log and must decide which before-images to reinstate. This may not be a simple task. There may be many transactions executing concurrently on the system, and each transaction may be updating several files. A backout failure on one file can have a domino effect, causing a string of subsequent backout failures on the same file or others. Also, if the problem that caused the backout failure concerned a whole storage element such as, for example, a magnetic disk, then 2~118~
many files on that magnetic disk could be affected and many backout failures could occur in rapid succession.
When a backout failure occurs in a transaction processing syste~, data with uncomnlitted changes, ie. any unwanted data update, must be isolated so that other transactions cannot access it. In one embodiment of the present invention this is done by closing the file in which the data containing the uncommitted data is stored.
However, it may not be possible to close the file immediately because other transactions may be in the process of updating it, and it is preferable to allow these transactions to run to completion if possible. Of course, some of these transactions may not be able to complete (possibly because of the same problem) and they may terminate abnormally, and backout of their updates may fail. Thus subsequent backout failures on the same file could occur between the first backout failure and the point when the file can be closed and the unwanted updates isolated.
Additional complexity can arise if the whole system itself fails during a backout failure, for instance because of a power cut, system error, operating system fault etc.
-If a transaction processing system such as, forexample the CICS system fails (i.e. terminates abnormally) then it is restarted in emergency restart mode. In this mode, the system tries to get things back to the state they were in when it failed, before continuing. The system can work out which transactions were in progress (i.e.
executing) at the time of failure, and will treat these transactions as having terminated abnormally, and it will attempt to backout: their uncommitted updates. Obviously these attempts could end in yet more backout failures.
In the CICS system and other related systems several files may actually represent the same collection of data (i.e. dataset), but be accessed via different paths, and so two backout failures for different files may be for the same dataset. All this complexity is reflected in the records that appear on the system log, and so the batch backout utility may have a very complex task, trying to unravel this interlaced stream of records and keep track of them. Preferably the batch backout utility should be able to perform backout for multiple files in one run, to backout an update as soon as it has found the relevant before-image (so that it does not have to store the before-image) and to do this with only one scan through the system log. Also, the batch backout utility must be able to decide whether it should backout a particular update, or if it should leave it to emergency restart. Likewise, emergency restart must be able to decide which backouts to attempt, and which to leave to the batch backout utility.
The present invention provides a set of system log records to, in effect, bracket the duration (ie mark the beginning and end) of a backout failure for a particular file. Using these records, the batch backout utility can track:
whether the system completed its backout failure processing (ie. if the system failed before the backout failure processing was completed, then the batch backout utility should not attempt backout, because the emergency restart of the system will retry it);
which transactions had failed backout against which files ~and therefore which changes to backout);
which transaction was the first to have a backout failure (and so when to stop looking for any others).
The three types of backout failure log record of the invention show the start, middle and end of each backout failure process, and, although the processes themselves may be occurring concurrently, and hence the log records may appear interlaced on the system log, it is relatively easy for the batch backout utility to keep track of each individual backout failure process.
In a preferred embodiment of the present invention, whenever a backout failure occurs, the system calls a user-written program to allow the user to handle the 2~89~2 UK9-89-Ot8 6 situation if possible, but would change transaction processing on re~urn from the call to the user to execute the following "backout failure control" code, rather than just i~noring the backout failure. Whenever a backout failure occurs, a "backout failed" flag in a file status control block is examined. If it is not set, then this backout failure must be the first for this file, and the file is put into a "backout faile~" status, by switching the flag on. The system will check this status whenever a transaction requests access to the file. If it is found to be set, then the system will deny access to the file. The file is also put into a "closing" state. This means that the file will be closed as soon as the last transaction that has been accessing the file is finished with i-t.
(There may be more than one transaction at a time updating the file). A "backout failed" record is written to the log, containing information such as file name, transaction identifier, etc., and a "irst" flag to show that this backout failure is the first for this file. If the transaction processing system finds that the "backout failed" status flag has already been set when it examines it at the occurrence of a backout failure, then it knows that a previous backout failure has occurred on this file, and it will not try to set the flag again or close the file, but will just write out a backout failed record to the system log containing the similar information to that before except that the flag will be set to "subsequent"
rather than "first".
When the file eventually closes ~i.e. when all transactions accessing the file before the first backout failure have finished), then the unwanted updates on the file have now become isolated because the system will not allow any attempt to open a file which has the "backout failed'7 flag set. ~t this point another type of backout failed log record is written to signify that the backout failure process for this file is complete and that it is the responsibility of the batch backout utility to backout updates to this f:ile. The flag on this log record is set to t'end" to signify this9 and also extra inormation, which informs the batch backout utility of the filename to 2~ 3'-~J
dataset name connections -that it neecls, is included on the log record.
The batch backout utility need only read backwards through the system log and use the backout failed log records it sees to create and maintain an internal table which tells it exactly what it needs to know (i.e. which before-images to reinstate, whether to go ahead with a backout or to leave it to the emergency restart procedure, and also when it has finished all the backout processing that is needed).
In a further preferred embodiment of the invention the batch backout utility only needs to be given the names of the datasets that the user wants to backout. It reads backwards through (a copy of~ the system log looking for an "end" type of backout failed record containing a matching dataset name. If it finds one, then it stores the dataset name-to-filename connections from the record in its table.
It will now also look for "subsequent" or "first" backout failed log records which contain the stored filenames. It will add a transaction identifier from these records to its table and use the stored filename-transaction identifier combinations to select the correct before-images to reinstate. (-As it reads each before-image on the log, it will compare the filename and transaction identifier on the before-image with those stored in its table. If there is a table entry with a matching filename and transaction identifier then the batch backout utility knows that a backout failure must have occurred for this file and this transaction, and also that the file has been closed. I-t therefore knows that it should reinstate this before-image and backout an uncommitted update). It will also mark its table to remember if a "first" type record has been seen, so that it will ignore any more backout failed log records for this file (because they would be from a set of backout failure records earlier in time but later in sequence since the batch backout utility is reading backwards through the system log).
2~932 The batch backout u-tility also looks at the "start-of-task" flag in the records it reads. This flag shows the first record writ-ten for a transaction, and the batch backout utility will remove any matching transaction identifier from its table when it comes across the start of a transaction. In addi-tion, if the table shows that a "first" backout failed log record has been seen for a fil~
and the batch backout utility has just removed the last remaining transaction that had a backout failure on this file, then it can also remove this file entry since no more transaction entries will be made for this file. When there are no more file entries in the table, then the batch backout utility knows that it has backed-out all the uncommitted updates and it can stop.
According to the preferred embodiment of the invention the transaction processing system contains a backout failed log record. The backout failed log record contains the information that the batch backout utility will need, to calculate if it is safe to attempt backout, where to start, where to stop and which original records it must re-apply in order to back out uncommitted changes. Fields in tha backout failure log record include: FIRST flag (This is set if it is the first backout failure log record for a given dataset); dataset name; and transaction identifier.
There are three different types of backout failure log record:
(i) backout failure log record FIRST, this indicates which task was the first to have a backout failure for a given dataset.
(ii) backout failure log record SUBSEQUENT, this indicates which tasks had backout failures subsequently for a given dataset.
(iii) backou-t failure log record END, this indicates that backout failure has completed for a given dataset.
The batch backout utility will read the system log backwards looking for "backout failure" log records. When it finds one, it will check the dataset name in it, to see if it matches one that the user has specified as one which ' ~
9'~ 3 UKs-89-018 9 should be backe~ out. If so, then the batch backout utility will store the extra information in the "backout failure" log record and use this stored information to control which original records it should re-apply and w~en to stop processing.
The following pseudo code examples show how backout failure log records are used. The term "database" is used to represent the entity to be backed out. The backout failure code is called whenever the backout process fails to apply the before-image to the database. The following pseudo code shows an outline of suitable backout failure logic:
Example 1.
If this data base has already had a backout failure then Write a backout failure log record S~BSEQUENT
Else Write a backout failure log record FIRST
Record that thi~ data base has had a backout failure Ensure subsequent attempts to update the data base are disallowed ~nd if When all tasks using this data base have completed Close the data base Write a backout failure log record END
The batch backout utility reads the system log backwards searching for backout failure log records. A
table is held, within the batch backout utility, containing information from the previously processed backout failure log records. When the batch backout utility finds a backout failure log record it updates this table, depending on the type of backout failure log record read.
The following pseudo code shows an example of suitable batch backout utility logic:
.
~Q ~8~
Example 2.
Read system log backwards If backout failure log record then If for a database that the user has asked to be backed out then If first backout failure log record read backwards for this database then ¦ ¦ If backout failure log record END then Set flag to remember backout failure log rscord END
read Else ERROR. The system did not complete backout, backout for this clata base is the responsibility of the system.
End if Else ¦ If backout failure log record FIRST has been read Eor this database then ¦ Ignore this backout failure log record. ~ sst of backout failure log records has been read for this ¦ database, and this backout failure log record is for a previous backout failure.
Else ¦ ~ Record the transaction identifier in this backout failure log record, against the database If backout failure log record FIRST then Record this against data base End if End if Else ¦ Ignore End if End if If before-image then If before-image for a database that the user has asked to be backed out then --¦ If transaction identifier of before-imags has been found in a backout failure log record then ¦ Apply the before-image Else Ignore as there has not been a backout failure log record for this transaction identifier, this task did NOT
9 ¦ fail backout.
¦ End if Else ¦ Ignore End if End if If first log record for a -transaction then If this transaction identifier is recorded against any databases then Remove this record If there are any date bases that had the transaction identifier and a backout failure log record FIRST has been read for the database then Removs the database from the list of databases to be backed out If there are no databases to be backed out left t.hsn ¦ End of job End if End if End if End if .
Claims (7)
1. A method of removing uncommitted changes made to stored data by a database management system, wherein the system stores a duplicate copy of data prior to effecting changes to the original copy, and upon failure of the system, or a transaction or transactions effecting the changes replaces the copy containing uncommited changes by the duplicate copy, characterised in that, in the case of failure of the system to replace the original copy (ie in the case of a backout failure), the system prevents subsequent attempts by the system to further change the data until the uncommitted changes have been removed by a process of backout failure processing, and wherein a batch backout utility program uses a set of system log records to bracket the duration of the backout failure for a particular file to track:
whether the system completed its backout failure processing;
which transactions had failed backout; and which transaction was the first to have a backout failure.
whether the system completed its backout failure processing;
which transactions had failed backout; and which transaction was the first to have a backout failure.
2. A method as claimed in claim 1 wherein the database management system is a transaction processing system.
3. A method as claimed in claim 1 wherein the system contains a backout failure log record, comprising the information that the batch backout utility will need, to calculate if it is safe to attempt backout, where to start, where to stop and which original records it must re-apply in order to backout uncommitted changes.
4. A method as claimed in claim 3 wherein fields in the backout failure log record include a FIRST flag set if it is the first backout failure log record for a given data set; dataset name; and transaction identifier.
5. A method as claimed in claim 3 wherein there are three different types of backout failure log record:
(i) backout failure log record FIRST, indicating which task was the first to have a backout failure for a given dataset;
(ii) backout failure log record SUBSEQUENT, indicating which tasks had backout failures subsequently for a given dataset;
(iii) backout failure log record END, indicating that backout failure has completed for a given dataset.
(i) backout failure log record FIRST, indicating which task was the first to have a backout failure for a given dataset;
(ii) backout failure log record SUBSEQUENT, indicating which tasks had backout failures subsequently for a given dataset;
(iii) backout failure log record END, indicating that backout failure has completed for a given dataset.
6. A method as claimed in claim 4 wherein there are three different types of backout failure log record:
(i) backout failure log record FIRST, indicating which task was the first to have a backout failure for a given dataset;
(ii) backout failure log record SUBSEQUENT, indicating which tasks had backout failures subsequently for a given dataset;
(iii) backout failure log record END, indicating that backout failure has completed for a given dataset.
(i) backout failure log record FIRST, indicating which task was the first to have a backout failure for a given dataset;
(ii) backout failure log record SUBSEQUENT, indicating which tasks had backout failures subsequently for a given dataset;
(iii) backout failure log record END, indicating that backout failure has completed for a given dataset.
7. A method as claimed in claim 5 or claim 6 wherein the backout failure logic is represented by pseudo code as follows:
If this data base has already had a backout failure then Write a backout failure log record SUBSEQUENT
Else Write a backout failure log record FIRST
Record that this data base has had a backout failure Ensure subsequent attempts to update the data base are disallowed End if When all tasks using this data base have completed Close the data base Write a backout failure log record END.
If this data base has already had a backout failure then Write a backout failure log record SUBSEQUENT
Else Write a backout failure log record FIRST
Record that this data base has had a backout failure Ensure subsequent attempts to update the data base are disallowed End if When all tasks using this data base have completed Close the data base Write a backout failure log record END.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP89305987A EP0402542B1 (en) | 1989-06-13 | 1989-06-13 | Method of removing uncommitted changes to stored data by a database management system |
| EP89305987.3 | 1989-06-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2018932A1 true CA2018932A1 (en) | 1990-12-13 |
Family
ID=8202715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002018932A Abandoned CA2018932A1 (en) | 1989-06-13 | 1990-06-13 | Method of removing uncommitted changes made to stored data by a database management system |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US5437026A (en) |
| EP (1) | EP0402542B1 (en) |
| JP (1) | JP2531835B2 (en) |
| AT (1) | ATE148241T1 (en) |
| BR (1) | BR9002804A (en) |
| CA (1) | CA2018932A1 (en) |
| DE (1) | DE68927705T2 (en) |
Families Citing this family (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2276737A (en) * | 1993-03-30 | 1994-10-05 | Ibm | Fault-tolerant transaction-oriented data processing |
| US6920467B1 (en) * | 1993-11-26 | 2005-07-19 | Canon Kabushiki Kaisha | Avoiding unwanted side-effects in the updating of transient data |
| GB2301910B (en) * | 1995-06-07 | 1999-07-21 | Ibm | Management of units of work on a computer system log |
| US5745750A (en) * | 1995-12-15 | 1998-04-28 | International Business Machines Corporation | Process and article of manufacture for constructing and optimizing transaction logs for mobile file systems |
| US5761678A (en) * | 1996-06-26 | 1998-06-02 | International Business Machines Corporation | Creation of clone storage area with identification of base storage area and deferred cloning of metadata |
| US5873098A (en) * | 1997-04-23 | 1999-02-16 | Oracle Corporation | Dynamic snapshot set adjustment |
| US5951695A (en) * | 1997-07-25 | 1999-09-14 | Hewlett-Packard Company | Fast database failover |
| US6173292B1 (en) | 1998-03-04 | 2001-01-09 | International Business Machines Corporation | Data recovery in a transactional database using write-ahead logging and file caching |
| US6092086A (en) * | 1998-03-31 | 2000-07-18 | Bmc Software | System and method for handling backout processing during capture of changed data in an enterprise computer system |
| GB2343020A (en) | 1998-10-19 | 2000-04-26 | Ibm | Handling transaction failures in a transaction processing system |
| JP4237354B2 (en) * | 1999-09-29 | 2009-03-11 | 株式会社東芝 | Transaction processing method and transaction processing system |
| US6587962B1 (en) * | 1999-10-20 | 2003-07-01 | Hewlett-Packard Development Company, L.P. | Write request protection upon failure in a multi-computer system |
| GB0006413D0 (en) * | 2000-03-17 | 2000-05-03 | Ibm | A dynamic shortcut to reverse autonomous computer program actions |
| US6990466B1 (en) * | 2000-08-08 | 2006-01-24 | International Business Machines Corporation | Method and system for integrating core banking business processes |
| US20030088783A1 (en) * | 2001-11-06 | 2003-05-08 | Dipierro Massimo | Systems, methods and devices for secure computing |
| US7103603B2 (en) * | 2003-03-28 | 2006-09-05 | International Business Machines Corporation | Method, apparatus, and system for improved duplicate record processing in a sort utility |
| US7577806B2 (en) | 2003-09-23 | 2009-08-18 | Symantec Operating Corporation | Systems and methods for time dependent data storage and recovery |
| US7730222B2 (en) | 2004-08-24 | 2010-06-01 | Symantec Operating System | Processing storage-related I/O requests using binary tree data structures |
| US7827362B2 (en) | 2004-08-24 | 2010-11-02 | Symantec Corporation | Systems, apparatus, and methods for processing I/O requests |
| US7577807B2 (en) | 2003-09-23 | 2009-08-18 | Symantec Operating Corporation | Methods and devices for restoring a portion of a data store |
| US7631120B2 (en) | 2004-08-24 | 2009-12-08 | Symantec Operating Corporation | Methods and apparatus for optimally selecting a storage buffer for the storage of data |
| US7725760B2 (en) | 2003-09-23 | 2010-05-25 | Symantec Operating Corporation | Data storage system |
| US7409587B2 (en) * | 2004-08-24 | 2008-08-05 | Symantec Operating Corporation | Recovering from storage transaction failures using checkpoints |
| US7991748B2 (en) | 2003-09-23 | 2011-08-02 | Symantec Corporation | Virtual data store creation and use |
| US7904428B2 (en) | 2003-09-23 | 2011-03-08 | Symantec Corporation | Methods and apparatus for recording write requests directed to a data store |
| US7287133B2 (en) | 2004-08-24 | 2007-10-23 | Symantec Operating Corporation | Systems and methods for providing a modification history for a location within a data store |
| US20050125423A1 (en) * | 2003-12-04 | 2005-06-09 | Hsien-Cheng Chou | Method to provide a filter for the capture program of IBM/DB2 data replication |
| US7756838B2 (en) * | 2005-12-12 | 2010-07-13 | Microsoft Corporation | Robust end-of-log processing |
| US8015215B2 (en) * | 2006-08-24 | 2011-09-06 | Oracle America, Inc. | Delegation in a file system with distributed components |
| US20090063807A1 (en) * | 2007-08-29 | 2009-03-05 | International Business Machines Corporation | Data redistribution in shared nothing architecture |
| KR101548222B1 (en) * | 2013-10-18 | 2015-08-31 | ㈜윈웨이시스템 | Data Backup System and Method thereof |
| CN109696902B (en) * | 2018-12-06 | 2022-06-10 | 奇瑞汽车股份有限公司 | Signal recording device, fault point detection circuit and method |
| CN111221671B (en) * | 2019-11-27 | 2024-03-22 | 中国银行股份有限公司 | Processing method and device for abnormal exit of application program |
| CN111444039B (en) * | 2019-12-30 | 2023-03-21 | 中国银联股份有限公司 | Cache data rollback method and cache data rollback device |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4507751A (en) * | 1982-06-21 | 1985-03-26 | International Business Machines Corporation | Method and apparatus for logging journal data using a log write ahead data set |
| US5155678A (en) * | 1985-10-29 | 1992-10-13 | International Business Machines Corporation | Data availability in restartable data base system |
| US4751702A (en) * | 1986-02-10 | 1988-06-14 | International Business Machines Corporation | Improving availability of a restartable staged storage data base system that uses logging facilities |
| US4868744A (en) * | 1986-03-03 | 1989-09-19 | International Business Machines Corporation | Method for restarting a long-running, fault-tolerant operation in a transaction-oriented data base system without burdening the system log |
| US4878167A (en) * | 1986-06-30 | 1989-10-31 | International Business Machines Corporation | Method for managing reuse of hard log space by mapping log data during state changes and discarding the log data |
| US4888681A (en) * | 1987-10-19 | 1989-12-19 | International Business Machines Corporation | Space management system for data files having shared access |
| US5077658A (en) * | 1987-10-19 | 1991-12-31 | International Business Machines Corporation | Data access system for a file access processor |
| US5043866A (en) * | 1988-04-08 | 1991-08-27 | International Business Machines Corporation | Soft checkpointing system using log sequence numbers derived from stored data pages and log records for database recovery |
| US4945474A (en) * | 1988-04-08 | 1990-07-31 | Internatinal Business Machines Corporation | Method for restoring a database after I/O error employing write-ahead logging protocols |
| US5123104A (en) * | 1988-04-08 | 1992-06-16 | International Business Machines Corporation | Method and apparatus for concurrent modification of an index tree in a transaction processing system utilizing selective indication of structural modification operations |
-
1989
- 1989-06-13 DE DE68927705T patent/DE68927705T2/en not_active Expired - Fee Related
- 1989-06-13 EP EP89305987A patent/EP0402542B1/en not_active Expired - Lifetime
- 1989-06-13 AT AT89305987T patent/ATE148241T1/en not_active IP Right Cessation
-
1990
- 1990-06-11 JP JP2150088A patent/JP2531835B2/en not_active Expired - Lifetime
- 1990-06-13 BR BR909002804A patent/BR9002804A/en unknown
- 1990-06-13 CA CA002018932A patent/CA2018932A1/en not_active Abandoned
-
1993
- 1993-06-28 US US08/083,860 patent/US5437026A/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| DE68927705T2 (en) | 1997-07-17 |
| ATE148241T1 (en) | 1997-02-15 |
| EP0402542B1 (en) | 1997-01-22 |
| US5437026A (en) | 1995-07-25 |
| DE68927705D1 (en) | 1997-03-06 |
| EP0402542A1 (en) | 1990-12-19 |
| JP2531835B2 (en) | 1996-09-04 |
| JPH0324649A (en) | 1991-02-01 |
| BR9002804A (en) | 1991-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5437026A (en) | Removing uncommitted changes made to stored data by a database management system | |
| US6185699B1 (en) | Method and apparatus providing system availability during DBMS restart recovery | |
| Verhofstad | Recovery techniques for database systems | |
| US5561795A (en) | Method and apparatus for audit trail logging and data base recovery | |
| US6993537B2 (en) | Data recovery system | |
| EP0336546B1 (en) | System and method for data recovery in a database on a computer system after an error in the system | |
| US6324548B1 (en) | Database backup and recovery using separate history files for database backup and audit backup | |
| EP0516900B1 (en) | Data backup and recovery in a data processing system | |
| US5778387A (en) | Database automated recovery system | |
| US5740434A (en) | System for maintenance of database integrity | |
| US7836267B1 (en) | Open computer files snapshot | |
| US20040181560A1 (en) | Method and apparatus for interrupting updates to a database to provide read-only access | |
| CN110209527B (en) | Data recovery method, device, server and storage medium | |
| US11010256B1 (en) | Method and system for implementing current, consistent, and complete backup copy by rolling a change log backwards against a storage device | |
| US5745674A (en) | Management of units of work on a computer system log | |
| WO2002082276A1 (en) | System and method for reorganizing stored data | |
| Eich | Main memory database recovery | |
| Dudjak et al. | Survey of database backup management | |
| Sullivan | System support for software fault tolerance in highly available database management systems | |
| CN112882861B (en) | Service configuration data loading and recovering system and method | |
| JPS63262737A (en) | Data base updating and recording processing method | |
| EP0834128A1 (en) | Data set backup in a shared environment | |
| JPH01293452A (en) | Data base fault recovering system | |
| Kumar et al. | Recovery in Oracle | |
| Emanuel | Database recovery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDE | Discontinued |