Image Transaction Logging and True Online Backup
If you are planning to implement a combination of Online Backup with Image logging, or have already done so, then you will want to pay close attention to the following information. We recently solved a problem involving the recovery of data from Image Log files that was, to say the least, not straightforward. In the context of this article, Online Backup refers to any backup product that features the option to generate a full system backup without logging off any users.
Before explaining the problem let me give a brief history of Image logging and recovery. Image transaction logging is a facility that records each and every transaction that modifies a dataset into log files for later playback in the event of a disk failure, or any other problem, which requires restoring a database. The log files are typically created on a different volume set than the database, and are named something like LOG001, LOG002, LOG003 and so on. Image transaction recovery is designed to look at the time a database was last backed up, and recover all transactions that occurred beyond that timestamp. Unfortunately only the DBSTORE utility sets this flag (some of the third party backup solutions have an option to set the DBSTORE flag,) and DBSTORE is quite limited in its capabilities. Consequently, nobody uses DBSTORE, and none of the Image databases have the correct date/time stamp.
A workaround to the lack of a reliable timestamp and one which we have recommended for years, is to begin a new logging cycle after each backup, setting the beginning log file name back to LOG001. The recovery program (DBRECOV) has a mechanism for overriding the timestamp, allowing you to recover all of the transactions starting with LOG001. This allows for a clean, concise logging cycle. You know when it began, when your last backup was, and what needs to be recovered.
In a traditional backup scheme you log off all of your users and run your backup, therefore you know exactly when your backup is “as of.” Enter Online Backup, a solution that allows you to perform a complete system backup while all of your users remain online. Online Backup uses what is known as a “sync point” to determine the point in time when your files are consistent. When the sync point occurs, the backup program will quiesce (pause) all of the users, sync all of the system and Image log files, and record the date/time stamp and current log file number in all of the Image root files.
This is where the problem begins. The Image root files have a flag indicating the current log file number is LOGxxx because that is the log file that was active at the sync point. After the backup completes the logging cycle is restarted to LOG001. The DBRECOV program has options to override the traditional DBSTORE timestamp; however, these same control options have no affect in an Online Backup environment. That is to say DBRECOV ignores the control options. As the log file number (LOG001) does not match what is saved in the Image root file, DBRECOV will not recover any of the transactions.
In the example I am citing a customer had a disk failure and had to restore their databases from a backup that was more than 24-hours old, and DBRECOV would not salvage any of the transactions no matter how many of its checks and balances we disabled. I am pleased to tell you we were able to recover the data with a combination of wiping out the timestamp/log file information in the root files and using the override controls of DBRECOV. Furthermore it required the use of several third party utilities including Adager and MPEX. Nevertheless I would not want to rely on that solution as a plan.
We have determined that the best method to employ when combining Image logging and True Online Backup is to not refresh the logging cycle. Simply let the log files automatically roll over, and if you must recover transactions at some point in the future let DBRECOV work with the timestamp/log file number recorded in the root files.