Transaction back out

Overview

Transaction back out is a file integrity facility that can logically undo the effects of incomplete transactions on file data. This logical inversion of a transaction is called a back out.

Back outs can be initiated for incomplete transactions by an application or by Model 204. Model 204 automatically backs out an incomplete transaction for a transaction back out (TBO) file if a user's request is canceled, if Model 204 detects a file problem, such as a table full condition, or if Model 204 is restarting the user. Transaction back out is the default file designation.

This page describes the components of the transaction back out facility and how it works, its effects on a file and on file recovery, and some factors to consider when choosing whether to implement transaction back out for a file.

Components of the transaction back out facility

The two components of the transaction back out facility, lock pending updates and the back out mechanism, are separate file specific options that are enabled or disabled on a file-by-file basis. To use the transaction back out facility against a file, both of the following components must be enabled:

Component	Function
Lock pending updates	When lock pending updates is enabled for a transaction back out file, all records updated within a transaction are enqueued exclusively. Each updated record is locked by the first update of the record in a transaction. When the transaction completes successfully or is backed out, the exclusive lock on the set of updated records is released.
Back Out mechanism	The back out mechanism logically inverts the effects of an update to a file by issuing compensating updates. For example, a back out of a STORE RECORD statement is logically equivalent to processing a DELETE RECORD statement. The back out of a transaction can be initiated only on active transactions; completed transactions cannot be backed out. Invoking the back out mechanism ends the transaction. A back out operation itself cannot be backed out.

Benefits of transaction back out

Through the locking and back out mechanisms, transaction back out provides many benefits, including logical consistency and data integrity. Some of these advantages are:

• Eliminating the possibility of cascading roll back (see the discussion about update units during recovery in Understanding update units.
• Roll forward of transaction back out files maintains logical consistency while minimizing the loss of work, because only incomplete transactions are not rolled forward.
• Transaction back out files are less likely to become physically inconsistent, because file table full conditions do not break transaction back out files, and because an incomplete transaction is automatically backed out of a transaction back out file during a user restart.

Designing transaction back out applications

To maximize these benefits, carefully consider the effects of the locking mechanism and transaction definition when designing a transaction back out application.

Concurrency control and locking mechanisms

Concurrency control manages the reading and writing of stored data so that concurrently executing programs do not interfere with each other. Concurrency control is achieved by the use of locking (or enqueuing) mechanisms.

User control of locking mechanisms

Model 204 locking mechanisms can be controlled to some extent by the user to provide the level of concurrency control required:

• Users can choose between the FIND statement, which locks the found set of records in share mode, and the FIND AND RESERVE statement, which locks the found set of records in exclusive mode.
• Update statements within FOR loops attempt to lock the current record in exclusive mode. For files that are not lock pending updates files, the exclusive lock on the record is always reduced to a share lock at the bottom of the FOR loop.

When to use locking mechanisms

The locking mechanism allows a high degree of data sharing but might not be desirable for certain applications. Because the update unit might be longer than the FOR loop exclusive lock on the updated record, other users or applications might update the same record before the first update unit completes. When update units overlap in this way, logical consistency problems arise, and transaction back out is not possible. (Update units are fully described in Update units and transactions.)

The Logical consistency errors figure illustrates the logical consistency problem. The example involves two record types: checking and savings. Each record contains NAME, BALANCE, and record type (RECTYPE) fields.

In this example, update A transfers $100 from Fred's checking account to his savings account. Update B finds the total balance of Fred's checking and savings accounts in order to determine whether to bill Fred for a service charge. Fred is billed if his combined balance is less than $100. Fred actually has more than $100 in the bank, but if update B is allowed to read update A's uncommitted data, a logical inconsistency might occur and Fred is billed.

If update A's records are locked until the update unit is complete, no inconsistency occurs. The FIND AND RESERVE statement can be used to lock update A's records, but no other request can access any data in the FIND AND RESERVE set from the time the reserve is initiated until the reserve is released. If the found set of records is larger than the set of records actually updated this technique locks records unnecessarily.

The lock pending updates mechanism can also be used to lock update A's records. In many cases, lock pending updates allows for more data sharing than FIND AND RESERVE because only those records actually updated are locked exclusively.

Logical consistency errors

Using lock pending updates

Lock pending updates files

In a lock pending updates file, the first update to a record places an exclusive lock on the record by adding it to the set of updated, locked records called the pending update pool. The record is not released from this exclusive lock at the end of the FOR loop. Instead, the record remains locked until the end of the update unit when the entire pending update pool is released. This locking mechanism ensures that when only lock pending updates files are involved in an application, the records involved in concurrent update units do not overlap.

Locking mechanism

The locking mechanism prevents updated records in one transaction from being used by other applications until Model 204 establishes whether the update will or will not be backed out. An update can be backed out without affecting the logical validity of any other update. Lock pending updates can be used without the back out mechanism of transaction back out, in order to provide logical consistency without requiring the use of the FIND AND RESERVE statement. Lock pending updates does not affect the use of the FIND AND RESERVE statement to avoid record enqueuing conflicts during updating, but it does postpone the release of the exclusive enqueue on the record until the update unit has ended.

Updates that do not lock records

Deleting records

Even in lock pending update files, there are some updates that do not lock the records involved. Records updated by DELETE RECORD or DELETE RECORDS statements, or the corresponding Host Language Interface functions, IFDREC and IFDSET, are not added to the pending update pool, and are, therefore, not locked for the duration of the update unit. Deleted records are not locked, because they no longer exist in the file and, thus, cannot be accessed by other update units. (For an example of a logical inconsistency arising from a DELETE RECORD statement, see Operating considerations.)

Filing records

Records updated by a FILE RECORDS statement or IFFILE Host Language function are not added to the pending update pool, and, therefore, are not exclusively locked, allowing logical inconsistencies to occur in some applications. These logical inconsistencies can be prevented by maintaining a FIND or FIND AND RESERVE on the set of records updated by the FILE RECORDS statement until the update unit ends.

DELETE statements and FILE RECORDS updates are the only exceptions to the lock pending updates lock on updated records.

Creating a lock pending updates file

A file is set to lock pending updates if the FOPT parameter has the X'02' bit turned off. This is the default. For instance, an FOPT setting of X'00' has lock pending updates enabled. Turning on the X'02' bit in the FOPT parameter disables lock pending updates. With lock pending updates disabled, the current record is released from its exclusive lock at the end of the FOR loop and the file cannot use transaction back out.

Lock pending updates considerations

Consider the following issues before deciding whether or not to select the lock pending updates file option:

• Lock pending updates option is a prerequisite for using the back out mechanism of transaction back out.
• Use of lock pending updates helps ensure logical consistency.
• Application that uses lock pending updates files might have more enqueuing conflicts than an application involving files that do not have lock pending updates.
• Application that uses lock pending updates files might have fewer enqueuing conflicts than an application that uses the FIND AND RESERVE statement to ensure logical consistency.

Using the back out mechanism

A back out can be requested by the application or issued automatically by Model 204, as in the cases of:

• Request cancellation
• Hard restart
• Run-time error

To issue a back out, the updated records must be locked exclusively so that no other update units are affected. This exclusive locking is provided by the lock pending updates file option.

The file also must be a transaction back out file (see Transaction back out file parameter settings) so that transaction back out logging is active. A transaction back out log stores compensating updates, and a constraints log prevents reuse of freed file resources until the transaction ends.

The following sections provide further information about requested back outs and automatic back outs.

Requested back outs

Requesting back outs in SOUL

A SOUL application can back out an incomplete transaction using the BACKOUT statement. The BACKOUT statement is valid only for transaction back out files. For a complete description and example of the use of the BACKOUT statement, see Manual backout.

For information about using BACKOUT with remote files for PQO sites, see PQO: Remote files and scattered groups.

Requesting back outs through the Host Language Interface

A Host Language Interface application can back out an incomplete transaction using the IFBOUT function.

Automatic back outs

Model 204 automatically backs out the current transaction for a transaction back out file if:

• SOUL request or Host Language function is canceled by Model 204
• File problem, such as a table full condition, occurs
• Model 204 is restarting a user who has a transaction in progress

When a SOUL request accessing a transaction back out file is canceled:

1. The user receives a message in the format:

   *** M204.nnnn: CANCELLING REQUEST: message

   Where:

   message is a Model 204 error message explaining why the request was canceled.

2. The back out mechanism automatically backs out the current transaction.
3. Model 204 returns to the terminal command level.

User restarts

If the user is restarted because of a file problem or a system problem that affects the file, Model 204 attempts to back out the active transaction in order to preserve file and data integrity.

Back outs caused by table full conditions

For example, a file table full condition that arises during an active transaction causes a user request cancellation and an automatic back out of the transaction. The file is not marked physically inconsistent, and Model 204 returns to terminal command level without restarting the user:

• If the request cancellation is caused by a Table A full condition, the file is marked full.
• If the request cancellation is caused by a Table B, Table C, or Table D full condition, the file is not marked full, because the back out of the active transaction reclaims some of the table space.

In any case, the file usually needs to be reorganized or have space added using the INCREASE command. File table full conditions that arise during non-backoutable update units cause a hard restart. (Examples of non-backoutable update units are the DEFINE FIELD command and updates to a non-transaction back out file.)

Canceling requests with *CANCEL or the attention key

A request is canceled and control is returned to terminal command level if the user presses the attention key or types *CANCEL at a terminal I/O point when an ON ATTENTION unit is not active in the request. (Attention keys are designated ATTN, BREAK, or PA1, depending on the terminal type.) The following message is sent to the audit trail explaining why the request was canceled:

*** CANCELLING REQUEST: M204.1338: NO ON ATTENTION UNIT

The request cancellation also causes an automatic back out of the current transaction if the file is a transaction back out file.

Using ON units with transaction back out files

Using ON ERROR

The ON ERROR unit allows the application to regain control of the request after the transaction has been backed out. ON ERROR is an ON unit that allows the user to specify the action to be taken if Model 204 cancels a request. After a request is canceled, or for transaction back out files after the active transaction is backed out, the ON ERROR unit is processed instead of returning control to the terminal command level.

An ON ERROR unit can be specified in any SOUL request, and it is further described in ON units.

Using ON ATTENTION

Use an ON ATTENTION unit to avoid inadvertently backing out an active transaction. If an ON ATTENTION unit is specified in the request, Model 204 performs the action specified in the ON ATTENTION unit when the user presses the attention key or enters *CANCEL.

Transaction back out file parameter settings

Transaction back out is implemented for a file if:

• FOPT parameter has the X'02' bit turned off (enabling lock pending updates).
• FRCVOPT parameter has the X'08' bit turned off (enabling the back out mechanism).

This is the default.

Turning off transaction back out

For instance, a file that has the FOPT parameter set to X'00' and the FRCVOPT parameter set to X'00' is a transaction back out file. Turning on the X'08' bit of the FRCVOPT parameter disables the back out mechanism of transaction back out. The transaction back out facility requires that both the back out mechanism and the lock pending updates mechanism be activated. If lock pending updates is disabled for a file (FOPT has the X'02' bit turned on), Model 204 automatically disables transaction back out for the file by turning on the X'08' bit of the FRCVOPT parameter and issuing the following message:

M204.2099: FRCVOPT FORCED TO 'NO TRANSACTION BACKOUT'

With transaction back out disabled, updates become non-backoutable update units that cannot be logically reversed.

Non-backoutable update units are defined in Update units and transactions.

In addition to the individual FRCVOPT and FOPT parameter description pages, see the discussion of FRCVOPT and FOPT as used in file integrity and recovery.

Completed transactions

After a transaction has ended, whether Host Language Interface or SOUL, the transaction cannot be backed out, and its updates persist in the file. The application chooses when to end or complete a transaction by executing one of the Model 204 commands, SOUL statements, or Host Language functions that ends a transaction. See Update units and transactions for a list of these commands, statements, and functions.

Roll-forward for completed transactions

During the roll forward phase of recovery, all completed transactions are reapplied to transaction back out files.

Effects of transaction back out on files

A transaction back out issues updates that compensate for the transaction's effect on the file. It removes the effects of an update on the file data, but does not reclaim record numbers and might not reclaim all the file space used by the backed out transaction.

The effects of transaction back out on the file for specific transaction types are described in the following sections.

Updates that add data

SOUL statements that add data to a file include STORE RECORD, ADD, and INSERT. Backing out additions to the file has the same effect on the file as if the additions were deleted.

For example, backing out a STORE RECORD statement leaves the file in the same state as if the stored record had been deleted with a DELETE RECORD statement. Backing out an ADD or INSERT statement has a similar effect on field name = value pairs in a record; it is as though a DELETE fieldname = value statement had been issued. Additions backed out from a file are reflected in the file parameters as deletes. For example, the back out of a STORE RECORD statement increments the MSTRDEL parameter.

Record numbers in backed-out files

None of the record numbers used are made available for reuse unless the transaction back out file has the unordered file type. As is the case with any deletion, no Table A updates are removed by the back out of a transaction. Field names generated by NUMERIC RANGE entries, CODED field values, and FOR EACH VALUE entries remain in Table A after a back out.

Updates that delete data

Backing out a DELETE transaction returns the file data to its state before the DELETE was processed. When the DELETE RECORDS statement is backed out, the deleted record reappears in the file. The back out of a DELETE fieldname statement has a similar effect on field name value pairs in a record.

DELETE statements backed out from a file remove the increment to the file parameters resulting from the DELETE. For example, DELETE RECORD statements from an entry order file are reflected in the parameter MSTRDEL. When a record is deleted, the value of the MSTRDEL parameter is increased by 1. If the deletion of that record is backed out, the value of the MSTRDEL parameter is decreased by 1. The MSTRDEL parameter is left at its original value.

Updates that change data

Back outs of changes to the file data are processed as inverse changes. When the CHANGE fieldname TO value statement is backed out, the new field value is replaced with the value of the field before the transaction began. Changes backed out from a file are not reflected in the file parameters.

Updates that build extension records

Backing out an update that built an extension record deletes the extension record. The record number used by the deleted extension record is not available for reuse, unless the reuse record number option (RRN) of the FILEORG parameter is turned on. This is true for any normal file update that might delete an extension record (such as a DELETE fieldname statement).

Incrementing update statistics

An update that creates an extension record increments the EXTNADD statistic, whether or not the update occurs as part of a back out. Similarly, any update that deletes an extension record increments the EXNTDEL statistic, whether or not that update is part of a back out operation.

Transaction back out logging

The back out mechanism must have the necessary information to perform a back out on each active transaction in the Model 204 run. For example, if a DELETE RECORD transaction is processed, all the information necessary to rebuild the entire deleted record must be available to Model 204, in case the transaction containing the DELETE RECORD operation is backed out. A log of compensating updates is built for each active transaction. After a transaction ends or is backed out, the log for that particular transaction is discarded.

Transaction back out logging keeps two logs in CCATEMP space: a transaction back out log, which logs the active transactions; and a constraints log, which tracks resources freed by file deletes and changes.

Transaction back out and constraints logs

For performance reasons, the size of the transaction back out and constraints logs should be substantially smaller than CCATEMP. Factors that increase the number of transaction back out and constraint records generated are:

• Number of users actively performing updating transactions
• Complexity and length of the transactions
• Number of records being accessed during the transaction.

To keep the number of log records to a minimum, make transactions as brief as possible and execute COMMIT statements whenever feasible. For more information about calculating the size of the logs and CCATEMP, see Logging ROLL FORWARD processing.

Transaction back out log

The transaction back out log is kept in CCATEMP. At least one CCATEMP page is always used for transaction back out logging. Theoretically, the only upper limit on the number of pages that can be used is the CCATEMP size. However, other uses of CCATEMP actually limit the number of pages much further.

Constraints log

DELETE RECORD, DELETE field name, and (sometimes) CHANGE are updates that free table space and other file resources. It is important that such freed file resources are not reused by other active transactions until the transaction that freed the resources ends. Otherwise, it is impossible to guarantee the success of a back out operation for the transaction. Model 204 keeps a constraints log, in specially formatted CCATEMP pages, to guarantee the availability of resources freed by an active transaction.

Constraints exist only during the life of a transaction. When a transaction ends, all of that particular transaction's constraint records are discarded.

Recovery and transaction back out

The transaction back out facility improves the recoverability of Model 204 files should system recovery be necessary. Because of the logical independence of transactions in the original run, transaction back out files can be rolled forward through the last completed transaction with logical consistency guaranteed. Loss of work is minimized, because only incomplete transactions are not rolled forward.

Transaction back out files are less likely to become physically inconsistent, because file table full conditions do not break transaction back out files, and because an incomplete transaction is automatically backed out of a transaction back out file during a hard restart.

For a complete description of recovery and the Roll Forward operation, see ROLL FORWARD facility.

Operating considerations

This section describes the factors to consider when:

• Designing an application that uses transaction back out files
• Using transaction back out files for an existing application
• Managing a system that has transaction back out files

Lock pending updates files

Transaction back out files must be lock pending updates files. Because of the lock pending updates exclusive lock on updated records, an application involving lock pending updates files can have many enqueuing conflicts. Keep this in mind when designing transaction back out applications or using transaction back out files in existing applications.

Files used with Parallel Query Option/204

For Parallel Query Option/204 (PQO), files that are accessed remotely must be transaction back out files. If you attempt to open a non-TBO file in remote context the open fails with the following error message:

M204.1973: NON-TBO REMOTE FILE

Logical consistency and DELETE RECORD or FILE RECORDS

Although transaction back out does increase the logical consistency enforced on a file through the lock pending updates option, there are still some possibilities for logical inconsistency when an update transaction is backed out. Because deleted records are not locked by lock pending updates, a logical inconsistency can arise when a transaction involving DELETE RECORD is backed out.

A similar logical inconsistency can arise using the FILE RECORDS statement, because FILE RECORDS does not lock any records. In the case of the FILE RECORDS statement, a FIND statement can be used to lock the records to be updated in share mode, or FIND AND RESERVE can be used to lock the records in exclusive mode. The application can choose whichever locking mechanism is desired. If the FIND set is locked exclusively and not released until after the update unit containing the FILE RECORDS statement has ended, logical inconsistency is prevented.

Example

The following example shows how DELETE RECORDS can cause logical inconsistencies in files.

User 1	User 2
BEGIN SMH: FIND ALL RECORDS FOR WHICH NAME = SMITH END FIND FOR EACH RECORD IN SMH DELETE RECORD END FOR JNS: FIND ALL RECORDS - FOR WHICH NAME = JONES END FIND FOR EACH RECORD IN JNS ADD TYPE = SPECIAL . . . . . . . . . . DEL: BACKOUT ALL: FIND ALL RECORDS END FIND FOR EACH RECORD IN ALL PRINT NAME AND TYPE END FOR END	BEGIN RECS: FIND ALL RECORDS FOR WHICH NAME = SMITH OR - NAME = SAUNDERS END FIND FOR EACH RECORD IN RECS ADD TYPE = SPECIAL END FOR SPEC: FIND ALL RECORDS - FOR WHICH TYPE = SPECIAL END FIND FOR EACH RECORD IN SPEC PRINT NAME AND TYPE END FOR END

User 1 deletes the set of records with NAME=SMITH. The transaction is not complete, but the deleted set of records is not locked. Before user 1's transaction completes, user 2 finds the set of records with either NAME=SMITH or NAME=SAUNDERS. There is no enqueuing conflict, because the deleted records are not locked. User 2 adds a field to his found set, prints a report, and ends. User 1 finally gets to a point where he decides to back out the transaction in progress. The deletion of the Smith records is in the backed out transaction and the Smith records reappear in the file.

However, because the Smith records do not have the TYPE=SPECIAL field added by user 2, there is a logical inconsistency. The chances of this type of inconsistency can be reduced by keeping transactions short, especially when the transactions are used to delete records. If user 1 had a COMMIT statement after the delete record loop, no inconsistency would arise unless the back out mechanism were automatically activated during that short piece of the user request. It is also quite likely that the records an application deletes are meant to be deleted, and a logical inconsistency in a set of records intended for deletion is unimportant.

CCATEMP space

Transaction back out files make demands on CCATEMP space. Both transaction back out logs and constraint records are kept in CCATEMP. A larger CCATEMP space might be needed if transaction back out files are used. The CCATEMP space required for logging and constraints can be minimized if transactions are kept short. If transaction back out logging or constraints fill up CCATEMP, Model 204 backs out the active transaction and cancels the user request. The user is not restarted.

Performance considerations for rerunable BATCH jobs

For better performance while loading rerunable BATCH jobs, turn transaction back out off (FRCVOPT X'08' on). However, before running the batch job with transaction back out off, take one of the following precautions:

• Back up the file. This allows you to run in BATCH very quickly. If the system crashes, however, you must restore the entire file.
• Turn on checkpoint logging. Although this slows down the BATCH run, it allows for faster recovery in case of a system crash. If you use checkpoint logging, use Roll-Back Recovery to restore the file. Rocket Software recommends this method if you are updating only a small number of pages.

Performance considerations for transaction back out and constraints logs

Transaction back out logging and constraint records' requirements might degrade performance if transactions are allowed to continue over many file updates, especially in multiuser environments. Performance is dependent on the complexity of the simultaneously active transactions in the system. Again, if the transactions are kept short so that they contain only a few file updates, the effect on performance is minimal.

Terminal I/O with transactions

Rocket Software recommends that terminal I/O points not be placed between the start of a transaction and the transaction's end or back out. This precaution stems from the exclusive lock placed on updated records. If a response is required from a terminal either to complete or back out a transaction, there is a possibility that a set of records might be locked for a long time quite unintentionally.

The ability to use COMMIT inside FOR EACH RECORD loops and the flexibility of boundary setting for transactions and non-backoutable update units allows applications to avoid this situation and places terminal I/O points outside of transactions or non-backoutable update units.

Considerations when using transaction back out files

A transaction back out file has many advantages, but the feature must be used carefully to obtain the benefits of these advantages. The key point is not to abuse the application's privilege of defining the number of operations in a transaction. If possible, ensure that transactions do not include many updates to a file or span terminal I/O points in the application.

Updates to TBO and non-TBO files in the same request

You can update a non-TBO file, commit it, and then update a TBO file — or reverse file order — in the same request.

Requests that attempt to update TBO and non-TBO files without an intervening COMMIT statement will compile, but they will fail during evaluation with the following message:

*** 1 Cancelling request; M204.2771: Attempt to update TBO and non-TBO files in the same transaction

Note: If you need to store temporary application data, consider defining a class and using a SOUL collection.