HLI: Transactions

From m204wiki
Revision as of 16:40, 29 April 2016 by ELowell (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Overview

This topic describes Model 204 transaction processing for the application programmer who is using the Host Language Interface facility. It details the action of HLI calls that perform update operations against the database and gives information about backing out an incomplete transaction for a TBO file.

Read the sections that pertain to multiple cursor IFSTRT threads if you are using multiple cursor functionality in your host language program for the first time.

For more information about recovery and checkpointing, see:

File integrity and recovery and Checkpoints: Storing before-images of changed pages

Transaction processing

In general terms, a transaction is a sequence of operations against the database to access data. The order of the operations is defined by the user in the application program. In data processing terms, there are two kinds of transactions, based on the type of data access:

  • Read-only
  • Updating

An updating transaction has particular significance when processing against the Model 204 database.

Update unit

An updating transaction corresponds to an HLI call or series of calls that perform(s) an update operation, that is, an operation that modifies the database. A Model 204 updating transaction is called an update unit. There are two types of update units:

  • Backoutable (that is, update units can be backed out), which includes:
    • Data manipulation updates to transaction back out (TBO) files.
  • Nonbackoutable (that is, update units cannot be backed out), including:
    • Data manipulation updates to non-TBO files
    • Procedure updates
    • Data definition updates, for example, using IFDFLD (DEFINE FIELD)

Transaction is an update unit

In the Model 204 environment, the following terminology applies:

  • The term transaction refers to a backoutable (TBO file) update unit.
  • An updating call is an HLI call that performs an update operation on a TBO file.

See Transaction back out facility for information about backing out transactions for TBO files.

Update unit boundaries

Update units for a multiple cursor IFSTRT thread

On a multiple cursor IFSTRT thread, a Model 204 transaction becomes active, that is, a backoutable update unit is started, when the first updating call is executed against the database. An updating call performs an update operation, that is, an operation that modifies the database.

The update unit continues until it is explicitly ended by an IFCMMT, IFCMTR, or IFBOUT call. The IFCMMT call ends the current transaction. A new update unit does not begin until the next updating call is issued. Until the new update unit begins, no transaction is active on the thread.

If the host language program using a multiple cursor IFSTRT thread does not perform any update processing, no update units are allocated and no transaction is activated for the thread.

Model 204 manages update units for a multiple cursor IFSTRT thread similarly to SOUL.

Update units for a single cursor IFSTRT thread

The boundaries of update units for single cursor IFSTRT threads are different from those for multiple cursor IFSTRT threads and SOUL. The thread that starts an update unit must be the thread that ends the update unit. When using IFDTHRD, your must assure in your coding that a change in threads does not occur within the context of a single update unit. Always call IFCMT or IFCMTR before calling IFDTHRD to assure that update unit processing starts and completes on the same thread.

For single cursor IFSTRT threads, an update unit is started by the first IFSTRT call issued by the host language program that starts an update thread (THRD_IND is 0 in IFAM1; THRD_IND is 1 in IFAM2 / IFAM4).

In general, a new update unit starts as soon as the previous update unit ends and a transaction remains active until host language processing is finished. The only HLI functions that end the current update unit and do not immediately start a new update unit are IFCHKPT and IFFNSH.

If an HLI job contains more than one IFSTRT thread with update privileges, all those threads participate in the same transaction. And if the transaction is backed out, all the updates for all the IFSTRT threads with update privileges are backed out.

Note: Read-only IFSTRT threads are not included in update units. Only single cursor IFSTRT threads with update privileges participate in Model 204 transactions.

When an update unit ends

When an update (backoutable or nonbackoutable) ends, Model 204 performs the following actions:

  • Writes the current journal buffer out to the journal (CCAJRNL)
  • Releases the exclusive lock on updated records in the pending updates pool for any files with the LPU option set
  • If the update unit is backoutable, frees the back out log and constraints log for the transaction (which means that the transaction can no longer be backed out)
  • Commits the update unit to the database (or databases) that were modified

When a transaction backs out

When a transaction backs out, all the updates performed by that transaction are undone and affected files are returned to the same state as before the transaction began.

After the back out is complete, the transaction ends, and Model 204 performs the same actions as for the end of an update unit. (See above.)

See HLI: Recovery and checkpoints for more information about backing out a transaction.

Update units: designing your application

Placing terminal I/O points outside update units

If a response is required from a terminal either to complete or back out a transaction, a set of records that is exclusively locked for the update unit might be unintentionally locked for a long time.

Avoid placing terminal I/O points between the start of a transaction or backoutable update unit and the transaction's end or back out. Place terminal I/O points outside of transactions.

Note: When using a single cursor IFSTRT thread, update units inhibit checkpoints and thus increase the amount of work to be done for recovery. See HLI: Model 204 recovery and checkpoints for more information about checkpointing in your host language application.

Unit of work for recovery

Update units have particular significance for Model 204 recovery. Recovery uses the starting and ending points of update units to return files to logically and physically consistent states. An update unit is a unit of work for recovery.

To plan for recovery when you are coding your HLI application program, you must know where update units begin and end.

HLI updating calls and update units

HLI calls that end the current update unit

On a multiple cursor IFSTRT thread, each of the following HLI calls ends the current update unit.

HLI call Equivalent SOUL command
IFBOUT BACK OUT
IFCMMT COMMIT
IFCMTR COMMIT RELEASE
IFCLOSE CLOSE
IFDTHRD, if the thread has a file open  
IFFNSH  

Note that, in group file context, IFCLOSE ends the transaction only if it results in the closing of a file and all files opened outside of the group have been closed. IFCLOSE does not end the transaction if all files in the group are opened singly.

On a single cursor IFSTRT thread with update privileges, all the calls listed above (except for IFCMTR) end the current update unit and start a new update unit, and CLOSE ALL is the only option for the IFCLOSE call.

HLI calls that start an undesignated update unit

On a single cursor IFSTRT thread with update privileges, each of the following HLI calls performs different actions depending on whether or not an update unit is active.

HLI call Equivalent SOUL command
IFDELF DELETE FIELD
IFDFLD DEFINE FIELD
IFINIT INITIALIZE
IFNFLD RENAME FIELD
IFRFLD REDEFINE FIELD
IFRPRM
IFSPRM		 RESET

SET

  If used to reset a file parameter

If no update unit is active

If no update unit is active when one of the previous HLI calls is issued, Model 204 performs the following actions:

  • Starts a nonbackoutable update unit
  • Processes the update
  • Ends the nonbackoutable update unit
  • Starts a new update unit

If an update unit is active

If an update unit is active when one of the previous HLI calls is issued, Model 204 performs the following actions:

  • Ends the active update unit
  • Starts a nonbackoutable update unit
  • Processes the update
  • Ends the nonbackoutable update unit
  • Starts a new update unit

HLI calls that start a backoutable update unit

On a single cursor IFSTRT thread with update privileges, each of the following HLI calls performs the following actions:

  • Ends a nonbackoutable update unit if one is active
  • Starts a backoutable update unit or continues a previously started backoutable update unit
    • HLI call Equivalent SOUL command
    IFBREC STORE RECORD
    IFDALL DELETE ALL
    IFDREC DELETE RECORD
    IFDVAL DELETE fieldname=value
    IFDSET DELETE RECORDS IN
    IFFILE FILE RECORDS
    IFPUT  

HLI threads and transactions

Model 204 manages transactions for the threads that are started by a host language application. Model 204 manages transactions differently for different types of threads, depending in which environment (IFAM1, IFAM2, IFAM4) the application is running.

Logical relationship of threads and transactions

The logical relationship between HLI threads and Model 204 transactions are shown in simplified form in the diagrams in the following sections.

The diagrams do not show the detailed elements of the interface, which are particular to each environment. Also, the diagrams provide an overview of HLI transactions and do not represent the preferred HLI job, which is recommended by Technical Support.

The figures use the following symbols:

Symbol Meaning
MC Multiple cursor IFSTRT thread
ST Single cursor IFSTRT thread
DI IFDIAL thread
T-n Model 204 transaction

See HLI: Threads for more information about using threads in HLI applications.

IFAM1 transaction

Single thread

In IFAM1, you can start a single thread in your HLI application program. The updating calls that are issued on the thread are processed in the transaction.

The following figure shows the logical relationship between an HLI thread in an IFAM1 application and a Model 204 transaction.

IFAM1 Transaction

IFAM2 transactions

One or more threads

In IFAM2, you can start one or more threads in your HLI application program.

Each multiple cursor IFSTRT thread and IFDIAL thread processes as a single transaction. In a multithreaded application, all single cursor IFSTRT threads process as a single transaction, as shown in the following figures.

The following figure shows the logical relationship between HLI threads in an IFAM2 application under z/OS or z/VSE and Model 204 transactions.

IFAM2 Transactions in z/OS or z/VSE

Note: Technical Support does not recommend that you mix multiple cursor and single cursor IFSTRT threads in a HLI application.

The following figure shows the logical relationship between HLI threads in an IFAM2 application under VM/CMS and Model 204 transactions. Under VM, you can start a single thread in your IFAM2 job if you are using IFDIAL.

IFAM2 Transactions in VM

Note: Technical Support does not recommend that you mix multiple cursor and single cursor IFSTRT threads in a HLI application.

IFAM4 transactions

One or more threads

In IFAM4, you can start one or more IFSTRT threads in your HLI application program.

Each multiple cursor IFSTRT thread processes as a single transaction. In a multithreaded application, all single cursor IFSTRT threads process together as a single transaction.

The following figure shows the logical relationship between HLI threads in an IFAM4 application and Model 204 transactions.

IFAM4 Transactions

Note: Technical Support does not recommend that you mix multiple cursor and single cursor IFSTRT threads in a HLI application.

Multithreaded IFAM2 and IFAM4 transactions

In IFAM2 and IFAM4, a host language job can start more than one thread. When you start more than one single cursor IFSTRT thread in your HLI application, Model 204 treats all of those threads as a single transaction.

For example, an IFBOUT executed on one of the single cursor IFSTRT threads backs out the updates for all of the single cursor IFSTRT threads in the application program. Likewise, an IFCMMT executed on one of the single cursor IFSTRT threads commits the updates for all of the single cursor IFSTRT threads in the program.

Multiple single cursor IFSTRT threads function as a separate transaction from an IFDIAL thread or any multiple cursor IFSTRT threads that are started by the application.

Note: Technical Support does not recommend that you mix multiple cursor and single cursor IFSTRT threads in a HLI application. An HLI job that mixes single cursor and multiple cursor IFSTRT threads processes multiple transactions separately against the database. This might produce undesired results because a job normally performs a logical unit of work.

See HLI: Threads for more information about multithreaded host language applications.

Multithreaded transaction with read-only IFSTRT threads

Single cursor IFSTRT threads with read-only privileges are not included in update units. This can cause undesired results in a multithreaded application. Such threads can access data from uncommitted nonbackoutable update units, which can lead to logical inconsistencies if updates are made based on this data.

If your HLI job uses multiple single cursor IFSTRT threads and performs any updating operations, start all single cursor IFSTRT threads with update privileges so that all of the threads participate in transaction processing.

To use a single cursor IFSTRT thread with updating privileges for retrieval operations, open files on the thread with read-only passwords. This prevents problems that result from accessing uncommitted updates when you use single cursor IFSTRT threads with read-only privileges.

Using IFCMMT in a multithreaded transaction

A call that ends a multithreaded transaction (IFCMMT, IFCMTR, IFBOUT) releases only the record and resource locks associated with the thread on which it is issued.

Whenever one of these calls is issued in a multithreaded application, IFCMMT must be issued on all the remaining threads that participate in the transaction. This ensures that all associated record and resource locks are released.

Note: If IFCMMT is not issued on all remaining threads in the multithreaded transaction, it is possible for the HLI application to generate locking conflicts with itself.

Committing transactions for lock pending updates files

Lock Pending Updates (LPU) locking mechanism

A TBO file, or any file with the LPU option enabled, exclusively locks all records updated within a transaction. When the transaction completes (when the update unit is committed) or is backed out, the exclusive lock on the entire set of updated records in the pending update pool is released.

The locking mechanism prevents updated records in one update unit from being used by other applications until Model 204 establishes whether the update is committed. An update can be backed out without affecting the logical validity of any other update.

Minimizing enqueuing conflicts

A TBO file, or any file with the LPU option enabled, generates an exclusive lock on updated records until the updates are committed. The LPU locking behavior may result in many additional enqueuing conflicts.

You can minimize the occurrence of enqueuing conflicts on those files by frequently issuing a call to IFCMMT. If you issue IFCMMT frequently, that is, after each updating call, each record update is immediately committed and each LPU lock is released.

For example, the following COBOL excerpt shows a frequent call to IFCMMT:

GET-NEXT RECORD. CALL "IFGET" USING... IF RETCODE IS EQUAL TO 2 GO TO NO-MORE-RECORDS. * * PERFORM RECORD PROCESSING ROUTINES * . . . CALL "IFPUT" USING... CALL "IFCMMT" USING... GO TO GET-NEXT-RECORD. * NO-MORE-RECORDS.

In this example, the LPU exclusive lock is obtained on each record as it is updated with IFPUT, then it is released with a call to IFCMMT.

Alternative for minimizing enqueuing conflicts

When you issue frequent calls to IFCMMT, as described in the preceding section, although record locking conflicts are reduced, a particular application might actually take longer to run because of the overhead associated with commit processing.

An alternative to immediate updating is to issue IFCMMT less frequently, so that LPU locks on updated records in the pending update pool are held until the entire found set is processed.

For example, the COBOL excerpt below shows an infrequent call to IFCMMT:

GET-NEXT RECORD. CALL "IFGET" USING... IF RETCODE IS EQUAL TO 2 GO TO NO-MORE-RECORDS. * * PERFORM RECORD PROCESSING ROUTINES * . . . CALL "IFPUT" USING... GO TO GET-NEXT-RECORD. * NO-MORE-RECORDS. CALL "IFCMMT" USING...

In this example, the LPU exclusive lock is obtained on each record as it is updated with IFPUT. The LPU locks are held on the updated records in the pending update pool until the entire found set is processed, then they are released with a call to IFCMMT.

Note that IFDREC, IFDSET, and IFFILE might result in logical data inconsistencies. See Logical inconsistencies with deleted records for more information.

See HLI: Locking behavior of HLI calls for more information about using HLI calls to end transactions.

Transaction back out facility

The Model 204 transaction back out facility provides a mechanism to undo the effects of incomplete updates to TBO files. A back out is a logical inversion of an update, which restores a file to its original state before the current update unit started.

The back out mechanism logically inverts the effects of an update to a TBO file by issuing compensating updates. The back out of a transaction can be initiated only on active transactions. Completed transactions cannot be backed out.

Invoking the back out mechanism ends the current transaction. A back out operation itself cannot be backed out. See Using the transaction back out facility for more information about backing out a transaction.

See HLI updating calls and update units for a description of update units. For complete information about the transaction back out facility, see Transaction back out.

Requirements for a back out

Model 204 requires that the following conditions be met for a back out to be successful:

  • All files in the transaction must be TBO files, so that transaction back out logging is active.

    A back out log stores compensating updates and a constraints log prevents reuse of freed file resources until the transaction ends. See Constraints log for a description of these logs.

  • The updated records must be locked exclusively so that no other update units are affected.

Note: TBO is the default file type. Lock pending updates is an option of the FOPT parameter that is enabled or disabled on a file by file basis.

See HLI calls that do not lock records for LPU files for information about backing out LPU files. See FOPT parameter and File integrity and recovery for more information about the FOPT parameter.

Using the transaction back out facility

Model 204 automatically backs out an incomplete update for a TBO file under any of the following conditions:

  • Model 204 cancels a HLI call or SOUL request
  • Model 204 detects a file problem, such as a table full condition
  • Model 204 restarts a user who has a transaction in progress (hard restart)

Upon canceling a HLI call in an active transaction, Model 204 returns a completion code (RETCODE) of 10. Model 204 automatically backs out the active transaction and returns control to the HLI application program.

You can use the IFGERR call to retrieve the text of the latest cancellation and counting error messages generated by Model 204. Note that IFGERR returns a null value if no such message exists.

Using IFBOUT to back out updates

A host language application can initiate a back out to undo an incomplete update to a TBO file. To activate the back out mechanism in your HLI program, issue the IFBOUT call. Note that IFBOUT is valid for use only with TBO files.

When the IFBOUT call is successfully executed, Model 204 performs the following actions:

  • Backs out the current transaction
  • Ends the transaction
  • Releases the LPU exclusive lock on updated records in the pending updates pool

Note: Upon executing IFBOUT, Model 204 does not release found sets. A call to IFBOUT does not change the current record.

Using transaction back out logs

Back out log

Model 204 builds a log of compensating updates, called a back out log, for each active transaction. When a transaction ends or is backed out, Model 204 discards the log for that particular transaction.

The back out log contains the necessary information to do a transaction back out on each active transaction in the Model 204 run. For example, if an IFDREC (DELETE RECORD) call is processed, all the information necessary to rebuild the entire deleted record must be available to Model 204 in case the transaction containing the IFDREC operation is backed out.

Model 204 stores the back out log in CCATEMP. At least one CCATEMP page is always used for transaction back out logging.

Constraints log

Model 204 builds a log of freed resources, called a constraints log, for each active transaction. When a transaction ends or is backed out, Model 204 discards the constraints log entries for that particular transaction.

Model 204 keeps the constraints log to guarantee the availability of resources freed by an active transaction. For example, the IFDREC, IFDVAL, IFDALL, and IFPUT calls free table space and other file resources. It is important that such freed file resources are not reused by other active transactions until the transaction that freed the resources ends.

Model 204 stores the constraint information in CCATEMP, in specially formatted pages.

Issue frequent calls to IFCMMT

To help keep both the back out log and the constraints log a manageable size, make transactions as brief as possible and issue IFCMMT as often as is feasible.

See HLI: Locking behavior of HLI calls for information about using IFCMMT.

Back out logging and CCATEMP space

Model 204 stores the back out and constraints logs in CCATEMP.

Transactions that are allowed to continue over many file updates increase the size of back out and constraint logs and can degrade the performance slightly for users who are doing back out logging.

Large back out and constraint logs also increase the possibility of CCATEMP filling. If transaction logging or constraint records fill up CCATEMP, Model 204 backs out the active transaction and cancels the user request, but does not restart the user.

Lessening CCATEMP space requirements

You can lessen the CCATEMP space required for logging and constraint records if you keep transactions short in your host language application, so that they contain a few file updates, by issuing frequent calls to IFCMMT.

Note: Theoretically, the only upper limit on the number of pages that can be used is the CCATEMP size. However, other uses of CCATEMP space limit the back out and constraint log pages that are available to the transaction back out facility. You might need a larger CCATEMP if you use TBO files.

For more information about the size calculation of CCATEMP, see Using the system scratch file (CCATEMP).

Transaction back out for LPU files

HLI calls that do not lock records for LPU files

Three updating calls function differently from normal LPU file processing. When processing against files that have the LPU option enabled, the following HLI calls do not lock records:

HLI call Equivalent SOUL command
IFDREC DELETE RECORD
IFDSET DELETE SET
IFFILE FILE

The records that are updated with any of these calls are not added to the pending update pool and are not locked for the remainder of the update unit. Deleted records are not locked because they no longer exist in the file, and thus cannot be accessed by other update units.

Note: These calls are the only exceptions to the exclusive locking behavior of LPU files on updated records.

Logical inconsistencies can occur when a transaction involving IFDREC, IFDSET, and IFFILE is backed out.

Logical inconsistencies with deleted records

When you are using TBO files with the LPU option enabled, logical inconsistencies can occur when a transaction involving IFDREC or IFDSET to delete records is backed out.

To prevent logical inconsistencies when deleting records, issue frequent calls to IFCMMT.

The following example shows why there is a need to keep transactions brief when deleting records, using IFCMMT to reduce the likelihood of logical inconsistencies.

In this example, events occur in the following order:

  1. User 1 begins processing, deletes (IFDSET) the set of records with NAME = SMITH, and continues to update other records.

    Note: The deleted set of records is not locked (IFDSET does not lock records), and User 1's transaction is not complete.

  2. User 2 begins processing and finds the set of records with either NAME = SMITH or NAME = SAUNDERS.
  3. There is no enqueuing conflict because the deleted records do not exist.
  4. User 2 adds a field to his found set (which includes only SAUNDERS records), prints a report, and ends his transaction.
  5. Subsequently, User 1's application comes to a point where it calls for the transaction in progress to be backed out. Because the deletion of the SMITH records is in the backed out transaction, the SMITH records reappear on the file.

    However, the original SMITH records do not have the field added by User 2. There is a logical inconsistency in the file.

If, in the first step, User 1 had issued an IFCMMT immediately after deleting the SMITH records, no inconsistency would have occurred unless the back out mechanism had been automatically activated during that short piece of the user request.

Logical inconsistencies using IFFILE

When you are using TBO files with the LPU option enabled, logical inconsistencies can occur when a transaction involving IFFILE to update records is backed out.

To prevent logical inconsistencies when updating records with IFFILE, use IFFNDX to lock records exclusively and hold the lock until after the update unit that contains the IFFILE call ends.

See also

Retrieved from "https://m204wiki.rocketsoftware.com/index.php?title=HLI:_Transactions&oldid=83940"