Multistep File Load utility

Multistep File Load phases

The first phase of the multistep File Load is performed in one job step; the second phase can involve many separate job steps. Each deferred update data set generated during the first phase of the File Load, and the FRV deferred update data set if one is present, adds two job steps to the File Load process. Each data set must be processed in the following separate job steps:

• One job step executes a sort program (such as IBM sort) to sort the deferred update data set into an order that allows efficient processing in the next job step.
• The next job step executes the BATCH204 load module to process the sorted deferred index information and to update Tables C and D.

Phase 1

The first phase of the multistep File Load is the execution of the file load program, which formats and loads the data into Table A and Table B of the Model 204 file. The first phase also generates the deferred updates for the Model 204 file index (Tables C and D), and it places these deferred updates in the deferred update data set or data sets.

Phase 2

The second phase of the multistep File Load consists of:

• Sorting each of the deferred update data sets generated in the first phase and the FRV deferred update data set, if one is generated.
• Applying the sorted deferred index updates to Tables C and D.

Number of job steps needed

The number of job steps in the entire File Load process depends on the attributes of the fields being updated and the number of deferred update data sets specified in the JCL. The File Load can have three, five, or seven steps. The "Deferring NON-DEFERRABLE fields" is a quick reference chart for determining the number of job steps needed (including the first-phase job step) for the File Load.

For a description of the three-, five-, or seven-step File Load process, refer to Three-step deferred update process, Five-step deferred update process, and Seven-step deferred update process. The steps are the same for both processes except that:

• The first-phase step of the File Load is a file load program, not a SOUL request or Host Language program.
• When using the File Load utility, you cannot omit the variable-length deferred update data set for ORDERED fields. If an ORDERED field is updated by the file load program, a variable-length deferred update data set must be available in the first-phase step so that Model 204 can write out the Ordered Index deferred updates to that data set.

Multistep File Load, Phase 1: Overview

This section describes in detail the first phase of the File Load process.

One-step file loads are discussed in detail beginning with One-Step File Load, Phase 1: Overview.

Multistep File Load statements

The JCL for the first phase of the multistep version normally includes the statements summarized in this section. These statements are discussed in greater detail, along with descriptions of other Model 204 configurations, in Defining the runtime environment (CCAIN).

EXEC statement

The EXEC statement performs the following functions:

• Invokes a program, usually named BATCH204
• Allows you to specify the size of the memory area allocated for File Load runs
• Allows you to specify a time limit for the job and a set of parameters to be passed to the job

The EXEC statement and its parameters are discussed in Specifying EXEC statement parameters.

Multistep File Load, Phase 1: Defining Model 204 system data sets

STEPLIB DD statement

STEPLIB points to the load module library into which the Model 204 BATCH204 load module has been linked. For z/VSE users, install the Model 204 BATCH204 module in a system or private core image library.

CCAAUDIT DD statement

CCAAUDIT allows you to log run statistics and other information in the audit trail. The CCAAUDIT DD statement normally specifies a SYSOUT data set. Tracking system activity (CCAJRNL, CCAAUDIT, CCAJLOG) discusses the use of the audit trail and the journals — CCAJRNL and CCAJLOG — in more detail. Specify CCAAUDIT rather than a journal data set, because File Load is run as a batch job, and you most likely want to print the run information directly.

CCATEMP, the scratch file DD statement

Model 204 requires a data set that has the file name CCATEMP to serve as a scratch file for the run. This data set can be permanent or temporary and it is initialized with each run. It must not be shared with other Model 204 jobs.

The scratch file is a special Model 204 file whose pages are used as work areas during the run. A small scratch file (20-30 pages) usually suffices for a File Load run.

CCASNAP and SYSUDUMP DD statements

Model 204 can trap program checks and file integrity problems at appropriate times and print SNAP dumps of selected portions of storage to the CCASNAP data set. These dumps are invaluable to Technical Support in locating and correcting errors. The SYSUDUMP data set also is required in the case of severe errors. Both normally are specified as SYSOUT-type data sets.

Under z/VSE, CCASNAP output is directed to the system logical unit SYSLST.

CCAPRINT and CCAIN DD statements

Include the FLOD, FILELOAD, or Z command and its associated file load program in the CCAIN data set, along with any other Model 204 commands needed to prepare the file for loading or to manipulate or check the data after loading.

Including Model 204 commands

The BATCH204 module accepts any Model 204 commands that are acceptable in a BATCH204 environment, in addition to the FLOD, FILELOAD, or Z commands.

Note: You can specify only one FLOD, FILELOAD, or Z command in any given File Load job step.

Do not issue FLOD or FILELOAD commands in any run where the NSERVS parameter is set in the CCAIN stream to less than NUSERS.

CCAIN input line length

The Model 204 system defaults expect the CCAIN data set to consist of 80-character lines that have data in columns 1 through 71. If input lines are longer than 71 characters, any nonblank character in column 72 indicates continuation of the line onto the next one. Model 204 EXEC statement parameters can be used to reset these defaults, as discussed in Specifying EXEC statement parameters.

For z/OS

CCAPRINT defines a simple sequential output data set (usually SYSOUT=A in z/OS JCL) that contains a summary of the user parameter lines and output generated by User 0.

A sequential input data set, CCAIN, often is defined under z/OS by the statement:

//CCAIN DD *

This data set defines the input for User 0, the batch user. This input defines the run-time parameters for the current run of the File Load utility and contains all the commands used to create and load the file.

For z/VSE

Under z/VSE, CCAPRINT output is directed to system logical unit SYSLST, and User 0 input statements usually follow the program execution statement. For example:

// EXEC BATCH204 // User 0 input . . .

CCASTAT DD statement

All runs that use the Model 204 security features require a CCASTAT definition statement pointing to a previously created password table data set that contains user and file security information. Refer to the discussion of this password table in File passwords and privileges. To require login, specify a SYSOPT of 16 and include a CCASTAT statement.

Multistep File Load, Phase 1: Defining Model 204 file data sets

The control statements must include a definition statement for each data set of each Model 204 file to be used in the run.

Defining TAPEI

The TAPEI data set contains the data that is to be loaded into the Model 204 file by the file load program. The file load program automatically defers updates to the index and writes the updates to the deferred update data set or data sets. It may be necessary to provide a fixed-length deferred update data set, a variable-length deferred update data set, or both. For information on deferred update data sets, see Deferred update feature.

Defining TAPE2

The default file name of the fixed-length deferred update data set is TAPE2. Only hashed index updates (KEY and NUMERIC RANGE fields) can be written to the fixed-length deferred update data set.

Defining TAPE3

The default file name of the variable-length deferred update data set is TAPE3. Either hashed index updates (KEY and NUMERIC RANGE fields) or Ordered Index updates (ORDERED fields) can be written to the variable-length deferred update data set. If ORDERED fields are updated by the file load program, there must be a variable-length deferred update data set supplied in the JCL of the first-phase step.

Overriding default file names

The default file names of TAPE2 and TAPE3 can be overridden simply by specifying the desired file name or names on the OPEN command. See Deferred update OPEN syntax for the correct syntax to use for declaring deferred update data sets.

Note: Never specify a deferred update data set as DUMMY.

Multistep File Load, Phase 1: Setting the SPCORE parameter

The SPCORE parameter specifies the minimum amount of storage to leave allocated at the end of Model 204 initialization. When viewed during the run, SPCORE indicates the amount of storage left unallocated just after initialization. Set SPCORE on User 0's parameter line to a value large enough to accommodate the output buffers of the deferred update data set(s), the input buffers for TAPEI, and the storage required to open the Model 204 file.

Calculating dynamic storage

The formula for calculating dynamic storage required for each deferred update data set is:

128 + deferred-update-data-set-blocksize * number-of-buffers

The default for the deferred update data set blocksize is 6000, and for the number of buffers is 3.

The storage required for input buffers for TAPEI is usually twice the blocksize of the TAPEI data set.

The dynamic storage required for opening the Model 204 file depends on the number of data sets and extents involved. 128 is sufficient for single data set, single extent files. The exact formula is:

32 + 8 * number-of-logical-extents + 16 * number-of-data-sets + 24 * number-of-physical-extents-in-all-data-sets

A new file has 6 logical extents.

Multistep File Load, Phase 1 examples

The following example of the first step of a simple multistep File Load illustrates the JCL statements included in a sample invocation of the File Load utility. PEOPLE is a Model 204 file created and cataloged at some earlier time. See Seven-Step File Load examples for a complete example of a multistep File Load.

The following program performs these steps:

1. Loads the PEOPLE file with the data on the statements that follow the TAPEI DD statement, and opens the PEOPLE file.
2. The FLOD command signals the start of a file load program that loads as many as 5000 records into PEOPLE. The File load program statements are described beginning with File Load statements: overview. The G statement causes a statement from TAPEI to be read for processing.
3. The next four statements load one record with a Social Security field taken from columns 1 through 9, a name field taken from columns 10 through 23, a department field taken from columns 24 through 25, and a sex field taken from columns 26 through 31.
4. The END statement signals the end of the file load program.

z/OS JCL

//JOB FILE LOAD STEP 1 //M204FLOD EXEC PGM=BATCH204,REGION=3072K //STEPLIB DD DSN=M204.LOADLIB,DISP=SHR //CCAPRINT DD SYSOUT=A //CCAAUDIT DD SYSOUT=A //CCASNAP DD SYSOUT=A //SYSUDUMP DD SYSOUT=A //CCATEMP DD UNIT=3380,SPACE=(TRK,3), // DISP=(NEW,DELETE) //PEOPLE DD DSN=M204.FILE.PEOPLE,DISP=SHR //TAPEI DD *,DCB=(LRECL=80,BLKSIZE=80) 012345678JOHNSON 03FEMALE 048343520KITTREDGE 11MALE . . . /* //TAPE2 DD DSN=M204.DEFER.FIXED.PEOPLE,DISP=(NEW,CATLG), // UNIT=TAPE,DCB=(RECFM=FB,LRECL=24,BLKSIZE=6000) //TAPE3 DD DSN=M204.DEFER.VAR.PEOPLE,DISP=(NEW,CATLG), // UNIT=TAPE,DCB=(RECFM=VB,LRECL=270,BLKSIZE=6000) //CCAIN DD * SPCORE=20000 OPEN PEOPLE FLOD 5000,5000,0 G SOC.SEC.NO=1,9,X'8000' NAME=10,14 DEPT=24,2 SEX=26,6 END EOJ /* // E.

z/VSE JCL

// JOB FILE LOAD STEP 1 // DLBL CCATEMP,'M204.CCATEMP',,DA // EXTENT SYSnnn,volser // DLBL PEOPLE,'M204.FILE.PEOPLE',,DA // EXTENT SYSnnn,volser // DLBL TAPEI,'PEOPLE.INPUT.DATA' // EXTENT SYSnnn,volser // UPSI 10000000 // TLBL SYS015,'M204.DEFERF.PEOPLE' // ASSGN SYS015,TAPE // TLBL SYS016,'M204.DEFERV.PEOPLE' // ASSGN SYS016,TAPE // EXEC BATCH204,SIZE=AUTO MAXBUF=20 DEFINE DATASET TAPEI WITH SCOPE=SYSTEM - LRECL=input-rec-length BLKSIZE=input-blk-length - RECFM=input-record-format DEFINE DATASET TAPE2 WITH SCOPE=SYSTEM - LRECL=24 BLKSIZE=6000 RECFM=FB DDNAME=SYS015 DEFINE DATASET TAPE3 WITH SCOPE=SYSTEM - LRECL=270 BLKSIZE=6000 RECFM=VB DDNAME=SYS016 OPEN PEOPLE FLOD 5000,5000,0 G SOC.SEC.NO=1,9,X'8000' NAME=10,14 DEPT=24,2 SEX=26,6 END EOJ /* /&

z/VM EXEC

To load a file using the File Load process, use the FASTLOAD EXEC. For example:

FASTLOAD PEOPLE (7STEP

This command invokes the PEOPLE EXEC, which looks similar to the following multistep example.

The CCAIN file has the same contents as in the other operating environments. For a complete example of a multistep File Load, see Seven-Step File Load examples.

Multistep File Load, z/VM EXECs

&CONTROL OFF &ERROR &EXIT &RETCODE . . . access disks . . . -FLOAD1 FILEDEF PEOPLE E DSN M204 PEOPLE FILEDEF TAPE2 E DSN PEOPLE DEFFIX (BLOCK 24 LRECL 24 RECFM FB FILEDEF TAPE3 E DSN PEOPLE DEFVAR (BLOCK 6000 LRECL 270 RECFM VB FILEDEF TAPEI DISK PEOPLE FLODINPT D (BLOCK 990 LRECL 986 RECFM VB FILEDEF CCAIN DISK PEOPLE CCAIN * &GOTO -COMMON . . . steps 2-7 . . . -COMMON FILEDEF CCAPRINT DISK FLOD&1 CCAPRINT A FILEDEF CCAAUDIT DISK FLOD&1 CCAAUDIT A FILEDEF CCATEMP E DSN M204 CCATEMP FILEDEF CCASNAP PRINTER &STACK SYSOPT 128 &EXIT