One-Step File Load utility

From m204wiki
Revision as of 23:09, 13 December 2013 by ELowell (talk | contribs)
Jump to navigation Jump to search

Overview

When a sequential data set contains the raw data for records to be stored in a Model 204 file, the File Load utility generally provides the most efficient means of loading that data into Model 204.

In a one-step File Load, the two phases are run as one job step in which the second phase is invoked automatically.

Operating system dependencies

The one-step File Load is not available under z/VSE.

Under z/VM, the one-step File Load is run by a single command (FASTLOAD) that drives a user-written EXEC through the desired number of steps.

Using one-step File Load

For small amounts of data, the one-step File Load can be much more efficient than the multistep procedure. The major reason for this efficiency is that each deferred update record has four fewer I/O operations performed on it in the one-step File Load. The one-step File Load bypasses:

  • Output from the file load program
  • Input to the sort utility
  • Output from the sort utility
  • Input to the Z commands

Another reason is that by using the one-step File Load, the sort key is shorter if the file load program performs no deletions. (This is true only if no ORDERED fields are updated in the one-step File Load.) In addition, the one-step method is operationally easier, because only one job step is involved. However, a one-step File Load can be less efficient for large amounts of data because there is less memory available for sorting.

One-Step File Load, Phase 1: Overview

The control statements for the one-step version of the File Load utility include the standard Model 204 DD statements and also the DD statements required by the sort program. Examples of one-step File Loads for z/OS and z/VM users follow.

Note

The one-step File Load utility is not available under z/VSE.

DD statements

The following DD statements have the meanings described earlier for the multistep procedure:

EXEC STEPLIB CCAAUDIT CCATEMP CCASNAP SYSUDUMP CCAPRINT CCASTAT TAPEI CCAIN File DD statements

The following sections summarize the differences between one-step and multistep control statements.

Providing space for sorting

Setting the REGION parameter of the EXEC statement

When running the one-step version of the File Load utility, the REGION parameter of the EXEC statement must be increased by at least 100K for each sort/Z sequence that is needed (see the description of CCA1WKxx DD statements on CCA1WKxx DD statement). To provide improved performance, increase the region more than the minimum and also increase the memory used by each sort (especially the first).

Setting SPCORE or MAXBUF

Set either the MAXBUF or the SPCORE parameter on User 0's parameter line to reserve the additional memory specified for the sorts. Otherwise, Model 204 uses the additional memory for disk buffers. If sufficient memory is not provided for the sorts to run, the step terminates with an error.

For more information about the sort space, refer to the discussion of the s1 and s2 parameters in FILELOAD command.

Sort program library

Depending upon the installation, you might need to include a SORTLIB DD statement to identify the library in which the sort program is stored. You might also need to specify sort work (SORTWKxx) data sets.

Valid sort programs with the File Load utility

Any sort program that is compatible with the IBM SORT SM-023 can be invoked by the File Load utility. (For more information about interfaces for sort utilities, see Hash key files.) The sort program invoked is the program that has the entry point, SORT, contained in one of the following libraries:

JOBLIB Job library
STEPLIB Step library
LINKLIB System link library

 

One-Step File Load, Phase 1: Defining CCA1WK and CCA2WK data sets (z/OS only)

When running the one-step version of the File Load utility under z/OS, deferred update index records are passed directly to the sort program as they are produced by the file load program. When the file load program is complete, the sort utility orders the deferred index records.

Sorting process

The sort works as follows:

  1. Index entries are sorted using CCA1WKxx, and the first Z command is automatically invoked.
  2. First Z command applies the deferred update index record entries to the file.
  3. If the first Z command encounters an index entry for an FRV field, the sort utility starts a sort of the FRV index records using CCA2WKxx.
  4. Second Z command, which adds the deferred FRV entries to the index, is invoked automatically after the first Z completes.

CCAnWKxxx data set names

Because two sorts can be invoked, you might need two sets of CCAnWKxx and SYSOUT DD statements: CCA1WKxx and CCA2WKxx. Specify the names of the CCAnWKxx sequentially, beginning with 01. For example:

//CCA1WK01 //CCA1WK02 //CCA1WK03 //CCA2WK01 //CCA2WK02 //CCA2WK03

If there are no FRV records, the second sort is not used, and neither CCA2WKxx nor SYSOUT2 statements are required.

CCA1WKxx DD statement

You need to define at least three CCA1WKxx DD statements (CCA1WK01-CCA1WK03). The actual number of CCA1WKxx statements you need depends on your sort package. The parameters on these statements are required by the particular sort utility being used to define work data sets.

The CCA1WKxx DD statements provide temporary storage for the first sort while it is reordering the deferred index records. These DD statements are required. If they are not included, the execution phase of the file load program is aborted, and an error message issued.

CCA2WKxx DD statement

CCCA2WKxx DD statements are required only if you are generating FRV records with the File Load statements. If you are not generating FRV records, then the second sort is not invoked and you can omit both the SYSOUT2 and CCA2WKxx DD statements.

If there are more deferred FRV index records than the second sort can handle in a memory sort, or if the sort program you are using does not support a memory sort, three CCA2WKxx DD statements are required.

If a second sort is needed, but a CCA2WK01 DD statement has not been provided, Model 204 issues the warning message:

*** M204.0759: SORT WORK DD CARD, CCA2WK01, MISSING

Under z/VM, there are no CCA1WK or CCA2WK statements. Sort space is allocated dynamically by the sort subtasks.

SYSOUT1 and SYSOUT2 DD statement

The SYSOUT DD statements specify the data sets to which messages from the sort program are directed. Because two sorts can be invoked, include two SYSOUT DD statements, one for the messages from each sort. SYSOUT1 corresponds to the CCA1WKxx DD statements, and SYSOUT2 corresponds to the CCA2WKxx DD statements.

One-Step File Load, Phase 1: Setting SPCORE and MAXBUF parameters

Either the SPCORE, the MAXBUF parameter, or both must be set on User 0's parameter line, as discussed for the EXEC statement earlier. See the discussion of the SPCORE parameter in Multistep File Load, Phase 1: Setting the SPCORE parameter.

To provide sufficient memory for the sorts, do one of the following:

  • Set SPCORE to the amount of space to be reserved for the sorts.
  • Limit the number of disk buffers that Model 204 obtains with MAXBUF.

Setting SPCORE

If SPCORE is set, it must be set on User 0's parameter line to a value large enough to accommodate all of the following:

  • TAPEI buffer
  • Two copies of the sort
  • Area that is used for file load program, sort, and Z communication
  • Two work areas, one for each invocation of the sort utility

If no deferred FRV records are generated by the file load program, only one copy of the sort is invoked. In this case, SPCORE can be reduced by the size occupied by one copy of the sort plus the size of the default work space for the sort. This work space is the value specified for the s2 argument in the FILELOAD command. For more information on the s2 parameter, refer to FILELOAD command.

In general, the best performance is obtained when the amount of memory that the first sort is allowed to use is expanded. However, if any FRV fields are loaded, memory must be left for the second sort. This sort is run concurrently with the first sort, so there must be sufficient memory for both sorts to be active simultaneously.

Setting MAXBUF

As an alternative to setting SPCORE, the MAXBUF parameter can be set to reserve memory for the sorts by limiting the number of buffers that are allocated. Never set MAXBUF to less than 4. A File Load run loading a typical file with no ORDERED fields should not require more than ASIZE + 2 buffers. Files with unordered, sorted, or hash key organizations, or with ORDERED fields might require additional buffers. In general, files with ORDERED fields run more efficiently with a large number of buffers: up to ASIZE+DSIZE+2.

One-Step File Load, Phase 1 examples

The following JCL illustrates a sample invocation of the one-step File Load.

z/OS JCL

//M204FLOD EXEC PGM=BATCH204,REGION=5120K, // PARM='SYSOPT=128',TIME=10 //STEPLIB DD DSN=DVM204.V220.LOADLIB,DISP=SHR //CCATEMP DD UNIT=SYSDA,SPACE=(CYL,1) //CENSUS DD DSN=M204.CENSUS,DISP=OLD //CCASTAT DD DSN=M204.CCASTAT,DISP=SHR //CCAAUDIT DD SYSOUT=C //CCAPRINT DD SYSOUT=C //SYSOUT1 DD SYSOUT=C //SYSOUT2 DD SYSOUT=C //CCASNAP DD SYSOUT=C //SYSUDUMP DD SYSOUT=C //TAPEI DD DSN=M204.CENSUS.INPUT,DISP=SHR //CCA1WK01 DD UNIT=SYSDA,SPACE=(CYL,5,,CONTIG) //CCA1WK02 DD UNIT=(SYSDA,SEP=CCA1WK01), // SPACE=(CYL,5,,CONTIG) //CCA1WK03 DD UNIT=(SYSDA,SEP=(CCA1WK01,CCA1WK02)), // SPACE=(CYL,5,,CONTIG) //CCA2WK01 DD UNIT=SYSDA, // SPACE=(CYL,5,,CONTIG) //CCA2WK02 DD UNIT=(SYSDA,SEP=CCA2WK01), // SPACE=(CYL,5,,CONTIG) //CCA2WK03 DD UNIT=(SYSDA,SEP=(CCA2WK01,CCA2WK02)), // SPACE=(CYL,5,,CONTIG) //CCAIN DD * MAXBUF=100,SPCORE=1126400 LOGON SUPERKLUGE PIGFLOUR OPEN CENSUS *PASSWORD* FILELOAD -1,-1,0,1048576,,,,28 . . . /* //

z/VM EXECs

To load a file using the one-step File Load process, use the FASTLOAD EXEC, described fully in the Rocket Model 204 Installation Guide for IBM z/VM. For example:

FASTLOAD CENSUS (1STEP

This command invokes the CENSUS EXEC, which looks similar to the following example.

&CONTROL OFF &ERROR &EXIT &RETCODE access disks . . . -FLOAD1 FILEDEF TAPEI DISK CENSUS FLODINPT D ( BLOCK 990 LRECL 986 RECFM VB FILEDEF CCAIN DISK CENSUS CCAIN1 * FILEDEF CCAPRINT DISK FLOD&1 CCAPRINT A FILEDEF CCAAUDIT DISK FLOD&1 CCAAUDIT A FILEDEF CCATEMP E DSN M204 CCATEMP FILEDEF CCASTAT E DSN M204 CCASTAT FILEDEF CENSUS E DSN M204 CENSUS FILEDEF CCASNAP PRINTER &STACK SYSOPT 128 &EXIT

The CCAIN file is the same as in the preceding z/OS example.

 

Retrieved from "https://m204wiki.rocketsoftware.com/index.php?title=One-Step_File_Load_utility&oldid=66230"