Multistep File Load utility: Difference between revisions

From m204wiki
Jump to navigation Jump to search
(Automatically generated page update)
 
m (link repair)
 
(14 intermediate revisions by 5 users not shown)
Line 1: Line 1:
==Multistep File Load phases==
==Multistep File Load phases==
<p>The first phase of the multistep File Load is performed in one job step; the second phase can involve many separate job steps. Each deferred update data set generated during the first phase of the File Load, and the FRV deferred update data set if one is present, adds two job steps to the File Load process. Each data set must be processed in the following separate job steps:</p>
<p>
The first phase of the multistep File Load is performed in one job step; the second phase can involve many separate job steps. Each deferred update data set generated during the first phase of the File Load, and the FRV deferred update data set if one is present, adds two job steps to the File Load process. Each data set must be processed in the following separate job steps:</p>
<ul>
<ul>
<li>One job step executes a sort program (such as IBM sort) to sort the deferred update data set into an order that allows efficient processing in the next job step.   </li>
<li>One job step executes a sort program (such as IBM sort) to sort the deferred update data set into an order that allows efficient processing in the next job step. </li>
<li>Next job step executes the BATCH204 load module to process the sorted deferred index information and to update Tables C and D.     </li>
 
<li>The next job step executes the BATCH204 load module to process the sorted deferred index information and to update Tables C and D. </li>
</ul>
</ul>
====Phase 1====
====Phase 1====
<p>The first phase of the multistep File Load is the execution of the file load program, which formats and loads the data into Table A and Table B of the <var class="product">Model&nbsp;204</var> file. The first phase also generates the deferred updates for the <var class="product">Model&nbsp;204</var> file index (Tables C and D) and places these deferred updates in the deferred update data set or data sets.</p>
<p>
The first phase of the multistep File Load is the execution of the file load program, which formats and loads the data into Table A and Table B of the <var class="product">Model&nbsp;204</var> file. The first phase also generates the deferred updates for the <var class="product">Model&nbsp;204</var> file index (Tables C and D), and it places these deferred updates in the deferred update data set or data sets.</p>
 
====Phase 2====
====Phase 2====
<p>The second phase of the multistep File Load consists of sorting each of the deferred update data sets generated in the first phase and the FRV deferred update data set, if one is generated, and then applying the sorted deferred index updates to Tables C and D.   </p>
<p>
The second phase of the multistep File Load consists of: </p>
<ul>
<li>Sorting each of the deferred update data sets generated in the first phase and the FRV deferred update data set, if one is generated. </li>
<li>Applying the sorted deferred index updates to Tables C and D. </li>
</ul>
 
====Number of job steps needed====
====Number of job steps needed====
<p>The number of job steps in the entire File Load process depends on the attributes of the fields being updated and the number of deferred update data sets specified in the JCL. The File Load can have three, five, or seven steps. The File Load Utility chapter provides a quick reference chart for determining the number of job steps needed (including the first-phase job step) for the File Load.</p>
<p>
<p>For a description of the three-, five-, or seven-step File Load process, refer to the three-, five-, or seven-step deferred update process descriptions in the File Load chapters. The steps are the same for both processes except that:</p>
The number of job steps in the entire File Load process depends on the attributes of the fields being updated and the number of deferred update data sets specified in the JCL. The File Load can have three, five, or seven steps. The [[Deferred update feature#Deferred updates for NON-DEFERRABLE fields|"Deferring NON-DEFERRABLE fields"]] is a quick reference chart for determining the number of job steps needed (including the first-phase job step) for the File Load.</p>
<p>
For a description of the three-, five-, or seven-step File Load process, refer to [[Deferred update feature#Three-step deferred update process|Three-step deferred update process]], [[Deferred update feature#Five-step deferred update process|Five-step deferred update process]], and [[Deferred update feature#Seven-step deferred update process|Seven-step deferred update process]]. The steps are the same for both processes except that:</p>
<ul>
<ul>
<li>First-phase step of the File Load is a file load program, not a User Language request or Host Language program.</li>
<li>The first-phase step of the File Load is a file load program, not a [[SOUL]] request or Host Language program.</li>
<li>When using the File Load utility, you cannot omit the variable-length deferred update data set for ORDERED fields. If an ORDERED field is updated by the file load program, there must be a variable-length deferred update data set available in the first-phase step so that <var class="product">Model&nbsp;204</var> can write out the Ordered Index deferred updates to that data set.  </li>
 
<li>When using the File Load utility, you cannot omit the variable-length deferred update data set for <var>ORDERED</var> fields. If an <var>ORDERED</var> field is updated by the file load program, a variable-length deferred update data set must be available in the first-phase step so that <var class="product">Model&nbsp;204</var> can write out the Ordered Index deferred updates to that data set.  </li>
</ul>
</ul>
==Multistep File Load, Phase 1: Overview==
==Multistep File Load, Phase 1: Overview==
<p>This section describes in detail the first phase of the File Load process.</p>
<p>
<p>One-step file loads are discussed in detail beginning with [[ One-Step File Load Utility#One-Step File Load, Phase 1: Overview|One-Step File Load, Phase 1: Overview]].</p>
This section describes in detail the first phase of the File Load process.</p>
<p>
One-step file loads are discussed in detail beginning with [[One-Step File Load utility#One-Step File Load, Phase 1: Overview|One-Step File Load, Phase 1: Overview]].</p>
 
===Multistep File Load statements===
===Multistep File Load statements===
<p>The JCL for the first phase of the multistep version normally includes the statements summarized in this section. These statements are discussed in greater detail, along with descriptions of other <var class="product">Model&nbsp;204</var> configurations, in the Rocket <var class="product">Model&nbsp;204</var> System Manager's Guide. </p>
<p>
The JCL for the first phase of the multistep version normally includes the statements summarized in this section. These statements are discussed in greater detail, along with descriptions of other <var class="product">Model&nbsp;204</var> configurations, in [[Defining the runtime environment (CCAIN)]]. </p>
 
===EXEC statement===
===EXEC statement===
<p>The EXEC statement performs the following functions:</p>
<p>
The EXEC statement performs the following functions:</p>
<ul>
<ul>
<li>Invokes a program, usually named BATCH204</li>
<li>Invokes a program, usually named BATCH204</li>
Line 28: Line 49:
<li>Allows you to specify a time limit for the job and a set of parameters to be passed to the job</li>
<li>Allows you to specify a time limit for the job and a set of parameters to be passed to the job</li>
</ul>
</ul>
<p>The EXEC statement and its parameters are discussed in the Rocket <var class="product">Model&nbsp;204</var> System Manager's Guide. </p>
<p>
The EXEC statement and its parameters are discussed in [[Defining the runtime environment (CCAIN)#Specifying EXEC statement parameters|Specifying EXEC statement parameters]]. </p>
 
==Multistep File Load, Phase 1: Defining Model 204 system data sets==
==Multistep File Load, Phase 1: Defining Model 204 system data sets==
===STEPLIB DD statement===
===STEPLIB DD statement===
<p>STEPLIB points to the load module library into which the <var class="product">Model&nbsp;204</var> BATCH204 load module has been linked. For z/VSE users, install the <var class="product">Model&nbsp;204</var> BATCH204 module in a system or private core image library.     </p>
<p>
STEPLIB points to the load module library into which the <var class="product">Model&nbsp;204</var> BATCH204 load module has been linked. For z/VSE users, install the <var class="product">Model&nbsp;204</var> <code>BATCH204</code> module in a system or private core image library. </p>
 
===CCAAUDIT DD statement===
===CCAAUDIT DD statement===
<p>CCAAUDIT allows you to log run statistics and other information in the audit trail. The CCAAUDIT DD statement normally specifies a SYSOUT data set. The Rocket <var class="product">Model&nbsp;204</var> System Manager's Guide discusses the use of the audit trail and the journals-CCAJRNL and CCAJLOG-in more detail. Specify CCAAUDIT rather than a journal data set, because File Load is run as a batch job, and you most likely want to print the run information directly.  </p>
<p>
CCAAUDIT allows you to log run statistics and other information in the audit trail. The CCAAUDIT DD statement normally specifies a SYSOUT data set. [[Tracking system activity (CCAJRNL, CCAAUDIT, CCAJLOG)]] discusses the use of the audit trail and the journals &mdash; CCAJRNL and CCAJLOG &mdash; in more detail. Specify CCAAUDIT rather than a journal data set, because File Load is run as a batch job, and you most likely want to print the run information directly.  </p>
 
===CCATEMP, the scratch file DD statement===
===CCATEMP, the scratch file DD statement===
<p><var class="product">Model&nbsp;204</var> requires a data set that has the file name CCATEMP to serve as a scratch file for the run. This data set can be permanent or temporary and it is initialized with each run. It must not be shared with other <var class="product">Model&nbsp;204</var> jobs. </p>
<p>
<p>The scratch file is a special <var class="product">Model&nbsp;204</var> file whose pages are used as work areas during the run. A small scratch file (20-30 pages) usually suffices for a File Load run. </p>
<var class="product">Model&nbsp;204</var> requires a data set that has the file name [[Using the system scratch file (CCATEMP)|CCATEMP]] to serve as a scratch file for the run. This data set can be permanent or temporary and it is initialized with each run. It must not be shared with other <var class="product">Model&nbsp;204</var> jobs. </p>
<p>
The scratch file is a special <var class="product">Model&nbsp;204</var> file whose pages are used as work areas during the run. A small scratch file (20-30 pages) usually suffices for a File Load run. </p>
 
===CCASNAP and SYSUDUMP DD statements===
===CCASNAP and SYSUDUMP DD statements===
<p><var class="product">Model&nbsp;204</var> can trap program checks and file integrity problems at appropriate times and print SNAP dumps of selected portions of storage to the CCASNAP data set. These dumps are invaluable to Technical Support in locating and correcting errors. The SYSUDUMP data set also is required in the case of severe errors. Both normally are specified as SYSOUT-type data sets.       </p>
<p>
<p>Under z/VSE, CCASNAP output is directed to the system logical unit SYSLST. </p>
<var class="product">Model&nbsp;204</var> can trap program checks and file integrity problems at appropriate times and print SNAP dumps of selected portions of storage to the [[Storing diagnostic information (CCASNAP)|CCASNAP]] data set. These dumps are invaluable to Technical Support in locating and correcting errors. The SYSUDUMP data set also is required in the case of severe errors. Both normally are specified as SYSOUT-type data sets. </p>
<p>
Under z/VSE, CCASNAP output is directed to the system logical unit SYSLST. </p>
 
===CCAPRINT and CCAIN DD statements===
===CCAPRINT and CCAIN DD statements===
<p>Include the FLOD, FILELOAD, or Z command and its associated file load program in the CCAIN data set, along with any other <var class="product">Model&nbsp;204</var> commands needed to prepare the file for loading or to manipulate or check the data after loading. </p>
<p>
Include the <var>FLOD</var>, <var>FILELOAD</var>, or <var>Z</var> command and its associated file load program in the CCAIN data set, along with any other <var class="product">Model&nbsp;204</var> commands needed to prepare the file for loading or to manipulate or check the data after loading. </p>
 
====Including Model 204 commands====
====Including Model 204 commands====
<p>The BATCH204 module accepts any <var class="product">Model&nbsp;204</var> commands that are acceptable in a BATCH204 environment, in addition to the FLOD, FILELOAD, or Z commands.</p>
<p>
<b>Note</b>
The BATCH204 module accepts any <var class="product">Model&nbsp;204</var> commands that are acceptable in a BATCH204 environment, in addition to the <var>[[File Load utility: FLOD and FILELOAD_commands#FLOD command|FLOD]]</var>, <var>[[File Load utility: FLOD and FILELOAD_commands#FILELOAD command|FILELOAD]]</var>, or <var>[[Deferred update feature#Z command|Z]]</var> commands.</p>
<p>You can specify only one FLOD, FILELOAD, or Z command in any given File Load job step. </p>
<p class="note"><b>Note:</b>
<p>Do not issue FLOD or FILELOAD commands in any run where the NSERVS parameter is set in the CCAIN stream to less than NUSERS.</p>
You can specify only one <var>FLOD</var>, <var>FILELOAD</var>, or <var>Z</var> command in any given File Load job step. </p>
<p>
Do not issue <var>FLOD</var> or <var>FILELOAD</var> commands in any run where the <var>[[NSERVS parameter|NSERVS]]</var> parameter is set in the CCAIN stream to less than <var>[[NUSERS parameter|NUSERS]]</var>.</p>
 
====CCAIN input line length====
====CCAIN input line length====
<p>The <var class="product">Model&nbsp;204</var> system defaults expect the CCAIN data set to consist of 80-character lines that have data in columns 1 through 71. If input lines are longer than 71 characters, any nonblank character in column 72 indicates continuation of the line onto the next one. <var class="product">Model&nbsp;204</var> EXEC statement parameters can be used to reset these defaults, as discussed in the Rocket <var class="product">Model&nbsp;204</var> System Manager's Guide.</p>
<p>
The <var class="product">Model&nbsp;204</var> system defaults expect the CCAIN data set to consist of 80-character lines that have data in columns 1 through 71. If input lines are longer than 71 characters, any nonblank character in column 72 indicates continuation of the line onto the next one. <var class="product">Model&nbsp;204</var> EXEC statement parameters can be used to reset these defaults, as discussed in [[Defining the runtime environment (CCAIN)#Specifying EXEC statement parameters|Specifying EXEC statement parameters]].</p>
 
====For z/OS====
====For z/OS====
<p>CCAPRINT defines a simple sequential output data set (usually SYSOUT=A in z/OS JCL) that contains a summary of the user parameter lines and output generated by User 0. </p>
<p>
<p>A sequential input data set, CCAIN, often is defined under z/OS by the statement:</p>
CCAPRINT defines a simple sequential output data set (usually <code>SYSOUT=A</code> in z/OS JCL) that contains a summary of the user parameter lines and output generated by User 0. </p>
<p>
A sequential input data set, CCAIN, often is defined under z/OS by the statement:</p>
<p class="code">//CCAIN DD *   
<p class="code">//CCAIN DD *   
</p>
</p>
<p>This data set defines the input for User 0, the batch user. This input defines the run-time parameters for the current run of the File Load utility and contains all the commands used to create and load the file.</p>
<p>
This data set defines the input for User 0, the batch user. This input defines the run-time parameters for the current run of the File Load utility and contains all the commands used to create and load the file.</p>
 
====For z/VSE====
====For z/VSE====
<p>Under z/VSE, CCAPRINT output is directed to system logical unit SYSLST, and User 0 input statements usually follow the program execution statement. For example:     </p>
<p>
Under z/VSE, CCAPRINT output is directed to system logical unit SYSLST, and User 0 input statements usually follow the program execution statement. For example: </p>
<p class="code">// EXEC BATCH204
<p class="code">// EXEC BATCH204
// User 0 input
// User 0 input
Line 63: Line 109:
.
.
</p>
</p>
===CCASTAT DD statement===
===CCASTAT DD statement===
<p>All runs that use the <var class="product">Model&nbsp;204</var> security features require a CCASTAT definition statement pointing to a previously created password table data set that contains user and file security information. Refer to the discussion of this password table in "File passwords and privileges". To require login, specify a SYSOPT of 16 and include a CCASTAT statement.     </p>
<p>
All runs that use the <var class="product">Model&nbsp;204</var> security features require a CCASTAT definition statement pointing to a previously created password table data set that contains user and file security information. Refer to the discussion of this password table in [[Model 204 security features#File passwords and privileges|File passwords and privileges]]. To require login, specify a <var>[[SYSOPT parameter|SYSOPT]]</var> of 16 and include a CCASTAT statement. </p>
 
==Multistep File Load, Phase 1: Defining Model 204 file data sets==
==Multistep File Load, Phase 1: Defining Model 204 file data sets==
<p>The control statements must include a definition statement for each data set of each <var class="product">Model&nbsp;204</var> file to be used in the run. </p>
<p>
The control statements must include a definition statement for each data set of each <var class="product">Model&nbsp;204</var> file to be used in the run. </p>
 
===Defining TAPEI===
===Defining TAPEI===
<p>The TAPEI data set contains the data that is to be loaded into the <var class="product">Model&nbsp;204</var> file by the file load program. The file load program automatically defers updates to the index and writes the updates to the deferred update data set or data sets. It may be necessary to provide a fixed-length deferred update data set, a variable-length deferred update data set, or both. For information on deferred update data sets, see the deferred updates chapter.</p>
<p>
The <code>TAPEI</code> data set contains the data that is to be loaded into the <var class="product">Model&nbsp;204</var> file by the file load program. The file load program automatically defers updates to the index and writes the updates to the deferred update data set or data sets. It may be necessary to provide a fixed-length deferred update data set, a variable-length deferred update data set, or both. For information on deferred update data sets, see [[Deferred update feature]].</p>
 
===Defining TAPE2===
===Defining TAPE2===
<p>The default file name of the fixed-length deferred update data set is TAPE2. Only hashed index updates (KEY and NUMERIC RANGE fields) can be written to the fixed-length deferred update data set.</p>
<p>
The default file name of the fixed-length deferred update data set is <code>TAPE2</code>. Only hashed index updates (<var>KEY</var> and <var>NUMERIC RANGE</var> fields) can be written to the fixed-length deferred update data set.</p>
 
===Defining TAPE3===
===Defining TAPE3===
<p>The default file name of the variable-length deferred update data set is TAPE3. Either hashed index updates (KEY and NUMERIC RANGE fields) or Ordered Index updates (ORDERED fields) can be written to the variable-length deferred update data set. If ORDERED fields are updated by the file load program, there must be a variable-length deferred update data set supplied in the JCL of the first-phase step.</p>
<p>
The default file name of the variable-length deferred update data set is <code>TAPE3</code>. Either hashed index updates (<var>KEY</var> and <var>NUMERIC RANGE</var> fields) or Ordered Index updates (<var>ORDERED</var> fields) can be written to the variable-length deferred update data set. If <var>ORDERED</var> fields are updated by the file load program, there must be a variable-length deferred update data set supplied in the JCL of the first-phase step.</p>
 
===Overriding default file names===
===Overriding default file names===
<p>The default file names of TAPE2 and TAPE3 can be overridden simply by specifying the desired file name or names on the OPEN command. See "Opening deferred update files" for the correct syntax to use for declaring deferred update data sets.</p>
<p>
<b>Note</b>
The default file names of TAPE2 and TAPE3 can be overridden simply by specifying the desired file name or names on the <var>OPEN</var> command. See [[Deferred update feature#Deferred update OPEN syntax|Deferred update OPEN syntax]] for the correct syntax to use for declaring deferred update data sets.</p>
<p>Never specify a deferred update data set as DUMMY.       </p>
<p class="note"><b>Note:</b>
Never specify a deferred update data set as <code>DUMMY</code>. </p>
 
==Multistep File Load, Phase 1: Setting the SPCORE parameter==
==Multistep File Load, Phase 1: Setting the SPCORE parameter==
<p>The SPCORE parameter specifies the minimum amount of storage to leave allocated at the end of <var class="product">Model&nbsp;204</var> initialization. When viewed during the run, SPCORE indicates the amount of storage left unallocated just after initialization. Set SPCORE on User 0's parameter line to a value large enough to accommodate the output buffers of the deferred update data set(s), the input buffers for TAPEI, and the storage required to open the <var class="product">Model&nbsp;204</var> file.  </p>
<p>
The <var>[[SPCORE parameter|SPCORE]]</var> parameter specifies the minimum amount of storage to leave allocated at the end of <var class="product">Model&nbsp;204</var> initialization. When viewed during the run, <var>SPCORE</var> indicates the amount of storage left unallocated just after initialization. Set <var>SPCORE</var> on User 0's parameter line to a value large enough to accommodate the output buffers of the deferred update data set(s), the input buffers for TAPEI, and the storage required to open the <var class="product">Model&nbsp;204</var> file.  </p>
 
===Calculating dynamic storage===
===Calculating dynamic storage===
<p>The formula for calculating dynamic storage required for each deferred update data set is:</p>
<p>
<p class="code">128 + deferred update data set blocksize * number of  
The formula for calculating dynamic storage required for each deferred update data set is:</p>
buffers
<p class="code">128 + <i>deferred-update-data-set-blocksize</i> * <i>number-of-buffers</i>
</p>
</p>
<p>The default for the deferred update data set blocksize is 6000, and for the number of buffers is 3.</p>
<p>
<p>The storage required for input buffers for TAPEI is usually twice the blocksize of the TAPEI data set.</p>
The default for the deferred update data set blocksize is 6000, and for the number of buffers is 3.</p>
<p>The dynamic storage required for opening the <var class="product">Model&nbsp;204</var> file depends on the number of data sets and extents involved. 128 is sufficient for single data set, single extent files. The exact formula is: </p>
<p>
<p class="code">32 + 8 * number of logical extents + 16 * number of data sets + 24 * number of physical extents in all data sets
The storage required for input buffers for TAPEI is usually twice the blocksize of the TAPEI data set.</p>
<p>
The dynamic storage required for opening the <var class="product">Model&nbsp;204</var> file depends on the number of data sets and extents involved. 128 is sufficient for single data set, single extent files. The exact formula is: </p>
<p class="code">32 + 8 * <i>number-of-logical-extents</i> + 16 * <i>number-of-data-sets</i> + 24 * <i>number-of-physical-extents-in-all-data-</i>sets
</p>
</p>
<p>A new file has 6 logical extents.</p>
<p>
A new file has 6 logical extents.</p>
 
==Multistep File Load, Phase 1 examples==
==Multistep File Load, Phase 1 examples==
<p>The following example of the first step of a simple multistep File Load illustrates the JCL statements included in a sample invocation of the File Load utility. PEOPLE is a <var class="product">Model&nbsp;204</var> file created and cataloged at some earlier time. See [[ Seven-Step File Load Examples#Seven-Step File Load Examples|Seven-Step File Load Examples]] for a complete example of a multistep File Load.</p>
<p>
<p>The following program performs these steps:</p>
The following example of the first step of a simple multistep File Load illustrates the JCL statements included in a sample invocation of the File Load utility. PEOPLE is a <var class="product">Model&nbsp;204</var> file created and cataloged at some earlier time. See [[Seven-Step File Load examples]] for a complete example of a multistep File Load.</p>
<p>
The following program performs these steps:</p>
<ol>
<ol>
<li>PEOPLE file is loaded with the data on the statements that follow the TAPEI DD statement, and the PEOPLE file is opened.</li>
<li>Loads the <code>PEOPLE</code> file with the data on the statements that follow the TAPEI DD statement, and opens the <code>PEOPLE</code> file.</li>
<li>FLOD command signals the start of a file load program that loads as many as 5000 records into PEOPLE. File load program statements are described beginning with [[ File Load Utility: FLOD and FILELOAD Commands#File Load statements: overview|File Load statements: overview]]. The G statement causes a statement from TAPEI to be read for processing.</li>
 
<li>Next four statements load one record with a Social Security field taken from columns 1 through 9, a name field taken from columns 10 through 23, a department field taken from columns 24 through 25, and a sex field taken from columns 26 through 31.</li>
<li>The <var>FLOD</var> command signals the start of a file load program that loads as many as 5000 records into <code>PEOPLE</code>. The File load program statements are described beginning with [[File Load utility: FLOD and FILELOAD commands#File Load statements: overview|File Load statements: overview]]. The <var>G</var> statement causes a statement from TAPEI to be read for processing.</li>
<li>END statement signals the end of the file load program. </li>
 
<li>The next four statements load one record with a Social Security field taken from columns 1 through 9, a name field taken from columns 10 through 23, a department field taken from columns 24 through 25, and a sex field taken from columns 26 through 31.</li>
 
<li>The <var>END</var> statement signals the end of the file load program. </li>
</ol>
</ol>
====z/OS JCL====
====z/OS JCL====
<p class="code">//JOB FILE LOAD STEP 1
<p class="code">//JOB FILE LOAD STEP 1
Line 136: Line 208:
E.
E.
</p>
</p>
====z/VSE JCL====
====z/VSE JCL====
<p class="code">// JOB FILE LOAD STEP 1
<p class="code">// JOB FILE LOAD STEP 1
Line 170: Line 243:
/&amp;
/&amp;
</p>
</p>
====z/VM EXEC====
====z/VM EXEC====
<p>To load a file using the File Load process, use the FASTLOAD EXEC. For example:</p>
<p>
To load a file using the File Load process, use the <var>[[Deferred update feature#Using deferred updates with FASTLOAD to load files|FASTLOAD]]</var> EXEC. For example:</p>
<p class="code">FASTLOAD PEOPLE (7STEP
<p class="code">FASTLOAD PEOPLE (7STEP
</p>
</p>
<p>This command invokes the PEOPLE EXEC, which looks similar to the following example. </p>
<p>
<p>The CCAIN file has the same contents as in the other operating environments. For a complete example of a multistep File Load, see [[ Seven-Step File Load Examples#Seven-Step File Load Examples|Seven-Step File Load Examples]].   </p>
This command invokes the <code>PEOPLE</code> EXEC, which looks similar to the following multistep example. </p>
<p>
The CCAIN file has the same contents as in the other operating environments. For a complete example of a multistep File Load, see [[Seven-Step File Load examples]]. </p>
 
====Multistep File Load, z/VM EXECs====
====Multistep File Load, z/VM EXECs====
<p class="code">&amp;CONTROL OFF
<p class="code">&amp;CONTROL OFF
Line 196: Line 274:
   .
   .
   .
   .
steps 2-7
<i>steps 2-7</i>
   .
   .
   .
   .
Line 208: Line 286:
&amp;EXIT
&amp;EXIT
</p>
</p>
<p>&nbsp;</p>
 
[[Category:File manager]]
[[Category:File loading and reorganization]]

Latest revision as of 21:36, 7 April 2017

Multistep File Load phases

The first phase of the multistep File Load is performed in one job step; the second phase can involve many separate job steps. Each deferred update data set generated during the first phase of the File Load, and the FRV deferred update data set if one is present, adds two job steps to the File Load process. Each data set must be processed in the following separate job steps:

  • One job step executes a sort program (such as IBM sort) to sort the deferred update data set into an order that allows efficient processing in the next job step.
  • The next job step executes the BATCH204 load module to process the sorted deferred index information and to update Tables C and D.

Phase 1

The first phase of the multistep File Load is the execution of the file load program, which formats and loads the data into Table A and Table B of the Model 204 file. The first phase also generates the deferred updates for the Model 204 file index (Tables C and D), and it places these deferred updates in the deferred update data set or data sets.

Phase 2

The second phase of the multistep File Load consists of:

  • Sorting each of the deferred update data sets generated in the first phase and the FRV deferred update data set, if one is generated.
  • Applying the sorted deferred index updates to Tables C and D.

Number of job steps needed

The number of job steps in the entire File Load process depends on the attributes of the fields being updated and the number of deferred update data sets specified in the JCL. The File Load can have three, five, or seven steps. The "Deferring NON-DEFERRABLE fields" is a quick reference chart for determining the number of job steps needed (including the first-phase job step) for the File Load.

For a description of the three-, five-, or seven-step File Load process, refer to Three-step deferred update process, Five-step deferred update process, and Seven-step deferred update process. The steps are the same for both processes except that:

  • The first-phase step of the File Load is a file load program, not a SOUL request or Host Language program.
  • When using the File Load utility, you cannot omit the variable-length deferred update data set for ORDERED fields. If an ORDERED field is updated by the file load program, a variable-length deferred update data set must be available in the first-phase step so that Model 204 can write out the Ordered Index deferred updates to that data set.

Multistep File Load, Phase 1: Overview

This section describes in detail the first phase of the File Load process.

One-step file loads are discussed in detail beginning with One-Step File Load, Phase 1: Overview.

Multistep File Load statements

The JCL for the first phase of the multistep version normally includes the statements summarized in this section. These statements are discussed in greater detail, along with descriptions of other Model 204 configurations, in Defining the runtime environment (CCAIN).

EXEC statement

The EXEC statement performs the following functions:

  • Invokes a program, usually named BATCH204
  • Allows you to specify the size of the memory area allocated for File Load runs
  • Allows you to specify a time limit for the job and a set of parameters to be passed to the job

The EXEC statement and its parameters are discussed in Specifying EXEC statement parameters.

Multistep File Load, Phase 1: Defining Model 204 system data sets

STEPLIB DD statement

STEPLIB points to the load module library into which the Model 204 BATCH204 load module has been linked. For z/VSE users, install the Model 204 BATCH204 module in a system or private core image library.

CCAAUDIT DD statement

CCAAUDIT allows you to log run statistics and other information in the audit trail. The CCAAUDIT DD statement normally specifies a SYSOUT data set. Tracking system activity (CCAJRNL, CCAAUDIT, CCAJLOG) discusses the use of the audit trail and the journals — CCAJRNL and CCAJLOG — in more detail. Specify CCAAUDIT rather than a journal data set, because File Load is run as a batch job, and you most likely want to print the run information directly.

CCATEMP, the scratch file DD statement

Model 204 requires a data set that has the file name CCATEMP to serve as a scratch file for the run. This data set can be permanent or temporary and it is initialized with each run. It must not be shared with other Model 204 jobs.

The scratch file is a special Model 204 file whose pages are used as work areas during the run. A small scratch file (20-30 pages) usually suffices for a File Load run.

CCASNAP and SYSUDUMP DD statements

Model 204 can trap program checks and file integrity problems at appropriate times and print SNAP dumps of selected portions of storage to the CCASNAP data set. These dumps are invaluable to Technical Support in locating and correcting errors. The SYSUDUMP data set also is required in the case of severe errors. Both normally are specified as SYSOUT-type data sets.

Under z/VSE, CCASNAP output is directed to the system logical unit SYSLST.

CCAPRINT and CCAIN DD statements

Include the FLOD, FILELOAD, or Z command and its associated file load program in the CCAIN data set, along with any other Model 204 commands needed to prepare the file for loading or to manipulate or check the data after loading.

Including Model 204 commands

The BATCH204 module accepts any Model 204 commands that are acceptable in a BATCH204 environment, in addition to the FLOD, FILELOAD, or Z commands.

Note: You can specify only one FLOD, FILELOAD, or Z command in any given File Load job step.

Do not issue FLOD or FILELOAD commands in any run where the NSERVS parameter is set in the CCAIN stream to less than NUSERS.

CCAIN input line length

The Model 204 system defaults expect the CCAIN data set to consist of 80-character lines that have data in columns 1 through 71. If input lines are longer than 71 characters, any nonblank character in column 72 indicates continuation of the line onto the next one. Model 204 EXEC statement parameters can be used to reset these defaults, as discussed in Specifying EXEC statement parameters.

For z/OS

CCAPRINT defines a simple sequential output data set (usually SYSOUT=A in z/OS JCL) that contains a summary of the user parameter lines and output generated by User 0.

A sequential input data set, CCAIN, often is defined under z/OS by the statement:

//CCAIN DD *

This data set defines the input for User 0, the batch user. This input defines the run-time parameters for the current run of the File Load utility and contains all the commands used to create and load the file.

For z/VSE

Under z/VSE, CCAPRINT output is directed to system logical unit SYSLST, and User 0 input statements usually follow the program execution statement. For example:

// EXEC BATCH204 // User 0 input . . .

CCASTAT DD statement

All runs that use the Model 204 security features require a CCASTAT definition statement pointing to a previously created password table data set that contains user and file security information. Refer to the discussion of this password table in File passwords and privileges. To require login, specify a SYSOPT of 16 and include a CCASTAT statement.

Multistep File Load, Phase 1: Defining Model 204 file data sets

The control statements must include a definition statement for each data set of each Model 204 file to be used in the run.

Defining TAPEI

The TAPEI data set contains the data that is to be loaded into the Model 204 file by the file load program. The file load program automatically defers updates to the index and writes the updates to the deferred update data set or data sets. It may be necessary to provide a fixed-length deferred update data set, a variable-length deferred update data set, or both. For information on deferred update data sets, see Deferred update feature.

Defining TAPE2

The default file name of the fixed-length deferred update data set is TAPE2. Only hashed index updates (KEY and NUMERIC RANGE fields) can be written to the fixed-length deferred update data set.

Defining TAPE3

The default file name of the variable-length deferred update data set is TAPE3. Either hashed index updates (KEY and NUMERIC RANGE fields) or Ordered Index updates (ORDERED fields) can be written to the variable-length deferred update data set. If ORDERED fields are updated by the file load program, there must be a variable-length deferred update data set supplied in the JCL of the first-phase step.

Overriding default file names

The default file names of TAPE2 and TAPE3 can be overridden simply by specifying the desired file name or names on the OPEN command. See Deferred update OPEN syntax for the correct syntax to use for declaring deferred update data sets.

Note: Never specify a deferred update data set as DUMMY.

Multistep File Load, Phase 1: Setting the SPCORE parameter

The SPCORE parameter specifies the minimum amount of storage to leave allocated at the end of Model 204 initialization. When viewed during the run, SPCORE indicates the amount of storage left unallocated just after initialization. Set SPCORE on User 0's parameter line to a value large enough to accommodate the output buffers of the deferred update data set(s), the input buffers for TAPEI, and the storage required to open the Model 204 file.

Calculating dynamic storage

The formula for calculating dynamic storage required for each deferred update data set is:

128 + deferred-update-data-set-blocksize * number-of-buffers

The default for the deferred update data set blocksize is 6000, and for the number of buffers is 3.

The storage required for input buffers for TAPEI is usually twice the blocksize of the TAPEI data set.

The dynamic storage required for opening the Model 204 file depends on the number of data sets and extents involved. 128 is sufficient for single data set, single extent files. The exact formula is:

32 + 8 * number-of-logical-extents + 16 * number-of-data-sets + 24 * number-of-physical-extents-in-all-data-sets

A new file has 6 logical extents.

Multistep File Load, Phase 1 examples

The following example of the first step of a simple multistep File Load illustrates the JCL statements included in a sample invocation of the File Load utility. PEOPLE is a Model 204 file created and cataloged at some earlier time. See Seven-Step File Load examples for a complete example of a multistep File Load.

The following program performs these steps:

  1. Loads the PEOPLE file with the data on the statements that follow the TAPEI DD statement, and opens the PEOPLE file.
  2. The FLOD command signals the start of a file load program that loads as many as 5000 records into PEOPLE. The File load program statements are described beginning with File Load statements: overview. The G statement causes a statement from TAPEI to be read for processing.
  3. The next four statements load one record with a Social Security field taken from columns 1 through 9, a name field taken from columns 10 through 23, a department field taken from columns 24 through 25, and a sex field taken from columns 26 through 31.
  4. The END statement signals the end of the file load program.

z/OS JCL

//JOB FILE LOAD STEP 1 //M204FLOD EXEC PGM=BATCH204,REGION=3072K //STEPLIB DD DSN=M204.LOADLIB,DISP=SHR //CCAPRINT DD SYSOUT=A //CCAAUDIT DD SYSOUT=A //CCASNAP DD SYSOUT=A //SYSUDUMP DD SYSOUT=A //CCATEMP DD UNIT=3380,SPACE=(TRK,3), // DISP=(NEW,DELETE) //PEOPLE DD DSN=M204.FILE.PEOPLE,DISP=SHR //TAPEI DD *,DCB=(LRECL=80,BLKSIZE=80) 012345678JOHNSON 03FEMALE 048343520KITTREDGE 11MALE . . . /* //TAPE2 DD DSN=M204.DEFER.FIXED.PEOPLE,DISP=(NEW,CATLG), // UNIT=TAPE,DCB=(RECFM=FB,LRECL=24,BLKSIZE=6000) //TAPE3 DD DSN=M204.DEFER.VAR.PEOPLE,DISP=(NEW,CATLG), // UNIT=TAPE,DCB=(RECFM=VB,LRECL=270,BLKSIZE=6000) //CCAIN DD * SPCORE=20000 OPEN PEOPLE FLOD 5000,5000,0 G SOC.SEC.NO=1,9,X'8000' NAME=10,14 DEPT=24,2 SEX=26,6 END EOJ /* // E.

z/VSE JCL

// JOB FILE LOAD STEP 1 // DLBL CCATEMP,'M204.CCATEMP',,DA // EXTENT SYSnnn,volser // DLBL PEOPLE,'M204.FILE.PEOPLE',,DA // EXTENT SYSnnn,volser // DLBL TAPEI,'PEOPLE.INPUT.DATA' // EXTENT SYSnnn,volser // UPSI 10000000 // TLBL SYS015,'M204.DEFERF.PEOPLE' // ASSGN SYS015,TAPE // TLBL SYS016,'M204.DEFERV.PEOPLE' // ASSGN SYS016,TAPE // EXEC BATCH204,SIZE=AUTO MAXBUF=20 DEFINE DATASET TAPEI WITH SCOPE=SYSTEM - LRECL=input-rec-length BLKSIZE=input-blk-length - RECFM=input-record-format DEFINE DATASET TAPE2 WITH SCOPE=SYSTEM - LRECL=24 BLKSIZE=6000 RECFM=FB DDNAME=SYS015 DEFINE DATASET TAPE3 WITH SCOPE=SYSTEM - LRECL=270 BLKSIZE=6000 RECFM=VB DDNAME=SYS016 OPEN PEOPLE FLOD 5000,5000,0 G SOC.SEC.NO=1,9,X'8000' NAME=10,14 DEPT=24,2 SEX=26,6 END EOJ /* /&

z/VM EXEC

To load a file using the File Load process, use the FASTLOAD EXEC. For example:

FASTLOAD PEOPLE (7STEP

This command invokes the PEOPLE EXEC, which looks similar to the following multistep example.

The CCAIN file has the same contents as in the other operating environments. For a complete example of a multistep File Load, see Seven-Step File Load examples.

Multistep File Load, z/VM EXECs

&CONTROL OFF &ERROR &EXIT &RETCODE . . . access disks . . . -FLOAD1 FILEDEF PEOPLE E DSN M204 PEOPLE FILEDEF TAPE2 E DSN PEOPLE DEFFIX (BLOCK 24 LRECL 24 RECFM FB FILEDEF TAPE3 E DSN PEOPLE DEFVAR (BLOCK 6000 LRECL 270 RECFM VB FILEDEF TAPEI DISK PEOPLE FLODINPT D (BLOCK 990 LRECL 986 RECFM VB FILEDEF CCAIN DISK PEOPLE CCAIN * &GOTO -COMMON . . . steps 2-7 . . . -COMMON FILEDEF CCAPRINT DISK FLOD&1 CCAPRINT A FILEDEF CCAAUDIT DISK FLOD&1 CCAAUDIT A FILEDEF CCATEMP E DSN M204 CCATEMP FILEDEF CCASNAP PRINTER &STACK SYSOPT 128 &EXIT