Hash key files: Difference between revisions

From m204wiki
Jump to navigation Jump to search
No edit summary
 
(16 intermediate revisions by 3 users not shown)
Line 1: Line 1:
==Overview==
==Overview==
<p>Some <var class="product">Model&nbsp;204</var> applications require each record to contain a unique or nearly unique identifier field, such as a serial number or a Social Security number. These applications often retrieve and process records one at a time according to this field.</p>
<p>
<p><var class="product">Model&nbsp;204</var> typically requires two disk transfers to accomplish this: one to the index and one to the record in Table B. By making the file a hash key file, the disk read of the index is eliminated and the whole operation is quicker and more efficient. If the retrieval is for more than a single hash key value, or if the records are not processed immediately, however, a hash key file might not provide any savings.     </p>
Some <var class="product">Model&nbsp;204</var> applications require each record to contain a unique or nearly unique identifier field, such as a serial number or a Social Security number. These applications often retrieve and process records one at a time according to this field.</p>
<p>
<var class="product">Model&nbsp;204</var> typically requires two disk transfers to accomplish this: one to the index and one to the record in Table B. By making the file a hash key file, the disk read of the index is eliminated and the whole operation is quicker and more efficient. If the retrieval is for more than a single hash key value, or if the records are not processed immediately, however, a hash key file might not provide any savings. </p>
 
===Hash key field===
===Hash key field===
<p>In a hash key file, one and only one field is designated the hash key when the file is initialized. When a record is stored in a hash key file, it is stored on an apparently random page of Table B; the page number actually depends upon the value of the record's hash key field. This hash key field is similar to the sort key of sorted <var class="product">Model&nbsp;204</var> files and often appears syntactically where a sort field would appear (see [[Sorted Files]] for a discussion of sorted files). A file can be either a sorted file or a hash key file, but not both.</p>
<p>
<p>This chapter describes the hash keys, parameters, creation, and loading of hashed files.</p>
In a hash key file, one and only one field is designated the hash key when the file is initialized. When a record is stored in a hash key file, it is stored on an apparently random page of Table B; the page number actually depends upon the value of the record's hash key field. This hash key field is similar to the sort key of sorted <var class="product">Model&nbsp;204</var> files and often appears syntactically where a sort field would appear (see [[Sorted files]] for a discussion of sorted files). A file can be either a sorted file or a hash key file, but not both.</p>
<p>
This page describes the hash keys, parameters, creation, and loading of hashed files.</p>
 
==Characteristics of hash key files==
==Characteristics of hash key files==
<p>Hash keys have the following characteristics:</p>
<p>
Hash keys have the following characteristics:</p>
<ul>
<ul>
<li>Any one record can have only one value for the hash key field.</li>
<li>Any one record can have only one value for the hash key field.</li>
<li>Several different records can have the same value for their hash key fields. These records are stored near each other in Table B.</li>
<li>Several different records can have the same value for their hash key fields. These records are stored near each other in Table B.</li>
<li>Value of a record's hash key field cannot be changed in any way. You can, however, delete the entire record and store it again with a different hash key value.</li>
<li>Value of a record's hash key field cannot be changed in any way. You can, however, delete the entire record and store it again with a different hash key value.</li>
<li>Special forms of the User Language STORE RECORD statement and the Host Language IFSTOR and IFBREC functions allow a hash key to be specified for a new record. See the Rocket <var class="product">Model&nbsp;204</var> User Language Manual and the Rocket <var class="product">Model&nbsp;204</var> Host Language Interface Reference Manual for details.     </li>
 
<li>Special forms of the SOUL [[Data maintenance#STORE RECORD statement|STORE RECORD statement]] and the Host Language <var>[[IFSTOR (HLI function)|IFSTOR]]</var> and <var>[[IFBREC (HLI function)|IFBREC]]</var> functions allow a hash key to be specified for a new record.</li>
</ul>
</ul>
==Hash key file parameters==
==Hash key file parameters==
<p>The following parameters are relevant to hash key files.  </p>
<p>
The following parameters are relevant to hash key files.  </p>
 
===BSIZE parameter===
===BSIZE parameter===
<p>BSIZE determines the number of pages to be assigned to Table B. Its computation is described in [[File Size Calculation]] .</p>
<p>
<p>BSIZE must be computed particularly carefully, because the size of a hash key file's Table B cannot be changed with an INCREASE command. If Table B fills up, the entire file must be reloaded.  </p>
<var>[[BSIZE parameter|BSIZE]]</var> determines the number of pages to be assigned to Table B. Its computation is described in [[File sizing introduction]] .</p>
<p>
<var>BSIZE</var> must be computed particularly carefully, because the size of a hash key file's Table B cannot be changed with an <var>INCREASE</var> command. If Table B fills up, the entire file must be reloaded.  </p>
 
===FILEORG parameter===
===FILEORG parameter===
<p>FILEORG controls various hash key file options. It normally defaults to 0, indicating an ordinary file. Sorted file options are summarized in [[Sorted Files]]. The available options for hash key files are:  </p>
<p>
<var>[[FILEORG parameter|FILEORG]]</var> controls various hash key file options. It normally defaults to 0, indicating an ordinary file. Sorted file options are summarized in [[Sorted files]]. The available options for hash key files are:  </p>
<table>
<table>
<tr class="head">
<tr class="head">
Line 25: Line 42:
<th>Meaning</th>
<th>Meaning</th>
</tr>
</tr>
<tr>
<tr>
<td align="right"> X'08'</td>
<td align="right"> X'08'</td>
<td>
<td>Hash key file.  
<p>Hash key file. </p>
<p>User designates the field that becomes the hash key. <var class="product">Model&nbsp;204</var> generates a hash key for each record supplied without one. The generated key determines where to place the record but is not stored with the record.</p>
<p>User designates the field that becomes the hash key. <var class="product">Model&nbsp;204</var> generates a hash key for each record supplied without one. The generated key determines where to place the record but is not stored with the record.</p>
</td>
</td></tr>
</tr>
 
<tr>
<tr>
<td align="right"> X'04'</td>
<td align="right"> X'04'</td>
<td>
<td>Reuse record numbers (RRN).  
<p>Reuse record numbers (RRN). </p>
<p>Record numbers of deleted records are reused for new records added, if available on the Table B page to which the new record is being added.    </p>
<p>Record numbers of deleted records are reused for new records added, if available on the Table B page to which the new record is being added.    </p>
</td>
</td></tr>
</tr>
 
<tr>
<tr>
<td align="right"> X'02'</td>
<td align="right"> X'02'</td>
<td>
<td>Hash key required.  
<p>Hash key required. </p>
<p>
<p>Every record in the file must have a value for the hash key, or a compilation error results.</p>
Every record in the file must have a value for the hash key, or a compilation error results.</p>
</td>
</td></tr>
</tr>
</table>
</table>
<p>Set FILEORG to the sum of the desired options. For a complete description of the FILEORG parameter, see the <var class="product">Model&nbsp;204</var> Parameter and Command Reference.</p>
<p>
Set <var>FILEORG</var> to the sum of the desired options.</p>
 
===HASHKEY parameter===
===HASHKEY parameter===
<p>The VIEW HASHKEY command displays the hash key field of the file, if the file is a hashed file and the user has a sufficient security level to view the field name. This parameter is set by <var class="product">Model&nbsp;204</var> at the time of file initialization according to the hash key specified for the file.  </p>
<p>
The <code>VIEW HASHKEY</code> command displays the hash key field of the file, if the file is a hashed file and the user has a sufficient security level to view the field name. This parameter is set by <var class="product">Model&nbsp;204</var> at the time of file initialization according to the hash key specified for the file.  </p>
 
==Creating a hash key file==
==Creating a hash key file==
<p>A hash key file is created in the normal way, as described in the file creation chapter. Set the FILEORG parameter at this time.</p>
<p>
A hash key file is created in the normal way, as described in [[Creating a file]]. Set the <var>FILEORG</var> parameter at this time.</p>
 
===Initializing hash key files===
===Initializing hash key files===
<p>A special form of the INITIALIZE command is provided for hash key files. This form enables you to establish the field name of the file's hash key. This special INITIALIZE command must be used each time the file is initialized. </p>
<p>
A special form of the <var>[[INITIALIZE command|INITIALIZE]]</var> command is provided for hash key files. This form enables you to establish the field name of the file's hash key. This special <var>INITIALIZE</var> command must be used each time the file is initialized. </p>
 
====Defining hash key fields====
====Defining hash key fields====
<p>Unlike ordinary field names, the field name of the hash key is not defined with a DEFINE command. Specify the desired field description for the hash key in the INITIALIZE command. Hash key fields cannot have the CODED, INVISIBLE, BINARY, FLOAT, or KEY attributes, and they cannot have the UPDATE option specified for them. The file initialization chapter provides a full description of the INITIALIZE command and its use with hash key fields.</p>
<p>
Unlike ordinary field names, the field name of the hash key is not defined with a <var>DEFINE</var> command. Specify the desired field description for the hash key in the <var>INITIALIZE</var> command. Hash key fields cannot have the <var>CODED</var>, <var>INVISIBLE</var>, <var>BINARY</var>, <var>FLOAT</var>, or <var>KEY</var> attributes, and they cannot have the <var>UPDATE</var> option specified for them. [[Initializing files]] provides a full description of the <var>INITIALIZE</var> command and its use with hash key fields.</p>
 
==Storing records in a hash key file==
==Storing records in a hash key file==
<p>Special forms of the User Language STORE RECORD statement and the Host Language IFBREC function allow a hash key to be specified for a new record. They are identical to the special forms used with sorted files. Refer to the Rocket <var class="product">Model&nbsp;204</var> User Language Manual and Rocket <var class="product">Model&nbsp;204</var> Host Language Interface Reference Manual for details.  </p>
<p>
Special forms of the SOUL <var>[[Data maintenance#STORE RECORD statement|STORE RECORD]]</var> statement and the Host Language <var>[[IFBREC (HLI function)|IFBREC]]</var> function allow a hash key to be specified for a new record. They are identical to the special forms used with sorted files. Refer to [[HLI: Model 204 files and records#Data files|Data files]] for a general description.  </p>
 
===File Load utility restrictions===
===File Load utility restrictions===
<p>If the records are stored with the File Load utility (see the File Load chapter), the following restrictions apply:  </p>
<p>
If the records are stored with the [[File Load utility]], the following restrictions apply:  </p>
<ul>
<ul>
<li>If the record being loaded has a hash key value, the hash key must be the first field loaded. That is, the read-and-load-a-field statement that loads the hash key must have the X'8000' mode bit specified. For example:</li>
<li>If the record being loaded has a hash key value, the hash key must be the first field loaded. That is, the read-and-load-a-field statement that loads the hash key must have the X'8000' mode bit specified. For example:
<p class="code">SOC.SEC.NO=1,9,X'8000'</p></li>
 
<li>If the record being loaded does not have a hash key value, the read-and-load-a-field statement that loads the first field in the record must specify the X'2000' mode bit, indicating no hash key, as well as X'8000'. For example:
 
<p class="code">NAME=10,15,X'A000'</p></li>
</ul>
</ul>
<b>SOC.SEC.NO=1,9,X'8000'</b>
<p>
<ul>
The following discussion describes a <var class="product">Model&nbsp;204</var> utility, M204HASH, that is designed to expedite the process of storing records in a hash key file with the File Load utility.</p>
<li>If the record being loaded does not have a hash key value, the read-and-load-a-field statement that loads the first field in the record must specify the X'2000' mode bit, indicating no hash key, as well as X'8000'. For example:</li>
 
</ul>
<b>NAME=10,15,X'A000'</b>
<p>The following discussion describes a <var class="product">Model&nbsp;204</var> utility, M204HASH, that is designed to expedite the process of storing records in a hash key file with the File Load utility.</p>
==M204HASH utility==
==M204HASH utility==
<p>The M204HASH utility optimizes the performance of loading Table B during the file load program step of the File Load utility for <var class="product">Model&nbsp;204</var> hash key files. The utility accomplishes this by sorting the file load program input records by the Table B page to which they will hash during the program. When the output file from M204HASH is used as the TAPEI file for the file load program, records are added to the <var class="product">Model&nbsp;204</var> file one page at a time. The file is thereby loaded in one pass of Table B.    </p>
<p>
<p>The utility is composed of a user exit to the standard IBM Sort/Merge or any compatible sort package such as SYNCSORT. It can be used on the z/OS, z/VM, and z/VSE operating systems, and supports fixed and variable record format files.</p>
The M204HASH utility optimizes the performance of loading Table B during the file load program step of the File Load utility for <var class="product">Model&nbsp;204</var> hash key files. The utility accomplishes this by sorting the file load program input records by the Table B page to which they will hash during the program. When the output file from M204HASH is used as the TAPEI file for the file load program, records are added to the <var class="product">Model&nbsp;204</var> file one page at a time. The file is thereby loaded in one pass of Table B.    </p>
<p>
The utility is composed of a user exit to the standard IBM Sort/Merge or any compatible sort package such as <code>SYNCSORT</code>. It can be used on the z/OS, z/VM, and z/VSE operating systems, and supports fixed and variable record format files.</p>
 
===M204HASH and Table B page numbers===
===M204HASH and Table B page numbers===
<p>To determine the Table B page number for a given record, the hash key input field is hashed and divided by the number of pages in Table B (BSIZE). The resulting Table B page number is a 3-byte binary field, which is appended to the front of the input record before sort processing. </p>
<p>
To determine the Table B page number for a given record, the hash key input field is hashed and divided by the number of pages in Table B (<var>BSIZE</var>). The resulting Table B page number is a 3-byte binary field, which is appended to the front of the input record before sort processing. </p>
 
====Removing the Table B page number====
====Removing the Table B page number====
<p>This 3-byte field is used as the sort key, which, if file load statements that correspond to the input record format are to be used for loading records into the file from the output data set, must be removed before the record is written to the output data set. </p>
<p>
This 3-byte field is used as the sort key, which, if file load statements that correspond to the input record format are to be used for loading records into the file from the output data set, must be removed before the record is written to the output data set. </p>
 
====For z/OS====
====For z/OS====
<p>In the z/OS environment, an additional user exit is provided for removing the 3-byte page number from each record. The additional user exit is necessary, because the IBM z/OS Sort/Merge program does not recognize any control statement for reformatting records before output.     </p>
<p>
In the z/OS environment, an additional user exit is provided for removing the 3-byte page number from each record. The additional user exit is necessary, because the IBM z/OS Sort/Merge program does not recognize any control statement for reformatting records before output. </p>
 
====For z/VM and z/VSE====
====For z/VM and z/VSE====
<p>In the z/VM and z/VSE environments, a sort program control statement can be used to remove the 3-byte page number before the records are written to the output data set.</p>
<p>
In the z/VM and z/VSE environments, a sort program control statement can be used to remove the 3-byte page number before the records are written to the output data set.</p>
 
===M204HASH utility input requirements===
===M204HASH utility input requirements===
====Input record requirements====
====Input record requirements====
<p>Every input record to M204HASH must contain at least the hash key for the <var class="product">Model&nbsp;204</var> record to which the data on that sequential record belongs. The output of the standard PAI FLOD method of reorganizing a file does not meet this requirement and is unsuitable for use with M204HASH. </p>
<p>
Every input record to M204HASH must contain at least the hash key for the <var class="product">Model&nbsp;204</var> record to which the data on that sequential record belongs. The output of the standard <var>PAI FLOD</var> method of reorganizing a file does not meet this requirement and is unsuitable for use with M204HASH. </p>
 
====Using multiple sequential records====
====Using multiple sequential records====
<p>The input to M204HASH typically is a sequential file of records, each record containing all data for a single <var class="product">Model&nbsp;204</var> record. If any <var class="product">Model&nbsp;204</var> record contains more data than can fit on a single 256-byte record (256 being the maximum record length allowed for sequential input), multiple sequential records, each containing the hash key, are required for that <var class="product">Model&nbsp;204</var> record. In this case, modify the SORT FIELDS sort control statement (described in [[#SORT statement|SORT statement]]) to include the position of the hash key as well as the position of the three-byte page number, to keep in order all sequential records that contain data for each single <var class="product">Model&nbsp;204</var> record. </p>
<p>
<p>Note</p>
The input to M204HASH typically is a sequential file of records, each record containing all data for a single <var class="product">Model&nbsp;204</var> record. If any <var class="product">Model&nbsp;204</var> record contains more data than can fit on a single 256-byte record (256 being the maximum record length allowed for sequential input), multiple sequential records, each containing the hash key, are required for that <var class="product">Model&nbsp;204</var> record. In this case, modify the <var>SORT FIELDS</var> sort control statement (described in [[#SORT statement|SORT statement]]) to include the position of the hash key as well as the position of the three-byte page number, to keep in order all sequential records that contain data for each single <var class="product">Model&nbsp;204</var> record. </p>
<p>The position of the hash key for the SORT FIELDS statement appears three bytes later on the record than it appears on input to M204HASH because of the three-byte page number appended to the front of the record. </p>
 
===$HSH function===
<p class="note"><b>Note:</b>
<p>The User Language $HSH function allows you to explicitly call the M204HASH utility. For information about using $HSH, see the Rocket <var class="product">Model&nbsp;204</var> User Language Manual.</p>
The position of the hash key for the <var>SORT FIELDS</var> statement appears three bytes later on the record than it appears on input to M204HASH because of the three-byte page number appended to the front of the record. </p>
 
===$Hsh function===
<p>
The SOUL <var>[[$Hsh]]</var> function allows you to explicitly call the M204HASH utility. </p>
 
==Using M204HASH in the z/OS environment==
==Using M204HASH in the z/OS environment==
<p>The object library distributed with the z/OS installation contains the object modules HA15OS and HA35OS. Link-edit HA15OS and place it in a load library.</p>
<p>
<p>The examples in [[#MODS statement|MODS statement]] and in [[#Using M204HASH in the z/VM environment|Using M204HASH in the z/VM environment]] assume that the link-edit module has been given the name HASH15. If you use the IBM z/OS Sort/Merge and you remove the 3-byte page number from the front of the output records, you must link-edit the HA35OS object module and place it in a load library. The following examples assume that this link-edit module has been given the name HASH35.</p>
The object library distributed with the z/OS installation contains the object modules <code>HA15OS</code> and <code>HA35OS</code>. Link-edit <code>HA15OS</code> and place it in a load library.</p>
<p>
The examples in [[#MODS statement|MODS statement]] and in [[#Using M204HASH in the z/VM environment|Using M204HASH in the z/VM environment]] assume that the link-edit module has been given the name <code>HASH15</code>. If you use the IBM z/OS Sort/Merge and you remove the 3-byte page number from the front of the output records, you must link-edit the <code>HA35OS</code> object module and place it in a load library. The following examples assume that this link-edit module has been given the name <code>HASH35</code>.</p>
 
===Required DD statements===
===Required DD statements===
<p>Follow the instructions in the sort package documentation for setting up the sort job. Five additional DD statements are required:</p>
<p>
Follow the instructions in the sort package documentation for setting up the sort job. Five additional DD statements are required:</p>
<table>
<table>
<tr class="head">
<tr class="head">
Line 99: Line 152:
<th>Defines... </th>
<th>Defines... </th>
</tr>
</tr>
<tr>
<tr>
<td>CCAIN</td>
<td>CCAIN</td>
<td>Control statement input data set containing 80-byte records. The control statements are described in [[#CCAIN control statements under z/OS|CCAIN control statements under z/OS]].  </td>
<td>Control statement input data set containing 80-byte records. The control statements are described in [[#CCAIN control statements under z/OS|CCAIN control statements under z/OS]].  </td>
</tr>
</tr>
<tr>
<tr>
<td>CCAPRINT</td>
<td>CCAPRINT</td>
<td>Error message output data set, which contains 80-byte records.  </td>
<td>Error message output data set, which contains 80-byte records.  </td>
</tr>
</tr>
<tr>
<tr>
<td>UNSORTED</td>
<td>UNSORTED</td>
<td>Data set that contains the records to be sorted. Any SORTIN DD statement present is ignored by the M204HASH utility. </td>
<td>Data set that contains the records to be sorted. Any SORTIN DD statement present is ignored by the M204HASH utility. </td>
</tr>
</tr>
<tr>
<tr>
<td>SORTED</td>
<td>SORTED</td>
<td>Data set into which the sorted records are placed as output. The M204HASH utility overwrites the record format, logical record length, and block size of the SORTED data set with corresponding values from the UNSORTED file. </td>
<td>Data set into which the sorted records are placed as output. The M204HASH utility overwrites the record format, logical record length, and block size of the <code>SORTED</code> data set with corresponding values from the <code>UNSORTED</code> file. </td>
</tr>
</tr>
<tr>
<tr>
<td>CCAEXITS</td>
<td>CCAEXITS</td>
<td>Library that contains the link-edited exit modules HASH15 and HASH35.</td>
<td>Library that contains the link-edited exit modules <code>HASH15</code> and <code>HASH35</code>.</td>
</tr>
</tr>
</table>
</table>
==CCAIN control statements under z/OS==
==CCAIN control statements under z/OS==
<p>The CCAIN data set must contain two control statements, the BSIZE and HASH KEY statements. It can contain one additional statement, the MODE statement. The statements need not begin in the first column, and they can appear in any order. Comment lines beginning with an asterisk are allowed. Descriptions of these statements follow. </p>
<p>
The CCAIN data set must contain two control statements, the <var>BSIZE</var> and <var>HASH KEY</var> statements. It can contain one additional statement, the <var>MODE</var> statement. The statements need not begin in the first column, and they can appear in any order. Comment lines beginning with an asterisk are allowed. Descriptions of these statements follow. </p>
 
===BSIZE statement===
===BSIZE statement===
<p>The syntax for the BSIZE statement is:</p>
<p>
====Syntax====
The syntax for the <var>BSIZE</var> statement is:</p>
<p class="code">BSIZE=<var class="term">number</var>  
<p class="syntax">BSIZE=<span class="term">number</span>  
</p>
</p>
<p>where:</p>
<p>
<p>number is the actual number of Table B pages of the file into which the records are loaded. This number must be in decimal format.</p>
Where:</p>
<p>
<var class="term">number</var> is the actual number of Table B pages of the file into which the records are loaded. This number must be in decimal format.</p>
 
===HASH KEY statement===
===HASH KEY statement===
<p>The syntax for the HASH KEY statement is:</p>
<p>
====Syntax====
The syntax for the <var>HASH KEY</var> statement is:</p>
<p class="code">HASH KEY=<var class="term">position</var>,<var class="term">length</var>   
 
<p class="syntax">HASH KEY=<span class="term">position</span>,<span class="term">length</span>   
</p>
</p>
<p>where:</p>
<p>
<p>position is a decimal number that defines the position of the hash key field on the input record</p>
Where:</p>
<p>length is its length.</p>
<ul>
<li><var class="term">position</var> is a decimal number that defines the position of the hash key field on the input record.</li>
 
<li><var class="term">length</var> is its length.</li>
</ul>
 
====HASH KEY statement with fixed length records====
====HASH KEY statement with fixed length records====
<p>For fixed length records, these values must be the same as the position and length values that appear in the file load statement for loading the hash key field (the statement on which the mode bits for starting a new record appear). </p>
<p>
For fixed length records, these values must be the same as the position and length values that appear in the file load statement for loading the hash key field (the statement on which the mode bits for starting a new record appear). </p>
 
====HASH KEY statement with variable length records====
====HASH KEY statement with variable length records====
<p>For variable length records, do not consider the 4-byte Record Descriptor Word as part of the record. That is, if the hash key field begins on the first data byte of the record, code 1 for position, not 5, as in the file load statement.</p>
<p>
For variable length records, do not consider the 4-byte Record Descriptor Word as part of the record. That is, if the hash key field begins on the first data byte of the record, code 1 for position, not 5, as in the file load statement.</p>
 
===MODE statement===
===MODE statement===
<p>The syntax for the MODE statement is:</p>
<p>
====Syntax====
The syntax for the <var>MODE</var> statement is:</p>
<p class="code">MODE=<var>X'hexadecimal_digits</var>'  
<p class="syntax">MODE = X'<span class="term">hexadecimal_digits</span>'  
</p>
</p>
<p>where:</p>
<p>
<p>hexadecimal_digits within the quotes are the same as that of the mode bits in the file load statement, as described in "File load statements: mode bits."</p>
Where:</p>
<p>Because the hash field is always the first field of each record loaded into a hash file, the first digit of the mode bits in the file load statement is always eight. The eight means start a new record.</p>
<p>
<var class="term">hexadecimal_digits</var> within the quotes are the same as that of the mode bits in the file load statement, as described in [[File Load utility: FLOD and FILELOAD commands#File Load statements: mode bits|File load statements: mode bits]].</p>
<p>
Because the hash field is always the first field of each record loaded into a hash file, the first digit of the mode bits in the file load statement is always 8. The 8 means start a new record.</p>
 
====File load mode bits====
====File load mode bits====
<p>The mode bits indicate whether or not to strip leading and trailing blanks and/or leading zeroes, and whether to store an all-zero field as 0 or not to store it at all. Valid values for the mode bits in hash key files are listed in [[#Mode bits for hash key files|Mode bits for hash key files]]. </p>
<p>
The mode bits indicate whether or not to strip leading and trailing blanks and/or leading zeroes, and whether to store an all-zero field as 0 or not to store it at all. Valid values for the mode bits in hash key files are listed in [[#Mode bits for hash key files|Mode bits for hash key files]]. </p>
 
===Mode bits for hash key files===
===Mode bits for hash key files===
<p>The following mode bits can be used with hash key files. For a complete description of the mode bits, see "File load statements: mode bits." </p>
<p>
The following mode bits can be used with hash key files. For a complete description of the mode bits, see [[File Load utility: FLOD and FILELOAD commands#File Load statements: mode bits|File load statements: mode bits]]. </p>
<table>
<table>
<tr class="head">
<tr class="head">
Line 158: Line 239:
<th>Meaning</th>
<th>Meaning</th>
</tr>
</tr>
<tr>
<tr>
<td align="right">X'8000'</td>
<td align="right">X'8000'</td>
<td>Begin a new <var class="product">Model&nbsp;204</var> record.</td>
<td>Begin a new <var class="product">Model&nbsp;204</var> record.</td>
</tr>
</tr>
<tr>
<tr>
<td align="right">X'2000'</td>
<td align="right">X'2000'</td>
<td>Sort or hash key omitted (specified with X'8000')</td>
<td>Sort or hash key omitted (specified with X'8000')</td>
</tr>
</tr>
<tr>
<tr>
<td align="right">X'0800'</td>
<td align="right">X'0800'</td>
<td>Suppress deletion of blanks</td>
<td>Suppress deletion of blanks</td>
</tr>
</tr>
<tr>
<tr>
<td align="right">X'0200'</td>
<td align="right">X'0200'</td>
<td>Load all-zero fields as '0'</td>
<td>Load all-zero fields as '0'</td>
</tr>
</tr>
<tr>
<tr>
<td align="right">X'0100'</td>
<td align="right">X'0100'</td>
Line 179: Line 265:
</tr>
</tr>
</table>
</table>
<p>Mode bits can be summed.</p>
<p>
Mode bits can be summed.</p>
 
==SYSIN control statements under z/OS==
==SYSIN control statements under z/OS==
<p>The sort input control file SYSIN requires three control statements:</p>
<p>
The sort input control file <code>SYSIN</code> requires three control statements:</p>
<ul>
<ul>
<li>RECORD statement</li>
<li><var>RECORD</var> statement</li>
<li>SORT statement</li>
<li><var>SORT</var> statement</li>
<li>MODS statement </li>
<li><var>MODS</var> statement </li>
</ul>
</ul>
<p>These statements are described in the following sections.</p>
<p>
These statements are described in the following sections.</p>
 
===RECORD statement===
===RECORD statement===
<p>Two parameters of the RECORD statement are required: </p>
<p>
Two parameters of the <var>RECORD</var> statement are required: </p>
<table>
<table>
<tr class="head">
<tr class="head">
Line 195: Line 287:
<th>Set...</th>
<th>Set...</th>
</tr>
</tr>
<tr>
<tr>
<td>TYPE</td>
<td>TYPE</td>
<td>To either F or V, depending on whether the input file is in fixed or variable record format. </td>
<td>To either <code>F</code> or <code>V</code>, depending on whether the input file is in fixed or variable record format. </td>
</tr>
</tr>
<tr>
<tr>
<td>LENGTH</td>
<td>LENGTH</td>
<td>
<td>At least the first three values (the fourth through seventh are optional):
<p>At least the first three values (the fourth through seventh are optional):</p>
<ul>
<ul>
<li>Set L1 to the maximum record length in the input file. For fixed length records, this is equivalent to the logical record length (LRECL). </li>
<li>Set <var>L1</var> to the maximum record length in the input file. For fixed length records, this is equivalent to the logical record length (<code>LRECL</code>). </li>
<li>Set L2 to the value of L1 plus 3 bytes for the appended Table B page number. </li>
 
<li>Set L3 to the same value as L1, if the Table B page number is removed before the record is written to the output file. </li>
<li>Set <var>L2</var> to the value of <var>L1</var> plus 3 bytes for the appended Table B page number. </li>
<li>If you choose to provide an L4 setting, set it to a value greater than or equal to 3 bytes.</li>
 
<li>Set <var>L3</var> to the same value as <var>L1</var>, if the Table B page number is removed before the record is written to the output file. </li>
 
<li>If you choose to provide an <var>L4</var> setting, set it to a value greater than or equal to 3 bytes.</li>
</ul>
</ul>
<p>For variable record format, add 4 bytes to L1, L2, L3, and L4 to include the 4-byte Record Descriptor Word. </p>
<p>
For variable record format, add 4 bytes to <var>L1</var>, <var>L2</var>, <var>L3</var>, and <var>L4</var> to include the 4-byte Record Descriptor Word. </p>
</td>
</td>
</tr>
</tr>
</table>
</table>
===SORT statement===
===SORT statement===
====For fixed record format files====
====For fixed record format files====
<p>For fixed record format the syntax is:</p>
<p>
====Syntax====
For fixed record format the syntax is:</p>
<p class="code">SORT FIELDS=(1,3,CH,A)  
<p class="syntax">SORT FIELDS=(1,3,CH,A)  
</p>
</p>
====For variable record format files====
====For variable record format files====
<p>For variable record format the syntax is:</p>
<p>
====Syntax====
For variable record format the syntax is:</p>
<p class="code">SORT FIELDS=(5,3,CH,A)     
<p class="syntax">SORT FIELDS=(5,3,CH,A)     
</p>
</p>
===MODS statement===
===MODS statement===
====Using MODS to remove the page number====
====Using MODS to remove the page number====
<p>To remove the 3-byte page number from the front of the output records the syntax is:</p>
<p>
====Syntax====
To remove the 3-byte page number from the front of the output records the syntax is:</p>
<p class="code">MODS E15=(HASH15,4000,CCAEXITS,N),E35=
<p class="syntax">MODS E15=(HASH15,4000,CCAEXITS,N),E35=
   (HASH35,900,CCAEXITS,N)  
   (HASH35,900,CCAEXITS,N)  
</p>
</p>
====Using MODS without removing the page number====
====Using MODS without removing the page number====
<p>If you do not need to remove the 3-byte page number, omit the E35 parameter from the MODS statement. The syntax is:</p>
<p>
====Syntax====
If you do not need to remove the 3-byte page number, omit the <var>E35</var> parameter from the <var>MODS</var> statement. The syntax is:</p>
<p class="code">MODS E15=(HASH15,4000,CCAEXITS,N)  
<p class="syntax">MODS E15=(HASH15,4000,CCAEXITS,N)  
</p>
</p>
===Output data set===
===Output data set===
<p>The output data set DD name used by the HASH35 exit is SORTED. If the E35 parameter is omitted, the output data set DD name must be SORTOUT, which is the sort program's standard output data set.</p>
<p>
<p>If the 3-byte page number is not removed from the output record, the file load statements used for loading the file must refer to the fourth byte of each record (eighth byte for variable-length data sets) as the beginning of the actual data.</p>
The output data set DD name used by the <var>HASH35</var> exit is <code>SORTED</code>. If the E35 parameter is omitted, the output data set DD name must be <code>SORTOUT</code>, which is the sort program's standard output data set.</p>
<p>
If the 3-byte page number is not removed from the output record, the file load statements used for loading the file must refer to the fourth byte of each record (eighth byte for variable-length data sets) as the beginning of the actual data.</p>
 
====z/OS JCL====
====z/OS JCL====
<p>The following JCL shows a sample SORT step:</p>
<p>
The following JCL shows a sample SORT step:</p>
<p class="code">//SORT EXEC PGM=SORT
<p class="code">//SORT EXEC PGM=SORT
//CCAPRINT DD SYSOUT=A
//CCAPRINT DD SYSOUT=A
Line 262: Line 370:
/*
/*
</p>
</p>
==Using M204HASH in the z/VSE environment==
==Using M204HASH in the z/VSE environment==
<p>Instructions for using M204HASH on z/VSE are similar to those for z/OS, with appropriate Job Control Language changes. Additional SYSIN control statements might also be required to define the data sets being used. In addition, the IBM z/VSE Sort/Merge program allows use of the OUTREC statement in the SYSIN file to remove the 3-byte page number from the front of the output record. This replaces the function of the HASH35 exit used under z/OS.  </p>
<p>
<p>Note the following about using M204HASH in a z/VSE environment:</p>
Instructions for using M204HASH on z/VSE are similar to those for z/OS, with appropriate Job Control Language changes. Additional SYSIN control statements might also be required to define the data sets being used. In addition, the IBM z/VSE Sort/Merge program allows use of the <var>OUTREC</var> statement in the SYSIN file to remove the 3-byte page number from the front of the output record. This replaces the function of the <code>HASH35</code> exit used under z/OS.  </p>
<p>
Note the following about using M204HASH in a z/VSE environment:</p>
<ul>
<ul>
<li>Syntax for the OUTREC statement for fixed record format is:</li>
<li>Syntax for the <var>OUTREC</var> statement for fixed record format is:
</ul>
<p class="syntax">OUTREC FIELDS=(4, <span class="term">length</span>)</p>
<b>OUTREC FIELDS=(4,length)</b>
<p>
<p>where length is the length of the original input record.</p>
Where <var class="term">length</var> is the length of the original input record.</p>
<p>The OUTREC statement syntax for variable record format is:</p>
<p>
<b>OUTREC FIELDS=(1,4,8)</b>
The <var>OUTREC</var> statement syntax for variable record format is:</p>
<ul>
<p class="syntax">OUTREC FIELDS=(1,4,8)</p> </li>
<li>HA15DOS module is contained in the object library in object format. Link-edit HA15DOS and place it in a load library, giving it the name HASH15. The HASH35 exit is not included with the installation software, because its function is performed by use of the OUTREC statement.</li>
<li><code>HA15DOS</code> module is contained in the object library in object format. Link-edit <code>HA15DOS</code> and place it in a load library, giving it the name <code>HASH15</code>. The <code>HASH35</code> exit is not included with the installation software, because its function is performed by use of the <var>OUTREC</var> statement.</li>
 
<li>CCAIN control statements must be included as in-stream data in the JCL. CCAPRINT messages appear in the output. </li>
<li>CCAIN control statements must be included as in-stream data in the JCL. CCAPRINT messages appear in the output. </li>
</ul>
</ul>
====z/VSE JCL====
====z/VSE JCL====
<p>The following JCL shows a sample SORT step:</p>
<p>
The following JCL shows a sample SORT step:</p>
<p class="code"><b></b>* $$ JOB JNM=LOAD,CLASS=G
<p class="code"><b></b>* $$ JOB JNM=LOAD,CLASS=G
// JOB LOAD
// JOB LOAD
Line 303: Line 418:
<b></b>* $$ EOJ  
<b></b>* $$ EOJ  
</p>
</p>
==Using M204HASH in the z/VM environment==
==Using M204HASH in the z/VM environment==
<p>The M204HASH utility can be used in the z/VM environment if SYNCSORT CMS is installed. </p>
<p>
<p>The procedure for installing M204HASH using SYNCSORT CMS is the same as for the standard IBM z/OS Sort/Merge described earlier in this section with the following exceptions: </p>
The M204HASH utility can be used in the z/VM environment if SYNCSORT CMS is installed. </p>
<p>
The procedure for installing M204HASH using SYNCSORT CMS is the same as for the standard IBM z/OS Sort/Merge described earlier in this section with the following exceptions: </p>
<ul>
<ul>
<li>HA15CMS exit appears on the tape in TEXT format. The HASH35 exit is not required when using SYNCSORT CMS and, therefore, does not appear with the distributed software.</li>
<li><code>HA15CMS</code> exit appears on the tape in TEXT format. The <code>HASH35</code> exit is not required when using SYNCSORT CMS and, therefore, does not appear with the distributed software.</li>
 
<li>FILEDEF commands replace DD statements. For variable record format, set the file mode number on the FILEDEF command for the SORTED file to 0, 1, 2, or 5 (or allow it to default to 1). You must change the file mode number to 4 before using the file as input to the file load program.</li>
<li>FILEDEF commands replace DD statements. For variable record format, set the file mode number on the FILEDEF command for the SORTED file to 0, 1, 2, or 5 (or allow it to default to 1). You must change the file mode number to 4 before using the file as input to the file load program.</li>
<li>HA15CMS TEXT module does not need to be link-edited before it is used, nor is a FILEDEF command required for it, because a search for the TEXT module is performed in the order described in the SYNCSORT CMS Programmer's Guide under the description of the MODS control statement. </li>
 
<li><code>HA15CMS</code> TEXT module does not need to be link-edited before it is used, nor is a FILEDEF command required for it, because a search for the TEXT module is performed in the order described in the SYNCSORT CMS Programmer's Guide under the description of the MODS control statement. </li>
</ul>
</ul>
====SYNCSORT z/VM issues====
====SYNCSORT z/VM issues====
<p>Note the following about SYNCSORT CMS:</p>
<p>
<ul>
Note the following about SYNCSORT CMS:</p>
<li>SSORT command initiates SYNCSORT. When using SSORT, specify the E15 option, giving an exit name of HA15CMS and a length of L4000.</li>
<li>SYNCSORT allows use of the OUTREC control statement in the SYSIN file to remove the 3-byte page number from the record before writing the record to the output file. This replaces the function of the HASH35 exit used in the z/OS environment.
<p>Syntax for the OUTREC statement, for fixed record format, is:</p>
</li>
</ul>
<b>OUTREC FIELDS=(4,length)</b>
<p>where length is the length of the original input record.</p>
<p>The OUTREC statement syntax for variable record format is:</p>
<b>OUTREC FIELDS=(1,4,8)</b>
<ul>
<ul>
<li><code>SSORT</code> command initiates SYNCSORT. When using <code>SSORT</code>, specify the <code>E15</code> option, giving an exit name of <code>HA15CMS</code> and a length of L4000.</li>
<li>SYNCSORT allows use of the <var>OUTREC</var> control statement in the SYSIN file to remove the 3-byte page number from the record before writing the record to the output file. This replaces the function of the <code>HASH35</code> exit used in the z/OS environment.
<p>
The syntax for the <var>OUTREC</var> statement, for fixed record format, is:</p>
<p class="syntax">OUTREC FIELDS=(4,<span class="term">length</span>)</p>
<p>
Where <var class="term">length</var> is the length of the original input record.</p>
<p>
The <var>OUTREC</var> statement syntax for variable record format is:</p>
<p class="syntax">OUTREC FIELDS=(1,4,8)</p> </li>
<li>Omit the MODS control statement in the SYSIN file, because it is ignored by SYNCSORT.  </li>
<li>Omit the MODS control statement in the SYSIN file, because it is ignored by SYNCSORT.  </li>
</ul>
</ul>
====z/VM EXEC====
====z/VM EXEC====
<p>The following sample EXEC defines the files and initiates SYNCSORT:</p>
<p>
<p>&nbsp;</p>
The following sample EXEC defines the files and initiates SYNCSORT:</p>
 
<p class="code">&amp;CONTROL OFF
<p class="code">&amp;CONTROL OFF
FILEDEF UNSORTED DISK UNSORTED DATA A (LRECL 80 BLKSIZE 80)
FILEDEF UNSORTED DISK UNSORTED DATA A (LRECL 80 BLKSIZE 80)
Line 336: Line 462:
SSORT E15 HASH15 L4000 SORTOUT DATA A SYSIN DATA A
SSORT E15 HASH15 L4000 SORTOUT DATA A SYSIN DATA A
</p>
</p>
<p>If the previous EXEC is used, the CCAIN DATA A file is:</p>
<p>
If the previous EXEC is used, the CCAIN DATA A file is:</p>
<p class="code">BSIZE=397
<p class="code">BSIZE=397
MODE=X'8B00'
MODE=X'8B00'
HASH KEY=9,4  
HASH KEY=9,4  
</p>
</p>
<p>The SYSIN DATA A file is:</p>
<p>
The SYSIN DATA A file is:</p>
<p class="code">RECORD TYPE=F,LENGTH=(80,83)
<p class="code">RECORD TYPE=F,LENGTH=(80,83)
SORT FIELDS=(1,3,CH,A)
SORT FIELDS=(1,3,CH,A)
OUTREC FIELDS=(4,80)     
OUTREC FIELDS=(4,80)     
</p> 


   
[[Category:Sorted and hash key FILEORG files]]
 
[[Category:File management]]

Latest revision as of 20:51, 20 March 2018

Overview

Some Model 204 applications require each record to contain a unique or nearly unique identifier field, such as a serial number or a Social Security number. These applications often retrieve and process records one at a time according to this field.

Model 204 typically requires two disk transfers to accomplish this: one to the index and one to the record in Table B. By making the file a hash key file, the disk read of the index is eliminated and the whole operation is quicker and more efficient. If the retrieval is for more than a single hash key value, or if the records are not processed immediately, however, a hash key file might not provide any savings.

Hash key field

In a hash key file, one and only one field is designated the hash key when the file is initialized. When a record is stored in a hash key file, it is stored on an apparently random page of Table B; the page number actually depends upon the value of the record's hash key field. This hash key field is similar to the sort key of sorted Model 204 files and often appears syntactically where a sort field would appear (see Sorted files for a discussion of sorted files). A file can be either a sorted file or a hash key file, but not both.

This page describes the hash keys, parameters, creation, and loading of hashed files.

Characteristics of hash key files

Hash keys have the following characteristics:

  • Any one record can have only one value for the hash key field.
  • Several different records can have the same value for their hash key fields. These records are stored near each other in Table B.
  • Value of a record's hash key field cannot be changed in any way. You can, however, delete the entire record and store it again with a different hash key value.
  • Special forms of the SOUL STORE RECORD statement and the Host Language IFSTOR and IFBREC functions allow a hash key to be specified for a new record.

Hash key file parameters

The following parameters are relevant to hash key files.

BSIZE parameter

BSIZE determines the number of pages to be assigned to Table B. Its computation is described in File sizing introduction .

BSIZE must be computed particularly carefully, because the size of a hash key file's Table B cannot be changed with an INCREASE command. If Table B fills up, the entire file must be reloaded.

FILEORG parameter

FILEORG controls various hash key file options. It normally defaults to 0, indicating an ordinary file. Sorted file options are summarized in Sorted files. The available options for hash key files are:

Option Meaning
X'08' Hash key file.

User designates the field that becomes the hash key. Model 204 generates a hash key for each record supplied without one. The generated key determines where to place the record but is not stored with the record.

X'04' Reuse record numbers (RRN).

Record numbers of deleted records are reused for new records added, if available on the Table B page to which the new record is being added.

X'02' Hash key required.

Every record in the file must have a value for the hash key, or a compilation error results.

Set FILEORG to the sum of the desired options.

HASHKEY parameter

The VIEW HASHKEY command displays the hash key field of the file, if the file is a hashed file and the user has a sufficient security level to view the field name. This parameter is set by Model 204 at the time of file initialization according to the hash key specified for the file.

Creating a hash key file

A hash key file is created in the normal way, as described in Creating a file. Set the FILEORG parameter at this time.

Initializing hash key files

A special form of the INITIALIZE command is provided for hash key files. This form enables you to establish the field name of the file's hash key. This special INITIALIZE command must be used each time the file is initialized.

Defining hash key fields

Unlike ordinary field names, the field name of the hash key is not defined with a DEFINE command. Specify the desired field description for the hash key in the INITIALIZE command. Hash key fields cannot have the CODED, INVISIBLE, BINARY, FLOAT, or KEY attributes, and they cannot have the UPDATE option specified for them. Initializing files provides a full description of the INITIALIZE command and its use with hash key fields.

Storing records in a hash key file

Special forms of the SOUL STORE RECORD statement and the Host Language IFBREC function allow a hash key to be specified for a new record. They are identical to the special forms used with sorted files. Refer to Data files for a general description.

File Load utility restrictions

If the records are stored with the File Load utility, the following restrictions apply:

  • If the record being loaded has a hash key value, the hash key must be the first field loaded. That is, the read-and-load-a-field statement that loads the hash key must have the X'8000' mode bit specified. For example:

    SOC.SEC.NO=1,9,X'8000'

  • If the record being loaded does not have a hash key value, the read-and-load-a-field statement that loads the first field in the record must specify the X'2000' mode bit, indicating no hash key, as well as X'8000'. For example:

    NAME=10,15,X'A000'

The following discussion describes a Model 204 utility, M204HASH, that is designed to expedite the process of storing records in a hash key file with the File Load utility.

M204HASH utility

The M204HASH utility optimizes the performance of loading Table B during the file load program step of the File Load utility for Model 204 hash key files. The utility accomplishes this by sorting the file load program input records by the Table B page to which they will hash during the program. When the output file from M204HASH is used as the TAPEI file for the file load program, records are added to the Model 204 file one page at a time. The file is thereby loaded in one pass of Table B.

The utility is composed of a user exit to the standard IBM Sort/Merge or any compatible sort package such as SYNCSORT. It can be used on the z/OS, z/VM, and z/VSE operating systems, and supports fixed and variable record format files.

M204HASH and Table B page numbers

To determine the Table B page number for a given record, the hash key input field is hashed and divided by the number of pages in Table B (BSIZE). The resulting Table B page number is a 3-byte binary field, which is appended to the front of the input record before sort processing.

Removing the Table B page number

This 3-byte field is used as the sort key, which, if file load statements that correspond to the input record format are to be used for loading records into the file from the output data set, must be removed before the record is written to the output data set.

For z/OS

In the z/OS environment, an additional user exit is provided for removing the 3-byte page number from each record. The additional user exit is necessary, because the IBM z/OS Sort/Merge program does not recognize any control statement for reformatting records before output.

For z/VM and z/VSE

In the z/VM and z/VSE environments, a sort program control statement can be used to remove the 3-byte page number before the records are written to the output data set.

M204HASH utility input requirements

Input record requirements

Every input record to M204HASH must contain at least the hash key for the Model 204 record to which the data on that sequential record belongs. The output of the standard PAI FLOD method of reorganizing a file does not meet this requirement and is unsuitable for use with M204HASH.

Using multiple sequential records

The input to M204HASH typically is a sequential file of records, each record containing all data for a single Model 204 record. If any Model 204 record contains more data than can fit on a single 256-byte record (256 being the maximum record length allowed for sequential input), multiple sequential records, each containing the hash key, are required for that Model 204 record. In this case, modify the SORT FIELDS sort control statement (described in SORT statement) to include the position of the hash key as well as the position of the three-byte page number, to keep in order all sequential records that contain data for each single Model 204 record.

Note: The position of the hash key for the SORT FIELDS statement appears three bytes later on the record than it appears on input to M204HASH because of the three-byte page number appended to the front of the record.

$Hsh function

The SOUL $Hsh function allows you to explicitly call the M204HASH utility.

Using M204HASH in the z/OS environment

The object library distributed with the z/OS installation contains the object modules HA15OS and HA35OS. Link-edit HA15OS and place it in a load library.

The examples in MODS statement and in Using M204HASH in the z/VM environment assume that the link-edit module has been given the name HASH15. If you use the IBM z/OS Sort/Merge and you remove the 3-byte page number from the front of the output records, you must link-edit the HA35OS object module and place it in a load library. The following examples assume that this link-edit module has been given the name HASH35.

Required DD statements

Follow the instructions in the sort package documentation for setting up the sort job. Five additional DD statements are required:

Statement Defines...
CCAIN Control statement input data set containing 80-byte records. The control statements are described in CCAIN control statements under z/OS.
CCAPRINT Error message output data set, which contains 80-byte records.
UNSORTED Data set that contains the records to be sorted. Any SORTIN DD statement present is ignored by the M204HASH utility.
SORTED Data set into which the sorted records are placed as output. The M204HASH utility overwrites the record format, logical record length, and block size of the SORTED data set with corresponding values from the UNSORTED file.
CCAEXITS Library that contains the link-edited exit modules HASH15 and HASH35.

CCAIN control statements under z/OS

The CCAIN data set must contain two control statements, the BSIZE and HASH KEY statements. It can contain one additional statement, the MODE statement. The statements need not begin in the first column, and they can appear in any order. Comment lines beginning with an asterisk are allowed. Descriptions of these statements follow.

BSIZE statement

The syntax for the BSIZE statement is:

BSIZE=number

Where:

number is the actual number of Table B pages of the file into which the records are loaded. This number must be in decimal format.

HASH KEY statement

The syntax for the HASH KEY statement is:

HASH KEY=position,length

Where:

  • position is a decimal number that defines the position of the hash key field on the input record.
  • length is its length.

HASH KEY statement with fixed length records

For fixed length records, these values must be the same as the position and length values that appear in the file load statement for loading the hash key field (the statement on which the mode bits for starting a new record appear).

HASH KEY statement with variable length records

For variable length records, do not consider the 4-byte Record Descriptor Word as part of the record. That is, if the hash key field begins on the first data byte of the record, code 1 for position, not 5, as in the file load statement.

MODE statement

The syntax for the MODE statement is:

MODE = X'hexadecimal_digits'

Where:

hexadecimal_digits within the quotes are the same as that of the mode bits in the file load statement, as described in File load statements: mode bits.

Because the hash field is always the first field of each record loaded into a hash file, the first digit of the mode bits in the file load statement is always 8. The 8 means start a new record.

File load mode bits

The mode bits indicate whether or not to strip leading and trailing blanks and/or leading zeroes, and whether to store an all-zero field as 0 or not to store it at all. Valid values for the mode bits in hash key files are listed in Mode bits for hash key files.

Mode bits for hash key files

The following mode bits can be used with hash key files. For a complete description of the mode bits, see File load statements: mode bits.

Mode Bit Meaning
X'8000' Begin a new Model 204 record.
X'2000' Sort or hash key omitted (specified with X'8000')
X'0800' Suppress deletion of blanks
X'0200' Load all-zero fields as '0'
X'0100' Strip leading zeroes

Mode bits can be summed.

SYSIN control statements under z/OS

The sort input control file SYSIN requires three control statements:

  • RECORD statement
  • SORT statement
  • MODS statement

These statements are described in the following sections.

RECORD statement

Two parameters of the RECORD statement are required:

Parameter Set...
TYPE To either F or V, depending on whether the input file is in fixed or variable record format.
LENGTH At least the first three values (the fourth through seventh are optional):
  • Set L1 to the maximum record length in the input file. For fixed length records, this is equivalent to the logical record length (LRECL).
  • Set L2 to the value of L1 plus 3 bytes for the appended Table B page number.
  • Set L3 to the same value as L1, if the Table B page number is removed before the record is written to the output file.
  • If you choose to provide an L4 setting, set it to a value greater than or equal to 3 bytes.

For variable record format, add 4 bytes to L1, L2, L3, and L4 to include the 4-byte Record Descriptor Word.

SORT statement

For fixed record format files

For fixed record format the syntax is:

SORT FIELDS=(1,3,CH,A)

For variable record format files

For variable record format the syntax is:

SORT FIELDS=(5,3,CH,A)

MODS statement

Using MODS to remove the page number

To remove the 3-byte page number from the front of the output records the syntax is:

MODS E15=(HASH15,4000,CCAEXITS,N),E35= (HASH35,900,CCAEXITS,N)

Using MODS without removing the page number

If you do not need to remove the 3-byte page number, omit the E35 parameter from the MODS statement. The syntax is:

MODS E15=(HASH15,4000,CCAEXITS,N)

Output data set

The output data set DD name used by the HASH35 exit is SORTED. If the E35 parameter is omitted, the output data set DD name must be SORTOUT, which is the sort program's standard output data set.

If the 3-byte page number is not removed from the output record, the file load statements used for loading the file must refer to the fourth byte of each record (eighth byte for variable-length data sets) as the beginning of the actual data.

z/OS JCL

The following JCL shows a sample SORT step:

//SORT EXEC PGM=SORT //CCAPRINT DD SYSOUT=A //SYSOUT DD SYSOUT=A //SYSUDUMP DD SYSOUT=A //UNSORTED DD DSN=UNSORTED.DATA,DISP=SHR //SORTED DD DSN=FLOD.INPUT,DISP=SHR //SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,500) //SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,500) //SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,500) //CCAEXITS DD DSN=M204.LINKLIB,DISP=SHR //CCAIN DD * BSIZE=397 HASH KEY=9,4 MODE=X'8B00' /* //SYSIN DD * SORT FIELDS=(1,3,CH,A) RECORD TYPE=F,LENGTH=(80,83,80) MODS E15=(HASH15,4000,CCAEXITS,N),E35=(HASH35,900,CCAEXITS,N) /*

Using M204HASH in the z/VSE environment

Instructions for using M204HASH on z/VSE are similar to those for z/OS, with appropriate Job Control Language changes. Additional SYSIN control statements might also be required to define the data sets being used. In addition, the IBM z/VSE Sort/Merge program allows use of the OUTREC statement in the SYSIN file to remove the 3-byte page number from the front of the output record. This replaces the function of the HASH35 exit used under z/OS.

Note the following about using M204HASH in a z/VSE environment:

  • Syntax for the OUTREC statement for fixed record format is:

    OUTREC FIELDS=(4, length)

    Where length is the length of the original input record.

    The OUTREC statement syntax for variable record format is:

    OUTREC FIELDS=(1,4,8)

  • HA15DOS module is contained in the object library in object format. Link-edit HA15DOS and place it in a load library, giving it the name HASH15. The HASH35 exit is not included with the installation software, because its function is performed by use of the OUTREC statement.
  • CCAIN control statements must be included as in-stream data in the JCL. CCAPRINT messages appear in the output.

z/VSE JCL

The following JCL shows a sample SORT step:

* $$ JOB JNM=LOAD,CLASS=G // JOB LOAD // DLBL UNSORT,'UNSORTED.VARIABLE.RECORDS',60 // EXTENT SYS023,SYSWK3 // DLBL SORT,'SORTED.VARIABLE.RECORDS',60 // EXTENT SYS023,SYSWK3,,,33409,200 // DLBL HASHCL,'M204.LOAD.LIBRARY' // EXTENT ,SYSWK3 // LIBDEF CL,SEARCH=HASHCL // EXEC SORT,SIZE=100K SORT FIELDS=(5,3,CH,A),FILES=1,WORK=1 RECORD TYPE=V,LENGTH=(28,31) INPFIL BLKSIZE=2400 OUTFIL BLKSIZE=2400 OPTION SORTIN=(023),SORTOUT=(023),FILNM=(SORT,UNSORT) OUTREC FIELDS=(1,4,8) MODS PH1=(HASH15,L4000,E15) /* BSIZE=397 MODE=X'8B00' HASH KEY=9,4 /* /& * $$ EOJ

Using M204HASH in the z/VM environment

The M204HASH utility can be used in the z/VM environment if SYNCSORT CMS is installed.

The procedure for installing M204HASH using SYNCSORT CMS is the same as for the standard IBM z/OS Sort/Merge described earlier in this section with the following exceptions:

  • HA15CMS exit appears on the tape in TEXT format. The HASH35 exit is not required when using SYNCSORT CMS and, therefore, does not appear with the distributed software.
  • FILEDEF commands replace DD statements. For variable record format, set the file mode number on the FILEDEF command for the SORTED file to 0, 1, 2, or 5 (or allow it to default to 1). You must change the file mode number to 4 before using the file as input to the file load program.
  • HA15CMS TEXT module does not need to be link-edited before it is used, nor is a FILEDEF command required for it, because a search for the TEXT module is performed in the order described in the SYNCSORT CMS Programmer's Guide under the description of the MODS control statement.

SYNCSORT z/VM issues

Note the following about SYNCSORT CMS:

  • SSORT command initiates SYNCSORT. When using SSORT, specify the E15 option, giving an exit name of HA15CMS and a length of L4000.
  • SYNCSORT allows use of the OUTREC control statement in the SYSIN file to remove the 3-byte page number from the record before writing the record to the output file. This replaces the function of the HASH35 exit used in the z/OS environment.

    The syntax for the OUTREC statement, for fixed record format, is:

    OUTREC FIELDS=(4,length)

    Where length is the length of the original input record.

    The OUTREC statement syntax for variable record format is:

    OUTREC FIELDS=(1,4,8)

  • Omit the MODS control statement in the SYSIN file, because it is ignored by SYNCSORT.

z/VM EXEC

The following sample EXEC defines the files and initiates SYNCSORT:

&CONTROL OFF FILEDEF UNSORTED DISK UNSORTED DATA A (LRECL 80 BLKSIZE 80) FILEDEF SORTED DISK SORTED DATA A (LRECL 80 BLKSIZE 80) FILEDEF CCAIN DISK CCAIN DATA A FILEDEF CCAPRINT DISK CCAPRINT DATA A (LRECL 80 BLKSIZE 80) SSORT E15 HASH15 L4000 SORTOUT DATA A SYSIN DATA A

If the previous EXEC is used, the CCAIN DATA A file is:

BSIZE=397 MODE=X'8B00' HASH KEY=9,4

The SYSIN DATA A file is:

RECORD TYPE=F,LENGTH=(80,83) SORT FIELDS=(1,3,CH,A) OUTREC FIELDS=(4,80)