File reorganization and table compaction - Revision history

JAL: /* COMPACTB data compaction */ replace hyphen with emdash

2018-04-18T20:15:01Z

COMPACTB data compaction: replace hyphen with emdash

← Older revision		Revision as of 20:15, 18 April 2018
Line 852:		Line 852:
	</ul>		</ul>
	<p>		<p>
	<var>COMPACTB</var> processing can be used to avoid frequent file reorganizations needed to reduce the number of extension records and can also be used to reclaim space occupied by logically deleted records~~-some times~~ referred to as dirty data.</p>		<var>COMPACTB</var> processing can be used to avoid frequent file reorganizations needed to reduce the number of extension records and can also be used to reclaim space occupied by logically deleted records — sometimes referred to as dirty data. </p>
	<p>		<p>
	Recovery of the compaction process is fully supported. </p>		Recovery of the compaction process is fully supported. </p>
Line 862:		Line 862:
	====Compaction for files with Table X====		====Compaction for files with Table X====
	<p>		<p>
	<var>COMPACTB</var> processing supports files defined with a Table X. Compaction for files defined with Table X have better performance, because compaction operates on a page basis instead of a record basis and locks fewer resources, especially the existence bit map.</p>		<var>COMPACTB</var> processing supports files defined with a Table X. Compaction for files defined with Table X have better performance, because compaction operates on a page basis instead of a record basis and locks fewer resources, especially the existence bit map.</p>

	====Physical delete of logically deleted records====		====Physical delete of logically deleted records====

JAL: link repair

2016-02-19T16:58:45Z

link repair

← Older revision		Revision as of 16:58, 19 February 2016
Line 26:		Line 26:
	<td>Products available to speedily reorganize even the largest Model 204 files. In addition, some of the limitations in other reorganization methods (invisible fields, procedures and Large Objects for example) are handled seamlessly.</td>		<td>Products available to speedily reorganize even the largest Model 204 files. In addition, some of the limitations in other reorganization methods (invisible fields, procedures and Large Objects for example) are handled seamlessly.</td>
	<td>[[Fast/Unload]]		<td>[[Fast/Unload]]
	<p>[[~~Media:FrelrNew.pdf\|~~Fast/Reload]]</p></td>		<p>[[Fast/Reload]]</p></td>
	</tr>		</tr>

JAL: link repair

2016-01-22T19:23:05Z

link repair

← Older revision		Revision as of 19:23, 22 January 2016
Line 24:		Line 24:
	<tr>		<tr>
	<td>Fast/Unload and Fast/Reload</td>		<td>Fast/Unload and Fast/Reload</td>
	<td>Products available to speedily reorganize even the largest Model 204 files. In addition, some of the limitations in other reorganization methods (invisible fields, procedures and Large Objects for example) are handled seamlessly</td>		<td>Products available to speedily reorganize even the largest Model 204 files. In addition, some of the limitations in other reorganization methods (invisible fields, procedures and Large Objects for example) are handled seamlessly.</td>
	<td>[~~http://m204wiki.rocketsoftware.com/images/e/e2/FunrNew.pdf~~ Fast/Unload]		<td>[[Fast/Unload]]
	<p>[~~http~~:~~//m204wiki.rocketsoftware.com/images/9/95/~~FrelrNew.pdf Fast/Reload]</p></td>		<p>[[Media:FrelrNew.pdf\|Fast/Reload]]</p></td>
	</tr>		</tr>

	<tr>		<tr>
	<td>Fixed or variable unload and reload</td>		<td>Fixed or variable unload and reload</td>
	<td>Requires the data to be unloaded into a structure and then reloaded</td>		<td>Requires the data to be unloaded into a structure and then reloaded.</td>
	<td>[[#Techniques for reorganization\|Techniques for reorganization]]</td>		<td>[[#Techniques for reorganization\|Techniques for reorganization]]</td>
	</tr>		</tr>
Line 37:		Line 37:
	<tr>		<tr>
	<td>[[REORGANIZE command\|REORGANIZE OI]]:<br>Reorganizing just the Ordered Index</td>		<td>[[REORGANIZE command\|REORGANIZE OI]]:<br>Reorganizing just the Ordered Index</td>
	<td>Improve the file's performance by cleansing the Ordered Index</td>		<td>Improve the file's performance by cleansing the Ordered Index.</td>
	<td>[[#Reorganizing the Ordered Index\|Reorganizing the Ordered Index]]</td>		<td>[[#Reorganizing the Ordered Index\|Reorganizing the Ordered Index]]</td>
	</tr>		</tr>
Line 43:		Line 43:
	<tr>		<tr>
	<td nowrap>[[COMPACTB command\|Compacting Tables B and X]]</td>		<td nowrap>[[COMPACTB command\|Compacting Tables B and X]]</td>
	<td>Combines extension records to improve performance. Optionally cleans up logically deleted records</td>		<td>Combines extension records to improve performance. Optionally cleans up logically deleted records.</td>
	<td>[[#Dynamic data compactor for Table B\|Dynamic data compactor for Table B]]</td>		<td>[[#Dynamic data compactor for Table B\|Dynamic data compactor for Table B]]</td>
	</tr>		</tr>
Line 49:		Line 49:
	<tr>		<tr>
	<td>[[COMPACTE command\|Compacting Table E]]</td>		<td>[[COMPACTE command\|Compacting Table E]]</td>
	<td>For [[Table E and non-FILEORG X'100' files (File architecture)\|files without the FILEORG X'100' bit]] set, defragments Table E to create larger contiguous areas of free pages</td>		<td>For [[Table E and non-FILEORG X'100' files (File architecture)\|files without the FILEORG X'100' bit]] set, defragments Table E to create larger contiguous areas of free pages.</td>
	<td>[[#Dynamic data compactor for Table E\|Dynamic data compactor for Table E]]</td>		<td>[[#Dynamic data compactor for Table E\|Dynamic data compactor for Table E]]</td>
	</tr>		</tr>
Line 83:		Line 83:
	Some of these conditions are described in the following sections.</p>		Some of these conditions are described in the following sections.</p>
	<p>		<p>
	To a very large extent, deciding on which (and when) files to reorganize depends on either of these:		To a very large extent, deciding on which (and when) files to reorganize depends on either of these: </p>
	<ul>		<ul>
	<li>How the files are updated: how often records are added and deleted; and how often data is added to (or deleted from) records. </li>		<li>How the files are updated: how often records are added and deleted; and how often data is added to (or deleted from) records. </li>
Line 98:		Line 98:
	====Monitoring Table A and Table C====		====Monitoring Table A and Table C====
	<p>		<p>
	To avoid filling Table A or Table C, regularly monitor table page retries (ARETRIES and CRETRIES, respectively), more often in highly volatile databases. If the number of page retries shows a noticeable increase from one run to another, the file might be approaching a table full condition and might require reorganization. The <var>[[TABLEC command\|TABLEC]]</var> command monitors the use of Table C.</p>		To avoid filling Table A or Table C, regularly monitor table page retries (<var>ARETRIES</var> and <var>CRETRIES</var> parameters, respectively), more often in highly volatile databases. If the number of page retries shows a noticeable increase from one run to another, the file might be approaching a table full condition and might require reorganization. The <var>[[TABLEC command\|TABLEC]]</var> command monitors the use of Table C.</p>

	====Monitoring hash key files====		====Monitoring hash key files====

JAL: link repair

2015-07-08T20:18:42Z

link repair

← Older revision		Revision as of 20:18, 8 July 2015
Line 797:		Line 797:
	===Rebuilding the Ordered Index with the Z command===		===Rebuilding the Ordered Index with the Z command===
	<p>		<p>
	To rebuild the Ordered Index, issue the <var>Z</var> command. If the settings of the [[File architecture#Ordered Index spacing parameters\|spacing parameters]] <var>LRESERVE</var>, <var>NRESERVE</var>, and <var>IMMED</var> are changed for any field before the <var>Z</var> command is issued, the Ordered Index is rebuilt using the new parameter values for that field. The new parameter values result in a different space utilization. </p>		To rebuild the Ordered Index, issue the <var>Z</var> command. If the settings of the [[Table D (File architecture)#Ordered Index spacing parameters\|spacing parameters]] <var>LRESERVE</var>, <var>NRESERVE</var>, and <var>IMMED</var> are changed for any field before the <var>Z</var> command is issued, the Ordered Index is rebuilt using the new parameter values for that field. The new parameter values result in a different space utilization. </p>
	<p>		<p>
	The data sets used by the <var>Z</var> command with <var>REORGANIZE</var> are the same as in ordinary File Load or deferred update runs:</p>		The data sets used by the <var>Z</var> command with <var>REORGANIZE</var> are the same as in ordinary File Load or deferred update runs:</p>

JAL: link repair

2015-01-09T19:49:59Z

link repair

← Older revision		Revision as of 19:49, 9 January 2015
Line 797:		Line 797:
	===Rebuilding the Ordered Index with the Z command===		===Rebuilding the Ordered Index with the Z command===
	<p>		<p>
	To rebuild the Ordered Index, issue the <var>Z</var> command. If the settings of the spacing parameters <var>~~[[LRESERVE parameter\|~~LRESERVE]]</var>, <var>~~[[NRESERVE parameter\|~~NRESERVE]]</var>, and <var>~~[[IMMED parameter\|~~IMMED]]</var> are changed for any field before the <var>Z</var> command is issued, the Ordered Index is rebuilt using the new parameter values for that field. The new parameter values result in a different space utilization. </p>		To rebuild the Ordered Index, issue the <var>Z</var> command. If the settings of the [[File architecture#Ordered Index spacing parameters\|spacing parameters]] <var>LRESERVE</var>, <var>NRESERVE</var>, and <var>IMMED</var> are changed for any field before the <var>Z</var> command is issued, the Ordered Index is rebuilt using the new parameter values for that field. The new parameter values result in a different space utilization. </p>
	<p>		<p>
	The data sets used by the <var>Z</var> command with <var>REORGANIZE</var> are the same as in ordinary File Load or deferred update runs:</p>		The data sets used by the <var>Z</var> command with <var>REORGANIZE</var> are the same as in ordinary File Load or deferred update runs:</p>

JAL: more conversion cleanup

2014-12-19T22:37:31Z

more conversion cleanup

Show changes

JAL: more conversion cleanup

2014-12-17T01:26:10Z

more conversion cleanup

← Older revision		Revision as of 01:26, 17 December 2014
Line 266:		Line 266:
	====DCB information====		====DCB information====
	<p>		<p>
	OUTTAPE is the name of the DD statement in the JCL of the job:</p>		<code>OUTTAPE</code> is the name of the DD statement in the JCL of the job:</p>
	<p class="code">//OUTTAPE DD DSN=COPY,DISP=(,KEEP),UNIT=TAPE,		<p class="code">//OUTTAPE DD DSN=COPY,DISP=(,KEEP),UNIT=TAPE,
	// DCB=(RECFM=VB,LRECL=260,		// DCB=(RECFM=VB,LRECL=260,
Line 299:		Line 299:
	====Parameter settings for the sequential data set====		====Parameter settings for the sequential data set====
	<p>		<p>
	As shown in the layout above, the field name begins in column 5, and spaces are required before and after the equal sign. If the field name=value portion of any line exceeds the value of the LOBUFF parameter, reset LOBUFF in the EXEC statement, and set LRECL in the OUTTAPE DD statement to LOBUFF plus 4. </p>		As shown in the layout above, the field name begins in column 5, and spaces are required before and after the equal sign. If the field name=value portion of any line exceeds the value of the <var>[[LOBUFF parameter\|LOBUFF]]</var> parameter, reset <var>LOBUFF</var> in the EXEC statement, and set <var>LRECL</var> in the OUTTAPE DD statement to: <var>LOBUFF</var> plus 4. </p>
	<p>		<p>
	The default values of 1,000,000 for the MUDD, MOUT, and MDKRD parameters might need to be adjusted upward for PAI dumps on large files.</p>		The default values of 1,000,000 for the <var>[[MUDD parameter\|MUDD]]</var>, <var>[[MOUT parameter\|MOUT]]</var>, and <var>[[MDKRD parameter\|MDKRD]]</var> parameters might need to be adjusted upward for <var>PAI</var> dumps on large files.</p>

	====Reloading files using the File Load utility====		====Reloading files using the File Load utility====
	<p>		<p>
	For step 4 of [[#Techniques for reorganization\|Techniques for reorganization]], the COPY data set is input to a file load program that uses the D statement; see the description of the File Load utility. The following sequence reloads the records from COPY.</p>		For step 4 in [[#Techniques for reorganization\|Techniques for reorganization]], the <code>COPY</code> data set is input to a file load program that uses the <var>D</var> statement (see the description of the [[File Load utility]]). The following sequence reloads the records from <code>COPY</code>.</p>
	<p class="code">CREATE FILE PERSONEL		<p class="code">CREATE FILE PERSONEL
	parameters		parameters
Line 343:		Line 343:
	</p>		</p>
	<p>		<p>
	The line after label 24 starts a new record. With minor modifications to this generalized file load program you can reorganize different <var class="product">Model 204</var> files.</p>		The line after label 24 starts a new record. With minor modifications to this generalized file load program, you can reorganize different <var class="product">Model 204</var> files.</p>

	===Reorganizing or loading a file that contains Large Objects===		===Reorganizing or loading a file that contains Large Objects===
	<p>		<p>
	A <var class="product">Model 204</var> file that contains Large Object data can be reorganized as described in [[#Using PAI FLOD for files with varying record formats\|Using PAI FLOD for files with varying record formats]]~~, which uses the User Language~~ PAI (PRINT ALL INFORMATION) statement ~~to unload~~ the data from a file, and a standard FLOD program ~~to reload~~ the data. </p>		A <var class="product">Model 204</var> file that contains Large Object data can be reorganized as described below in [[#Using PAI FLOD for files with varying record formats\|Using PAI FLOD for files with varying record formats]]. The SOUL <var>PAI</var> (<var>PRINT ALL INFORMATION</var>) statement unloads the data from a file, and a standard <var>FLOD</var> program reloads the data. </p>
	<table>		<table>
	<tr class="head">		<tr class="head">
Line 355:		Line 355:

	<tr>		<tr>
	<td>Non-LOB field </td>		<td><p>Non-LOB field </p></td>
	<td>		<td>
	<p class="codeInTable">FILELDNAME=FIELDVALUE		<p class="codeInTable">FILELDNAME=FIELDVALUE
Line 362:		Line 362:

	<tr>		<tr>
	<td>LOB field </td>		<td><p>LOB field </p></td>
	<td>		<td>
	<p class="codeInTable">FIELDNAME=DESCRIPTOR		<p class="codeInTable">FIELDNAME=DESCRIPTOR
Line 372:		Line 372:
	</tr>		</tr>
	</table>		</table>
	<p>For a LOB field, the data value is replaced by a LOB descriptor, which contains information such as the LOB length and RESERVE value. The LOB descriptor contains unprintable characters. The actual LOB value then follows on subsequent records; as many as are required to hold the LOB value-depending on the OUTMRL of the output data set to which the PAI output is directed.</p>
	<p>		<p>
	~~When a FLOD runs a statement to load~~ a field ~~value~~, ~~it checks whether~~ the ~~field~~ is a LOB, and ~~if so processing assumes that the data is formatted~~ as ~~illustrated in~~ the ~~previous table-namely~~, the ~~FIELDVALUE~~ is ~~actually a descriptor-and loads it accordingly~~.</p>		For a LOB field (<var>BLOB</var> or <var>CLOB</var>), the data value is replaced by a LOB descriptor, which contains information such as the LOB length and <var>RESERVE</var> value. The LOB descriptor contains unprintable characters. The actual LOB value then follows on subsequent records (as many as are required to hold the LOB value, depending on the <var>[[OUTMRL parameter\|OUTMRL]]</var> of the output data set to which the <var>PAI</var> output is directed).</p>
	<p>		<p>
	~~The descriptor format is normally of no concern~~ to the ~~file manager or FLOD programmer~~, as the ~~descriptor~~ is ~~built by~~ the ~~PAI statement that unloaded~~ the ~~data~~. </p>		When a <var>FLOD</var> runs a statement to load a field value, it checks whether the field is a LOB, and if so processing assumes that the data is formatted as illustrated in the previous table — namely, the FIELDVALUE is actually a descriptor — and loads it accordingly.</p>
	<p>		<p>
	If you need to build the LOB descriptor yourself, for example, for an initial load of a file that contains Large Object data, see "Building a Large Object descriptor".</p>		The descriptor format is normally of no concern to the file manager or <var>FLOD</var> programmer, as the descriptor is built by the <var>PAI</var> statement that unloaded the data. </p>
			<p>
			If you need to build the LOB descriptor yourself, for example, for an initial load of a file that contains Large Object data, see [[File Load utility#"Building a Large Object descriptor\|Building a Large Object descriptor]].</p>

	==Reorganizing sorted and hash key files==		==Reorganizing sorted and hash key files==

JAL: more conversion cleanup

2014-12-16T23:15:08Z

more conversion cleanup

Show changes

ELowell at 16:54, 12 May 2014

2014-05-12T16:54:57Z

← Older revision		Revision as of 16:54, 12 May 2014
Line 752:		Line 752:
	You might want to modify the spacing parameters of one or more of the ORDERED fields in the Ordered Index in response to or in anticipation of a change in the nature of the data being stored. (For example, you might anticipate a different pattern or frequency of the updates to the Ordered Index.) </p>		You might want to modify the spacing parameters of one or more of the ORDERED fields in the Ordered Index in response to or in anticipation of a change in the nature of the data being stored. (For example, you might anticipate a different pattern or frequency of the updates to the Ordered Index.) </p>
	<p>		<p>
	If you need to change the spacing parameters of one or more of the ORDERED fields for future updating, use the REDEFINE command with the field(s) involved. See the discussion of ORDERED fields in [[Field design ~~(File management)~~#ORDERED and NON-ORDERED attributes\|ORDERED and NON-ORDERED attributes]], and the REDEFINE command discussion in [[Defining fields manually#Defining field retrieval attributes\|Defining field retrieval attributes]], for more information about the Ordered Index B-tree structure and alteration. </p>		If you need to change the spacing parameters of one or more of the ORDERED fields for future updating, use the REDEFINE command with the field(s) involved. See the discussion of ORDERED fields in [[Field design#ORDERED and NON-ORDERED attributes\|ORDERED and NON-ORDERED attributes]], and the REDEFINE command discussion in [[Defining fields manually#Defining field retrieval attributes\|Defining field retrieval attributes]], for more information about the Ordered Index B-tree structure and alteration. </p>

	===REORGANIZE OI command===		===REORGANIZE OI command===

DmeWiccan: Automatically generated page update

2014-05-09T22:45:47Z

Automatically generated page update

← Older revision		Revision as of 22:45, 9 May 2014
Line 958:		Line 958:
	</div> <!-- end div for toc limit -->		</div> <!-- end div for toc limit -->

	[[Category:~~File management~~]]		[[Category:Model 204 files]]
	[[Category:File loading and reorganization]]		[[Category:File loading and reorganization]]