COMPACTB command: Difference between revisions
m (Automatically generated page update) |
|||
(19 intermediate revisions by 5 users not shown) | |||
Line 4: | Line 4: | ||
<dd>File Manager | <dd>File Manager | ||
<dt>Function | <dt>Function | ||
<dd>Combine as many extension records as possible into one extension record for the current or specified file. | <dd>Combine as many [[Record_(File_architecture)#Extension_records|extension records]] as possible into one extension record for the current or specified file. | ||
<p class="note"><b>Note:</b> | |||
An automatic optimization occurs if a [[Record_(File_architecture)#Base_records|base record]] has only one extension record, and the combined base plus extension record will fit on the page currently occupied by the base record. In that case, the result of compaction is a single base record and the elimination of the extension.</p> | |||
</dl> | </dl> | ||
==Syntax== | ==Syntax== | ||
<p class="syntax">COMPACTB [FROM ssss][TO eeee][FREE nn][MAXE nn] [DELETE] | <p class="syntax">COMPACTB [FROM <span class="term">ssss</span>][TO <span class="term">eeee</span>][FREE <span class="term">nn</span>][MAXE <span class="term">nn</span>] [DELETE] | ||
</p> | </p> | ||
<p>Where </p> | <p> | ||
< | Where: </p> | ||
< | <ul> | ||
< | <li><var>FROM</var> <var class="term">ssss</var> indicates the starting [[Table_B_(File_architecture)|Table B]] page number where the compactor starts looking for extensions. The default is zero.</li> | ||
<li><var>TO</var> <var class="term">eeee</var> indicates the ending logical record number where the compactor stops working.</li> | |||
<li><var>FREE</var> <var class="term">nn</var> indicates the percentage of unused or free pages (Table B or [[Table_X_(File_architecture)|Table X]]) that can be used by the data compactor for new extension records. The default is 10 percent. The percent of free pages is calculated as follows: | |||
<ul> | <ul> | ||
<li | <li>Table B: <code>((<var>[[BSIZE_parameter|BSIZE]]</var> - <var>[[BHIGHPG_parameter|BHIGHPG]]</var>) / BSIZE) * 100</code> </li> | ||
</li> | |||
<li>Table X ((XSIZE - XHIGHPG / XSIZE) * 100</li> | <li>Table X: <code>((<var>[[XSIZE_parameter|XSIZE]]</var> - <var>[[XHIGHPG_parameter|XHIGHPG]]</var> / XSIZE) * 100</code> </li> | ||
</ul></li> | |||
< | |||
< | <li><var>MAXE</var> <var class="term">nn</var> specifies the percentage of a page size (6144 bytes) that defines the maximum extension record size eligible for compaction. Larger extensions are not moved. The default is 80 percent.</li> | ||
<li><var>DELETE</var> specifies to physically delete logically deleted records. The default behavior is to not delete the logically deleted records.</li> | |||
</ul> | |||
==Usage== | ==Usage== | ||
< | <ul> | ||
<li>The data compactor is a CPU- and I/O-intensive process, so do not compact data at your site during high load periods. To avoid monopolizing system resources, the data compactor checks for a <var>[[BUMP_command|BUMP]]</var> command or for long request conditions at the following intervals: | |||
<ul> | <ul> | ||
<li | <li>When compacting files with Table X, the check for a bump or for long request conditions is done | ||
when one of the following milestones (whichever comes first) is reached: | |||
<ul> | <ul> | ||
<li | <li>30 compacted records</li> | ||
</li> | |||
<li | <li>30 processed Table B pages</li> | ||
</li> | |||
<li | <li>30 logically deleted records</li> | ||
After the check, the counter is reset and processing continues until the next time one of the milestones is reached; and so on.</ul></li> | |||
</li> | |||
<li | <li>When compacting files with no Table X, <var>COMPACTB</var> allows higher priority users to run at every 30 records read, whether the records are compacted or not. </li> | ||
</ul></li> | |||
</ | |||
<li>The data compactor commits at every record compacted. | |||
< | <p> | ||
If your system crashes, only one record compaction might be lost. Data is never lost. The worst that might happen when the system crashes is that after recovery processing, one record, and only one, would have either of these:</p> | |||
<ul> | <ul> | ||
<li> | <li>The old extension chain with the new extension chain being orphaned.</li> | ||
</li> | |||
<li> | <li>The new extension chain with the old extension chain being orphaned.</li> | ||
</ul> | |||
<p> | |||
You cannot reclaim space for orphaned chains without a file reorganization. There is no backout for the data compactor.</p> | |||
</li> | </li> | ||
< | <li>The compactor takes preimages of all the pages it will change and writes a journal record, containing all compacted extensions for each compaction, which might require that you increase the checkpoint and journal data set sizes.</li> | ||
< | <li>There is no restriction on record length or number of extension records for the data compactor, as long as there is enough free space on pages to hold the compacted records.</li> | ||
< | |||
<li>The <var>COMPACTB</var> command maintains the same logical order (which is visible to programs) of extension records in the compacted records as in the original record. The physical order of extension records (that is, the order of the page numbers) might differ.</li> | |||
</ul> | |||
====Setting the DELETE argument==== | ====Setting the DELETE argument==== | ||
<p>To use the DELETE argument of the COMPACTB command for a file, that file must already have a Table X defined for it | <p> | ||
To use the <var>DELETE</var> argument of the <var>COMPACTB</var> command for a file, that file must already have a Table X defined for it: <var>XSIZE</var> is greater than zero in the file <var>CREATE</var> arguments.</p> | |||
====Setting the MAXE argument==== | ====Setting the MAXE argument==== | ||
<p>The larger an extension, the less likely that it will be combined with other extensions, because the largest single extension record | <p> | ||
The larger an extension, the less likely that it will be combined with other extensions, because the largest single extension record cannot be larger than a page size. When extension chains are very long and contain mainly very short extensions, a smaller <var>MAXE</var> setting might produce better results more quickly. You might want to test various <var>MAXE</var> settings to find which page percentage size is most advantageous for your data.</p> | |||
====Running the data compactor==== | ====Running the data compactor==== | ||
<p>The data compactor tries to lock records on a one-at-a-time basis. Records with extensions that are subject to compaction are locked exclusively as long as compaction for the record takes place. If record lock is not available, then the record is skipped. Avoid running the data compactor with applications that lock large numbers of records for a long time. </p> | <p> | ||
The data compactor tries to lock records on a one-at-a-time basis. Records with extensions that are subject to compaction are locked exclusively as long as compaction for the record takes place. If record lock is not available, then the record is skipped. Avoid running the data compactor with applications that lock large numbers of records for a long time. </p> | |||
====Reviewing data compactor statistics==== | ====Reviewing data compactor statistics==== | ||
<p>At the end of processing, the compactor prints the following statistics:</p> | <p> | ||
<p class="code">M204.2749: NUMBER OF BASIC RECORDS PROCESSED: nnnn | At the end of processing, the compactor prints the following statistics:</p> | ||
M204.2750: NUMBER OF EXTENSION RECORDS BEFORE | <p class="code">M204.2749: NUMBER OF BASIC RECORDS PROCESSED: <var class="term">nnnn</var> | ||
M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION: nnnn | M204.2750: NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: <var class="term">nnnn</var> | ||
M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS: nnnn | M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION: <var class="term">nnnn</var> | ||
M204.2753: NUMBER OF FREE PAGES USED: nnnn | M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS: <var class="term">nnnn</var> | ||
M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS: nnnn | M204.2753: NUMBER OF FREE PAGES USED: <var class="term">nnnn</var> | ||
M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS: <var class="term">nnnn</var> | |||
</p> | </p> | ||
==Example: file statistic changes== | ==Example: file statistic changes== | ||
<p class="code" | <p class="code"><b>> IN TEST 1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN</b> | ||
BHIGHPG 39 TABLE B HIGHEST ACTIVE PAGE | BHIGHPG 39 TABLE B HIGHEST ACTIVE PAGE | ||
EXTNADD 42 EXTENSION RECORDS ADDED | EXTNADD 42 EXTENSION RECORDS ADDED | ||
EXTNDEL 15 EXTENSION RECORDS DELETED | EXTNDEL 15 EXTENSION RECORDS DELETED | ||
BQLEN 2 TABLE B QUEUE LENGTH | BQLEN 2 TABLE B QUEUE LENGTH | ||
<b>> IN TEST1 COMPACTB FROM 0 TO 9999999 FREE 100 MAXE 100</b> | |||
NUMBER OF BASIC RECORDS PROCESSED: 20 | NUMBER OF BASIC RECORDS PROCESSED: 20 | ||
NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: 27 | NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: 27 | ||
Line 88: | Line 103: | ||
NUMBER OF NOT PROCESSED (LOCKED) RECORDS: 0 | NUMBER OF NOT PROCESSED (LOCKED) RECORDS: 0 | ||
NUMBER OF FREE PAGES USED: 2 | NUMBER OF FREE PAGES USED: 2 | ||
<b>> IN TEST1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN</b> | |||
BHIGHPG 41 TABLE B HIGHEST ACTIVE PAGE | BHIGHPG 41 TABLE B HIGHEST ACTIVE PAGE | ||
EXTNADD 49 EXTENSION RECORDS ADDED | EXTNADD 49 EXTENSION RECORDS ADDED | ||
Line 94: | Line 109: | ||
BQLEN 5 TABLE B QUEUE LENGTH | BQLEN 5 TABLE B QUEUE LENGTH | ||
</p> | </p> | ||
<p> | <p> | ||
The value of the <var>[[BQLEN_parameter|BQLEN]]</var> parameter changed. When the original extensions were deleted, a page became suitable for reuse and was placed on the Reuse Queue. In this example unused pages were used for new extensions. If new extensions are placed on a page from the Reuse Queue, <var>BQLEN</var> might decrease.</p> | |||
</ | |||
==Example: compacting extensions== | ==Example: compacting extensions== | ||
<p>A single record has nine extension records with the following length. The lengths of the extensions in this example were chosen arbitrarily to illustrate how and when extensions are combined | <p> | ||
<table> | A single record has nine extension records with the following length. The lengths of the extensions in this example were chosen arbitrarily to illustrate how and when extensions are combined or not combined.</p> | ||
<table> | |||
<tr> <th | <tr class="head"> | ||
<th>Ext.1</th> <th>Ext.2</th> <th>Ext.3</th> <th>Ext.4</th> <th>Ext.5</th> <th>Ext.6</th> <th>Ext.7</th> <th>Ext.8</th> <th>Ext.9</th> </tr> | |||
<tr> < | <tr> | ||
<td>40</td> <td>1200</td> <td>2400</td> <td>3200</td> <td>4300</td> <td>2300</td> <td>60</td> <td>90</td> <td>120</td> </tr> | |||
</table> | </table> | ||
<p>The chain is reduced to four parts:</p> | <p> | ||
< | The chain is reduced to four parts:</p> | ||
< | <ul> | ||
< | <li>Part 1: Extensions 1, 2, and 3 have a combined length of 3640 that compacts into one extension record.</li> | ||
< | |||
<p>After compacting, there are four extensions instead of nine. </p> | <li>Part 2: Extension 4 is not moved, because combined with previous extensions it will not fit the page.</li> | ||
<li>Part 3: Extension 5 is not moved, because it cannot be combined with either Extension 4 or Extension 6.</li> | |||
<li>Part 4: Extensions 6, 7, 8, and 9 have a combined length of 2570 that compacts into one extension record.</li> | |||
</ul> | |||
<p> | |||
After compacting, there are four extensions instead of nine. </p> | |||
[[Category:Commands]] | [[Category:Commands]] |
Latest revision as of 12:50, 11 February 2019
Summary
- Privileges
- File Manager
- Function
- Combine as many extension records as possible into one extension record for the current or specified file.
Note: An automatic optimization occurs if a base record has only one extension record, and the combined base plus extension record will fit on the page currently occupied by the base record. In that case, the result of compaction is a single base record and the elimination of the extension.
Syntax
COMPACTB [FROM ssss][TO eeee][FREE nn][MAXE nn] [DELETE]
Where:
- FROM ssss indicates the starting Table B page number where the compactor starts looking for extensions. The default is zero.
- TO eeee indicates the ending logical record number where the compactor stops working.
- FREE nn indicates the percentage of unused or free pages (Table B or Table X) that can be used by the data compactor for new extension records. The default is 10 percent. The percent of free pages is calculated as follows:
- MAXE nn specifies the percentage of a page size (6144 bytes) that defines the maximum extension record size eligible for compaction. Larger extensions are not moved. The default is 80 percent.
- DELETE specifies to physically delete logically deleted records. The default behavior is to not delete the logically deleted records.
Usage
- The data compactor is a CPU- and I/O-intensive process, so do not compact data at your site during high load periods. To avoid monopolizing system resources, the data compactor checks for a BUMP command or for long request conditions at the following intervals:
- When compacting files with Table X, the check for a bump or for long request conditions is done
when one of the following milestones (whichever comes first) is reached:
- 30 compacted records
- 30 processed Table B pages
- 30 logically deleted records After the check, the counter is reset and processing continues until the next time one of the milestones is reached; and so on.
- When compacting files with no Table X, COMPACTB allows higher priority users to run at every 30 records read, whether the records are compacted or not.
- When compacting files with Table X, the check for a bump or for long request conditions is done
when one of the following milestones (whichever comes first) is reached:
- The data compactor commits at every record compacted.
If your system crashes, only one record compaction might be lost. Data is never lost. The worst that might happen when the system crashes is that after recovery processing, one record, and only one, would have either of these:
- The old extension chain with the new extension chain being orphaned.
- The new extension chain with the old extension chain being orphaned.
You cannot reclaim space for orphaned chains without a file reorganization. There is no backout for the data compactor.
- The compactor takes preimages of all the pages it will change and writes a journal record, containing all compacted extensions for each compaction, which might require that you increase the checkpoint and journal data set sizes.
- There is no restriction on record length or number of extension records for the data compactor, as long as there is enough free space on pages to hold the compacted records.
- The COMPACTB command maintains the same logical order (which is visible to programs) of extension records in the compacted records as in the original record. The physical order of extension records (that is, the order of the page numbers) might differ.
Setting the DELETE argument
To use the DELETE argument of the COMPACTB command for a file, that file must already have a Table X defined for it: XSIZE is greater than zero in the file CREATE arguments.
Setting the MAXE argument
The larger an extension, the less likely that it will be combined with other extensions, because the largest single extension record cannot be larger than a page size. When extension chains are very long and contain mainly very short extensions, a smaller MAXE setting might produce better results more quickly. You might want to test various MAXE settings to find which page percentage size is most advantageous for your data.
Running the data compactor
The data compactor tries to lock records on a one-at-a-time basis. Records with extensions that are subject to compaction are locked exclusively as long as compaction for the record takes place. If record lock is not available, then the record is skipped. Avoid running the data compactor with applications that lock large numbers of records for a long time.
Reviewing data compactor statistics
At the end of processing, the compactor prints the following statistics:
M204.2749: NUMBER OF BASIC RECORDS PROCESSED: nnnn M204.2750: NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: nnnn M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION: nnnn M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS: nnnn M204.2753: NUMBER OF FREE PAGES USED: nnnn M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS: nnnn
Example: file statistic changes
> IN TEST 1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN BHIGHPG 39 TABLE B HIGHEST ACTIVE PAGE EXTNADD 42 EXTENSION RECORDS ADDED EXTNDEL 15 EXTENSION RECORDS DELETED BQLEN 2 TABLE B QUEUE LENGTH > IN TEST1 COMPACTB FROM 0 TO 9999999 FREE 100 MAXE 100 NUMBER OF BASIC RECORDS PROCESSED: 20 NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: 27 NUMBER OF EXTENSION RECORDS AFTER COMPACTION: 20 NUMBER OF NOT PROCESSED (LOCKED) RECORDS: 0 NUMBER OF FREE PAGES USED: 2 > IN TEST1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN BHIGHPG 41 TABLE B HIGHEST ACTIVE PAGE EXTNADD 49 EXTENSION RECORDS ADDED EXTNDEL 29 EXTENSION RECORDS DELETED BQLEN 5 TABLE B QUEUE LENGTH
The value of the BQLEN parameter changed. When the original extensions were deleted, a page became suitable for reuse and was placed on the Reuse Queue. In this example unused pages were used for new extensions. If new extensions are placed on a page from the Reuse Queue, BQLEN might decrease.
Example: compacting extensions
A single record has nine extension records with the following length. The lengths of the extensions in this example were chosen arbitrarily to illustrate how and when extensions are combined or not combined.
Ext.1 | Ext.2 | Ext.3 | Ext.4 | Ext.5 | Ext.6 | Ext.7 | Ext.8 | Ext.9 |
---|---|---|---|---|---|---|---|---|
40 | 1200 | 2400 | 3200 | 4300 | 2300 | 60 | 90 | 120 |
The chain is reduced to four parts:
- Part 1: Extensions 1, 2, and 3 have a combined length of 3640 that compacts into one extension record.
- Part 2: Extension 4 is not moved, because combined with previous extensions it will not fit the page.
- Part 3: Extension 5 is not moved, because it cannot be combined with either Extension 4 or Extension 6.
- Part 4: Extensions 6, 7, 8, and 9 have a combined length of 2570 that compacts into one extension record.
After compacting, there are four extensions instead of nine.