COMPACTB command: Difference between revisions

From m204wiki
Jump to navigation Jump to search
m (Automatically generated page update)
 
 
(19 intermediate revisions by 5 users not shown)
Line 4: Line 4:
<dd>File Manager
<dd>File Manager
<dt>Function
<dt>Function
<dd>Combine as many extension records as possible into one extension record for the current or specified file.
<dd>Combine as many [[Record_(File_architecture)#Extension_records|extension records]] as possible into one extension record for the current or specified file.
<p class="note"><b>Note:</b>
An automatic optimization occurs if a [[Record_(File_architecture)#Base_records|base record]] has only one extension record, and the combined base plus extension record will fit on the page currently occupied by the base record. In that case, the result of compaction is a single base record and the elimination of the extension.</p>
</dl>
</dl>
==Syntax==
==Syntax==
<p class="syntax">COMPACTB [FROM ssss][TO eeee][FREE nn][MAXE nn] [DELETE]
<p class="syntax">COMPACTB [FROM <span class="term">ssss</span>][TO <span class="term">eeee</span>][FREE <span class="term">nn</span>][MAXE <span class="term">nn</span>] [DELETE]
</p>
</p>
<p>Where </p>
<p>
<p><var>FROM ssss</var> indicates the starting Table B page number where the compactor starts looking for extensions. The default is zero.</p>
Where: </p>
<p><var>TO eeee</var> indicates the ending logical record number where the compactor stops working.</p>
<ul>
<p><var>FREE nn</var> indicates the percentage of unused or free pages (Table B or Table X) that may be used by the data compactor for new extension records. The default is 10 percent. The percent of free pages is calculated as follows: </p>
<li><var>FROM</var> <var class="term">ssss</var> indicates the starting [[Table_B_(File_architecture)|Table B]] page number where the compactor starts looking for extensions. The default is zero.</li>
 
<li><var>TO</var> <var class="term">eeee</var> indicates the ending logical record number where the compactor stops working.</li>
 
<li><var>FREE</var> <var class="term">nn</var> indicates the percentage of unused or free pages (Table B or [[Table_X_(File_architecture)|Table X]]) that can be used by the data compactor for new extension records. The default is 10 percent. The percent of free pages is calculated as follows:  
 
<ul>  
<ul>  
<li>
<li>Table B: <code>((<var>[[BSIZE_parameter|BSIZE]]</var> - <var>[[BHIGHPG_parameter|BHIGHPG]]</var>) / BSIZE) * 100</code> </li>
<p>Table B: ((BSIZE - BHIGHPG) / BSIZE) * 100 </p>
</li>
   
   
<li>Table X ((XSIZE - XHIGHPG / XSIZE) * 100</li>
<li>Table X: <code>((<var>[[XSIZE_parameter|XSIZE]]</var> - <var>[[XHIGHPG_parameter|XHIGHPG]]</var> / XSIZE) * 100</code> </li>
</ul>
</ul></li>
<p><var>MAXE nn</var> specifies the percentage of a page size (6144 bytes) that defines the maximum extension record size eligible for compaction. Larger extensions are not moved. The default is 80 percent.</p>
 
<p><var>DELETE</var> specifies to physically delete logically deleted records. The default behavior is to not delete the logically deleted records.</p>
<li><var>MAXE</var> <var class="term">nn</var> specifies the percentage of a page size (6144 bytes) that defines the maximum extension record size eligible for compaction. Larger extensions are not moved. The default is 80 percent.</li>
 
<li><var>DELETE</var> specifies to physically delete logically deleted records. The default behavior is to not delete the logically deleted records.</li>
</ul>
 
==Usage==
==Usage==
<p>The data compactor is a CPU and I/O intensive process; Rocket recommends that you do not compact data at your site during high load periods. To avoid monopolizing system resources, the data compactor checks for bump or long request conditions at the following intervals:</p>
<ul>
<li>The data compactor is a CPU- and I/O-intensive process, so do not compact data at your site during high load periods. To avoid monopolizing system resources, the data compactor checks for a <var>[[BUMP_command|BUMP]]</var> command or for long request conditions at the following intervals:
<ul>  
<ul>  
<li>
<li>When compacting files with Table X, the check for a bump or for long request conditions is done  
<p>When compacting files with Table X, the check for bump or long request conditions is done for every:</p>
when one of the following milestones (whichever comes first) is reached:
<ul>  
<ul>  
<li>
<li>30 compacted records</li>
<p>30 compacted records, or</p>
</li>
   
   
<li>
<li>30 processed Table B pages</li>
<p>30 processed Table B pages, or</p>
</li>
   
   
<li>
<li>30 logically deleted records</li>
<p>30 logically deleted records,</p>
After the check, the counter is reset and processing continues until the next time one of the milestones is reached; and so on.</ul></li>
</li>
</ul>
whichever comes first.  </li>
   
   
<li>
<li>When compacting files with no Table X, <var>COMPACTB</var> allows higher priority users to run at every 30 records read, whether the records are compacted or not. </li>
<p>When compacting files with no Table X, COMPACTB allows higher priority users to run at every 30 records read, regardless of whether the records are compacted or not. </p>
</ul></li>
</li>
 
</ul>
<li>The data compactor commits at every record compacted.  
<p> The data compactor commits at every record compacted.</p>
<p>
<p>If your system crashes, only one record compaction may be lost. Data is never lost. The worst that may happen when the system crashes is that after recovery processing, one record-and only one-will have either the:</p>
If your system crashes, only one record compaction might be lost. Data is never lost. The worst that might happen when the system crashes is that after recovery processing, one record, and only one, would have either of these:</p>
<ul>  
<ul>  
<li>
<li>The old extension chain with the new extension chain being orphaned.</li>
<p>Old extension chain with the new extension chain being orphaned, or</p>
</li>
   
   
<li>
<li>The new extension chain with the old extension chain being orphaned.</li>
<p>New extension chain with the old extension chain being orphaned</p>
</ul>
<p>
You cannot reclaim space for orphaned chains without a file reorganization. There is no backout for the data compactor.</p>
</li>
</li>
</ul>
 
<p>You cannot reclaim space for orphaned chains without a file reorganization. There is no back out for the data compactor.</p>
<li>The compactor takes preimages of all the pages it will change and writes a journal record, containing all compacted extensions for each compaction, which might require that you increase the checkpoint and journal data set sizes.</li>
<p>The compactor takes preimages of all the pages it will change and writes a journal record, containing all compacted extensions for each compaction, which may require that you increase the checkpoint and journal data set sizes.</p>
 
<p>There is no restriction on record length or number of extension records for the data compactor, as long as there is enough free space on pages to hold the compacted records.</p>
<li>There is no restriction on record length or number of extension records for the data compactor, as long as there is enough free space on pages to hold the compacted records.</li>
<p>The COMPACTB command maintains the same logical order-that is visible to programs-of extension records in the compacted records as in the original record. Their physical order-that is, the order of the page numbers-may differ.</p>
 
<li>The <var>COMPACTB</var> command maintains the same logical order (which is visible to programs) of extension records in the compacted records as in the original record. The physical order of extension records (that is, the order of the page numbers) might differ.</li>
</ul>
 
====Setting the DELETE argument====
====Setting the DELETE argument====
<p>To use the DELETE argument of the COMPACTB command for a file, that file must already have a Table X defined for it-XSIZE is greater than zero in the file CREATE arguments.</p>
<p>
To use the <var>DELETE</var> argument of the <var>COMPACTB</var> command for a file, that file must already have a Table X defined for it: <var>XSIZE</var> is greater than zero in the file <var>CREATE</var> arguments.</p>
 
====Setting the MAXE argument====
====Setting the MAXE argument====
<p>The larger an extension, the less likely that it will be combined with other extensions, because the largest single extension record may not be larger than a page size. When extension chains are very long and contain mainly very short extensions, a smaller MAXE setting may produce better results more quickly. You might want to test various settings to find what percent is most advantageous for your data.</p>
<p>
The larger an extension, the less likely that it will be combined with other extensions, because the largest single extension record cannot be larger than a page size. When extension chains are very long and contain mainly very short extensions, a smaller <var>MAXE</var> setting might produce better results more quickly. You might want to test various <var>MAXE</var> settings to find which page percentage size is most advantageous for your data.</p>
 
====Running the data compactor====
====Running the data compactor====
<p>The data compactor tries to lock records on a one-at-a-time basis. Records with extensions that are subject to compaction are locked exclusively as long as compaction for the record takes place. If record lock is not available, then the record is skipped. Avoid running the data compactor with applications that lock large numbers of records for a long time. </p>
<p>
The data compactor tries to lock records on a one-at-a-time basis. Records with extensions that are subject to compaction are locked exclusively as long as compaction for the record takes place. If record lock is not available, then the record is skipped. Avoid running the data compactor with applications that lock large numbers of records for a long time. </p>
 
====Reviewing data compactor statistics====
====Reviewing data compactor statistics====
<p>At the end of processing, the compactor prints the following statistics:</p>
<p>
<p class="code">M204.2749: NUMBER OF BASIC RECORDS PROCESSED:              nnnn
At the end of processing, the compactor prints the following statistics:</p>
M204.2750: NUMBER OF EXTENSION RECORDS BEFORE COMPACRTION: nnnn
<p class="code">M204.2749: NUMBER OF BASIC RECORDS PROCESSED:              <var class="term">nnnn</var>
M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION:    nnnn
M204.2750: NUMBER OF EXTENSION RECORDS BEFORE COMPACTION:   <var class="term">nnnn</var>
M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS:        nnnn
M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION:    <var class="term">nnnn</var>
M204.2753: NUMBER OF FREE PAGES USED:                      nnnn
M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS:        <var class="term">nnnn</var>
M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS:    nnnn
M204.2753: NUMBER OF FREE PAGES USED:                      <var class="term">nnnn</var>
M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS:    <var class="term">nnnn</var>
</p>
</p>
==Example: file statistic changes==
==Example: file statistic changes==
<p class="code">><b>IN TEST 1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN</b>
<p class="code"><b>> IN TEST 1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN</b>
BHIGHPG  39          TABLE B HIGHEST ACTIVE PAGE
BHIGHPG  39          TABLE B HIGHEST ACTIVE PAGE
EXTNADD  42          EXTENSION RECORDS ADDED
EXTNADD  42          EXTENSION RECORDS ADDED
EXTNDEL  15          EXTENSION RECORDS DELETED
EXTNDEL  15          EXTENSION RECORDS DELETED
BQLEN    2          TABLE B QUEUE LENGTH
BQLEN    2          TABLE B QUEUE LENGTH
  IN TEST1 COMPACTB FROM 0 TO 9999999 FREE 100 MAXE 100
<b>> IN TEST1 COMPACTB FROM 0 TO 9999999 FREE 100 MAXE 100</b>
NUMBER OF BASIC RECORDS PROCESSED:            20
NUMBER OF BASIC RECORDS PROCESSED:            20
NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: 27
NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: 27
Line 88: Line 103:
NUMBER OF NOT PROCESSED (LOCKED) RECORDS:      0
NUMBER OF NOT PROCESSED (LOCKED) RECORDS:      0
NUMBER OF FREE PAGES USED:                    2
NUMBER OF FREE PAGES USED:                    2
  IN TEST1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN
<b>> IN TEST1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN</b>
BHIGHPG  41          TABLE B HIGHEST ACTIVE PAGE
BHIGHPG  41          TABLE B HIGHEST ACTIVE PAGE
EXTNADD  49          EXTENSION RECORDS ADDED
EXTNADD  49          EXTENSION RECORDS ADDED
Line 94: Line 109:
BQLEN    5          TABLE B QUEUE LENGTH
BQLEN    5          TABLE B QUEUE LENGTH
</p>
</p>
<p>At the end of processing, the compactor prints the following statistics: </p>
<p>
<p class="code">M204.2749: NUMBER OF BASIC RECORDS PROCESSED:              nnnn
The value of the <var>[[BQLEN_parameter|BQLEN]]</var> parameter changed. When the original extensions were deleted, a page became suitable for reuse and was placed on the Reuse Queue. In this example unused pages were used for new extensions. If new extensions are placed on a page from the Reuse Queue, <var>BQLEN</var> might decrease.</p>
M204.2750: NUMBER OF EXTENSION RECORDS BEFORE COMPACTION:  nnnn
 
M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION:    nnnn
M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS:        nnnn
M204.2753: NUMBER OF FREE PAGES USED:                      nnnn
M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS:    nnnn
</p>
<p>The value of BQLEN parameter also changed. When the original extensions were deleted, a page became suitable for reuse and was placed on the Reuse Queue. In this example unused pages were used for new extensions, if new extensions are placed on a page from the Reuse Queue, BQLEN may decrease.</p>
==Example: compacting extensions==
==Example: compacting extensions==
<p>A single record has nine extension records with the following length. The lengths of the extensions in this example were chosen arbitrarily to illustrate how and when extensions are combined, or not combined.</p>
<p>
<table>
A single record has nine extension records with the following length. The lengths of the extensions in this example were chosen arbitrarily to illustrate how and when extensions are combined or not combined.</p>
   
<table>  
<tr> <th><var>Ext.1</var></th> <td>Ext.2</td> <td>Ext.3</td> <td>Ext.4</td> <td>Ext.5</td> <td>Ext.6</td> <td>Ext.7</td> <td>Ext.8</td> <td>Ext.9</td> </tr>
<tr class="head">  
<th>Ext.1</th> <th>Ext.2</th> <th>Ext.3</th> <th>Ext.4</th> <th>Ext.5</th> <th>Ext.6</th> <th>Ext.7</th> <th>Ext.8</th> <th>Ext.9</th> </tr>
   
   
<tr> <th align="right"><var>40</var></th> <td>1200</td> <td>2400</td> <td>3200</td> <td>4300</td> <td>2300</td> <td>60</td> <td>90</td> <td>120</td> </tr>
<tr>
 
<td>40</td> <td>1200</td> <td>2400</td> <td>3200</td> <td>4300</td> <td>2300</td> <td>60</td> <td>90</td> <td>120</td> </tr>  
</table>
</table>
<p>The chain is reduced to four parts:</p>
<p>
<p>Part 1: Extensions 1,2, and 3 have a combined length of 3640 that compacts into one extension record.</p>
The chain is reduced to four parts:</p>
<p>Part 2: Extension 4 is not moved, because combined with previous extensions it will not fit the page.</p>
<ul>
<p>Part 3: Extension 5 is not moved, because it cannot be combined with either Extension 4 or Extension 6.</p>
<li>Part 1: Extensions 1, 2, and 3 have a combined length of 3640 that compacts into one extension record.</li>
<p>Part 4: Extensions 6,7,8, and 9 have combined length of 2570 that compacts into one extension record.</p>
 
<p>After compacting, there are four extensions instead of nine. </p>
<li>Part 2: Extension 4 is not moved, because combined with previous extensions it will not fit the page.</li>
 
<li>Part 3: Extension 5 is not moved, because it cannot be combined with either Extension 4 or Extension 6.</li>
 
<li>Part 4: Extensions 6, 7, 8, and 9 have a combined length of 2570 that compacts into one extension record.</li>
</ul>
<p>
After compacting, there are four extensions instead of nine. </p>
 
[[Category:Commands]]
[[Category:Commands]]

Latest revision as of 12:50, 11 February 2019

Summary

Privileges
File Manager
Function
Combine as many extension records as possible into one extension record for the current or specified file.

Note: An automatic optimization occurs if a base record has only one extension record, and the combined base plus extension record will fit on the page currently occupied by the base record. In that case, the result of compaction is a single base record and the elimination of the extension.

Syntax

COMPACTB [FROM ssss][TO eeee][FREE nn][MAXE nn] [DELETE]

Where:

  • FROM ssss indicates the starting Table B page number where the compactor starts looking for extensions. The default is zero.
  • TO eeee indicates the ending logical record number where the compactor stops working.
  • FREE nn indicates the percentage of unused or free pages (Table B or Table X) that can be used by the data compactor for new extension records. The default is 10 percent. The percent of free pages is calculated as follows:
  • MAXE nn specifies the percentage of a page size (6144 bytes) that defines the maximum extension record size eligible for compaction. Larger extensions are not moved. The default is 80 percent.
  • DELETE specifies to physically delete logically deleted records. The default behavior is to not delete the logically deleted records.

Usage

  • The data compactor is a CPU- and I/O-intensive process, so do not compact data at your site during high load periods. To avoid monopolizing system resources, the data compactor checks for a BUMP command or for long request conditions at the following intervals:
    • When compacting files with Table X, the check for a bump or for long request conditions is done when one of the following milestones (whichever comes first) is reached:
      • 30 compacted records
      • 30 processed Table B pages
      • 30 logically deleted records
      • After the check, the counter is reset and processing continues until the next time one of the milestones is reached; and so on.
    • When compacting files with no Table X, COMPACTB allows higher priority users to run at every 30 records read, whether the records are compacted or not.
  • The data compactor commits at every record compacted.

    If your system crashes, only one record compaction might be lost. Data is never lost. The worst that might happen when the system crashes is that after recovery processing, one record, and only one, would have either of these:

    • The old extension chain with the new extension chain being orphaned.
    • The new extension chain with the old extension chain being orphaned.

    You cannot reclaim space for orphaned chains without a file reorganization. There is no backout for the data compactor.

  • The compactor takes preimages of all the pages it will change and writes a journal record, containing all compacted extensions for each compaction, which might require that you increase the checkpoint and journal data set sizes.
  • There is no restriction on record length or number of extension records for the data compactor, as long as there is enough free space on pages to hold the compacted records.
  • The COMPACTB command maintains the same logical order (which is visible to programs) of extension records in the compacted records as in the original record. The physical order of extension records (that is, the order of the page numbers) might differ.

Setting the DELETE argument

To use the DELETE argument of the COMPACTB command for a file, that file must already have a Table X defined for it: XSIZE is greater than zero in the file CREATE arguments.

Setting the MAXE argument

The larger an extension, the less likely that it will be combined with other extensions, because the largest single extension record cannot be larger than a page size. When extension chains are very long and contain mainly very short extensions, a smaller MAXE setting might produce better results more quickly. You might want to test various MAXE settings to find which page percentage size is most advantageous for your data.

Running the data compactor

The data compactor tries to lock records on a one-at-a-time basis. Records with extensions that are subject to compaction are locked exclusively as long as compaction for the record takes place. If record lock is not available, then the record is skipped. Avoid running the data compactor with applications that lock large numbers of records for a long time.

Reviewing data compactor statistics

At the end of processing, the compactor prints the following statistics:

M204.2749: NUMBER OF BASIC RECORDS PROCESSED: nnnn M204.2750: NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: nnnn M204.2751: NUMBER OF EXTENSION RECORDS AFTER COMPACTION: nnnn M204.2752: NUMBER OF NOT PROCESSED (LOCKED) RECORDS: nnnn M204.2753: NUMBER OF FREE PAGES USED: nnnn M204.2754: NUMBER OF DELETED LOCIGALLY DELETED RECORDS: nnnn

Example: file statistic changes

> IN TEST 1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN BHIGHPG 39 TABLE B HIGHEST ACTIVE PAGE EXTNADD 42 EXTENSION RECORDS ADDED EXTNDEL 15 EXTENSION RECORDS DELETED BQLEN 2 TABLE B QUEUE LENGTH > IN TEST1 COMPACTB FROM 0 TO 9999999 FREE 100 MAXE 100 NUMBER OF BASIC RECORDS PROCESSED: 20 NUMBER OF EXTENSION RECORDS BEFORE COMPACTION: 27 NUMBER OF EXTENSION RECORDS AFTER COMPACTION: 20 NUMBER OF NOT PROCESSED (LOCKED) RECORDS: 0 NUMBER OF FREE PAGES USED: 2 > IN TEST1 VIEW BHIGHPG,EXTNADD,EXTNDEL,BQLEN BHIGHPG 41 TABLE B HIGHEST ACTIVE PAGE EXTNADD 49 EXTENSION RECORDS ADDED EXTNDEL 29 EXTENSION RECORDS DELETED BQLEN 5 TABLE B QUEUE LENGTH

The value of the BQLEN parameter changed. When the original extensions were deleted, a page became suitable for reuse and was placed on the Reuse Queue. In this example unused pages were used for new extensions. If new extensions are placed on a page from the Reuse Queue, BQLEN might decrease.

Example: compacting extensions

A single record has nine extension records with the following length. The lengths of the extensions in this example were chosen arbitrarily to illustrate how and when extensions are combined or not combined.

Ext.1 Ext.2 Ext.3 Ext.4 Ext.5 Ext.6 Ext.7 Ext.8 Ext.9
40 1200 2400 3200 4300 2300 60 90 120

The chain is reduced to four parts:

  • Part 1: Extensions 1, 2, and 3 have a combined length of 3640 that compacts into one extension record.
  • Part 2: Extension 4 is not moved, because combined with previous extensions it will not fit the page.
  • Part 3: Extension 5 is not moved, because it cannot be combined with either Extension 4 or Extension 6.
  • Part 4: Extensions 6, 7, 8, and 9 have a combined length of 2570 that compacts into one extension record.

After compacting, there are four extensions instead of nine.