Table B (File architecture): Difference between revisions

From m204wiki
Jump to navigation Jump to search
(replaced graphic and updated free space pointer description, per Don's feedback)
m (→‎Use in indexing: link repair)
 
(30 intermediate revisions by 4 users not shown)
Line 1: Line 1:
This topic covers the internal architecture of a Model 204 Table B page.  
This page describes the internal architecture of a Model 204 Table B page.  


For a discussion of the ways a File Manager may organize these pages in a file, please refer to [[File Design (File Management)]].  
For a discussion of the ways a File Manager can organize these pages in a file, refer to [[File design]].  


At a minimum, Table B contains all of the base records in a Model 204 file. If [[Table X (File Architecture)|Table X]] is not enabled it also contains all extension records. If [[Table E (File Architecture)|Table E]] is not enabled, then any data whose contents is greater than 255 bytes must be held as a series of repeating fields.
At a minimum, Table B contains all of the base records in a Model 204 file.  
<p>Up to Model 204 version 7.4, Table B has a maximum number of 16.7 million record numbers (IRNs for Internal Record Numbers).
As of Model 204 version 7.5, Table B has a maximum number of 48 million record numbers (if the <var>[[FILEORG parameter|FILEORG]]</var> X'200' bit is set). </p>
If [[Table X (File architecture)|Table X]] is <i>not</i> enabled, these "slots" are used for both base records and extension records. If [[Table E (File architecture)|Table E]] is <i>not</i> enabled, any data with contents greater than 255 bytes must be stored as a series of repeating fields.


==Structure of a Table B page==


== The structure of a Table B page ==
:[[File:Table_B_structure.jpg|border|400 px]]


::[[File:Table_B_structure.jpg|border|400 px]]


Not shown is a 4 byte 'Reuse Queue Page Number' for Unordered Files. If it is present, it is in the lower left hand corner ('before' (working backwards) the first record on the page).  
Not shown in the figure above is a 4-byte "Reuse Queue page number" for Unordered files. If it is present, it is in the lower left-hand corner ("before," working backwards, the first record on the page).  


Each item in the graphic is described below.
Each item in the figure is described below.


=== Pointers ===
===Pointers===


<p>'''Free Space Pointer''' </p>
====Free space pointer====
 
<p>
<p>The free space pointer points to the last cell in the currently allocated  pointer array, which in turn points to the beginning of free space on a page, where the next record can begin. (See the above diagram.)
The free space pointer points to the last cell in the currently allocated  pointer array, which in turn points to the beginning of free space on a page, where the next record can begin. (See the above diagram.)
On pages with no records yet stored, the free space pointer points to the last byte on the page (either the very end, of just before the Reuse Queue Page Number described above).</p>   
On pages with no records yet stored, the free space pointer points to the last byte on the page (either the very end, of just before the Reuse Queue Page Number described above).</p>   


<p>'''Record Pointer Array'''</p>
====Record pointer array====
 
<p>
<p>There are ''up to'' [[BRECPPG parameter|BRECPPG]] record pointers. These are not physically put onto the page until the record is being added to the file.</p> 
There are ''as many as'' <var>[[BRECPPG parameter|BRECPPG]]</var> record pointers. These are not physically put onto the page until the record is being added to the file. ''As many as'' because, (except as noted below) it is possible that, as large records are stored, there will not be room for the full number of <var>BRECPPG</var> on the page, and so they would not be stored.</p>
 
<p>''Up to'' because, (except as noted below) it is possible that, as large records are stored, there will not be room for the full number of BRECPPG on the page, and so they would not be stored.</p>
 
<p>For Enhanced Data Handling ([[FILEORG parameter|FILEORG]] x'100') files, with [[RECRDOPT parameter|RECRDOPT]] set to x'01', you will always have space for the full number of BRECPPG record pointers, as their length is included in the calculation of the [[BRLIMSZ parameter]]. They still are not present on the page until the record is stored.</p>   
 
<div id="Internal Record Number"></div>
<p>'''Internal Record Number'''</p>
<p>The record pointer is a useful identifier of a record, and is referred to as the 'Internal Record Number' (IRN).</p>
 
<p>If, for example, [[BRECPPG parameter|BRECPPG]] is set to 10, the first Table B page will, at most) contain records 0 to 9 (the first record in a Model 204 file is IRN 0); the second page contains IRNs 10 to 19, and so on.</p>


<p>Even if all the record slots are not used, they are treated as if they were, meaning that, in the exmple above, the second page in the file will still contain IRNs 10 to 19 even if only one record is physically present on the first page.</p>
<p>For Enhanced Data Handling (<var>FILEORG</var> X'100') files, with <var>[[RECRDOPT parameter|RECRDOPT]]</var> set to X'01', you will always have space for the full number of <var>BRECPPG</var> record pointers, as their length is included in the calculation of the <var>[[BRLIMSZ parameter|BRLIMSZ]]</var>. They still are not present on the page until the record is stored.</p>  


<p>See [[$CURREC]] for a handy way to retrieve the IRN.</p>  
====Internal Record Number====
<p>
The record pointer is a useful identifier of a record, and is referred to as the "Internal Record Number" (IRN).</p>


<p>Note that the IRN is not a permanent record identifier. If the file is reorganized it is highly likely that almost all records will have a different IRN afterward.</p>
<p>If, for example, <var>BRECPPG</var> is set to 10, the first Table B page contains, at most, records 0 to 9 (the first record in a Model 204 file is IRN 0). The second page contains IRNs 10 to 19, and so on.</p>


=== Space Management ===
<p>Even if all the record slots are not used, they are treated as if they were. That is, in the example above, the second page in the file will still contain IRNs 10 to 19, even if only one record is physically present on the first page.</p> 


<p>A Table B page grows from the top down (the pointers) and the bottom up (data area) simultaneously.<p>
<p>See <var>[[$CURREC]]</var> for a handy way to retrieve the IRN.</p>  


<p>When [[Adding Records (File Architecture)|adding records]], there must be enough room to add all the preallocated fields on that page. Also, no new record will be added to the page if there are less than BRESERVE parameter|BRESERVE bytes available on the page. In each of these cases, the 'next' page is used (see [[Adding Records (File Architecture)|adding records]]).</p>
<p>Note that the IRN is not a permanent record identifier. If the file is reorganized, it is highly likely that almost all records will have a different IRN afterward.</p>


<p>Similarly, when adding fields to existing records, when it reaches a point where the new field (or fieldgroup) will not fit on the current page. When that happens, data is moved from the record into the extension record next in the chain, with, perhaps, a new extension being created at the end of the chain.</p>
===Space management===
<p>
A Table B page grows from the top down (the pointers) and the bottom up (data area) simultaneously.</p>


<p>When adding records, there must be enough room to add all the preallocated fields on that page. Also, no new record will be added to the page if there are less than <var>[[BRESERVE parameter|BRESERVE]]</var> bytes available on the page. In each of these cases, the "next" page is used (see [[Adding records (File architecture)|adding records]]).</p>


<p>Similarly, when adding fields to existing records and the new field (or fieldgroup) does not fit on the current page, data is moved from the record into the extension record next in the chain, with, perhaps, a new extension being created at the end of the chain.</p>


=== Further information on Records ===
=== Further information about records ===


Refer to [[Record (File Architecture)|Records]] and [[Adding Records (File Architecture)|Adding Records]] for a full discussion of record architecture.
Refer to [[Record (File architecture)|Record]] and [[Adding records (File architecture)|Adding records]] for a full discussion of record architecture.


Refer also to [[Record Design (File management)|Record design]] for the best use of Model 204 Records.
Refer also to [[Record design]] for the best use of Model 204 records.


=== <div id="Table B (File Architecture) Segments">Table B Segments</div> ===   
===Table B segments===   


==== Definition ====
====Definition====
<p>
As discussed below, [[Bitmaps (File architecture)|bitmaps]] are used in a number of ways in Model&nbsp;204's processing. As bitmaps can contain bits covering 49152 records (usable space of 6144 * 8) records, this number of records is called a <b>segment</b>.</p>


<p>As discussed below, [[Bit Maps (File Architecture)|bit maps]] are used in a number of ways in Model 204's processing. As bit maps can contain bits covering 49152 records (usable space of 6144 * 8) records, this number of records is called a segment</p>
====Importance of and use====
<p>
Segments are used in indexing (as discussed below). In addition, when a procedure is compiled, the maximum number of Table B segments that the code needs to be able to handle is tracked (remember that sets of records are tracked with bitmaps). </p>


==== Importance of and Use ====
====INCREASE command boundaries====
<p>
When Model 204 code is compiled, one of the critical things noted is the number of segments in the file. This is necessary so that space can be reserved for found sets and other record tracking methodology.</p>


<p>In addition to indexing (discussed below), segments are used, when a procedure is compiled, to track the maximum number of Table B segments that the code needs to be able to handle (remember that sets of records are tracked via bit maps). </p>
<p>Because of this, if you want to increase Table B, and other users or subsystems have the file open, you can only increase it to the next segment boundary (that is, the next multiple of 49152), and you must use the <var>DYNAMIC</var> option of the <var>[[INCREASE command|INCREASE]]</var> command. Otherwise, you can use the <var>INCREASE</var> command to extend Table B beyond the current segment boundary, but this requires exclusive access to the file.</p>


==== [[INCREASE command]] boundaries ====
<p>For flexibility, consider using the <var>[[MAXINCBP parameter|MAXINCBP]]</var> parameter. This parameter (expressed as a percentage) is the additional number of segments reserved in compilation for growth. See the <var>MAXINCBP</var> explanation in [[Managing file and table sizes#MAXINCBP parameter|Managing file and table sizes]] for details.</p>


<p>When Model 204 code is compiled, one of the critical things noted is the number of segments in the file. This is necessary so that space can be reserved for found sets and other record tracking methodology.</p>
====Use in indexing====
<p>
The segment concept is inherent to Model 204 indexing in a number of ways. Specifically:</p>


<p>Because of this, if you want to increase Table B, and other users or subsystems have the file open, you can only do so to the next segment boundary (i.e.: the next multiple of 49152), and you must use the DYNAMIC option of the INCREASE command . Otherwise, you can use the INCREASE command to extend Table B beyond the current segment boundary, but this requires exclusive access to the file..</p>
<ul>
<li>See [[Table C (File architecture)#Segment entries |Table C segment entries]] for the effect of segments in the hash index.


<p>For flexibility, Rocket Software recommends that you consider the use of the [[MAXINCBP parameter]]. This parameter (expressed as a percentage) is the additional number of segements reserved in compilation for growth.<p>
<li>See [[Table D (File architecture)#B-tree index structure|B-tree index structure]] for the effect of segments in the ordered index.


<p>See [[Managing File and Table Sizes#The MAXINCBP parameter|the MAXINCBP parameter]] for a full explanation of how it works.</p>
<li>See [[Table D (File architecture)#Bitmaps used in indexing|Bitmaps used in indexing]] for how segments relate to the detailed level of both types of indexes.
</ul>


==== Use in Indexing ====
===Reuse queue===
 
<p>
<p>The segment concept is inherent to Model 204 indexing in a number of ways. Specifically:</p>
For Reuse Record Number (RRN) files (<var>FILEORG</var> X'04' bit set), pages with sufficient space (see the <var>[[BREUSE parameter]]</var>) and one or more available record slots are kept in a reuse queue.</p>
 
* See [[#Table C (File Architecture)#Segment Entries |Table C Segment Entries]] for the effect of segments in the hash index.
 
* See [[#Table D (File Architecture)#B-Tree index structure|B-Tree index structure]] for the effect of segments in the ordered index. 
 
* See [[#Table D (File Architecture)#Bit Maps Used In Indexing|bit maps used in indexing]] for how segments relate to the detailed level of both types of indexes.
 
=== <div id="Table B (File Architecture) The Reuse Queue">The Reuse Queue</div>===
 
<p>For Reuse Record Number (RRN) files ([[FILEORG parameter|FILEORG]] x'04' bit set) pages with sufficient space (see the [[BREUSE parameter]]) and one or more available record slots are kept in a reuse queue.</p>


<p>This may happen due to records (or large numbers of fields) being deleted.</p>  
<p>This may happen due to records (or large numbers of fields) being deleted.</p>  


<p>These use the 4 byte reuse queue page number to track the pages's availability.</p>
<p>These use the 4-byte reuse queue page number to track the pages's availability.</p>
 
<p>RRN files add records to the pages from the queue before adding records to [[BHIGHPG parameter|BHIGHPG]].</p>
 
==The following Parameters relate to the Use of Table B==
 
<p>This table excludes the parameters used for [[Hash Key Files|Hash Key]] and [[Sorted Files]]. Please see those topics for a full discussion of their special parameters (and distinction in meanings for some of these).</p>
<p>Also not shown are the parameters which control the [[#Resizing Tables (File Management) Automatic Increases|automatic increase]] in table size, which see.</p> 
 
::{|
|[[BHIGHPG parameter]]
|Table B highest active page. Normally the high water mark of pages <br>used (except for Hash Key and Sorted files). Viewable only.
|-
|[[BQLEN parameter]]
|The number of pages in the reuse queue. Viewable only.
|-
|[[BRECPPG parameter]]
|The maximum number of record slots on a Table B page.
|-
|[[BRESERVE parameter]]
|Table B Reserved space.
|-
|[[BREUSE parameter]]
|The minimum percentage of free space necessary to add a page to the reuse queue, over and above BRESERVE.
|-
|[[BREUSED parameter]]
|The numberof record slots reused.
|-
|[[BRLIMSZ parameter]]
|Base record size maximum. Viewable only. 
|-
|[[BSIZE parameter]]
|The number of pages in Table B.
|-
|[[RECRDOPT parameter]]
|Record storage options.
|-
 
 
|}
 


<p>RRN files add records to the pages from the queue before adding records to <var>[[BHIGHPG parameter|BHIGHPG]]</var>.</p>


==Parameters related to the use of Table B==
<p>
This table excludes the parameters used for [[Hash key files|Hash Key]] and [[Sorted files]]. See those topics for a full discussion of their special parameters (and distinction in meanings for some of the parameters below).</p>
<p>
Also not shown are the parameters that control the [[Managing file and table sizes#Automatic increases|automatic increase]] in table size.</p> 


<table  class="thJustBold" style="Width:80%">
<tr><th>[[BHIGHPG parameter|BHIGHPG]]</th>
<td>Table B highest active page. Normally the high water mark of pages used (except for Hash Key and Sorted files). Viewable only. </td></tr>
<tr><th>[[BQLEN parameter|BQLEN]]</th>
<td>The number of pages in the reuse queue. Viewable only. </td></tr>
<tr><th>[[BRECPPG parameter|BRECPPG]]</th>
<td>The maximum number of record slots on a Table B page. </td></tr>
<tr><th>[[BRESERVE parameter|BRESERVE]]</th>
<td>Table B Reserved space. </td></tr>
<tr><th>[[BREUSE parameter|BREUSE]]</th>
<td>The minimum percentage of free space necessary to add a page to the reuse queue, over and above BRESERVE. </td></tr>
<tr><th>[[BREUSED parameter|BREUSED]]</th>
<td>The numberof record slots reused.  </td></tr>
<tr><th>[[BRLIMSZ parameter|BRLIMSZ]]</th>
<td>Base record size maximum. Viewable only. </td></tr> 
<tr><th>[[BSIZE parameter|BSIZE]]</th>
<td>The number of pages in Table B. </td></tr>
<tr><th nowrap>[[RECRDOPT parameter|RECRDOPT]]</th>
<td>Record storage options. </td></tr>
</table>


[[Category:File architecture]]
[[Category:File architecture]]

Latest revision as of 17:00, 22 April 2016

This page describes the internal architecture of a Model 204 Table B page.

For a discussion of the ways a File Manager can organize these pages in a file, refer to File design.

At a minimum, Table B contains all of the base records in a Model 204 file.

Up to Model 204 version 7.4, Table B has a maximum number of 16.7 million record numbers (IRNs for Internal Record Numbers). As of Model 204 version 7.5, Table B has a maximum number of 48 million record numbers (if the FILEORG X'200' bit is set).

If Table X is not enabled, these "slots" are used for both base records and extension records. If Table E is not enabled, any data with contents greater than 255 bytes must be stored as a series of repeating fields.

Structure of a Table B page


Not shown in the figure above is a 4-byte "Reuse Queue page number" for Unordered files. If it is present, it is in the lower left-hand corner ("before," working backwards, the first record on the page).

Each item in the figure is described below.

Pointers

Free space pointer

The free space pointer points to the last cell in the currently allocated pointer array, which in turn points to the beginning of free space on a page, where the next record can begin. (See the above diagram.) On pages with no records yet stored, the free space pointer points to the last byte on the page (either the very end, of just before the Reuse Queue Page Number described above).

Record pointer array

There are as many as BRECPPG record pointers. These are not physically put onto the page until the record is being added to the file. As many as because, (except as noted below) it is possible that, as large records are stored, there will not be room for the full number of BRECPPG on the page, and so they would not be stored.

For Enhanced Data Handling (FILEORG X'100') files, with RECRDOPT set to X'01', you will always have space for the full number of BRECPPG record pointers, as their length is included in the calculation of the BRLIMSZ. They still are not present on the page until the record is stored.

Internal Record Number

The record pointer is a useful identifier of a record, and is referred to as the "Internal Record Number" (IRN).

If, for example, BRECPPG is set to 10, the first Table B page contains, at most, records 0 to 9 (the first record in a Model 204 file is IRN 0). The second page contains IRNs 10 to 19, and so on.

Even if all the record slots are not used, they are treated as if they were. That is, in the example above, the second page in the file will still contain IRNs 10 to 19, even if only one record is physically present on the first page.

See $CURREC for a handy way to retrieve the IRN.

Note that the IRN is not a permanent record identifier. If the file is reorganized, it is highly likely that almost all records will have a different IRN afterward.

Space management

A Table B page grows from the top down (the pointers) and the bottom up (data area) simultaneously.

When adding records, there must be enough room to add all the preallocated fields on that page. Also, no new record will be added to the page if there are less than BRESERVE bytes available on the page. In each of these cases, the "next" page is used (see adding records).

Similarly, when adding fields to existing records and the new field (or fieldgroup) does not fit on the current page, data is moved from the record into the extension record next in the chain, with, perhaps, a new extension being created at the end of the chain.

Further information about records

Refer to Record and Adding records for a full discussion of record architecture.

Refer also to Record design for the best use of Model 204 records.

Table B segments

Definition

As discussed below, bitmaps are used in a number of ways in Model 204's processing. As bitmaps can contain bits covering 49152 records (usable space of 6144 * 8) records, this number of records is called a segment.

Importance of and use

Segments are used in indexing (as discussed below). In addition, when a procedure is compiled, the maximum number of Table B segments that the code needs to be able to handle is tracked (remember that sets of records are tracked with bitmaps).

INCREASE command boundaries

When Model 204 code is compiled, one of the critical things noted is the number of segments in the file. This is necessary so that space can be reserved for found sets and other record tracking methodology.

Because of this, if you want to increase Table B, and other users or subsystems have the file open, you can only increase it to the next segment boundary (that is, the next multiple of 49152), and you must use the DYNAMIC option of the INCREASE command. Otherwise, you can use the INCREASE command to extend Table B beyond the current segment boundary, but this requires exclusive access to the file.

For flexibility, consider using the MAXINCBP parameter. This parameter (expressed as a percentage) is the additional number of segments reserved in compilation for growth. See the MAXINCBP explanation in Managing file and table sizes for details.

Use in indexing

The segment concept is inherent to Model 204 indexing in a number of ways. Specifically:

Reuse queue

For Reuse Record Number (RRN) files (FILEORG X'04' bit set), pages with sufficient space (see the BREUSE parameter) and one or more available record slots are kept in a reuse queue.

This may happen due to records (or large numbers of fields) being deleted.

These use the 4-byte reuse queue page number to track the pages's availability.

RRN files add records to the pages from the queue before adding records to BHIGHPG.

Parameters related to the use of Table B

This table excludes the parameters used for Hash Key and Sorted files. See those topics for a full discussion of their special parameters (and distinction in meanings for some of the parameters below).

Also not shown are the parameters that control the automatic increase in table size.

BHIGHPG Table B highest active page. Normally the high water mark of pages used (except for Hash Key and Sorted files). Viewable only.
BQLEN The number of pages in the reuse queue. Viewable only.
BRECPPG The maximum number of record slots on a Table B page.
BRESERVE Table B Reserved space.
BREUSE The minimum percentage of free space necessary to add a page to the reuse queue, over and above BRESERVE.
BREUSED The numberof record slots reused.
BRLIMSZ Base record size maximum. Viewable only.
BSIZE The number of pages in Table B.
RECRDOPT Record storage options.