Record (File architecture)

Overview

A Model 204 Record is a collection of fields (either individually; in Repeating Field Groups (RFGs); or any mixture of these) containing a set of data about a thing. Admittedly, this is a very vague description, because the flexibility of Model 204 permits the File Manager to create files whose records are best suited to the required purpose, whatever it may be.

Each record is variable in length and need contain only the fields (some or all of those contained in the Internal File Dictionary in Table A) that pertain to it. The limit of the number of field value pairs in a record is in the tens of millions.

There is only a limited fixed format for data within a record (pre-allocated fields). Almost any number of fields can appear almost any number of times in almost any order. Each record is automatically assigned a unique internal record number (IRN) that is used by the system to build index entries for the record. The IRN is temporary and will likely change when the file is reorganized.

For example, the following code:

IN filename STORE RECORD END STORE

does, in fact, physically store a record. It has been assigned an Internal Record Number and would contain the pre-allocated field structure (if any), but no data.

Records. Base Records and Extension Records

The term 'record' is actually used three ways:

• the record as a full collection of all of the data about a 'thing'
• base records: the initial portion of that data
• extension records: a (possible) string of additional portions of the data

These are discussed below:

the Record as a container

Is just that: a way to describe all of the data held against the record key (presuming there is one)...

Base Records

The base record is the first (and, often, only) part of the record. It is assigned a slot in Table B, and its internal record number is how Model 204 identifies the record for indexing purposes as well as in record sets and lists.

Extension Records

When a full record is not able to fit on a particular page (either, it itself may be more than 6140 bytes, or other records on the page do not leave sufficient space for the record) one or more extension records are stored.

If Table X is enabled, all such extensions are stored there, otherwise the extension(s) are stored in Table B.

Record Growth

The ADD statement (or the file load process) always places new data at the end of a record, and so, the record growth is straightforward: add a field at the end of the record, if it doesn't fit, start a new extension.

INSERTs are more complicated. Whether updating fields or repeating field groups, the correct location must be found within a record, and the data inserted at that point.

It is, of course, quite possible, that there will not be enough space on the page to insert it, in which case space is made by moving data from this record (after the insert point) down the extension chain. So, with large extension records, it is conceivable that a single field added to the base record (for example) might cause all of the extensions to need to change.

The Internal Record Number (IRN)

The internal record number (IRN) is the base record number (or slot) assigned when the record is first stored. While, once assigned, it can be passed around inside a program as an easy way to refer back to the set of data (see the FOR RECORD NUMBER statement) it is not a permanent identifier: when the file is reorganized the position where a record is reloaded may be quite different than where it was before.

The IRN (represented by a single bit in a bit map) is used as a target for Model 204 indexing (both hashed and B-tree indexing). See Bit Maps for more information.

The Structure of a Model 204 Record

As shown in Table B, Model 204 records grow from the end toward the beginning of the page it is stored upon. So, a proper representation of the record shows it as:

In any discussion of the record, the 'beginning' of the record is to the left, and the growth (represented by the blue arrow) is toward the right.

Extension Pointer

Every 'portion' of every Model 204 record (base or one of perhaps many extensions) begins with an extension record pointer.

For files without a Table X, this pointer is 3 bytes in length.

For files with Table X enabled, this pointer is 4 bytes in length (to handle the larger number of slots in Table X).

Regardless of the length, this pointer links this portion of the record to its next portion. Where that location is, of course, dependent on whether Table X is enabled.

Data

If there are any preallocated fields in a file (see the OCCURS attribute) every record in that file will begin with the set of such fields, immediately after the extension pointer. The structure of the fields is held in the Record Map in Table D.

Note: For the Sort or Hash key (for such files), if it is preallocated (and it should be unless it is wildly variable in size) it will be the first field in the record map. If it is not preallocated, it will be the first 'field value pair' in the record.

After the preallocated fields, is the rest of the data; a series of: field value pairs; repeating field groups; and/or pointers to large objects. These will be physically positioned in: either the order they were ADDed (which always adds the entity at the end of the record); or positioned by an INSERT statement. Hence, the application code controls the relative order of the data in the record, once past the preallocated fields.

Each of the four is briefly described below, but refer to the specific topic for each for further detail:

Preallocated Fields

All records stored in a Model 204 file will contain all of the preallocated fields (those with the OCCURS attribute) (if any) for that file. These are stored based on the Record Map of preallocated fields stored in Table D.

Field Value Pairs

Field Value Pairs, or, more precisely, Field Name 'Equals' Value Pairs, are the most common way that data is held in a Model 204 Record.

Even the Repeating Field Group structure is a set of Field Value Pairs which occur in tandem.

Repeating Field Groups (RFGs)

A Repeating Field Group (RFG) allows sets of data to be managed together. Inside of the RFG is a series of field value pairs (as above) but these fields are not independent of each other.

Large objects

There are two ways that large objects (data longer than 255 bytes) can be handled in a Model 204 record.

Native large objects in Table E

Large objects stored as part of a record contain a pointer locally (within the record) to the starting point of the large object in Table E.

Starting in Model 204 V7R5, large objects can be handled by using the FILEORG x'100' bit. See Table E for the full discussion.

Large objects without Table E

If Table E is not enabled, and the File Manager still wants to store data that is more than 255 bytes long, storage must be done by the application code.

The technique is to store the data as a series of repeating fields, each containing (up to) 255 bytes.

The application would need to parse the input data into 255 byte chunks to store it, and then string it together again to read it.

For efficient storage of large objects, Rocket Software recommends enabling Table E.