Record (File architecture)

Overview

A Model 204 record is a collection of fields (either individual, in field groups, or any mixture of these) containing a set of data about a thing. Admittedly, this is a very vague description, because the flexibility of Model 204 permits the File Manager to create files whose records are best suited to the required purpose, whatever it may be.

Each record is variable in length and need contain only the fields (some or all of those contained in the internal file dictionary in Table A) that pertain to it. The limit of the number of field value pairs in a record is in the tens of millions.

There is only a limited fixed format for data within a record (preallocated fields). Almost any number of fields can appear almost any number of times in almost any order. Each record is automatically assigned a unique internal record number (IRN) that is used by the system to build index entries for the record. The IRN is temporary and will likely change when the file is reorganized.

For example, the following code does, in fact, physically store a record:

IN filename STORE RECORD END STORE

The record is assigned an Internal Record Number and contains the preallocated field structure (if any) but no data.

Records, base records, and extension records

The term "record" is actually used three ways:

• The record as a full collection of all of the data about a "thing"
• Base record: the initial portion of the record data
• Extension record(s): a (possible) string of additional portions of the data

These "record" terms are discussed in the subsections that follow.

Record as container

A record as a container is just that: a way to describe all of the data associated with the record key (if a record key exists).

Base records

The base record is the first (and often, only) part of the record. It is assigned a slot in Table B, and its internal record number is how Model 204 identifies the record for indexing purposes as well as in record sets and lists.

Extension records

When a full record is not able to fit on a particular page (either, it itself may be more than 6140 bytes, or other records on the page do not leave sufficient space for the record), one or more extension records are stored.

If Table X is enabled, all such extensions are stored there, otherwise the extension(s) are stored in Table B.

Record growth

The ADD statement (or the file load process) always places new data at the end of a record, so the record growth is straightforward: add a field at the end of the record, if it doesn't fit, start a new extension.

insertions are more complicated. Whether updating fields or field groups, the correct location must be found within a record, and the data inserted at that point.

It is quite possible, that there will not be enough space on the page to insert a record, in which case space is made by moving data from this record (after the insert point) on to an extension record. So, with large extension records, it is conceivable that a single field added to the base record might cause all of the extensions for a record.

Internal record number

The internal record number (IRN) is the base record number (or slot) assigned when the record is first stored. While, once assigned, it can be passed around inside a program as an easy way to refer back to the set of data (see the For Record Number statement), it is not a permanent identifier: when the file is reorganized, the position where a record is reloaded may be quite different than where it was before.

The IRN (represented by a single bit in a bitmap) is used as a target for Model 204 indexing (both hashed and B-tree indexing). See Bitmaps for more information.

The structure of a Model 204 record

As shown in Table B, Model 204 records grow from the end toward the beginning of the page it is stored upon. So, a proper representation of the record shows it as:

In any discussion of the record, the "beginning" of the record is to the left, and the growth (represented by the blue arrow) is toward the right.

Extension pointer

Every "portion" of every Model 204 record (base or one of perhaps many extensions) begins with an extension record pointer.

For files without a Table X, this pointer is three bytes in length.

For files with Table X enabled, this pointer is four bytes in length (to handle the larger number of slots in Table X).

Regardless of the length, this pointer links this portion of the record to its next portion. Where that location is, of course, depends on whether Table X is enabled.

Data

If there are any preallocated fields in a file (see the OCCURS attribute) every record in that file will begin with the set of such fields, immediately after the extension pointer. The structure of the fields is held in the record map in Table D.

Note: For the sort or hash key (for such files), if it is preallocated (and it should be unless it is wildly variable in size), it will be the first field in the record map. If it is not preallocated, it will be the first "field value pair" in the record.

After the preallocated fields comes the rest of the data: a series of field value pairs, field groups, and/or pointers to large objects. These will be physically positioned according to either of the following:

• The order they were ADDed (which always adds the entity at the end of the record)
• An INSERT statement.

Hence, once past the preallocated fields, the application code controls the relative order of the data in the record.

The following subsections further describe each of the four record data components, but refer to the specific topic for each for further detail.

Preallocated fields

All records stored in a Model 204 file will contain all of the preallocated fields (those with the OCCURS attribute) (if any) for that file. These are stored based on the Record map of preallocated fields stored in Table D.

Field value pairs

Field value pairs, or, more precisely, field name "equals" value pairs, are the most common way that data is held in a Model 204 record.

Even the field group structure is a set of field value pairs which occur in tandem.

Field groups

A field group allows sets of data to be managed together. Inside of the field group is a series of field value pairs (as above), but these fields are not independent of each other.

Large objects

Large objects (data longer than 255 bytes) can be handled in a Model 204 record in two ways:

• Native large objects in Table E

  Large objects stored as part of a record contain a pointer locally (within the record) to the starting point of the large object in Table E.

  Starting in Model 204 V7.5, large objects can be handled by using the FILEORG X'100' bit. See Table E for the full discussion.

• Large objects without Table E

  If Table E is not enabled, and the File Manager still wants to store data that is more than 255 bytes long, storage must be done by the application code.

  The technique is to store the data as a series of repeating fields, each containing (up to) 255 bytes. The application must:

  1. Parse the input data into 255-byte chunks to store it.
  2. String the chunks together again to read it.

For efficient storage of large objects, Rocket Software recommends enabling Table E.