Large request considerations

Overview

There is a practical limit to the number of statements allowed in each request. This limit varies depending on the installation and the type of statements used.

Requests that are too large exceed the amount of memory allotted for one or more of the internal work areas, also referred to as server tables. Such an event produces an error message of the form:

REQUEST TOO LONG - xTBL

You can take a combination of any of these approaches to handle large requests:

• Rewrite large requests to make them shorter and less complex.
• Increase the sizes of internal work areas (server tables).
• Use the request continuation feature.

This page focuses on the second two approaches.

User Language internal work areas

Model 204 uses several internal tables as work areas for processing User Language requests.

The size of these tables (which can be set by the user) determines the number of statements allowed in each request. The number of statements allowed also depends on the types of statements used.

Summary of work areas

The internal work areas required by the User Language compiler and evaluator are listed below:

User Language internal work areas

Table	Contents
FSCB	Menus, screens, and images
FTBL	File groups
GTBL	Global variables
ITBL	Dummy strings and $READ responses
NTBL	Statement labels, list names, variables
QTBL	Statements in internal form
STBL	Character strings
TTBL	List of temporary work pages
VTBL	Compiler variables

Each of these tables is described in more detail in Description of tables.

Resetting table sizes

The size of these tables can be reset by the user with the UTABLE command. Table sizes are controlled by the LxTBL parameters described in pages you can locate via the List of Model 204 parameters. Resetting these table sizes can increase the total server size requirement at your site. If this is the case, you must refine your request using the techniques discussed in this section.

The size of the FSCB is controlled by the LFSCB parameter.

Pushdown list and QTBL size increase

Note that if you get unrecoverable errors when running a request, you might need to increase the size of the pushdown lists or QTBL parameters or both.

Use the UTABLE command or see your system administrator to increase and reset these tables:

• The pushdown list parameter, LPDLST, approximately 10 to 20 percent. (See Pattern matching for more information on pattern matching.)
• The server pushdown list parameter, LSERVPD, approximately 10 percent.
• The LQTBL approximately 1 percent.

These percentages might vary at your site.

Description of tables

Each table and its use by the User Language compiler and evaluator is described separately below. The descriptions should help you to restructure large requests to make more efficient use of the available table space.

FSCB (full-screen buffer)

The FSCB (full-screen buffer) is used to store menu, screen, and image definitions and the values of screen variables and image data blocks. FSCB space is reused by each logical menu definition, logical screen definition, or block definition. Therefore the FSCB need only be large enough to hold the largest menu, screen, or image definition. FSCB requirements are:

• Every menu requires 144 bytes of fixed overhead in the FSCB, which includes the menu title. In addition, each menu prompt adds another 144 bytes in the FSCB.
• Every screen requires 432 bytes of fixed overhead for the first panel and 144 bytes (includes the screen title) for any other panels. An additional 32 bytes is used for each screen prompt and input item, as well as a 32-byte entry for every screen line containing at least one input item. Each defined screen line adds 80 bytes to FSCB requirements, including skipped lines.

  Additional space is used in the FSCB if automatic validation options are used. Each validation option adds 2 or 4 bytes to the FSCB. In addition, VERIFY adds 256 bytes the first time a particular character set is used in a logical panel. Additional occurrences of the same character set do not add extra space. ONEOF and character RANGE store each character string plus one byte for each string's length. NUMERIC RANGE adds 8 bytes for each number (16 bytes for each range pair).

• Every block used in image definition requires space in the FSCB. The amount of space required is computed as the sum of the following:
  • 32 bytes
  • 16 bytes for each IMAGE statement
  • 32 bytes for each ARRAY statement
  • 16 bytes for each item definition
  • 32 bytes for each OCCURS clause in an item definition
  • 32 bytes for each INITIAL clause in an item definition that specifies a value or number (other than the ZERO keyword) plus the length of the value or number
  • Total length of the block in bytes

FTBL (file group table)

Data structures relating to file groups are stored in FTBL.

Two types of FTBL entries are allocated by Model 204 when file groups are used:

• The first type is allocated each time a group is opened (explicitly by OPEN or implicitly for an ad hoc group) and is released when the group is closed. This type of entry has a fixed-size portion of 62 bytes, plus 2 bytes for each file in the group definition.
• The second type is used for collecting field name codes and properties during a request. An entry is allocated each time a new field name is encountered in the request. The entry size is variable, consisting of 9 fixed bytes plus a number of bytes equal to the length of the field name plus 11 bytes for each file in the group. These field entries are deleted at the END statement (including END MORE).

In addition to the space required by the two types of entries described above, Model 204 allocates a fixed amount of space in FTBL equal to 4 bytes times the value of the NGROUP runtime parameter.

GTBL (global variable table)

Global variables, lists, found sets, images, screens, menus, and REMEMBERed positions in FOR loops are special mechanisms for communicating across User Language request boundaries, as described in Global features.

A GTBL entry is created when any global object is declared. In addition to the entries created by individual declarations, the last 32 bytes of your GTBL space allocation is reserved for a GTBL trailer.

You can delete entries from GTBL by using different forms of the CLEAR statement, or the CLEARG or CLEARGO commands. See Clearing the GTBL work area for detailed instructions on clearing objects.

The types of GTBL entries are listed below:

• A global variable (declared or changed with the $SETG or $INCRG function) creates an entry with a length of 4 bytes plus the length of the variable name plus the length of the current value. If a global variable is redefined, its old entry is deleted and a new entry is added.
• A global list or found set in single file context creates an entry with a length of 28 bytes.
• A global list or found set in group context creates an entry with a length of 28 bytes plus eight bytes for each file in the group.
• A sorted found set creates an entry with a length of 40 bytes.
• A position in an unordered or sorted record set creates an entry with a base length of 32 bytes.
• A position in a non-pattern driven ordered (BTREE) record set creates an entry with a length of 60 bytes plus a number of bytes equal to the size of the BTREE key.
• A position in a pattern driven ordered (BTREE) record set creates an entry with a length of 68 bytes.
• A position in a sorted file record set creates an entry with a length of 80 byes.

For a break down of GTBL usage for global screens and images, refer to FSCB (full-screen buffer) for details of these global object types.

ITBL (dummy string and $READ response table)

ITBL is used to hold dummy string and $READ responses entered as arguments to an INCLUDE statement or command.

Argument strings are saved as entered, including delimiters, with an additional 4 bytes of overhead for each saved string.

Space taken by a string is released when the included procedure is exhausted.

NTBL (statement labels/list names/variables table)

One entry (12 bytes) is allocated in NTBL for each statement label, list name, %variable, and image, menu, and screen variable. An entry also is allocated for each partner process opened by a request. Data defined as common takes two entries for the first COMMON declaration and one additional entry for each additional COMMON declaration.

One entry is allocated for each unlabeled FIND. An extra entry is allocated for each FOR EACH VALUE statement or each FOR statement with the IN ORDER clause. Most NTBL entries are preserved by MORE, but the unlabeled FIND and secondary FOR entries are deleted.

During request evaluation, one entry is required for each sequential or VSAM file opened simultaneously.

QTBL (internal statement table)

Each statement is compiled into internal Model 204 instructions in QTBL. After compilation, the entries in QTBL are used to drive the evaluator. The entries generated by each User Language statement vary in number and in size.

Under certain circumstances, users sharing precompiled procedures can use a shared version of QTBL, thus reducing server I/O. See the Rocket Model 204 System Manager's Guide for more information.

Some common examples are listed below:

Statement or other functions	Bytes generated	Comments
ADD	20	Each fieldname=value
CALL	16
CHANGE	44	fieldname=value TO
CLEAR ON	16
CLEAR TAG	16
CLOSE	16
CLOSE PROCESS	16
COMMIT	4
COMMIT RELEASE	20
COUNT RECORDS	52	Plus 20 or a group
DELETE	24	fieldname=vaue
DELETE RECORD	16	Each
END	4
Expressions	Varies	Depends on the complexity of the expression and on whether conversion between data types is required.
FIND ALL RECORDS (with no qualification)	64	Record security adds 52 bytes. Group adds 20 more. Each inverted condition adds about 36 bytes; NUMERIC RANGE conditions a little less. Each direct and Ordered Index condition (for example, AFTER, GREATER THAN) generates an entry of 20 bytes in length. The IN RANGE FROM and TO clause and BETWEEN operator clause generate entries of 28 bytes, plus 16 for each AND (except where AND is part of the BETWEEN clause). An additional 16 bytes are added for each FIND statement with at least one direct condition. AND and OR as operators generate 16 bytes. At request evaluation time, additional space can be required in QTBL for a FIND with direct conditions; this requirement never exceeds the number of direct bytes already compiled by more than 16 bytes. The extra QTBL space is released after evaluation of the FIND. Numeric conditions, when used with NUMERIC RANGE fields, also generate bytes during evaluation that are released as soon as the FIND has been executed.
FIND ALL VALUES	32 74	For an ORDERED field For an FRV field
FOR EACH RECORD	20	In addition to the body of the loop.
FOR EACH VALUE OF	88	In addition to the body of the loop: IN ORDER adds 68 bytes A group adds 20 more
Function call	16	Plus the bytes necessary to evaluate the arguments.
IDENTIFY	16
IF	32 for each line in the IF statement	THEN clause generates 16 bytes, plus additional for the body of the clause. Each operator in the IF statement generates 16 bytes. Conversions between strings and numbers take 16 bytes each. Each ELSE and ELSEIF generates 16 bytes, plus additional for the body of the ELSE or ELSEIF.
Index loop	40	In addition to the bytes necessary to evaluate expressions used in the statement, plus any generated within the body of the loop.
INSERT	24	Each fieldname=value
MODIFY	16
NOTE	20	Each fieldname=value
ON units	16	Each ON unit also generates bytes for the statements within the ON unit and 16 bytes for the automatically generated STOP statement.
OPEN	16	For external entities.
OPEN PROCESS	16
PAUSE	16
POSITION	20
PREPARE IMAGE	8
PREPARE MENU	8
PREPARE SCREEN	8
PRINT	16	For each term in the print specification: Field from a record takes an additional 20 bytes. AND connective generates 16 bytes. TAB generates 4 bytes. WITH generates no bytes.
PRINT MENU	16
PRINT SCREEN	16
READ IMAGE	16
READ MENU	20
READ SCREEN	20
RECEIVE	16
References to a subscripted %variable	16	in addition to the bytes required to evaluate the expression for each dimension.
RELEASE POSITION	8
REPEAT	16	Additional bytes are required to evaluate each condition for the WHILE clause.
REREAD SCREEN	20
RETRY	16	Each pending statement.
RETURN	16
SEND	16
SIGNAL PROCESS	12
STORE RECORD	16	Plus 16 for each field to be included in the record.
TAG	16
TRANSFER	16
WRITE IMAGE	12

QTBL is emptied by END and END MORE (see Starting and ending requests and continuations for a discussion of END MORE).

STBL (character string table)

All character strings are stored in STBL. The strings are kept in counted form, with a 1-byte length preceding the string itself.

The following types of strings are stored in STBL:

• Quoted strings
• Values in FIND specifications
• Literal values in ADD, CHANGE, DELETE, and INSERT statements
• %Variable values (space reserved for maximum length) for STRING %variables
• Array %variables

During evaluation, the following types of strings also are stored in STBL:

• Results of functions that return character strings
• Intermediate string results during arithmetic expression evaluation
• NOTE values and fields to be printed
• FOR EACH VALUE values
• FOR EACH OCCURRENCE values
• FOR EACH RECORD IN ORDER BY values when the Ordered Index is used to drive the loop

The space utilized for the storage of intermediate results during the evaluation of an arithmetic expression is freed when the evaluation of that expression is completed.

The last three categories above occur only within FOR EACH RECORD, FOR EACH OCCURRENCE, or FOR EACH VALUE loops. Each pass through the loop reuses the STBL space from the previous pass. When the loop is finished, the last NOTE, FOR EACH OCCURRENCE, or FOR EACH VALUE values remain in STBL (the space is not freed). Thus, if FOR statements are executed a large number of times or if many NOTE statements are issued, STBL fills up rather quickly.

A FIXED or FLOAT %variable array uses 8 bytes of STBL space for each element. If the FIELD SAVE option is used when a STRING %variable array is declared (if field information is saved where the element is used for a field name variable), then 13 bytes plus the maximum length of the string plus 1 byte are reserved in STBL for each element of the array. If the NO FIELD SAVE option is specified, the extra 13 bytes for field information are not reserved. Using NO FIELD SAVE can result in a significant saving in a multi-dimension array. For example, in an array with dimensions 100 by 2, specifying NO FIELD SAVE saves 100*2*13 (or 2600) bytes.

The MORE command (see MORE command) releases most STBL space, keeping only the entries used for %variables and arrays. STBL is filled in during evaluation, when a value is assigned to the %variable. The space in STBL is reused when the %variable is reassigned.

TTBL (temporary work page list table)

TTBL entries keep track of scratch file (CCATEMP) pages. The FIND statement uses scratch pages as a work space for evaluating Boolean expressions. The number of TTBL entries depends on the complexity of the Boolean operation. TTBL entries are released at the end of the evaluation of the FIND statement.

VTBL (compiler variable table)

Entries in VTBL vary in size. Most range from 8 to 20 bytes; some are much larger. Many User Language statements and some constructs cause one or more compiler variables to be allocated in VTBL. Common examples are listed in the following sections.

Expressions, commands, and statements

• Arithmetic expressions allocate 8-byte entries for string expressions and 16-byte entries for numeric expressions, some of which are used for intermediate results. The intermediate result space is reused by subsequent expressions.
• COUNT statement allocates one 8-byte entry.
• SORT statement allocates a 12-byte entry, plus one 20-byte entry for each field referenced in the sorted records (or one entry for a PRINT ALL INFORMATION statement), plus one 28-byte entry for each sort key.
• User Language function calls allocate one entry that is 4 + (4*number-of-arguments) bytes long. The arguments themselves have their own VTBL entries that are independent of the function call.
• Each subroutine declaration takes 16 bytes, plus the associated space for %variables and lists used as parameters.
• FIND allocates one basic 8-byte entry for a single file or 8+(8*number-of-files) bytes for a group. At least two 20-byte entries and one 28-byte entry are allocated for work space (more for complex Boolean operations). Also, one entry is allocated for each fieldname = value (property) pair mentioned in the FIND. The length of a property entry is at least 20 bytes; more for large files. Each direct retrieval requires an additional 28 bytes in VTBL. All space except that allocated by the basic entry is released after the FIND has been evaluated and is reused by subsequent FINDs.

  Each Ordered Index retrieval requires an entry of 4 bytes plus 4 bytes per segment in the file.

Increased VTBL requirements for FIND criteria

An extra 32 bytes of VTBL is needed for each FIND criteria in the FIND statement with the largest number of criteria. For example, if your request has a FIND statement with six criteria, the VTBL requirements would increase as follows:

6 FIND statement criteria * 32 = 192 bytes

If you sized VTBL at precisely what is needed to compile in an earlier release, compilation may fail in this release with error:

M204.0198: REQUEST TOO LONG -- VTBL

Defined objects

• Starting in Version 5.1, the size of the header record of a value set is increased from 20 bytes to 48 bytes.
• Each %variable adds one entry. The entry is 16 bytes for a FIXED %variable, 12 bytes for a FLOAT %variable, and 16 bytes for a STRING %variable. Each %variable array adds one 24-byte entry. There is also a 12-byte entry for every reference to an array element, regardless of the number of dimensions in the subscript.
• Every menu definition adds a 48-byte entry. Every screen definition adds a 68-byte entry, plus 4 bytes for each physical screen panel. A one-panel screen entry in VTBL is 72 bytes.
• Every set of related images adds a 12-byte entry plus 4 bytes for approximately every 256 items in the block.
• Lists allocate 8-byte entries for single files or 8+(8*number-of-files) bytes for groups.

CALL statements

• Each CALL statement with parameters allocates one entry that is 4 + (4 * number of arguments) bytes in length.
• Evaluation of a CALL statement generates one 28-byte entry that is released by RETURN. These entries are placed in a LIFO stack area.

Value loops

• FOR EACH RECORD without the IN ORDER clause allocates a 16-byte entry.
• FOR EACH VALUE with FROM, TO or LIKE specified, and/or using an ORDERED field allocates a 40-byte entry.
• FOR EACH VALUE and FOR EACH RECORD IN ORDER BY, using an ORDERED field, allocate an additional 44-byte entry.
• Every FOR loop position declared with a REMEMBER statement adds an 8-byte entry.
• When an ON unit is invoked, one 28-byte entry is used. The entry is released by BYPASS, RETRY, or STOP.

Many VTBL entries are deleted by the MORE command (refer to MORE command). Entries are retained in the following cases:

• %Variables
• COUNT's 8-byte entry (Not retained if the original User Language statement was unlabeled.)
• FIND's basic 8-byte (larger for groups) entry (Not retained if the original User Language statement was unlabeled.)
• Entries for lists
• Image, menu, and screen
• SORT's basic 12-byte entry plus the 20-byte sort key entries. (Not retained if the original User Language statement was unlabeled.)

Entries in the last three categories are not retained if the original User Language statement was not labeled.

Request continuation

A continuation is a request that refers to certain items in the preceding request.

Request continuation is used to increase user interaction with Model 204 without increasing retrieval costs, or to break up into smaller logical units requests that would be too large to run otherwise (requests that would exceed work table space, as described in the previous sections).

Request continuation is particularly effective when the request consists of a large and complex retrieval section followed by a similarly complicated report generation section.

Specific benefits of request continuation include:

• The opportunity to enter certain Model 204 commands between a request terminated by END MORE and its continuation.
• The ability to decide whether to continue a request based on its (intermediate) results.
• The opportunity to reduce the size of your QTBL, since this work area is cleared with the execution of an END MORE statement (unlike other work areas, which retain their contents when END MORE is encountered).

See the next section for rules relating to the use of request continuation.

MORE command

Description

If a BEGIN command is replaced by the MORE command, the request that follows is considered to be a continuation of the previous request and can refer to statement labels and other information from that request.

Example 1

Suppose the user is interested in high vehicle premiums for a particular garaging location. The user might write the following request:

BEGIN HIGH.PREM: FIND ALL RECORDS FOR WHICH VEHICLE PREMIUM IS GREATER THAN 200 GARAGING LOCATION = VA03 END FIND END MORE

Before printing information for the record set, the user can determine the amount of records to be printed by adding the following statements:

MORE LOW.DEDUCT: FIND AND PRINT COUNT FIND$ HIGH.PREM END FIND END MORE

The LOW.DEDUCT statement refers to the records retrieved by the HIGH.PREM statement in the original request. In this way, the expense of retrieving that set of records a second time is avoided.

If this continuation indicates that there are less than 40 records meeting the extended retrieval conditions, the user can continue the request again:

MORE FOR EACH RECORD IN LOW.DEDUCT PRINT OWNER POLICY - WITH PRINCIPLE DRIVER TO COLUMN 25 END FOR END

Example 2

In the preceding example, the user also could redirect the output, depending on the number of records found. For example, if there were more than 100 records in the set the user could precede the request continuation with a USE command:

USE OUTPRINT MORE FOR EACH RECORD IN LOW.DEDUCT PRINT OWNER POLICY - WITH PRINCIPLE DRIVER TO COLUMN 25 END FOR END

In this way the user can decrease the output time without having to reduce the number of records found in the initial request.

Rules for request continuation

Avoid too many continuations

In theory, requests can be continued indefinitely. However, information saved from the pieces of a basic request can fill the internal work areas (see User Language internal work areas), preventing further continuations.

Sets of records retrieved in one request remain locked through all continuations.

For these reasons, you should issue a fresh BEGIN command when information from the last request is no longer needed.

Starting and ending requests and continuations

A request is started by a BEGIN command; a continuation is started by a MORE command. Both are terminated by END or END MORE statements (with a USE specification if desired).

Multiple continuations

You can continue a request several times, as long as you issue an END MORE statement at the end of each section to be continued. Each new continuation, the initial request, and all previous continuations constitute the basic request. Once a request or a continuation ends with an END statement without the MORE option, it is no longer continued.

References in a continuation

A continuation can refer to the following elements of the basic request:

• Statement labels for COUNT, FIND, and SORT statements
• List names
• %Variables
• Menus and screens
• Images

Statement labels for statements other than COUNT, FIND, and SORT in the basic request can be reused in the continuation.

When a request is continued, all found sets, lists, and %variables of the main request are preserved for later use. The found sets, lists, and %variables of complex subroutines are discarded after END MORE unless they are common.

If a request opens a data set, and you want to leave the data set open after the request terminates, you can end the request with END MORE USE. This allows the continuation to add to the USE output without resetting the page number.

Restrictions applying to request continuations

A request continuation cannot:

• JUMP TO statement labels or CALL subroutines in the basic request
• Alter the type, length, or decimal place parameters of any %variable mentioned in the basic request
• Use a FOR RECORD NUMBER IN statement that refers to a label in the basic request

A basic request cannot continue if the user issues any of the following commands:

• BEGIN (a new request is started)
• CLOSE
• FILELOAD
• FLOD
• INITIALIZE
• LOGIN
• LOGOUT
• MONITOR
• UTABLE

ON units

An ON unit definition is not preserved across an END MORE statement and MORE command. Each new request continuation must define its own ON units.

Interaction with SORT statement

The records produced by a SORT statement contain only fields necessary to satisfy the initial request (or continuation) in which the SORT statement appears. If the initial request contains these statements:

SORT.RECS: SORT RECORDS IN FIND.RECS BY FULLNAME FOR EACH RECORD IN SORT.RECS %A = AGENT

a continuation retrieves only the AGENT field from the sorted records. All other fields are treated as missing in the sorted records.