m204wiki - User contributions [en]

RegexSplit (String function)

2022-03-26T01:51:54Z

ATitelbaum: /* Examples */

{{Template:String:RegexSplit subtitle}}

<var>RegexSplit</var> repeatedly applies a regular expression, or "regex," to the method object string until it has tested the entire string. This splits the string into the substrings that are matched by the regex (the "separators") and the substrings that are not matched. By default, the method saves each unmatched substring as a separate item in the <var>Stringlist</var> result object.
The leftmost unmatched substring is the first item in the <var>Stringlist</var>, the next leftmost is the second item, and so on.

A simple application of <var>RegexSplit</var> is to extract only the data items from a string of comma-separated data items. If the specified regex is a comma, each of the resulting <var>Stringlist</var>
items will contain one of the data items.

The <var>Stringlist</var> that is returned by a default invocation of <var>RegexSplit</var> will contain at least as many items as there are instances of matched substrings. Upon each match, the input string characters preceding the matched ones (and since the previous matched ones) are saved as a <var>Stringlist</var> item. If there are consecutive matching substrings (no unmatched characters between the matching ones), the corresponding <var>Stringlist</var> item for the second matching substring is empty.

<var>RegexSplit</var> also has non-default alternatives that let you save the following in the <var>Stringlist</var>:
<ul><li>only the matched substrings
<li>both the matched and unmatched substrings
<li>only the substrings that are matched by capturing groups in the specified regex
<li>the unmatched substrings and the substrings matched by capturing groups</ul>

Within a regex, characters enclosed by a pair of unescaped parentheses form a "subexpression". A subexpression is a capturing group if the opening parenthesis is not followed by a
question mark (?).

<var>RegexSplit</var> uses the Regex [[Regex_processing#Regex_rules|rules]] of regular expression matching.

==Syntax==
{{Template:String:RegexSplit syntax}}

===Syntax terms===
<table class="syntaxTable">
<tr><th>%sl</th>
<td>A <var>Stringlist</var> object created and returned by the <var>RegexSplit</var> function. </td></tr>

<tr><th>string</th>
<td>The input string, to which the regular expression <var class="term">regex</var> is applied. </td></tr>
<tr><th>regex</th>
<td>A string that is interpreted as a regular expression and is applied in a matching operation to the method object string. </td></tr>

<tr><th><var>Options</var></th>
<td>This is an optional, but [[Notation conventions for methods#Named parameters|name required]], parameter supplying a string of single letter options, which may be specified in uppercase or lowercase, in any combination, and blank separated or not as you prefer. For more information about these options, see [[Regex_processing#Common_regex_options|"Common regex options"]].
</td></tr>

<tr><th><var>Add</var></th>
<td>The <var>Add</var> argument (name required) is a <var>RegexSplitOutputOptions</var> enumeration value (see below) that specifies what substrings of the input string <var class="term">string</var> to store in the output <var>Stringlist</var> <var class="term">%outList</var>.
</td></tr></table>

===RegexSplitOutputOptions enumeration===
The values of this [[Enumerations|enumeration]] are the following:
<table class="thJustBold">
<tr><th>Unmatched</th>
<td>Store only each unmatched substring and any empty substrings due to adjacent separators (consecutive matching substrings). For example, if the value of the <var>RegexSplit</var> parameter <var class="term">regex</var> is <code>#</code>, and <var class="term">string</var> is <code>C###D</code>, the <var>Unmatched</var> option adds four <var>Stringlist</var> items: <code>C</code>, two empty items, then <code>D</code>.

<var>Unmatched</var> is the default.</td></tr>

<tr><th>Matched</th>
<td>Store each matched substring only. Include those characters matched by capturing or non-capturing groups.</td></tr>

<tr><th>MatchedAndUnmatched</th>
<td>Store each matched and each unmatched substring in alternating <var>Stringlist</var> items. The first item contains the first unmatched substring, the second item contains the first matched substring, and so on, ending with the last matched substring and the last unmatched substring.</td></tr>

<tr><th>Captured</th>
<td>Store only those substrings matched by capturing groups in the <var class="term">regex</var> argument — as if <var>[[RegexCapture_(Stringlist_function)|RegexCapture]]</var> were applied repeatedly.</td></tr>

<tr><th>CapturedAndUnmatched</th>
<td>Store the following in alternating <var>Stringlist</var> items:
<ol type="a">
<li>Those substrings matched by capturing groups in <var class="term">regex</var>
<li> Each unmatched substring
</ol>

The first <var>Stringlist</var> item contains the first unmatched substring, if any; otherwise, it contains the substring captured by the first capturing group. The next item contains the substring captured by the next, if any, capturing group; otherwise, it contains the next unmatched string. And so on.</td></tr>
</table>

'''Note:''' As with all enumerations, the <var>ToString</var> method implicitly converts an enumeration value to a character string whose value is the name of the enumeration value. For more information about methods available to all enumerations, see [[Enumerations#Common enumeration methods|"Common enumeration methods"]]. 

==Exceptions==
<var>RegexSplit</var> can throw the following exception:
<dl>
<dt><var>[[InvalidRegex_class|InvalidRegex]]</var>
<dd>If the <var class="term">regex</var> parameter does not contain a valid regular expression. The exception object indicates the position of the character in the <var class="term">regex</var> parameter where it was determined that the regular expression is invalid, and a description of the nature of the error.
</dl>

==Usage notes==
<ul>
<li>It is strongly recommended that you protect your environment from regex processing demands on PDL and STBL space by setting, say, <code>UTABLE LPDLST 3000</code> and <code>UTABLE LSTBL 9000</code>. See [[Regex_processing#SOUL programming considerations|SOUL programming considerations]].

<li>The <var>String</var> intrinsic class <var>RegexSplit</var> function performs the same processing as the <var>Stringlist</var> class <var>[[RegexSplit_(Stringlist_function)|RegexSplit]]</var> function. The only differences are:

<ol>
<li>The target <var>Stringlist</var> for the <var>Stringlist</var> class method is its method object, whereas it is the function output for the <var>String</var> class method.
<li>The <var>Stringlist</var> class method appends to the target stringlist, the <var>String</var> class method creates a new <var>Stringlist</var>.
<li>The <var>String</var> class method has no status parameter, the <var>Stringlist</var> class method does.
<li>The only way the <var>String</var> class method has of indicating an error is via an exception.
</ol>

<li>Basically, <var>RegexSplit</var> divides a string into pieces, some of which match a regular expression and some of which don't. Although, you may be more interested in the pieces that are matched than the pieces that are unmatched, this documentation often refers to the matched pieces as "separators," which displays a bias towards the default paradigm of a comma-delimited list. In this case (which is equivalent to the method's <code>Add=Unmatched</code> option), the regex identifies the "commas," and the method extracts the "list elements" that remain. In the algorithm for extracting the list items in the default case:
<ol>
<li>The <var class="term">regex</var> makes its initial match in the input string; this is the first "comma" separator.
<li>The characters to the left of the matched substring (if none, then the null string) are the first "list element" unmatched piece.
<li>The <var class="term">regex</var> finds its next match, the next separator. The substring between the final character in the previous match and the first character in the current match (if none, then the null string) becomes the second unmatched piece.
<li>When the regex fails to make a next match, the entire substring remaining after the last match (if none, then the null string) becomes the last unmatched piece.
</ol>
Worth noting about this algorithm:
<ul>
<li>The matching by the <var class="term">regex</var> is not affected by any capturing strings in the <var class="term">regex</var>. Capturing, per se, is not involved in the default case.
<li>A "comma" (separator) is always assumed to be preceded and followed by a "list element" (unmatched piece). Consequently, the extracted <var>Stringlist</var> often contains at least one null item. Only a comma-delimited list of the form '<code>a,b,c</code>' (where there are no repeated, list-starting, or list-ending commas) results in a <var>Stringlist</var> with no nulls.
<li>There is always one more unmatched piece than matched, because the pieces must alternate (consecutive matched pieces are separated by an unmatched null), and they begin and end with an unmatched piece.
</ul>
<li>If a <var>RegexSplit</var> <var class="term">regex</var> contains multiple capturing groups and the <code>Add=Captured</code> option is used, each time the <var class="term">regex</var> matches in the input string, the number of <var>Stringlist</var> items added is the number of capturing groups. For example, if you have three capturing groups and you specify an <code>Add=Captured</code> option, the <var>Stringlist</var> item numbers always line up as follows:
1. First capturing group
2. Second capturing group
3. Third capturing group
4. First capturing group
5. Second capturing group
6. Third capturing group
7. First capturing group
8. Second capturing group
9. Third capturing group
...

If you are also capturing the non-matched pieces (<code>Add=CapturedAndUnmatched</code>), here is the item order:
1. First non-matched piece
2. First capturing group
3. Second capturing group
4. Third capturing group
5. Second non-matched piece
6. First capturing group
7. Second capturing group
8. Third capturing group
9. Third non-matched piece
10. First capturing group
11. Second capturing group
12. Third capturing group
...

<li>An empty item in the output <var>Stringlist</var> may represent consecutive separators in the input string (with <code>Add=Unmatched</code>).

</ul>

==Examples==
<ol>
<li>This example demonstrates how <var>RegexSplit</var> operating in default mode against a simple comma-delimited list assigns items to the result <var>Stringlist</var>.
UTABLE LPDLST 3000
begin
%str longstring
%regex longstring
%sl object stringlist
%i is float

%str = 'Barry,Mildred'
%regex = ','
%sl = %str:regexSplit(%regex)
[[PrintText statement|printText]] {~} is: {%sl:count}
for %i from 1 to %sl:count
printText %sl({%i}) is: {%sl(%i)}
end for
end

For the input string <code>Barry,Mildred</code> and for a comma (<code>,</code>) as the regex, the result is:
%sl:Count is 2
%sl(1) is: Barry
%sl(2) is: Mildred

For the input string <code>,Barry,Mildred</code> and the same regex, the result includes a null first item:
%sl:Count is 3
%sl(1) is:
%sl(2) is: Barry
%sl(3) is: Mildred

And similarly, the input string <code>Barry,Mildred,</code> and the same regex produce a null third item; and the input string <code>,Barry,Mildred,</code> and the same regex produces a null first item and fourth items.

<li>This example shows the utility of an <code>Add=Matched</code> option.
 ...
%str = ' Barry Mildred Jack Faust '
%sl = %str:regexSplit(' +')
printText ---------- Unmatched:
%sl:Print

%c is string len 10
%c = '[' with '5F':hexToString With ' ]+'
%sl = %str:regexSplit(%c, add=matched)
printText ---------- Matched:
%sl:Print
printText ----------*
...

The result shows that using an <code>Add=Matched</code> option along with a regex that matches the data you want to extract is an alternative that lets you avoid the nulls in the <var>Stringlist</var> output:
<nowiki>---------- Unmatched:

Barry
Mildred
Jack
Faust

---------- Matched:
Barry
Mildred
Jack
Faust
----------*
</nowiki>

<li>In the following example, an <code>Add=Captured</code> option is used with a regex that includes multiple capture groups to strip the labels but capture the data values of an input string. Using the CreateLines method as shown is a way to use <var>RegexSplit</var> against a <var>Stringlist</var>.
begin
%troops is object stringList
%tokens is object stringList

%troops = new
text to %troops
Name: Clegg
Rank: Corporal
Missing: Leg

Name: Ryan
Rank: Private
Missing: Brothers

Name: Bilko
Rank: Sergeant
Missing: Discipline
end text

%tokens= %troops:createLines:regexSplit( -
name: *(\S+)\s*Rank: (\S+)\s*Missing: *(\S+)', -
add=captured)
%tokens:print(,3)
end

This is the result:
5 Clegg
8 Corporal
3 Leg
4 Ryan
7 Private
8 Brothers
5 Bilko
8 Sergeant
10 Discipline

</ol>

==See also==
<ul>
<li>For information about additional methods that support regular expressions, see [[Regex_processing|"Regex Processing"]].
</ul>

{{Template:String:RegexSplit footer}}
[[Category:Regular expression processing]]

RegexSplit (String function)

2022-03-26T01:45:55Z

ATitelbaum: /* RegexSplitOutputOptions enumeration */

{{Template:String:RegexSplit subtitle}}

<var>RegexSplit</var> repeatedly applies a regular expression, or "regex," to the method object string until it has tested the entire string. This splits the string into the substrings that are matched by the regex (the "separators") and the substrings that are not matched. By default, the method saves each unmatched substring as a separate item in the <var>Stringlist</var> result object.
The leftmost unmatched substring is the first item in the <var>Stringlist</var>, the next leftmost is the second item, and so on.

A simple application of <var>RegexSplit</var> is to extract only the data items from a string of comma-separated data items. If the specified regex is a comma, each of the resulting <var>Stringlist</var>
items will contain one of the data items.

The <var>Stringlist</var> that is returned by a default invocation of <var>RegexSplit</var> will contain at least as many items as there are instances of matched substrings. Upon each match, the input string characters preceding the matched ones (and since the previous matched ones) are saved as a <var>Stringlist</var> item. If there are consecutive matching substrings (no unmatched characters between the matching ones), the corresponding <var>Stringlist</var> item for the second matching substring is empty.

<var>RegexSplit</var> also has non-default alternatives that let you save the following in the <var>Stringlist</var>:
<ul><li>only the matched substrings
<li>both the matched and unmatched substrings
<li>only the substrings that are matched by capturing groups in the specified regex
<li>the unmatched substrings and the substrings matched by capturing groups</ul>

Within a regex, characters enclosed by a pair of unescaped parentheses form a "subexpression". A subexpression is a capturing group if the opening parenthesis is not followed by a
question mark (?).

<var>RegexSplit</var> uses the Regex [[Regex_processing#Regex_rules|rules]] of regular expression matching.

==Syntax==
{{Template:String:RegexSplit syntax}}

===Syntax terms===
<table class="syntaxTable">
<tr><th>%sl</th>
<td>A <var>Stringlist</var> object created and returned by the <var>RegexSplit</var> function. </td></tr>

<tr><th>string</th>
<td>The input string, to which the regular expression <var class="term">regex</var> is applied. </td></tr>
<tr><th>regex</th>
<td>A string that is interpreted as a regular expression and is applied in a matching operation to the method object string. </td></tr>

<tr><th><var>Options</var></th>
<td>This is an optional, but [[Notation conventions for methods#Named parameters|name required]], parameter supplying a string of single letter options, which may be specified in uppercase or lowercase, in any combination, and blank separated or not as you prefer. For more information about these options, see [[Regex_processing#Common_regex_options|"Common regex options"]].
</td></tr>

<tr><th><var>Add</var></th>
<td>The <var>Add</var> argument (name required) is a <var>RegexSplitOutputOptions</var> enumeration value (see below) that specifies what substrings of the input string <var class="term">string</var> to store in the output <var>Stringlist</var> <var class="term">%outList</var>.
</td></tr></table>

===RegexSplitOutputOptions enumeration===
The values of this [[Enumerations|enumeration]] are the following:
<table class="thJustBold">
<tr><th>Unmatched</th>
<td>Store only each unmatched substring and any empty substrings due to adjacent separators (consecutive matching substrings). For example, if the value of the <var>RegexSplit</var> parameter <var class="term">regex</var> is <code>#</code>, and <var class="term">string</var> is <code>C###D</code>, the <var>Unmatched</var> option adds four <var>Stringlist</var> items: <code>C</code>, two empty items, then <code>D</code>.

<var>Unmatched</var> is the default.</td></tr>

<tr><th>Matched</th>
<td>Store each matched substring only. Include those characters matched by capturing or non-capturing groups.</td></tr>

<tr><th>MatchedAndUnmatched</th>
<td>Store each matched and each unmatched substring in alternating <var>Stringlist</var> items. The first item contains the first unmatched substring, the second item contains the first matched substring, and so on, ending with the last matched substring and the last unmatched substring.</td></tr>

<tr><th>Captured</th>
<td>Store only those substrings matched by capturing groups in the <var class="term">regex</var> argument — as if <var>[[RegexCapture_(Stringlist_function)|RegexCapture]]</var> were applied repeatedly.</td></tr>

<tr><th>CapturedAndUnmatched</th>
<td>Store the following in alternating <var>Stringlist</var> items:
<ol type="a">
<li>Those substrings matched by capturing groups in <var class="term">regex</var>
<li> Each unmatched substring
</ol>

The first <var>Stringlist</var> item contains the first unmatched substring, if any; otherwise, it contains the substring captured by the first capturing group. The next item contains the substring captured by the next, if any, capturing group; otherwise, it contains the next unmatched string. And so on.</td></tr>
</table>

'''Note:''' As with all enumerations, the <var>ToString</var> method implicitly converts an enumeration value to a character string whose value is the name of the enumeration value. For more information about methods available to all enumerations, see [[Enumerations#Common enumeration methods|"Common enumeration methods"]]. 

==Exceptions==
<var>RegexSplit</var> can throw the following exception:
<dl>
<dt><var>[[InvalidRegex_class|InvalidRegex]]</var>
<dd>If the <var class="term">regex</var> parameter does not contain a valid regular expression. The exception object indicates the position of the character in the <var class="term">regex</var> parameter where it was determined that the regular expression is invalid, and a description of the nature of the error.
</dl>

==Usage notes==
<ul>
<li>It is strongly recommended that you protect your environment from regex processing demands on PDL and STBL space by setting, say, <code>UTABLE LPDLST 3000</code> and <code>UTABLE LSTBL 9000</code>. See [[Regex_processing#SOUL programming considerations|SOUL programming considerations]].

<li>The <var>String</var> intrinsic class <var>RegexSplit</var> function performs the same processing as the <var>Stringlist</var> class <var>[[RegexSplit_(Stringlist_function)|RegexSplit]]</var> function. The only differences are:

<ol>
<li>The target <var>Stringlist</var> for the <var>Stringlist</var> class method is its method object, whereas it is the function output for the <var>String</var> class method.
<li>The <var>Stringlist</var> class method appends to the target stringlist, the <var>String</var> class method creates a new <var>Stringlist</var>.
<li>The <var>String</var> class method has no status parameter, the <var>Stringlist</var> class method does.
<li>The only way the <var>String</var> class method has of indicating an error is via an exception.
</ol>

<li>Basically, <var>RegexSplit</var> divides a string into pieces, some of which match a regular expression and some of which don't. Although, you may be more interested in the pieces that are matched than the pieces that are unmatched, this documentation often refers to the matched pieces as "separators," which displays a bias towards the default paradigm of a comma-delimited list. In this case (which is equivalent to the method's <code>Add=Unmatched</code> option), the regex identifies the "commas," and the method extracts the "list elements" that remain. In the algorithm for extracting the list items in the default case:
<ol>
<li>The <var class="term">regex</var> makes its initial match in the input string; this is the first "comma" separator.
<li>The characters to the left of the matched substring (if none, then the null string) are the first "list element" unmatched piece.
<li>The <var class="term">regex</var> finds its next match, the next separator. The substring between the final character in the previous match and the first character in the current match (if none, then the null string) becomes the second unmatched piece.
<li>When the regex fails to make a next match, the entire substring remaining after the last match (if none, then the null string) becomes the last unmatched piece.
</ol>
Worth noting about this algorithm:
<ul>
<li>The matching by the <var class="term">regex</var> is not affected by any capturing strings in the <var class="term">regex</var>. Capturing, per se, is not involved in the default case.
<li>A "comma" (separator) is always assumed to be preceded and followed by a "list element" (unmatched piece). Consequently, the extracted <var>Stringlist</var> often contains at least one null item. Only a comma-delimited list of the form '<code>a,b,c</code>' (where there are no repeated, list-starting, or list-ending commas) results in a <var>Stringlist</var> with no nulls.
<li>There is always one more unmatched piece than matched, because the pieces must alternate (consecutive matched pieces are separated by an unmatched null), and they begin and end with an unmatched piece.
</ul>
<li>If a <var>RegexSplit</var> <var class="term">regex</var> contains multiple capturing groups and the <code>Add=Captured</code> option is used, each time the <var class="term">regex</var> matches in the input string, the number of <var>Stringlist</var> items added is the number of capturing groups. For example, if you have three capturing groups and you specify an <code>Add=Captured</code> option, the <var>Stringlist</var> item numbers always line up as follows:
1. First capturing group
2. Second capturing group
3. Third capturing group
4. First capturing group
5. Second capturing group
6. Third capturing group
7. First capturing group
8. Second capturing group
9. Third capturing group
...

If you are also capturing the non-matched pieces (<code>Add=CapturedAndUnmatched</code>), here is the item order:
1. First non-matched piece
2. First capturing group
3. Second capturing group
4. Third capturing group
5. Second non-matched piece
6. First capturing group
7. Second capturing group
8. Third capturing group
9. Third non-matched piece
10. First capturing group
11. Second capturing group
12. Third capturing group
...

<li>An empty item in the output <var>Stringlist</var> may represent consecutive separators in the input string (with <code>Add=Unmatched</code>).

</ul>

==Examples==
<ol>
<li>This example demonstrates how <var>RegexSplit</var> operating in default mode against a simple comma-delimited list assigns items to the result <var>Stringlist</var>.
UTABLE LPDLST 3000
begin
%str longstring
%regex longstring
%sl object stringlist
%i is float

%str = 'Barry,Mildred'
%regex = ','
%sl = %str:regexSplit(%regex)
[[PrintText statement|printText]] {~} is: {%sl:count}
for %i from 1 to %sl:count
printText %sl({%i}) is: {%sl(%i)}
end for
end

For the input string <code>Barry,Mildred</code> and for a comma (<code>,</code>) as the regex, the result is:
%sl:Count is 2
%sl(1) is: Barry
%sl(2) is: Mildred

For the input string <code>,Barry,Mildred</code> and the same regex, the result includes a null first item:
%sl:Count is 3
%sl(1) is:
%sl(2) is: Barry
%sl(3) is: Mildred

And similarly, the input string <code>Barry,Mildred,</code> and the same regex produce a null third item; and the input string <code>,Barry,Mildred,</code> and the same regex produces a null first item and fourth items.

<li>This example shows the utility of an <code>Add=Matched</code> option.
 ...
%str = ' Barry Mildred Jack Faust '
%sl = %str:regexSplit(' +')
printText ---------- Unmatched:
%sl:Print

%c is string len 10
%c = '[' with '5F':hexToString With ' ]+'
%sl = %str:regexSplit(%c, add=matched)
printText ---------- Matched:
%sl:Print
printText ----------*
...

The result shows that using an <code>Add=Matched</code> option along with a regex that matches directly the data you want to extract is a successful alternative that also lets you avoid the nulls in the <var>Stringlist</var> output:
<nowiki>---------- Unmatched:

Barry
Mildred
Jack
Faust

---------- Matched:
Barry
Mildred
Jack
Faust
----------*
</nowiki>

<li>In the following example, an <code>Add=Captured</code> option is used with a regex that includes multiple capture groups to strip the labels but capture the data values of an input string. Using the CreateLines method as shown is a way to use <var>RegexSplit</var> against a <var>Stringlist</var>.
begin
%troops is object stringList
%tokens is object stringList

%troops = new
text to %troops
Name: Clegg
Rank: Corporal
Missing: Leg

Name: Ryan
Rank: Private
Missing: Brothers

Name: Bilko
Rank: Sergeant
Missing: Discipline
end text

%tokens= %troops:createLines:regexSplit( -
name: *(\S+)\s*Rank: (\S+)\s*Missing: *(\S+)', -
add=captured)
%tokens:print(,3)
end

This is the result:
5 Clegg
8 Corporal
3 Leg
4 Ryan
7 Private
8 Brothers
5 Bilko
8 Sergeant
10 Discipline

</ol>

==See also==
<ul>
<li>For information about additional methods that support regular expressions, see [[Regex_processing|"Regex Processing"]].
</ul>

{{Template:String:RegexSplit footer}}
[[Category:Regular expression processing]]

RegexReplace (String function)

2022-03-26T01:43:47Z

ATitelbaum: /* Examples */

{{Template:String:RegexReplace subtitle}}

The <var>RegexReplace</var> [[Intrinsic classes|intrinsic]] function searches a given string for matches of a regular expression, and replaces matches with, or according to, a specified replacement string.
The function stops after the first match and replace, or it can continue searching and replacing until no more matches are found.

Matches are obtained according to the [[Regex_processing#Regex_rules|rules]] of regular expression matching.

==Syntax==
{{Template:String:RegexReplace syntax}}

===Syntax terms===
<table class="syntaxTable">
<tr><th>%outString</th>
<td>A string set to the value of method object <var class="term">string</var> with each matched substring replaced by the value of <var class="term">replacement</var>. </td></tr>

<tr><th>string</th>
<td>The method object string, within which matches for <var class="term">regex</var> are sought.</td>

<tr><th>regex</th>
<td>A string that is interpreted as a regular expression and that is applied to the method object <var class="term">string</var> to find the one or more substrings matched by <var class="term">regex</var> </td></tr>

<tr><th>replacement</th>
<td>The string that replaces the substrings of <var class="term">string</var> that <var class="term">regex</var> matches. Except when the <code>A</code> option is specified (as described below for the <var>Options</var> argument), you can include markers in the <var class="term">replacement</var> value to indicate where to insert corresponding captured strings — strings matched by capturing groups (parenthesized subexpressions) in <var class="term">regex</var>, if any.

As in Perl, these markers are in the form <var class="term">$n</var>, where n is the number of the capture group, and 1 is the number of the first capture group. n must not be 0 or contain more than 9 digits. If a capturing group makes no matches (is positional, for example), or if there was no nth capture group corresponding to the <var class="term">$n</var> marker in a replacement string, the (literal) value of <var class="term">$n</var> is used in the replacement string instead of the empty string. <code>xxx$1</code> is an example of a valid replacement string, and <code>$0yyy</code> is an example of an invalid one. Or you can use the format <var class="term">$mn</var>, where m is one of the following modifiers:

<table class="syntaxNested">
<tr><th><var>U</var> or <var class="camel">u</var></th>
<td>Specifies that the specified captured string should be uppercased when inserted.</td></tr>

<tr><th><var>L</var> or <var class="camel">l</var></th>
<td>Indicates that the captured string should be lowercased when inserted.</td></tr>
</table>
The only characters you can escape in a replacement string are dollar sign (<code>$</code>), backslash (<code>\</code>), and the digits <code>0</code> through <code>9</code>. So only these escapes are respected: <code>\\</code>, <code>\$</code>, and <code>\0</code> through <code>\9</code>. No other escapes are allowed in a replacement string — this includes "shorthand" escapes like <code>\d</code> — and an "unaccompanied" backslash (<code>\</code>) is an error. For example, since the scan for the number that accompanies the meta-$ stops at the first non-numeric, you use <code>1$1\2</code> to indicate that the first captured string should go between the numbers 1 and 2 in the replacement string.
An invalid replacement string results in request cancellation.</td></tr>

<tr><th><var>Options</var></th>
<td>This optional, [[Notation conventions for methods#Named parameters|name required]], parameter is a string of single-letter options, which may be specified in uppercase or lowercase, in any combination, and blank separated or not, as you prefer. For more information about these options, see [[Regex_processing#Common_regex_options|Common regex options]].
</td></tr>
</table>

==Usage notes==
<ul>
<li>It is strongly recommended that you protect your environment from regular expression processing demands on PDL and STBL space by setting, say, <code>UTABLE LPDLST 3000</code> and <code>UTABLE LSTBL 9000</code>. See [[Regex processing#SOUL programming considerations|SOUL programming considerations]].

<li>Within a regular expression, characters enclosed by a pair of unescaped parentheses form a "subexpression." A subexpression is a capturing group if the opening parenthesis is not followed by a question mark (<tt>?</tt>). A capturing group that is nested within a non-capturing subexpression is still a capturing group.

<li>In Perl, <var class="term">$n</var> markers (<code>$1</code>, for example) enclosed in single quotes are treated as literals instead of as "that which was captured by the first capturing parentheses." <var>RegexReplace</var> uses the <code>A</code> option of the <var>Options</var> argument for this purpose.

<li>Matching of <var class="term">regex</var> may "succeed" but yet match no characters. For example, a quantifier like <code>?</code> is allowed by definition to match no characters, though it tries to match one. <var>RegexReplace</var> honors such a zero-length match by substituting the specified replacement string at the current position. If the global option is in effect, the <var class="term">regex</var> is then applied again one position to the right in the input string, and again, until the end of the string. The regex <code>9?</code> globally applied to the string <code>abc</code> with a comma-comma (<tt>,,</tt>) replacement string results in this output string: <code>,,a,,b,,c,,</code>.

<li>For information about additional methods that support regular expressions, see [[Regex processing]].
</ul>

==Examples==
<ol>
<li>In the following example, the regex <code>(5.)</code> is applied repeatedly (global option) to the string <code>5A5B5C5D5E</code> to replace the uppercase letters with their lowercase counterparts. The <code>$L1</code> <var class="term">replacement</var> value makes the replacement string equal to whatever is matched by the capturing group, <code>(5.)</code>, in the <var class="term">regex</var> (the <code>L</code> causes the lowercase versions of the captured letters to be used).
begin
%regex longstring
%inStr longstring
%replacement longstring
%outStr longstring
%opt string len 10

%inStr='5A5B5C5D5E'
%regex='(5.)'
%replacement='$L1'
%opt='g'
%outStr = %inStr:regexReplace(%regex, %replacement, options=%opt)
[[PrintText statement|printText]] OutputString: '{%outStr}'
end

The example result is:
OutputString: '5a5b5c5d5e'''Bold text''''

The non-capturing regex <code>5.</code> matches and replaces the same substrings as the capturing group <code>(5.)</code>, but <code>(5.)</code> is used above to take advantage of the self-referring marker for the replacement string, <code>$L1</code>, which is valid only for capturing groups.

<div id="greedy"></div>
<li>Say you want to supply end tags to items of the form <code><img foo="bar"></code>, converting them to <code><img foo="bar"></img></code>. You decide to use the following regex to capture <code>img</code> tags that have attributes:
(<img .*>)
And you use the following replacement string to replace the captured string with the captured string plus an appended <code></img></code>:
$1</img>
However, if the regex above is applied to the string <code><body><img src="foo" width="24"></body></code>, the end tag <code></img></code> is not inserted after the first closing angle bracket (<code>></code>) after <code>24</code> as you want. Instead, the matched string greedily extends to the second closing angle bracket, and the tag <code></img></code> is positioned at the end:
<body><img src="foo" width="24"></body></img>
One remedy for this situation is to use the following <var class="term">regex</var>, which employs a negated character class to match non-closing-bracket characters:
(<img [^>]*>)
This <var class="term">regex</var> does not extend beyond the first closing angle bracket in the target input string, and the resulting output string is:
<body><img src="foo" width="24"></img></body></ol>

==See also==
{{Template:String:RegexReplace footer}}
[[Category:Regular expression processing]]

SERVGA parameter

2022-03-25T21:21:16Z

ATitelbaum: /* Description */

{{Template:SERVGA parameter subtitle}}
==Summary==
<dl>
<dt>Default value
<dd>0
<dt>Parameter type
<dd>System
<dt>Where set
<dd>On User 0's parameter line
<dt>Related products
<dd>All
<dt>Introduced
<dd><var class="product">Model 204 V7.5</var>
</dl>

==Description==

The <var>SERVGA</var> parameter controls which server tables are allocated in the ATB-swappable-server area.
In Model 204 version 7.5, only <var>[[LNTBL_parameter|NTBL]]</var> and <var>[[LQTBL_parameter|QTBL]]</var> can be placed above-the-bar in a swappable area. 
Each server table to be allocated in that area is controlled by a bit in <var>SERVGA</var>. If the bit is on, the corresponding server table is allocated in the ATB-swappable-server area.

The bits are:
<table class="thJustBold">
<tr><th>Bit</th><th>Server table placed above the bar</th><th>Required version of Model 204</th></tr>
<tr><td><code>X'02000000'</code></td><td><var>[[LFTBL_parameter|FTBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'01000000'</code></td><td><var>[[LXTBL_parameter|XTBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00800000'</code></td><td><var>[[LGTBL_parameter|GTBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00400000'</code></td><td><var>[[LITBL_parameter|ITBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00200000'</code></td><td><var>[[LPDLST_parameter|PDL]]</var></td><td>7.9 or later</td></tr>
<tr><td><code>X'00100000'</code></td><td><var>[[LFSCB_parameter|FSCB]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00080000'</code></td><td><var>[[LSTBL_parameter|STBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00040000'</code></td><td><var>[[LVTBL_parameter|VTBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00020000'</code></td><td><var>[[LTTBL_parameter|TTBL]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00010000'</code></td><td><var>[[HTBFRS]]</var></td><td>7.6 or later</td></tr>
<tr><td><code>X'00004000'</code></td><td><var>[[LNTBL_parameter|NTBL]]</var> </td><td>7.5 or later</td></tr>
<tr><td><code>X'00002000'</code></td><td><var>[[LQTBL_parameter|QTBL]]</var></td><td>7.5 or later</td></tr>
<tr><td><code>X'00001000'</code></td><td>Server-swap table</td><td>7.9 or later</td></tr>
</table>

Note: These bits can also be set in <var>[[SERVNSA_parameter|SERVNSA]]</var> but should not be set in both <var>SERVGA</var> and <var>SERVNSA</var> — if they are, an error message is issued and the Online fails to come up.
<var>[[SERVGSZ parameter|SERVGSZ]]</var> is the amount of space in bytes required for the swappable above-the-bar server tables per server. The total amount of storage allocated for swappable above-the-bar server areas equals <var>SERVGSZ</var> rounded to 4K and multiplied by <var>[[NSERVS parameter|NSERVS]]</var>.
When sizing <var>SERVGSZ</var>, it should accommodate the largest swappable above-the-bar table sizes that might be needed.

Some server tables can alternatively be placed above-the-bar in a non-swappable area. This can be indicated with the <var>[[SERVNSA parameter|SERVNSA]]</var> and <var>[[SERVNSSZ parameter|SERVNSSZ]]</var> parameters.
In Model 204 7.9 and later, if PDL is placed above-the-bar it is recommended that LPDLST be set to its maximum value of 65536.

{{Template:SERVGA/SERVGSZ common}}

==See also==
<ul>
<li><var>[[ZPAGEOPT parameter|ZPAGEOPT]]</var></li>
</ul>

[[Category:System parameters]]
[[Category:Parameters]]

Defining the runtime environment (CCAIN)

2022-03-25T21:20:10Z

ATitelbaum: /* ATB storage for procedure pages */

==Overview==

CCAIN is the <var class="product">Model 204</var> control file used to define the runtime and user environments, and to control system operations.

When all the CCAIN input stream is read, <var class="product">Model 204</var> automatically terminates.

The discussion of the CCAIN file is presented in these pages:

<ul>
<li>The page you are reading describes the structure of the CCAIN file and the parameters used to define the runtime environment. Topics directly relating to the runtime parameters such as resource locking, disk buffers, and server areas are included.</li>

<li>[[Defining the user environment (CCAIN)]] discusses setting up the user environment.</li>

<li>[[Controlling system operations (CCAIN)]] discusses the commands used to control system operations. </li>
</ul>

Special considerations relating to <var class="product">Model 204</var> configurations or operating systems follow each discussion whenever appropriate.

==Structure of CCAIN==

The CCAIN file is divided into three sections, as described here:

===Runtime environment===
The runtime environment specifications line, which is known as the '''User 0 parameter line''', sets system characteristics and default user parameters during <var class="product">Model 204</var> initialization.


User 0, which acts as the system operator, is the name given to the input stream used by <var class="product">Model 204</var> initialization routines. The input stream is read from the CCAIN file and echoed on the CCAPRINT file. (For more information on CCAPRINT, see [[Obtaining User 0 output (CCAPRINT)]].) 


The User 0 parameter line, which is described in this page, includes:

<ul>
<li>System table sizes</li>

<li>I/O buffer sizes</li>

<li>Scheduler and performance options</li>

<li>Recovery options</li>
</ul>

<div id="preUser0Cmds">
====Pre-user 0 commands====


Parameters on the User 0 line are specified on the first line of the CCAIN input stream unless certain commands are used. The following commands are the only ones that can precede the User 0 parameter line:
<ul>


<li>[[*LOWER command|*LOWER]]</li>
<li>[[*UPPER command|*UPPER]]</li>
<li>[[APPDATE command|APPDATE]]</li>
<li>[[DEFINE DATASET command|DEFINE DATASET]]</li>
<li>[[DEFINE STREAM command|DEFINE STREAM]]</li>
<li>[[MSGCTL command|MSGCTL]]</li>
<li>[[UNICODE command|UNICODE]]</li>
</ul>
<ul>
<li>* (a comment) is also allowed, however, a blank must follow the <code>*</code>.</li>
<li>An all-blank line is also allowed.</li>
</ul>

In addition, starting with version 7.5, you can [[PRECCAIN|provide a common configuration]] for all <var class="product">Model 204</var> jobs by linking <code>PRECCAIN</code> into your ONLINE, IFAM1, and/or IFAM4 load modules. <code>PRECCAIN</code>, an assembler module which you modify and link into the load modules, contains <var class="product">Model 204</var> commands and parameters set and executed, respectively, during <var class="product">Model 204</var> initialization.

===User environments and definitions===
User environment definitions and specifications are set for each user on a separate line for:
<ul>
<li> Device type and terminal communication network</li>

<li> Compiler table defaults</li>

<li> Server configuration</li>

<li> Default output options</li>
</ul>

===System control commands===
System control commands are entered on successive lines. System commands include:
<ul>
<li> Recovery procedures</li>

<li> Suspension of User 0</li>

<li> Message control</li>

<li> Allocation and freeing of data sets</li>

<li> Definition of data sets and printers</li>

<li> End-of-data and end-of-job statements</li>

<li> [[SOUL]] requests</li>

<li> FLOD programs </li>
</ul>

==ONLINE data streams with CCAIN==

The following pages present z/OS, z/VSE, and z/VM examples for ONLINE data streams. The ONLINE examples illustrate the structure of CCAIN, including the runtime environment specifications line (User 0), user environment definitions and specifications, and system control commands.

A z/OS example for a BATCH204 run follows the ONLINE data streams.

===Parameter lines===

A CCAIN parameter line consists of one or more 80-column card images with parameter keywords and values in columns 1 through 71. 

If the line exceeds 71 characters, any non-blank character in column 72 indicates continuation to the next card image. 

The maximum length of the parameter area is controlled by the <var>LIBUFF</var> parameter, which is listed in the [[#Runtime environment specifications|Common runtime parameters]] and [[#Calculating fixed table size|Fixed server table values]] tables. 

<var class="product">Model 204</var> accepts comment lines or blank lines after the User 0 parameters in the CCAIN input stream. <var>IODEV</var> lines can have comments and blank lines before, between and after them.

As shown in the following examples, these rules governing parameter lines apply in all operating environments.

===z/OS JCL===

The following example shows z/OS JCL for an ONLINE data stream containing the CCAIN file.

 //M204ONLN JOB CLASS=A,MSGCLASS=A,NOTIFY=LEN
//RUN EXEC PGM=ONLINE,REGION=4096K,
// TIME=1440,PARM=('SYSOPT=149,LIBUFF=1024,LOBUFF=3000')
//STEPLIB DD DSN=M204.RLSE.LOADLIB,DISP=SHR
//CCAJRNL DD DSN=M204.JOURNAL,DISP=SHR
//CHKPOINT DD DSN=M204.CHKPOINT,DISP=SHR
//CCATEMP DD UNIT=3380,SPACE=(TRK,90),
// DISP=(NEW,DELETE)
//CCASNAP DD SYSOUT=A
//SYSMDUMP DD SYSOUT=A
//SYSUDUMP DD SYSOUT=A
//CCAAUDIT DD SYSOUT=A
//CCASTAT DD DSN=M204.CCASTAT,DISP=SHR
//CCAGRP DD DSN=M204.CCAGRP,DISP=SHR
//CLIENTS DD DSN=M204.FILE.CLIENTS,DISP=SHR
//VEHICLES DD DSN=M204.FILE.VEHICLES,DISP=SHR
//CLAIMS89 DD DSN=M204.FILE.CLAIMS89,DISP=SHR
//CLAIMS90 DD DSN=M204.FILE.CLAIMS90,DISP=SHR
//CCASERVR DD UNIT=3380,DISP=(NEW,DELETE),
// SPACE=(CYL,5,,CONTIG)
//CCAPRINT DD SYSOUT=A
//CCAIN DD * X Runtime
NUSERS=9,NSERVS=2,MINBUF=18,MAXBUF=1000, X environment
TERMBUF=5,NFILES=4,NDCBS=4,NDIR=4,SPCORE=50000, X definitions
IFAMBS=4000,LRETBL=800,VTAMNAME=M204,CRFSCHNL=M204FULL, X User0
CRIOCHNL=M204PROD,IFAMCHNL=IFAMPROD, X
RCVOPT=9,CPMAX=1,CPTIME=30,CPTO=5,CPTQ=5, X
LFSCB=7000,LGTBL=500,LSTBL=3000,LVTBL=200, X
LOUTPB=3000,NBKPG=5,OUTCCC=80,SERVSIZE=72456
IODEV=7,NOTERM=3,POLLNO=1,SERVSIZE=72456 User environment
IODEV=7,POLLNO=2 defining 8 users
IODEV=7,POLLNO=3 device types, and
IODEV=29,NOTERM=2,POLLNO=1 the communication
IODEV=29,POLLNO=2 network
IODEV=11,NOTERM=2,POLLNO=1
IODEV=11,POLLNO=2
IODEV=23
HALT 27,MODEL 204 IS UP AND RUNNING,3,EOD
EOD
*SLEEP 300
BROADCAST URGENT 1***SYSTEM GOES DOWN IN 5 MINUTES
*SLEEP 300
HALT 24,WAIT FOR USERS TO LOGOUT,3,EOJ
CHECKPOINT System operation
BUMP ALL control commands
*SLEEP 300
EOJ


===z/VSE JCL===

The following example shows z/VSE JCL for an ONLINE data stream containing the CCAIN File:

 // JOB ONLINE
// DLBL M204LIB,'M204.PROD.LIBRARY'
// EXTENT SYSnnn,...
// LIBDEF PHASE.SEARCH=(M204LIB.V210,PRD1.BASE)
// DLBL CCAJRNL,'MODEL204.CCAJRNL'
// EXTENT SYS001,balance of extent information
// DLBL CHKPNT,'MODEL204.CHKPOINT'
// EXTENT SYS001,balance of extent information
// DLBL CCATEMP,'MODEL204.CCATEMP',,DA
// EXTENT SYS001
// DLBL CCASRVR,'MODEL204.CCASERVR',,DA
// EXTENT SYS001
// DLBL CCASRV0,'MODEL204.CCASERV0',,DA
// EXTENT SYS001
// DLBL CCAGRP,'MODEL204.CCAGRP,,DA
// EXTENT SYS001
// DLBL CCASYS,'MODEL204.CCASYS',,DA
// EXTENT SYS001
// DLBL CCASTAT,'MODEL204.CCASTAT'
// EXTENT SYS001,balance of extent information
// ASSGN SYS001,X'cuu'
// ASSGN SYS002,X'108' ---- Local 3270 terminal
// ASSGN SYS003,X'019' ---- Local 3270 terminal
// ASSGN SYS008,PRINTER ---- Audit trail
*** INSERT LABEL INFORMATION AND ASSIGNMENTS
*** FOR USER DATABASE FILES HERE
// UPSI 10011011
// OPTION SYSPARM='LIB=512'
// EXEC ONLINE,SIZE=AUTO
NUSERS=6,MAXBUF=1000,LFSCB=4900, Runtime environment
LOUTPB=2400,NDIR=10,NSERVS=2,SERVSIZE=95000, definition (User 0)
RCVOPT=9
IODEV=35,INPUT=SYS002,SERVSIZE=95000 User environment
IODEV=35,INPUT=SYS003 defining 5 users,
IODEV=41,NOTERM=2,POLLNO=1 device types, and the
IODEV=11,POLLNO=2 communication network
IODEV=23,NOTERM=1,POLLNO=1
*** INSERT USER ZERO REQUESTS HERE
HALT 22,MODEL 204 IS AVAILABLE,3,EOJ System operation
CHECKPOINT
EOJ control commands
/*
/&


===CMS CCAIN file===

PAGESIZE=6184,NUSERS=5,NSERVS=2, X
NFILES=3,NDCBS=3,MINBUF=18,MAXBUF=1000, X
RCVOPT=9,SERVSIZE=95000
IODEV=41,NOTERM=2,POLLNO=1
IODEV=41,POLLNO=2
IODEV=39,NOTERM=1,POLLNO=1
IODEV=43,NOTERM=1,POLLNO=1
HALT 19,MODEL 204 IS NOW UP,10,END OF DAY
EOD
HALT 24,WAIT FOR USERS TO LOGOUT,3,EOJ
CHECKPOINT
EOJ


===z/VM ONLINE processing===

A <var class="product">Model 204</var> ONLINE environment is created in a z/VM environment by: 

<ul>
<li> Defining runtime parameters in the User 0 input file. </li>

<li> Executing the [[#CMS ONLINE command|CMS ONLINE command]], which causes an EXEC procedure to:

<ul>
<li> Execute a user-created EXEC procedure to define the file recovery environment</li>

<li> Invoke <var class="product">Model 204</var> to perform file recovery</li>

<li> Execute a user-created EXEC procedure to define the ONLINE environment</li>

<li> Invoke <var class="product">Model 204</var> to establish the ONLINE environment
</li>
</ul></li>
</ul>

Samples of the components necessary to invoke a <var class="product">Model 204</var> ONLINE environment follow. 

===EXEC procedure defining the ONLINE environment===

&TRACE OFF
&C FILEDEF * CLEAR
&C LABELDEF * CLEAR
&C FILEDEF CCAIN DISK DOCONLN CCAIN A
&C FILEDEF CCAPRINT DISK DOCONLN CCAPRINT A
&C FILEDEF CCAAUDIT J DSN M204 CCAAUDIT
&C FILEDEF CCASNAP PRINTER
&C FILEDEF CCAJRNL G DSN M204 CCAJRNL
&C FILEDEF CHKPOINT G DSN M204 CHKPNT
&C FILEDEF CCATEMP H DSN M204 CCATEMP
&C FILEDEF CCASERVR I DSN M204 CCASRVR
&C FILEDEF CCAGRP J DSN M204 CCAGRP
&C FILEDEF CCASYS J DSN M204 CCASYS
&C FILEDEF CCASTAT J DSN M204 CCASTAT
&C FILEDEF CARS J DSN M204 CARS
&STACK SYSOPT 146 LIBUF 1000


==CMS ONLINE command==


The ONLINE command brings up <var class="product">Model 204</var> in the service machine, allowing multiple users to log on. 

====Syntax====

The format of the ONLINE command is:

ONLINE [TEST] [NODCSS] [IFDIAL] [exec1] [BYPASS] [exec2] [BYPASS] [execargs]

Where:

<ul>
<li> TEST specifies that a TEST version of the <var class="product">Model 204</var> ONLINE module or shared segment, such as T204, is to be invoked. The default is the production version <code>M204</code>.</li>

<li> NODCSS specifies that shared segments are not to be used, even though they exist.</li>

<li> IFDIAL specifies that a single user IFDIAL connection is to be made (saved segments mandatory). The IFDIAL connection must be made on the main (nonrecovery) step.

A single-user IFDIAL EXEC procedure, SAMPDIAL, is supplied as part of the distributed material. Customize and install SAMPDIAL on an accessible minidisk. The M204 EXEC expects the IFDIAL EXEC to be named <code>SINGDIAL</code>. 

For more information, refer to the <var class="book">[http://docs.rocketsoftware.com/nxt/gateway.dll/RKBnew556%2Fmodel%20204%2Fprevious%20versions%2Fv7.4%2Fm204_installzvm_v74.pdf Rocket Model 204 Installation Guide for IBM z/VM, version 7.4]</var>.
</li>

<li><var class="term">exec1</var> specifies the name of the EXEC procedure that contains the ACCESS commands for the required minidisks and the file definition (FILEDEF or M204FDEF) commands for <var class="product">Model 204</var> recovery purposes. You must create the EXEC in accordance with the file requirements for the <var class="product">Model 204</var> ONLINE environment to be recovered.</li>

<li>BYPASS specifies not to use the EXEC procedure name in the ONLINE command and bypasses the recovery or Online steps, or both.</li>

<li><var class="term">exec2</var> specifies the name of the EXEC procedure that contains the ACCESS commands for the required minidisks and the file definition (FILEDEF or M204FDEF) commands for <var class="product">Model 204</var> regular online production files. You must create the EXEC in accordance with the file requirements for the <var class="product">Model 204</var> online environment to be initiated.</li>

<li><var class="term">execargs</var> are any user arguments, which are passed directly to the EXECs (1 and 2). </li>
</ul>

====Usage notes====

The EXEC can also include the Dictionary and Access/204 file definitions, if they are installed.

The following considerations apply to CMS online command processing:

<ul>
<li> If no operands are specified on the <var>ONLINE</var> command, the default name of the restart EXEC procedure is <code>M204REST</code>.</li>

<li> The default name of the Online EXEC procedure is <code>M204DEF</code>.</li>

<li> If one operand is specified, it is assumed to be the name of the Online EXEC procedure.</li>

<li> EXEC procedures invoked by <var>ONLINE</var> provide the necessary <var class="product">Model 204</var> parameters.</li>

<li> Required options must be placed in the stack (the &STACK command) as keyword-value pairs, separated by blanks.</li>

<li> If IFDIAL is specified, the main (nonrecovery) EXEC must provide only one parameter, the user program name, in the stack. The program name must be placed in the CMS stack before returning to the ONLINE EXEC.

The <var>IFSETUP</var> function (see the <var class="book">[[Media:M204 HLIReference V75.pdf|Rocket Model 204 Host Language Interface Reference Manual]]</var>) is used to send the CCAIN parameters via the user program. Neither CCAIN nor CCAPRINT are used for IFDIAL connections. </li>

<li> A single-user <var class="product">Model 204</var> interactive Online environment uses an EXEC procedure, <code>SAMPSING</code>, which is supplied as part of the distributed material. An IODEV statement is not required. (See [[#Server areas|Server areas]].)

Customize and install SAMPSING and its companion, SAMPSING CCAIN, on a generally accessible CMS minidisk. Name the customized files <code>SINGLUSR EXEC</code> and <code>SINGLUSR CCAIN</code> to sustain compatibility with the standard distributed user interfaces. </li>
</ul>

Return codes are evaluated as follows:

<ul>
<li> A return code of zero from any one of the EXEC procedures invokes <var class="product">Model 204</var>.</li>

<li> A return code of one (1) bypasses the invocation of <var class="product">Model 204</var>.</li>

<li> Any other return code is considered an error and causes the ONLINE EXEC to terminate immediately. </li>
</ul>

====Example====

ONLINE BYPASS DOCONLN

The parameters used in the sample above:

<ul>
<li> Use the version of <var class="product">Model 204</var> with saved segments</li>

<li> Provide no recovery arguments</li>

<li> Execute a user-written EXEC (DOCONLN) that defines the ONLINE environment </li>
</ul>

==CMS utilities and EXECs==

The "CMS utilities" table explains how to use CMS utility command modules relating to <var class="product">Model 204</var> in the CMS environment. None of the commands can be issued within the <var class="product">Model 204</var> environment. For information about syntax, refer to [[CMS utility command modules]]. 

<table>
<caption>CMS utilities</caption>
<tr class="head">
<th>
Utility
</th>
<th>
Purpose
</th>
<th>
Comments
</th>
</tr>

<tr>
<td>
M204APND
</td>
<td>
Concatenates file definitions.
</td>
<td>
A DDNAME of CLEAR removes all file definitions.
</td>
</tr>

<tr>
<td>
M204CMS
</td>
<td>
The interface between CMS and <var class="product">Model 204</var> that provides system services during execution of load modules, for example M204ONLN, M204IFM1, and M204UTILC.
</td>
<td>
 
</td>
</tr>

<tr>
<td>
M204FDEF
</td>
<td>
Creates file definitions for files on unaccessed variable format disks without accessing the resident disk.
</td>
<td>
An example defining the file CLI:

CP LINK MVS 201 201 MW WRITE
M204FDEF CLI 201 DSN M204 CLI
 </td>
</tr>

<tr>
<td>
M204LDEF
</td>
<td>
Specifies magnetic tape label information for tape volumes using the M204APND module.
</td>
<td>
Standard LABELDEF command parameters and options are used.
</td>
</tr>

<tr>
<td>
M204UTIL
</td>
<td>
Initializes, labels, allocates, erases, renames, and lists variable-format volumes.
</td>
<td>
Not recommended for space allocation on DOS or OS format disks owned by a guest operating system, because the catalog is not updated and does not work on indexed VTOCs. You can use IEFBR14 for space allocation and M204FDEF or a FILEDEF for access. 
</td>
</tr>

<tr>
<td>
M204XFER
</td>
<td>
Transfers control to the version of the M204CMS module that executes in a saved segment (DCSS).
</td>
<td>
M204XFER can also load a second DCSS containing the <var class="product">Model 204</var> program invoked by M204CMS.
</td>
</tr>
</table>

===Using the M204UTIL utility===

The M204UTIL utility uses RESERVE/RELEASE logic when updating the VTOC of a variable-format disk. You can use it to manipulate a volume being used by one or more <var class="product">Model 204</var> service virtual machines. 

Note: Erasing data sets that are in use by <var class="product">Model 204</var> on such volumes produces unpredictable results.

The "M204UTIL options" table lists the options that are available with M204UTIL. 

<table>
<caption>M204UTIL options</caption>
<tr class="head">
<th>
Allocation unit
</th>
<th>
Option
</th>
</tr>

<tr>
<td>
BLKSIZE
</td>
<td>
Block size
</td>
</tr>

<tr>
<td>
DSORG
</td>
<td>
Either PS or DA
</td>
</tr>

<tr>
<td>
LRECL
</td>
<td>
Record length
</td>
</tr>

<tr>
<td>
PRIMARY
</td>
<td>
CYL TRK BLK
</td>
</tr>

<tr>
<td>
RECFM
</td>
<td>
F/FA/FBA/V/VA/VBA/U
</td>
</tr>
</table>

You must specify the primary option. Other options are not mandatory. The operands and options are function-dependent.

For example, the first statement below initializes a temporary minidisk. The second creates a data set named DEV.SCRATCH.CCATEMP to be stored on it. The third statement erases the data set.

M204UTIL INITIAL 291 TMP291
M204UTIL ALLOCATE DEV SCRATCH CCATEMP 291 (PRIMARY 5 CYL)
M204UTIL ERASE DEV SCRATCH CCATEMP 291


===Using the M204MOUN EXEC to mount and dismount tapes===

When a tape must be mounted, the CMS interface to <var class="product">Model 204</var> invokes the EXEC procedure M204MOUN, passing the DDname, device address, volume serial number, volume sequence, and access type (READ or WRITE) as arguments.

The M204MOUN EXEC determines the status of the tape device, issues appropriate Control Program and CMS commands, and sends a message to the system operator for a tape mount and drive attachment to the service virtual machine, if necessary. 

Based on criteria at its disposal, the EXEC can reject the attempt to mount the tape. You can alter the M204MOUN EXEC defaults to meet site requirements.

When a tape volume is dismounted, the CMS interface to <var class="product">Model 204</var> invokes the EXEC procedure M204UNLD, passing the DDname, device address, volume serial number, volume sequence, access type (READ or WRITE), and file status (EOV or EOF) as arguments.
<ul>
<li> EOV (request another volume) indicates an entry at end-of-volume.</li>

<li> EOF (no further requests) indicates an entry due to end-of-file. </li>
</ul>

==BATCH204 JCL with CCAIN==

===Sample z/OS JCL for invoking a BATCH204 run===

<nowiki>//RUN EXEC PGM=BATCH204,REGION=1200K,
// TIME=10,PARM='SYSOPT=144'

//STEPLIB DD DSN=M204.LOADLIB,DISP=SHR
//CCAAUDIT DD SYSOUT=A
//CCASTAT DD DSN=M204.CCASTAT,DISP=SHR
//CCAJRNL DD DSN=M204.JOURNAL,DISP=SHR
//CCATEMP DD UNIT=3330,SPACE=(TRK,20),
// DISP=(NEW,DELETE)
//CCASNAP DD SYSOUT=A
//SYSMDUMP DD SYSOUT=A
//SYSUDUMP DD SYSOUT=A
//CENSUS DD DSN=M204.FILE.CENSUS,DISP=SHR
//CCAPRINT DD SYSOUT=A
//CCAIN DD *
PAGESZ=6184,NFILES=1,SNAPCTL=2
.
.
.
</nowiki>
STEPLIB points to the load module library where the <var class="product">Model 204</var> program is linked.

<ul>
<li> If the load module library is added to the LINKLIB concatenation, STEPLIB is not necessary.</li>

<li> If EXCPVR under z/OS is used, STEPLIB must be authorized. </li>
</ul>

==Runtime environment specifications==

You can specify <var class="product">Model 204</var> runtime environment parameters on the EXEC statement and on the User 0 parameter line of the CCAIN input stream. 

The following sections explain how to set EXEC parameters, discuss the most commonly used User 0 parameters, and detail procedures specific to z/VSE and z/VM environments.

===Specifying EXEC statement parameters===

The JCL EXEC statement includes the following parameters that affect the runtime environment (see the sample in [[#z/OS JCL|z/OS JCL]]). 

<ul>
<li>PGM specifies the <var class="product">Model 204</var> configuration.</li>

<li> TIME specifies how long <var class="product">Model 204</var> can run before being canceled by the operating system. Specify at least 10 seconds for system initialization and normal termination.

Under z/OS, if TIME is set to 1440, the operating system's automatic cancellation of the run is bypassed. If the <var class="product">Model 204</var> automatic shutdown facility is also bypassed, then <var class="product">Model 204</var> can run indefinitely until brought down by other means.</li>

<li> PARM sets various <var class="product">Model 204</var> parameters, including the <var>[[LIBUFF parameter|LIBUFF]]</var> parameter and the <var>[[SYSOPT parameter|SYSOPT]]</var> parameter, which is described in the [[#Setting the SYSOPT parameter|SYSOPT parameter options]] table.

Note: The value set for <var>LIBUFF</var> takes effect immediately, so you must set <var>LIBUFF</var> large enough to accommodate CCAIN.</li>

<li> REGION specifies the size of the memory area allocated for the <var class="product">Model 204</var> configuration. The next section explains how to estimate the value of this parameter. </li>
</ul>

===Calculating region size===

Consider the following factors when estimating region size for an Online system (z/OS) or z/VM machine size (CMS):

<ul>
<li> Size of the load module, which varies depending on the use of optional modules </li>

<li> Spare core (<var>SPCORE</var>) specification, the default is 8192 bytes

Increase the default, if you use deferred update, the FLOD exit (FLODXT) feature, directed output, or active subsystems. Additional memory is also required to open sequential data sets. The requirements for FLODXT are given under <var>SPCORE</var> in the [[#User 0 parameters|"Common runtime parameters" table]]. </li>

<li> Number of buffers used for each server

Four buffers for each server is the minimum requirement. Each buffer requires slightly more than one <var class="product">Model 204</var> page. The main memory required is dependent upon the <var>SERVSIZE</var> parameter setting. 

No work space is required. Under normal conditions, five active users or application threads can be serviced efficiently by one server. </li>

<li> Number of servers allocated

The size of each user's area is dependent on the settings of the compiler table parameters governing the size and complexity of the SOUL requests. The formula to calculate server area size is given in [[#Sizing user server areas|Sizing user server areas]].

In exceptional cases, processing needs might require a distinct server for each user.
</li>

<li> Storage overhead

Approximately 500 bytes per user and 700 bytes per file are required. The actual amount depends on the number of data sets and extents. 
</li>
</ul>

===Setting the SYSOPT parameter===

<var>SYSOPT</var> values, which can be summed, define actions taken during a run. Individual <var>SYSOPT</var> options are shown in the following table: 

<table>
<caption>SYSOPT parameter options</caption>
<tr class="head">
<th>Option </th>
<th>Specifies... </th>
</tr>

<tr>
<td>
128 </td>
<td>
Log for the CCAJRNL and the CCAAUDIT data sets </td>
</tr>

<tr>
<td>
64 </td>
<td>
Abend without a dump when the return code is nonzero </td>
</tr>

<tr>
<td>
32 </td>
<td>
Print lines relating to system initialization or IFAM function calls (RK lines) on the audit trail or journal </td>
</tr>

<tr>
<td>
16 </td>
<td>
Login is required </td>
</tr>

<tr>
<td>
8 </td>
<td>
Automatic disconnect operation in response to the LOGOUT command </td>
</tr>

<tr>
<td>
4 </td>
<td>
Execution of data definition commands within a particular run only through the File Management facility of Dictionary </td>
</tr>

<tr>
<td>
2 </td>
<td>
Existing permanent group file (CCAGRP) is required </td>
</tr>

<tr>
<td>
1 </td>
<td>
CCASYS file, which is required for subsystem applications </td>
</tr>
</table>

===SYSIPT logical unit in z/VSE===

In the z/VSE environment, CCAIN is replaced by the logical unit SYSIPT, a device-independent input reader. Typically, the SYSIPT logical unit is assigned to the same device as that used by the Job Control program for reading JCL. It is usually unnecessary to provide a z/VSE ASSGN statement for this logical unit. 

You must place CCAIN data in one of the following: 

<ul>
<li> Job stream immediately following the z/VSE EXEC Job Control statement:

// EXEC ONLINE,SIZE=AUTO
PAGESZ=6184,RCVOPT=1
.
.
.
.
/*
/&

For a full example, see [[#z/VSE JCL|z/VSE JCL]].</li>

<li> A disk data set. Use any utility that takes card input and writes it to a file of 80-byte unblocked records. You must also supply a DLBL and EXTENT for either CCAIN or IJSYSIN: </li>

<li> For CCAIN, the symbolic unit referenced on the EXTENT statement must be a programmer logical unit, for example, SYS022.</li>

<li> For IJSYSIN, the symbolic unit referenced on the EXTENT statement must be SYSIPT. </li>

<li> Procedure with the <code>DATA=YES</code> option on the CATALP statement.</li>
</ul>

===User 0 input file in the z/VM environment===

In z/VM, the User 0 output file is specified as a file stored on a minidisk. There are no restrictions on the choice of the CMS file identifier. For example:

FILEDEF CCAIN DISK DOCONLN CCAIN A

<var class="product">Model 204</var> also runs within a user's virtual machine in a single-user mode. In this case the file is defined with a FILEDEF command, similar to the example shown above, in the single-user EXEC. Runtime parameters are set up the same as for multiple users, except that the number of users is set to one (<code>NUSERS=1</code>).

===Stacking z/VM runtime parameters===

In a z/VM environment, the EXEC containing the FILEDEF commands can specify SYSOPT (and any other runtime parameters) before initialization by stacking the parameters:

&C FILEDEF CCAIN DISK DOCONLN CCAIN A
&C FILEDEF CCAPRINT DISK DOCONLN CCAPRINT A
.
.
.
*
*STACK THE PARM FIELD VALUE FOR MODEL 204
*
&STACK SYSOPT 128 LIBUFF 2048

===User 0 parameters===

The runtime environment specifications entered on the User 0 parameter line further define system options, user default values, and work area sizes.

Parameters common to many <var class="product">Model 204</var> configurations are summarized in the "Common runtime parameters" table. 

<table>
<caption>Common runtime parameters</caption>
<tr class="head">
<th>Parameter</th>
<th>Specifies...</th>
</tr>

<tr>
<td>
<var>[[CFRLOOK parameter|CFRLOOK]]</var>
</td><td>
Collect critical file resources conflict statistics.
</td></tr>

<tr><td>
<var>[[CPMAX parameter|CPMAX]]</var></td>
<td>Maximum number of checkpoints
</td></tr>

<tr><td>
<var>[[CPTIME parameter|CPTIME]]</var></td>
<td>
Checkpoint time intervals
</td></tr>

<tr><td>
<var>[[LENQTBL parameter|LENQTBL]]</var></td>
<td>
Number of entries in each user's resource enqueuing table. (See [[#Resource locking table|Resource locking table]] for sizing formulas.) The default is 6. 
</td></tr>

<tr><td>
<var>[[LIBUFF parameter|LIBUFF]]</var>
</td>
<td>
Length of the input buffer used for input lines from CCAIN or the user's terminal. <var>LIBUFF</var> must be three bytes longer than the longest line or record read into it. Longer lines are rejected with an error message. The default is 255.

If an input line is continued with a nonblank character, the number of characters in the original line and all continuations (not including continuation characters) must fit in the <var>LIBUFF</var> specification. </td></tr>

<tr><td>
<var>[[LOBUFF parameter|LOBUFF]]</var></td>
<td>
Length of the output buffer used for output lines to
CCAAUDIT, CCAPRINT, for a user's terminal, or for a directed output (USE) data set. LOBUFF can be reset on individual user parameter lines. The default is 256. The recommended value for SQL processing is 5000.
</td></tr>

<tr><td>
<var>[[LOGADD parameter|LOGADD]]</var></td>
<td>
Number of slots reserved for adding new password (CCASTAT) entries. The default is 0. 
</td></tr>

<tr><td>
<var>[[LOGFAIL parameter|LOGFAIL]]</var></td>
<td>
Action taken when the number of consecutive login failures exceeds the value of the <var>LOGTRY</var> parameter. Default is 0. 
</td></tr>

<tr><td>
<var>[[LOGONENQ parameter|LOGONENQ]]</var></td>
<td>
Use of unique user IDs for systemwide logons to a single ONLINE system. (See the <var>LOGONENQ</var> entry in the [[#userenvTable|"User environment control parameters" table]] to specify unique user IDs for specific terminals.) The default is 0. 

The Subsystem Management facility is not affected by LOGONENQ.
</td></tr>

<tr><td>
<var>[[LOGTRY parameter|LOGTRY]]</var></td>
<td>
Maximum number of login attempts allowed. The default is 0. 
</td></tr>

<tr><td>
<var>[[LRETBL parameter|LRETBL]]</var></td>
<td>
Length of each user's part of the record enqueuing table. (See [[#Resource locking|Resource locking]].) Default is 200.
</td></tr>

<tr><td>
<var>[[LRUTIM parameter|LRUTIM]]</var></td>
<td>
Disk page reference interval for references considered obsolete. Use <var>LRUTIM</var> to calculate DKRR statistics. The default is 0.
</td></tr>

<tr><td>
<var>[[MAXBUF parameter|MAXBUF]]</var></td>
<td>
Maximum number of file page buffers allocated below the bar during <var class="product">Model 204</var> initialization. (See [[#Disk buffers and Model 204 storage|Disk buffers and Model 204 storage]].) The default is 256.
</td></tr>

<tr><td>
<var>[[MINBUF parameter|MINBUF]]</var></td>
<td>
Minimum number of file page buffers allocated below the bar during <var class="product">Model 204</var> initialization. (See [[#Disk buffers and Model 204 storage|Disk buffers and Model 204 storage]].) The default is 18.
</td></tr>

<tr><td>
<var>[[NDCBS parameter|NDCBS]]</var></td>
<td>
Number of <var class="product">Model 204</var> file DCBs that can be used at any one time. The default is 10.
</td></tr>

<tr><td>
<var>[[NDIR parameter|NDIR]]</var></td>
<td>
Maximum number of <var class="product">Model 204</var> files that can be opened during a run. The default is 5.
</td></tr>

<tr><td>
<var>[[NFILES parameter|NFILES]]</var></td>
<td>
Maximum number of <var class="product">Model 204</var> files that can be open at any one time. Files remain open until an explicit CLOSE is issued or the session ends. The default is 2. 

<var>NFILES</var>, <var>NDCBS</var>, and <var>NDIR</var> specifications are automatically incremented during system initialization if <code>SYSOPT=2</code> (permanent group file). With the <var>VIEW</var> command, you can view parameter values during a run.
</td></tr>

<tr><td>
<var>[[NGROUP parameter|NGROUP]]</var>
</td>
<td>
Maximum number of file groups each user can have open at the same time. The default is 5.
</td></tr>

<tr>
<td>
<var>[[NSERVS parameter|NSERVS]]</var>
</td>
<td>
Number of servers. The default is <var>NUSERS</var> (see below). 
</td></tr>

<tr><td>
<var>[[NSUBTKS parameter|NSUBTKS]]</var></td>
<td>
Maximum number of pseudo subtasks that can be generated in a <var class="product">Model 204</var> run. The default is 4.
</td></tr>

<tr><td>
<var>[[NUMBUFG parameter|NUMBUFG]]</var></td>
<td>
Number of file page buffers allocated above the bar during <var class="product">Model 204</var> initialization. (See [[#Disk buffers and Model 204 storage|Disk buffers and Model 204 storage]].) The default is 0.
</td></tr>

<tr><td>
<var>[[NUSERS parameter|NUSERS]]</var></td>
<td>
Total number of SOUL users and IFAM2 or IFAM4 threads supported. The default is 1. The value of <var>NUSERS</var> consists of the total number of I/O device types (IODEVs) specified on the user parameter lines plus 1 for User 0. Online users are defined as terminal users with a particular type of terminal or as a host language thread, if IFAM is supported in the online region.
</td></tr>

<tr><td>
<var>[[PAGEFIX parameter|PAGEFIX]]</var></td>
<td>
Fixes areas in memory; can be useful in reducing paging traffic under z/OS or z/VSE. The default is X `0000000', which means nothing is page fixed.

For z/VSE only: A PAGEFIX request is valid only if real pages have been assigned by the ALLOC command. If ALLOC is 0 or if the total number of pages to be fixed exceeds the number of real pages, the PAGEFIX request fails. 
</td></tr>

<tr><td>
<var>[[RETRVKEY parameter|RETRVKEY]]</var></td>
<td>
PF key used to retrieve a previous terminal input line. 280 bytes of spare core is required for each user that has a defined retrieve key. The default is 0.
</td></tr>

<tr><td>
<var>[[SEQOPT parameter|SEQOPT]]</var>
</td>
<td>
Activation of the prefetch (look-ahead) feature for SOUL applications requiring entry order retrieval. Values are 1 (on) or 0 (off). The default is 0. Activation of SEQOPT requires resizing the <var>[[MAXBUF parameter|MAXBUF]]</var> parameter by using the formula: 

MAXBUF = NUSERS * (4 + 2 * [maximum FOR EACH RECORD loop nest level])


You can modify SEQOPT with the RESET command. 
</td></tr>

<tr><td>
<var>[[SERVSIZE parameter|SERVSIZE]]</var>
</td>
<td>
Size of each server area. The default, formerly 0, is 65536. (See [[#Server areas|Server areas]].) 
</td></tr>

<tr><td>
<var>[[SPCORE parameter|SPCORE]]</var></td>
<td>
Minimum amount of storage within the <var class="product">Model 204</var> address space to leave unallocated at the end of <var class="product">Model 204</var> initialization. You can set <var>SPCORE</var> in the EXEC statement PARM field or on the User 0 parameter line. The default is 8192. Spare core is used by: 
<ul>
<li> IFAM4 application program storage. The default is 12288. </li>

<li> Active subsystems. (See the formula given in [[System requirements for Application Subsystems#SPCORE size|SPCORE size]].)</li>

<li> <var>FILELOAD</var> or <var>FLOD</var> commands for tape input and deferred update output. When using the FLODXT feature, you must allocate 100 bytes for each FLODXT program.</li>

<li> Each defined retrieval key for previous terminal input (180 bytes per key). </li>
</ul>

In z/OS, a number of bytes of virtual storage equal to your <var>SPCORE</var> setting is reserved above the line, and the same number is reserved below the line. The BTB amount is ignored in Model 204 7.7 and higher if the <var>[[NUMBUFG parameter|NUMBUFG]]</var> parameter setting is greater than 0. </td>
</tr>

<tr><td>
<var>[[TIMESTOP parameter|TIMESTOP]]</var></td>
<td>
Amount of time (CPU milliseconds) before automatic termination processing begins or before initiation of commands or SOUL requests stops. TIMESTOP cannot be reset. The default is 1500.


If TIMESTOP is set to 0, the <var class="product">Model 204</var> timing facility's automatic shutdown is bypassed. If, in addition, the JCL EXEC statement parameter TIME is set to 1440, the z/OS automatic shutdown is bypassed, and <var class="product">Model 204</var> continues to run indefinitely until brought down by other means.
</td></tr>

<tr><td><var>[[XMEMOPT parameter|XMEMOPT]]</var></td>
<td>
<var class="product">Model 204</var> Cross-Memory Services facility used by Timer PC and IOS Branch Entry for z/OS systems, as well as using SUSPEND/RESUME instead of WAIT/POST for communication between <var class="product">Model 204</var> real subtasks. Coordinates with <var>XMEMSVC</var>.

It is also required for:

<ul>
<li> [[IOS Branch Entry]]</li>

<li> [[UL/DB2]]</li>

<li> [[Defining the user environment (CCAIN)#CRAM options for z/OS|CRAM-XDM]]</li>

<li> CPUID check (product license verification; prior to Model 204 7.5)</li>
</ul>

VIO is incompatible with IOSB and EXCPVR.
</td></tr>

<tr><td><var>[[XMEMSVC parameter|XMEMSVC]]</var></td>
<td>
SVC number of the <var class="product">Model 204</var> Cross-Memory Services module ([[Cross-memory facilities CRAM and M204XSVC#M204XSVC|M204XSVC]]). Installation of this module with an SVC is deprecated as of Model 204 7.5. <var>XMEMSVC</var> coordinates with <var>XMEMOPT</var>. The default is 0. 

M204XSVC is also used by:
<ul>
<li> IOS Branch Entry</li>

<li> UL/DB2</li>

<li> CRAM-XDM</li>

<li> CPUID check (product license verification; prior to Model 204 7.5)</li>
</ul>
</td></tr>
</table>

===z/VSE UPSI Job Control statement===

In a z/VSE environment:
<ul>
<li> Use the <var>SYSPARM</var> parameter of the z/VSE OPTION Job Control statement to specify a limited number of parameters that are normally specified on User 0 or any user's parameter lines, if the maximum length of the data does not exceed eight characters. For example, you can set the <var>LIBUFF</var> parameter with the following statement:
// OPTION SYSPARM='LIB=2048'
 </li>

<li> Set the <var>SYSOPT</var> value before initialization by using the z/VSE UPSI Job Control statement. Specify the value as a series of weighted UPSI bit values.

For example, including RK lines on the audit trail can be specified with the following statement, which is the equivalent of SYSOPT=32:

00100000

The "UPSI/SYSOPT settings" table summarizes the UPSI bit settings and <var>SYSOPT</var> equivalents: 

<table>
<caption>UPSI/SYSOPT settings</caption>
<tr class="head">
<th>UPSI setting</th>
<th>SYSOPT value</th>
<th>Meaning</th>
</tr>

<tr>
<td>
10000000</td>
<td>
128</td>
<td>
Generate audit trail and/or journal</td>
</tr>

<tr>
<td>
01000000</td>
<td>
64</td>
<td>
Abend without a dump when the return code is nonzero</td>
</tr>

<tr>
<td>
00100000</td>
<td>
32</td>
<td>
Include RK lines on the audit trail</td>
</tr>

<tr>
<td>
00010000</td>
<td>
16 </td>
<td>
Login required</td>
</tr>

<tr>
<td>
00001000</td>
<td>
8</td>
<td>
Automatic disconnect in response to the LOGOUT command</td>
</tr>

<tr>
<td>
00000100
</td>
<td>
4</td>
<td>
Restrict the use of data definition commands within a run</td>
</tr>

<tr>
<td>
00000010</td>
<td>
2</td>
<td>
Open CCAGRP for use of permanent file group</td>
</tr>

<tr>
<td>
00000001</td>
<td>
1</td>
<td>
Open CCASYS for use of application subsystem definitions</td>
</tr>
</table></li>
</ul>

==Resource locking==

To maintain data integrity, resource locking requests and reserves system and file resources for share (SHR) and exclusive (EXCL) use with other users. Data corruption is prevented by using linked lists of system and file resources. Conflict in the locking table results from attempts to lock exclusively on a file resource. 

Resource locking messages indicate a wait for a file, a conflict for a resource, or that the table is full and needs to be increased with the LENQTBL parameter.

Locking occurs on:

<ul>
<li> File resources, which are usually locked in SHR mode, such as:
<ul>
<li> File access</li>

<li> Record locking table</li>

<li> Table B and Group index</li>

<li> Tables C and D</li>

<li> Permanent procedures</li>

<li> Access Control Table </li>
</ul></li>

<li> System resources, which are usually locked in EXCL mode, such as:
<ul>
<li> Access to the CCASTAT file</li>

<li> Group definition table in CCATEMP</li>

<li> Updates to CCAGRP</li>

<li> Names of a file and subsystem</li>

<li> User defined resources</li>
</ul></li>

<li> Records </li>
</ul>

The following sections explain the resource locking tables and the details of resource locking on single and multiple CPUs.

===Record locking table===

The record locking table (also called: record enqueuing table), whose length is the product of the number of users (<var>[[NUSERS parameter|NUSERS]]</var>) and the length in bytes of one user's part of the table (<var>[[LRETBL parameter|LRETBL]]</var>), contains control information necessary to detect conflicts between users trying to update records simultaneously. 

You can issue a <code>MONITOR ENQ</code> command to determine current usage in the record locking table.

====Calculating allocated size of record locking table====

The amount of space required by a request is roughly proportional to the number of lists and <var>FIND</var> statements contained in the request. Each <var>FIND</var> statement or list requires about 46 bytes per file for files less than or equal to 300,000 records. Space requirements increase at the rate of 2.25 bytes per segment. The maximum value is 65,535.

====SYSOPT2=X'40'====

Record sets — found sets, including <var>FDWOL</var> found sets, sorted sets, lists, and LPU lists — are traced through entries in the record locking table. One entry is required for each segment (49,152 records) in the record set. These entries are CCATEMP page numbers. 

When the <var>[[SYSOPT2 parameter|SYSOPT2]]</var> X'40' bit is set, the entries contain 4-byte CCATEMP page numbers. Setting the X'40' bit provides a substantial increase in the number of simultaneous record sets that can be concurrently active in a given <var class="product">Model 204</var> run. Therefore, if you set the <var>SYSOPT2</var> X'40' bit, you should also at least double the <var>LRETBL</var> value.

<ul>
<li> When the <var>SYSOPT2</var> setting does not include X'40', then at any given time the bit maps corresponding to all users holding found sets of any kind must fit into CCATEMP pages designated as the small model page pool no matter how large CCATEMP has been allocated.</li>

<li> When the <var>SYSOPT2</var> setting does include X'40', the CCATEMP page restriction is removed and user found sets can be placed anywhere within CCATEMP. This includes both the small model page pool and the CCATEMP expansion area, allowing for the possibility of a greater number of concurrent found sets being held by all users.</li>
</ul>

===Resource locking table===

You can use the <var>[[$CenqCt]]</var> function to obtain information on the number of unused entries in the resource locking table (also called: resource enqueuing table). 

====Sizing the resource locking table====

For details on estimating the size of the resource locking table, see <var>[[LENQTBL parameter|LENQTBL]]</var>.

For z/OS, <var class="product">Model 204</var> allocates the resource locking table above the 16-megabyte line. 

===Multiple jobs running on one CPU===

Locking occurs at the file level when application files are shared between multiple jobs. 

File locking modes is EXCL when:

<ul>
<li>Files are opened from a SOUL thread or an IFAM1 thread that has file update privileges.</li>

<li>Files are opened from an IFAM2 thread or an IFAM4 thread that has allowed updates with thread update and file update privileges.</li>

<li>Command is entered to create or delete a permanent group.</li>
</ul>

File locking is SHR mode when:

<ul>
<li>Files are opened under any other circumstances than those listed above.</li>

<li>Files are opened in deferred update mode. Such files remain in SHR mode for the duration of the run.
Note:
Up to 192 files can be opened in deferred update mode.
An attempt to exceed 192 files results in an error message.</li>

<li>Files have the file recovery option (<var>FRCVOPT</var>) parameter set to include X'10'. Locking on an application file does not occur when it is closed, unless <var>FRCVOPT</var> is set to X'10'. <var>FRCVOPT</var> is discussed extensively in [[File integrity and recovery]]. </li>

<li>The <var class="product">Model 204</var> job uses the CCAGRP data set. </li>
</ul>

The following sections explain how locking conflicts are handled by <var class="product">Model 204</var>, and how data integrity is ensured when multiple jobs run on one CPU.

===Handling locking conflicts===

Locking conflicts for application files are handled first by the operating system and next by <var class="product">Model 204</var>. 

The operating system initially examines the disposition for all application files, as specified in the JCL for a job. If two jobs specify SHR for the same application file, the operating system allows both jobs access to the file. When the second job attempts to process the application file, <var class="product">Model 204</var> determines that another job poses a locking conflict.

<var class="product">Model 204</var> reads the first page of the file and examines the lock, which is located on that page. If a conflict is detected, <var class="product">Model 204</var> waits until the job's time limit is reached for the file to become available. Then, if the file is not available, <var class="product">Model 204</var> sends error messages to the operator's console and to the output device of the user who attempted to open the file.

Error messages are issued once every five minutes until the file becomes available, the job time limit for an online job is reached, or the maximum number of error messages for a batch job (<var>ERMX</var>) is reached.

Messages sent to the operator's console are:

<ul>
<li> For an Online job:

*** M204.0582: ACCESS TO FILE filename PREVENTED BY: jobname
*** M204.0584: FILE IS IN USE — filename
 </li>

<li>For a batch job the message sent from <var class="product">Model 204</var> to the operator:

*** M204.0582: ACCESS TO FILE filename PREVENTED BY: jobname

*** M204.0581: ENQ'ING TO SHR FOR FILE filename volname

*** M204.0582: ACCESS TO FILE filename PREVENTED BY: jobname

*** M204.0583: ENQ'ING TO EXCL FOR FILE filename volname
 </li>
</ul>

===Data integrity===

When multiple jobs run on the same CPU, data integrity is ensured by using:

<ul>
<li> Operating system locking and unlocking facility for shared application files and the system file containing file group definitions (CCAGRP).</li>

<li> Special lock stored in the system file containing user and file security information (CCASTAT).</li>

<li> Restriction on sharing the system file that provides space for user work tables (CCASERVR) and the system scratch file (CCATEMP) between jobs.
</li>
</ul>

===Multiple Model 204 versions running on separate CPUs===

The operating system file locking mechanism prevents concurrent updating and retrieval of data sets by jobs that run on separate CPUs. The RESERVE/RELEASE hardware feature restricts use of a device to a single CPU in the following ways:

<ul>
<li> Device containing a file's first data set can be reserved when control of the file is gained on one CPU and one of the following conditions is true:</li>

<li> File is opened in a <var class="product">Model 204</var> job for the first time.</li>

<li> File that was read-only is first opened for update.</li>

<li> Last updating user closes the file.</li>

<li> File is completely closed. </li>

<li> Device is released in each case after a single disk read and disk write have been performed.
</li>
</ul>

Each file contains a list of jobs that have control of the file. The list is read and updated only while the device is reserved. If control of the file cannot be obtained, then the list is not updated, and the list of jobs preventing access, with their system IDs, is sent to the operator. 

Each list entry contains the following information:

<ul>
<li> SMF system ID lock type (SHR or EXCL)</li>

<li> Job and step names</li>

<li> Date and time that the list entry was created</li>
</ul>

===How Model 204 resolves locking conflicts===

Locking conflicts can be handled automatically or by issuing the <var>[[ENQCTL command|ENQCTL]]</var> command. 

Conflicts are handled automatically in single CPU error cases where a
<var class="product">Model 204</var> job has a file open and either the operating system crashes or the <var class="product">Model 204</var> job is canceled. In these instances, the locking list in the file still shows the file as locked and the following process occurs: 

<ol>
<li> When a file is opened, the locking list is processed after the operating system enqueuing. If the operating system enqueuing succeeds, there is no conflicting job on the requesting job's system. Any conflicting list entries for the same system are obsolete.</li>

<li> If the locking request is exclusive, any list entries for the current system are eliminated as obsolete.</li>

<li> If the request is shared, any exclusive entries for the current system and any shared entry having the same system ID, job, and step names are eliminated.</li>
</ol>

===Locking conflicts between CPUs===

You can clear conflicts occurring between CPUs by issuing the <var>ENQCTL</var> command to interrogate or modify the status of a file's locking list. The following rules apply to the <var>ENQCTL</var> command: 

<ul>
<li> If an <var>ENQCTL</var> command is issued with only a file name, all list entries for the file are displayed.</li>

<li> If an <var>ENQCTL</var> command is issued with arguments, all the entries satisfying the arguments are deleted.</li>
</ul>

For example, if a system crash occurs for system S133, the operator at another system can issue the following command to remove all the locking list entries from CENSUS that were added by jobs running on system S133:

ENQCTL CENSUS S133

Indiscriminate use of the <var>ENQCTL</var> command can cause shared DASD integrity exposure through the removal of entries of active systems or jobs. 

===Shared DASD locking conflicts===

If a shared DASD locking conflict occurs, <var class="product">Model 204</var> sends an error message to the operator's console. Error messages are issued once every five minutes until the conflict is resolved and the file becomes available.

Messages sent to the operator's console are: 
<ul>
<li> For a batch job:
*** M204.0582: ACCESS TO FILE filename PREVENTED BY: jobname
*** M204.0584: FILE IS IN USE -- filename

Note:
Both of these messages increment the error count for the batch job. Because the batch OPEN command is aborted after the specified maximum number of errors (ERMX) is reached, <var class="product">Model 204</var> does not wait indefinitely for the conflict to be resolved. </li>

<li>For active system or job locking list entries deleted by using the ENQCTL command:
*** M204.0585: SHARED DASD ENQ LIST OVERLAID FOR
filename AT hh:mm:ss ON yy.ddd


The date and time identify the most recent update of the file. Used in conjunction with the operating system job logs, this information can determine the cause of the problem. 
</li></ul>

===z/VSE considerations===

The following considerations apply to resource locking in a z/VSE environment:
<ul>
<li> File locking is available for z/VSE releases that support LOCK and UNLOCK (SVC 110).</li>

<li> Multiple copies of <var class="product">Model 204</var> running in separate z/VSE systems cannot share any <var class="product">Model 204</var> database files.</li>
</ul>

===Resource locking in z/VM===

The following considerations apply to resource locking in z/VM environments:

<ul>
<li>CMS-format disks cannot be shared in read/write mode by multiple virtual or real machines. Any attempt at SHR access destroys the data.

The file allocation techniques that are used and the lack of support in CMS for access serialization prevent effective read/write file sharing.

Files on CMS-format disks do not require preallocation. The files are created automatically the first time they are referenced and continue to increase in size as more data is added. File size is restricted only by the available space defined on the minidisk.</li>

<li> Several virtual machines can share variable-format disks by using virtual RESERVE/RELEASE facilities.

RESERVE/RELEASE permits access to a volume restricted to a particular (real or virtual) access path. Because allocations are static in nature, a file can be read and written without further reference to the allocation information, unless secondary allocation functions are required.

Files on variable-format disks require preallocation. A primary allocation must be provided. Secondary extents can be specified to permit limited extension of the file. The file allocation information is recorded on a disk in the Volume Table of Contents (VTOC).

Some files on variable-format disks can be read/write shared by multiple <var class="product">Model 204</var> virtual machines while others cannot:
<ul>
<li>Files that can be shared are CCAGRP, CCAIN, CCASTAT, CCASYS, and <var class="product">Model 204</var> files. </li>

<li>Files that cannot be shared are CCAAUDIT, CCAJRNL, CCAPRINT, CCARF, CCASERVR, CCASNAP, CCATEMP, CHKPOINT, RESTART, USE data sets, and deferred update data sets. </li>
</ul></li>

<li>SHR mode access on a read-only device can cause data inconsistencies.

If a CMS user has SHR access to a <var class="product">Model 204</var> file on a read-only minidisk, SHR does not prevent another user from upgrading to the EXCL mode. </li>
</ul>

==Disk buffers and Model 204 storage==

The disk buffer pool holds pages from database files and from CCATEMP and CCAGRP. CCATEMP pages consist of found sets, sorted sets, backpage images, temporary procedures and other data structures. Database pages consist of pages from Tables FCT, A, B, C, D, E and X. Pages are read into and written from this buffer pool by the disk buffer manager which manages this resource using a least recently used algorithm. This pool of buffers is shared by all users.

<var class="product">Model 204</var> utilizes the following areas of storage, depending on the operating system architecture your site supports:

<ul>
<li> Below the line, for 24-bit storage for non-XA systems: z/OS, z/VM, and z/VSE</li>

<li> Above the line, for sites that support 31-bit storage for OS/390, XA, ESA, z/OS, z/VM, and z/VSE</li>

<li> Above the bar, for sites that support 64-bit storage for z/OS, z/VM, and z/VSE</li>
</ul>

The buffer pool consists of the following data structures: 
<ul>
<li> Disk Buffer Control Blocks: 160 bytes each, one per buffer</li>

<li> Hash cells: 16 bytes each. The number of hash cells allocated for each buffer is equal to the <var>[[HASHCELL parameter|HASHCELL]]</var> parameter value, which defaults to 3. </li>

<li> Page fix lists: 16 bytes per buffer</li>

<li> Buffers: 6184 bytes, plus 8-byte overrun detection area

The disk buffer overrun detection area, the space between each buffer in the disk buffer pool, is eight bytes of hexadecimal FF, so for each buffer, 6192 bytes is allocated. The total size of the buffer pool is then:

(NUMBUF + NUMBUFG) * (6192 + 160 + (16*HASHCELL) + 16)


<var>[[NUMBUF parameter|NUMBUF]]</var> is a viewable parameter that is equal to the number of buffers allocated above the line. <var>NUMBUF</var> is equal to or less than <var>[[MAXBUF parameter|MAXBUF]]</var>, depending on the amount of virtual storage available to the job.

<var>[[NUMBUFG parameter|NUMBUFG]]</var> is a settable and viewable parameter that is equal to the number of buffers allocated above the bar (ATB). If <var>NUMBUFG</var> buffers cannot be allocated in available above the bar storage, the run terminates.
Note: As of version 7.7 of Model 204, either
<var>NUMBUF</var> or <var>NUMBUFG</var> is 0: all disk buffer storage
is allocated BTB, or all is allocated ATB, depending on the setting of <var>NUMBUFG</var>.</li>
</ul>

===Using 31-bit storage===

When BTB buffers are being used (that is, <var>NUMBUFG</var> is 0 under version 7.7 or higher, or <var>NUMBUFG</var> is any value under Model 204 prior to 7.7) in systems that support 31-bit addressing, <var class="product">Model 204</var> disk buffers are allocated above the 16-megabyte line. This frees virtual storage for other data that must remain below the line, and it allows for the allocation of a larger buffer pool, since there is more virtual storage above the line. As the number of buffers increases, database pages can remain in memory for longer periods of time, and repeated reads (I/O) for the same pages are reduced.

<ul>
<li> If [[IOS Branch Entry]] (<var>[[XMEMOPT parameter|XMEMOPT]]</var> X'02') is used, control blocks, hash cells, and page fix lists are allocated above the line. </li>

<li> If IOS Branch Entry is not used, the disk buffer control blocks and hash cells (and page fix lists if EXCPVR is used) are allocated below the line. The buffers themselves are allocated above the line.</li>
</ul>

Note:
For information about controlling the allocation of hash cells per buffer pool page, see the <var>[[HASHCELL parameter|HASHCELL]]</var> parameter. 

A minimum number of BTB buffers is required to bring up an Online configuration:
<ul>
<li>Model 204 7.6 or lower:

The minimum number of buffers is equal to the result of the following calculation: 
NLRUQ * ((NSERVS + NSUBTKS) * MAXOBUF + 15)


If <var>[[MINBUF parameter|MINBUF]]</var> is set smaller than the above result, Model 204 resets <var>MINBUF</var> to the result value. 

If <var>MAXBUF</var> is smaller than <var>MINBUF</var> after the previous calculation, <var>MAXBUF</var> is reset to the value of <var>MINBUF</var>. 

Rocket Software recommends that you start with a minimum setting of <code>MAXBUF=10000</code> and monitor performance statistics to determine if that number is adequate. Generally, performance will improve as the size of the buffer pool increases. That will not be the case, however, if real storage is limited and system paging increases. Many sites are running with <code>MAXBUF=50000</code>, <code>100000</code>, or more.</li>

<li>Model 204 7.7 and higher:

If <var>NUMBUFG</var> is 0, only BTB buffers are used, and the minimum number of buffers calculation and the effects on <var>MINBUF</var> and <var>MAXBUF</var> are as described in the preceding item. 

If <var>NUMBUFG</var> is greater than 0, only ATB buffers are used, and the minimum number of ATB buffers is described in [[#Managing ATB storage with NUMBUFG|Managing ATB storage with NUMBUFG]].

</li>
</ul>

===Using 64-bit addressing and Above the Bar (ATB) storage===

In systems that support 64-bit virtual storage, you can place Table B and Table X pages (and other entities listed in [[Model 204 configurations and operating environments#Model 204 entities in above the bar storage|Model 204 entities in above the bar storage]]) in the above-the-bar buffer pool, or above the two gigabyte (2GB) address line. Pages that are not stored above the bar reside in the buffer pool above the line.

<blockquote class="note">Important:
<ul>
<li>For Model 204 version 7.4, you should ensure that your user-written and third-party $functions can work with non-swappable server tables in 64-bit storage above the bar.</li>

<li>For Model 204 version 7.5 and higher, any assembler language functions that you have written for use within Model 204 version 7.5 must be 64-bit compliant. [[Contacting Rocket Software Technical Support|Contact Technical Support]] if you need help with conversion of your functions.</li>

<li>For Model 204 version 7.7 and higher, activating usage of ATB buffers (setting <var>NUMBUFG</var> greater than 0) causes all buffer storage to be ATB and none to be BTB.

Prior to the gradual introduction of ATB storage for Model 204, storage was rather limited, so Model 204 took all the storage it could, returned <var>[[SPCORE parameter|SPCORE]]</var> bytes, and allocated the rest into the buffer pool.
If the resulting number of buffers was not between <var>MINBUF</var> and <var>MAXBUF</var>, the run terminated.
On more contemporary systems, virtual storage is abundant, making obsolete the concept of calculating the amount of storage (BTB) that will fit. 

The following documentation retains many remarks that apply to versions preceding 7.7 where ATB and BTB buffers might both be used together. </li>
</ul></blockquote>

<ul>
<li> In most cases, Table B pages constitute the biggest portion of all pages in the buffer pool. Moving Table B pages to an above-the-bar buffer pool lets <var class="product">Model 204</var> place more pages from all other tables in the below-the-bar buffer pool and thereby reduce I/O and CPU time to read and write pages to and from disk. </li>

<li> When a buffer is allocated above the bar, the corresponding disk buffer control blocks (one per buffer, 160 bytes each) and hash cells (three per buffer, 16 bytes each) are also allocated above the bar. This means there is no below the bar storage penalty for allocating above-the-bar buffers.</li>

<li> Having these two buffer pools rather than one improves <var class="product">Model 204</var> scalability by reducing MP collisions when using buffer pool resources. </li>

<li> Eight bytes added to the end of every buffer, above and below the bar, detect buffer overruns. The buffer size per page is 6192 bytes (or 6184 plus 8).</li>
</ul>

====Managing ATB storage with NUMBUFG====

Note: If you set <var>NUMBUFG</var> greater than zero to use storage above the bar, IBM requires that you set a limit on how much of that virtual storage each address space can use. This limit is called [[Model 204 configurations and operating environments#Set the IBM MEMLIMIT system option|MEMLIMIT]].

When <var>NUMBUFG</var> is set to a nonzero value, an above-the-bar buffer pool is allocated with <var>NUMBUFG</var> buffers. As of Model 204 7.7, this ATB buffer pool handles all disk buffer storage, and you get the exact amount of storage you specify with <var>NUMBUFG</var>. Before version 7.7 of Model 204, nonzero <var>NUMBUFG</var> activates an ATB buffer pool in addition to the BTB buffer pool.


If ATB and BTB buffer pools are both being used: 
<ul>
<li>The ATB and the BTB buffer pool are allocated with at least a minimum number of buffers, and the calculations for those minimums are very similar:

The ATB minimum, calculated by <var class="product">Model 204</var>, is:
[[NLRUQG parameter|NLRUQG]] * ((NSERVS + NSUBTKS) * MAXOBUF + 15)


If you set <var>NUMBUFG</var> to a lower value, it is reset to the calculated value.

The BTB minimum is:
[[NLRUQ parameter|NLRUQ]] * ((NSERVS + NSUBTKS) * MAXOBUF + 15)


If you set <var>MINBUF</var> to a lower value, it is reset to the calculated value.</li>

<li>Table B and Table X pages (and other entities listed in [[Model 204 configurations and operating environments#Model 204 entities in above the bar storage|Model 204 entities in above the bar storage]]) use the above-the-bar buffer pool. Those pages are not read into the below-the-bar buffer pool. Consequently, most sites can reduce the size of the below-the-bar buffer pool by the highwater mark of Table B pages currently resident in that buffer pool.</li>

<li>To quickly implement the above-the-bar feature, initially set <var>NUMBUFG</var> equal to your <var>MAXBUF</var> setting and leave <var>MAXBUF</var> at its current setting.</li>

<li>If <var>NUMBUFG</var> is greater than zero, the buffer hash pool is allocated above the bar. In addition, control blocks associated with ATB buffers are also allocated above the bar. <var>NUMBUFG</var> is limited to buffer pools of 4.2 terabytes or fewer.</li>

<li>To use the above-the-bar buffer pool in z/OS, IOS Branch Entry is required. This means <var>[[XMEMOPT parameter|XMEMOPT]]</var> must be set to include X'02'. You can explicitly exclude allocating above-the-bar buffers by setting <code>NUMBUFG=0</code>.

If <var>NUMBUFG</var> is greater than zero and <var>XMEMOPT</var> does not include X'02', the following message is issued, <var>NUMBUFG</var> is not reset, and the job terminates.

M204.2581: XMEMOPT=2 (IOS BRANCH) REQUIRED FOR NUMBUFG > 0


If you cannot get the number of buffers you requested, the job fails. 
</li>
</ul>

====Determining the NUMBUFG setting====

The number of buffers you want to allocate above the bar and below the bar is dependent on the mix of work that is being done on your system. See [[#Model 204 entities in above the bar storage|Model 204 entities in above the bar storage]] for a list of entities that can go above the bar. 

<ul>
<li> The <var>[[LDKBMWNG parameter|LDKBMWNG]]</var> parameter, which applies to above-the-bar buffers, corresponds to the <var>[[LDKBMWND parameter|LDKBMWND]]</var> parameter, which applies to below-the-bar buffers.</li>

<li> If <var>NLRUQG</var> is set greater than 1, the value of <var>LDKBMWNG</var> is rounded up to a multiple of <var>NLRUQ</var>. <var>LDKBMWND</var> has a minimum size of one (1).
</li>
</ul>

High values of <var>LDKBMWNG</var> might unnecessarily increase the number of writes done (measured by the DKWR statistic). Low values might cause excessive waiting for buffers (measured by the MAXIOX statistic). Rocket Software recommends starting values for <var>LDKBMWND</var> and <var>LDKBMWNG</var> at 10% of <var>NUMBUF</var> and <var>NUMBUFG</var>, respectively.

If you do not set <var>LDKBMWNG</var>, it is set to the same value as <var>LDKBMWND</var>.

====Designating non-swappable and swappable server space====

You can allocate designated server tables for each user in storage above the bar that will not be subject to server swapping. This feature enables you to divide the server storage into two parts: swappable and non-swappable. This makes the swappable part of the server storage smaller and faster to swap.

The <var>[[SERVNSSZ parameter|SERVNSSZ]]</var> and <var>[[SERVNSA parameter|SERVNSA]]</var> parameters control non-swappable server areas.
<ul>
<li><var>SERVNSSZ</var> (server non-swappable size) is the amount of space in bytes required for the above the bar server tables per user.
</li>

<li><var>SERVNSA</var> (server non-swappable areas) specifies the tables above the bar.</li>
</ul>
In order to store a table in ATB storage:
<ul>
<li>Increase the <var>SERVNSSZ</var> parameter by the corresponding table size.</li>

<li>Set the proper bit in <var>SERVNSA</var>; for example:
<ul>
<li>For FTBL, set the first byte to X'02', so the value of <var>SERVNSA</var> is X'02000000'.</li>

<li>For GTBL, set the second byte to X'80', so the value of <var>SERVNSA</var> is X'00800000'.</li>

<li>For NTBL, set the third byte to X'40', so the value of <var>SERVNSA</var> is X'00004000'.</li>

<li>For QTBL, set the third byte to X'20', so the value of <var>SERVNSA</var> is X'00002000'.</li>
</ul>
</ul>
Note: The settings for each server table above the bar are independent of each other. So if FTBL, GTBL, NTBL, and QTBL are all placed above the bar, <var>SERVNSA</var> should be set to X'02806000'.

The non-swappable server part can be used with server swapping done in storage or on disk. It can also be used when there is no server swapping (<var>NUSERS</var>=<var>NSERVS</var>) to make servers below the bar smaller by using the non-swappable server part above the bar and saving storage below the bar.

To obtain below-the-bar storage savings when the non-swappable server part is used, decrease the value of the <var>SERVSIZE</var> parameter by the <var>LFTBL</var> value used in calculating the server size.
Note: When a table such as FTBL is placed in the non-swappable server part above the bar, the <var>[[UTABLE command|UTABLE]]</var> command decreasing the table size will not free any storage in the regular server below the bar.

When using non-swappable ATB server space, each user will get <var>SERVNSSZ</var> bytes of ATB space, even if the thread is logged out or running resident requests.

For greater efficiency, Model 204 version 7.5 also provides swappable ATB server areas that can supplement or replace the non-swappable areas.

The <var>[[SERVGA parameter|SERVGA]]</var> and <var>[[SERVGSZ parameter|SERVGSZ]]</var> parameters control swappable server areas.
<ul>
<li><var>SERVGA</var> (server swappable areas) specifies the tables above the bar. Each server table is controlled by a bit in SERVGA.</li>

<li><var>SERVGSZ</var> (server non-swappable size) is the amount of space in bytes required for the swappable above-the-bar server tables per server. The total amount of storage allocated for swappable above-the-bar server areas equals <var>SERVGSZ</var> rounded to 4K and multiplied by <var>[[NSERVS parameter|NSERVS]]</var>.</li>
</ul>

<blockquote class="note">Note: In Model 204 V7.5 and higher, server tables can be in three different places:
<ul>
<li>The BTB swappable server area </li>
<li>The ATB swappable server area </li>
<li>The ATB non-swappable server area </li>
</ul>

But server tables cannot be in two places at the same time. For example, you cannot make servers swappable and non-swappable at the same time. If, for instance, you set the NTBL bit on for the <var>SERVNSA</var> parameter, and you also set it on for <var>SERVGA</var>, the result would be: 
M204.1399: Same server area defined for server above the bar and non-swappable server

However, you can make parts of your server swappable and other parts non-swappable: the non-swappable areas are like having <code>NUSERS=NSERVS</code>, but only for some tables. So if you had large STBLs and GTBLs, you might want to make them non-swappable to reduce server swap times. And if you had small TTBLs and ITBLs, you might want to put them in the swappable area. Any table in the above-the-bar SRVR (the swappable part of the server) is not taking up space in the below-the-bar server. The same is true for any table in the ATB-non-swappable part of the server. 

The goal is to reduce the requirement for below-the-bar server area so that you can increase <var>NSERVS</var> (as <var>NSERVS</var> increases, so does the need for below-the-bar virtual storage). 
</blockquote>

====ATB storage for APSY precompiled procedures====
The <var>[[RESPAGE parameter|RESPAGE]]</var> parameter activates the APSY [[Performance monitoring and tuning#Resident Request feature for precompiled procedures|precompiled procedures in storage]] feature using above-the-bar pages by specifying a number of 4K-byte operating system pages.

Specifying <var>RESPAGE</var> stores precompiled procedures as resident requests above the bar in the <var>RESPAGE</var> area. Using <var>RESPAGE</var> allows you to substantially increase the resident request area and decrease server swapping of QTBL and NTBL pages.

Also, when <var>RESPAGE</var> is greater than 0, <var>RESSIZE</var> is reset to -1, and no storage is allocated for <var>RESSIZE</var>.

<var>RESPAGE</var> is independent of <var>[[SERVNSA parameter|SERVNSA]]</var>. That is, if <var>RESPAGE</var> is greater than 0, NTBL and QTBL for resident requests will always be stored in the resident request area, outside of the non-swappable server areas, even if the <var>SERVNSA</var> bits specify NTBL or QTBL are to be stored in the non-swappable server areas. This avoids the duplication of storing NTBL and QTBL in two different locations. In other words, a resident request is only stored in the resident request area and nowhere else.

Storing resident requests above the bar is independent of tables above the bar. You can use a combination of resident requests and swappable servers ATB to reduce BTB-server sizes and thus increase the number of servers.

====ATB storage for EBM pages====

Existence Bit Map (EBM) pages reside in above the bar storage. 

Each <var class="product">Model 204</var> file contains one EBM page for each segment in a file. If a file has five segments, that means there are five EBM pages for that file. 

To allow more ATB buffers for the EBM pages, increase the <var>NUMBUFG</var> setting by a value that accommodates all the EBM pages for all files that might be open concurrently in your job. If in addition the version of Model 204 is lower than 7.7, BTB buffers are also being used, and you can reduce the <var>MAXBUF</var> parameter by the same amount you increased <var>NUMBUFG</var> for EBM pages.

====ATB storage for procedure pages====

Procedure text pages, located in Table D, are also eligible to reside in the ATB buffer pool. Each SOUL procedure is stored in one or more procedure text pages, the initial page of which is pointed to by the procedure dictionary. (Pages from the procedure dictionary, which is also stored in Table D, are read into the BTB buffer pool.)

In a development environment where procedure page volume is high and where <var>NUMBUFG</var> is allocated, you might need to increase <var>NUMBUFG</var> to accommodate the procedure text pages. In this case, if BTB buffers are also being used (Model 204 version less than 7.7), you can reduce the <var>MAXBUF</var> parameter by the same amount you increased <var>NUMBUFG</var>.

====ATB storage for screens and images====

Pages used for <var class="product">Model 204</var> <var>SCREEN</var> and <var>IMAGE</var> items reside in the buffer pool above the bar.

====ATB storage for hash table cells====

The hash table is used to locate pages in the buffer pool based on the file and page number. You control the number of hash cells allocated in the hash table with the parameter <var>[[HASHCELL parameter|HASHCELL]]</var>. If <var>NUMBUFG</var> is also set to a value greater than zero to allocate buffers above the bar, the hash cells are also allocated above the bar. 

====ATB storage for XmlDoc object pages====
As of Model 204 version 7.5, the CCATEMP pages used for <var>XmlDoc</var> objects use the ATB buffer pool, which might allow the BTB buffer pool to be reduced, perhaps providing more storage for server areas. It also might provide a reduction in CPU utilization, especially when the <var>[[TEMPPAGE parameter|TEMPPAGE]]</var> parameter is used to allocate CCATEMP in memory.

===Monitoring disk buffers===

To get started using ATB buffers in an environment where both ATB and BTB buffers will be used, you might want to implement <var>NUMBUFG</var>, then do more analysis later. A suggested starting value for <var>NUMBUFG</var> in this case is equal to your <var>MAXBUF</var> setting (while leaving <var>MAXBUF</var> at its current setting). Then use the [[MONITOR command: Disk buffers|MONITOR DISKBUFF commands]] to analyze the buffer pool utilizations. 

==Managing delayed disk updates==

The disk update process allows delayed disk updates, which avoids duplicate database writes in certain situations.

When a user completes an update unit for a file and there are no other update units active against that file, <var class="product">Model 204</var> writes the buffer to disk with all of the file's modified pages, marks the file as physically consistent, and issues a message stating that the disk update is complete. 

===Specifying delayed updates with the DKUPDTWT parameter===

The CCAIN parameter <var>[[DKUPDTWT parameter|DKUPDTWT]]</var> specifies delayed disk updates. The value of <var>DKUPDTWT</var> determines how many seconds a disk buffer containing a file's modified pages must have aged before it can be written to disk. 

When <var>DKUPDTWT</var> is zero, the default value, <var class="product">Model 204</var> writes all of the file's modified pages to disk at the end of the last in-flight update of the file. The user who completed the last in-flight update waits for this disk update process to complete and for the message stating that the disk update is complete. 

If <var>DKUPDTWT</var> is not zero, <var class="product">Model 204</var> delays the start of the disk update process for at least <var>DKUPDTWT</var> seconds, after which it may be the checkpoint pseudo subtask (CHKPPST) that performs the disk update. The user who completed the last in-flight update does not have to wait for the disk update process to complete. 

When <var>DKUPDTWT</var> is not zero, the CHKPPST and CHKPTIMR pseudo subtasks are started automatically at <var class="product">Model 204</var> initialization. An error message informs you if the <var>NSUBTKS</var> parameter is set too low to start these pseudo subtasks.

The maximum value of <var>DKUPDTWT</var> depends on the value of the <var>CPTIME</var> parameter. If <var>CPTIME</var> is nonzero, <var>DKUPDTWT</var> must be less than or equal to <code>CPTIME*30</code>. The absolute maximum value of <var>DKUPDTWT</var> is 60. 

The system manager can reset <var>DKUPDTWT</var> to zero while the online is running. It can be reset to a nonzero value as long as the CHKPPST and CHKPTIMR pseudo subtasks were started during <var class="product">Model 204</var> initialization.

===Handling delayed updates and CHKPPST===

The CHKPPST pseudo subtask plays a central role in handling delayed disk updates. When <var>DKUPDTWT</var> is set to 0, CHKPPST does the following:
<ol>
<li>Sleeps for <var>CPTIME</var> minutes.</li>

<li>Tries to quiesce updates, for up to <var>CPTQ</var>, plus <var>CPTO</var> seconds.</li>

<li>Takes the checkpoint, if all updates are quiesced.</li>
</ol>

If <var>DKUPDTW</var>T is greater than 0, CHKPPST has substantially more processing to perform: 

<ol>
<li>Sleeps for <var>DKUPDTWT</var> divided by four seconds.</li>

<li>Further processing depends on the value, rounded to the nearest integer of:
N = (CPTIME * 60) / (DKUPDTWT / 4)
</li>
</ol>

===Attempting a checkpoint===

If the number of wake-up calls since the last checkpoint is <var>N</var>, CHKPPST takes a new checkpoint, as follows:

<table>
<tr class="head">
<th>Attempt </th>
<th>Purpose </th></tr>

<tr>
<td>1.</td>
<td>Performs the disk update process on all files that are not being updated, regardless of how long since they were marked disk-update-needed. </td></tr>

<tr>
<td>2. </td>
<td>Attempts to deactivate Host Language Interface (HLI) updates for CPTQ seconds. Performs the disk update process on all files that are not being updated each time <var class="product">Model 204</var> determines that there are more HLI jobs to wait for.</td></tr>
<tr>
<td>3. </td>
<td>Attempts to deactivate all other updates for CPTO seconds. Performs the disk update process on all files that are not being updated each time <var class="product">Model 204</var> determines that there are more users to wait for. </td></tr>

<tr>
<td>4. </td>
<td>If all updates have been deactivated, performs the disk update process on all remaining files marked disk-update-needed. Otherwise, abandons the checkpoint attempt. </td></tr>

<tr>
<td>5. </td>
<td>Performs the disk update process again. Then takes the checkpoint. </td></tr>

<tr>
<td>6. </td>
<td>If the number of wake-up calls since the last checkpoint is not <var>N</var>, CHKPPST performs the disk update process on all files that have aged sufficiently — that is, marked disk-update-needed for at least DKUPDTWT seconds. </td></tr>
</table>

===Factors affecting disk update and checkpoint processing===

Several important factors affect the processing of disk updates and checkpoints:

<ul>
<li> Some disk updates might be interrupted by another thread's request for the file's UPDATE resource. Attempts 1, 2, 3, 4, and 6 might be interrupted by an attempt to start a new update or by a user doing a disk update as part of CLOSE FILE processing. In these cases, the following message is issued, and the next file is processed:
M204.0440: DISK UPDATED ABORTED
 </li>

<li> Attempts 2, 3, and 4 might be interrupted by the expiration of the waiting time set with <var class="product">Model 204</var> parameters CPTQ and CPTO, respectively. In these cases, the M204.0440 message is issued, all remaining files are bypassed, and the checkpoint is timed out.</li>

<li> If CPTQ and CPTO are zero (no time-out intervals specified) and the DKUPDTWT parameter is nonzero, updates are deactivated for as long as required to perform the disk update process for all available files. If you do not want checkpoint attempts to deactivate updates, set DKUPDTWT to zero.</li>

<li> If Attempts 2, 3, or 4 abort due to a CPTO or CPTQ time-out, the checkpoint time-out message indicates the name of the file being written at the time of the time-out:

M204.0843: CHECKPOINT TIMED OUT ON date/time UPDATING FILE file
</li>

<li> Attempt 5 cannot be interrupted once it begins. Therefore, CPTO and CPTQ intervals are not honored for a CLOSE FILE command that is blocking a checkpoint. This type of event is likely to be infrequent and of short duration.</li>

<li> The sleep intervals of CHKPPST are not adjusted by the amount of time required to perform Attempt 6. Therefore, checkpoints might be spaced out by more than CPTIME minutes. If this is a frequent problem, adjust CPTIME downward.</li>
</ul>

==Understanding file statistics==

For descriptions of the file statistics DKUPTIME, UPDTTIME, and PDNGTIME, see [[Performance monitoring and tuning#Disk buffer monitor statistics and parameters|Disk buffer monitor statistics and parameters]]. 

===Handling 64-bit statistics===

To support very long running <var class="product">Model 204</var> regions, Model 204 V7R1 included modifications to the capacity of statistical counters (by increasing the size of some statistics and also exploiting 64-bit processing where appropriate). For sites upgrading to V7R1 or later, in-house or third-party support applications that process statistical counters need to review the statistics generated:

<ul>
<li>As some of the statistics fields became double-words, check [[Using system statistics]] for a revised layout of the System, Final, and Partial statistics. Also, additional Disk Buffer Monitor, MP/204, and File statistics were updated.</li>

<li>Examine your in-house or third-party support applications for changes you need to make because of the increased length of some of the statistics. Make any changes necessary to your applications, then reassemble with the release upgrade.</li>

<li>If your in-house or third party support applications don't reference any of these double word statistics, you only need to reassemble your program with the updated offsets.</li>
</ul>

==Server areas==

Server areas are the internal work areas allocated to each user. Each area is divided into a fixed and variable portion. The fixed portion, which includes logical I/O buffers and user resettable parameters, is calculated by <var class="product">Model 204</var> at initialization. The variable portion can be changed dynamically with the <var>[[UTABLE command|UTABLE]]</var> command or the <var>[[IFUTBL (HLI function)|IFUTBL]]</var> IFAM function. 

===Sizing user server areas===

The default size of all user server areas is set on User 0's parameter line. If the default is used, the allocated server area is exactly large enough to contain the tables for each user specified on each user's parameter line. If <var>[[SERVSIZE parameter|SERVSIZE]]</var> is also specified on a particular user's parameter line, the default is overridden for that user.

The value of <var>SERVSIZE</var> must be as large as, or larger than, the user's aggregate table size. It is calculated by examining the user's server area requirements and monitoring the system statistics (described in [[ONLINE monitoring]]) that provide information about the installation work load. The following formula gives the approximate size:

SERVSIZE = fixed-table-size + variable-table-size

Where:

<ul>
<li><var class="term">fixed-table-size</var> represents settings, defined during initialization, that cannot be modified during the run. </li>

<li><var class="term">variable-table-size</var> represents settings that can be varied using the <var>UTABLE</var> command or its IFAM equivalent, <var>IFUTBL</var>.
</li>
</ul>

====SERVSIZE and server page alignment====

Servers and some server tables are always aligned on a 4K page boundary. In pre-7.4 releases, server and tables alignment took place only when <var>[[DSPOPT parameter|DSPOPT]]</var> had settings of bits X'01' or X'02', or when the <var>[[APSYPAGE parameter|APSYPAGE]]</var> parameter was indicated. 

If you used server alignment previously, there is no change in your server size requirements.

If you did not use server alignment previously, then you might notice an increase in server size that in the worst case could be up to 24528 bytes per server.

When you calculate server size, take into account that FSCB, HEAP, NTBL, QTBL, STBL, and VTBL are each rounded on a 4K page boundary, so in the worst case each area could require up to 4088 bytes of server space, compared to servers with no alignment in previous releases.

The following sections explain how server area sizing parameters are processed, which parameters determine fixed and variable table sizes, and the ranges of values these parameters can take.

===Initialization and error handling===

During initialization, each user, except User 0, is identified in the output before the user's parameter line is read. The aggregate size of each user's tables and the size of tables fixed during initialization are printed after the user's parameters are read. 

If errors are detected, they are reported and initialization continues whenever possible. If errors are detected during initialization, the run is canceled at the end of initialization. Error conditions in initializing the server cause the run to end immediately with a return code of 96. 

The results of user changes to the sizes of FTBL, GTBL, ITBL, and XTBL are discussed in the <var>UTABLE</var> command [[UTABLE command#Usage notes|Usage notes]]. 

===Calculating fixed table size===

Use the following formula to calculate fixed table size, the <var>[[FIXSIZE parameter|FIXSIZE]]</var> parameter value:

Fixed table size = 2520
+ ((MAXINCL+6) * 80)dwr
+ ((LAUDPROC + 9) * (MAXINCL-1))dwr
+ (LIBUFF + 4)
+ (LOBUFF + 5)dwr
+ (LOUTPB)dwr
+ (((NGROUP + 12) * (NRMTFILE + NFILES + 1)) + (NRMTFILE + 1))dwr
+ ((NORQS*3) + 2)dwr
+ (3 * ERRMSGL)


Each term of this formula that is followed by <code>dwr</code> must be double word rounded to the next multiple of eight. For example, if the value of <var>LOBUFF</var> is 500, the term <code>(LOBUFF + 5)</code> equals 505, which must be rounded to 512, the next multiple of 8. 

If <var>SYSOPT</var> is 1 or 2 (indicating CCASYS or CCAGRP), add 1 to the value of <var>NFILES</var> used in the formula. If <var>SYSOPT</var> is 3 (indicating both CCASYS and CCAGRP), add 2.

If any SQL threads are specified in CCAIN (IODEVs 13, 17, or 19), add 6712 bytes for C language work areas.

The "Fixed server table values" table, below, shows the minimum, maximum, and default values for parameters that affect fixed server table sizing. The rightmost columns show the relevant units of measure; for example, the maximum value of <var>NORQS</var> is 32767 entries (not bytes). The values of <var>LIBUFF</var> and <var>LOBUFF</var> may need to be increased for SQL processing. Recommended values are <code>LIBUFF=3000</code> and <code>LOBUFF=5000</code>.

<table>
<caption>Fixed server table values</caption>
<tr class="head">
<th>Parameter</th>
<th>Default</th>
<th>Max </th>
<th>Bytes/entries</th>
</tr>

<tr>
<td>
ERRMSGL</td>
<td>
80</td>
<td>
256</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LAUDPROC</td>
<td>
21</td>
<td>
253</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LIBUFF</td>
<td>
255</td>
<td>
32767</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LOBUFF</td>
<td>
256</td>
<td>
32767</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LOUTPB</td>
<td>
0</td>
<td>
3000</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
MAXINCL</td>
<td>
5</td>
<td>
40</td>
<td>
90+LAUDPROC bytes per INCLUDE level</td>
</tr>

<tr>
<td>
NFILES</td>
<td>
2</td>
<td>
16383</td>
<td>
Entries</td>
</tr>

<tr>
<td>
NGROUP</td>
<td>
5</td>
<td>
16383</td>
<td>
Entries</td>
</tr>

<tr>
<td>
NORQS</td>
<td>
5</td>
<td>
32767</td>
<td>
Entries</td>
</tr>

<tr>
<td>
NRMTFILE</td>
<td>
0</td>
<td>
16383</td>
<td>
Entries</td>
</tr>
</table>

===Calculating variable table size===

Use the following formula to calculate variable table size:

Variable table size = 96
+ ((HTLEN+5) * (MAXHDR + MAXTRL)) dwr
+ (LFSCB) dwr
+ (LFTBL) dwr
+ (LGTBL) dwr
+ (LHEAP) dwr
+ (LITBL) dwr
+ (LNTBL * 12)
+ (LPDLST +32) dwr
+ (LQTBL * 16)
+ (LSTBL) dwr
+ (LTTBL * 4)dwr
+ (LVTBL * 32)
+ (LXTBL) dwr


Each term of this formula that is followed by <code>dwr</code> must be doubleword rounded to the next multiple of eight. The "Variable server table values" table shows minimum and maximum values:

<table>
<caption>Variable server table values</caption>
<tr class="head">
<th>
Parameter</th>
<th>
Default</th>
<th>
Max </th>
<th>
Bytes/entries/lines</th>
</tr>

<tr>
<td>
HTLEN</td>
<td>
132</td>
<td>
32767</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LFSCB</td>
<td>
8 (as of V7.7) 
0 (7.6 or earlier) </td>
<td>
65528</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LFTBL</td>
<td>
1000</td>
<td>
30 million</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LGTBL</td>
<td>
288</td>
<td>
2 billion</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LHEAP</td>
<td>
0</td>
<td>
2 million</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LITBL</td>
<td>
8 (as of V7.7) 
0 (7.6 or earlier) </td>
<td>
32760</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LNTBL</td>
<td>
50</td>
<td>
32760</td>
<td>
12-byte entries</td>
</tr>

<tr>
<td>
LPDLST</td>
<td>
2600</td>
<td>
32760</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LQTBL</td>
<td>
400</td>
<td>
262,143</td>
<td>
16-byte entries</td>
</tr>

<tr>
<td>
LSTBL</td>
<td>
600</td>
<td>
16M</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
LTTBL</td>
<td>
50</td>
<td>
8190</td>
<td>
4-byte entries</td>
</tr>

<tr>
<td>
LVTBL</td>
<td>
50</td>
<td>
524287</td>
<td>
32-byte entries</td>
</tr>

<tr>
<td>
LXTBL</td>
<td>
1000</td>
<td>
32760</td>
<td>
Bytes</td>
</tr>

<tr>
<td>
MAXHDR</td>
<td>
5</td>
<td>
32767</td>
<td>
Lines</td>
</tr>

<tr>
<td>
MAXTRL</td>
<td>
5</td>
<td>
32767</td>
<td>
Lines</td>
</tr>
</table>

==Server tables==

Server tables are sections of the server area used by the SOUL compiler and evaluator to store all the information necessary to run a request. Some server tables are also used by the editor and HLI functions.

Each user has a copy of the server tables in the server. Table sizes are controlled by the parameters shown in the [[#User 0 parameters|"Common runtime parameters" table]]. Parameter settings on the user's parameter line affect the size of the servers and the region.

The "Summary of server tables" lists the server tables. Further information on individual tables is contained in the subsections that follow.

<table>
<caption>Summary of server tables</caption>
<tr class="head">
<th>Table</th>
<th>Contents</th>
</tr>

<tr>
<td>FSCB</td>
<td>Menu, screen, and image information</td>
</tr>

<tr>
<td>FTBL</td>
<td>File groups</td>
</tr>

<tr>
<td>GTBL</td>
<td>Global variables</td>
</tr>

<tr>
<td>ITBL</td>
<td>Dummy string and $READ responses</td>
</tr>

<tr>
<td>NTBL</td>
<td>Statement labels, list names, and variables</td>
</tr>

<tr>
<td>QTBL</td>
<td>Statements in internal form (quadruples)</td>
</tr>

<tr>
<td>RTBL</td>
<td>User privileges, class, and field level security information</td>
</tr>

<tr>
<td>STBL</td>
<td>Character strings</td>
</tr>

<tr>
<td>TTBL</td>
<td>Temporary work pages</td>
</tr>

<tr>
<td>VTBL</td>
<td>Compiler variables</td>
</tr>

<tr>
<td>XTBL</td>
<td>Procedure security</td>
</tr>
</table>

===Full-screen buffer table (FSCB)===

The full-screen buffer table (FSCB) stores menu, screen, and image definitions, in addition to the values of screen variables and image data blocks. FSCB space is reused by each logical menu definition, logical screen definition, or block definition.

The FSCB must be large enough to hold the largest screen, image, or menu definition. The following space is required:

<ul>
<li>144 bytes of fixed overhead for every menu, including the menu title</li>

<li>144 bytes for each menu prompt</li>

<li>432 bytes of fixed overhead for the first panel of every screen:</li>

<li>144 bytes for each subsequent panel, including the screen title</li>

<li>32 bytes for each screen prompt and input item</li>

<li>32 bytes for every screen line containing at least one input item</li>

<li>80 bytes for each defined screen line, including skipped lines</li>

<li>Additional space for automatic validation, including:</li>

<li>2 or 4 bytes for each automatic validation option</li>

<li>256 bytes for the VERIFY command when a particular character set is used in a compiled screen for the first time</li>
</ul>

Additional occurrences of the same character set do not add extra space. ONEOF and character RANGE store each character string plus one byte for each string's length.

<ul>
<li>8 bytes for each number in a NUMERIC RANGE statement (16 bytes for each range pair)</li>

<li>Space for every block used in image definition</li>
</ul>

The amount of required space is computed as the sum of the specific [[SOUL]] statements, clauses, and items. For a complete list of these values, see [[Full-screen feature]]. 

===File group table (FTBL)===

Data structures particular to file groups are stored in FTBL. FTBL entries are:

<ul>
<li> Sixty-two-byte fixed size entry plus six bytes for each file in the group definition.

This entry is allocated each time a group is opened (explicitly by the OPEN command or implicitly for an ad hoc group) and is released when the group is closed.</li>

<li> Variable entry, consisting of nine fixed-bytes plus a number of bytes equal to the length of the field name plus 11 bytes per file in the group

This entry is created for collecting field-name codes and properties during a SOUL request. An entry is allocated each time a new field name is encountered in the request. The field entries are deleted at every END statement (including END MORE). </li>
</ul>

When the Inverted File Access Method (IFAM) is used: 

<ul>
<li>Host Language threads use FTBL under the same circumstances as SOUL.</li>

<li>Field entries are not deleted until the group is closed or until IFFNSH is called. </li>

<li>Increase the total FTBL requirement by NGROUP times four bytes.
</li>
</ul>

For Parallel Query Option/204 server nodes, the required size of FTBL increases by eleven bytes times the total number of group members for temporary scattered groups opened at the client and containing a member on the server node.

For information on using FTBL in 64-bit storage, see [[#Using 64-bit addressing and Above the Bar (ATB) storage|Above the bar storage]].

===Understanding the global variable table (GTBL)===

GTBL contains information about:

<ul>
<li>Global variables </li>

<li>Global images, screens, and menus</li>
</ul>

GTBL entries are created by the <var>$Incrg</var>, <var>$Getg</var>, and <var>$Setg</var> functions. When a global variable is redefined, its old entry is deleted from GTBL and a new entry is added. 

In addition, a 32-byte trailer stores information about offsets. 

For information on using GTBL in 64-bit storage, see [[#Using_64-bit addressing and Above the Bar .28ATB.29 storage|Above the bar storage]].

====Clearing GTBL entries====

The <var>CLEARGO</var> command deletes all images, screens, and menus. You can also use the <var>CLEAR GLOBALS</var> statement to delete selected types of GTBL entries. For details, see [[Global features]].

====Space allocation====

The space allocation for a global variable includes:

<ul>
<li>4 bytes indicating the length of GTBL</li>

<li>1 byte for the length of the variable name</li>

<li>Variable name</li>

<li>1 byte for the length of the current name</li>

<li>Current value </li>
</ul>


Global images, screens, and menus require space for a 20-byte header in addition to the size of the object.


When allocating GTBL space, always remember to add 32 bytes for the trailer.


The minimum length of GTBL is 40 bytes (X'28').

====Improving global variable processing====

You can improve global variable processing by setting the <var>[[FASTGLOB parameter|FASTGLOB]]</var>, <var>[[GTBLPCT parameter|GTBLPCT]]</var>, and <var>[[GTBLHASH parameter|GTBLHASH]]</var> parameters. See [[Global features]] for a discussion of how these parameters affect performance. 

===Dummy string and $READ table (ITBL)===

ITBL holds dummy string and $READ responses that are entered as arguments to an INCLUDE statement or command.

The space allocation for an ITBL entry includes:

<ul>
<li>Argument strings, including delimiters, which are saved as they are entered</li>

<li>4 bytes of overhead for each saved string</li>
</ul>

Space taken by a string is released when the included procedure is executed.

===Labels, names, and variables table (NTBL)===

NTBL holds labels, names, and variables. Each entry is allocated 12 bytes. NTBL has two entries for each first occurrence of a COMMON declaration.

NTBL has one entry for each of the following elements: 

<ul>
<li>Statement label</li>

<li>List name</li>

<li>%variable</li>

<li>Image, menu, and screen variable</li>

<li>Partner process opened by a request</li>

<li>Additional <var>COMMON</var> declaration</li>

<li>Unlabeled <var>Find</var></li>

<li><var>For Each Value</var> statement</li>

<li><var>For</var> statement with the <var>In Order</var> clause</li>

<li>Sequential or VSAM file opened simultaneously</li>
</ul>

Most NTBL entries are preserved by the <var>MORE</var> command, except for the unlabeled <var>Find</var> and secondary <var>For</var> entries, which are deleted.

A host language thread requires NTBL entries for list names, compilation names, and variables. 

<var>FLOD</var> uses NTBL entries for tags and index registers. The size of NTBL determines the highest tag or index that can be specified. In this case, NTBL must be at least 12 bytes multiplied by the highest tag or index size desired. 

You need not allow extra space for runtime NTBL storage used during request evaluation of an <var>Open Dataset</var>, <var>Open External</var>, or <var>Open Terminal</var> statement. Use the compiler highwater mark to set LNTBL. 

For information on using NTBL in 64-bit storage, see [[#Using 64-bit addressing and Above the Bar .28ATB.29 storage|Above the bar storage]].

===Internal statement table (QTBL)===

QTBL holds internal <var class="product">Model 204</var> instructions that result from compilation of each internal statement. After compilation, the entries in QTBL drive the evaluator. QTBL is emptied by <var>End</var> and <var>End More</var> statements.

The Editor formats all of QTBL into 16-byte entries, which provide a map of text being edited. The number of entries used depends on the number and position of insertions and deletions, not on the amount of text.

Quadruple (QTBL) entries generated by SOUL statements vary in number and size. The "Sample QTBL entries" table, below, shows typical values for each entry. For more QTBL information, see [[Large request considerations#QTBL (internal statement table)|QTBL (internal statement table)]]. 

You can reduce server I/O by allowing users executing shared precompiled procedures to use a shared copy of QTBL.

For information on using QTBL in 64-bit storage, see [[#Using 64-bit addressing and Above the Bar .28ATB.29 storage|Above the bar storage]].

===QTBL and IFAM processing===

When using the Inverted File Access Method (IFAM):

<ul>
<li><var>IFFIND</var>, <var>IFCOUNT</var>, and list manipulations build quadruples so that the SOUL evaluator routines can be used.</li>

<li><var>IFGET</var>, <var>IFMORE</var>, and <var>IFPUT</var> build field name lists in QTBL.</li>

<li>Other calls are evaluated directly.</li>
</ul>

IFAM's use of QTBL and the effect of the compiled IFAM feature are discussed in detail in [[HLI: Model 204 tables#Internal statements.2Fquad table .28QTBL.29|Internal statements/quad table (QTBL)]] and [[HLI: IFSTRT processing#Using the compiled IFAM facility|Using the compiled IFAM facility]]. 

===QTBL and FLOD processing===

FLOD builds its own quadruples (flodruples) in QTBL. Flodruple sizes vary, but most are 20 bytes or less. 

Major exceptions are:

<ul>
<li>Read-and-load-field flodruple, which expands four bytes for each entry in a translation table</li>

<li>CASE statement, which requires eight bytes for each comparison string
</li>
</ul>

===Sample QTBL entries===
"Sample QTBL entries" shows typical QTBL entry sizes for various SOUL statements and program structures: 

<table>
<caption>Sample QTBL entries</caption>
<tr class="head">
<th>
SOUL statement</th>
<th>
QTBL entry size</th>
</tr>

<tr>
<td>
$functions</td>
<td>
16 + 3 per argument</td>
</tr>

<tr>
<td>
%variable, subscripted (reference to)</td>
<td>
16 + expression evaluation</td>
</tr>

<tr>
<td>
ADD</td>
<td>
20</td>
</tr>

<tr>
<td>
AFTER</td>
<td>
20</td>
</tr>

<tr>
<td>
AND (except where AND is part of BETWEEN)</td>
<td>
16</td>
</tr>

<tr>
<td>
Conversion between a string and a number</td>
<td>
16</td>
</tr>

<tr>
<td>
CALL</td>
<td>
16</td>
</tr>

<tr>
<td>
CHANGE</td>
<td>
44</td>
</tr>

<tr>
<td>
CLEAR LIST</td>
<td>
20</td>
</tr>

<tr>
<td>
CLEAR ON</td>
<td>
16</td>
</tr>

<tr>
<td>
CLEAR TAG</td>
<td>
16</td>
</tr>

<tr>
<td>
CLOSE</td>
<td>
16</td>
</tr>

<tr>
<td>
CLOSE PROCESS</td>
<td>
16</td>
</tr>

<tr>
<td>
COMMIT</td>
<td>
4</td>
</tr>

<tr>
<td>
COMMIT RELEASE</td>
<td>
20</td>
</tr>

<tr>
<td>
COUNT RECORDS (with a group)</td>
<td>
52 + 20</td>
</tr>

<tr>
<td>
DELETE</td>
<td>
24</td>
</tr>

<tr>
<td>
DELETE RECORD</td>
<td>
16</td>
</tr>

<tr>
<td>
ELSE</td>
<td>
16 + body of the clause</td>
</tr>

<tr>
<td>
ELSEIF</td>
<td>
16 + body of the clause END</td>
</tr>

<tr>
<td>
END</td>
<td>
4</td>
</tr>

<tr>
<td>
Function call</td>
<td>
16 + argument evaluation</td>
</tr>

<tr>
<td>
FIND (with at least one direct condition)</td>
<td>
64 + 16</td>
</tr>

<tr>
<td>
FIND (with inverted condition)</td>
<td>
64 + 36</td>
</tr>

<tr>
<td>
FIND ALL RECORDS (no qualification)</td>
<td>
64</td>
</tr>

<tr>
<td>
FIND ALL RECORDS (with record security and group)</td>
<td>
64 + 52 + 20</td>
</tr>

<tr>
<td>
FIND ALL RECORDS (with record security)</td>
<td>
64 + 52</td>
</tr>

<tr>
<td>
FIND ALL VALUES (FRV field)</td>
<td>
74</td>
</tr>

<tr>
<td>
FIND ALL VALUES (ORDERED field)</td>
<td>
32</td>
</tr>

<tr>
<td>
FOR EACH RECORD</td>
<td>
24 + body of the loop</td>
</tr>

<tr>
<td>
FOR EACH VALUE OF (with IN ORDER, a group)</td>
<td>
88 + body of the loop + 68 + 24</td>
</tr>

<tr>
<td>
GREATER THAN</td>
<td>
20</td>
</tr>

<tr>
<td>
IDENTIFY</td>
<td>
16</td>
</tr>

<tr>
<td>
IF</td>
<td>
32</td>
</tr>

<tr>
<td>
IF (with operator)</td>
<td>
32 + 16</td>
</tr>

<tr>
<td>
Index loop</td>
<td>
40 + expression evaluation + body of the loop</td>
</tr>

<tr>
<td>
IN RANGE FROM and TO, BETWEEN</td>
<td>
28</td>
</tr>

<tr>
<td>
INSERT</td>
<td>
24</td>
</tr>

<tr>
<td>
MODIFY</td>
<td>
16</td>
</tr>

<tr>
<td>
NOTE</td>
<td>
20</td>
</tr>

<tr>
<td>
ON</td>
<td>
16 + included statements</td>
</tr>

<tr>
<td>
OPEN</td>
<td>
20</td>
</tr>

<tr>
<td>
OPEN PROCESS</td>
<td>
20</td>
</tr>

<tr>
<td>
OR</td>
<td>
16</td>
</tr>

<tr>
<td>
PAUSE</td>
<td>
16</td>
</tr>

<tr>
<td>
POSITION</td>
<td>
20</td>
</tr>

<tr>
<td>
PREPARE IMAGE</td>
<td>
8</td>
</tr>

<tr>
<td>
PREPARE MENU</td>
<td>
8</td>
</tr>

<tr>
<td>
PREPARE SCREEN</td>
<td>
8</td>
</tr>

<tr>
<td>
PRINT</td>
<td>
16 for each term</td>
</tr>

<tr>
<td>
PRINT (a field)</td>
<td>
16 + 20</td>
</tr>

<tr>
<td>
PRINT MENU</td>
<td>
16</td>
</tr>

<tr>
<td>
PRINT SCREEN</td>
<td>
16</td>
</tr>

<tr>
<td>
READ IMAGE</td>
<td>
16</td>
</tr>

<tr>
<td>
READ MENU</td>
<td>
20</td>
</tr>

<tr>
<td>
READ SCREEN</td>
<td>
20</td>
</tr>

<tr>
<td>
RECEIVE</td>
<td>
16</td>
</tr>

<tr>
<td>
RELEASE RECORD</td>
<td>
20</td>
</tr>

<tr>
<td>
RELEASE POSITION</td>
<td>
8</td>
</tr>

<tr>
<td>
REPEAT</td>
<td>
16 + evaluation of WHILE clause</td>
</tr>

<tr>
<td>
REREAD SCREEN</td>
<td>
20</td>
</tr>

<tr>
<td>
RETRY</td>
<td>
16</td>
</tr>

<tr>
<td>
RETURN</td>
<td>
16</td>
</tr>

<tr>
<td>
SEND</td>
<td>
16</td>
</tr>

<tr>
<td>
SIGNAL PROCESS</td>
<td>
12</td>
</tr>

<tr>
<td>
STOP (automatically generated)</td>
<td>
16</td>
</tr>

<tr>
<td>
STORE RECORD (with each field)</td>
<td>
16 + 16</td>
</tr>

<tr>
<td>
TAB</td>
<td>
4</td>
</tr>

<tr>
<td>
TAG</td>
<td>
16</td>
</tr>

<tr>
<td>
THEN</td>
<td>
16 + body of the clause</td>
</tr>

<tr>
<td>
TRANSFER</td>
<td>
16</td>
</tr>

<tr>
<td>
WITH</td>
<td>
0</td>
</tr>

<tr>
<td>
WRITE IMAGE</td>
<td>
12</td>
</tr>
</table>

==User, field, group security table (RTBL)==

RTBL contains a user's privileges, class, field-level security (FLS) levels for each open file, and classes for open, permanent groups.

The size of RTBL is calculated from the formula:

(((NGROUPS + 11) * (NFILES + 1)) + (NRMTFILE + 1))dwr


For Parallel Query Option/204 client nodes, the required size of RTBL increases by <code>(NRMTFILE=1)</code> bytes.

===Character string table (STBL)===

STBL stores all character strings in counted form, with a 1-byte length preceding the string itself. The following considerations apply to space usage:

<ul>
<li> Space used to store intermediate results during an arithmetic expression evaluation is freed when the evaluation is completed.</li>

<li>Space used by <var>FOR EACH OCCURRENCE</var>, <var>FOR EACH RECORD</var>, and <var>FOR EACH VALUE</var> loops is reused until the end of the loop.</li>

<li>The last value of a <var>Note</var> statement remains in STBL.</li>

<li><var>FIXED</var> or <var>FLOAT</var> %variable array uses eight bytes for each element. </li>
</ul>

When the %variable is reassigned, the STBL space is reused.

<ul>
<li>The FIELD SAVE option requires 10 bytes plus the maximum length of the string plus one byte for each element.</li>
</ul>

NO FIELD SAVE does not reserve the extra 10 bytes and results in significant saving when using a multidimensional array.

<ul>
<li>MORE releases all but the space required by %variables and arrays. </li>

<li>If a user is using the pattern matcher in an IF statement with an IMAGE or SCREEN ITEM as the pattern and an IMAGE or SCREEN ITEM as the comparison string, then the value of the pattern is stored in STBL. The space is freed after each comparison, so the maximum increase is equal to the size of the largest pattern in a request that meets the above criteria.</li>
</ul>

====When the Inverted File Access Method (IFAM) is used====
<ul>
<li><var>IFDVAL</var> and <var>IFFILE</var> store the value string from the input parameter in STBL. </li>

<li>Strings enclosed in quotation marks or values specified for <var>IFFIND</var> that are stored in STBL are the same types as those stored in STBL by SOUL.</li>

<li>EDIT form of <var>IFPUT</var> uses STBL for each value. Space from previous values is reused. </li>

<li><var>FLOD</var> stores translation table values and <var>CASE</var> statement comparison values in STBL.
</li>
</ul>

====Parallel Query Option/204 STBL requirements====

STBL requires an increase for Parallel Query Option/204 $functions used in remote file or scattered context and for SOUL pattern matching processes

<ul>
<li>Using one of the following $functions in remote context requires 12 additional bytes in STBL:

<var>$Curfile</var>

<var>$RlcFile</var>

<var>$Update</var>

<var>$UpdFile</var></li>

<li>If you are using the pattern matcher in any SOUL statement, the full size of the pattern is used and stored in STBL. The space is freed after each statement block using the pattern matcher, for example, <var>FIND</var> or <var>FOR</var>.</li>
</ul>

===Temporary work page list table (TTBL)===

TTBL entries keep track of scratch file (CCATEMP) pages. The entries are four bytes each and used by the Editor and <var>FIND</var> (or <var>IFFIND</var>) evaluator routines.

<ul>
<li>The Editor uses scratch pages to make a private copy of the procedure being edited.</li>

<li><var>FIND</var> uses scratch pages as work space for evaluating Boolean expressions.</li>
</ul>

The number of TTBL entries required by <var>FIND</var> depends on the complexity of the Boolean expression. Entries are released at the end of the <var>FIND</var> statement evaluation. 

===Compiler variable table (VTBL)===

VTBL stores values of simple variable types and refers to the String Table (STBL) in cases of more complex variables. The variables are local (%variables), internal, screen, and image variables. 


Many SOUL statements and some constructs cause one or more compiler variables to be allocated in VTBL.

Entries in VTBL vary in size. The "Sample VTBL entries" table shows typical values. For more information, refer to [[Large request considerations#VTBL (compiler variable table)|VTBL (compiler variable table)]].

<table>
<caption>Sample VTBL entries</caption>
<tr class="head">
<th>
SOUL statement </th>
<th>
VTBL entry size (bytes)</th>
</tr>

<tr>
<td>
% variable:</td>
<td>
 </td>
</tr>

<tr>
<td>
FIXED</td>
<td>
16</td>
</tr>

<tr>
<td>
FLOAT</td>
<td>
12</td>
</tr>

<tr>
<td>
STRING</td>
<td>
24</td>
</tr>

<tr>
<td>
array</td>
<td>
20</td>
</tr>

<tr>
<td>
array element reference</td>
<td>
12</td>
</tr>

<tr>
<td>
CALL</td>
<td>
4 + (4 * arguments)</td>
</tr>

<tr>
<td>
CALL statement evaluation</td>
<td>
28</td>
</tr>

<tr>
<td>
COUNT</td>
<td>
8</td>
</tr>

<tr>
<td>
FIND (basic entry, single file)</td>
<td>
8</td>
</tr>

<tr>
<td>
workspace</td>
<td>
20 + 20 + 28</td>
</tr>

<tr>
<td>
fieldname = value pair</td>
<td>
20 or more</td>
</tr>

<tr>
<td>
retrieval</td>
<td>
28</td>
</tr>

<tr>
<td>
Ordered Index retrieval</td>
<td>
4 + 4 per segment</td>
</tr>

<tr>
<td>
FIND (basic entry, group)</td>
<td>
8 + (8 * files)</td>
</tr>

<tr>
<td>
FOR EACH RECORD (no IN ORDER clause)</td>
<td>
16</td>
</tr>

<tr>
<td>
FOR EACH VALUE (with FROM, TO, LIKE, ORDERED</td>
<td>
48</td>
</tr>

<tr>
<td>
FOR EACH RECORD IN ORDER BY (with ORDERED field)</td>
<td>
48</td>
</tr>

<tr>
<td>
FOR EACH VALUE (with ORDERED field)</td>
<td>
40</td>
</tr>

<tr>
<td>
Function (SOUL)</td>
<td>
4 + (4 * arguments)</td>
</tr>

<tr>
<td>
Lists:</td>
<td>
 </td>
</tr>

<tr>
<td>
single files</td>
<td>
8</td>
</tr>

<tr>
<td>
groups</td>
<td>
8 + (8 * files)</td>
</tr>

<tr>
<td>
Menu definition</td>
<td>
48
</td>
</tr>

<tr>
<td>
Numeric expression</td>
<td>
16</td>
</tr>

<tr>
<td>
ON</td>
<td>
28</td>
</tr>

<tr>
<td>
Related image set</td>
<td>
12 + 4 for every 256 items</td>
</tr>

<tr>
<td>
Screen definition</td>
<td>
68 + 4 for each panel</td>
</tr>

<tr>
<td>
Screen entry (one panel)</td>
<td>
72</td>
</tr>

<tr>
<td>
SORT</td>
<td>
 </td>
</tr>

<tr>
<td>
each referenced field</td>
<td>
20</td>
</tr>

<tr>
<td>
each sort key</td>
<td>
32</td>
</tr>

<tr>
<td>
String expression</td>
<td>
8</td>
</tr>

<tr>
<td>
Subroutine declaration</td>
<td>
16 + parameters </td>
</tr>
</table>

====Effect of the MORE parameter on VTBL====

The MORE parameter deletes all VTBL entries except %variables, list entries, images, menus, screens, labeled SORT and sort key entries, labeled FIND entries, and labeled COUNT entries.

====IFAM and FLOD considerations====

The following considerations apply to VTBL: 

IFAM requires VTBL space for <var>IFFIND</var>, <var>IFCOUNT</var>, and lists (matching SOUL requirements), and a few extra bytes for temporary work space. 

FLOD needs a minimal amount of VTBL space if the L statement (locate loop) is used. 

====VTBL requirement for Application Subsystem users====

The size of VTBL for [[Application Subsystem development|Application Subsystem]] users loading saved compilations does not have to be as large as the compiling user's VTBL. As long as the loading user's VTBL size accommodates the request's VTBL requirement, the loading user can have a smaller VTBL than the compiling user. 

===Procedure security table (XTBL)===

XTBL contains procedure security information for each file and permanent group member that a user has open. The size of each file entry depends on the number of user-class/procedure-class mappings defined in the file for the user's class. 

[[Category:System management]]
[[Category:Model 204 operational issues]]

REGXSTBL parameter

2022-03-25T20:08:11Z

ATitelbaum: /* Description */

{{Template:REGXSTBL parameter subtitle}}

==Summary==
<dl>
<dt>Default value
<dd>2048
<dt>Parameter type
<dd>System
<dt>Where set
<dd>User 0 parameter line
<dt>Related products
<dd>All
<dt>Introduced
<dd><var class="product">Model 204</var> version 7.9
</dl>

==Description==
This parameter indicates the amount of STBL to be assigned to each swapped-in [[Regex class|Regex object]].

When the [[New (Regex constructor)|Regex constructor]] is invoked, the regular expression string and replacement string are compiled and the original strings and compilations are saved in the STBL part of the object. If the total size of these strings and compilations exceeds <var>REGXSTBL</var>, the request is cancelled.

Since the size of the regular expression and replacement compilation rarely exceed 16 bytes per source character (it's usually less), the default value of 2048 can accommodate almost all regular expressions with replacement strings of at least 120 characters. Regular expressions much longer than this are rare.

Increasing the value of this parameter increases the STBL requirements for every request that uses <var>Regex</var> objects, usually by twice the amount of increase (one for each swapped-in object slot allocated).

[[Category:Parameters]]
[[Category:System parameters]]

Regex class

2022-03-25T20:05:54Z

ATitelbaum:

The <var>Regex</var> class provides a facility that allows reuse of runtime compiled regular expressions. It was first available in Model 204 version 7.9.

==Regex usage==
Before Model 204 7.9, every regular expression function would recompile the regular expression specified as a string parameter on each call. So, in a program like the following:
%num is float
%stringlist is object stringlist
...
%num = 1
repeat forever
%num = %stringlist:regexLocate("(%[a-z_][a-z_.0-9]*) *=.*?\1", options="i")
if %num eq 0 then loop end; end if
...
end repeat

the regular expression in the <var>RegexLocate</var> call would be parsed and compiled in every iteration of the loop. As one might guess, this adds a lot of unnecessary overhead. In Model 204 7.9 and later, the regular expression would be compiled at the time the surrounding SOUL code would be compiled so no compilation would be necessary at runtime. The Model 204 regular expression engine was also rewritten in 7.9 to make it significantly more efficient so code like that above would be significantly more efficient in Model 204 7.9 and later.

For most purposes, these improvements can be taken advantage of without any effort. However, as one might guess, a regular expression can only be compiled at SOUL compile-time if the regular expression is a literal. Otherwise, how could the compiler know what to compile? There are cases, however, where a regular expression is only available at runtime. A classic example is in a program that allows end users to type in arbitrary regular expressions for matching against a set of values. In such a case, it might be useful to be able to compile the user-specified expression once and then apply it to a variety of values.

It might also be useful to use the same regular expression in multiple places or to even create a system or subsystem global that contains a regular expression. Or one might simply want to separate the regular expression creation from the code that uses it for code-readability or maintenance reasons.

It is for cases like these that the <var>Regex</var> class was created. To illustrate how the <var>Regex</var> class might be used, consider the above example where user-specified regular expression is in string variable <var>%userRegex</var>. In that case, the above code can be written as:
%num is float
%regex is object regex
%stringlist is object stringlist
%userRegex is string len 255
...
%num = 1
%regex = new(%userRegex, options="i")
repeat forever
%num = %regex:locate(%stringlist, %num)
if %num eq 0 then loop end; end if
...
end repeat

In this case, the user-specified regular expression is only compiled once in the <var>Regex</var> constructor call (<var>new</var>) and reused multiple times after that, for optimal efficiency.

==The Regex class and Unicode==
The Model 204 regular expression engine supports both EBCDIC and Unicode regular expressions and match strings – one would use a Unicode regular expression against a Unicode match string and an EBCDIC regular expression against an EBCDIC match string. To facilitate this, the non-<var>Regex</var> class methods often have Unicode variants. For example, <var>RegexMatch</var> should be used for EBCDIC regular expressions and match strings and <var>UnicodeRegexMatch</var> against Unicode. Note that, the Model 204 <var>String</var> and <var>Longstring</var> types are agnostic about their contents (they could very well be binary or UTF-8), but in most cases they contain EBCDIC data.

However, there is no Unicode version of the <var>Regex</var> class. Instead, the <var>Regex</var> constructor (<var>New</var>) compiles a Unicode or EBCDIC regular expression based on the type of the regular expression parameter. For example, in the following:
%regex is object regex
...
%regex = new("foo.*?bar")
...
%regex = new("foo.*?bar":u)

the first constructor call would compile an EBCDIC regular expression and the second Unicode. While, in this case, the Unicode regular expression contains no Unicode characters that can't be converted to EBCDIC, the difference is still significant. If the first value of <var>%regex</var> was run against a Unicode string, that string would first have to be translated to EBCDIC so might encounter a translation error and, in any case, would suffer the extra overhead of character translation. Similarly, if the second value of <var>%regex</var> was run against an EBCDIC string, the EBCDIC string would first have to be translated to Unicode, incurring extra overhead.

While the distinction between EBCDIC and Unicode input parameters can be handled at runtime, method output types are determined at compile-time so any <var>Regex</var> method that returns a string, must have Unicode and non-Unicode versions. For example, the <var>Replace</var> function returns a string, so there is a <var>ReplaceUnicode</var> function that should be used when the <var>Regex</var> object contains Unicode. The <var>Regex</var> <var>IsUnicode</var> method can be used to determine if a <var>Regex</var> object was created for Unicode strings.

In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.

==Replacement==
When using the <var>Replace</var> or <var>ReplaceUnicode</var> methods, one can specify a replacement string on the <var>Replace</var> or <var>ReplaceUnicode</var> method call. However, if one is always using the same replacement value for a <var>Regex</var> object, it can be slightly more efficient to specify the replacement string on the <var>Regex</var> object constructor call. For example:
%regex is object regex
%stringlist is object stringlist
%userRegex is string len 255
%userReplace is string len 255
...
%num = 1
%regex = new(%userRegex, options="i")
repeat forever
%num = %regex:locate(%stringlist, %num)
if %num eq 0 then loop end; end if
%stringlist:replace(%i, %regex:replace(%num, %stringlist(%num), %userReplace)
end repeat

could be done more efficiently as:
%regex is object regex
%stringlist is object stringlist
%userRegex is string len 255
%userReplace is string len 255
...
%num = 1
%regex = new(%userRegex, replace=%userReplace, options="i")
repeat forever
%num = %regex:locate(%stringlist, %num)
if %num eq 0 then loop end; end if
%stringlist:replace(%num, %regex:replace(%stringlist(%num))
end repeat


==Introspection==
The <var>Regex</var> class has several methods that allow a <var>Regex</var> object to describe itself. This can be useful when looking at a <var>Regex</var> object in a <var>SirFact</var> dump or when using the debugger. The introspection methods are:
<table>
<tr><th>IsUnicode</th><td>A boolean value indicating whether or not the object is for Unicode.</td></tr>
<tr><th>Options</th><td>The <var>Options</var> parameter value specified in the constructor call.</td></tr>
<tr><th>RegexString</th><td>The regular expression specified in the constructor call.</td></tr>
<tr><th>RegexUnicode</th><td>The regular expression Unicode specified in the constructor call.</td></tr>
<tr><th>ReplaceString</th><td>The replacement specified in the constructor call.</td></tr>
<tr><th>ReplaceUnicode</th><td>The replacement Unicode specified in the constructor call.</td></tr>
</table>

==Regular expression size limits==
When the [[New (Regex constructor)|Regex constructor]] is invoked, the regular expression string and replacement string are compiled and the original strings and compilations are saved in the STBL part of the object. If the total size of these strings and compilations exceeds the value of the [[REGXSTBL parameter|REGXSTBL system parameter]], the request is cancelled.

The value of the <var>REGXSTBL</var> parameter can be increased if this is a problem though, since the parameter is a user0 parameter only, it requires a recycle of the Online to make the change. If <var>REGXSTBL</var> is increased, STBL requirements will also increase for every request that uses <var>Regex</var> objects.

Note that the default value of <var>REGXSTBL</var> is 2048 which is probably large enough to accommodate almost any regular expression one is likely to encounter in the "real world".

==List of Regex methods==
The individual <var>Regex</var> methods are summarized in [[List of Regex methods]].

[[Category:Regular expression processing]]