Managing file groups: Difference between revisions

From m204wiki
Jump to navigation Jump to search
mNo edit summary
 
(6 intermediate revisions by 5 users not shown)
Line 1: Line 1:
== Overview ==
==Overview==
<p>File groups are sets of files which process as if they were one.</p>
<p>
A file group is a set of files which process as if they were one file.</p>


<p>There are three types:</p>
<p>There are three types of file groups:</p>
* Permanent groups
<ul>
* Temporary groups, and
<li>Permanent </li>
* Ad-hoc groups
<li>Temporary </li>
<li>Ad hoc </li>
</ul>


== Permanent, Temporary and Ad-hoc Groups ==
==Permanent, temporary, and ad hoc groups==


===CREATE GROUP command and the types of groups ===
===CREATE GROUP command and the types of groups===
<p>Permanent and Temporary file groups are created using the CREATE GROUP command. Ad-hoc groups exist only for a particular 'ín' clause (and any label references which refer to the statement). The following chart explains how they are created: </p>
<p>
Permanent and temporary file groups are created using the <var>CREATE PERM GROUP</var> and <var>CREATE TEMP GROUP</var> commands. Ad hoc groups exist only for a particular <var>IN</var> clause (and any label references that refer to the statement). The following table explains how they are created: </p>
<table>
<table>
<tr class="head">
<tr class="head">
Line 16: Line 20:
<th>Created...</th>
<th>Created...</th>
</tr>
</tr>
<tr>
<tr>
<td>Permanent </td>
<td>Permanent </td>
<td>
<td>By the system manager using the <var>CREATE</var> command.
<p>By the system manager using the CREATE command.</p>
<p>
<p>User access is determined by the privileges defined for the file group.</p>
User access is determined by the privileges defined for the file group.</p>
<p>Use the following syntax to create a permanent group: </p>
<p>
<p class="code">CREATE   PERM GROUP groupname
Use the following syntax to create a permanent group: </p>
 
<p class="syntax">CREATE PERM GROUP <span class="term">groupname</span>
FROM  {filename  [AT location] [(OPTIONAL)  |  (MANDATORY)]} ,... [PARAMETER  parameter[=value] [PARAMETER  parameter[=value]]...
FROM  {<span class="term">filename</span> [AT <span class="term">location</span>] [(OPTIONAL)  |  (MANDATORY)]} ,...  
[PARAMETER  <span class="term">parameter</span>[=<span class="term">value</span>]  
[PARAMETER  <span class="term">parameter</span>[=<span class="term">value</span>]]...
.
.
.
.
Line 30: Line 37:
END
END
</p>
</p>
<p>For information about creating permanent groups, see [[CREATE command: Permanent group|CREATE PERM GROUP]]</p>
<p>
For information about creating permanent groups, see <var>[[CREATE command: Permanent group|CREATE PERM GROUP]]</var>.</p>
</td>
</td>
</tr>
</tr>


<tr><td>Temporary</td>
<tr><td>Temporary</td>
<td><p>By an individual user with the CREATE command and can be referenced only by that user. </p>
<td>By an individual user with the <var>CREATE</var> command, and can be referenced only by that user.  
<p>Temporary group names exist only during the current login session and are deleted automatically when you log out, unless you delete them during the session.   </p>
<p>
<p>Use the following syntax to create a temporary group:</p>
Temporary group names exist only during the current login session and are deleted automatically when you log out, unless you delete them during the session. </p>
<p class="code">CREATE [TEMP] GROUP groupname FROM filename
<p>
  [,filename ...][PARAMETER parameter_list]
Use the following syntax to create a temporary group:</p>
<p class="syntax">CREATE [TEMP] GROUP <span class="term">groupname</span> FROM <span class="term">filename</span> [,<span class="term">filename</span> ...]
[PARAMETER <span class="term">parameter_list</span>]
  .     
  .     
  .
  .
Line 45: Line 55:
END  
END  
</p>
</p>
<p>The parameters available for temporary file groups are detailed in [[CREATE command: Temporary group|CREATE TEMP GROUP]].</p>
<p>
The parameters available for temporary file groups are described in <var>[[CREATE command: Temporary group|CREATE TEMP GROUP]]</var>.</p>
</td>
</td>
</tr>
</tr>
Line 51: Line 62:
<tr>
<tr>
<td>Ad hoc</td>
<td>Ad hoc</td>
<td><p>Within a User Language request by prefacing a statement with the clause: </p>
<td>Within a SOUL request by prefacing a statement with the clause:  
<p class="code">IN <var class="term">file1</var>, <var class="term">file2</var> [,<var class="term">file3</var> ...]   
<p class="syntax">IN <span class="term">file1</span>, <span class="term">file2</span> [,<span class="term">file3</span> ...]   
</p>
</p>
<p>Ad hoc groups have no name and exist only for the current request. They allow record retrieval from many files at once without defining a group in advance. As in any IN clause, all of the files in an ad hoc group must be opened before the group is mentioned. For more information, see the description of Ad hoc groups in the <var>Rocket Model&nbsp;204 User Language Manual</var>.</p></td></tr>
<p>
Ad hoc groups have no name and exist only for the current request. They allow record retrieval from many files at once without defining a group in advance. As in any <var>IN</var> clause, all of the files in an ad hoc group must be opened before the group is mentioned. For more information, see [[Files, groups, and reference context#Ad hoc groups|Ad hoc groups]].</p></td></tr>
</table>
</table>


== Using Groups ==
==Using groups==
<p>There are a number of reasons why a file manager might set up file groups:</p>
<p>
 
This section describes a number of reasons why a file manager might set up file groups.</p>
=== Files with large numbers of record ===
<p>A single Model 204 file may only contain 16.7 million record slots. As applications grow, this limit may not be enough.</p>
 
<p>Defining a permanent file group can support much larger sets of data: there can be up to 254 files in a file group.</p> 
 
<p>Normally, if you take this approach, you would not randomly place the records anywhere in the group, but rather you would partition the files based on some attribute. For example, you may use the Social Security Number to decide which file in a group to place a particular record in. '''Hint:''' because some identifiers have hidden meanings (the high order digits of the Social Security Numbers used to identify the region where the application was made, for example), often the low order digits make a better randomizer.</p>
 
=== Partitioning for performance ===
<p>But even files where you do not expect to hit the record limit might benefit from partitioning. If you were, for example, to most commonly read data for a particular year, you would probably find that placing all the records for a year in different files would perform better than putting all of the records in a single file and indexing the year for easy access.</p>
 
<p>See [[Files, Groups, and Reference Context#IN GROUP MEMBER clause|IN GROUP MEMBER clause]] for more information.</p>
 
=== Procedure groups in APSYs ===
 
<p>When creating Application Subsystems (APSYs) you may (and are encouraged to) specify that the procedure file is actually a group.</p>


<p>You may, in your application design, have large amounts of code identical in multiple APSYs, but you may require different global setting between the APSYs. Because, at compile time, procedure files are searched in the order they appear in the group. you can put different 'first' procedure files, each followed by the same code base. </p>
===Files with large numbers of records===
<p>
A single Model 204 file may only contain 16.7 million record slots. As applications grow, this limit may not be enough.</p>
<p>
Defining a permanent file group enables support of much larger sets of data: a file group can have as many as 256 files.<!--Tested: GROUP001.DME.QA--></p> 
<p>
Normally, if you take this approach, you do not randomly place the records anywhere in the group, but rather you partition the files based on some attribute. For example, you might use the Social Security number to decide in which file in a group to place a particular record. </p>


<p>See "Overview of the Subsystem Management facility" in [http://docs.rocketsoftware.com/nxt/gateway.dll/RKB6/system%20%26%20file%20management/m204_v7r4_system_manager.pdf The Model 204 System Managers Guide] for more information on APSYs.</p>
===Partitioning for performance===
<p>
Even files with a record limit you do not expect to hit might benefit from partitioning. For example, say you most commonly read data for a particular year. You would probably find that placing each year's records in a different file would perform better than putting all of the records in a single file, then indexing the year for easy access.</p>
<p>
See [[Files, groups, and reference context#IN GROUP MEMBER clause|IN GROUP MEMBER clause]] for more information.</p>


=== Postponing reorganizations with Groups ===
===Procedure groups in APSYs===
<p class="note">'''Note:''' the following technique cannot be used if there are references in your code to Internal Record Numbers (IRNs). IRNs (without also specifying a group member) cannot be used against group files.</p>
<p>
When creating [[Application Subsystem development|Application Subsystems]] (APSYs) you may (and are encouraged to) specify that the procedure file is actually a group.</p>
<p>
In your application design, you may have large amounts of code that is identical in multiple APSYs, but you may require different global settings between the APSYs. Since at compile time, procedure files are searched in the order they appear in the group, you can set different "first" procedure files, each followed by the same code base. </p>
<p>
See [[System requirements for Application Subsystems]] for more information about APSYs.</p>


* File X is almost full
===Using a group to postpone a file reorganization===
* Define File Y, which looks like File X (same field names)
<p class="note">'''Note:''' The following technique cannot be used if there are references in your code to Internal Record Numbers (IRNs). IRNs (without also specifying a group member) cannot be used against group files.</p>
* File Y can be defined to have whatever table sizes it needs
The following sequence describes a technique for postponing the reorganization of file <code>X</code>, which is almost full:
* Group File X and Y together, with File Y as the specified UPDTFILE
<ol>
* Run your application against grouped files X and Y
<li>Define file <code>Y</code>, which looks like file <code>X</code> (same field names).</li>
<p>Example:</p>     
<li>Define file <code>Y</code> to have whatever table sizes it needs.</li>
<li>Group file <code>X</code> and file <code>Y</code> together, with file <code>Y</code> as the specified <var>UPDTFILE</var> (<var>CREATE GROUP</var> command parameter).</li>
<li>Run your application against grouped files <code>X</code> and <code>Y</code>.</li>
</ol>
<p>
Example:</p>     
<p class="code">CREATE PERM GROUP X FROM X, Y
<p class="code">CREATE PERM GROUP X FROM X, Y
PARAMETER UPDTFILE = Y  
PARAMETER UPDTFILE = Y  
Line 94: Line 111:
</p>
</p>


[[Category:File management]]
[[Category:Model 204 files]]

Latest revision as of 22:20, 18 September 2017

Overview

A file group is a set of files which process as if they were one file.

There are three types of file groups:

  • Permanent
  • Temporary
  • Ad hoc

Permanent, temporary, and ad hoc groups

CREATE GROUP command and the types of groups

Permanent and temporary file groups are created using the CREATE PERM GROUP and CREATE TEMP GROUP commands. Ad hoc groups exist only for a particular IN clause (and any label references that refer to the statement). The following table explains how they are created:

Type of group Created...
Permanent By the system manager using the CREATE command.

User access is determined by the privileges defined for the file group.

Use the following syntax to create a permanent group:

CREATE PERM GROUP groupname FROM {filename [AT location] [(OPTIONAL) | (MANDATORY)]} ,... [PARAMETER parameter[=value] [PARAMETER parameter[=value]]... . . . END

For information about creating permanent groups, see CREATE PERM GROUP.

Temporary By an individual user with the CREATE command, and can be referenced only by that user.

Temporary group names exist only during the current login session and are deleted automatically when you log out, unless you delete them during the session.

Use the following syntax to create a temporary group:

CREATE [TEMP] GROUP groupname FROM filename [,filename ...] [PARAMETER parameter_list] . . . END

The parameters available for temporary file groups are described in CREATE TEMP GROUP.

Ad hoc Within a SOUL request by prefacing a statement with the clause:

IN file1, file2 [,file3 ...]

Ad hoc groups have no name and exist only for the current request. They allow record retrieval from many files at once without defining a group in advance. As in any IN clause, all of the files in an ad hoc group must be opened before the group is mentioned. For more information, see Ad hoc groups.

Using groups

This section describes a number of reasons why a file manager might set up file groups.

Files with large numbers of records

A single Model 204 file may only contain 16.7 million record slots. As applications grow, this limit may not be enough.

Defining a permanent file group enables support of much larger sets of data: a file group can have as many as 256 files.

Normally, if you take this approach, you do not randomly place the records anywhere in the group, but rather you partition the files based on some attribute. For example, you might use the Social Security number to decide in which file in a group to place a particular record.

Partitioning for performance

Even files with a record limit you do not expect to hit might benefit from partitioning. For example, say you most commonly read data for a particular year. You would probably find that placing each year's records in a different file would perform better than putting all of the records in a single file, then indexing the year for easy access.

See IN GROUP MEMBER clause for more information.

Procedure groups in APSYs

When creating Application Subsystems (APSYs) you may (and are encouraged to) specify that the procedure file is actually a group.

In your application design, you may have large amounts of code that is identical in multiple APSYs, but you may require different global settings between the APSYs. Since at compile time, procedure files are searched in the order they appear in the group, you can set different "first" procedure files, each followed by the same code base.

See System requirements for Application Subsystems for more information about APSYs.

Using a group to postpone a file reorganization

Note: The following technique cannot be used if there are references in your code to Internal Record Numbers (IRNs). IRNs (without also specifying a group member) cannot be used against group files.

The following sequence describes a technique for postponing the reorganization of file X, which is almost full:

  1. Define file Y, which looks like file X (same field names).
  2. Define file Y to have whatever table sizes it needs.
  3. Group file X and file Y together, with file Y as the specified UPDTFILE (CREATE GROUP command parameter).
  4. Run your application against grouped files X and Y.

Example:

CREATE PERM GROUP X FROM X, Y PARAMETER UPDTFILE = Y END