SirFile

Overview

SirFile is a comprehensive facility for monitoring the physical storage utilization of Model 204 database files and warning users of the need for file reorganizations.

SirFile requires only a single screen of setup information to determine "thresholds," after which it automatically performs a pass through the Online database files, checking to see if any file has exceeded a threshold. The process that evaluates files is called the refresh process.

The refresh process performs two types of checks:

• It compares the current state of each file against the threshold settings to see if any file statistic is above the critical value for that statistic.
• It maintains a database of table usage information which it uses to determine rate of growth and to predict when thresholds will be exceeded.

During the initial database load, all files added to the SirFile database are compared to the system default thresholds. After the initial load, you can override the system default thresholds with file-specific thresholds for files that require higher or lower levels of monitoring.

If any file table is above a threshold, SirFile flags the file with a warning, causing it to be highlighted in SirFile. Once SirFile has stored at least one sample snapshot of a file, it begins calculating the date at which each table will fill — assuming any table is showing growth. If the predicted fill date falls within a user-settable number of days, SirFile posts a prediction to the file, and the file is highlighted in the SirFile screens. When a prediction or warning is posted using a periodic refresh, a message can also be sent to the audit trail, as well as to a list of user IDs.

The file parameters and thresholds that are evaluated for table-full status are described below in Getting started.

Although the database refresh portion of SirFile is quite efficient, it is the most expensive operation in SirFile. Therefore, several options are provided for determining when to:

• Perform a refresh.
• Calculate CFULLP, the percentage full of Table C, which is generally the most expensive of the refresh calculations.

For more information, see SirFile refresh process, for a complete description of the refresh process, including a method that can be useful for calculating CFULLP.

The SirFile refresh process is always in "learn" mode. Any file that is open when a SirFile refresh is run is automatically added to the SirFile database, using the default thresholds for comparison.

Setting up

Make files available for SirFile monitoring

For SirFile to collect information on a file, it must be open in the region by some user or subsystem. To guarantee that certain files are included in the initial load, those files can be opened manually before initiating the refresh process. Because storing user-entered passwords would violate most sites' security, SirFile uses a variety of methods to make files available to itself without needing to store passwords:

• If a file is open in the region, or SirFile manages to open the file with default privileges, the file is added to the database and no user intervention is required.

  If a file requires a password, you receive a prompting screen.

• The first time SirFile collects data for a file, it adds the file to its Application Subsystem definition (with low privileges in the "Users" SCLASS and high privileges in the "Admin" SCLASS). Thereafter, no password is required for SirFile to open the file. Files that are prefixed "CCA" are exempted from this operation, as these files cause problems in SUBSYSMGMT.
• If SirFile has a file in its database that requires a password, and a non-periodic refresh is running (other than SIRFILE BATCH), you are prompted for the password.

  If SIRFILE BATCH or a periodic refresh is running, SirFile skips any file it cannot open.

File password screen

When SirFile needs to open a file, it first tries to do it without a password. If that fails you are prompted for a password with the following screen:

File open screen

SirFile requires only read privileges in files being monitored.

If you are being prompted for passwords in the middle of a refresh operation, the PF key choices are as shown in the figure, and have the following actions:

PF3	Cancel the refresh and return to the previous screen.
PF6	Skip the current file in the refresh.
PF9	Skip the this and the rest of the files in the refresh that require manual entry of passwords.

Otherwise, for example if you are prompted for a password due to an OPEN command, the only PF key choice is:

PF3	Cancel the open and return to the previous screen.

Ensure file update access

Besides its subsystem procedure file and the RKTools file SIRLOCAL, SirFile requires update access to CCASYS.

All other files, even if they are defined to SirFile as optional subsystem files, are closed and freed from the subsystem at the end of each user's SirFile session. This is done to reduce enqueuing problems.

Getting started

1. SirFile is installed as a private application subsystem, as described in RKTools installation. To access the system, you enter SIRFILE on the Model 204 command line of the Online containing the software.
2. You are presented with an initial system default thresholds screen. You may accept the defaults or change any of the supplied settings. The table below explains each of the threshold entities.

   SirFile records are fairly small and you shouldn't be too concerned about minimizing the amount of stored data: A year's worth of data for a single file will occupy only about one Model 204 page if a record is kept per week.

   Before you press the PF12 key to save the settings and initiate the SirFile database refresh process, consider opening additional files (see step 3, below).

   System default thresholds

   

   This screen is also accessible by the PF11 key from the File Monitor screen or from the SirFile file tables screen screen, if the cursor is not on a file name or file statistics line, respectively.

   The default thresholds screen is initially populated with a set of defaults that are adequate for most file monitoring needs. The thresholds are:

   ARETRIES	Table A retries indicate a failure of the Table A hashing algorithm to find a slot on a Table A page to store field information. Table A is usually the smallest table in a Model 204 database file, but it is accessed every time a field name is referred to in SOUL or IFAM code, so it is important that Table A operate as efficiently as possible. In most cases, any ARETRIES are too many, and for this reason, the ARETRIES default is very small.
   BFULLP	Percentage full of Table B. This is the percentage of record slots currently used, calculated as BHIGHPG/BSIZE. Table B holds the actual data in a Model 204 file and is usually the largest table.
   CRETRIES	Table C hashing algorithm retries. This statistic indicates that Table C, a hashed data structure, is becoming full, so the algorithm which determines placement of new pointers in the table must run repeatedly in order to find an empty slot. There is no fixed measure of how many retries indicate a critical condition for Table C.
   CFULLP	Percentage full of Table C. Like Table A, Table C begins to exhibit hash retries when the data in it reaches a certain density. There is no strict formula for the maximum fullness of Table C for a given file, but a rule of thumb is that retries will begin to appear when Table C is 70 to 80% full. CFULLP is a more accurate measure of Table C full status than CRETRIES, but it is very expensive to calculate. For this reason, unless you determine CFULLP calculation file-by-file, it is only calculated by the SIRFILE BATCH command.
   DFULLP	Table D percentage full. Table D can fill either as a result of index information or SOUL procedures. Percentage full in Table D is calculated as the number of D pages used (DPGUSED) divided by DSIZE.
   EXTNADD	Number of extension records. Extension records occur when the information added to an existing record exceeds the usable space left on a Table B page. Extension records are not a sign of a file full condition pending, and in some record structures they may be unavoidable. For this reason, the EXTNADD threshold may be set very high. However, extension records cause inefficiencies in storage and retrieval, and for some file designs should be taken as a sign that Table B parameters are out of balance.
   OVFLADD	Addition of records to overflow areas. Overflow records may occur in either sorted or hash files.
   EOVFLADD	Addition of records to extra overflow areas. Records may be added to Extra Overflow areas in sorted files when the Overflow area is full.
   SPILLADD	Number of records that have "spilled" from the preferred overflow area into secondary overflow areas or the overflow area prior to the appropriate sort group. Spilled records are stored and retrieved less efficiently than other records.
   Number of historical records to keep per file	Number of sample records to keep for each file. SirFile requires at least one sample to be stored for any given file in order to make predictions about file-full conditions. The default value of 99999 records means that SirFile does not throw away old data, as long as the file CREATE date does not change.
   Minimum number of days between stored samples	Minimum number of days that must pass between samples being stored for any particular file. This setting allows users to run the refresh as often as they like without SirFile storing excessive amounts of data. The default of 7 ensures that no more than one record a week is maintained for any file unless that file breaks a threshold.
   # of days advance warning on threshold exceeded	Number of days prior to a predicted "threshold exceeded" condition SirFile should post a warning. During a refresh, SirFile uses the current file table values and the historical file information to calculate when the file will exceed each threshold (if positive growth is detected in a threshold value). If the number of days predicted falls within this value, a prediction and message are posted to the file, and the file appears highlighted on the SirFile main menu.
   Collect CFULLP only on specified files?	Specifying Y to this prompt allows SirFile to collect the CFULLP statistic only on files specified by the user. Collecting CFULLP requires a TABLEC command, which can be very I/O intensive. When Y is specified at this prompt, you must specifically request each file the TABLEC command is to be run against, entering Y at the following prompt in the SirFile single file thresholds screen: Collect CFULLP for this file When N is specified at this prompt, CFULLP calculations are performed only by the SIRFIELD BATCH command.

3. Open or close additional files.

   Use the following commands:

   OPEN filename	Opens a file. As data can only be collected for files that SirFile already knows about, or files that are currently open by any user, you may want to manually open a file before running a refresh to guarantee the file is included in the sample.
   CLOSE filename	Closes a file. Any file that the SirFile user has open can be closed, except SIRFILE, SIRLOCAL and CCASYS (the required application subsystem files for SirFile).

   PF keys:

   PF1	Accesses full-screen help.
   PF3	Return to command level or previous screen.
   PF9	Repeats the last command-line command.
   PF12	Saves the file threshold defaults and invokes a SirFile one-time database refresh.

SirFile topics

The SirFile documentation consists of the pages listed below. This list is also available as a "See also" link from each of the pages.

For information about product changes and Model 204 feature support per SirFile version, see the RKTools release notes.

For information about product error messages, see MSIR. messages.

• SirFile
• SirFile main menu
• SirFile refresh process
• SirFile single file thresholds
• SirFile Background File Monitor screen‎
• SirFile file tables screen
• SirFile file detail screen
• SirFile messages screen