Managing file and table sizes

When you create a file and its tables, whether you use the calculations described in File Size Calculation to size the tables or the rules of thumb that an experienced File Manager develops over time, there is always some space for growth included. When a Model 204 file is created, exact disk space requirements are seldom known. Even if perfect, eventually, it will be necessary to revise the space utilized in a file.

There are three ways to maintain table sizes: the FILEMGMT subsystem (which see); the File Manager may use a number of commands to maintain the table sizes; or some of those functions may be automated by setting up the 'Auto Increase' parameters in the file so that free space is added to the tables as needed.

The latter two are described below.

Overview of Capabilities

Because of the hashed nature of Tables A and C, these tables can only be changed with the CREATE command during a file reorganization.

The other tables may be modified using the INCREASE and DECREASE command, as follows:

Command	Description	Exclusive Access?	Limitations
INCREASE TABLEB	Move pages from free space to Table B	No, but the DYNAMIC sub-parameter is required if you don't have exclusive access	segment boundaries need to be taken into account the MAXINCBP parameter needs to be set if you wish to increase Table B beyond the current segment boundary. INCREASE cannot be used for Table B in a hashed organisation file ( FILEORG = x'08' )
INCREASE TABLED TABLEE or TABLEX	Move pages from free space to this table	No	Up to maximum table size. XSIZE cannot be increased from 0.
DECREASE TABLEB TABLED TABLEE TABLEX	Move pages to free space from the respective table	Yes	Only if the ?HIGHPG (for the respective table) statistic permits it
INCREASE DATASET	Add an additional dataset to a Model 204 file.	No	A model 204 file may have up to 255 datasets The additional pages are formatted as free space

Increasing Tables B, D, E, and X

You can increase manually Tables B, D, E, and X by assigning pages from free space (with special considerations for Table B described below). Therefore, you must examine the FREESIZE parameter using the VIEW command to make sure that unassigned pages are available. If they are not, you can use INCREASE DATASETS to add new pages.

The format of the INCREASE command is:

Syntax

INCREASE {TABLEB | TABLED | TABLEE | TABLEX} n [DYNAMIC]

where:

n is a decimal number from 1 to FREESIZE.

Example

INCREASE TABLEE 2

Usage notes

Executing the command immediately changes the values of FREESIZE and either BSIZE, DSIZE, ESIZE or XSIZE, respectively.

The DYNAMIC option applies to only an INCREASE TABLEB command.

For Tables B and X, you must take into account BRECPPG and the maximum number of slots.

Special considerations Increasing Table B without exclusive access

The DYNAMIC Option

If you do not have exclusive access to the file and still need to increase the size of Table B, you must specify DYNAMIC (or DYN) on the INCREASE TABLEB command to force the increase to occur (even though requests have already been compiled against the file). The command would look something like:

INCREASE TABLEB 10000 DYNAMIC

The MAXINCBP parameter

Even with the DYNAMIC option, you may only increase the size to the next segment boundary. (The number of segments in a file is used as part of the procedure compilation.)

This limitation can be alleviated by setting the MAXINCBP parameter.

The MAXINCBP system parameter sets a limit, as a percentage of BSIZE, on how much Table B can increase by, either by the INCREASE TABLE B command or by automatic increase (if BAUTOINC is also set). This percentage is then used to calculate the maximum number of segments, rounded up to a whole number, that the file could have if Table B were increased by the entire amount. The actual number of pages that BSIZE can increase to is determined by the calculated number of segments, divided by BRECPPG.

The compiler uses the maximum possible number of segments to determine the correct length of the entries it allocates in the resource locking table and in VTBL. During compilation the file is treated as though it has already been increased to the maximum number of segments (MAXINCBP) even though no increase may yet have occurred.

The total Table B expansion with no MAXINCBP is limited to existing number of segments. If a file is opened without MAXINCBP set yet, then no increase in number of segments is possible. MAXINCBP must be set (or reset) before the file is opened.

The MAXINCBP parameter is ignored for any sorted or hash key files.

Example of Table B increase

For example, take a file with BSIZE=100000, BRECPPG=10. For purposes of allocating entries in the record locking table and VTBL, the compiler treats this file as though all records are present and it is therefore, a 21-segment ((10*100000)/49152) file. However, if the system parameter MAXINCBP=20, then Table B in this file could increase by 20,000 pages. The resulting Table B of 120,000 pages would be 24.4-segment ((120000*10)/49152) segment file. This would be rounded up to 25 segments, so BSIZE could actually be increased to 122,880 before crossing into the twenty-sixth segment.

These calculations occur when each file is opened for the first time and affect the compilation of requests from that time forward, for that run.

Increasing LRETBL to accommodate MAXINCBP

Setting MAXINCBP might require a corresponding increase to LRETBL. That is, if MAXINCBP is 10, you might want to increase LRETBL by 10%. MAXINCBP is the upper boundary on the required increase-the actual increase will be somewhat less.

Decreasing free space

Free space will be decreased by increasing Table(s) B, D, and/or E.

If you wish to make the overall size of the file smaller because there is too much free space, you can use the DUMP and RESTORE commands to decrease the size or number of datasets.

Decreasing Tables B, D, or E

Tables B, D, and E are decreased by reassigning unused pages in them to free space. Therefore, the parameters BSIZE and BHIGHPG, DSIZE and DHIGHPG, or ESIZE and EHIGHPG must be viewed to determine the number of unused pages in the table in question.

The format of the DECREASE command is:

Syntax

DECREASE {TABLEB | TABLED | TABLEE} n

where:

n is a decimal number.

• For TABLEB, n can be in the range 1 through the value of BSIZE, minus BHIGHPG, minus 1.
• For TABLED, n can be in the range 1 through DSIZE, minus DHIGHPG, minus 1. DHIGHPG represents the highest Table D page used.

  DECREASE processing will remove only a contiguous set of pages from the end of a table. Because of the way in which Table D pages are managed, the unused pages might not be contiguous. This is especially likely in files where procedures are heavily updated. Under these circumstances, Table D cannot be decreased by the full number of unused pages. If the DECREASE command specifies more pages than can be reassigned to free space, you receive the message "NOT ENOUGH ROOM IN TABLE." If this happens, try the DECREASE command again with a smaller number.

• For TABLEE, n can be in the range of 0 through ESIZE, minus EHIGHPG, minus 1.

You must open the file exclusively before you issue the DECREASE command. Processing the command automatically changes the values of FREESIZE and BSIZE, DSIZE, or ESIZE, respectively.

Space Management

The number of pages in a Model 204 table can be increased using the INCREASE TABLEB, INCREASE TABLED, or INCREASE TABLEE commands. If there are insufficient pages in FREESIZE to accommodate the increase, new pages can be added, also dynamically, to FREESIZE using the INCREASE DATASETS command.

When data growth can be anticipated, these increases can be made in advance and file availability remains unaffected. However, these increases are not automatic. You must anticipate the increase required and issue the commands manually prior to running update transactions that require the additional pages.

When it becomes necessary to increase the size of Table B, D, E, or X, free space might not contain enough unassigned pages.

Choose one of the following methods to increase free space:

• Decrease some other table.

  For example, if only two pages are in the free space of file CARS, and three unused pages are in Table B, the following sequence of commands adds five pages to Table D:

  OPEN CARS DECREASE TABLEB 3 INCREASE TABLED 5

• Use DUMP and RESTORE to increase the size or number of the file data sets (see File Dumping and Restoring).

Adding new pages with the INCREASE DATASETS command

When free space is exhausted, you can manually add new pages dynamically with the INCREASE DATASETS command.

• First, allocate a new data set, perhaps using the ALLOCATE command as follows:

  ALLOCATE YOURF2 WITH SCOPE=SYSTEM SEQUENTIAL CATALOG - VOLUME=CCAVOL UNIT=3390 PRI 10 CYL BLKSIZE=6184 - DSN=CCA.YOURFILE.YOURF2.M204

  For information regarding dynamic creation of data sets, see the ALLOCATE command .

• The INCREASE command allows you to add data sets to the file. The pages of the new data sets are automatically added to free space. The format of this command is:

  INCREASE DATASETS WITH name1[,name2 ...]

  where name1, name2, and so on are the names of the new data sets.

For example, you can issue the following commands to add free space in YOURF2 to YOURFILE.

OPEN YOURFILE UPDATE PASSWORD INCREASE DATASETS WITH YOURF2

The lettered blocks in Adding new pages with the INCREASE DATASETS command and Adding new pages with the INCREASE DATASETS command represent pages for Tables A, B, C, D, and E; the F blocks represent free pages (or FREESIZE). Now you have added all of the pages in YOURF2 to FREESIZE in YOURFILE.

YOURFILE

A	A	A	B	B
B	B	C	D	D
D	D	D	D	D
D	D	D	E	E
F	F	F	F	F

Combine YOURF2 with YOURFILE

YOURF2

F	F	F	F	F
F	F	F	F	F
F	F	F	F	F
F	F	F	F	F
F	F	F	F	F

The logical Model 204 file named YOURFILE is now composed of two data sets and two DDNAMES: YOURFILE and YOURF2. You can issue an INCREASE DATASETS command as many times as necessary to add new data sets and new pages to FREESIZE.

For example, suppose that you want to add a third data set to the CARS file, which already has two data sets.

z/OS example

//JOB INCREASE DATASETS IN FILE CARS // EXEC (Include here the basic JCL or cataloged procedure invocation provided by the Model204 system manager to run BATCH204). //CARS DD DSN=M204.FILE.CARS,DISP=SHR //CARS2 DD DSN=M204.FILE.CARS2,DISP=SHR //CARS3 DD DSN=M204.FILE.CARS3, // DISP=(NEW,CATLG),VOL=SER=DSK012, // UNIT=3380,SPACE=(TRK,27) //CCAIN DD * User 0 parameters OPEN CARS INCREASE DATASETS WITH CARS3 CLOSE ALL EOJ /*

z/VSE example

// JOB INCREASE DATASETS IN FILE CARS . . . // DLBL CARS,'M204.FILE.CARS',,DA // EXTENT ,volser // DLBL CARS2,'M204.FILE.CARS2',,DA // EXTENT ,volser // DLBL CARS3,'M204.FILE.CARS3',,DA // EXTENT ,volser // UPSI 10000000 // EXEC BATCH204,SIZE=AUTO User 0 parameters OPEN CARS INCREASE DATASETS WITH CARS3 CLOSE ALL EOJ /* /&

z/VM example

For z/VM, use the following INCREASE EXEC. First, allocate CARS3:

M204UTIL ALLOCATE M204 FILE CARS3 M (PRIMARY 95 TRK

Then invoke INCREASE EXEC:

ONLINE BYPASS INCREASE

The INCREASE EXEC follows:

&CONTROL OFF FILEDEF CLEAR * FILEDEF CCASNAP PRINTER FILEDEF CCATEMP DISK CCATEMP WORK ( XTENT 20 LRECL 6184 FILEDEF CCASTAT M DSN WORK CCASTAT FILEDEF CCAPRINT DISK INCREASE CCAPRINT A FILEDEF CCAAUDIT DISK INCREASE CCAAUDIT A FILEDEF CARS M DSN M204 FILE CARS FILEDEF CARS2 M DSN M204 FILE CARS2 FILEDEF CARS3 M DSN M204 FILE CARS3 FILEDEF CCAIN DISK INCREASE CCAIN A &STACK SYSOPT 128 LIBUFF 600

where the CCAIN file, INCREASE CCAIN, is:

PAGESZ=6184 OPEN CARS INCREASE DATASETS WITH CARS3 EOJ

Automatic Increases

Perhaps the worst thing that can happen to a File Manager are system outages due to tables within files becoming full during processing. To prevent this, some of the functions described above can be automated so that table sizes are changed as needed.

Increasing page size automatically for Tables B, D, and X

The automatic increase option for Tables B, D, and X lets table increases occur without manual intervention. If a file has been created with this capability, an automatic increase will occur when the table becomes full due to lack of allocated pages in Tables B, D, and/or X.

Large transaction volumes or significant data growth can increase page requirements beyond what is currently allocated to the file tables. When this occurs, files are marked full, requests are cancelled, and transactions are backed out. Consequently, files may be left unusable for update processing and applications until there can be manual intervention to increase the table size.

Implementation of automatic increase capabilities in your files can reduce request cancellations and transaction back outs due to table full conditions. This feature expands the ability to provide 24/7 file and application availability.

The page increase is based on the values of the controlling parameters that are set by the file manager or system defaults that can be set by a system manager.

• For files created prior to V6R1.0 the system parameters, if set, will be used to increase supported tables that fill.
• For files created at V6R1.0 or later the file and system parameters, if set, will be used for automatic increase operations.

If the file parameters are not set, but the system parameters are set, they will be used. If both file and system parameters are set, the file parameters take precedence. If none of the automatic increase parameters are set, no automatic increases can occur.

Automatic increase usage

The automatic increase operation for Tables B, D, and X can add up to FREESIZE pages to those tables when the file full condition occurs and to avoid the negative consequences of a table filling. Automatic increase will reallocate pages from FREESIZE, if pages are available, to the appropriate table. The page increase is based on the values of the controlling parameters that are set by the file and system managers.

• When one of these tables fills, only that table is increased.
• A following message is sent to the audit trail regarding an automatic increase event. Also, this event creates a discontinuity and the information is written to the journal.

  M204.2835: %F TABLE %C INCREASED BY %C PAGES AUTOMATICALLY

• The automatic increase operation is a non-backoutable transaction, even if the file full event is backed out. As is the case with all file discontinuities, recovery cannot roll back across the automatic increase discontinuity.
• Automatic increase for Tables B and D is supported for entry order, unordered, and sorted files. Entry order and unordered organizations also support the automatic increase of Table X. For hash key files automatic increase is supported only for Table D.

Enabling automatic increase

To use automatic increase the files must be created under V6R1.0 or later. The file and system parameters that control automatic increase are: BAUTOINC, DAUTOINC, XAUTOINC, and / or the DEFINCP parameter.

To implement automatic increase capabilities for files created prior to V6R1.0 you must reorganize the files under a supporting Model 204 release.

DEFINCP system parameter

DEFINCP can be set in the job stream or set by a system manager.

The DEFINCP value represents the percentage of the current table size by which that table can be automatically increased if it fills during processing. DEFINCP applies to Tables B, D, and X based on file organization (Hash key files cannot have BAUTOINC set).

The MAXINCBP system parameter sets a limit, as a percentage of BSIZE, on how much Table B can auto increase by if BAUTOINC is also set.

When an automatic increase takes place

If FREESIZE is 0 or all automatic increase parameter values are 0, no automatic increase can occur and the default processing for a table full condition results.

When a file is opened, no automatic increase occurs for any table. This avoids increasing a file at open time that may have problems with recovery. The DEFINCP parameter can be set with a default percentage increase for Tables B, D, and X and the table lacking space will be increased dynamically. When a table becomes full, Model 204 checks the corresponding automatic increase value--BAUTOINC, DAUTOINC, or XAUTOINC. If it is zero, then it checks DEFINCP. If it is not zero, the table is increased by DEFINCP percent.

MAXINCBP with Table B automatic increase

The MAXINCBP parameter works for Table B automatic increase the same way as for manual increases (described above).

Parameters that affect table size

The following file, system, and table parameters are relevant to the INCREASE and DECREASE commands and the Automatic Increase feature, because they enable you to determine space usage and space availability in Tables B, D, and X.

Parameter	Type	Specifies...
BAUTOINC	FILE	Number of pages for Table B automatic increase
BHIGHPG	TABLES	Highest active Table B page number. Because page numbers start at zero, the number of Table B pages concurrently in use is equal to (BHIGHPG + 1).
BRLIMSZ	TABLES	Table B base record limit size
BSIZE	TABLES	Current number of pages assigned to Table B
DAUTOINC	FILE	Number of pages for Table d automatic increase
DEFINCP	SYSTEM	Default automatic table size percentage increase
DHIGHPG	TABLES	Highest Table D page number that has been used (the Table D highwater mark for the file)
DPGSUSED	FILE	Number of Table D pages in use
DSIZE	TABLES	Current number of pages assigned to Table D
EHIGHPG	TABLES	Highest active Table E page number that has been used-the Table E highwater mark for the file
EPGSUSED	FILE	Number of Table E pages in use
ESIZE	TABLES	Current number of pages assigned to Table E
FREESIZE	TABLES	Current number of pages of free space (that is, unassigned pages)
MAXINCBP	SYSTEM	Maximum dynamic Table B percentage increase
RECRDOPT	TABLES	Record options
XAUTOINC	FILE	Number of pages for Table X automatic increase
XHIGHPG	TABLES	Table X highest active page
XSIZE	TABLES	Pages in Table X