Managing file and table sizes

From m204wiki
Revision as of 22:12, 30 March 2013 by Rob (talk | contribs)
Jump to navigation Jump to search

When you create a file and its tables, whether you use the calculations described in File Size Calculation to size the tables or the rules of thumb that an experienced File Manager developr over time, there is always some space for growth included. When a Model 204 file is created, exact disk space requirements are seldom known. Even if perfect, eventually, it will be necessary to revise the space utilized in s file.

There are three ways to maintain table sizes: the FILEMGMT subsystem (which see); the File Manager may use a number of commands to maintain the table sizes; or some of those functions may be automated by setting up the 'Auto Increase' parameters in the file so that free space is added to the tables as needed.

The latter two are described below.

Overview of Capabilities

Because of the hashed nature of Tables A and C, these tables can only be changed with the CREATE command during a file reorganisation.

The other tables may be modified using the INCREASE and DECREASE command, as follows:

Command Description Exclusive Access? Limitations
INCREASE TABLEB Move pages from free space to Table B No, but the DYNAMIC sub-parameter
is required if you don't have exclusive
access 		segment boundaries need to be taken into account
the MAXINCBP parameter needs to be set if you wish to increase Table B beyond the current segment boundary.
INCREASE TABLED
TABLEE or
TABLEX 		Move pages from free space to this table No Up to maximum table size
DECREASE TABLEB
TABLED
TABLEE
TABLEX 		Move pages to free space from the respective table Yes Only if the ?HIGHPG (for the respective table) statistic permits it
INCREASE DATASET Add an additional dataset to a Model 204 file. No A model 204 file may have up to 255 datasets

The additional pages are formatted as free space


Increasing Tables B, D, E, and X

You can increase manually Tables B, D, E, and X by assigning pages from free space (with special considerations for Table B described below). Therefore, you must examine the FREESIZE parameter using the VIEW command to make sure that unassigned pages are available. If they are not, see Increasing free space manually.

The format of the INCREASE command is:

Syntax

INCREASE {TABLEB | TABLED | TABLEE | TABLEX} n [DYNAMIC]

where:

n is a decimal number from 1 to FREESIZE.

Example

INCREASE TABLEE 2

Usage notes

The file must be open exclusively to you before you issue a command to increase a table. Executing the command immediately changes the values of FREESIZE and either BSIZE, DSIZE, ESIZE or XSIZE, respectively.

The DYNAMIC option applies to only an INCREASE TABLEB command.

Increasing Table B with the DYNAMIC option

You can specify DYNAMIC (or DYN) on the INCREASE TABLEB command to force the increase to occur even though requests have already been compiled against the file. The command could be:

INCREASE TABLEB 10000 DYNAMIC

The increase is limited to the number of pages specified, as a percentage of BSIZE, by the MAXINCBP parameter. If MAXINCBP is zero, the increase is limited to a number of pages that ensure that the current number of file segments does not increase.

For more informaiton about MAXINCBP, see MAXINCBP system parameter.

You can set the DEFINCP system parameter to specify a default percentage increase in Table B size for a file that fills up the current Table B allocation. For example, if a file has 100,000 Table B pages and DEFINCP is set to 10, when Table B fills, Table B will be increased to 110,000; then, 121,000; then, 133,100 pages and so on. The amount of increase is reduced subject to FREESIZE and MAXINCBP restrictions.

Decreasing free space

Free space can be decreased either by decreasing Table(s) B, D, and/or E using:

Decreasing Tables B, D, or E

Tables B, D, and E are decreased by reassigning unused pages in them to free space. Therefore, the parameters BSIZE and BHIGHPG, DSIZE and DHIGHPG, or ESIZE and EHIGHPG must be viewed to determine the number of unused pages in the table in question.

The format of the DECREASE command is:

Syntax

DECREASE {TABLEB | TABLED | TABLEE} n

where:

n is a decimal number.

  • For TABLEB, n can be in the range 1 through the value of BSIZE, minus BHIGHPG, minus 1.
  • For TABLED, n can be in the range 1 through DSIZE, minus DHIGHPG, minus 1. DHIGHPG represents the highest Table D page used.

    DECREASE processing can remove only a contiguous set of pages from the end of a table. Because of the way in which Table D pages are managed, the unused pages might not be contiguous. This is especially likely in files where procedures are heavily updated. Under these circumstances, Table D cannot be decreased by the full number of unused pages. If the DECREASE command specifies more pages than can be reassigned to free space, you receive the message "NOT ENOUGH ROOM IN TABLE." If this happens, try the DECREASE command again with a smaller number.

  • For TABLEE, n can be in the range of 0 through ESIZE, minus EHIGHPG, minus 1.

You must open the file exclusively before you issue the DECREASE command. Processing the command automatically changes the values of FREESIZE and BSIZE, DSIZE, or ESIZE, respectively.

Parameters that affect table size

The following file, system, and table parameters are relevant to the INCREASE and DECREASE commands and the Automatic Increase feature, because they enable you to determine space usage and space availability in Tables B, D, and X.

Parameter Type Specifies...
BAUTOINC FILE Number of pages for Table B automatic increase
BHIGHPG TABLES Highest active Table B page number. Because page numbers start at zero, the number of Table B pages concurrently in use is equal to (BHIGHPG + 1).
BRLIMSZ TABLES Table B base record limit size
BSIZE TABLES Current number of pages assigned to Table B
DAUTOINC FILE Number of pages for Table d automatic increase
DEFINCP SYSTEM Default automatic table size percentage increase
DHIGHPG TABLES Highest Table D page number that has been used (the Table D highwater mark for the file)
DPGSUSED FILE Number of Table D pages in use
DSIZE TABLES Current number of pages assigned to Table D
EHIGHPG TABLES Highest active Table E page number that has been used-the Table E highwater mark for the file
EPGSUSED FILE Number of Table E pages in use
ESIZE TABLES Current number of pages assigned to Table E
FREESIZE TABLES Current number of pages of free space (that is, unassigned pages)
MAXINCBP SYSTEM Maximum dynamic Table B percentage increase
RECRDOPT TABLES Record options
XAUTOINC FILE Number of pages for Table X automatic increase
XHIGHPG TABLES Table X highest active page
XSIZE TABLES Pages in Table X

 

Increase Table B

While generally similar to the increases to the other tables, there are some limitations when you do not have exclusive access to the files. If you do have exclusive access to the file, Table B may be increased

MAXINCBP system parameter

The MAXINCBP system parameter sets a limit, as a percentage of BSIZE, on how much Table B can increase by, either by the INCREASE TABLE B command or by automatic increase (if BAUTOINC is also set). This percentage is then used to calculate the maximum number of segments, rounded up to a whole number, that the file could have if Table B were increased by the entire amount. The actual number of pages that BSIZE can increase to is determined by the calculated number of segments, divided by BRECPPG.

The compiler uses the maximum possible number of segments to determine the correct length of the entries it allocates in the resource locking table and in VTBL. During compilation the file is treated as though it has already been increased to the maximum number of segments (MAXINCBP) even though no increase may yet have occurred.

The total Table B expansion with no MAXINCBP is limited to existing number of segments. MAXINCBP defines the maximum size of Table B, but if DEFINCP is not set nor the file's BAUTOINC, then auto increase does not happen. If a file is opened without MAXINCBP set yet, then no increase in number of segments is possible. MAXINCBP must be set (or reset) before the file is opened.

The MAXINCBP parameter is ignored for any hash key files.

Example of Table B automatic increase

For example, take a file with BSIZE=100000, BRECPPG=10. For purposes of allocating entries in the record locking table and VTBL, the compiler treats this file as though all records are present and it is therefore, a 21-segment ((10*100000)/49152) file. However, if the system parameter MAXINCBP=20, then Table B in this file could increase by 20,000 pages. The resulting Table B of 120,000 pages would be 24.4-segment ((120000*10)/49152) segment file. This would be rounded up to 25 segments, so BSIZE could actually be increased to 122,880 before crossing into the twenty-sixth segment.

These calculations occur when each file is opened for the first time and affect the compilation of requests from that time forward, for that run.

MAXINCBP and DEFINCP precedence over the BAUTOINC parameter

The MAXINCBP and DEFINCP parameters are system parameters and take precedence over the file-level BAUTOINC parameter.

If the MAXINCBP parameter is set to a non-zero value, when a file is first opened the value of the BAUTOINC parameter is ignored. This is because the MAXINCBP parameter is already set up to facilitate an increase later as needed.

If BAUTOINC is non-zero in a file that needs an automatic increase, it is used to determine the size of the increase. If BAUTOINC is zero, DEFINCP is used. If DEFINCP is also zero, no increase is made to Table B.

Increasing LRETBL to accommodate MAXINCBP

Setting MAXINCBP might require a corresponding increase to LRETBL. That is, if MAXINCBP is 10, you might want to increase LRETBL by 10%. MAXINCBP is the upper boundary on the required increase-the actual increase will be somewhat less.

Space Management

The number of pages in a Model 204 table can be increased using the INCREASE TABLEB, INCREASE TABLED, or INCREASE TABLEE commands. If there are insufficient pages in FREESIZE to accommodate the increase, new pages can be added, also dynamically, to FREESIZE using the INCREASE DATASETS command.

When data growth can be anticipated, these increases can be made in advance and file availability remains unaffected. However, these increases are not automatic. You must anticipate the increase required and issue the commands manually prior to running update transactions that require the additional pages.

When it becomes necessary to increase the size of Table B, D, E, or X, free space might not contain enough unassigned pages.

Choose one of the following methods to increase free space:

  • Decrease some other table.

    For example, if only two pages are in the free space of file CARS, and three unused pages are in Table B, the following sequence of commands adds five pages to Table D:

    OPEN CARS DECREASE TABLEB 3 INCREASE TABLED 5

  • Use DUMP and RESTORE to increase the size or number of the file data sets (see File Dumping and Restoring).

Adding new pages with the INCREASE DATASETS command

When free space is exhausted, you can manually add new pages dynamically with the INCREASE DATASETS command.

  • First, allocate a new data set, perhaps using the ALLOCATE command as follows:

    ALLOCATE YOURF2 WITH SCOPE=SYSTEM SEQUENTIAL CATALOG - VOLUME=CCAVOL UNIT=3390 PRI 10 CYL BLKSIZE=6184 - DSN=CCA.YOURFILE.YOURF2.M204

    For information regarding dynamic creation of data sets, see the ALLOCATE command .

  • The INCREASE command allows you to add data sets to the file. The pages of the new data sets are automatically added to free space. The format of this command is:

    INCREASE DATASETS WITH name1[,name2 ...]

    where name1, name2, and so on are the names of the new data sets.

For example, you can issue the following commands to add free space in YOURF2 to YOURFILE.

OPEN YOURFILE UPDATE PASSWORD INCREASE DATASETS WITH YOURF2

The lettered blocks in Adding new pages with the INCREASE DATASETS command and Adding new pages with the INCREASE DATASETS command represent pages for Tables A, B, C, D, and E; the F blocks represent free pages (or FREESIZE). Now you have added all of the pages in YOURF2 to FREESIZE in YOURFILE.

YOURFILE
A A A B B
B B C D D
D D D D D
D D D E E
F F F F F

Combine YOURF2 with YOURFILE

YOURF2
F F F F F
F F F F F
F F F F F
F F F F F
F F F F F

The logical Model 204 file named YOURFILE is now composed of two data sets and two DDNAMES: YOURFILE and YOURF2. You can issue an INCREASE DATASETS command as many times as necessary to add new data sets and new pages to FREESIZE.

For example, suppose that you want to add a third data set to the CARS file, which already has two data sets.

z/OS example

//JOB INCREASE DATASETS IN FILE CARS // EXEC (Include here the basic JCL or cataloged procedure invocation provided by the Model204 system manager to run BATCH204). //CARS DD DSN=M204.FILE.CARS,DISP=SHR //CARS2 DD DSN=M204.FILE.CARS2,DISP=SHR //CARS3 DD DSN=M204.FILE.CARS3, // DISP=(NEW,CATLG),VOL=SER=DSK012, // UNIT=3380,SPACE=(TRK,27) //CCAIN DD * User 0 parameters OPEN CARS INCREASE DATASETS WITH CARS3 CLOSE ALL EOJ /*

z/VSE example

// JOB INCREASE DATASETS IN FILE CARS . . . // DLBL CARS,'M204.FILE.CARS',,DA // EXTENT ,volser // DLBL CARS2,'M204.FILE.CARS2',,DA // EXTENT ,volser // DLBL CARS3,'M204.FILE.CARS3',,DA // EXTENT ,volser // UPSI 10000000 // EXEC BATCH204,SIZE=AUTO User 0 parameters OPEN CARS INCREASE DATASETS WITH CARS3 CLOSE ALL EOJ /* /&

z/VM example

For z/VM, use the following INCREASE EXEC. First, allocate CARS3:

M204UTIL ALLOCATE M204 FILE CARS3 M (PRIMARY 95 TRK

Then invoke INCREASE EXEC:

ONLINE BYPASS INCREASE

The INCREASE EXEC follows:

&CONTROL OFF FILEDEF CLEAR * FILEDEF CCASNAP PRINTER FILEDEF CCATEMP DISK CCATEMP WORK ( XTENT 20 LRECL 6184 FILEDEF CCASTAT M DSN WORK CCASTAT FILEDEF CCAPRINT DISK INCREASE CCAPRINT A FILEDEF CCAAUDIT DISK INCREASE CCAAUDIT A FILEDEF CARS M DSN M204 FILE CARS FILEDEF CARS2 M DSN M204 FILE CARS2 FILEDEF CARS3 M DSN M204 FILE CARS3 FILEDEF CCAIN DISK INCREASE CCAIN A &STACK SYSOPT 128 LIBUFF 600

where the CCAIN file, INCREASE CCAIN, is:

PAGESZ=6184 OPEN CARS INCREASE DATASETS WITH CARS3 EOJ

Automatic Increases

Increasing page size automatically for Tables B, D, and X

The automatic increase option for Tables B, D, and X lets table increases occur without manual intervention. If a file has been created with this capability, an automatic increase will occur when the table becomes full due to lack of allocated pages in Tables B, D, and/or X.

Large transaction volumes or significant data growth can increase page requirements beyond what is currently allocated to the file tables. When this occurs, files are marked full, requests are cancelled, and transactions are backed out. Consequently, files may be left unusable for update processing and applications until there can be manual intervention to increase the table size.

Implementation of automatic increase capabilities in your files can reduce request cancellations and transaction back outs due to table full conditions. This feature expands the ability to provide 24/7 file and application availability.

The page increase is based on the values of the controlling parameters that are set by the file manager or system defaults that can be set by a system manager.

  • For files created prior to V6R1.0 the system parameters, if set, will be used to increase supported tables that fill.
  • For files created at V6R1.0 or later the file and system parameters, if set, will be used for automatic increase operations.

If the file parameters are not set, but the system parameters are set, they will be used. If both file and system parameters are set, the file parameters take precedence. If none of the automatic increase parameters are set, no automatic increases can occur.

Automatic increase usage

The automatic increase operation for Tables B, D, and X can add up to FREESIZE pages to those tables when the file full condition occurs and to avoid the negative consequences of a table filling. Automatic increase will automatically reallocate pages from FREESIZE, if pages are available, to the appropriate table. The page increase is based on the values of the controlling parameters that are set by the file and system managers.

  • When one of these tables fills, only that table is increased.
  • A following message is sent to the audit trail regarding an automatic increase event. Also, this event creates a discontinuity and the information is written to the journal.

    M204.2835: %F TABLE %C INCREASED BY %C PAGES AUTOMATICALLY

  • The automatic increase operation is a non-backoutable transaction, even if the file full event is backed out. As is the case with all file discontinuities, recovery cannot roll back across the automatic increase discontinuity.
  • Automatic increase for Tables B and D is supported for entry order, unordered, and sorted files. Entry order and unordered organizations also support the automatic increase of Table X. For hash key files automatic increase is supported only for Table D.

Enabling automatic increase

To use automatic increase the files must be created under V6R1.0 or later. The file and system parameters that control automatic increase are: BAUTOINC, DAUTOINC, XAUTOINC, DEFINCP, and MAXINCBP. See Model 204 Parameter and Command Reference for the details on setting and resetting these parameters.

To implement automatic increase capabilities for files created prior to V6R1.0 you must reorganize the files under a supporting Model 204 release.

When an automatic increase takes place

If FREESIZE is 0 or all automatic increase parameter values are 0, no automatic increase can occur and the default processing for a table full condition results.

When a file is opened, no automatic increase occurs for any table. This avoids increasing a file at open time that may have problems with recovery. The DEFINCP parameter can be set with a default percentage increase for Tables B, D, and X and the table lacking space will be increased dynamically. When a table becomes full, Model 204 checks the corresponding automatic increase value--BAUTOINC, DAUTOINC, or XAUTOINC. If it is zero, then it checks DEFINCP. If it is not zero, the table is increased by DEFINCP percent.

Table B automatic increase - BAUTOINC, DEFINCP, and MAXINCBP

Set the BAUTOINC file parameter to the number of pages you want reallocated to Table B from FREESIZE pages when a Table B full condition occurs. If BAUTOINC is set to 0, the value of the DEFINCP system parameter is checked. If DEFINCP is a non-zero value, Table B is increased by DEFINCP percentage of BSIZE pages.

DEFINCP system parameter

DEFINCP can be set in the job stream or set by a system manager.

The DEFINCP value represents the percentage of the current table size by which that table can be automatically increased if it fills during processing. DEFINCP applies to Tables B, D, and X based on file organization.

MAXINCBP system parameter

The MAXINCBP system parameter sets a limit, as a percentage of BSIZE, on how much Table B can increase by, either by the INCREASE TABLE B command or by automatic increase (if BAUTOINC is also set). This percentage is then used to calculate the maximum number of segments, rounded up to a whole number, that the file could have if Table B were increased by the entire amount. The actual number of pages that BSIZE can increase to is determined by the calculated number of segments, divided by BRECPPG.

The compiler uses the maximum possible number of segments to determine the correct length of the entries it allocates in the resource locking table and in VTBL. During compilation the file is treated as though it has already been increased to the maximum number of segments (MAXINCBP) even though no increase may yet have occurred.

The total Table B expansion with no MAXINCBP is limited to existing number of segments. MAXINCBP defines the maximum size of Table B, but if DEFINCP is not set nor the file's BAUTOINC, then auto increase does not happen. If a file is opened without MAXINCBP set yet, then no increase in number of segments is possible. MAXINCBP must be set (or reset) before the file is opened.

The MAXINCBP parameter is ignored for any hash key files.

Example of Table B automatic increase

For example, take a file with BSIZE=100000, BRECPPG=10. For purposes of allocating entries in the record locking table and VTBL, the compiler treats this file as though all records are present and it is therefore, a 21-segment ((10*100000)/49152) file. However, if the system parameter MAXINCBP=20, then Table B in this file could increase by 20,000 pages. The resulting Table B of 120,000 pages would be 24.4-segment ((120000*10)/49152) segment file. This would be rounded up to 25 segments, so BSIZE could actually be increased to 122,880 before crossing into the twenty-sixth segment.

These calculations occur when each file is opened for the first time and affect the compilation of requests from that time forward, for that run.

MAXINCBP and DEFINCP precedence over the BAUTOINC parameter

The MAXINCBP and DEFINCP parameters are system parameters and take precedence over the file-level BAUTOINC parameter.

If the MAXINCBP parameter is set to a non-zero value, when a file is first opened the value of the BAUTOINC parameter is ignored. This is because the MAXINCBP parameter is already set up to facilitate an increase later as needed.

If BAUTOINC is non-zero in a file that needs an automatic increase, it is used to determine the size of the increase. If BAUTOINC is zero, DEFINCP is used. If DEFINCP is also zero, no increase is made to Table B.

Increasing LRETBL to accommodate MAXINCBP

Setting MAXINCBP might require a corresponding increase to LRETBL. That is, if MAXINCBP is 10, you might want to increase LRETBL by 10%. MAXINCBP is the upper boundary on the required increase-the actual increase will be somewhat less.

Table D automatic increase - DAUTOINC and DEFINCP

Set the DAUTOINC file parameter to the number of pages you want reallocated to Table D from FREESIZE pages when a Table D full condition occurs. If DAUTOINC is found to be set to 0, the value of the DEFINCP system parameter is checked and Table D is increased by DEFINCP percentage of DSIZE.

For information about DEFINCP, see DEFINCP system parameter.

Table X automatic increase - XAUTOINC and DEFINCP

Set the XAUTOINC file parameter to the number of pages you want reallocated to Table X from FREESIZE pages when a Table X full condition occurs. If XAUTOINC is found to be set to 0, the value of the DEFINCP system parameter is checked and Table X is increased by DEFINCP percentage of XSIZE.

For more information about DEFINCP, see DEFINCP system parameter.

You can implement the XAUTOINC parameter for files created in V7R1.0 or later that have a Table X, and you can view it. A file has a Table X when XSIZE is greater than zero.

  • If there are enough FREESIZE pages, the auto increase takes place.
  • If there are some FREESIZE pages, but not enough to satisfy the XAUTOINC value, the auto increase proceeds until FREESIZE is exhausted.
  • More pages may be added to Table X to add one more segment to the file. See the Rocket Model 204 File Manager's Guide.

When the file becomes full from lack of space in Table X, an automatic increase is done by comparing and using the smaller number of the following:

  • The value of XAUTOINC
  • The number of pages in the free space
Retrieved from "https://m204wiki.rocketsoftware.com/index.php?title=Managing_file_and_table_sizes&oldid=54649"