Fast/Unload job statistics

Because Fast/Unload is essentially a performance product, it is important to maintain statistics that indicate the cost of performing a Fast/Unload and provide information that might be useful in tuning future unloads. These statistics are reported on the report data set. These statistics are reported for the compile step, the unload step, and Ordered Index unload step separately.

One of these stats, "Number of extension pages in base buffer," along with some of the Table B statistics that are reported as part of FSTATS processing, can also help you detect a need for a reorganization for improving your file's performance with Model 204. (See Description of Table B statistics for an explanation of the Table B statistics generated by FSTATS processing.)

If you use Fast/Unload in a z/OS environment, you can choose to generate SMF records in addition to the Fast/Unload report. See Fast/Unload SMF record format for more information about Fast/Unload SMF records. All of the statistics, of course, are reported on SMF records; the printing of statistics on the FUNPRINT data set, however, is restricted to those statistics that have a value greater than zero.

Many of the statistics refer to I/O processing of the input Model 204 file (or files, in the case of a group). There are two groups of these I/O statistics:

Base buffer statistics

These occur only during the Unload phase, and are for accessing Table B pages in page number order, that is, accessing the the base records processed during the unload.

Extension page statistics

These can apply during any phase. They are for directly reading pages from the Model 204 file in no predetermined order. These are called extension page statistics, because one type of direct page access is for extension record access during the Unload phase. There are other types of direct page access, such as reading the Table A field and coded value information during the Compile phase, reading the Existence Bit Map, and reading B-tree, list, and index bitmap pages during the Index unload phase. Note that the extension pages actually accessed in the unload phase depend on the type of processing in your FUEL program: A UAI operation accesses all extensions of all records being unloaded. FSTATS processing accesses all extensions of all records processed. Some FUEL constructs, such as DELETE or using # or * for a field occurrence, cause all extensions of a record to be processed. Otherwise, only as much of a logical Model 204 record is processed as needed to reference the fields used in the FUEL program.

The following is a description of statistics that are reported:

Base buffer reads	Total number of EXCPs issued to read a track or group of tracks into a base buffer. It is directly affected by the SEBUFF parameter and the number of Model 204 input records (that is, base records) processed. This statistic was new in Fast/Unload version 4.0.
Base buffer waits	Total number of WAITs issued after initiating reads into the base buffers. The ratio of EXCPs to WAITs indicates, to some extent, the degree of overlap of CPU with I/O processing. This statistic was new in Fast/Unload version 4.0.
Extension pg in base buffs	Total number of Model 204 file pages accessed "directly" which were currently available in pages read into base buffers. This indicates how many extension records were "physically close" to their base record during the Unload phase; specifically, within the SBBUFF times NBBUFF tracks starting with the group of SBBUFF tracks containing the base record. A large number for this statistic indicates that Model 204 database processing can efficiently access the extension records accompanying a base record, with little physical disk manipulation (or even none, with caching devices). As mentioned in Description of Table B statistics, this information can be used in addition to the FSTATS information about extension records, to help you determine possible benefits obtained by reorganizing a file. If extension records are frequently accessed in Model 204 processing and a large number of them are not physically close to the base record, performance may suffer to some degree. This statistic was new in Fast/Unload version 4.0.
Extension pg in exten pool	Total number of Model 204 file pages accessed "directly" which were not currently available in pages read into base buffers but which were available in the pool of extension pages. The larger this number is, the fewer EXCPs are needed to re-read pages, and this statistic will increase (up to the limit of the number of pages needed) as NEBUFF is increased. You can make the tradeoff of costs for real memory due to increased NEBUFF versus the the costs for I/O. This statistic was new in Fast/Unload version 4.0.
Extension buffer reads	Total number of EXCPs to read pages accessed "directly." It is the number of such pages that could not be found either in the base buffers or in the extension buffer pool. This statistic was new in Fast/Unload version 4.0.
CPU time	This is the total CPU time used. If data is going out to a sort package, this CPU time value does not include the CPU used by the sort task. If this value is large, there is probably not much you can do about speeding up the unload without making changes to your FUEL program.
Waiting for CPU time	Total real time spent waiting for a CPU to run on. This figure is calculated by subtracting the total of all the wait times and the total CPU time from the total real time for the job. This value could actually include page wait time, or waits for synchronous BSAM I/O under CMS. If you have a high value for waiting for CPU time, it probably indicates severe system-wide contention for CPU or real memory resources.
Report buffer wait time	Total real time spent waiting for a report buffer to be written. This value should always be relatively small, unless an unload produces a huge number of reported errors (conversion, hard or missing value). Fast/Unload is optimized for placing data into the output data set (FUNOUT). A large value for report buffer wait time indicates a misuse of the report data set for outputting large amounts of data. One can reduce report buffer wait time by using a relatively large block size on the report data sets (FUNPRINT) DD card or FILEDEF statement.
Input wait time	Total real time spent waiting for a program input record to be read in.
Open wait time	Total real time spent waiting for OPEN's of input or output data sets to complete. This would ordinarily be a relatively small value, unless one is using a data set that requires mounting or staging from a removable storage medium such as a magnetic tape.
Output buffer wait time	Total real time spent waiting for an output buffer to be written. If this value is a relatively large chunk of total real time, you might be able to get better performance by increasing the blocksize on the output data set (on the output data set DD card or FILEDEF statement), or by using more output buffers (by setting NOBUFF). If the output data set is on a disk device, better performance might be achieved by using a tape device. If you are going to an external sort routine and this value is large, your efforts might be best spent tuning the sort portion of the job.
Base buffer wait time	Total real time spent waiting for the next base record to be read in. If this value is a relatively large chunk of total real time, one might be able to get better performance by increasing the base record buffer size (by setting SBBUFF) or by increasing the number of base record buffers (by setting NBBUFF). In general, however, one is not likely to see a significant improvement from having more than two base record buffers.
Extension buffer wait time	Total real time spent waiting for an extension record to be read in. If this value is a relatively large chunk of total real time, you might be able to get better performance by increasing the number of extension record buffers (by setting (NEBUFF) or their size (by setting SEBUFF). If this value is extremely high (more than half of total run time), better performance might be achieved by actually setting the number of base buffers to 1 (setting NBBUFF to 1), preventing base buffer read ahead from blocking extension record retrieval.
PST wait time	Total real time spent waiting for the Fast/Unload SOUL Interface PST. This can be either to access the next file in the Model 204 GROUP being unloaded, or to access a Model 204 page that was marked as being modified when the Fast/Unload job started.
Total time	Total time spent in the compile or unload step. This is both figuratively and literally the bottom line for Fast/Unload statistics. Since the object of Fast/Unload is to unload data as quickly as possible, this figure is a measure of your success. If you get an unexpectedly high total time value, you should examine the individual components of total time and attempt to correct the dominating components.
Storage for occurrence pointer arrays	Storage usage for pointers to field and fieldgroup occurrences that are referenced in the FUEL program. This statistic was new in Fast/Unload version 4.7.

See also

Fast/Unload topics