Fast/Unload job statistics - Revision history

Dme at 14:35, 11 December 2015

2015-12-11T14:35:33Z

← Older revision		Revision as of 14:35, 11 December 2015
Line 12:		Line 12:
	All of the statistics, of course, are reported on SMF records; the printing of		All of the statistics, of course, are reported on SMF records; the printing of
	statistics on the <code>FUNPRINT</code> data set, however, is restricted to those statistics		statistics on the <code>FUNPRINT</code> data set, however, is restricted to those statistics
	that have a value greater than zero.		that have a value greater than zero. You must apply a [[Fast/Unload custom zap#smfRecn\|zap with the SMF record number]] to enable SMF record generation.

	Many of the statistics refer to I/O processing of the input <var class="product">Model 204</var> file (or files, in the case of a group).		Many of the statistics refer to I/O processing of the input <var class="product">Model 204</var> file (or files, in the case of a group).

JAL: add link

2015-04-01T15:50:16Z

add link

← Older revision		Revision as of 15:50, 1 April 2015
Line 1:		Line 1:
	<!-- Page name: Fast/Unload job statistics-->		<!-- Page name: Fast/Unload job statistics-->
	Because <var class="product">Fast/Unload</var> is essentially a performance product, it is important to maintain		Because <var class="product">Fast/Unload</var> is essentially a performance product, it is important to maintain statistics that indicate the cost of performing a <var class="product">Fast/Unload</var> and provide information that might be useful in tuning future unloads.
	statistics that indicate the cost of performing a <var class="product">Fast/Unload</var> and provide information that might be useful in tuning future unloads.
	These statistics are reported on the report data set.		These statistics are reported on the report data set.
	These statistics are reported for		These statistics are reported for the compile step, the unload step, and Ordered Index unload step separately.
	the compile step, the unload step, and Ordered Index unload step separately.

	One of these stats, "Number of extension pages in base buffer," along		One of these stats, "Number of extension pages in base buffer," along
	with some of the Table B statistics that are reported as part of ~~<var>~~[[Fast/Unload program parameters#~~fstp~~\|FSTATS]]~~</var> processing~~, can also help you detect a need for a reorganization for improving your file's performance with <var class="product">Model 204</var>.		with some of the [[Fast/Unload Extraction Language (FUEL)#fstb\|Table B statistics]] that are reported as part of [[Fast/Unload program parameters#fst\|FSTATS processing]], can also help you detect a need for a reorganization for improving your file's performance with <var class="product">Model 204</var>.
	~~(See [[Fast/Unload Extraction Language (FUEL)#fstb\|Description of Table B statistics]] for an explanation of the Table B statistics generated by <var>FSTATS</var> processing.)~~

	If you use <var class="product">Fast/Unload</var> in a z/OS environment, you can choose to generate SMF records in addition to the <var class="product">Fast/Unload</var> report.		If you use <var class="product">Fast/Unload</var> in a z/OS environment, you can choose to generate SMF records in addition to the <var class="product">Fast/Unload</var> report.

JAL: add template

2015-03-11T19:43:50Z

add template

← Older revision		Revision as of 19:43, 11 March 2015
Line 114:		Line 114:

	==See also==		==See also==
	[[Fast/Unload ~~overview#WIKFUN$$topics\|Fast/Unload topics]]~~		{{Template:Fast/Unload topic list}}

JAL: 4.7 changes

2015-03-04T21:40:42Z

4.7 changes

← Older revision		Revision as of 21:40, 4 March 2015
Line 109:		Line 109:
	<tr><th>Storage for occurrence pointer arrays</th>		<tr><th>Storage for occurrence pointer arrays</th>
	<td>Storage usage for pointers to field and fieldgroup occurrences that are referenced in the FUEL program.		<td>Storage usage for pointers to field and fieldgroup occurrences that are referenced in the FUEL program.
			<p>
			This statistic was new in Fast/Unload version 4.7.</p>
	</table>		</table>

	==See also==		==See also==
	[[Fast/Unload overview#WIKFUN$$topics\|Fast/Unload topics]]		[[Fast/Unload overview#WIKFUN$$topics\|Fast/Unload topics]]

JAL: 4.7 changes

2015-03-04T21:39:01Z

4.7 changes

← Older revision		Revision as of 21:39, 4 March 2015
Line 106:		Line 106:
	<tr><th>Total time</th>		<tr><th>Total time</th>
	<td>Total time spent in the compile or unload step. This is both figuratively and literally the bottom line for <var class="product">Fast/Unload</var> statistics. Since the object of <var class="product">Fast/Unload</var> is to unload data as quickly as possible, this figure is a measure of your success. If you get an unexpectedly high total time value, you should examine the individual components of total time and attempt to correct the dominating components.</td></tr>		<td>Total time spent in the compile or unload step. This is both figuratively and literally the bottom line for <var class="product">Fast/Unload</var> statistics. Since the object of <var class="product">Fast/Unload</var> is to unload data as quickly as possible, this figure is a measure of your success. If you get an unexpectedly high total time value, you should examine the individual components of total time and attempt to correct the dominating components.</td></tr>

			<tr><th>Storage for occurrence pointer arrays</th>
			<td>Storage usage for pointers to field and fieldgroup occurrences that are referenced in the FUEL program.
	</table>		</table>

	==See also==		==See also==
	[[Fast/Unload overview#WIKFUN$$topics\|Fast/Unload topics]]		[[Fast/Unload overview#WIKFUN$$topics\|Fast/Unload topics]]

JAL: link repair

2015-03-02T21:27:07Z

link repair

← Older revision		Revision as of 21:27, 2 March 2015
Line 7:		Line 7:

	One of these stats, "Number of extension pages in base buffer," along		One of these stats, "Number of extension pages in base buffer," along
	with some of the Table B statistics that are reported as part of <var>[[~~FSTATS parameter~~\|FSTATS]]</var> processing, can also help you detect a need for a reorganization for		with some of the Table B statistics that are reported as part of <var>[[Fast/Unload program parameters#fstp\|FSTATS]]</var> processing, can also help you detect a need for a reorganization for improving your file's performance with <var class="product">Model 204</var>.
	improving your file's performance with <var class="product">Model 204</var>.
	(See [[Fast/Unload Extraction Language (FUEL)#fstb\|Description of Table B statistics]] for an explanation of the Table B statistics generated by <var>FSTATS</var> processing.)		(See [[Fast/Unload Extraction Language (FUEL)#fstb\|Description of Table B statistics]] for an explanation of the Table B statistics generated by <var>FSTATS</var> processing.)

JAL: more conversion cleanup

2015-02-24T19:40:49Z

more conversion cleanup

Show changes

DmeWiccan: Automatically generated page update

2015-01-15T19:40:08Z

Automatically generated page update

← Older revision		Revision as of 19:40, 15 January 2015
Line 14:		Line 14:
	processing, can also help you detect a need for a reorganization for		processing, can also help you detect a need for a reorganization for
	improving your file's performance with <var class="product">Model 204</var>.		improving your file's performance with <var class="product">Model 204</var>.
	(See [[#fstb\|Description of Table B statistics]] for an explanation of the Table B statistics generated		(See [[Fast/Unload Extraction Language (FUEL)#fstb\|Description of Table B statistics]] for an explanation of the Table B statistics generated
	by FSTATS processing.)		by FSTATS processing.)
	<p></p>		<p></p>
Line 38:		Line 38:
	<tr><th>Base buffer reads</th><td>Total number of EXCPs issued to read a track or group of tracks into a base buffer. It is directly affected by the SEBUFF parameter and the number of <var class="product">Model 204</var> input records (that is, base records) processed. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>		<tr><th>Base buffer reads</th><td>Total number of EXCPs issued to read a track or group of tracks into a base buffer. It is directly affected by the SEBUFF parameter and the number of <var class="product">Model 204</var> input records (that is, base records) processed. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
	<tr><th>Base buffer waits</th><td>Total number of WAITs issued after initiating reads into the base buffers; the ratio of EXCPs to WAITs indicates, to some extent, the degree of overlap of CPU with I/O processing. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>		<tr><th>Base buffer waits</th><td>Total number of WAITs issued after initiating reads into the base buffers; the ratio of EXCPs to WAITs indicates, to some extent, the degree of overlap of CPU with I/O processing. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
	<tr><th>Extension pg in base buffs</th><td>Total number of <var class="product">Model 204</var> file pages accessed "directly" which were currently available in pages read into base buffers. This indicates how many extension records were "physically close" to their base record during the Unload phase; specifically, within the SBBUFF times NBBUFF tracks starting with the group of SBBUFF tracks containing the base record. A large number for this indicates that <var class="product">Model 204</var> database processing can efficiently access the extension records accompanying a base record, with little physical disk manipulation (or even none, with caching devices). As mentioned in [[#fstb\|Description of Table B statistics]], this information can be used in addition to the FSTATS information about extension records, to help you determine possible benefits obtained by reorganizing a file; if extension records are frequently accessed in <var class="product">Model 204</var> processing and a large number of them are not physically close to the base record, performance may suffer to some degree. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>		<tr><th>Extension pg in base buffs</th><td>Total number of <var class="product">Model 204</var> file pages accessed "directly" which were currently available in pages read into base buffers. This indicates how many extension records were "physically close" to their base record during the Unload phase; specifically, within the SBBUFF times NBBUFF tracks starting with the group of SBBUFF tracks containing the base record. A large number for this indicates that <var class="product">Model 204</var> database processing can efficiently access the extension records accompanying a base record, with little physical disk manipulation (or even none, with caching devices). As mentioned in [[Fast/Unload Extraction Language (FUEL)#fstb\|Description of Table B statistics]], this information can be used in addition to the FSTATS information about extension records, to help you determine possible benefits obtained by reorganizing a file; if extension records are frequently accessed in <var class="product">Model 204</var> processing and a large number of them are not physically close to the base record, performance may suffer to some degree. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
	<tr><th>Extension pg in exten pool</th><td>Total number of <var class="product">Model 204</var> file pages accessed "directly" which were not currently available in pages read into base buffers but which were available in the pool of extension pages. The larger this number is, the fewer EXCPs are needed to re-read pages, and this statistic will increase (up to the limit of the number of pages needed) as NEBUFF is increased; you can make the tradeoff of costs for real memory due to increased NEBUFF versus the the costs for I/O. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>		<tr><th>Extension pg in exten pool</th><td>Total number of <var class="product">Model 204</var> file pages accessed "directly" which were not currently available in pages read into base buffers but which were available in the pool of extension pages. The larger this number is, the fewer EXCPs are needed to re-read pages, and this statistic will increase (up to the limit of the number of pages needed) as NEBUFF is increased; you can make the tradeoff of costs for real memory due to increased NEBUFF versus the the costs for I/O. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
	<tr><th>Extension buffer reads</th><td>Total number of EXCPs to read pages accessed "directly". It is the number of such pages that could not be found either in the base buffers or in the extension buffer pool. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>		<tr><th>Extension buffer reads</th><td>Total number of EXCPs to read pages accessed "directly". It is the number of such pages that could not be found either in the base buffers or in the extension buffer pool. <p></p> This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>

DmeWiccan: Automatically generated page update

2015-01-14T00:49:10Z

Automatically generated page update

New page

Because <var class="product">Fast/Unload</var> is essentially a performance product, it is important to
maintain
statistics that indicate the cost of performing a <var class="product">Fast/Unload</var> and provide
information that might be useful in tuning future unloads.
These statistics are reported on the report data set.
These statistics are
reported for
the compile step, the unload step, and Ordered Index unload step separately.

One of these stats, "Number of extension pages in base buffer", along
with some of the Table B statistics that are reported as part of FSTATS
processing, can also help you detect a need for a reorganization for
improving your file's performance with <var class="product">Model 204</var>.
(See [[#fstb|Description of Table B statistics]] for an explanation of the Table B statistics generated
by FSTATS processing.)

If you use <var class="product">Fast/Unload</var> in a z/OS environment, you can choose to generate SMF
records in addition to the <var class="product">Fast/Unload</var> report.
See [[Fast/Unload SMF record format]] for more
information about <var class="product">Fast/Unload</var> SMF records.
All of the statistics, of course, are reported on SMF records; the printing of
statistics on the FUNPRINT dataset, however, is restricted to those statistics
which have a value greater tha zero.

Many of the statistics refer to I/O processing of the input <var class="product">Model 204</var> file
(or files, in the case of a group).
There are two groups of these I/O statistics:
<table>
<tr><th>Base buffer statistics</th><td>These occur only during the Unload phase, and are for accessing Table B pages in page number order, that is, accessing the the base records processed during the unload.</td></tr>
<tr><th>Extension page statistics</th><td>These can apply during any phase, and are for directly reading pages from the <var class="product">Model 204</var> file in no pre-determined order. These are called extension page statistics because one type of direct page access is for extension record access during the Unload phase. There are other types of direct page access, such as reading the Table A field and coded value information during the Compile phase, reading the Existence Bit Map, and reading B-tree, list, and index bitmap pages during the Index unload phase. Note that the extension pages actually accessed in the unload phase depend on the type of processing in your FUEL program. A UAI operation will access all extensions of all records being unloaded; FSTATS processing will access all extensions of all records processed; some FUEL constructs, such as DELETE or using "#" or "*" for a field occurrence, cause all extensions of a record to be processed; otherwise, only as much of a logical <var class="product">Model 204</var> record is processed as needed to reference the fields used in the FUEL program.</td></tr>
</table>

The following is
a description of statistics that are reported:
<table>
<tr><th>Base buffer reads</th><td>Total number of EXCPs issued to read a track or group of tracks into a base buffer. It is directly affected by the SEBUFF parameter and the number of <var class="product">Model 204</var> input records (that is, base records) processed. This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
<tr><th>Base buffer waits</th><td>Total number of WAITs issued after initiating reads into the base buffers; the ratio of EXCPs to WAITs indicates, to some extent, the degree of overlap of CPU with I/O processing. This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
<tr><th>Extension pg in base buffs</th><td>Total number of <var class="product">Model 204</var> file pages accessed "directly" which were currently available in pages read into base buffers. This indicates how many extension records were "physically close" to their base record during the Unload phase; specifically, within the SBBUFF times NBBUFF tracks starting with the group of SBBUFF tracks containing the base record. A large number for this indicates that <var class="product">Model 204</var> database processing can efficiently access the extension records accompanying a base record, with little physical disk manipulation (or even none, with caching devices). As mentioned in [[#fstb|Description of Table B statistics]], this information can be used in addition to the FSTATS information about extension records, to help you determine possible benefits obtained by reorganizing a file; if extension records are frequently accessed in <var class="product">Model 204</var> processing and a large number of them are not physically close to the base record, performance may suffer to some degree. This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
<tr><th>Extension pg in exten pool</th><td>Total number of <var class="product">Model 204</var> file pages accessed "directly" which were not currently available in pages read into base buffers but which were available in the pool of extension pages. The larger this number is, the fewer EXCPs are needed to re-read pages, and this statistic will increase (up to the limit of the number of pages needed) as NEBUFF is increased; you can make the tradeoff of costs for real memory due to increased NEBUFF versus the the costs for I/O. This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
<tr><th>Extension buffer reads</th><td>Total number of EXCPs to read pages accessed "directly". It is the number of such pages that could not be found either in the base buffers or in the extension buffer pool. This statistic is new in <var class="product">Fast/Unload</var> version 4.0.</td></tr>
<tr><th>CPU time</th><td>This is the total CPU time used. If data is going out to a sort package, this CPU time value does not include the CPU used by the sort task. If this value is large, there is probably not much you can do about speeding up the unload without making changes to your FUEL program.</td></tr>
<tr><th>Waiting for CPU time</th><td>Total real time spent waiting for a CPU to run on. This figure is calculated by subtracting the total of all the wait times and the total CPU time from the total real time for the job. This value could actually include page wait time, or waits for synchronous BSAM I/O under CMS. If you have a high value for waiting for CPU time, it probably indicates severe system-wide contention for CPU or real memory resources.</td></tr>
<tr><th>Report buffer wait time</th><td>Total real time spent waiting for a report buffer to be written. This value should always be relatively small unless an unload produces a huge number of reported errors (conversion, hard or missing value). <var class="product">Fast/Unload</var> is optimized for placing data into the output data set (FUNOUT). A large value for report buffer wait time indicates a misuse of the report data set for outputting large amounts of data. One can reduce report buffer wait time by using a relatively large block size on the report data sets (FUNPRINT) DD card or FILEDEF statement.</td></tr>
<tr><th>Input wait time</th><td>Total real time spent waiting for a program input record to be read in.</td></tr>
<tr><th>Open wait time</th><td>Total real time spent waiting for OPEN's of input or output data sets to complete. This would ordinarily be a relatively small value, unless one is using a data set that requires mounting or staging from a removable storage medium such as a magnetic tape.</td></tr>
<tr><th>Output buffer wait time</th><td>Total real time spent waiting for an output buffer to be written. If this value is a relatively large chunk of total real time one might be able to get better performance by increasing the blocksize on the output data set (on the output data set DD card or FILEDEF statement) or by using more output buffers (by setting the NOBUFF). If the output data set is on a disk device, better performance might be achieved by using a tape device. If you are going to an external sort routine and this value is large, your efforts might be best spent tuning the sort portion of the job.</td></tr>
<tr><th>Base buffer wait time</th><td>Total real time spent waiting for the next base record to be read in. If this value is a relatively large chunk of total real time, one might be able to get better performance by increasing the base record buffer size (by setting SBBUFF) or by increasing the number of base record buffers (by setting NBBUFF). In general, however, one is not likely to see a significant improvement from having more than 2 base record buffers.</td></tr>
<tr><th>Extension buffer wait time</th><td>Total real time spent waiting for an extension record to be read in. If this value is a relatively large chunk of total real time one might be able to get better performance by increasing the number of extension record buffers (by setting (NEBUFF) or their size (by setting SEBUFF). If this value is extremely high (more than half of total run time) better performance might be achieved by actually setting the number of base buffers to 1 (setting NBBUFF to 1), preventing base buffer read ahead from blocking extension record retrieval.</td></tr>
<tr><th>PST wait time</th><td>Total real time spent waiting for the <var class="product">Fast/Unload SOUL Interface</var> PST. This can be either to access the next file in the <var class="product">Model 204</var> GROUP being unloaded, or to access a <var class="product">Model 204</var> page that was marked as being modified when the <var class="product">Fast/Unload</var> job started.</td></tr>
<tr><th>Total time</th><td>Total time spent in the compile or unload step. This is both figuratively and literally the bottom line for <var class="product">Fast/Unload</var> statistics. Since the object of <var class="product">Fast/Unload</var> is to unload data as quickly as possible, this figure is a measure of your success. If you get an unexpectedly high total time value, you should examine the individual components of total time and attempt to correct the dominating components.</td></tr>
</table>
==See also==
[[Fast/Unload overview#WIKFUN$$topics|Fast/Unload topics]]