SirTune user states
Model 204 user states
When the SirTune sampling program is collecting a sample it scans all logged on users. Each user is classified by its state. The user's state is a general indication of the type of activity occurring in a user thread. These states roughly correspond to the states reported by the Model 204 performance monitor, though broken down to a finer level of detail.
The following primary states are distinguished by SirTune:
|BLKIN||This includes any user that is blocked, that is waiting for something, in a server and not waiting for user input. This is distinguished from BLKIU because waits for things other than user input are generally viewed as a performance problem while waits for user input are not.|
|BLKIU||This includes any user that is blocked, that is waiting for something, in a server and waiting for user input. This is distinguished from BLKIN because waits for things other than user input are generally viewed as a performance problem while waits for user input are not.|
|BLKON||This includes any user that is blocked, that is waiting for something, not in a server and not waiting for user input. This is distinguished from BLKOU because waits for things other than user input are generally viewed as a performance problem while waits for user input are not.|
|BLKOU||This includes any user that is blocked, that is waiting for something, not in a server and waiting for user input. This is distinguished from BLKON because waits for things other than user input are generally viewed as a performance problem while waits for user input are not.|
|REDY||This includes any user that is ready to run, that is, in a server and not waiting on anything but not actually being run. Generally a user is in state REDY because another user is currently running.|
|RUNG||This includes any user that is running, that is, using CPU. Unless MP/204 is installed, there can never be more than one user in state RUNG per sample.|
|RUNGM||If MP/204 is installed, this includes any user that is running, that is, using CPU, in maintask mode. There can never be more than one user in state RUNGM per sample. See The RUNGM and RUNGS states.|
|RUNGS||If MP/204 is installed, this includes any user that is running, that is, using CPU, in subtask mode. See The RUNGM and RUNGS states.|
|SWPGI||This includes any user that is in the process of being swapped into a server.|
|SWPGOBN||This includes any user that is in the process of being swapped out of a server because it is waiting on something other than user input. If what the user was waiting on is still not completed at the point the user is swapped out, the user switches to state BLKON.|
|SWPGOBU||This includes any user that is in the process of being swapped out of a server because it is waiting on user input. If what the user was waiting on is still not completed at the point the user is swapped out, the user switches to state BLKOU.|
|SWPGOW||This includes any user that is in the process of being swapped out of a server because it is has been server sliced. If no servers of appropriate size are available at the point the user is swapped out, the user switches to state WTSV.|
|WPST||This includes any PST that is not running.|
|WTSV||This includes any user that is waiting for a server to become available so that the user could be run. The only reason a user would be in the WTSV state is that all servers of appropriate size are occupied by other users that cannot be swapped out of server.|
In this list, the phrase "waiting for user input" refers to a thread waiting for terminal or line input. In addition, a wait for a response to the console message issued by User 0 on a HALT command is also considered a user input wait. "Sleep" waits, that is, waits resulting from the *SLEEP command and the Pause statement, are not considered user input waits.
In addition to the above primary states, several composite states are provided for convenience and report generation. For example, composite state SWPG is made up of primary states SWPGI, SWPGOBN, SWPGOBU, and SWPGOW. Thus any user in any of the indicated primary states is also considered to be in state SWPG. The following are the available composite states, their component primary states, and an explanation that suggests the meaning of the composite state.
|ALL||This is a composite state that includes all primary states. Any logged on user or PST is considered in state ALL.|
|ALLI||This state is made up of RUNG, REDY, BLKIN, and BLKIU. It includes any user currently in a server and not being swapped out. It does not include non-running PSTs.|
|ALLN||This state is made up of RUNG, REDY, BLKIN, BLKON, WTSV, SWPGI, SWPGOBN, and SWPGOW. It includes any user not blocked for user input. It does not include non-running PSTs.|
|BLK||This state is made up of BLKIN, BLKIU, BLKON, BLKOU, SWPGOBN, and SWPGOBU. It includes any user that is blocked on anything.|
|BLKI||This state is made up of BLKIN and BLKIU. It includes any user that is in a server and blocked on anything.|
|BLKN||This state is made up of BLKIN, BLKON, and SWPGOBN. It includes any user that is blocked for something other than user input.|
|BLKO||This state is made up of BLKON and BLKOU. It includes any user that is not in a server but is blocked on something.|
|BLKU||This state is made up of BLKIU, BLKOU, and SWPGOBU. It includes any user that is waiting for user input.|
|OSERVN||This state is made up of SWPGOBN and BLKON. It includes any user that is either not in a server or being swapped out of a server because it is blocked on something other than user input.|
|OSERVU||This state is made up of SWPGOBU and BLKOU. It includes any user that is either not in a server or being swapped out of a server because it is blocked on user input.|
|OSERVW||This state is made up of SWPGOW and WTSV. It includes any user that is either waiting for a server or being swapped out of a server so that it can wait for a server to free up. This latter case only happens when a user is server sliced.|
|REDYR||This state is made up of RUNG and REDY. It includes any user that is not blocked on anything and is in a server. Users in state REDYR can be either running or waiting for the Model 204 scheduler to provide CPU to run.|
|RUNBL||This state is made up of RUNG, REDY, WTSV, and SWPGOW. It includes any user that is not blocked on anything, that is, is runnable. Users in state RUNBL can be either running or waiting for the Model 204 scheduler to provide the resources (CPU and/or server) to run.|
|SWPG||This state is made up of SWPGI, SWPGOBN, SWPGOBU, and SWPGOW. It includes any user that is being swapped into or out of a server.|
|SWPGO||This state is made up of SWPGOBN, SWPGOBU, and SWPGOW. It includes any user that is being swapped out of a server.|
|SWPGOB||This state is made up of SWPGOBN and SWPGOBU. It includes any user that is being swapped out of a server because it is blocked on something.|
Specifying states in COLLECT and REPORT STATE statements
Any of the above primary or composite states can be included on COLLECT statements for input to SIRTUNEI and on REPORT STATE statements for input to SIRTUNEREPORT or SIRTUNER. Some valid COLLECT statements are:
COLLECT BLKN SWPG COLLECT ALLN COLLECT BLKIN BLKON SWPGOBN WTSV SWPGOW SWPGI
Some valid REPORT STATE statements are
REPORT STATE BLKIN EVAL REPORT STATE SWPG CHUNK 100 REPORT STATE ALLN EVAL CHUNK 1000 CHUNK 4
In addition to user states, SirTune's COLLECT statement lets you request information about DISKIO and CFR. The following is a valid COLLECT statement:
COLLECT DISKIO CFR
But there is no REPORT STATE statement that allows DISKIO nor CFR.
Any state requested in a REPORT STATE statement must have had the corresponding primary states explicitly or implicitly specified on COLLECT statements for SirTune. The simplest way to ensure this is by explicitly specifying any state to be used in a REPORT STATE statement or a COLLECT statement. For example, if you intend to produce the following reports with SIRTUNEREPORT or SIRTUNER:
REPORT STATE BLKN CHUNK 10 REPORT STATE SWPG CHUNK 10
You can code the following COLLECT statement for SirTune:
COLLECT BLKN SWPG
This statement is functionally equivalent to
COLLECT BLKIN BLKON SWPGOBN SWPGOBU SWPGOW SWPGI
In general, if running a relatively small Online (an average of less than 20 logged on users), this statement should not produce a prohibitively large amount of data and makes all reports possible:
If running a midsize to large Online (an average 20+ logged on users), the following statement should collect a sufficient quantity of data to produce most interesting STATE reports without generating a prohibitively large sample data set:
COLLECT ALLN BLKIU SWPGOBU
Specifying the RUNGM and RUNGS states
When running the MP/204 feature with Model 204, a user that is in state RUNG can be further distinguished to be either running in maintask mode (RUNGM) or subtask mode (RUNGS) for the purposes of reporting. For example, these SIRTUNEREPORT or SIRTUNER statements generate two reports:
REPORT STATE RUNGM EVAL REPORT STATE RUNGS EVAL
The first report is a breakdown of users running in maintask mode by evaluating procedure, and the second is a breakdown of users running in subtask mode by evaluating procedure. Maintask mode is often referred to as "serial" mode, and subtask mode is often referred to as "parallel" mode.
The total observations for state RUNG in any sample is always equal to the total observations for state RUNGM plus the total observations for state RUNGS.
The distinction between maintask and subtask mode can be made either on the basis of the task on which a user is running (maintask or subtask), or on its virtual (or logical) MP mode (that is, whether it is capable of running in a subtask or not).
The default distinction is made on the basis of the
actual task on which a user is running.
This can be changed with the SIRTUNER
This is generally the preferred setting when using
REPORT STATE RUNGM report to try to reduce the amount of maintask (serial) SOUL code.
Specifying reports by wait type
Users in state BLK (blocked on anything), always have a wait type
associated with them.
These wait types are the same wait types that appear
next to the users in a Model 204 MONITOR command or in the SirMon WAITTYP statistic.
STATE reports can be requested by these wait types.
To produce these STATE reports by wait type, COLLECT statements (collecting data for all states in which a wait type might occur) must be added to SirTune's input stream (
For example, disk I/O wait types are not swappable, so it is only necessary to collect state BLKIN to produce a
REPORT STATE WDISK report.
Since critical file resource waits are swappable, states BLKIN, BLKON, and SWPGOBN must all be collected to produce a
REPORT STATE WCFREX report.
The available wait type reports along with the corresponding Model 204 wait type number, a description of the wait type, and the required states to be collected are listed here:
|WMISC||0 - Miscellaneous waits. Requires BLKN.|
|WDISK||1 - Wait for disk I/O. Requires BLKIN.|
|WUSERO||2 - Wait for user output. Requires BLKU.|
|WUSERI||3 - Wait for user input. Requires BLKU.|
|WOPERI||4 - Wait for operator input. Requires BLKU.|
|WDUMPO||5 - Wait for dump write. Requires BLKIN.|
|WDUMPI||6 - Wait for restore read. Requires BLKIN.|
|WENQUE||7 - Wait for miscellaneous enqueue. Requires BLKN.|
|WBUFF||8 - Wait for disk buffer. Requires BLKIN.|
|WPST||10 - Wait on PST. Requires BLKN.|
|WIFAM||11 - IFAM waits. Requires BLKN.|
|WSLEEP||12 - Waits for a time interval, including Pause statements and *SLEEP commands. Requires BLKN.|
|WJRNLO||15 - Wait for journal output. Requires BLKIN.|
|WCHKPO||16 - Wait for checkpoint output. Requires BLKIN.|
|WWRITE||17 - Wait for a checkpoint DECB. Requires BLKIN.|
|WARBMO||18 - Waits for output arbitration. Requires BLKN.|
|WCHKPR||19 - Waits for a checkpoint request. Requires WPST.|
|WDISK||20 - Waits for checkpoint completion. Requires BLKIN.|
|WDEAD||21 - Wait forever (dead thread). Requires BLKU.|
|WVSAMI||22 - Wait for VSAM input. Requires BLKN.|
|WLOGIN||23 - Wait after login failure. Requires BLKN.|
|WCFREX||24 - Wait for critical file resource in exclusive mode. Requires BLKN.|
|WCFRSH||25 - Wait for critical file resource in share mode. Requires BLKN.|
|WVTBUF||26 - Wait for VTAM buffer. Requires BLKN.|
|WCONVI||27 - Wait for inter-process input. Requires BLKN.|
|WCONVO||28 - Wait for inter-process output. Requires BLKN.|
|WSCTYI||29 - Wait for security interface. Requires BLKN.|
|WS$WAI||30 - Swappable $Wait call. Requires BLKN.|
|WN$WAI||31 - Non-swappable $Wait call. Requires BLKIN.|
|WULDB2||32 - Wait for DB2 subtask. Requires BLKN.|
|WOCSUB||33 - Waiting on Open/Close subtask. Requires BLKIN.|
|WDBUGU||38 - Wait for user being debugged. Requires BLKN.|
|WDBUGD||39 - Wait for user performing debugging. Requires BLKN.|
|WMQTSK||40 - Wait for MQ subtask to become available. Requires BLKN.|
|WMQAPI||41 - Wait for MQ subtask to run. Requires BLKIN.|
|WMQGWT||42 - Wait for MQGET with wait time specified. Requires BLKN.|
|WECLD||43 - Wait for ECF to load/delete a module. Requires BLKN.|
|WECMOD||44 - Wait for external module to become free. Requires BLKN.|
|WECTSK||45 - Wait for ECF subtask to become free. Requires BLKN.|
|WECRUN||46 - Wait for external module to run. Requires BLKN.|
|W$WTQZ||47 - User within $WAIT('CPQZ') wait; CHKPPST within extended quiesce. Requires BLKN.|
|W$WTXS||48 - User within $WAIT('QZSIG') wait. Requires BLKN.|
|W$NDEQ||49 - At end of extended quiesce, waiting for count of $WAIT('CPQZ') and $WAIT('QZSIG') users to go to zero. Requires BLKN.|
|WHSM||50 - Wait FOR HSM recall of a migrated dataset. Requires BLKN.|
|WCDS||51 - Wait for share mode constraints DB lock. Requires BLKIN.|
|WCDX||52 - Wait for exclusive mode constraints DB lock. Requires BLKIN.|
|WSBBOL||53 - Wait for SUB-TRANS CP processing to complete for this user. Requires BLKN.|
|WSBBFC||54 - SUB-TRAN CP postponement - waiting on blocking file command to complete. Requires BLKIN.|
|WSBTMR||55 - SUB-TRAN CP CPTS timer wait. Requires BLKIN.|
|WSBARY||56 - SUB-TRAN CP scanner array wait. Requires BLKIN.|
|WDMNM||57 - A daemon child waiting on its master. Requires BLK.|
|WDMND||58 - A daemon master waiting on its daemon. Requires BLK.|
|WCUSTn||80-89 - Customer reserved wait codes.|
|WFUNLD||97 - Fast Unload request. Requires BLKN.|
|WMAXAU||98 - MAXAUSER delay. Requires BLKN.|
|WSFQUI||99 - SirFact quiesce wait. Requires BLKN.|
Thus to produce a breakdown of disk I/O waits by evaluating procedure and by individual lines within the procedures, code the following in SIRTUNEI:
REPORT STATE WDISK EVAL CHUNK 4
To get a breakdown of waits for miscellaneous enqueues (including record locks) by evaluating procedure and by individual lines within the procedures, code the following in SIRTUNEI:
REPORT STATE WENQUE EVAL CHUNK 4
In addition to these primary wait types, there are a few composite wait types for which reports can be generated. These composite wait types, their component primary wait types, and a description of what the composite wait types measure are listed here:
|WCFR||This is made up of WCFREX and WCFRSH. It measures all waits on critical file resources whether for exclusive or share control.|
|WLOG||This is made up of WJRNLO, WCHKPO, WWRITE, and WARBMO. It measures all waits on activities associated with logging for Model 204 recovery, that is, all checkpoint and journal I/O related waits.|
To get a breakdown of waits for critical file resources by evaluating procedure and by individual lines with the procedures, code the following in SIRTUNEI:
REPORT STATE WCFR EVAL CHUNK 4
Critical file resource states
Critical file resources are used by Model 204 to provide multi-user concurrency control on a file level. This control mechanism will sometimes exacerbate some other performance bottleneck. A high value for number of users per sample with wait types CFREX and CFRSH in the SUMMARY report suggests that critical file resource enqueuing bears closer examination.
There are four critical file resources:
|DIRECT||Protects table B updates and accesses.|
|INDEX||Protects accesses and updates of table C and the ordered index.|
|EXISTS||Protects accesses and updates of the existence bit map.|
|RECENQ||Protects accesses and updates of the record enqueuing table. This is the only critical file resource that can be eliminated by the use of the Find Without Locks SOUL statement.|
Determining the cause of a wait
A first step to investigating a critical file resource enqueueing problem is to produce reports for the WCFR state. This will help isolate the programs or lines of code that encounter frequent or long critical file resource waits. Probably the most useful report would be produced by this statement:
REPORT STATE WCFR CHUNK 4
This will break down critical file resource waits by individual lines of SOUL code. Unfortunately, the problem with this type of analysis is that it focuses on the "victims" of critical file resource waits rather than the "culprits," the lines of code holding critical file resources causing other users to wait. While in some situations, the lines of code causing the critical file resource waits are the same lines that suffer from the waits, there is no way to be certain from the WCFR state report that this is indeed the case.
To determine the actual cause of critical file resource enqueuing, more data needs to be collected by the SirTune data collector.
To have this additional data collected, simply specify the parameter
CFR on a
COLLECT statement for
SirTune. This parameter can be specified alone or with other
COLLECT parameters as in this statement:
COLLECT BLKN DISKIO CFR
After this additional CFR (Critical File Resource) data is collected, SirTune is able to produce several additional reports to help isolate the cause of critical file resource enqueuing. The first report that might be useful is the CFRROOT report. This report indicates the base wait types that are behind critical file resource waits. The CFRROOT report does not provide information about which lines of code cause critical file resource waits, so it is not helpful for application tuning.
The CFRROOT report might indicate that
application tuning (rather than system tuning) might be required to reduce
critical file resource enqueuing.
This would be indicated by a primary root cause
DISK (disk I/O waits) or maybe
JRNLO (journal I/O waits).
Reducing wait times
You can attack a primary root cause for critical file resource waits by trying to reduce overall disk I/O's or journal I/O's (with application tuning), or by specifically targeting those instructions that hold critical file resources.
To facilitate this latter option, several CFR states can be requested on SirTune reports if CFR data had been collected by SirTune. These states are:
|CFRHANY||The state where a user holds any critical file resource.|
|CFRHDIR||The state where a user holds the |
|CFRHIND||The state where a user holds the |
|CFRHEXS||The state where a user holds the |
|CFRHREC||The state where a user holds the |
|CFRBANY||The state where a user holds any critical file resource and is preventing (blocking) another user from obtaining a critical file resource.|
|CFRBDIR||The state where a user holds the |
|CFRBIND||The state where a user holds the |
|CFRBEXS||The state where a user holds the |
|CFRBREC||The state where a user holds the |
It should be noted that the CFRBxxx states are weighted based on the number of other users holding the resource and the number of users waiting for the resource. For example, if a user at a line of code holds the DIRECT resource and 3 other users are waiting for the resource, that line of code is considered to have 3 observations in the CFRBDIR state.
On the other hand, if a user at a line of code holds the DIRECT resource (in share mode) along with 4 other users, and a single user is waiting for the DIRECT resource, the line of code is considered to have 1/5th of an observation in the CFRBDIR state.
Generally, the most useful reports for reducing critical file resource waits are the CFRB reports. This statement breaks down the state where a user is blocking another user from any critical file resource by lines of SOUL code:
REPORT STATE CFRBANY CHUNK 4
This is probably the most useful of the STATE CFRxxxx reports. Once critical file resource blocking is isolated to specific SOUL instructions, critical file resource enqueuing can be reduced by:
- Reducing the number of times the offending instructions are executed.
- Reducing the amount of disk I/O performed by the offending instructions.
- Reducing the amount of CPU used by the offending instructions.
It might be tempting to use the Find Without Locks SOUL statement to reduce the critical file resource enqueuing associated with a statement. This will only work if the resource causing conflicts is the RECENQ resource. All other critical file resources are processed exactly the same way, whether or not a locked record set is being used.
However, if the resource causing the conflict is indeed the RECENQ resource, it is still not recommended that the solution be Find Without Locks. A high conflict rate on the RECENQ resource indicates that the environment has a high update activity level, which means that operating on unenqueued found sets is a questionable tactic at best. A high conflict rate on the RECENQ resource might suggest examination of strategies for releasing found sets before any terminal I/O occurs.
The CFRHxxx reports can be useful for tracking potential critical file resource enqueuing problems (perhaps in a test environment) before they actually happen. These states include any user that holds a critical file resource, whether or not it is blocking anyone. These reports are difficult to interpret, however, since they require a fairly good estimate of expected future usage patterns to have any predictive value.
- SirTune introduction
- SirTune data collection under MVS
- SirTune data collection under CMS
- SirTune data collection statements
- SirTune MODIFY and SMSG commands
- SirTune report generation
- SirTune reports
- SirTune user states
- SirTune and Model 204 quad types
- SirTune statement wildcards
- SirTune date processing