Fast/Unload with an external sort package

Instead of having data for a particular output stream go directly to an output data set, it can be passed first to a SORT routine. This has the following advantages over sorting that data in a separate step:

• A total time savings, because sort processing and record extraction processing are overlapped.
• A disk space savings, because the intermediate output data set that would be used to pass data from Fast/Unload to your sort package is eliminated.

Prior to version 4.1, any SORT directives apply to the single output stream, FUNOUT, and there is no TO destination qualifier on such directives.

As of version 4.1 and the advent of multiple output streams in a single FUEL program, a SORT directive may indicate the output stream to which it applies:

SORT TO destination ...

The destination ties the SORT directive to the output stream that has the same destination declared in an OUT TO or UAI TO directive. These directives can occur in any order in your FUEL program.

Specifying the sort

For a UAI stream, you indicate that the output is to be sorted by using the SORT keyword on the UAI statement (see UNLOAD ALL INFORMATION or UAI). The only independent SORT directives you may use are the OPTION statement (at most one per stream) and the PGM statement (at most one per program).

For a non-UAI stream (one declared with an OUT TO destination directive), to pass data to a SORT routine you must code two or more SORT statements before the start of the FOR EACH RECORD loop in your FUEL program. Fast/Unload interprets the information on a SORT directive as a control statement to be passed to your SORT package (not including the TO destination qualifier).

The SORT FIELDS (Using SORT FIELDS) and SORT RECORD (Using SORT RECORD) statements are required for Fast/Unload to pass data to a SORT routine. In addition to these required SORT statements, Fast/Unload also supports the following statements:

• MODS
• DEBUG
• ALTSEQ
• SUM
• INCLUDE
• OMIT
• OPTION
• OUTREC
• INREC

In addition:

• When using a 31-bit extended parameter list to pass data to the sort package, any other sort statement accepted by your sort package can be used. For more information on using 31-bit parameter lists, see the documentation on the SORTP parameter, and also see Fast/Unload customization of defaults.
• You can identify your sort program by using the following form of the SORT statement:

  SORT PGM = pgmname

  Where pgmname is the name of your sort program (the default name is SORT). This form of the SORT statement does not allow the TO destination qualifier, and it may occur at most once in a FUEL program. You can also customize Fast/Unload to change the default SORT program name (see Default SORT program name).

Using SORT FIELDS

Since SORT FIELDS is a sort statement, one would expect to have to code the Fast/Unload version of this as SORT [TO destination] SORT FIELDS, where the first SORT indicates to Fast/Unload that what follows is a sort statement, and the second SORT is part of the actual sort statement. While this is, in fact, permitted, Fast/Unload also lets you specify simply SORT FIELD as a shorthand.

In addition to the standard form of the SORT FIELDS statement (explicitly specifying start position, length, and format), Fast/Unload provides a shorthand for specifying field positions in an output record: If you specify a field name as part of the SORT FIELDS statement, Fast/Unload will replace the name(s) of the field(s) with the starting position, length, and format in the output record.

In addition, you can specify a %variable or the special variables listed below as a shorthand for specifying the position of that %variable or special variable in an output record. The special variables usable in SORT FIELDS are:

• #ERROR
• #RECIN
• #GRPMEM
• #GRPSIZ

Note: Such a shorthand is designed for relatively simple FUEL programs. It is not available if the FUEL program contains more than one OUT TO stream. In that case, you must use the standard form of SORT FIELDS.

For example, if you code

SORT FIELDS=(FIELD1,A)

and later in your program you code

PUT FIELD1 AT 15 AS STRING(10)

Fast/Unload will pass

SORT FIELDS=(15,10,CH,A)

to the sort package if the record format is fixed, or

SORT FIELDS=(19,10,CH,A)

if the format is variable. The 19 position in the statement above also illustrates the fact that Fast/Unload does not consider the RDW (Record Descriptor Word) as part of the output record, while sorts do consider the RDW as part of the input record.

Constant occurrence numbers are valid with the field names in the SORT FIELDS statement, while (loop control variable or %variable) variable occurrences are not.

Using SORT RECORD

Fast/Unload uses the SORT RECORD statement to determine the output record format and length.

For example, Fast/Unload produces fixed-length records with a length of 300 if you code:

SORT RECORD TYPE=F,LENGTH=(300)

The blocksize used is the largest valid blocksize less than or equal to 4096, or it is the record length, if the record length exceeds 4096. For example, if TYPE=F,LENGTH=(300), the blocksize is 3900. If TYPE=V,LENGTH=(300), the blocksize is 4096. If TYPE=V,LENGTH=8000, the blocksize is 8000.

Data passed between Fast/Unload and the sort package is buffered, and the number of buffers used is set by the value of NOBUFF.

Sample code

The following program is an example of the use of an external sort package with Fast/Unload:

OPEN BIGFILE SORT TO SBIGFILE FIELDS=(KEY1,A,KEY3,D),EQUALS SORT TO SBIGFILE RECORD TYPE=F,LENGTH=(100) SORT TO SBIGFILE ALTSEQ - CODE=(F0B0,F1B1,F2B2,F3B3,F4B4,F5B5,F6B6,F7B7,F8B8,F9B9) OUT TO SBIGFILE DEFAULT FOR EACH RECORD PUT KEY1 AS STRING 10 PUT KEY2 AS STRING 10 PUT KEY3 AS FIXED 4 PUT DATA(*) AS STRING(10) OUTPUT END FOR

In this example, the SORT FIELDS statement passed to the sort would be

SORT FIELDS=(1,10,CH,A,21,4,BI,D),EQUALS

See also

• Fast/Unload overview
• Fast/Unload invocation
• Fast/Unload program parameters
• Fast/Unload Extraction Language (FUEL)
• Fast/Unload standard #functions
• Fast/Unload BLOB/CLOB processing considerations
• Fast/Unload datetime processing considerations
• Fast/Unload DATESTAT analysis
• Fast/Unload job statistics
• Fast/Unload SOUL Interface
• Fast/Unload with an external sort package
• Fast/Unload with DBCS data
• Fast/Unload customer-written assembler #function packages
• Fast/Unload user exits or filters
• Fast/Unload with Model 204 file groups
• Fast/Unload with Model 204 fieldgroups
• Fast/Unload with the Sir2000 Field Migration Facility
• Fast/Unload floating point arithmetic and numeric conversion
• Fast/Unload program return codes
• Fast/Unload installation
• Fast/Unload customization of defaults
• Fast/Unload SMF record format
• Fast/Unload release notes
• Fast/Unload messages