Fast/Unload floating point arithmetic and numeric conversion: Difference between revisions

Revision as of 17:36, 23 February 2015

Arithmetic operations in Fast/Unload (that is, arithmetic calculations, as defined in Expressions) are performed using the IBM/360 floating point arithmetic instructions. In addition, many operations in Fast/Unload involve converting to or from a numeric value, which uses a (power of 2) floating point representation.

Many values that are exactly expressed in base 10 (that is, the usual decimal number system) are not represented exactly as power of 2 floating point values. Also, the floating point arithmetic instructions will, in many cases, produce results that are only approximately equal to the given operation and operands.

To satisfy the requirements of manipulating these numeric values using the "hexadecimal" floating point instructions, certain approaches have been adopted in Model 204 SOUL. In order to produce results in Fast/Unload that are identical to the results obtained in SOUL, these same approaches have been adopted in Fast/Unload. However, this M204wiki page only serves as documentation of float handling in Fast/Unload. The similarity of results with SOUL has been extensively tested, but there may be edge cases in which Fast/Unload and SOUL differ.

Note also that Fast/Unload contains no exact provision for the floating point value handling provided by the Image feature in SOUL. Information in this M204wiki page must not be extrapolated to the operation with Images.

Note: The contents of this M204wiki page are refreshed as of Fast/Unload 4.3. It is highly recommended that you use at least Fast/Unload version 4.3 if your application involves calculations sensitive to the operation of floating point handling.

Overview

The IBM "hexadecimal" floating point instructions use an exponent based on a power of sixteen. If the value being represented contains a fractional part, or if it is beyond the range of integers that can be exactly represented, then the floating point representation of a value may be only an approximation of the value. For example, the value "one-tenth" is represented in base 16 as this infinite hexadecimal series:

.1AAAAAAA...

These approximations can lead to surprising results, especially when dealing with decimal fractions that one commonly considers as having an exact representation, for example, the base 10 number .1 for the value one-tenth.

To address this problem, the approach to floating point manipulation in Fast/Unload is based on the following:

Decimal external inputs and outputs	For external numeric inputs and outputs of `Model 204` applications, the original source (for example, data entry fields) and final destination (for example, printed values) is expressed in decimal (base 10) notation.
Arithmetic results mirror decimal operations	When two numeric values are operated upon, in particular with addition and subtraction, the resulting value, when expressed in decimal, is as close as practical to the value that would occur if the operation were performed in decimal.

The principles described above are accomplished using the following algorithms:

15-digit decimal significance	An 8-byte floating point value, in the IBM 360 architecture, contains 56 bits of significance for the fraction part. 56 bits can represent the numbers from 0 to approximately 7.2E16 (7.2 times 10 to the 16th power). Therefore, the maximum significance, in decimal, of an 8-byte `FLOAT` is 16 (decimal) digits. SOUL uses a more conservative limit to significance, and uses 15 decimal digit significance with numeric operations.
Float value preserved if target has same length	A float value is exactly preserved if copied to a float target that has the same length. Length-converting PUT statements specifies the rules for copying a float value to a float target that has a different length.
Float length 4 conversions differ from length 8 or 16	When a float value of length 4 is used in any context other than copying, it is converted as described in Using a float value, with decimal digit precision. When a float value of length 8 or 16 is used in any context other than copying or arithmetic, it is converted as described in Using a float value, with decimal digit precision.
FLoat length 8 or 16 values in arithmetic expressions	Float values of length 8 or 16 use the high order 8 bytes, without any modification, when they are used as entities in an arithmetic expression.

The purpose of the above rules is to achieve (approximately) the same results for float numeric operations as would be given by operations with decimal numbers.

Primitive operations

As a brief background, note the following:

Floating point values use the IBM hexadecimal floating point representation, which is a one-bit sign, a 7-bit base 16 exponent, and a binary fraction whose length is either 3 bytes (FLOAT LEN 4), 7 bytes (FLOAT LEN 8), or 14 bytes (FLOAT LEN 16).
In a normalized floating point number, the high-order nibble (the first four bits) of the fraction has a non-zero value.
The normalized 8-byte add (AD/R) and subtract (SD/R) instructions are used for addition and subtraction in arithmetic expressions. The 8-byte multiply (MD/R) and divide (MD/R) instructions are used for multiplication and division. These instructions produce normalized results (except at the limits of normalized values) and do not round.
See also Arithmetic expressions, which explains that after every float addition or subtraction, there is a decimal rounding step to preserve the proper number of significant digits.

Using a float value, with decimal digit precision

Except for the following cases:

When an 8- or 16-byte floating point value is the input to an arithmetic expression (described in Arithmetic expressions)
When a floating point value is the input to a PUT statement or is the right side of an assignment statement (described in Assignments and length-preserved PUT statements and Length-converting PUT statements)
When the argument of the #FLOAT8 function is a 4-byte floating point value (described in #FLOAT8: Get 8-byte float, padding 4-byte input with 0)

Whenever FUEL requires the value of a floating point value, the value obtained is approximately the closest 8-byte floating point representation of a value, which depends on the length of the "input" floating point value:

LEN 4

The result is the floating point value approximately closest to the decimal number with 6 significant digits closest to the LEN 4 float input. For example, if a FLOAT LEN 4 field contains the following value, shown in hexadecimal:

41100004

In decimal it is:

1.000003814697265625

Then the nearest 6-digit decimal value is:

1.00000

So that value is used, represented exactly by the 8-byte float:

4110000000000000

LEN 8 or 16

The result is the floating point value approximately closest to the decimal number with 15 significant digits closest to the first 8 bytes of the float input. For example, if a FLOAT LEN 8 field contains the following value, shown in hexadecimal:

4110000400000000

In decimal it is:

1.000003814697265625

Then the nearest 15-digit decimal value is:

1.00000381469727

So that value is used, represented by the 8-byte float that is approximately the nearest:

4110000400000013

Note: The low-order 8 bytes of a 16-byte float are ignored.

Obtaining numeric values from non-floats

When a numeric value is required in FUEL from a string or constant that is a decimal number, the value obtained is approximately the closest 8-byte floating point representation of the decimal value to 15 significant digits. For example, after this FUEL fragment:

%T = '1.000003814697265625' /* Note: string value %T = '1.000003814697265625' /* Note: string value CHANGE MYFIELD = %T /* So field set to string value %Z = MYFIELD * 1 /* %Z has float value %Y = %T * 1 /* %Y has (same) float value PUT MYFIELD /* Line 1: From string, 19 digits OUTPUT PUT %Y /* Line 2: From float, 15 digits OUTPUT IF %T EQ '1.000003814697265625' THEN /* No conversion here PUT 'String comparison 1 EQ, of course' OUTPUT END IF IF %T NE %X THEN /* Here converting %X->string: 15 digits PUT 'String comparison 2 should be NE' OUTPUT END IF IF %T EQ +%X THEN /* Here converting %T->float PUT 'Float comparison should be EQ' OUTPUT END IF

The output file will contain:

1.000003814697265625 1.00000381469727 String comparison 1 EQ, of course String comparison 2 should be NE Float comparison should be EQ

And %X, %Y, and %Z will each contain the 8-byte floating point number X'4100000400000013'.

Assignments and length-preserved PUT statements

Assignments between fields and %variables Whenever a FLOAT field occurrence is assigned to a %variable, the entire 4, 8, or 16 bytes are copied exactly to the %variable. Whenever a %variable that contains a floating point value is assigned (with the CHANGE or ADD[C] statement) as the value of a field occurrence, the %variable's entire 4, 8, or 16 bytes are copied exactly to the field occurrence.

Length-preserved PUT AS FLOAT

Whenever a field occurrence or %variable that contains a floating point value is used in a PUT AS FLOAT statement, and the FLOAT format specifies a length that is the same as that of the occurrence or %variable, the entire 4, 8, or 16 bytes are copied exactly to the ouput file.

For example, if field FLT4 contains a 4-byte float value that in hexadecimal is X'41000004', then the following FUEL fragment places two lines to the output file, each containing the 4 bytes that are hexadecimal X'41000004':

PUT FLT4 AS FLOAT(4) OUTPUT %X = FLT4 PUT %X AS FLOAT(4) OUTPUT

Length-converting PUT statements

Other than obtaining the value of a float (for example, as part of an IF statement comparison or as an argument to a #function), which is explained in Using a float value, with decimal digit precision, the only context in FUEL in which a float value is transformed to a float value with a different length is in the PUT statement with a FLOAT format whose format length differs from the length of the float "input." The cases are shown below:

AS FLOAT(4)	The first (and only, if the input is length 8) 8 bytes of the input are copied to a 4-byte float using the `LEDR` instruction, which produces the first 4 bytes of the input 8 bytes, or those 4 bytes plus 1 (times the sign of the value) if the high order bit of the second 4 bytes is 1. Note that there is no normalization of the input value prior to rounding. Thus the low-order 31 fraction bits of an 8-byte float are ignored, and the low-order `31 + 56` fraction bits of a 16-byte float are ignored.
LEN 16 -> 8	The `AS FLOAT(8)` clause for a 16-byte float input is obtained by taking the first 8 bytes of the 16-byte float value. Note that there is no rounding, as there is when converting from 8-byte to 4-byte floats, and there is no normalization. Thus the low order 56 fraction bits of the 16-byte float are ignored.
LEN 4 -> 8/16	When a 4-byte float value is the input to `AS FLOAT(8)` or `AS FLOAT(16)`, the 4-byte input float value is converted to an 8-byte value that is approximately nearest the input value expressed as the nearest 6-significant-digit decimal number, as described in the `LEN 4` case in Using a float value, with decimal digit precision. This is the result when the `PUT` format length is 8. An additional 8 bytes of zeroes are added when it is 16.
LEN 8 -> 16	The `AS FLOAT(16)` clause for a 8-byte float input is obtained by appending 8 additional bytes of zeroes to the unchanged 8-byte value.

Arithmetic expressions

The result of an arithmetic expression is always an 8-byte float; these are produced as described in Primitive operations. Also, after every addition or subtraction in the expression, a step is performed to ensure that the correct significance is retained as the result of that operation. This significance is based on the magnitude of the inputs and the result of the addition or subtraction.

For example, after performing the SDR to operate on the following two values:

%X = 123456789.1234 - 123456789 /* = X'4775BCD151F97247' /* - X'4775BCD150000000' /* = X'401F972470000000' /* = .123399998992682 (rounded to 15 digits)

Fast/Unload examines the magnitude of the absolute values of the result and addends, and it determines that 4 significant digits should be retained, so it rounds the result to 4 significant digits:

/* .1234 (result rounded to 4 digits) /* = X'401F972474538EF3' (float nearest to .1234)

Example

Following is one example of the behavior of Fast/Unload. Versions prior to 4.0 obtain the different result shown below.

%T2 = 1 FOR J FROM 1 TO 7 /* Get 1E-7 %T2 = %T2 / 10 END FOR PUT %T2 OUTPUT FOR J FROM 1 TO 5 /* Get .01 %T2 = %T2 * 10 END FOR PUT %T2 OUTPUT %T = 0 FOR J FROM 1 TO 10 /* Get .1 %T = %T + %T2 END FOR PUT %T OUTPUT

Results (starting with version 4.0):

0.0000001 0.00999999999999999 0.1

Results (prior to version 4.0):

0.0000001 0.00999999999999999 0.0999999999999999