Fast/Unload floating point arithmetic and numeric conversion: Difference between revisions

From m204wiki
Jump to navigation Jump to search
(Automatically generated page update)
 
No edit summary
 
(5 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<!-- Page name: Fast/Unload floating point arithmetic and numeric conversion-->
<!-- Page name: Fast/Unload floating point arithmetic and numeric conversion-->
<p></p>
Arithmetic operations in <var class="product">Fast/Unload</var> (that is, arithmetic
Arithmetic operations in <var class="product">Fast/Unload</var> (that is, arithmetic
calculations, as defined in [[#expr|Expressions]]) are performed using the IBM/360
calculations, as defined in [[Fast/Unload Extraction Language (FUEL)#expr|Expressions]]) are performed using the IBM/360
floating point arithmetic instructions.
floating point arithmetic instructions.
In addition,
In addition, many operations in <var class="product">Fast/Unload</var> involve converting to or from a numeric value, which uses a (power of 2) floating point representation.
many operations in <var class="product">Fast/Unload</var> involve converting to or from a numeric value, which
 
uses a (power of 2) floating point representation.
<p></p>
Many values that are exactly expressed in base 10 (that is, the usual decimal
Many values that are exactly expressed in base 10 (that is, the usual decimal
number system) are not represented exactly as power of 2 floating point values.
number system) are not represented exactly as power of 2 floating point values.
Also, the floating point arithmetic instructions will, in many cases, produce results
Also, the floating point arithmetic instructions will, in many cases, produce results
that are only approximately equal to the given operation and operands.
that are only approximately equal to the given operation and operands.
<p></p>
 
To satisfy
To satisfy the requirements of manipulating these numeric values using the
the requirements of manipulating these numeric values using the
"hexadecimal" floating
"hexadecimal" floating
point instructions, certain approaches have been adopted in <var class="product">Model 204</var> User Language.
point instructions, certain approaches have been adopted in <var class="product">Model 204</var> [[SOUL]].
In order to produce results in <var class="product">Fast/Unload</var> that are identical to the results obtained in
In order to produce results in <var class="product">Fast/Unload</var> that are identical to the results obtained in SOUL, these same approaches have been adopted in <var class="product">Fast/Unload</var>.
User Language, these same approaches have been adopted in <var class="product">Fast/Unload</var>.
However, this M204wiki page only serves as documentation of float
However, this wiki page only serves as documentation of float
handling in <var class="product">Fast/Unload</var>.
handling in <var class="product">Fast/Unload</var>.
For detailed information about SOUL vs. FUEL numeric differences, see:
<ul>
<li>[[Fast/Unload Extraction Language (FUEL)#soulVsFuelDataComparisons|Contrasting SOUL and FUEL comparisons]]
<li>[[Floating_point_conversion,_rounding,_and_precision_rules|SOUL  float conversion/rounding/precision rules]]
</ul>
The similarity of results with <var class="product">SOUL</var> has been extensively
The similarity of results with <var class="product">SOUL</var> has been extensively
tested, but there may be edge cases in which <var class="product">Fast/Unload</var> and
tested, but there may be edge cases in which <var class="product">Fast/Unload</var> and
<var class="product">SOUL</var> differ.
<var class="product">SOUL</var> differ.
<p></p>
 
Note also that <var class="product">Fast/Unload</var> contains no exact provision for the floating point
Note also that <var class="product">Fast/Unload</var> contains no exact provision for the floating point value handling provided by the <code>Image</code> feature in
value handling provided by the <code>Image</code> feature in
<var class="product">SOUL</var>.
<var class="product">SOUL</var>.
Information in this wiki page must
Information in this M204wiki page must
not be extrapolated to the operation with Images.
not be extrapolated to the operation with Images.
<blockquote class="note"><b>Note:</b> The contents of this wiki page are refreshed as of <var class="product">Fast/Unload</var> 4.3.
<p class="note"><b>Note:</b> The contents of this M204wiki page are refreshed as of <var class="product">Fast/Unload</var> 4.3.
It is highly recommended that you use at least <var class="product">Fast/Unload</var> version 4.3 if
It is highly recommended that you use at least <var class="product">Fast/Unload</var> version 4.3 if
your application involves calculations sensitive to the operation of
your application involves calculations sensitive to the operation of
floating point handling.
floating point handling.
</blockquote>
</p>
 
==Overview==
==Overview==
<p></p>
The IBM "hexadecimal" floating point instructions use an exponent based
The IBM "hexadecimal" floating point instructions use an exponent based
on a power of sixteen.
on a power of sixteen.
Line 42: Line 43:
the range of integers that can be exactly represented, then the floating point
the range of integers that can be exactly represented, then the floating point
representation of a value may be only an approximation of the value.
representation of a value may be only an approximation of the value.
For example, the value "one-tenth" is represented in base 16 as the
For example, the value "one-tenth" is represented in base 16 as this
infinite hexadecimal series .1AAAAAAA...
infinite hexadecimal series:
<p class="code">.1AAAAAAA...</p>
These approximations can lead to surprising results, especially when dealing
These approximations can lead to surprising results, especially when dealing
with <b>decimal</b> fractions that one commonly considers as having an
with <i>decimal</i> fractions that one commonly considers as having an
exact representation, for example, the base 10 number .1 for the value one-tenth.
exact representation, for example, the base 10 number <code>.1</code> for the value one-tenth.
<p></p>
 
To address this problem, the approach to floating point manipulation
To address this problem, the approach to floating point manipulation
in <var class="product">Fast/Unload</var> is based on the following:
in <var class="product">Fast/Unload</var> is based on the following:
<table>
<table class="thJustBold">
<tr><th>Decimal external inputs and outputs</th><td>For external numeric inputs and outputs of <var class="product">Model 204</var> applications, the original source (for example, data entry fields) and final destination (for example, printed values) is expressed in decimal (base 10) notation.</td></tr>
<tr><th>Decimal external inputs and outputs</th>
<tr><th>Arithmetic results mirror decimal operations</th><td>When two numeric values are operated upon, in particular with addition and subtraction, the resulting value, when expressed in decimal, is as close as practical to the value that would occur if the operation were performed in decimal.</td></tr>
<td>For external numeric inputs and outputs of <var class="product">Model 204</var> applications, the original source (for example, data entry fields) and final destination (for example, printed values) is expressed in decimal (base 10) notation.</td></tr>
 
<tr><th nowrap>Arithmetic results mirror decimal operations</th>
<td>When two numeric values are operated upon, in particular with addition and subtraction, the resulting value, when expressed in decimal, is as close as practical to the value that would occur if the operation were performed in decimal.</td></tr>
</table>
</table>
<p></p>
 
The principles described above are accomplished using the following
The principles described above are accomplished using the following algorithms:
algorithms:
<table class="thJustBold">
<table>
<tr><th nowrap>15-digit decimal significance</th>
<tr><th>15-digit decimal significance</th><td>An 8-byte floating point value, in the IBM 360 architecture, contains 56 bits of significance for the fraction part. 56 bits can represent the numbers from 0 to approximately 7.2E16 (7.2 times 10 to the 16th power). Therefore, the maximum significance, in decimal, of an 8-byte FLOAT is 16 (decimal) digits. User Language uses a more conservative limit to significance, and uses 15 decimal digit significance with numeric operations.</td></tr>
<td>An 8-byte floating point value, in the IBM 360 architecture, contains 56 bits of significance for the fraction part. 56 bits can represent the numbers from 0 to approximately 7.2E16 (7.2 times 10 to the 16th power). Therefore, the maximum significance, in decimal, of an 8-byte <var>FLOAT</var> is 16 (decimal) digits. SOUL uses a more conservative limit to significance, and uses 15 decimal digit significance with numeric operations.</td></tr>
<tr><th>Float value preserved if target has same length</th><td>A float value is exactly preserved if copied to a float target that has the same length. [[#flench|Length-converting PUT statements]] specifies the rules for copying a float value to a float target that has a different length.</td></tr>
 
<tr><th>Float length 4 conversions differ from length 8 or 16</th><td> When a float value of length 4 is used in any context other than copying, it is converted as described in [[#fval|Using a float value, with decimal digit precision]]. When a float value of length 8 or 16 is used in any context other than copying or arithmetic, it is converted as described in [[#fval|Using a float value, with decimal digit precision]].</td></tr>
<tr><th>Float value preserved if target has same length</th>
<tr><th>FLoat length 8 or 16 values in arithmetic expressions</th><td>Float values of length 8 or 16 use the high order 8 bytes, without any modification, when they are used as entities in an arithmetic expression.</td></tr>
<td>A float value is exactly preserved if copied to a float target that has the same length. [[#flench|Length-converting PUT statements]] specifies the rules for copying a float value to a float target that has a different length.</td></tr>
 
<tr><th>Float length 4 conversions differ from length 8 or 16</th>
<td> When a float value of length 4 is used in any context other than copying, it is converted as described in [[#fval|Using a float value, with decimal digit precision]]. When a float value of length 8 or 16 is used in any context other than copying or arithmetic, it is converted as described in [[#fval|Using a float value, with decimal digit precision]].</td></tr>
 
<tr><th>FLoat length 8 or 16 values in arithmetic expressions</th>
<td>Float values of length 8 or 16 use the high order 8 bytes, without any modification, when they are used as entities in an arithmetic expression.</td></tr>
</table>
</table>
<p></p>
 
The purpose of the above rules is to achieve (approximately) the same
The purpose of the above rules is to achieve (approximately) the same
results for float numeric operations as would be given by
results for float numeric operations as would be given by
Line 72: Line 83:
<!--Caution: <div> above-->
<!--Caution: <div> above-->
   
   
<p></p>
As a brief background, note the following:
As a brief background, note the following:
<ul>
<ul>
<li>
<li>Floating point values use the IBM
Floating point values use the IBM
hexadecimal floating point representation, which is a one-bit
hexadecimal floating point representation, which is a one-bit
sign, a 7-bit base 16 exponent, and a binary fraction whose
sign, a 7-bit base 16 exponent, and a binary fraction whose
length is either 3 bytes (FLOAT LEN 4), 7 bytes (FLOAT LEN 8),
length is either 3 bytes (<code>FLOAT LEN 4</code>), 7 bytes (<code>FLOAT LEN 8</code>),
or 14 bytes (FLOAT LEN 16).
or 14 bytes (<code>FLOAT LEN 16</code>).</li>
<li>
 
In a <b>normalized</b> floating point number, the high-order
<li>In a <b>normalized</b> floating point number, the high-order
nibble (the first four bits) of the fraction has a non-zero value.
nibble (the first four bits) of the fraction has a non-zero value.</li>
<li>
 
The normalized 8-byte add (AD/R) and subtract (SD/R) instructions are
<li>The normalized 8-byte add (AD/R) and subtract (SD/R) instructions are
used for addition and subtraction in arithmetic expressions;
used for addition and subtraction in arithmetic expressions.
the 8-byte multiply (MD/R) and divide (MD/R) instructions are used
The 8-byte multiply (MD/R) and divide (MD/R) instructions are used
for multiplication and division.
for multiplication and division.
These instructions produce normalized results (except at the limits of
These instructions produce normalized results (except at the limits of
normalized values) and do not round.
normalized values) and do not round.
<p></p>
<p>
See also [[#farith|Arithmetic expressions]], which explains that after every
See also [[#farith|Arithmetic expressions]], which explains that after every
float addition or subtraction, there is a decimal rounding step to
float addition or subtraction, there is a decimal rounding step to
preserve the proper number of significant digits.
preserve the proper number of significant digits.</p></li>
</ul>
</ul>
   
   
Line 101: Line 110:
<!--Caution: <div> above-->
<!--Caution: <div> above-->
   
   
<p></p>
<b>Except</b> for the following cases:
<b>Except</b> for the following cases:
<ul>
<ul>
<li>when an 8- or 16-byte floating point value is the input to an
<li>When an 8- or 16-byte floating point value is the input to an
arithmetic expression (described in [[#farith|Arithmetic expressions]]);
arithmetic expression (described in [[#farith|Arithmetic expressions]])</li>
<li>when a floating point value is the input to a PUT statement or is the
 
<li>When a floating point value is the input to a <var>PUT</var> statement or is the
right side of an assignment statement (described in [[#fltasg|Assignments and length-preserved PUT statements]]
right side of an assignment statement (described in [[#fltasg|Assignments and length-preserved PUT statements]]
and [[#flench|Length-converting PUT statements]]);
and [[#flench|Length-converting PUT statements]])</li>
<li>when the argument of the #FLOAT8 function is a 4-byte floating point value
 
(described in [[#flt8|#FLOAT8: Get 8-byte float, padding 4-byte input with 0]]);
<li>When the argument of the <var>#FLOAT8</var> function is a 4-byte floating point value
(described in [[Fast/Unload standard functions#flt8|#FLOAT8: Get 8-byte float, padding 4-byte input with 0]])</li>
</ul>
</ul>
whenever FUEL requires the <b><i>value</i></b> of a <b>floating point</b> value,
Whenever FUEL requires the <b><i>value</i></b> of a <b>floating point</b> value,
the value obtained is approximately
the value obtained is approximately
the closest 8-byte floating point representation of a value, which depends
the closest 8-byte floating point representation of a value, which depends
on the length of the "input" floating point value:
on the length of the "input" floating point value:
<table>
<table class="thJustBold">
<tr><th>LEN 4</th><td>The result is the floating point value approximately closest to the <b>decimal</b> number with <b>6</b> significant digits closest to the LEN 4 float input. For example, if a FLOAT LEN 4 field contains the following value, shown in hexadecimal:
<tr><th>LEN 4</th>
<td>The result is the floating point value approximately closest to the <b>decimal</b> number with <b>6</b> significant digits closest to the <code>LEN 4</code> float input. For example, if a <code>FLOAT LEN 4</code> field contains the following value, shown in hexadecimal:
<p class="code"><nowiki>41100004
<p class="code"><nowiki>41100004
</nowiki></p> which in decimal is:
</nowiki></p>  
<p class="code"><nowiki>1.000003814697265625
In decimal it is:
</nowiki></p> then the nearest <b>6</b>-digit decimal value is:
<p class="code">1.000003814697265625
<p class="code"><nowiki>1.00000
</p>  
</nowiki></p> so that value is used, represented exactly by the 8-byte float:
Then the nearest <b>6</b>-digit decimal value is:
<p class="code"><nowiki>4110000000000000
<p class="code">1.00000
</nowiki></p></td></tr>
</p>  
<tr><th>LEN 8 or 16</th><td>The result is the floating point value approximately closest to the <b>decimal</b> number with <b>15</b> significant digits closest to the first 8 bytes of the float input. For example, if a FLOAT LEN 8 field contains the following value, shown in hexadecimal:
So that value is used, represented exactly by the 8-byte float:
<p class="code"><nowiki>4110000400000000
<p class="code">4110000000000000
</nowiki></p> which in decimal is:
</p></td></tr>
<p class="code"><nowiki>1.000003814697265625
 
</nowiki></p> then the nearest <b>15</b>-digit decimal value is:
<tr><th nowrap>LEN 8 or 16</th>
<td>The result is the floating point value approximately closest to the <b>decimal</b> number with <b>15</b> significant digits closest to the first 8 bytes of the float input. For example, if a <code>FLOAT LEN 8</code> field contains the following value, shown in hexadecimal:
<p class="code">4110000400000000
</p>  
In decimal it is:
<p class="code">1.000003814697265625
</p>  
Then the nearest <b>15</b>-digit decimal value is:
<p class="code"><nowiki>1.00000381469727
<p class="code"><nowiki>1.00000381469727
</nowiki></p> so that value is used, represented by the 8-byte float that is approximately the nearest:
</nowiki></p>  
<p class="code"><nowiki>4110000400000013
So that value is used, represented by the 8-byte float that is approximately the nearest:
</nowiki></p>
<p class="code">4110000400000013
<blockquote class="note"><b>Note:</b> The low order 8 bytes of a 16-byte float are ignored. </blockquote></td></tr>
</p>
<p class="note"><b>Note:</b> The low-order 8 bytes of a 16-byte float are ignored. </p></td></tr>
</table>
</table>
===Obtaining numeric values from non-floats===
===Obtaining numeric values from non-floats===
<p></p>
<p></p>
When a numeric value is required in FUEL from a <b>string or constant
When a numeric value is required in FUEL from a <b>string or constant
that is a decimal number</b>, the value obtained is approximately the
that is a decimal number</b>, the value obtained is approximately the
Line 167: Line 185:
END IF
END IF
</nowiki></p>
</nowiki></p>
<p></p>
 
<p></p>
The output file will contain:
The output file will contain:
<p class="code"><nowiki>1.000003814697265625
<p class="output"><nowiki>1.000003814697265625
1.00000381469727
1.00000381469727
String comparison 1 EQ, of course
String comparison 1 EQ, of course
Line 176: Line 193:
Float comparison should be EQ
Float comparison should be EQ
</nowiki></p>
</nowiki></p>
And %X, %Y, and %Z will each contain the 8-byte floating
And <code>%X</code>, <code>%Y</code>, and <code>%Z</code> will each contain the 8-byte floating
point number <code>X'4100000400000013'</code>.
point number <code>X'4100000400000013'</code>.
   
   
Line 183: Line 200:
<!--Caution: <div> above-->
<!--Caution: <div> above-->
   
   
<table>
<table class="thJustBold">
<tr><th>Assignments between fields and %variables</th><td> Whenever a FLOAT field occurrence is assigned to a %variable, the entire 4, 8, or 16 bytes are copied exactly to the %variable. Whenever a %variable that contains a floating point value is assigned (via the CHANGE or ADD[C] statement) as the value of a field occurrence, the %variable's entire 4, 8, or 16 bytes are copied exactly to the field occurrence.</td></tr>
<tr><th>Assignments between fields and %variables</th>
<tr><th>Length-preserved PUT AS FLOAT</th><td> Whenever a field occurrence or %variable that contains a floating point value is used in a PUT AS FLOAT statement, and the FLOAT format specifies a length that is the same as that of the occurrence or %variable, the entire 4, 8, or 16 bytes are copied exactly to the ouput file. <p></p> For example, if field FLT4 contains a 4-byte float value that in hexadecimal is X'41000004', then the following FUEL fragment:
<td>Whenever a <var>FLOAT</var> field occurrence is assigned to a %variable, the entire 4, 8, or 16 bytes are copied exactly to the %variable. Whenever a %variable that contains a floating point value is assigned (with the <var>CHANGE</var> or <var>ADD[C]</var> statement) as the value of a field occurrence, the %variable's entire 4, 8, or 16 bytes are copied exactly to the field occurrence.</td></tr>
<p class="code"><nowiki>PUT FLT4 AS FLOAT(4)
 
<tr><th nowrap>Length-preserved PUT AS FLOAT</th>
<td> Whenever a field occurrence or %variable that contains a floating point value is used in a <var>PUT AS FLOAT</var> statement, and the <var>FLOAT</var> format specifies a length that is the same as that of the occurrence or %variable, the entire 4, 8, or 16 bytes are copied exactly to the ouput file.  
<p>
For example, if field <code>FLT4</code> contains a 4-byte float value that in hexadecimal is <code>X'41000004'</code>, then the following FUEL fragment places two lines to the output file, each containing the 4 bytes that are hexadecimal <code>X'41000004'</code>:</p>
<p class="code">PUT FLT4 AS FLOAT(4)
OUTPUT
OUTPUT
%X = FLT4
%X = FLT4
PUT %X AS FLOAT(4)
PUT %X AS FLOAT(4)
OUTPUT
OUTPUT
</nowiki></p> places two lines to the output file, each containing the 4 bytes that are hexadecimal <code>X'41000004'</code>.</td></tr>
</p> </td></tr>
</table>
</table>
   
   
Line 198: Line 220:
<!--Caution: <div> above-->
<!--Caution: <div> above-->
   
   
<p></p>
Other than obtaining the value of a float (for example, as part of an
Other than obtaining the value of a float (for example, as part of an
IF statement comparison or as an argument to a #function),
<var>IF</var> statement comparison or as an argument to a #function),
which is explained in [[#fval|Using a float value, with decimal digit precision]], the
which is explained in [[#fval|Using a float value, with decimal digit precision]], the
only context in FUEL in which a float value is transformed to a float
only context in FUEL in which a float value is transformed to a float
value with a different length is in the PUT statement with a FLOAT
value with a different length is in the <var>PUT</var> statement with a <var>FLOAT</var>
format whose format length differs from the length of the float "input."
format whose format length differs from the length of the float "input."
The cases are shown below:
The cases are shown below:
<table>
<table class="thJustBold">
<tr><th>AS FLOAT(4)</th><td>The first (and only, if the input is length 8) 8 bytes of the input are copied to a 4-byte float using the LEDR instruction, which produces the first 4 bytes of the input 8 bytes, or those 4 bytes plus 1 (times the sign of the value) if the high order bit of the second 4 bytes is 1. <p></p> Note that there is no normalization of the input value prior to rounding; thus the low-order 31 fraction bits of an 8-byte float are ignored, and the low-order 31 + 56 fraction bits of a 16-byte float are ignored.</td></tr>
<tr><th>AS FLOAT(4)</th>
<tr><th>LEN 16 -> 8</th><td> The AS FLOAT(8) clause for a 16-byte float input is obtained by taking the first 8 bytes of the 16-byte float value. <p></p> Note that there is no rounding, as there is when converting from 8-byte to 4-byte floats, and there is no normalization; thus the low order 56 fraction bits of the 16-byte float are ignored.</td></tr>
<td>The first (and only, if the input is length 8) 8 bytes of the input are copied to a 4-byte float using the <code>LEDR</code> instruction, which produces the first 4 bytes of the input 8 bytes, or those 4 bytes plus 1 (times the sign of the value) if the high order bit of the second 4 bytes is 1. <p>
<tr><th>LEN 4 -> 8/16</th><td> When a 4-byte float value is the input to AS FLOAT(8) or AS FLOAT(16), the 4-byte input float value is converted to an 8-byte value that is approximately nearest the input value expressed as the nearest 6-significant-digit decimal number, as described in the <b>LEN 4</b> case in [[#fval|Using a float value, with decimal digit precision]]. This is the result when the PUT format length is 8; an additional 8 bytes of zeroes are added when it is 16.</td></tr>
Note that there is no normalization of the input value prior to rounding. Thus the low-order 31 fraction bits of an 8-byte float are ignored, and the low-order <code>31 + 56</code> fraction bits of a 16-byte float are ignored.</p></td></tr>
<tr><th>LEN 8 -> 16</th><td> The AS FLOAT(16) clause for a 8-byte float input is obtained by appending 8 additional bytes of zeroes to the unchanged 8-byte value.</td></tr>
 
<tr><th>LEN 16 -> 8</th>
<td>The <code>AS FLOAT(8)</code> clause for a 16-byte float input is obtained by taking the first 8 bytes of the 16-byte float value.  
<p>
Note that there is no rounding, as there is when converting from 8-byte to 4-byte floats, and there is no normalization. Thus the low order 56 fraction bits of the 16-byte float are ignored.</p></td></tr>
 
<tr><th nowrap>LEN 4 -> 8/16</th>
<td>When a 4-byte float value is the input to <code>AS FLOAT(8)</code> or <code>AS FLOAT(16)</code>, the 4-byte input float value is converted to an 8-byte value that is approximately nearest the input value expressed as the nearest 6-significant-digit decimal number, as described in the <code>LEN 4</code> case in [[#fval|Using a float value, with decimal digit precision]]. This is the result when the <var>PUT</var> format length is 8. An additional 8 bytes of zeroes are added when it is 16.</td></tr>
 
<tr><th>LEN 8 -> 16</th>
<td>The <code>AS FLOAT(16)</code> clause for a 8-byte float input is obtained by appending 8 additional bytes of zeroes to the unchanged 8-byte value.</td></tr>
</table>
</table>
   
   
<div id="farith"></div>
==<b id="farith"></b> Arithmetic expressions==
==Arithmetic expressions==
<!--Caution: <div> above-->
   
   
<p></p>
The result of an arithmetic expression is always an 8-byte
The result of an arithmetic expression is always an 8-byte
float; these are produced as described in [[#fbg|Primitive operations]].
float; these are produced as described in [[#fbg|Primitive operations]].
Line 225: Line 253:
This significance is based on the magnitude of the inputs and
This significance is based on the magnitude of the inputs and
the result of the addition or subtraction.
the result of the addition or subtraction.
<p></p>
 
<p></p>
For example, after performing the <code>SDR</code> to operate on the following two values:
For example, after performing the SDR to operate on the following two values:
<p class="code"><nowiki>%X =  123456789.1234    -  123456789
<p class="code"><nowiki>%X =  123456789.1234    -  123456789
  /*  =  X'4775BCD151F97247'
  /*  =  X'4775BCD151F97247'
Line 234: Line 261:
  /*  =        .123399998992682  (rounded to 15 digits)
  /*  =        .123399998992682  (rounded to 15 digits)
</nowiki></p>
</nowiki></p>
<p></p>
 
<p></p>
<var class="product">Fast/Unload</var> examines the magnitude of the absolute values of the result and addends, and it determines that 4 significant digits should be retained, so
<var class="product">Fast/Unload</var> examines the magnitude of the absolute values of the result and
addends, and it determines that 4 significant digits should be retained, so
it rounds the result to 4 significant digits:
it rounds the result to 4 significant digits:
<p class="code"><nowiki> /*            .1234  (result rounded to 4 digits)
<p class="code"><nowiki> /*            .1234  (result rounded to 4 digits)
  /*  =  X'401F972474538EF3' (float nearest to .1234)
  /*  =  X'401F972474538EF3' (float nearest to .1234)
</nowiki></p>
</nowiki></p>
==Example==
==Example==
<p></p>
Following is one example of the behavior of <var class="product">Fast/Unload</var>.
Following is one example of the behavior of <var class="product">Fast/Unload</var>.
Versions prior to 4.0 obtain the different result shown below.
Versions prior to 4.0 obtain the different result shown below.
Line 265: Line 290:
OUTPUT
OUTPUT
</nowiki></p>
</nowiki></p>
<p></p>
 
Results (starting with version 4.0):
Results (starting with version 4.0):
<p class="code"><nowiki>0.0000001
<p class="output">0.0000001
0.00999999999999999
0.00999999999999999
0.1
0.1
</nowiki></p>
</p>
Results (prior to version 4.0):
Results (prior to version 4.0):
<p class="code"><nowiki>0.0000001
<p class="output">0.0000001
0.00999999999999999
0.00999999999999999
0.0999999999999999
0.0999999999999999
</nowiki></p>
</p>
 
==See also==
==See also==
[[Fast/Unload overview#WIKFUN$$topics|Fast/Unload topics]]
{{Template:Fast/Unload topic list}}

Latest revision as of 19:04, 3 March 2019

Arithmetic operations in Fast/Unload (that is, arithmetic calculations, as defined in Expressions) are performed using the IBM/360 floating point arithmetic instructions. In addition, many operations in Fast/Unload involve converting to or from a numeric value, which uses a (power of 2) floating point representation.

Many values that are exactly expressed in base 10 (that is, the usual decimal number system) are not represented exactly as power of 2 floating point values. Also, the floating point arithmetic instructions will, in many cases, produce results that are only approximately equal to the given operation and operands.

To satisfy the requirements of manipulating these numeric values using the "hexadecimal" floating point instructions, certain approaches have been adopted in Model 204 SOUL. In order to produce results in Fast/Unload that are identical to the results obtained in SOUL, these same approaches have been adopted in Fast/Unload. However, this M204wiki page only serves as documentation of float handling in Fast/Unload.

For detailed information about SOUL vs. FUEL numeric differences, see:

The similarity of results with SOUL has been extensively tested, but there may be edge cases in which Fast/Unload and SOUL differ.

Note also that Fast/Unload contains no exact provision for the floating point value handling provided by the Image feature in SOUL. Information in this M204wiki page must not be extrapolated to the operation with Images.

Note: The contents of this M204wiki page are refreshed as of Fast/Unload 4.3. It is highly recommended that you use at least Fast/Unload version 4.3 if your application involves calculations sensitive to the operation of floating point handling.

Overview

The IBM "hexadecimal" floating point instructions use an exponent based on a power of sixteen. If the value being represented contains a fractional part, or if it is beyond the range of integers that can be exactly represented, then the floating point representation of a value may be only an approximation of the value. For example, the value "one-tenth" is represented in base 16 as this infinite hexadecimal series:

.1AAAAAAA...

These approximations can lead to surprising results, especially when dealing with decimal fractions that one commonly considers as having an exact representation, for example, the base 10 number .1 for the value one-tenth.

To address this problem, the approach to floating point manipulation in Fast/Unload is based on the following:

Decimal external inputs and outputs For external numeric inputs and outputs of Model 204 applications, the original source (for example, data entry fields) and final destination (for example, printed values) is expressed in decimal (base 10) notation.
Arithmetic results mirror decimal operations When two numeric values are operated upon, in particular with addition and subtraction, the resulting value, when expressed in decimal, is as close as practical to the value that would occur if the operation were performed in decimal.

The principles described above are accomplished using the following algorithms:

15-digit decimal significance An 8-byte floating point value, in the IBM 360 architecture, contains 56 bits of significance for the fraction part. 56 bits can represent the numbers from 0 to approximately 7.2E16 (7.2 times 10 to the 16th power). Therefore, the maximum significance, in decimal, of an 8-byte FLOAT is 16 (decimal) digits. SOUL uses a more conservative limit to significance, and uses 15 decimal digit significance with numeric operations.
Float value preserved if target has same length A float value is exactly preserved if copied to a float target that has the same length. Length-converting PUT statements specifies the rules for copying a float value to a float target that has a different length.
Float length 4 conversions differ from length 8 or 16 When a float value of length 4 is used in any context other than copying, it is converted as described in Using a float value, with decimal digit precision. When a float value of length 8 or 16 is used in any context other than copying or arithmetic, it is converted as described in Using a float value, with decimal digit precision.
FLoat length 8 or 16 values in arithmetic expressions Float values of length 8 or 16 use the high order 8 bytes, without any modification, when they are used as entities in an arithmetic expression.

The purpose of the above rules is to achieve (approximately) the same results for float numeric operations as would be given by operations with decimal numbers.

Primitive operations

As a brief background, note the following:

  • Floating point values use the IBM hexadecimal floating point representation, which is a one-bit sign, a 7-bit base 16 exponent, and a binary fraction whose length is either 3 bytes (FLOAT LEN 4), 7 bytes (FLOAT LEN 8), or 14 bytes (FLOAT LEN 16).
  • In a normalized floating point number, the high-order nibble (the first four bits) of the fraction has a non-zero value.
  • The normalized 8-byte add (AD/R) and subtract (SD/R) instructions are used for addition and subtraction in arithmetic expressions. The 8-byte multiply (MD/R) and divide (MD/R) instructions are used for multiplication and division. These instructions produce normalized results (except at the limits of normalized values) and do not round.

    See also Arithmetic expressions, which explains that after every float addition or subtraction, there is a decimal rounding step to preserve the proper number of significant digits.

Using a float value, with decimal digit precision

Except for the following cases:

Whenever FUEL requires the value of a floating point value, the value obtained is approximately the closest 8-byte floating point representation of a value, which depends on the length of the "input" floating point value:

LEN 4 The result is the floating point value approximately closest to the decimal number with 6 significant digits closest to the LEN 4 float input. For example, if a FLOAT LEN 4 field contains the following value, shown in hexadecimal:

41100004

In decimal it is:

1.000003814697265625

Then the nearest 6-digit decimal value is:

1.00000

So that value is used, represented exactly by the 8-byte float:

4110000000000000

LEN 8 or 16 The result is the floating point value approximately closest to the decimal number with 15 significant digits closest to the first 8 bytes of the float input. For example, if a FLOAT LEN 8 field contains the following value, shown in hexadecimal:

4110000400000000

In decimal it is:

1.000003814697265625

Then the nearest 15-digit decimal value is:

1.00000381469727

So that value is used, represented by the 8-byte float that is approximately the nearest:

4110000400000013

Note: The low-order 8 bytes of a 16-byte float are ignored.

Obtaining numeric values from non-floats

When a numeric value is required in FUEL from a string or constant that is a decimal number, the value obtained is approximately the closest 8-byte floating point representation of the decimal value to 15 significant digits. For example, after this FUEL fragment:

%T = '1.000003814697265625' /* Note: string value %T = '1.000003814697265625' /* Note: string value CHANGE MYFIELD = %T /* So field set to string value %Z = MYFIELD * 1 /* %Z has float value %Y = %T * 1 /* %Y has (same) float value PUT MYFIELD /* Line 1: From string, 19 digits OUTPUT PUT %Y /* Line 2: From float, 15 digits OUTPUT IF %T EQ '1.000003814697265625' THEN /* No conversion here PUT 'String comparison 1 EQ, of course' OUTPUT END IF IF %T NE %X THEN /* Here converting %X->string: 15 digits PUT 'String comparison 2 should be NE' OUTPUT END IF IF %T EQ +%X THEN /* Here converting %T->float PUT 'Float comparison should be EQ' OUTPUT END IF

The output file will contain:

1.000003814697265625 1.00000381469727 String comparison 1 EQ, of course String comparison 2 should be NE Float comparison should be EQ

And %X, %Y, and %Z will each contain the 8-byte floating point number X'4100000400000013'.

Assignments and length-preserved PUT statements

Assignments between fields and %variables Whenever a FLOAT field occurrence is assigned to a %variable, the entire 4, 8, or 16 bytes are copied exactly to the %variable. Whenever a %variable that contains a floating point value is assigned (with the CHANGE or ADD[C] statement) as the value of a field occurrence, the %variable's entire 4, 8, or 16 bytes are copied exactly to the field occurrence.
Length-preserved PUT AS FLOAT Whenever a field occurrence or %variable that contains a floating point value is used in a PUT AS FLOAT statement, and the FLOAT format specifies a length that is the same as that of the occurrence or %variable, the entire 4, 8, or 16 bytes are copied exactly to the ouput file.

For example, if field FLT4 contains a 4-byte float value that in hexadecimal is X'41000004', then the following FUEL fragment places two lines to the output file, each containing the 4 bytes that are hexadecimal X'41000004':

PUT FLT4 AS FLOAT(4) OUTPUT %X = FLT4 PUT %X AS FLOAT(4) OUTPUT

Length-converting PUT statements

Other than obtaining the value of a float (for example, as part of an IF statement comparison or as an argument to a #function), which is explained in Using a float value, with decimal digit precision, the only context in FUEL in which a float value is transformed to a float value with a different length is in the PUT statement with a FLOAT format whose format length differs from the length of the float "input." The cases are shown below:

AS FLOAT(4) The first (and only, if the input is length 8) 8 bytes of the input are copied to a 4-byte float using the LEDR instruction, which produces the first 4 bytes of the input 8 bytes, or those 4 bytes plus 1 (times the sign of the value) if the high order bit of the second 4 bytes is 1.

Note that there is no normalization of the input value prior to rounding. Thus the low-order 31 fraction bits of an 8-byte float are ignored, and the low-order 31 + 56 fraction bits of a 16-byte float are ignored.

LEN 16 -> 8 The AS FLOAT(8) clause for a 16-byte float input is obtained by taking the first 8 bytes of the 16-byte float value.

Note that there is no rounding, as there is when converting from 8-byte to 4-byte floats, and there is no normalization. Thus the low order 56 fraction bits of the 16-byte float are ignored.

LEN 4 -> 8/16 When a 4-byte float value is the input to AS FLOAT(8) or AS FLOAT(16), the 4-byte input float value is converted to an 8-byte value that is approximately nearest the input value expressed as the nearest 6-significant-digit decimal number, as described in the LEN 4 case in Using a float value, with decimal digit precision. This is the result when the PUT format length is 8. An additional 8 bytes of zeroes are added when it is 16.
LEN 8 -> 16 The AS FLOAT(16) clause for a 8-byte float input is obtained by appending 8 additional bytes of zeroes to the unchanged 8-byte value.

Arithmetic expressions

The result of an arithmetic expression is always an 8-byte float; these are produced as described in Primitive operations. Also, after every addition or subtraction in the expression, a step is performed to ensure that the correct significance is retained as the result of that operation. This significance is based on the magnitude of the inputs and the result of the addition or subtraction.

For example, after performing the SDR to operate on the following two values:

%X = 123456789.1234 - 123456789 /* = X'4775BCD151F97247' /* - X'4775BCD150000000' /* = X'401F972470000000' /* = .123399998992682 (rounded to 15 digits)

Fast/Unload examines the magnitude of the absolute values of the result and addends, and it determines that 4 significant digits should be retained, so it rounds the result to 4 significant digits:

/* .1234 (result rounded to 4 digits) /* = X'401F972474538EF3' (float nearest to .1234)

Example

Following is one example of the behavior of Fast/Unload. Versions prior to 4.0 obtain the different result shown below.

%T2 = 1 FOR J FROM 1 TO 7 /* Get 1E-7 %T2 = %T2 / 10 END FOR PUT %T2 OUTPUT FOR J FROM 1 TO 5 /* Get .01 %T2 = %T2 * 10 END FOR PUT %T2 OUTPUT %T = 0 FOR J FROM 1 TO 10 /* Get .1 %T = %T + %T2 END FOR PUT %T OUTPUT

Results (starting with version 4.0):

0.0000001 0.00999999999999999 0.1

Results (prior to version 4.0):

0.0000001 0.00999999999999999 0.0999999999999999

See also