Translate tables: Difference between revisions

From m204wiki
Jump to navigation Jump to search
m (→‎Defining translate tables: misc formatting)
m (misc formatting)
Line 1: Line 1:
Janus Connectivity products like [[Janus Web Server]] and [[Janus SOAP]] typically communicate with applications on remote computers. In many cases, these remote computers use different encodings for symbols, numerals, and letters or “characters.A character usually occupies a single byte of memory on any computer, but the character displayed for a particular bit value may vary. For example, in an ASCII encoding, the numeral “1” is represented by the hexadecimal value X'31'. On a computer that uses an EBCDIC encoding (IBM mainframes), the representation is X'F1'. Therefore a numeral “1” transmitted from an ASCII-based computer to an EBCDIC-based computer, or vice versa, will be misinterpreted. To avoid such misinterpretation, a method of converting a byte from one encoding to another is needed. Janus translate tables were implemented to solve this problem.
Janus Connectivity products like [[Janus Web Server]] and [[Janus SOAP]] typically communicate with applications on remote computers. In many cases, these remote computers use different encodings for symbols, numerals, and letters or "characters." A character usually occupies a single byte of memory on any computer, but the character displayed for a particular bit value may vary.  
 
For example, in an ASCII encoding, the numeral "1" is represented by the hexadecimal value X'31'. On a computer that uses an EBCDIC encoding (IBM mainframes), the representation is X'F1'. Therefore a numeral "1" transmitted from an ASCII-based computer to an EBCDIC-based computer, or vice versa, will be misinterpreted. To avoid such misinterpretation, a method of converting a byte from one encoding to another is needed. Janus translate tables were implemented to solve this problem.


==Versions and capability==
==Versions and capability==
In Janus release 5.6 and earlier, a fixed translate table provided the most common type of translation: that is, it converted ASCII to EBCDIC when receiving, and it converted EBCDIC to ASCII when sending. In most cases, this is adequate for any of the Janus products. There are, however, many different types of EBCDIC and ASCII character sets, designed to display foreign language characters, currency, and other special symbols. To ensure these characters are transmitted and received correctly, a more flexible approach is required.
In Janus release 5.6 and earlier, a fixed translate table provided the most common type of translation: that is, it converted ASCII to EBCDIC when receiving, and it converted EBCDIC to ASCII when sending. In most cases, this is adequate for any of the Janus products. There are, however, many different types of EBCDIC and ASCII character sets, designed to display foreign language characters, currency, and other special symbols. To ensure these characters are transmitted and received correctly, a more flexible approach is required.


In Janus 6.0, the translate table facility was expanded to allow simultaneous use of many different translate tables, and further, to allow user-written or customized translate tables.
In Janus 6.0, the translate table facility was expanded to allow simultaneous use of many different translate tables, and further, to allow user-written or customized translate tables.


==Using Translate Tables==
==Using translate tables==
 
To use a Janus translate table other than the standard table, you must do one of the following:
To use Janus translate tables, you must do one of the following if you do not want to use the standard tables:
 
<ul>
<ul>
<li>Code your own translate table source.</li>
<li>Code your own translate table source.</li>
<li>Use an external translate table.</li>
<li>Use an external translate table.</li>
<li>Use or modify a Sirius-provided translate table.</li>
<li>Use or modify a Rocket-provided translate table.</li>
</ul>
</ul>


The [[JANUS LOADXT]] command then validates the new table, converts it to an internal format, and adds or replaces any existing table with the same name.
You then use the <var>[[JANUS LOADXT]]</var> command to validate the new table, convert it to an internal format, and add or replace any existing table with the same name.
 
To use the most up-to-date Sirius-provided translation file, specify the following to load and name the Unicode table that handles Janus SOAP XML document translations (you must be running Sirius Mods version 7.6 or higher):
 
<p class="code">  JANUS LOADXT xtabname UNICODE</p>


See [[JANUS LOADXT]].
To use the most up-to-date Rocket-provided translation file, specify the following to load and name the Unicode table that handles Janus SOAP XML document translations:


To view the translations in effect after issuing this command:
<p class="code">JANUS LOADXT <i>xtabname</i> UNICODE </p>


To view the translations in effect after issuing this <var>LOADXT</var> command:
<ol>
<ol>
<li>Determine the codepage in use at your site for translations between Unicode and EBCDIC:
<li>Determine the codepage in use at your site for translations between Unicode and EBCDIC:


<p class="code"> UNICODE Display Table Standard</p>
<p class="code">UNICODE Display Table Standard </p>


The result of this UNICODE command (described further in the Janus SOAP Reference Manual) will begin with something like:
The result of this <var>UNICODE</var> command begins with something like:


<p class="code"> * The following commands produce the current Unicode
<p class="code">* The following commands produce the current Unicode
     translation tables:
     translation tables:


  UNICODE Table Standard Base Codepage pppp
UNICODE Table Standard Base Codepage <i>pppp</i>
</p>
</p>


where pppp is the codepage in use.</li>
where <var class="term">pppp</var> is the codepage in use. </li>


<li> Using the above codepage, issue:
<li>Using the above codepage, issue:


<p class="code"> UNICODE Display Codepage pppp</p>
<p class="code">UNICODE Display Codepage <i>pppp</i></p>
 
<p>
The result will be many lines of single-character translations, one line for each
The result will be many lines of single-character translations, one line for each character translated between EBCDIC and Unicode. This format differs entirely from that described below in "Defining translate tables." </p></li>
character translated between EBCDIC and Unicode. This format differs entirely
from that described below in “Defining translate tables”.</li>
</ol>
</ol>


To provide backward compatibility with earlier releases, two additional translate tables are loaded during Model 204 initialization. The standard table, named “STANDARD,is an exact copy of the translate table used with earlier Janus releases. This is the default translate table for all Janus ports, unless you define Web ports with DBCS support. In this case, the “JAPANIBM” translate table is used. Both these tables are built-in to the [[Sirius Mods]] code. As described in the next section, “Defining translate tables”, the JANUS DISPXT and JANUS LOADXT commands let you view (or copy for modification purposes) and reload these translate tables.
To provide backward compatibility with earlier releases, two additional translate tables are loaded during Model&nbsp;204 initialization. The standard table, named "STANDARD," is an exact copy of the translate table used with earlier Janus releases. This is the default translate table for all Janus ports, unless you define Web ports with DBCS support. In this case, the "JAPANIBM" translate table is used. Both these tables are built-in to the [[Sirius Mods]] code. As described in the next section, "Defining translate tables," the <var>JANUS DISPXT</var> and <var>JANUS LOADXT</var> commands let you view (or copy for modification purposes) and reload these translate tables.


==Defining translate tables==
==Defining translate tables==
You can define your own translate tables and load them with the <var>[[JANUS LOADXT]]</var> command. These tables can be defined in a Model&nbsp;204 procedure file or group, or in a sequential file or MVS Partitioned Data Set
You can define your own translate tables and load them with the <var>JANUS LOADXT</var> command. These tables can be defined in a Model&nbsp;204 procedure file or group, or in a sequential file or MVS Partitioned Data Set
(PDS).
(PDS).


A translate table must define a 256-byte EBCDIC-to-ASCII table, followed by a 256-byte ASCII-to-EBCDIC table. Comment lines are allowed and can begin with a semicolon (<tt>;</tt>) or an asterisk (<tt>*</tt>).
A translate table must define a 256-byte EBCDIC-to-ASCII table, followed by a 256-byte ASCII-to-EBCDIC table. Comment lines are allowed, and they can begin with a semicolon (<tt>;</tt>) or an asterisk (<tt>*</tt>).


Comments are allowed on the same line as definition lines, provided they are preceded by either of the two comment characters.
Comments are allowed on the same line as definition lines, provided they are preceded by either of the two comment characters.
Line 91: Line 85:
Each character in the translate table is coded in hexadecimal format and must be two bytes long. Each two-byte hexadecimal string must be separated by one or more blanks.
Each character in the translate table is coded in hexadecimal format and must be two bytes long. Each two-byte hexadecimal string must be separated by one or more blanks.


Following is an abbreviated example of translate table source code. Each of the rows of hexadecimal characters define the EBCDIC encoding for an ASCII character at that offset in the table. So for example, to find out how this table would translate an ASCII 2 (hexadecimal X'32'), you locate offset X'32' in the table, which is in the fourth row, third pair of characters. Note that the top and right-side comments help to locate offsets quickly. At offset X'32' in this table, you find the value X'F2', which is the EBCDIC encoding for the numeral 2.
Following is an abbreviated example of translate table source code. Each of the rows of hexadecimal characters define the EBCDIC encoding for an ASCII character at that offset in the table. So for example, to find out how this table would translate an ASCII 2 (hexadecimal X'32'), you locate offset X'32' in the table, which is in the fourth row, third pair of characters. Note that the top and right-side comments (shown in bold) help to locate offsets quickly. At offset X'32' in this table, you find the value X'F2', which is the EBCDIC encoding for the numeral 2.


Of course, to send characters, you would want to translate EBCDIC encoding back into ASCII. Therefore, an EBCDIC to ASCII section must also be coded. This is not shown in the example.
Of course, to send characters, you would want to translate EBCDIC encoding back into ASCII. Therefore, an EBCDIC to ASCII section must also be coded. This is not shown in the example.
Line 97: Line 91:
<p class="code"> ;
<p class="code"> ;
  ; ASCII-to-EBCDIC table for English (US) CECP Code Page 037
  ; ASCII-to-EBCDIC table for English (US) CECP Code Page 037
  ; 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
  ; <b>00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F</b>
  ;
  ;
  00 01 02 03 37 2D 2E 2F 16 05 25 0B 0C 0D 0E 0F ; 00 ;
  00 01 02 03 37 2D 2E 2F 16 05 25 0B 0C 0D 0E 0F ; <b>00</b> ;
  10 11 12 13 3C 3D 32 26 18 19 3F 27 1C 1D 1E 1F ; 10 ;
  10 11 12 13 3C 3D 32 26 18 19 3F 27 1C 1D 1E 1F ; <b>10</b> ;
  40 5A 7F 7B 5B 6C 50 7D 4D 5D 5C 4E 6B 60 4B 61 ; 20 ;
  40 5A 7F 7B 5B 6C 50 7D 4D 5D 5C 4E 6B 60 4B 61 ; <b>20</b> ;
  F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 7A 5E 4C 7E 6E 6F ; 30 ;
  F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 7A 5E 4C 7E 6E 6F ; <b>30</b> ;
  7C C1 C2 C3 C4 C5 C6 C7 C8 C9 D1 D2 D3 D4 D5 D6 ; 40 ;
  7C C1 C2 C3 C4 C5 C6 C7 C8 C9 D1 D2 D3 D4 D5 D6 ; <b>40</b> ;
  D7 D8 D9 E2 E3 E4 E5 E6 E7 E8 E9 BA E0 BB B0 6D ; 50 ;
  D7 D8 D9 E2 E3 E4 E5 E6 E7 E8 E9 BA E0 BB B0 6D ; <b>50</b> ;
  79 81 82 83 84 85 86 87 88 89 91 92 93 94 95 96 ; 60 ;
  79 81 82 83 84 85 86 87 88 89 91 92 93 94 95 96 ; <b>60</b> ;
  97 98 99 A2 A3 A4 A5 A6 A7 A8 A9 C0 4F D0 A1 07 ; 70 ;
  97 98 99 A2 A3 A4 A5 A6 A7 A8 A9 C0 4F D0 A1 07 ; <b>70</b> ;
  68 DC 51 42 43 44 47 48 52 53 54 57 56 58 63 67 ; 80 ;
  68 DC 51 42 43 44 47 48 52 53 54 57 56 58 63 67 ; <b>80</b> ;
  71 9C 9E CB CC CD DB DD DF EC FC 70 B1 80 BF FF ; 90 ;
  71 9C 9E CB CC CD DB DD DF EC FC 70 B1 80 BF FF ; <b>90</b> ;
  45 55 CE DE 49 69 9A 9B AB AF 5F B8 B7 AA 8A 8B ; A0 ;
  45 55 CE DE 49 69 9A 9B AB AF 5F B8 B7 AA 8A 8B ; <b>A0</b> ;
  2B 2C 09 21 28 65 62 64 B4 38 31 34 33 4A B2 24 ; B0 ;
  2B 2C 09 21 28 65 62 64 B4 38 31 34 33 4A B2 24 ; <b>B0</b> ;
  22 17 29 06 20 2A 46 66 1A 35 08 39 36 30 3A 9F ; C0 ;
  22 17 29 06 20 2A 46 66 1A 35 08 39 36 30 3A 9F ; <b>C0</b> ;
  8C AC 72 73 74 0A 75 76 77 23 15 14 04 6A 78 3B ; D0 ;
  8C AC 72 73 74 0A 75 76 77 23 15 14 04 6A 78 3B ; <b>D0</b> ;
  EE 59 EB ED CF EF A0 8E AE FE FB FD 8D AD BC BE ; E0 ;
  EE 59 EB ED CF EF A0 8E AE FE FB FD 8D AD BC BE ; <b>E0</b> ;
  CA 8F 1B B9 B6 B5 E1 9D 90 BD B3 DA FA EA 3E 41 ; F0 ;
  CA 8F 1B B9 B6 B5 E1 9D 90 BD B3 DA FA EA 3E 41 ; <b>F0</b> ;
</p>
</p>


You can specify existing external translate table source when loading translate tables.
You can specify existing external translate table source when loading translate tables. For example, IBM supplies many translate tables with its TCP/IP product under z/OS. You can use those tables directly with <var>JANUS LOADXT</var>:


For example, IBM supplies many translate tables with its TCP/IP product under MVS. You can use these tables directly with <var>JANUS LOADXT</var>. To use these tables, you must first allocate the dataset and member you want to load. Use the Model&nbsp;204 <var>ALLOCATE</var> command to do this:
<ol>
<li>Allocate the data set and member you want to load, using the Model&nbsp;204 <var>ALLOCATE</var> command:


<p class="code">ALLOCATE IBMXTAB WITH SCOPE=SYSTEM DSN=TCPIP.AEZAXLT1 -
<p class="code">ALLOCATE IBMXTAB WITH SCOPE=SYSTEM DSN=TCPIP.AEZAXLT1 -
   MEMBER=US OLD SEQUENTIAL VOLUME=OS260R
   MEMBER=US OLD SEQUENTIAL VOLUME=OS260R
</p>
</p></li>


Then use the <var>JANUS LOADXT</var> command to load the translate table:
<li>Use the <var>JANUS LOADXT</var> command to load the translate table:


<p class="code">JANUS LOADXT <i>name</i> DDNAME IBMXTAB</p>
<p class="code">JANUS LOADXT <i>name</i> DDNAME IBMXTAB</p></li>
</ol>


You can obtain source copies of any previously loaded translate table (including internal tables) by using the <var>JANUS DISPXT</var> command and the Model&nbsp;204 <var>USE</var> command. For example, to obtain a copy of the current translate table "STANDARD," issue the following commands:
You can obtain source copies of any previously loaded translate table (including internal tables) by using the <var>JANUS DISPXT</var> command and the Model&nbsp;204 <var>USE</var> command. For example, to obtain a copy of the current translate table "STANDARD," issue the following commands:
Line 134: Line 130:
JANUS DISPXT STANDARD </p>
JANUS DISPXT STANDARD </p>


You can then modify the source and reload the table using the <var>JANUS LOADXT</var> command.
You can then modify the source, and reload the table using the <var>JANUS LOADXT</var> command.


<p class="note"><b>Note:</b> Because the <code>STANDARD</code> table in the example above may already be a modified version of the STANDARD table distributed with Janus TCP/IP Base, if you definitely want to copy the distributed STANDARD table, issue the <var>USE</var> and <var>JANUS DISPXT</var> commands in a "private" Online where you are confident the table has not been modified.
<p class="note"><b>Note:</b> Because the <code>STANDARD</code> table in the example above may already be a modified version of the STANDARD table distributed with Janus TCP/IP Base, if you definitely want to copy the distributed STANDARD table, issue the <var>USE</var> and <var>JANUS DISPXT</var> commands in a "private" Online where you are confident the table has not been modified.

Revision as of 23:33, 9 January 2018

Janus Connectivity products like Janus Web Server and Janus SOAP typically communicate with applications on remote computers. In many cases, these remote computers use different encodings for symbols, numerals, and letters or "characters." A character usually occupies a single byte of memory on any computer, but the character displayed for a particular bit value may vary.

For example, in an ASCII encoding, the numeral "1" is represented by the hexadecimal value X'31'. On a computer that uses an EBCDIC encoding (IBM mainframes), the representation is X'F1'. Therefore a numeral "1" transmitted from an ASCII-based computer to an EBCDIC-based computer, or vice versa, will be misinterpreted. To avoid such misinterpretation, a method of converting a byte from one encoding to another is needed. Janus translate tables were implemented to solve this problem.

Versions and capability

In Janus release 5.6 and earlier, a fixed translate table provided the most common type of translation: that is, it converted ASCII to EBCDIC when receiving, and it converted EBCDIC to ASCII when sending. In most cases, this is adequate for any of the Janus products. There are, however, many different types of EBCDIC and ASCII character sets, designed to display foreign language characters, currency, and other special symbols. To ensure these characters are transmitted and received correctly, a more flexible approach is required.

In Janus 6.0, the translate table facility was expanded to allow simultaneous use of many different translate tables, and further, to allow user-written or customized translate tables.

Using translate tables

To use a Janus translate table other than the standard table, you must do one of the following:

  • Code your own translate table source.
  • Use an external translate table.
  • Use or modify a Rocket-provided translate table.

You then use the JANUS LOADXT command to validate the new table, convert it to an internal format, and add or replace any existing table with the same name.

To use the most up-to-date Rocket-provided translation file, specify the following to load and name the Unicode table that handles Janus SOAP XML document translations:

JANUS LOADXT xtabname UNICODE

To view the translations in effect after issuing this LOADXT command:

  1. Determine the codepage in use at your site for translations between Unicode and EBCDIC:

    UNICODE Display Table Standard

    The result of this UNICODE command begins with something like:

    * The following commands produce the current Unicode translation tables: UNICODE Table Standard Base Codepage pppp

    where pppp is the codepage in use.
  2. Using the above codepage, issue:

    UNICODE Display Codepage pppp

    The result will be many lines of single-character translations, one line for each character translated between EBCDIC and Unicode. This format differs entirely from that described below in "Defining translate tables."

To provide backward compatibility with earlier releases, two additional translate tables are loaded during Model 204 initialization. The standard table, named "STANDARD," is an exact copy of the translate table used with earlier Janus releases. This is the default translate table for all Janus ports, unless you define Web ports with DBCS support. In this case, the "JAPANIBM" translate table is used. Both these tables are built-in to the Sirius Mods code. As described in the next section, "Defining translate tables," the JANUS DISPXT and JANUS LOADXT commands let you view (or copy for modification purposes) and reload these translate tables.

Defining translate tables

You can define your own translate tables and load them with the JANUS LOADXT command. These tables can be defined in a Model 204 procedure file or group, or in a sequential file or MVS Partitioned Data Set (PDS).

A translate table must define a 256-byte EBCDIC-to-ASCII table, followed by a 256-byte ASCII-to-EBCDIC table. Comment lines are allowed, and they can begin with a semicolon (;) or an asterisk (*).

Comments are allowed on the same line as definition lines, provided they are preceded by either of the two comment characters.

You can define any number of bytes of the translate table on a single line, but it is easier to read and maintain if you define 16 bytes per line. Certain characters must be translated to their equivalent values in EBCDIC and ASCII for basic Janus functions to work correctly. Therefore, these characters are reserved for use by Janus, and they may not be translated to any other characters. These reserved characters are the following:

0-9 a-z A-Z . period < less than ( left parenthesis + plus & ampersand  ! exclamation * asterisk ) right parenthesis  ; semicolon / forward slash , comma  % percent > greater than  ? question mark  : colon # pound sign ' single quote = equal sign " double quote

JANUS LOADXT will validate any table it attempts to load, and if these characters are not preserved when translating to ASCII and back, the table will not be loaded.

Each character in the translate table is coded in hexadecimal format and must be two bytes long. Each two-byte hexadecimal string must be separated by one or more blanks.

Following is an abbreviated example of translate table source code. Each of the rows of hexadecimal characters define the EBCDIC encoding for an ASCII character at that offset in the table. So for example, to find out how this table would translate an ASCII 2 (hexadecimal X'32'), you locate offset X'32' in the table, which is in the fourth row, third pair of characters. Note that the top and right-side comments (shown in bold) help to locate offsets quickly. At offset X'32' in this table, you find the value X'F2', which is the EBCDIC encoding for the numeral 2.

Of course, to send characters, you would want to translate EBCDIC encoding back into ASCII. Therefore, an EBCDIC to ASCII section must also be coded. This is not shown in the example.

 ;  ; ASCII-to-EBCDIC table for English (US) CECP Code Page 037  ; 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F  ; 00 01 02 03 37 2D 2E 2F 16 05 25 0B 0C 0D 0E 0F ; 00 ; 10 11 12 13 3C 3D 32 26 18 19 3F 27 1C 1D 1E 1F ; 10 ; 40 5A 7F 7B 5B 6C 50 7D 4D 5D 5C 4E 6B 60 4B 61 ; 20 ; F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 7A 5E 4C 7E 6E 6F ; 30 ; 7C C1 C2 C3 C4 C5 C6 C7 C8 C9 D1 D2 D3 D4 D5 D6 ; 40 ; D7 D8 D9 E2 E3 E4 E5 E6 E7 E8 E9 BA E0 BB B0 6D ; 50 ; 79 81 82 83 84 85 86 87 88 89 91 92 93 94 95 96 ; 60 ; 97 98 99 A2 A3 A4 A5 A6 A7 A8 A9 C0 4F D0 A1 07 ; 70 ; 68 DC 51 42 43 44 47 48 52 53 54 57 56 58 63 67 ; 80 ; 71 9C 9E CB CC CD DB DD DF EC FC 70 B1 80 BF FF ; 90 ; 45 55 CE DE 49 69 9A 9B AB AF 5F B8 B7 AA 8A 8B ; A0 ; 2B 2C 09 21 28 65 62 64 B4 38 31 34 33 4A B2 24 ; B0 ; 22 17 29 06 20 2A 46 66 1A 35 08 39 36 30 3A 9F ; C0 ; 8C AC 72 73 74 0A 75 76 77 23 15 14 04 6A 78 3B ; D0 ; EE 59 EB ED CF EF A0 8E AE FE FB FD 8D AD BC BE ; E0 ; CA 8F 1B B9 B6 B5 E1 9D 90 BD B3 DA FA EA 3E 41 ; F0 ;

You can specify existing external translate table source when loading translate tables. For example, IBM supplies many translate tables with its TCP/IP product under z/OS. You can use those tables directly with JANUS LOADXT:

  1. Allocate the data set and member you want to load, using the Model 204 ALLOCATE command:

    ALLOCATE IBMXTAB WITH SCOPE=SYSTEM DSN=TCPIP.AEZAXLT1 - MEMBER=US OLD SEQUENTIAL VOLUME=OS260R

  2. Use the JANUS LOADXT command to load the translate table:

    JANUS LOADXT name DDNAME IBMXTAB

You can obtain source copies of any previously loaded translate table (including internal tables) by using the JANUS DISPXT command and the Model 204 USE command. For example, to obtain a copy of the current translate table "STANDARD," issue the following commands:

USE ddname JANUS DISPXT STANDARD

You can then modify the source, and reload the table using the JANUS LOADXT command.

Note: Because the STANDARD table in the example above may already be a modified version of the STANDARD table distributed with Janus TCP/IP Base, if you definitely want to copy the distributed STANDARD table, issue the USE and JANUS DISPXT commands in a "private" Online where you are confident the table has not been modified.

See also