XmlDoc API serialization options

Multiple methods in the XmlDoc API serialize (produce the text-string representation of) the contents of an XmlDoc or XmlDoc subtree. These methods include Serial, WebSend, Xml, and Print, Audit, and Trace. The AddXml method of the HttpRequest class is also included below; it also serializes an XmlDoc and is comparable to the Websend method.

Each of the serialization methods has an "options" parameter that is a blank-delimited string (not case-sensitive) of one or more options which control aspects of the output format. These options are summarized in the table below. Following the table are several topics that concern XmlDoc serialization.

Option descriptions

The options parameter options available to an XmlDoc API serialization method vary with the individual method. The table below includes all the options from all the methods. The individual option descriptions specify the methods for which the option is available, and the individual method pages specify the options available for that method.

For direct access to a particular option in the table, you can use the following "index" of the options:

• AllowXmlDecl
• AttributeCompact
• BothCompact
• CharacterEncodeAll
• Compact
• CR
• CRLF
• EBCDIC
• ElementCompact
• ExclCanonical
• Expanded
• Indent n
• LF
• Newline
• NoEmptyElt
• NoXmlDecl
• OmitNullElement
• SortCanonical
• UTF-8
• WithComments
• XmlDecl

Option	Description
CharacterEncodeAll In methods: Audit Print Serial Trace	Use character encoding in all contexts to display Unicode characters that do not translate to EBCDIC. If this option is not specified (as of Sirius Mods 7.6), only non-translatable Unicode characters in Attribute or Element values are displayed as character references. For more information about this option, see "EBCDIC serialization of untranslatable Unicode characters", below. The CharacterEncodeAll option is available as of Sirius Mods version 7.6. It is available for the Serial method starting with Sirius Mods version 8.0.
EBCDIC In methods: Serial	This indicates that the serialization is to be in EBCDIC rather than UTF-8. The Serial method provides UTF-8 encoding by default. Since XmlDocs are stored in Unicode (under Sirius Mods 7.6 or higher,), serializing to UTF-8 involves no translation: the stored Unicode characters are merely encoded as UTF-8. Serializing to EBCDIC causes conversion of the subtree content via the Unicode tables.The serialization of untranslatable characters is desribed below. Prior to Sirius Mods 7.6, XmlDocs are stored in EBCDIC.
ExclCanonical In methods: Serial	This indicates that the output of the serialization will be in exclusive XML canonical form, as defined in the W3C "Exclusive XML Canonicalization" specification (http://www.w3.org/tr/xml-exc-c14n), which is an extension of the "XML Canonicalization" specification (http://www.w3.org/TR/xml-c14n). These specifications constrain serializations to facilitate processing such as digital signatures. This option, added in Sirius Mods version 7.0, is described in greater detail in "Canonicalization" below. Specifying any of the Serial method CR, LF, CRLF, or Indent options when you also specify ExclCanonical is allowed. Although the resulting output will not be completely canonical, it may be what you require for the purposes of a digital signature, for example. The formatting addressed by those options is defined in the Exclusive Canonicalization specification and covered by the ExclCanonical option. Similarly, the effect of the XmlDecl option contradicts the Exclusive Canonicalization specification. If you do specify the XmlDecl and ExclCanonical options together, however, the serialized XML Declaration is followed by a linefeed character.
Indent n In methods: AddXml Audit Print Serial Trace WebSend Xml	Inserts space characters (and line-ends, as described for the next option) into the serialized string such that if the string is broken at the line-ends and displayed as a tree, the display of each lower level (child element) in the subtree is indented n spaces from the starting point of the previous level (parent element). If serialized output with an Indent value of 2 is displayed as a tree, the spacing is as in the following: <top> <leaf1 xx="yy">value</leaf1> <sub> <leaf2>value</leaf2> </sub> </top> n is a non-negative integer, and its maximum value (as of Sirius Mods version 7.0) is 254. For the Print, Audit, and Trace methods only: if the Indent option is omitted, the default indent is 3 spaces. For the Serial method only: One of the line-end options, below, must also be specified. For AddXml and WebSend only: Indent may be used with one of the line-end options, below, including Newline. If Indent is specified and no line-end options are also specified, Newline is implied.
CR LF CRLF Newline In methods: AddXml Serial WebSend Xml	Line-end options for the method output: CR Insert a carriage-return character. LF Insert a linefeed character. CRLF Insert a carriage-return character followed by a linefeed character. Newline Insert the line-end sequence defined for this HttpRequest object, or for this Janus Web Server connection (by the JANUS DEFINE command or $Web_Set), as the line-end sequence in the above cases. Available for AddXml and WebSend only. You specify one of the line-end options above to provide line breaks in the output after any of the following is serialized: An element start-tag, if it has any non-text node children An element end tag An empty element tag A processing instruction (PI) A comment A text node, if it has any siblings Using one of these line-end options produces output that is similar to the BothCompact option.
NoEmptyElt In methods: AddXml Audit Print Serial Trace WebSend Xml	Deprecated as of Sirius Mods version 7.0, this option ensures that all empty elements are serialized with a start tag followed by an end tag. For example: <middleName></middleName> If NoEmptyElt is not specified, the default is to serialize an empty element with an empty element tag; using the same example as above, this would be: <middleName/> The ExclCanonical option provides the same empty element serialization as NoEmptyElt. Also, the NoEmptyElement XmlNode property specifies whether to serialize childless nodes using a separate start tag and end tag.
OmitNullElement In methods: AddXml Audit Print Serial Trace WebSend Xml	An Element node that has no children and no Attributes will not be serialized, unless it is the top level Element in the subtree being serialized. The serialization of a child-less and Attribute-less Element is omitted, even if the serialization of the Element would contain Namespace declarations in its start tag. If an Element node has no Attributes, but has (only) Element children (one or more), and all of its children are Attribute-less and child-less, then that parent Element is serialized, even though its content in the serialization is empty. That parent is serialized with a start tag and an end tag (and an inserted line separator, if called for by the serializing method's parameter options). For example, if the Serial method display of a particular XmlDoc in tree format is the following when OmitNullElement is not specified: <top> <middle> <empty/> <p:empty2 xmlns:p="uri:stuff"/> </middle> </top> Here is the display of the XmlDoc with the OmitNullElement option specified: <top> <middle> </middle> </top> But if you attempt to display only the empty subtree of the XmlDoc using OmitNullElement, the empty node is not suppressed, and the result is: </empty> The OmitNullElement option is available as of Sirius Mods version 7.3.
Compact Expanded AttributeCompact ElementCompact BothCompact In methods: Audit Print Trace	One of the following mutually exclusive output formats: Compact This is the default. An element's entire start tag is printed on a single line, which includes attributes and namespace declarations. If it has no children or has a single Text child, and does not have attributes nor namespace declarations, then the Text child is serialized on the same line as the start and end tags. For example: <top> <in1 a="xyz" b="foo"> content1 </in1> <in2>content2</in2> </top> Expanded A new line is started for each attribute, namespace declaration, and child. For example: <top> <in1 a="xyz" b="foo" > content1 </in1> <in2> content2 </in2> </top> AttributeCompact Attributes and namespace declarations are printed on the same line as the start tag. For example: <top> <in1 a="xyz" b="foo"> content1 </in1> <in2> content2 </in2> </top> ElementCompact An entire element is printed on one line, if it has no attributes nor namespace declarations and has no children other than possibly a Text child. For example: <top> <in1 a="xyz" b="foo" > content1 </in1> <in2>content2</in2> </top> BothCompact The most compacted format, this combines the effect of AttributeCompact and ElementCompact. It displays on one line an element that has no children or that has a single Text child. <top> <in1 a="xyz" b="foo">content1</in1> <in2>content2</in2> </top>
SortCanonical In methods: AddXml Audit Print Serial Trace WebSend Xml	Deprecated as of Sirius Mods version 7.0, SortCanonical serializes namespace declarations (based on the prefix being declared) and attributes (based on the namespace URI followed by the local name) in sorted order. This can be useful, for instance, when using Serial to serialize a portion of an XML document for a signature. The sort order for namespace declarations and attributes is from lowest to highest, and it uses the Unicode code ordering (for example, numbers are lower than letters). Added in Sirius Mods version 6.9 as a step towards support for canonicalization, this option is superseded by the ExclCanonical option.
UTF-8 In methods: Serial	This indicates that the serialization should be in UTF-8. This is the default.
WithComments In methods: Serial	This indicates that all Comment nodes in the specified subtree are to be included in the serialized output. Note: This option, added in Sirius Mods version 7.0, is only a supplement to the ExclCanonical option: specifying WithComments without specifying ExclCanonical has no effect. Specifying ExclCanonical without specifying WithComments causes all Comment nodes to be suppressed from the result.
XmlDecl In methods: Serial	This indicates that the serialized XmlDoc will contain the "XML Declaration" (<?xml version=...?>), if the value of the Version property is a non-null string, and if the XmlDoc is not empty. XmlDecl may only be specified if the top of the subtree being serialized is the Root node.
AllowXmlDecl NoXmlDecl In methods: AddXml Audit Print Trace WebSend Xml	These indicate whether or not the serialized XmlDoc will contain the "XML Declaration" (<?xml version=...?>). AllowXmlDecl (the default) may only be specified if the value of the Version property is a non-null string, and if the top of the subtree being serialized is the Root node. AllowXmlDecl and NoXmlDecl may not both be specified.

EBCDIC serialization of untranslatable Unicode characters

As of Sirius Mods version 7.6, XmlDoc content is stored in Unicode. The methods that support EBCDIC serialization (Serial with EBCDIC option, Print, Audit, Trace) use the Unicode tables to convert the XmlDoc content.

One feature of the conversion from Unicode is that the serializing method displays non-translatable Unicode characters stored in Attribute or Element values as character references. For example:

%doc:AddElement('top', '™':U) %doc:Print

The result of this fragment is:

<top>&#x2122;</top>

Note: prior to version 8.0 of the Sirius Mods, the Serial method did not convert non-translatable Unicode characters to character references; the result was, instead, a request cancellation.

However, when an untranslatable Unicode character occurs in a context other than Element or Attribute value (that is, a name, comment, or PI), and the default serialization options are in effect, character encoding is not used. Because it is an Element name, for example, the following statements result in a request cancellation:

%doc:AddElement('&#x2122;':U) %doc:Print

The Print method fails, attempting to translate the element name, the U+2122 character, to EBCDIC. This request cancellation can be prevented by using the CharacterEncodeAll option:

%doc:AddElement('&#x2122;':U) %doc:Print(, 'CharacterEncodeAll')

The result of the above fragment is:

<&#x2122;/>

Note: The result with CharacterEncodeAll can be misleading. Request cancellation is avoided, but it can produce a serialization that is not equivalent to the portion of the XmlDoc that was serialized.

The serialized result above is not a legal XML document, because the ampersand (&) is not a legal name character. Similarly, for an untranslatable Unicode character added to a document with AddComment or AddPI, EBCDIC serialization with CharacterEncodeAll produces a stream of characters that can be displayed, but, if those characters are deserialized, the result is not the same XmlDoc content. The XML standard does not provide for character references in names, Comments, and PIs.

For example:

%d:AddComment('&#x2122;') %d:AddComment('&#x2122;':U) %d:Print(, 'CharacterEncodeAll')

The above results in:

<!--&#x2122;--> <!--&#x2122;-->

This may lead you to believe that the two Comment nodes in the XmlDoc are identical, but the first one contains the 8 characters ™, whereas the second comment contains a single Unicode "trademark" character (™).

Serialization and the "xml:space" attribute

The effect of the xml:space attribute on the serialization of an Element that has the xml:space="preserve" or xml:space="default" attribute depends on the serialization method:

• Print, Audit, and Trace:
  The xml:space="preserve" and xml:space="default" attributes do not affect the serialized output.
• Serial, WebSend, and Xml:
  If one of the line-end options or Indent is specified, and an element to be serialized has the xml:space="preserve" attribute, then within the serialization of that element and its descendants, no line-end nor indent characters are inserted. The xml:space="default" attribute does not influence serialization, regardless of method options, nor does it cause resumption of the insertion of readability line-ends or indents if they were suspended by a containing xml:space="preserve".

Displaying whitespace characters in serializations

The serialization methods use the hexadecimal character references specified in the XML Canonicalization specification (http://www.w3.org/TR/xml-c14n) to display the following whitespace characters:

• For Attribute nodes: tab, carriage return, and linefeed
• For Text nodes: carriage return

Since the character references are not subject to the standard XML whitespace normalization, a serialized document (or subtree) that is then deserialized will retain this whitespace.

These character references are used:

tab		
carriage return	
linefeed	


The EBCDIC and corresponding ASCII encodings of the characters is:

	EBCDIC	ASCII
tab	X'05'	X'09'
carriage return	X'0D'	X'0D'
linefeed	X'25'	X'0A'

Canonicalization

Canonicalization refers to a particular serialization of an XML document that is unique, yet still a logically equivalent representation of the document. Exclusive canonicalization is canonicalization augmented by rules for preserving or excluding the namespace context (declaration) of nodes when only a portion of an XML document is serialized.

Therefore, if a portion (subtree) of an XML document is exclusively canonicalized, it is serialized uniquely and is "substantially independent of its XML context" (that is, contains all essential and no extraneous information from its ancestor nodes). This independence makes the subtree suitable for working with digital signatures.

Some of the many requirements for canonicalization are provided automatically by specifying the Serial method with no options specified. For example, UTF-8 encoding and exclusion of the XML declaration, if any, are provided by default by Serial. Specifying ExclCanonical, which is new as of Sirius Mods version 7.0, adds the following features to the no-option default:

• Sorting of namespace declarations (based on the prefix being declared) and of attributes (based on the namespace URI followed by the local name). The sort order is from lowest to highest, and it uses the Unicode code ordering (for example, numbers are lower than letters).
• For empty elements, serialization with both a start tag and an end tag, instead of using a single "empty element tag."
• The suppression of any Comment nodes that may be present in the subtree. Comment nodes are suppressed unless the WithComments option is specified along with ExclCanonical.

  For an example, see PIs and Comments.

• Special namespace declaration handling: A namespace declaration is produced only if it is utilized by an element or attribute in the subtree. The declaration is produced in the start-tag of an element that uses it (or has an attribute using it), unless the parent of the element is in the subtree and the declaration is in scope at the parent.

  For examples, see Namespace serialization and Namespace importing.

• Attribute values are always serialized within double-quotation-mark (") delimiters, and a double-quotation mark character in an attribute value is serialized as ". With or without the ExclCanonical option, these special characters in attribute values are serialized as entity and hexadecimal character references:
  • The ampersand (&) is serialized as &
  • The less-than symbol (<) is serialized as <
  • The carriage return (CR) character is serialized as 

  • The linefeed (LF) character is serialized as 

  • The tab character is serialized as 	
  For examples, see Character references.
• Within Text nodes, the following characters are serialized as entity and hexadecimal character references:

  If you specify Serial with no options:

  • The less-than symbol (<) is serialized as <
  • The ampersand (&) is serialized as &
  • The carriage return (CR) character is serialized as 


  If you specify the ExclCanonical option, the following is also true:

  • The greater-than symbol (>) is serialized as >

  For examples, see Character references.

• If serializing the Root of an XmlDoc, a linefeed character is inserted between the children of the Root. This character is represented exactly by X'25' if the EBCDIC option of Serial is used; otherwise it is represented by X'0A'.

  Note: No linefeed is inserted if the XmlDoc has one PI or Comment node and does not have an Element node. In this case (which is allowed by Janus SOAP), the XML document is not well-formed and therefore the canonicalization specifications ignore it.

• If the subtree to be serialized is a single node that is either of these:
  • A PI child of the Root
  • A single node that is a Comment child of the Root and the WithComments option is specified

  Then a linefeed character is added after the PI or Comment if there is a following Element sibling, or is added before the PI or Comment if there is a preceding Element sibling.

  Note: No linefeed is inserted if the XmlDoc does not have an Element node. In this case (which is allowed by Janus SOAP]]), the XML document is not well-formed and therefore the canonicalization specifications ignore it.

Qualifications/exceptions

• The canonicalization specifications, especially exclusive canonicalization, include references to the serialization of a subset of a document. The ExclCanonical option is based not on a subset but on a subtree.
• Although the ExclCanonical and SortCanonical options use the "Unicode" sort sequence, this is currently limited to Unicode values less than 256 (as of version 7.7 of Janus SOAP]]), so it is accomplished with an 8-byte EBCDIC to 8-byte Unicode table, which is (for all intents and purposes) merely an EBCDIC-to-ASCII translation.
• The specifications support an argument to canonicalization that is a list of namespace declarations that are to be "forced" into the serialization. The ExclCanonical option does not provide this support.

Examples

The following examples show various aspects of the ExclCanonical option. The examples use the EBCDIC option to display the result. If using ExclCanonical for digital signature processing, you probably should omit the EBCDIC option and use the default encoding, UTF-8.

Namespace serialization

Under exclusive canonicalization, a namespace is not serialized if it is not necessary. In this example, the subtree to be serialized is displayed in green font in the request code that follows:

Begin %doc is Object XmlDoc %doc = New %l is longstring %sl is object stringlist %sl = New text to %sl <top> <a xmlns:p3="urn:p3" xmlns:p2="urn:p2" xmlns:p1="urn:p1"> <p1:b/> <p2:b/> </a> </top> end text Call %doc:LoadXml(%sl) Print 'Exclcan via ParseLines:' %sl = New %l=%doc:Serial('top/a', 'EBCDIC exclcanonical indent 2 lf') %sl:Parselines(%l) %sl:Print End

The exclusive canonical serialization (displayed, after being parsed from string to Stringlist, with line breaks and indent for the sake of clarity) omits the declaration for p3, because it is not utilized in the serialized subtree:

<a> <p1:b xmlns:p1="urn:p1"></p1:b> <p2:b xmlns:p2="urn:p2"></p2:b> </a>

An element utilizes an in-scope namespace declaration in either of these cases:

• The element is prefixed and the declaration is of that prefix.
• The element is unprefixed and it is a default namespace declaration.

An attribute utilizes an in-scope namespace declaration if the attribute is prefixed and the declaration is of that prefix.

In the preceding example, there was no alternative to removing the non-utilized declaration for p3, but if it were utilized by a descendant element "lower" in the document tree, it would be moved to that element.

Another application of the utilization rule is shown in the next example.

Namespace importing

Under exclusive canonicalization, namespaces are imported to where they are needed.

Using the same type of request as in the preceding example, the w element is the subtree to serialize (green font):

<a xmlns:p3="urn:p3" xmlns:p2="urn:p2" xmlns:p1="urn:p1"> <w> <p1:b/> <p2:b/> </w> </a>

Exclusive canonical serialization (display form), which gets required namespace declarations from an ancestor of the serialized subtree:

<w> <p1:b xmlns:p1="urn:p1"></p1:b> <p2:b xmlns:p2="urn:p2"></p2:b> </w>

PIs and Comments

Using the same type of request as in the Namespace serialization example, this is the subtree to be serialized (display form):

<a> <!-- Comment 1 --> <w> <?pi-without-data?> </w> </a>

Exclusive canonical serialization (display form), which omits the Comment node:

<a> <w> <?pi-without-data?> </w> </a>

Note: To include the Comment node, specify also the WithComments option of Serial.

Character references

Using the same type of request as in the Namespace serialization example, this is the subtree to serialize (display form):

<doc> <comp>val>"0" val&lt;"10"</comp> <comp expr='val>"0"'></comp> <norm attr=' &apos; &#xD;&#xA;&#x9; &apos; '/> <white>&#x9;&#xD;&#xA;</white> </doc>

This is the result from the Serial method with no options specified (display form, and the <white> element has a line that wraps to emphasize the non-visible linefeed character it contains):

<doc> <comp>val>"0" val&lt;"10"</comp> <comp expr='val>"0"'></comp> <norm attr=" ' &#xD;&#xA;&#x9; ' "/> <white> &#xD; </white> </doc>

The exclusive canonical serialization follows (display form, wrapped <white> element line has no indent).

<doc> <comp>val&gt;"0" val&lt;"10"</comp> <comp expr="val>&quot;0&quot;"></comp> <norm attr=" ' &#xD;&#xA;&#x9; ' "></norm> <white> &#xD; </white> </doc>

The differences (green font) from no-option Serial include:

• The greater-than symbol (>) within a text node is serialized as >.
• Attribute values are enclosed in double-quotation marks (").
• A double-quotation mark in an attribute value is serialized as ".
• An empty element is serialized with two tags (a start tag followed by an end tag), not with a single empty-element tag.