m204wiki - User contributions [en]

List (SortOrder function)

2010-12-17T14:45:32Z

198.242.244.228:

Create SortOrder object from input list of SortOrder objects
[[Category:SortOrder methods|List constructor]]
[[Category:System methods]]



List is a member of the [[SortOrder class]].


This shared function is a virtual constructor, or factory method, for SortOrders.
The List
method invokes the creation of a new SortOrder instance, then populates that
instance with items from an input list that are themselves SortOrder instances.
Designed for specifying multiple sort criteria, the SortOrders in the object that
List returns are applied consecutively by the sorting method
that uses the SortOrder.
===Syntax===
%sord = [%(sortorder for itemtype):] List(sordlist)
===Syntax terms===
<dl>
<dt>%sord
<dd>A SortOrder object variable to contain the new object instance.
<dt>%(sortorder for itemtype)
<dd>This optional specification of the class and collection item type
in parentheses indicates
that the method is shared and does not operate on a specific instance.
<dt>sordlist
<dd>A comma-delimited set of SortOrder objects, which may be explicitly specified
[[Ascending and Descending (SortOrder constructors)|Ascending/Descending constructors]],
SortOrder object variables, or another SortOrder List specification.
These individual SortOrders are applied to the target collection items
in the order they are specified in this argument.

If a sortorder like ascending(price) is a single sort criterion, then the
maximum number of sort criteria a List call may produce is seven.

</dl>
===Usage Notes===
<ul>
<li>For more information about factory methods, see [[??]] refid=vconst..
<li>List is a constructor and as such can be called with no method
object, with an explicit class name, or with an object variable,
even if that object is null:
<pre>
%sord = List(&'italic(sordlist))
%sord = %(sortorder for float):List(&'italic(sordlist))
%sord = %sord:List(&'italic(sordlist))
</pre>
</ul>
===Examples===

The typical way to use SortOrder lists is in a Sort or SortNew
method invocation:
<pre>
%orders:sort(%myorder)
</pre>

Where, for example:
<pre>
%myorder = list(ascending(quantity), descending(price))
</pre>

Since the arguments to the SortOrder List method are also SortOrders, you could
also pass the %myorder variable in another List call.
The following List statement
inserts the %myorder criteria into the middle of two other sort criteria:
<pre>
%mynewOrder = list(ascending(length), %myorder, -
descending(reverse))
</pre>

You can even pass a List call within a List call:
<pre>
%myorder = list(ascending(change), -
list(ascending(quantity), descending (price)))
</pre>

Ascending and Descending (SortOrder functions)

2010-12-17T13:00:31Z

198.242.244.228:

Create SortOrder object to sort in ascending o
[[Category:SortOrder methods|Ascending and Descending constructors]]
[[Category:System methods]]



Ascending and Descending are members of the [[SortOrder class]].


These shared methods create a new instance of the SortOrder class.
The ordering direction of the new instance is given by the constructor name:
Ascending is low-to-high; Descending is high-to-low.

The sorting key of the new instance is specified by the input parameter
to the constructor.
This parameter is a function that gets
applied to each item in the collection that is being sorted, and it must be
a method that operates on the item type and returns a User Language intrinsic
datatype (Float, String, Longstring, or Unicode) value.
===Ascending/Descending syntax===
%sord = [%(sortorder for itemtype):] Ascending(function)

%sord = [%(sortorder for itemtype):] Descending(function)
===Syntax terms===
<dl>
<dt>%sord
<dd>A SortOrder object variable to contain the new object instance.
<dt>%(sortorder for itemtype)
<dd>This optional specification of the class and collection item type
in parentheses indicates
that the method is shared and does not operate on a specific instance.
<dt>function
<dd>A method value (a method name literal, a method variable, a class Variable,
or even a method that returns a method value) for a method
that operates on objects of the type specified on the
declaration of the collection being sorted, and that returns a numeric or
string value.

The default ''function'' value as of ''Sirius Mods'' version 7.6
is the special value <tt>This</tt>,
described further in the “Usage Notes,” below.

</dl>
===Usage Notes===
<ul>
<li>For more information about the ''function'' parameter,
see [[SortOrder class#Specifying a SortOrder's sort key method|Specifying a SortOrder's sort key method]].
<li>Ascending and Descending are constructors and as such can be called with
no method object, with an explicit class name, or with an object variable,
even if that object is null:
<pre>
%sord = Descending(&'italic(function))
%sord = %(sortorder for float):Descending(&'italic(function))
%sord = %sord:Descending(&'italic(function))
</pre>
<li>The Ascending or Descending parameter is a method value, not a User Language expression,
and you may not specify a function that itself has an argument.
Example [[??]] refid=xmpsord., however, shows a way to apply such a function
in a sort.
<li>As of ''Sirius Mods'' version 7.6, the
default SortOrder ''function'' value is <tt>This</tt>, a
method value that is valid for User Language intrinsic method objects only.
The identity method <tt>This</tt> returns the value of the
method object to which it is applied.

The following statements are therefore equivalent:
<pre>
%sord = Descending(this)
%sord = Descending
</pre>
'''Note:'''
Using the default SortOrder above on a non-intrinsic method object
produces a compilation error.
</ul>

StartOfString (StringTokenizer function)

2010-12-16T03:44:56Z

198.242.244.228:

Substring that precedes first character of current toke
[[Category:StringTokenizer methods|StartOfString function]]
[[Category:System methods]]



StartOfString is a member of the [[StringTokenizer class]].


This method returns the substring of the tokenizer string
that precedes the first character of the current token.
===Syntax===
%string = %tok:StartOfString
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to receive the characters that precede the current position.
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>StartOfString does not advance the position.
<li>The substring returned by StartOfString ends at (but does not include)
the first character of the current token,
the position of which is the value of [[CurrentTokenPosition (StringTokenizer property)|CurrentTokenPosition]].
The value of the current token is returned by [[CurrentToken (StringTokenizer function)|CurrentToken]].
<li>StartOfString returns a null string if it is issued before a token is
found or if the value of [[NextPosition (StringTokenizer property)|NextPosition]] is <tt>1</tt>.
<li>The [[RestOfString (StringTokenizer function)|RestOfString]] function
returns the substring of the tokenizer string that follows the
present tokenizing position.
</ul>
===Examples===

The following request fragment shows that consecutive StartOfString statements
do not advance the position:
<pre>
%tok:string = 'A StartOfString example'
%tok:findToken:('example')
printText {%tok:startOfString}
printText {%tok:startOfString}\\
</pre>

The result is:
<pre>
A StartOfString
A StartOfString \\
</pre>

Spaces (StringTokenizer property)

2010-12-16T03:40:46Z

198.242.244.228:

Characters which are the token-delimiting "whitespace" charact
[[Category:StringTokenizer methods|Spaces property]]
[[Category:System methods]]



Spaces is a member of the [[StringTokenizer class]].


This readWrite property returns or sets the characters that are recognized as
“whitespace”characters, that is, as characters that separate tokens.
Such a character is also called a “non-token delimiter,”
because it is a delimiter that is not itself a token.

You can specify “token delimiters,” which are delimiters that are also tokens,
using the [[TokenChars (StringTokenizer property)|TokenChars]] property or the [[New (StringTokenizer constructor)|New]] constructor TokenChars parameter.

One or multiple consecutive whitespace characters mark the end or beginning
of a token, except when these spaces are part of a quoted region.
In that case, the spaces become non-delimiters.

===Syntax===
%string = %tok:Spaces

%tok:Spaces = %string
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to contain the returned value of the current
whitespace characters or to be set as the new value(s).
The default value for a new tokenizer instance is a blank character.

If you are setting Spaces, each character in the string is a
space character — that is, you may not separate characters —
and no character may repeat (except for apostrophe, which may be doubled).
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>The Spaces characters are initially settable in the [[New (StringTokenizer constructor)|New]] call that creates the tokenizer instance.
<li>No character can be one of the Spaces characters and also
one of either the Quotes characters or the TokenChars characters.
</ul>
===Examples===

In the following example, a Stringtokenizer object is instantiated
with the default parameter settings for the New constructor.
Then the input string's tokens are returned in sequence:
<pre>
%tok is object stringtokenizer
%tok = new
%tok:string = 'Bad,F ?3mO ,7,{x Z}'
repeat while not %tok:atEnd
printText {~} is {%tok:nextToken}
end repeat
</pre>

The resulting tokens, which were delimited by the default space character (blank), are:
<pre>
%tok:nextToken is Bad,F
%tok:nextToken is ?3mO
%tok:nextToken is ,7,{x
%tok:nextToken is Z}
</pre>

If instead the tokenizer object is instantiated with a comma as the space character
 
(<tt>%tok = new(Spaces=',')</tt>), the resulting tokens are:
<pre>
%tok:nextToken is Bad
%tok:nextToken is F ?3mO
%tok:nextToken is 7
%tok:nextToken is {x Z}
</pre>

SkipTokens (StringTokenizer subroutine)

2010-12-16T03:30:26Z

198.242.244.228: /* Examples */

Move current token forward a specified number of tokens
[[Category:StringTokenizer methods|SkipTokens subroutine]]
[[Category:System methods]]



SkipTokens is a member of the [[StringTokenizer class]].


This method moves the current token and tokenizing position forward in the
tokenizer string the number of tokens specified in the method argument.
Skipping one token is equivalent to executing [[NextToken (StringTokenizer function)|NextToken]];
skipping three tokens is equivalent to executing NextToken three times consecutively.

If a token is found after advancing the designated number of tokens, the
tokenizing position in the string is moved after the found token, and
[[CurrentToken (StringTokenizer function)|CurrentToken]] will return the found token.
If the end of the string occurs before the specified number of tokens,
the request is cancelled.

The SkipTokens subroutine is available as of version 7.7 of the ''Sirius Mods''.
===Syntax===
%tok:SkipTokens(num)
===Syntax terms===
<dl>
<dt>%tok
<dd>A StringTokenizer object variable.
<dt>num
<dd>A numeric variable that is the number of tokens to skip.
The value of ''num'' must be greater than 0 and less
than the number of tokens remaining in the tokenizer string.

</dl>
===Usage Notes===
<ul>
<li>For a new StringTokenizer instance, issuing NextToken, SkipTokens,
or [[FindToken (StringTokenizer function)|FindToken]] is required before
[[CurrentToken (StringTokenizer function)|CurrentToken]] may be issued without error.
</ul>
===Examples===

The following code fragment shows multiple SkipTokens calls.
The second call follows a move of the tokenizing position
to the middle of the initial token in the tokenizer string.

begin
%tok is object stringTokenizer
%tok = new
%tok:string = 'Here is the skipTokens example.'
[[Targeted Text statements|printText]] {~} = '{%tok:string}'
%tok:skipTokens(3)
printText {~} = {%tok:currentToken}
printText {~} = {%tok:nextPosition}
%tok:nextPosition = 3
%tok:skipTokens(1)
printText {~} = {%tok:currentToken}
printText {~} = {%tok:nextPosition}
%tok:skipTokens(4)
printText {~} = {%tok:currentToken}
end

The result is:
<pre>
%tok:string = 'Here is the skipTokens example.'
%tok:currentToken = the
%tok:nextPosition = 12
%tok:currentToken = re
%tok:nextPosition = 5
%tok:currentToken = example.
%tok:nextPosition = 32
</pre>

RestOfString (StringTokenizer function)

2010-12-16T03:25:59Z

198.242.244.228:

Non-tokenized substring that follows the next token
[[Category:StringTokenizer methods|RestOfString function]]
[[Category:System methods]]



RestOfString is a member of the [[StringTokenizer class]].


This method returns the current non-tokenized substring of the tokenizer string.
This substring starts from the position after the next token ([[NextPosition (StringTokenizer property)|NextPosition]]),
and it includes the remaining characters going forward to the end of the string.
===Syntax===
%string = %tok:RestOfString
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to receive the characters that follow the next token.
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>The RestOfString operation does not advance the NextPosition value.
<li>RestOfString depends only on the value of the most recent NextPosition
or [[NextChar (StringTokenizer function)|NextChar]] call; it
is independent of the values of [[CurrentToken (StringTokenizer function)|CurrentToken]]
and [[CurrentTokenPosition (StringTokenizer property)|CurrentTokenPosition]].
<li>If the value of [[AtEnd (StringTokenizer function)|AtEnd]] is <tt>True</tt>,
issuing RestOfString is '''not''' an error but returns a null string.
<li>The [[StartOfString (StringTokenizer function)|StartOfString]] function
returns the substring of the tokenizer string that precedes the first character of
the current token (this first-character position is returned by CurrentTokenPosition).
</ul>
===Examples===

The following request fragment shows that consecutive RestOfString statements
return the same value so do not advance the NextPosition value, and it shows that
RestOfString returns a null string
if issued when the position is at the end of the string:
<pre>
%tok:string = 'restOfString example'
printText {%tok:nextToken}
printText {%tok:restOfString}
printText {%tok:restOfString}
printText {%tok:nextToken}
printText {%tok:restOfString} \\
</pre>

The result is:
<pre>
restOfString
example
example
example
\\
</pre>

RemoveQuotes (StringTokenizer property)

2010-12-16T03:21:33Z

198.242.244.228:

Whether to remove quote characters from returned values
[[Category:StringTokenizer methods|RemoveQuotes property]]
[[Category:System methods]]



RemoveQuotes is a member of the [[StringTokenizer class]].


This readWrite property returns or sets the Boolean value that indicates
whether to remove the quote characters from the returned values of quoted tokens.
The default value for a new tokenizer is <tt>True</tt>.

===Syntax===
%bool = %tok:RemoveQuotes

%tok:RemoveQuotes = %bool
===Syntax terms===
<dl>
<dt>%bool
<dd>An enumeration object of type Boolean to contain the returned value of
RemoveQuotes.
For more information about Boolean enumerations, see [[Using Boolean enumerations]].
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>The [[Quotes (StringTokenizer property)|Quotes]] property returns or resets the values of the
current quotation characters.
The Quotes characters are initially settable in the
[[New (StringTokenizer constructor)|New]] call that creates the tokenizer instance.
</ul>
===Examples===

In the following request fragment, the first [[PeekToken (StringTokenizer function)|PeekToken]] call
returns <tt>quotes example</tt>, removing the double quotation marks
from the tokenizer string because that is the default behavior.
Then, after RemoveQuotes is reset to <tt>False</tt>, the next peekToken call
displays the returned token value with its quotation
marks (<tt>"quotes example"</tt>):
<pre>
...
%tok = new(Quotes='"')
%tok:string = '"quotes example"'
Print %tok:peekToken
%tok:removeQuotes = False
Print %tok:peekToken
...
</pre>

Quotes (StringTokenizer property)

2010-12-16T03:17:31Z

198.242.244.228:

Characters which are quotation characters
[[Category:StringTokenizer methods|Quotes property]]
[[Category:System methods]]



Quotes is a member of the [[StringTokenizer class]].


This readWrite property returns or sets
the characters that are recognized as quotation characters.
The text between each disjoint pair of identical quotation characters
(a “quoted region”) is treated as a single token, and any
delimiter characters (Quote, Space, or TokenChar)
within a quoted region are treated as non-delimiters.

===Syntax===
%chars = %tok:Quotes

%tok:Quotes = %chars
===Syntax terms===
<dl>
<dt>%chars
<dd>A string variable to contain the returned value of the current quotation
characters or to be set as the new value.
The default value for a new tokenizer is a null string.

If you are setting Quotes, each character in the string is a quotation
character — that is, you may not separate characters — and no character
may repeat (except for apostrophe, which may be doubled).
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>No character can be one of the Quotes characters and also be
one of either the Spaces characters or the TokenChars characters.
<li>The Quotes characters are initially settable in the [[New (StringTokenizer constructor)|New]] call that creates the tokenizer instance.
<li>A quotation character may participate in only one quoted region.
The first quotation character from the beginning of the tokenization string is
paired with the following identical quotation character in the string.
An unpaired quotation character causes a request cancellation.
<li>A quoted region is not affected by the [[TokensToLower (StringTokenizer property)|TokensToLower]]
and [[TokensToUpper (StringTokenizer property)|TokensToUpper]] properties.
<li>The [[RemoveQuotes (StringTokenizer property)|RemoveQuotes]] property controls whether
quotes are removed from the returned values of quoted tokens.
</ul>
===Examples===

The following request fragment employs two quotation characters, and
it uses the RemoveQuotes method to include the quotation characters
in the returned tokens:
<pre>
%tok = new(Quotes='"+')
%tok:string = '+This Quotes example+ is "trivial"'
%tok:removeQuotes = False
Print %tok:nextToken
Print %tok:nextToken
Print %tok:nextToken
</pre>

The result is:
<pre>
+This Quotes example+
is
"trivial"
</pre>

PeekToken (StringTokenizer function)

2010-12-16T03:13:42Z

198.242.244.228:

Value of next token in current tokenizer string, don't adva
[[Category:StringTokenizer methods|PeekToken function]]
[[Category:System methods]]



PeekToken is a member of the [[StringTokenizer class]].


This method returns the string value of
the next token forward in the current tokenizer string.
The next token parsing begins from the tokenizing position,
which is the value returned by [[NextPosition (StringTokenizer property)|NextPosition]].

When the next token is determined,
PeekToken does not advance the tokenizing position,
which remains what it was when PeekToken was called.

The delimiters that are involved in determining the tokens
are set initially by the [[New (StringTokenizer constructor)|New]] method
that instantiates the StringTokenizer object.
The delimiters can be modified by the [[Spaces (StringTokenizer property)|Spaces]],
[[Quotes (StringTokenizer property)|Quotes]], and [[TokenChars (StringTokenizer property)|TokenChars]] properties.
===Syntax===
%string = %tok:PeekToken
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to receive the value of the current token.
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>If the value of [[AtEnd (StringTokenizer function)|AtEnd]] is <tt>True</tt>,
issuing PeekToken is invalid and cancels the request.
<li>The [[NextToken (StringTokenizer function)|NextToken]] and [[FindToken (StringTokenizer function)|FindToken]]
functions return the next token and
advance the tokenizing position to the character or position following that token.
[[SkipTokens (StringTokenizer subroutine)|SkipTokens]] similarly advances the tokenizing position,
but it returns no value.

After a successful execution of NextToken, FindToken, or SkipTokens, the token
located becomes the current token.
This is not true for PeekToken: after it completes successfully, the token located
does ''not'' become the current token.
</ul>
===Examples===

The following sequence of printText statements display,
respectively, <tt>example</tt>, <tt>example</tt>, and <tt>example</tt>,
showing that the tokenizing position does not move after PeekToken completes:
<pre>
%tok:string = 'example of peekToken'
printText {%tok:peekToken}
printText {%tok:peekToken}
printText {%tok:peekToken}
</pre>

PeekChar (StringTokenizer function)

2010-12-16T03:09:04Z

198.242.244.228:

Value of character at current tokenizing position, don't adv
[[Category:StringTokenizer methods|PeekChar function]]
[[Category:System methods]]



PeekChar is a member of the [[StringTokenizer class]].


This method returns the value of
the character that is at the tokenizing position.
The tokenizing position is the value returned by [[NextPosition (StringTokenizer property)|NextPosition]].

After returning the character value, PeekChar does not advance the
tokenizing position,
which remains what it was when PeekChar was called.
===Syntax===
%string = %tok:PeekChar
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to receive the value of the character at the
tokenizing position.
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>If the value of [[AtEnd (StringTokenizer function)|AtEnd]] is <tt>True</tt>,
issuing PeekChar is invalid and cancels the request.
<li>The [[NextChar (StringTokenizer function)|NextChar]] function also returns the value of the
next character, but it advances the tokenizing position past the next character.
</ul>
===Examples===

The following sequence of printText statements display,
respectively, <tt>A</tt>, <tt>A</tt>, <tt>A</tt>, and <tt>n</tt>,
showing that the tokenizing position does not move after PeekChar completes,
PeekChar and NextChar return the same value, and NextChar does
advance the tokenizing position:
<pre>
%tok:string = 'An example of peekChar'
printText {%tok:peekChar}
printText {%tok:peekChar}
printText {%tok:nextChar}
printText {%tok:nextChar}
</pre>

NotAtEnd (StringTokenizer function)

2010-12-16T03:06:38Z

198.242.244.228:

Is present tokenizing position at end of tokenizer string?
[[Category:StringTokenizer methods|NotAtEnd function]]
[[Category:System methods]]



NotAtEnd is a member of the [[StringTokenizer class]].


This method returns a Boolean value that indicates whether or not
the present tokenizing position is after the last token in the tokenizer string.
The value is <tt>True</tt> if the position is '''not''' after the
last token; otherwise it is <tt>False</tt>.

The tokenizing position is given by [[NextPosition (StringTokenizer property)|NextPosition]].
===Syntax===
%bool = %tok:NotAtEnd
===Syntax terms===
<dl>
<dt>%bool
<dd>An enumeration object of type Boolean to contain the returned value
of NotAtEnd.

For more information about Boolean enumerations, see [[Using Boolean enumerations]].
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>The tokenizing position may be after the last token but
not necessarily at the end of the token string.
In the following code sequence, the return from NotAtEnd is <tt>True</tt>
and the position is not at the end of the string:
<pre>
%tok:string = 'example '
%tok:nextPosition = 10
%tok:notAtEnd
</pre>
<li>[[AtEnd (StringTokenizer function)|AtEnd]] is the reverse of NotAtEnd:
it returns a value of <tt>False</tt>
if the current tokenizing position is not at the end of the string.
</ul>

NextToken (StringTokenizer function)

2010-12-16T03:04:50Z

198.242.244.228:

Next token forward in current tokenizer string
[[Category:StringTokenizer methods|NextToken function]]
[[Category:System methods]]



NextToken is a member of the [[StringTokenizer class]].


This method returns the string value of
the next token forward in the current tokenizer string.
The next token parsing begins from the tokenizing position,
which is the value returned by [[NextPosition (StringTokenizer property)|NextPosition]].

When the next token is determined,
NextToken advances the tokenizing position
to the position following that token, then returns the token value.

The delimiters that are involved in determining the tokens
are set initially by the [[New (StringTokenizer constructor)|New]] method
that instantiates the StringTokenizer object.
The delimiters can be modified by the [[Spaces (StringTokenizer property)|Spaces]],
[[Quotes (StringTokenizer property)|Quotes]], and [[TokenChars (StringTokenizer property)|TokenChars]] properties.
===Syntax===
%string = %tok:NextToken([Skip=num])
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to receive the value of the current token.
<dt>%tok
<dd>A StringTokenizer object variable.
<dt>Skip= num
<dd>This name required numeric argument specifies a number of tokens
to skip before returning the next token.
The value of ''num'' must be greater than 0, and it must be
be less than the number of tokens remaining in the tokenizer string.
For an example, see item [[??]] refid=skipxmp..

The Skip parameter is available as of version 7.7 of the ''Sirius Mods''.

</dl>
===Usage Notes===
<ul>
<li>If the value of [[AtEnd (StringTokenizer function)|AtEnd]] is <tt>True</tt>,
issuing NextToken is invalid and cancels the request.
<li>For a new StringTokenizer instance, issuing NextToken,
[[FindToken (StringTokenizer function)|FindToken]], or [[SkipTokens (StringTokenizer subroutine)|SkipTokens]] is
required before [[CurrentToken (StringTokenizer function)|CurrentToken]] may be issued without error.
<li>The NextPosition [[NextPosition (StringTokenizer property)|NextPosition]] returns the position of
the character after the NextToken value.
Conversely, the determination of the NextToken value always starts at the
position given by NextPosition.
If you reset NextPosition, the value returned by your next NextToken call
will be based on the new NextPosition value.
<li>The FindToken and [[PeekToken (StringTokenizer function)|PeekToken]]
functions also return the next token.
Like NextToken, FindToken
advances the tokenizing position to the position following that token.
PeekToken, however, does not advance the tokenizing position.
</ul>
===Examples===
<ol>
<li>In the following request fragment, the printText statements display,
respectively, <tt>example</tt>, <tt>of</tt>, and <tt>nextToken</tt>.
<pre>
%tok is object stringTokenizer
%tok = new
%tok:string = 'example of nextToken'
printText {%tok:nextToken}
printText {%tok:nextToken}
printText {%tok:nextToken}
</pre>
 
<li>The following request fragment shows the effect of the Skip parameter:
<pre>
%tok:string = ' * 0 * 1 * 2 * 3 * 4 * 5 * 6 * 7'
repeat while %tok:notAtEnd
printtext {~} = '{%tok:nextToken(skip=1)}'
end repeat
</pre>

The result is:
<pre>
%tok:nextToken(skip=1) = '0'
%tok:nextToken(skip=1) = '1'
%tok:nextToken(skip=1) = '2'
%tok:nextToken(skip=1) = '3'
%tok:nextToken(skip=1) = '4'
%tok:nextToken(skip=1) = '5'
%tok:nextToken(skip=1) = '6'
%tok:nextToken(skip=1) = '7'
</pre>
</ol>

NextPosition (StringTokenizer property)

2010-12-16T02:54:33Z

198.242.244.228: /* Usage Notes */

Position of character at current tokenizing position
[[Category:StringTokenizer methods|NextPosition property]]
[[Category:System methods]]



NextPosition is a member of the [[StringTokenizer class]].


This readWrite property returns
or sets the tokenizing position, the position of the character
from which the parsing of the next token begins.
Unless you explicitly set it otherwise, the tokenizing position is
the position that follows the (last) character in the value returned by the
most recent call of [[NextToken (StringTokenizer function)|NextToken]],
[[FindToken (StringTokenizer function)|FindToken]], or [[NextChar (StringTokenizer function)|NextChar]].

===Syntax===
%pos = %tok:NextPosition

%tok:NextPosition = %pos
===Syntax terms===
<dl>
<dt>%pos
<dd>A numeric variable to contain the returned value of the current tokenizing
position.
The default value is <tt>1</tt>.
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>The [[CurrentTokenPosition (StringTokenizer property)|CurrentTokenPosition]] property returns the position of
the initial character in the current token.
Setting NextPosition does not advance the current token position.
<li>To advance the current token position a single character, you can use
[[NextChar (StringTokenizer function)|NextChar]].
If a NextChar function call immediately follows a NextPosition call,
NextChar identifies the character whose position is the value of NextPosition.
<li>If you reset NextPosition, the value returned by your next NextToken,
FindToken, [[SkipTokens (StringTokenizer subroutine)|SkipTokens]], or NextChar call will be based on the reset NextPosition value.

The NextPosition value is entirely independent from the CurrentTokenPosition value,
and vice versa.
Both properties can be set to arbitrary locations in the tokenization string.
</ul>

===Examples===

In the following request fragment, NextToken calls retrieve the first three
tokens in the string, then NextPosition is explicitly set back to the second
character in the string, from which the next token is determined:
<pre>
%tok is object stringTokenizer
%tok = new
%tok:string = 'This is the nextPosition example.'
printText {~} is {%tok:nextToken}
printText {~} is {%tok:nextToken}
printText {~} is {%tok:nextToken}
printText {~} is {%tok:nextPosition}
%tok:nextPosition = 2
printText {~} is {%tok:nextToken}
printText {~} is {%tok:nextPosition}
</pre>

The result is:
<pre>
%tok:nextToken is This
%tok:nextToken is is
%tok:nextToken is the
%tok:nextPosition is 12
%tok:nextToken is his
%tok:nextPosition is 5
</pre>

NextChar (StringTokenizer function)

2010-12-16T02:45:30Z

198.242.244.228:

Value of character at current tokenizing position
[[Category:StringTokenizer methods|NextChar function]]
[[Category:System methods]]



NextChar is a member of the [[StringTokenizer class]].


This method returns the value of the character that is at the tokenizing position,
and it advances the tokenizing position past that character to the next character.
The tokenizing position is the value returned by [[NextPosition (StringTokenizer property)|NextPosition]].
===Syntax===
%string = %tok:NextChar
===Syntax terms===
<dl>
<dt>%string
<dd>A string variable to receive the returned character value.
<dt>%tok
<dd>A StringTokenizer object variable.

</dl>
===Usage Notes===
<ul>
<li>If the value of [[AtEnd (StringTokenizer function)|AtEnd]] is <tt>True</tt>,
issuing NextChar is invalid and cancels the request.
<li>Once NextChar executes, the number of the resulting position
can be returned by NextPosition.
<li>The [[PeekChar (StringTokenizer function)|PeekChar]] function also returns the value of the
next character, but it does not advance the tokenizing position.
</ul>
===Examples===

The following sequence of printText statements display,
respectively, <tt>E</tt>, <tt>x</tt>, <tt>a</tt>, and <tt>a</tt>,
showing that the tokenizing position moves after NextChar completes
but not after PeekChar completes:
<pre>
%tok:string = 'Example of peekChar'
Print %tok:nextChar
Print %tok:nextChar
Print %tok:peekChar
Print %tok:peekChar
</pre>

New (StringTokenizer constructor)

2010-12-16T02:32:38Z

198.242.244.228:

Create new StringTokenizer object
[[Category:StringTokenizer methods|New constructor]]
[[Category:System methods]]



New is a member of the [[StringTokenizer class]].


This method returns a new instance of a StringTokenizer object.
It has three optional arguments that let you specify the delimiter characters
that determine the tokens in the string that is being tokenized.
===Syntax===
%tok = New( [TokenChars=chars] -
[, Spaces=chars] -
[, Quotes=chars] )
===Syntax terms===
<dl>
<dt>%tok
<dd>A StringTokenizer object variable to contain the new object instance.
<dt>TokenChars= chars
<dd>This name required string argument (TokenChars) is a set of single-character
token-delimiters (delimiters that are also tokens) that may be
separated by whitespace characters.

TokenChars is an optional argument that defaults to a null string.
<dt>Spaces= chars
<dd>This name required string argument (Spaces) is a set of “whitespace”
characters, that is, characters that separate tokens.
Each of these characters is
a “non-token delimiter,” a delimiter that is not itself a token.

Spaces is an optional argument that defaults to a blank character.
<dt>Quotes= chars
<dd>This name required string argument (Quotes) is a set of quotation characters.
The text between each disjoint pair of identical quotation characters (a “quoted
region”) is treated as a single token, and any delimiter characters (Quote,
Space, or TokenChar) within a quoted region are treated as non-delimiters.

Quotes is an optional argument that defaults to a null string.

</dl>
===Usage Notes===
<ul>
<li>A character may belong to at most one of the Spaces,
Quotes, or TokenChars sets of characters.
<li>If you are specifying Spaces, Quotes, or TokenChars,
each character in the string is a quotation
character — that is, you may not separate characters — and no character
may repeat (except for apostrophe, which may be doubled).
<li>A quoted region is not affected by the [[TokensToLower (StringTokenizer property)|TokensToLower]]
and [[TokensToUpper (StringTokenizer property)|TokensToUpper]] properties.
</ul>
===Examples===

In the following request, a variety of delimiters is on display:
<pre>
begin
%tok is object stringtokenizer

%tok = new(spaces=' ;', tokenchars='=_', quotes='"')
%tok:string = '--=_alternative 0016B5A2CA2574DD_=; -
Content-Type: text/plain; charset="US-ASCII"'

repeat while not %tok:atEnd
printText {~} is {%tok:nextToken}
end repeat
end
</pre>

The result is:
<pre>
%tok:nextToken is --
%tok:nextToken is =
%tok:nextToken is _
%tok:nextToken is alternative
%tok:nextToken is 0016B5A2CA2574DD
%tok:nextToken is _
%tok:nextToken is =
%tok:nextToken is Content-Type:
%tok:nextToken is text/plain
%tok:nextToken is charset
%tok:nextToken is =
%tok:nextToken is US-ASCII
</pre>

FindToken (StringTokenizer function)

2010-12-16T02:30:12Z

198.242.244.228:

Search for specified token
[[Category:StringTokenizer methods|FindToken function]]
[[Category:System methods]]



FindToken is a member of the [[StringTokenizer class]].


This callable method returns a Boolean value that indicates whether
it finds a specified token.
The value is <tt>True</tt> if the token is found;
otherwise it is <tt>False</tt>.
The search starts from the tokenizing position,
the value of which is returned by [[NextPosition (StringTokenizer property)|NextPosition]].

If the token is found, the position in
the string is advanced past the found token, and [[CurrentToken (StringTokenizer function)|CurrentToken]]
will return the found token.
Otherwise, [[AtEnd (StringTokenizer function)|AtEnd]] is set to <tt>True</tt>, and CurrentToken
is set to the last token at the end of the string.

The delimiters that are involved in determining the tokens
are set initially by the [[New (StringTokenizer constructor)|New]] constructor
that instantiates the StringTokenizer object.
The delimiters can be modified by the [[Spaces (StringTokenizer property)|Spaces]],
[[Quotes (StringTokenizer property)|Quotes]], and [[TokenChars (StringTokenizer property)|TokenChars]] properties.
===Syntax===
[%bool =] %tok:FindToken(string)
===Syntax terms===
<dl>
<dt>%bool
<dd>An enumeration object of type Boolean to contain the returned value
of FindToken.

For more information about Boolean enumerations, see [[Using Boolean enumerations]].
<dt>%tok
<dd>A StringTokenizer object variable.
<dt>string
<dd>A string or string variable that is the token to be located.

</dl>
===Usage Notes===
<ul>
<li>For a new StringTokenizer instance, issuing FindToken,
[[NextToken (StringTokenizer function)|NextToken]], or [[SkipTokens (StringTokenizer subroutine)|SkipTokens]] is
required before [[CurrentToken (StringTokenizer function)|CurrentToken]] may be issued without error.
<li>The NextToken and [[PeekToken (StringTokenizer function)|PeekToken]]
functions also return the next token.
Like FindToken, NextToken
advances the tokenizing position to the character or position following that token.
PeekToken, however, does not advance the tokenizing position.
</ul>
===Examples===

The following code fragment shows a FindToken call that locates its target,
then a FindToken call that does not locate its target:
<pre>
%tok = new(tokenchars='-')
%tok:string = 'Content-Transfer-Encoding: 7bit'
PrintText {~} is {%tok:findToken('Transfer')}
PrintText {~} is {%tok:findToken('8bit')}
PrintText {~} is {%tok:AtEnd}
PrintText {~} is {%tok:currentToken}
</pre>

The result is:
<pre>
%tok:findToken('Transfer') is True
%tok:findToken('8bit') is False
%tok:AtEnd is True
%tok:currentToken is 7bit
</pre>
'''Note:'''
Using the PrintText statement is described in [[Targeted Text statements|PrintText]].
Using PrintText to print an enumeration value is described in the first “Note” ??
in [[Enumerations#Using enumerations|Using enumerations]].