Separators (StringTokenizer property): Difference between revisions
m (Automatically generated page update) |
mNo edit summary |
||
| (One intermediate revision by the same user not shown) | |||
| Line 1: | Line 1: | ||
{{Template:StringTokenizer:Separators subtitle}} | {{Template:StringTokenizer:Separators subtitle}} | ||
Separators provide a way to handle consecutive occurrences of the same token delimiter character, for example, in a comma-separated value (csv) file, where they indicate a missing value. | |||
<var>Separators</var> characters delimit, or separate, tokens as do <var>[[Spaces (StringTokenizer property)|Spaces]]</var> and <var>[[TokenChars (StringTokenizer property)|TokenChars]]</var> characters, but <var>Separators</var> characters differ in that they: | |||
<ul> | |||
<li>do not compress to a single separator (like <var>Spaces</var> characters) | |||
<li>are not themselves tokens (like <var>TokenChars</var> characters), so are not returned by repeated <var>[[NextToken (StringTokenizer function)|NextToken]]</var> calls that encounter consecutive <var>Separators</var> characters | |||
</ul> | |||
==Syntax== | ==Syntax== | ||
{{Template:StringTokenizer:Separators syntax}} | {{Template:StringTokenizer:Separators syntax}} | ||
===Syntax terms=== | ===Syntax terms=== | ||
<table class="syntaxTable"> | <table class="syntaxTable"> | ||
<tr><th>% | <tr><th>%currentString</th> | ||
<td>A string variable to contain the returned value of the current separator characters. | |||
Each character in the string is a separator.</td></tr> | |||
<tr><th>stringTokenizer</th> | <tr><th>stringTokenizer</th> | ||
<td>StringTokenizer object</td></tr> | <td>A <var>StringTokenizer</var> object.</td></tr> | ||
<tr><th>newString</th> | |||
<td>A string variable to contain the new <var>Separators</var> characters. Each character in the string is a separator, and no character may repeat (except for apostrophe, which may be doubled). | |||
</table> | </table> | ||
==Usage notes== | ==Usage notes== | ||
<ul> | |||
<li>Separators do '''not''' override explicitly defined <var>TokenChars</var> characters. If both separators and token characters are defined, all such characters act as token delimiters. | |||
<li>Separators override default and explicitly defined <var>Spaces</var> characters. | |||
For example, if the only change to the example above is that the tokenizer string is <code>"please, don't go"</code>, | |||
the result is: | |||
<p class="output"> %toke:nextToken = 'please' | |||
%toke:nextToken = 'don't go' | |||
</p> | |||
The blank after <code>don't</code> does not act as a token delimiter. | |||
</ul> | |||
==Examples== | ==Examples== | ||
The adjacent separators in the token string below are detected and returned as nulls by the <var>NextToken</var> method: | |||
<p class="code">b | |||
%toke is object StringTokenizer | |||
%toke = new(separators=',;') | |||
%toke:string = '0,1,2,,4,;6' | |||
repeat while %toke:notAtEnd | |||
printtext {~} = '{%toke:nextToken}' | |||
end repeat | |||
end | |||
</p> | |||
The result is: | |||
<p class="output">%toke:nextToken = '0' | |||
%toke:nextToken = '1' | |||
%toke:nextToken = '2' | |||
%toke:nextToken = <nowiki>''</nowiki> | |||
%toke:nextToken = '4' | |||
%toke:nextToken = <nowiki>''</nowiki> | |||
%toke:nextToken = '6' | |||
</p> | |||
==See also== | ==See also== | ||
{{Template:StringTokenizer:Separators footer}} | {{Template:StringTokenizer:Separators footer}} | ||
Latest revision as of 20:33, 5 July 2012
Characters to be interpreted as token separators (StringTokenizer class)
[Introduced in Sirius Mods 7.8]
Separators provide a way to handle consecutive occurrences of the same token delimiter character, for example, in a comma-separated value (csv) file, where they indicate a missing value.
Separators characters delimit, or separate, tokens as do Spaces and TokenChars characters, but Separators characters differ in that they:
- do not compress to a single separator (like Spaces characters)
- are not themselves tokens (like TokenChars characters), so are not returned by repeated NextToken calls that encounter consecutive Separators characters
Syntax
%currentString = stringTokenizer:Separators stringTokenizer:Separators = newString
Syntax terms
| %currentString | A string variable to contain the returned value of the current separator characters. Each character in the string is a separator. |
|---|---|
| stringTokenizer | A StringTokenizer object. |
| newString | A string variable to contain the new Separators characters. Each character in the string is a separator, and no character may repeat (except for apostrophe, which may be doubled). |
Usage notes
- Separators do not override explicitly defined TokenChars characters. If both separators and token characters are defined, all such characters act as token delimiters.
- Separators override default and explicitly defined Spaces characters.
For example, if the only change to the example above is that the tokenizer string is
"please, don't go", the result is:%toke:nextToken = 'please' %toke:nextToken = 'don't go'
The blank after
don'tdoes not act as a token delimiter.
Examples
The adjacent separators in the token string below are detected and returned as nulls by the NextToken method:
b %toke is object StringTokenizer %toke = new(separators=',;') %toke:string = '0,1,2,,4,;6' repeat while %toke:notAtEnd printtext {~} = '{%toke:nextToken}' end repeat end
The result is:
%toke:nextToken = '0' %toke:nextToken = '1' %toke:nextToken = '2' %toke:nextToken = '' %toke:nextToken = '4' %toke:nextToken = '' %toke:nextToken = '6'