Spaces (StringTokenizer property): Difference between revisions
mNo edit summary |
mNo edit summary |
||
| Line 7: | Line 7: | ||
You can specify "token delimiters," which are delimiters that are also tokens, | You can specify "token delimiters," which are delimiters that are also tokens, | ||
using the [[TokenChars (StringTokenizer property)|TokenChars]] property or the [[New (StringTokenizer constructor)|New]] constructor TokenChars parameter. | using the <var>[[TokenChars (StringTokenizer property)|TokenChars]]</var> property or the <var>[[New (StringTokenizer constructor)|New]]</var> constructor <var>TokenChars</var> parameter. | ||
One or multiple consecutive whitespace characters mark the end or beginning | One or multiple consecutive whitespace characters mark the end or beginning | ||
| Line 15: | Line 15: | ||
==Syntax== | ==Syntax== | ||
{{Template:StringTokenizer:Spaces syntax}} | {{Template:StringTokenizer:Spaces syntax}} | ||
===Syntax terms=== | ===Syntax terms=== | ||
<table class="syntaxTable"> | <table class="syntaxTable"> | ||
<tr><th>%currentString</th> | <tr><th>%currentString</th> | ||
<td>A string variable to contain the returned value of the current whitespace characters | <td>A string variable to contain the returned value of the current whitespace characters. Each character in the string is a space character. </td></tr> | ||
<tr><th>stringTokenizer</th> | <tr><th>stringTokenizer</th> | ||
<td>A <var>StringTokenizer</var> object expression.</td></tr> | <td>A <var>StringTokenizer</var> object expression.</td></tr> | ||
<tr><th>newString</th> | <tr><th>newString</th> | ||
<td>The String value to assign <var class="term">stringTokenizer</var>'s <var>Spaces</var> property.</td></tr> | <td>The <var>String</var> value to assign <var class="term">stringTokenizer</var>'s <var>Spaces</var> property. Each character in the string is a space character — that is, you may not separate characters — and no character may repeat (except for apostrophe, which may be doubled).</td></tr> | ||
</table> | </table> | ||
==Usage notes== | ==Usage notes== | ||
<ul> | <ul> | ||
<li>The <var>Spaces</var> characters are initially settable in the [[New (StringTokenizer constructor)|New]] call that creates the tokenizer instance. | <li>The <var>Spaces</var> characters are initially settable in the <var>[[New (StringTokenizer constructor)|New]]</var> call that creates the tokenizer instance. The default value for a new tokenizer instance is a blank character. | ||
<li>No character can be one of the <var>Spaces</var> characters and also | <li>No character can be one of the <var>Spaces</var> characters and also | ||
one of | one of the <var>[[Quotes (StringTokenizer property)|Quotes]]</var>, <var>TokenChars</var>, or <var>[[Separators (StringTokenizer property)|Separators]]</var> characters. | ||
</ul> | </ul> | ||
==Examples== | ==Examples== | ||
In the following example, a <var>StringTokenizer</var> object is instantiated | In the following example, a <var>StringTokenizer</var> object is instantiated | ||
with the default parameter settings for the New constructor. | with the default parameter settings for the <var>New</var> constructor. | ||
Then the input string's tokens are returned in sequence: | Then the input string's tokens are returned in sequence: | ||
<p class="code">%tok is object stringtokenizer | <p class="code">%tok is object stringtokenizer | ||
Latest revision as of 01:12, 6 July 2012
Characters to be interpreted as token-delimiting whitespace (StringTokenizer class)
This readWrite property returns or sets the characters that are recognized as
"whitespace" characters, that is, as characters that separate tokens.
Such a character is also called a "non-token delimiter,"
because it is a delimiter that is not itself a token.
You can specify "token delimiters," which are delimiters that are also tokens, using the TokenChars property or the New constructor TokenChars parameter.
One or multiple consecutive whitespace characters mark the end or beginning of a token, except when these spaces are part of a quoted region. In that case, the spaces become non-delimiters.
Syntax
%currentString = stringTokenizer:Spaces stringTokenizer:Spaces = newString
Syntax terms
| %currentString | A string variable to contain the returned value of the current whitespace characters. Each character in the string is a space character. |
|---|---|
| stringTokenizer | A StringTokenizer object expression. |
| newString | The String value to assign stringTokenizer's Spaces property. Each character in the string is a space character — that is, you may not separate characters — and no character may repeat (except for apostrophe, which may be doubled). |
Usage notes
- The Spaces characters are initially settable in the New call that creates the tokenizer instance. The default value for a new tokenizer instance is a blank character.
- No character can be one of the Spaces characters and also one of the Quotes, TokenChars, or Separators characters.
Examples
In the following example, a StringTokenizer object is instantiated with the default parameter settings for the New constructor. Then the input string's tokens are returned in sequence:
%tok is object stringtokenizer %tok = new %tok:string = 'Bad,F ?3mO ,7,{x Z}' repeat while not %tok:atEnd printText {~} is {%tok:nextToken} end repeat
The resulting tokens, which were delimited by the default space character (blank), are:
%tok:nextToken is Bad,F %tok:nextToken is ?3mO %tok:nextToken is ,7,{x %tok:nextToken is Z}
If instead the tokenizer object is instantiated with a comma as the space character
(%tok = new(Spaces=',')), the resulting tokens are:
%tok:nextToken is Bad %tok:nextToken is F ?3mO %tok:nextToken is 7 %tok:nextToken is {x Z}