RegexSplit (Stringlist function)
Split a string onto a Stringlist using regex (Stringlist class)
This callable method repeatedly applies a regular expression, or "regex," to a given input string until it has tested the entire string. This splits the string into the substrings that are matched by the regex (the "separators") and the substrings that are not matched.
By default, the method saves each unmatched substring as a separate item in the Stringlist method object. The leftmost unmatched substring is the first item in the Stringlist, the next leftmost is the second item, and so on. A simple application of the method is to extract only the data items from a string of comma-separated data items. If the specified regex is a comma, each of the resulting Stringlist items will contain one of the data items. The Stringlist that is returned by a default invocation of RegexSplit will contain at least as many items as there are instances of matched substrings. Upon each match, the input string characters preceding the matched ones (and since the previous matched ones) are saved as a Stringlist item. If there are consecutive matching substrings (no unmatched characters between the matching ones), the corresponding Stringlist item for the second matching substring is empty. RegexSplit also has non-default alternatives that let you save the following in the Stringlist:
- only the matched substrings
- both the matched and unmatched substrings
- only the substrings that are matched by capturing groups in the specified regex
- the unmatched substrings and the substrings matched by capturing groups
Within a regex, characters enclosed by a pair of unescaped parentheses form a "subexpression". A subexpression is a capturing group if the opening parenthesis is not followed by a question mark (?). RegexSplit uses the rules of regular expression matching (information about which is provided in "Regex processing rules").
Specifying an invalid argument results in request cancellation.
[%number =] sl:RegexSplit( inString, regex, [Options= string], - [Status= %output], [Add= regexSplitOutputOptions]) Throws InvalidRegex
|%number||If specified, a numeric variable that is set to 0 if the regular expression was invalid or no match was found or some error occurred, or it is the number of items added to the method Stringlist sl.|
|sl||A Stringlist object.|
|inString||The input string, to which the regular expression regex is applied.|
|regex||A string that is interpreted as a regular expression and is applied in a matching operation to the inString argument.|
|Options||This is an optional, name required, parameter supplying a string of single letter options, which may be specified in uppercase or lowercase, in any combination, and blank separated or not as you prefer. For more information about these options, see Common regex options.|
|Status||The Status argument (name required) is optional; if specified, it is set to an integer code. These values are possible:
If you omit this argument and an invalid regex pattern was specified, an InvalidRegex exception is thrown under Sirius Mods 7.2 and later, and the request is cancelled; under earlier versions of the Sirius Mods. If you omit this argument and another negative Status value is to be returned, the request is cancelled.
|Add||The Add argument (name required) is one of the following RegexSplitOutputOptions enumeration values, which specify what substrings of the input string inString to store in the method Stringlist sl:
RegexSplit can throw the following exceptions under Sirius Mods 7.2 and later.
- If the regex parameter does not contain a valid regular expression. The exception object indicates the position of the character in the regex parameter where it was determined that the regular expression is invalid, and a description of the nature of the error.
- RegexSplit is available in Sirius Mods Version 7.0 and later.
- It is strongly recommended that you protect your environment from regular expression processing demands on PDL and STBL space by setting, say,
UTABLE LPDLST 3000and
UTABLE LSTBL 9000. See SOUL programming considerations.
- The Stringlist class RegexSplit function performs the same processing as the intrinsic String class RegexSplit function. The only differences are:
- The target Stringlist for the Stringlist class method is its method object, whereas it's the function output for the String class method.
- The Stringlist class method appends to the target Stringlist, the String class method creates a new Stringlist.
- The String class method has no Status parameter, the Stringlist class method does. The only way the String class method has of indicating an error is via an exception.
- Basically, RegexSplit divides a string into pieces, some of which match a regular expression and some of which don't. Although, you may be more interested in the pieces that are matched than the pieces that are unmatched, this documentation often refers to the matched pieces as "separators," which displays a bias towards the default paradigm of a comma-delimited list. In this case (which is equivalent to the method's
Add=Unmatchedoption), the regex identifies the "commas," and the method extracts the "list elements" that remain. In the algorithm for extracting the list items in the default case:
- The regex makes its initial match in the input string; this is the first "comma" / separator.
- The characters to the left of the matched substring (if none, then the null string) are the first "list element" unmatched piece.
- The regex finds its next match, the next separator. The substring between the final character in the previous match and the first character in the current match (if none, then the null string) becomes the second unmatched piece.
- When the regex fails to make a next match, the entire substring remaining after the last match (if none, then the null string) becomes the last unmatched piece.
Worth noting about this algorithm:
- The matching by the regex is not affected by any capturing strings in the regex. Capturing, per se, is not involved in the default case.
- A "comma" (separator) is always assumed to be preceded and followed by a "list element" (unmatched piece). Consequently, the extracted Stringlist often contains at least one null item. Only a comma-delimited list of the form
a,b,c(where there are no repeated, list-starting, or list-ending commas) results in a Stringlist with no nulls. For an example, see Example 1 below.
- There is always one more unmatched piece than matched, because the pieces must alternate (consecutive matched pieces are separated by an unmatched null), and they begin and end with an unmatched piece.
- If a RegexSplit regex contains multiple capturing groups and the
Add=Captureoption is used, each time the regex matches in the input string, the number of Stringlist items added is the number of capturing groups. For example, if you have three capturing groups and you specify the
Add=Capturedoption, the Stringlist item numbers always line up as follows:
1. First capturing group 2. Second capturing group 3. Third capturing group 4. First capturing group 5. Second capturing group 6. Third capturing group 7. First capturing group 8. Second capturing group 9. Third capturing group ...
If you are also capturing the non-matched pieces (
Add=CapturedAndUnmatched), here is the item order:
1. First non-matched piece 2. First capturing group 3. Second capturing group 4. Third capturing group 5. Second non-matched piece 6. First capturing group 7. Second capturing group 8. Third capturing group 9. Third non-matched piece 10. First capturing group 11. Second capturing group 12. Third capturing group ...
For a code example, see Example 3, below.
- If %number is 0, either regex did not match inString, or there was an error in the regex. The Status argument returns additional information: If it is negative, it indicates an error. If it is zero, it indicates there was no error, but the regex did not match.
- An empty item in the output Stringlist may represent consecutive separators in the input string (with
- RegexSplit might add items to the method Stringlist, then subsequently encounter a zero-length match (which is treated as an error; status is set to -6). In this case, the modified Stringlist is not returned to its former state. You are responsible for preserving the unmodified state of the Stringlist if you want to restore that state when handling the error case.
- For information about additional methods and $functions that support regular expressions, see "Regex Processing".
- This example demonstrates how RegexSplit operating in default mode against a simple comma-delimited list assigns items to the result Stringlist.
UTABLE LPDLST 3000 Begin %inStr longstring %regex longstring %sl object stringlist %i is float %sl = new %str = 'Barry,Mildred' %regex = ',' %sl:RegexSplit (%str, %regex) Print '%sl:Count is ' %sl:Count For %i from 1 to %sl:Count Print '%sl(item' With %i With ') is: ' %sl:Item(%i) End For End
For the input string
Barry,Mildredand for a comma (
,) as the regex, the result is:
%sl:Count is 2 %sl(item1) is: Barry %sl(item2) is: Mildred
For the input string
,Barry,Mildredand the same regex, the result includes a null first item:
%sl:Count is 3 %sl(item1) is: %sl(item2) is: Barry %sl(item3) is: Mildred
And similarly, the input string
Barry,Mildred,and the same regex produce a null third item; and the input string
,Barry,Mildred,and the same regex produce a null first item and a null fourth item.
- This example shows the utility of the
... %str = ' Barry Mildred Jack Faust ' %sl:RegexSplit(%str, ' +') Print '---------- Unmatched: ' %sl:Print %sl = New %c String Len 10 %c = '[' With $X2C('5F') With ' ]+' %sl:RegexSplit(%str, %c, Add=Matched) Print '---------- Matched: ' %sl:Print Print '----------' ...
The result shows that using the
Add=Matchedoption along with a regex that matches directly the data you want to extract is a successful alternative that also lets you avoid the nulls in the Stringlist output:
---------- Unmatched: Barry Mildred Jack Faust ---------- Matched: Barry Mildred Jack Faust ----------
- In the following example, the
Add=Captureoption is used with a regex that includes multiple capture groups to strip the labels but capture the data values of an input string. Using the CreateLines method as shown is a way to use RegexSplit against a Stringlist.
b %troops is object stringList %tokens is object stringList %troops = new text to %troops Name: Clegg Rank: Corporal Missing: Leg Name: Ryan Rank: Private Missing: Brothers Name: Bilko Rank: Sergeant Missing: Discipline end text %tokens = new %tokens:regexSplit(%troops:createLines, - 'Name: *(\S+)\s*Rank: (\S+)\s*Missing: *(\S+)', - add=captured) %tokens:print(,3) end
This is the result:
5 Clegg 8 Corporal 3 Leg 4 Ryan 7 Private 8 Brothers 5 Bilko 8 Sergeant 10 Discipline