Regex class - Revision history

ATitelbaum at 20:05, 25 March 2022

2022-03-25T20:05:54Z

← Older revision		Revision as of 20:05, 25 March 2022
Line 15:		Line 15:
	the regular expression in the <var>RegexLocate</var> call would be parsed and compiled in every iteration of the loop. As one might guess, this adds a lot of unnecessary overhead. In Model 204 7.9 and later, the regular expression would be compiled at the time the surrounding SOUL code would be compiled so no compilation would be necessary at runtime. The Model 204 regular expression engine was also rewritten in 7.9 to make it significantly more efficient so code like that above would be significantly more efficient in Model 204 7.9 and later.		the regular expression in the <var>RegexLocate</var> call would be parsed and compiled in every iteration of the loop. As one might guess, this adds a lot of unnecessary overhead. In Model 204 7.9 and later, the regular expression would be compiled at the time the surrounding SOUL code would be compiled so no compilation would be necessary at runtime. The Model 204 regular expression engine was also rewritten in 7.9 to make it significantly more efficient so code like that above would be significantly more efficient in Model 204 7.9 and later.

	For most purposes, these improvements can be taken advantage of without any effort. However, as one might guess, a regular expression can only be compiled at SOUL compile-time if the regular expression is a literal. Otherwise how could the compiler know what to compile? There are cases, however, where a regular expression is only available at runtime. A classic example is in a program that allows end users to type in arbitrary regular expressions for matching against a set of values. In such a case, it might be useful to be able to compile the user-specified expression once and then apply it to a variety of values.		For most purposes, these improvements can be taken advantage of without any effort. However, as one might guess, a regular expression can only be compiled at SOUL compile-time if the regular expression is a literal. Otherwise, how could the compiler know what to compile? There are cases, however, where a regular expression is only available at runtime. A classic example is in a program that allows end users to type in arbitrary regular expressions for matching against a set of values. In such a case, it might be useful to be able to compile the user-specified expression once and then apply it to a variety of values.

	It might also be useful to use the same regular expression in multiple places or to even create a system or subsystem global that contains a regular expression. Or one might simply want to separate the regular expression creation from the code that uses it for code-readability or maintenance reasons.		It might also be useful to use the same regular expression in multiple places or to even create a system or subsystem global that contains a regular expression. Or one might simply want to separate the regular expression creation from the code that uses it for code-readability or maintenance reasons.
Line 45:		Line 45:
	%regex = new("foo.*?bar":u)		%regex = new("foo.*?bar":u)
	</p>		</p>
	the first constructor call would compile an EBCDIC regular expression and the second Unicode. While, in this case, the Unicode regular expression contains no Unicode characters that can't be converted to EBCDIC, the difference is still significant. If the first value of <var>%regex</var> ~~were~~ run against a Unicode string, that string would first have to be translated to EBCDIC so might encounter a translation error and, in any case, would suffer the extra overhead of character translation. ~~Simiarly~~, if the second value of <var>%regex</var> ~~were~~ run against an EBCDIC string, the EBCDIC string would first have to be translated to Unicode, incurring extra overhead.		the first constructor call would compile an EBCDIC regular expression and the second Unicode. While, in this case, the Unicode regular expression contains no Unicode characters that can't be converted to EBCDIC, the difference is still significant. If the first value of <var>%regex</var> was run against a Unicode string, that string would first have to be translated to EBCDIC so might encounter a translation error and, in any case, would suffer the extra overhead of character translation. Similarly, if the second value of <var>%regex</var> was run against an EBCDIC string, the EBCDIC string would first have to be translated to Unicode, incurring extra overhead.

	While the distinction between EBCDIC and Unicode input parameters can be handled at runtime, method output types are determined at compile-time so any <var>Regex</var> method that returns a string, must have Unicode and non-Unicode versions. For example, the <var>Replace</var> function returns a string, so there is a <var>ReplaceUnicode</var> function that should be used when the <var>Regex</var> object contains Unicode. The <var>Regex</var> <var>IsUnicode</var> method can be used to determine ~~whether or not~~ a <var>Regex</var> object was created for Unicode strings.		While the distinction between EBCDIC and Unicode input parameters can be handled at runtime, method output types are determined at compile-time so any <var>Regex</var> method that returns a string, must have Unicode and non-Unicode versions. For example, the <var>Replace</var> function returns a string, so there is a <var>ReplaceUnicode</var> function that should be used when the <var>Regex</var> object contains Unicode. The <var>Regex</var> <var>IsUnicode</var> method can be used to determine if a <var>Regex</var> object was created for Unicode strings.

	In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.		In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.
Line 77:		Line 77:
	%num = %regex:locate(%stringlist, %num)		%num = %regex:locate(%stringlist, %num)
	if %num eq 0 then loop end; end if		if %num eq 0 then loop end; end if
	%stringlist:~~relace~~(%num, %regex:replace(%stringlist(%num))		%stringlist:replace(%num, %regex:replace(%stringlist(%num))
	end repeat		end repeat
	</p>		</p>
Line 97:		Line 97:
	The value of the <var>REGXSTBL</var> parameter can be increased if this is a problem though, since the parameter is a user0 parameter only, it requires a recycle of the Online to make the change. If <var>REGXSTBL</var> is increased, STBL requirements will also increase for every request that uses <var>Regex</var> objects.		The value of the <var>REGXSTBL</var> parameter can be increased if this is a problem though, since the parameter is a user0 parameter only, it requires a recycle of the Online to make the change. If <var>REGXSTBL</var> is increased, STBL requirements will also increase for every request that uses <var>Regex</var> objects.

	Note that the default value of <var>REGXSTBL</var> is 2048 which is probably large enough to ~~accomodate~~ almost any regular expression one is likely to encounter in the "real world".		Note that the default value of <var>REGXSTBL</var> is 2048 which is probably large enough to accommodate almost any regular expression one is likely to encounter in the "real world".

	==List of Regex methods==		==List of Regex methods==

Alex: /* Regular expression size limits */

2022-03-24T21:56:21Z

Regular expression size limits

← Older revision		Revision as of 21:56, 24 March 2022
Line 96:		Line 96:

	The value of the <var>REGXSTBL</var> parameter can be increased if this is a problem though, since the parameter is a user0 parameter only, it requires a recycle of the Online to make the change. If <var>REGXSTBL</var> is increased, STBL requirements will also increase for every request that uses <var>Regex</var> objects.		The value of the <var>REGXSTBL</var> parameter can be increased if this is a problem though, since the parameter is a user0 parameter only, it requires a recycle of the Online to make the change. If <var>REGXSTBL</var> is increased, STBL requirements will also increase for every request that uses <var>Regex</var> objects.

			Note that the default value of <var>REGXSTBL</var> is 2048 which is probably large enough to accomodate almost any regular expression one is likely to encounter in the "real world".

	==List of Regex methods==		==List of Regex methods==

Alex at 21:54, 24 March 2022

2022-03-24T21:54:26Z

← Older revision		Revision as of 21:54, 24 March 2022
Line 1:		Line 1:
	The <var>Regex</var> class provides a facility that allows reuse of runtime compiled regular expressions. It was first available in Model 204 version 7.9.		The <var>Regex</var> class provides a facility that allows reuse of runtime compiled regular expressions. It was first available in Model 204 version 7.9.


	==Regex usage==		==Regex usage==
	Before ~~model~~ 204 7.9, every regular expression function would recompile the regular expression specified as a string parameter on each call. So, in a program like the following:		Before Model 204 7.9, every regular expression function would recompile the regular expression specified as a string parameter on each call. So, in a program like the following:
	<p class="code">%num is float		<p class="code">%num is float
	%stringlist is object stringlist		%stringlist is object stringlist
Line 92:		Line 91:
	<tr><th>ReplaceUnicode</th><td>The replacement Unicode specified in the constructor call.</td></tr>		<tr><th>ReplaceUnicode</th><td>The replacement Unicode specified in the constructor call.</td></tr>
	</table>		</table>

			==Regular expression size limits==
			When the [[New (Regex constructor)\|Regex constructor]] is invoked, the regular expression string and replacement string are compiled and the original strings and compilations are saved in the STBL part of the object. If the total size of these strings and compilations exceeds the value of the [[REGXSTBL parameter\|REGXSTBL system parameter]], the request is cancelled.

			The value of the <var>REGXSTBL</var> parameter can be increased if this is a problem though, since the parameter is a user0 parameter only, it requires a recycle of the Online to make the change. If <var>REGXSTBL</var> is increased, STBL requirements will also increase for every request that uses <var>Regex</var> objects.

	==List of Regex methods==		==List of Regex methods==

Alex at 15:18, 12 March 2022

2022-03-12T15:18:21Z

← Older revision		Revision as of 15:18, 12 March 2022
Line 51:		Line 51:

	In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.		In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.

			==Replacement==
			When using the <var>Replace</var> or <var>ReplaceUnicode</var> methods, one can specify a replacement string on the <var>Replace</var> or <var>ReplaceUnicode</var> method call. However, if one is always using the same replacement value for a <var>Regex</var> object, it can be slightly more efficient to specify the replacement string on the <var>Regex</var> object constructor call. For example:
			<p class="code">%regex is object regex
			%stringlist is object stringlist
			%userRegex is string len 255
			%userReplace is string len 255
			...
			%num = 1
			%regex = new(%userRegex, options="i")
			repeat forever
			%num = %regex:locate(%stringlist, %num)
			if %num eq 0 then loop end; end if
			%stringlist:replace(%i, %regex:replace(%num, %stringlist(%num), %userReplace)
			end repeat
			</p>
			could be done more efficiently as:
			<p class="code">%regex is object regex
			%stringlist is object stringlist
			%userRegex is string len 255
			%userReplace is string len 255
			...
			%num = 1
			%regex = new(%userRegex, replace=%userReplace, options="i")
			repeat forever
			%num = %regex:locate(%stringlist, %num)
			if %num eq 0 then loop end; end if
			%stringlist:relace(%num, %regex:replace(%stringlist(%num))
			end repeat
			</p>

	==Introspection==		==Introspection==

Alex at 15:09, 12 March 2022

2022-03-12T15:09:30Z

← Older revision		Revision as of 15:09, 12 March 2022
Line 51:		Line 51:

	In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.		In addition, some operations don't make sense for Unicode <var>Regex</var> objects. For example, there is no Unicode <var>Stringlist</var> object so the <var>Locate</var> function is not allowed with Unicode <var>Regex</var> objects.

			==Introspection==
			The <var>Regex</var> class has several methods that allow a <var>Regex</var> object to describe itself. This can be useful when looking at a <var>Regex</var> object in a <var>SirFact</var> dump or when using the debugger. The introspection methods are:
			<table>
			<tr><th>IsUnicode</th><td>A boolean value indicating whether or not the object is for Unicode.</td></tr>
			<tr><th>Options</th><td>The <var>Options</var> parameter value specified in the constructor call.</td></tr>
			<tr><th>RegexString</th><td>The regular expression specified in the constructor call.</td></tr>
			<tr><th>RegexUnicode</th><td>The regular expression Unicode specified in the constructor call.</td></tr>
			<tr><th>ReplaceString</th><td>The replacement specified in the constructor call.</td></tr>
			<tr><th>ReplaceUnicode</th><td>The replacement Unicode specified in the constructor call.</td></tr>
			</table>

	==List of Regex methods==		==List of Regex methods==

Alex at 14:42, 12 March 2022

2022-03-12T14:42:08Z

@@ Line 35: / Line 35: @@
 </p>
 In this case, the user-specified regular expression is only compiled once in the <var>Regex</var> constructor call (<var>new</var>) and reused multiple times after that, for optimal efficiency.
 ==List of Regex methods==

Alex: Created page with "The `Regex` class provides a facility that allows reuse of runtime compiled regular expressions. It was first available in Model 204 version 7.9. ==Regex usage== Be..."

2022-03-12T14:21:11Z

Created page with "The <var>Regex</var> class provides a facility that allows reuse of runtime compiled regular expressions. It was first available in Model 204 version 7.9. ==Regex usage== Be..."

New page

The <var>Regex</var> class provides a facility that allows reuse of runtime compiled regular expressions. It was first available in Model 204 version 7.9.

==Regex usage==
Before model 204 7.9, every regular expression function would recompile the regular expression specified as a string parameter on each call. So, in a program like the following:
<p class="code">%num is float
%stringlist is object stringlist
...
%num = 1
repeat forever
%num = %stringlist:regexLocate("(%[a-z_][a-z_.0-9]*) *=.*?\1", options="i")
if %num eq 0 then loop end; end if
...
end repeat
</p>
the regular expression in the <var>RegexLocate</var> call would be parsed and compiled in every iteration of the loop. As one might guess, this adds a lot of unnecessary overhead. In Model 204 7.9 and later, the regular expression would be compiled at the time the surrounding SOUL code would be compiled so no compilation would be necessary at runtime. The Model 204 regular expression engine was also rewritten in 7.9 to make it significantly more efficient so code like that above would be significantly more efficient in Model 204 7.9 and later.

For most purposes, these improvements can be taken advantage of without any effort. However, as one might guess, a regular expression can only be compiled at SOUL compile-time if the regular expression is a literal. Otherwise how could the compiler know what to compile? There are cases, however, where a regular expression is only available at runtime. A classic example is in a program that allows end users to type in arbitrary regular expressions for matching against a set of values. In such a case, it might be useful to be able to compile the user-specified expression once and then apply it to a variety of values.

It might also be useful to use the same regular expression in multiple places or to even create a system or subsystem global that contains a regular expression. Or one might simply want to separate the regular expression creation from the code that uses it for code-readability or maintenance reasons.

It is for cases like these that the <var>Regex</var> class was created. To illustrate how the <var>Regex</var> class might be used, consider the above example where user-specified regular expression is in string variable <var>%userRegex</var>. In that case, the above code can be written as:
<p class="code">%num is float
%regex is object regex
%stringlist is object stringlist
%userRegex is string len 255
...
%num = 1
%regex = new(%userRegex, options="i")
repeat forever
%num = %regex:locate(%stringlist, %num)
if %num eq 0 then loop end; end if
...
end repeat
</p>
In this case, the user-specified regular expression is only compiled once in the <var>Regex</var> constructor call (<var>new</var>) and reused multiple times after that, for optimal efficiency.

==List of Regex methods==
The individual <var>Regex</var> methods are summarized in [[List of Regex methods]].

[[Category:Regular expression processing]]