HTTP Helper

The HTTP Helper is a high level, object oriented interface to client sockets that permits you to write a User Language HTTP client without knowledge of socket level programming or the format of HTTP requests and responses.

The HTTP Helper consists of two object classes: HttpRequest and HttpResponse.

Note: To use the HTTP Helper objects, you must have licensed Janus Sockets and Janus SOAP. The Sirius object-oriented alternative to $functions is designed for Janus SOAP applications only, and information about using Sirius objects is provided in "Janus SOAP User Language Interface".

Feature summary

The following capabilities are provided by the HTTP Helper:

• Processing GET and POST HTTP requests, including XML support.
• Setting form fields and HTTP request headers and posting XML content.
• Sending arbitrary HTTP POST data (using a Longstring).
• Browser-like handling of file uploads using Multipart Form encoding
• After a request, accessing the HTTP Status line, the response headers, and the content.
• Accessing the content in its raw form and parsed into lines, or deserializing returned XML into an XmlDoc object.
• Optional automatic handling of redirects that a Get or Post method call receives from the server
• Using HTTPS, Secure Socket Layer (SSL) HTTP, as well as standard HTTP.
• Using HTTP Authentication to access password-protected URLs.
• Automatic and transparent translation from ASCII to and from EBCDIC.
• Using proxy servers.
• Using HTTP Version 1.1 or 1.0. Version 1.1 is the default.

Keep-Alive support

The client port definition parameter KEEPALIVE, introduced in Sirius Mods version 6.8, tells Janus Sockets to keep an HTTP connection to a web server open for a certain number of seconds so Janus Sockets can send another request on the same connection. This can reduce network traffic and, more significantly, HTTP request latency. Both of these benefits are magnified for SSL connections, where each HTTP request requires a TCP/IP and SSL connection-establishment handshake.

The Keepalive parameter must be followed by a single number between 1 and 32767 that indicates the number of seconds a TCP/IP connection is to be held open after an HTTP request on that connection. For the connection to be held open, the web server must indicate that it supports HTTP keep-alives. Most modern web servers can take advantage of keep-alives.

A keep-alive TCP/IP connection uses a Janus Sockets thread, so it is counted against both the maximum connections for a port and against a site's licensed-thread limits. Since a Janus Sockets application is often used to communicate with only a single or handful of web servers, this should not be a problem.

Keep-alives are highly specific to the HTTP protocol (there is a totally different keep-alive feature at the TCP protocol level with which HTTP keep-alives should not be confused), so Janus Sockets can only take advantage of them when it “knows” that HTTP is being used. This is only so when HTTP Helper objects (HttpRequest and HttpResponse) are used.

For an HttpRequest object Get or Post, if keep-alives are being used for a port (KEEPALIVE is non-zero on the port definition), Janus Sockets seeks an idle (keep-alive) connection for the target IP address. If one is found, that connection is used, avoiding connection establishment overhead. If none is found, a new connection is established. After the request is completed, rather than breaking the connection, it is held open for the KEEPALIVE timeout period. This approach reduces connection establishment overhead while in no way reducing parallelism — if a keep-alive connection is in use by one thread, another connection is used; if no unused ones are available, a new one is established.

There may be a timing problem with a keep-alive connection if a request tries to use a keep-alive connection just as the target server is closing the connection (most likely because the server's keep-alive time limit was exceeded). In such a case, the client side must try to use a different connection to the target server or try to establish a new one. This retry functionality is handled automatically and transparently by Janus Sockets. In fact, the use of keep-alives for a Janus Sockets HTTP client application should have no effect on application functionality.

The HttpRequest class

HttpRequest is the object with which you construct and issue an HTTP request to an HTTP server.

To construct a request, methods are provided, for example, for:

• Setting and inspecting the request's URL — specifying the entire URL (URL method) or using individual methods to build the URL's constituent parts (TCP/IP protocol, target host, target port, target page, respectively with the Protocol, Host, Port, and Page methods)
• Sending HTTP-form field and file data (AddField, LineEnd, MultiPartFormEncoding)
• Adjusting HTTP header fields (AddHeader, AutoSendXmlContentType)
• Providin Post data (AddLongstring, AddXml)
• Proxy server (Proxy, RemoveProxy)
• HTTP version
• Redirection handling , request-for-authentication handling (MaxRedirects, SetAuthentication, RemoveAuthentication)

To issue a request, HTTP method operations are provided by the Get, Post, and Send methods, which return an instance of the HttpResponse object. The Timeout property sets a maximum wait for these operations to complete, while Print provides a report of the request for debugging. FieldCount and HeaderCount return counts of the request's number of form fields and headers.

The HttpResponse class

HttpResponse is an object that encapsulates the items returned from an HTTP request. When you call a Get, Post, or Send method in an HttpRequest, an instance of HttpResponse is created, populated, and returned. HttpResponse methods return the HTTP status code/message, the document itself, and any HTTP response headers.

Although a New method exists to instantiate an HttpResponse object, the Get, Post, and Send methods of the HttpRequest class instantiate and return an instance of an HttpResponse object. HttpResponse methods access what was returned from the server.

The HttpResponse methods are of three principal types:

Status reporting

These methods take no parameters and let you check the results of the Get, Post, or Send operation most recently performed.

• The StatusLine method returns the entire HTTP status line, which consists of a code, a blank, and a message (for example, "200 OK").
• Code returns only the code portion of the status line (for example, "200").
• Message returns only the message portion of the status line (for example, "OK").
• Success returns 1 (true) if the HTTP status var is in the 200s (success range), and 0 (false) otherwise. If Success is true, content was retrieved and is available via the Content and ParseXml methods.
• URL returns the value of the URL from which the Get, Post, or Send response is obtained.

Accessing the returned document

These methods let you access the document/content from Get, Post, and Send operations. You call them if the Success method indicates a successful HTTP transaction has occurred.

• Content returns the entire document into a longstring with no parsing.
• ContentToStringlist returns the entire document into a Stringlist.
• ParseXML deserializes the returned data (assumed to be XML) into the XmlDoc that is passed.

Response header

These methods allow the retrieval of HTTP response headers returned by an HTTP request.

• HeaderCount returns the number of headers returned by the server, or the count of the number of occurrences of a particular header.
• HeaderNames returns a Stringlist object listing the names of the HTTP response headers returned.
• HeaderValue returns the value of the header whose name is passed.

Examples

Examples follow in which HttpRequest and HttpResponse objects are used to send a request to a server and to process the document the server sends in response. The first example uses HTTP GET; the second uses HTTP PUT. Not shown is the Janus Sockets CLSOCK port definition and CLSOCK ALLOW rules for the socket (“SOCKIT2ME”) named in the examples.

HTTP GET example

In this example, an XML document is fetched using HTTP GET.

Begin Variables are undefined %httpreq object httpRequest %httpresp object httpResponse %mydoc object XmlDoc %rc float %httpreq = new %mydoc = new * Identify the target URL * (Prepare the request) %httpreq:URL = 'foo.com/bar' * Send request to server with HTTP GET %httpresp = %httpreq:Get('SOCKIT2ME') * Check the status and process the results returned If ( %httpresp:Success ) then %rc = %httpresp:ParseXML( %mydoc ) * the whole document is now in %mydoc * do whatever .... End If End

HTTP POST example

This example simulates a form submission using HTTP POST.

Begin Variables are undefined %httpreq object httpRequest %httpresp object httpResponse %mydoc is longstring %httpreq = new * Identify the target URL * (Prepare the request) %httpreq:URL = 'foo.com/bin/lookup' * Set the form fields %httpreq:AddField('FirstName','Moe') %httpreq:AddField('LastName','Howard') * Send request to server with HTTP POST * Do Not cancel if comm or socket error %httpresp = %httpreq:Post( 'SOCKIT2ME', 0 ) * Check the status and process the results returned If %httpresp eq null then Print 'No Connection: ' $STATUSD ElseIf (%httpresp:Success) eq 0 then Print 'ErrInfo: ' %httpresp:StatusLine Else %mydoc = %httpresp:Content * the whole document is now in %mydoc * and can be processed as appropriate End If End