Uploading files to a Janus Web Server

Using Janus Web Server it is possible to create complete web applications using only the Model 204 editor. With the Model 204 editor, it is possible to create static HTML pages and User Language procedures that dynamically generate HTML and that accept input from HTML forms.

There might be a need, however, to include binary data generated outside of Model 204 in an application. This data can include graphical images, audio data, java applets and many other workstation generated binary datatypes. Furthermore, it might be desirable to generate static HTML pages using an HTML editor, some other workstation based editor or even a mainframe editor outside of Model 204. All these cases raise the issue of how the data generated outside of Model 204 can be loaded into Model 204 procedures.

Since Model 204 procedures are generally expected to contain text data, there is also an issue of how binary, non-text data can be stored in a Model 204 procedure. This problem is solved by Janus Web Server with the use of special format procedures that allow the storing of data as either binary or base64 encoded data. To keep these procedures from causing problems in Model 204, the data is "wrapped" with some character data that is stripped by Janus Web Server when the data is downloaded as part of a web request. More information on producing these special format procedures is available in Loading with the SIRPIPE and SIRPUT Utilities, HTTP PUT support, and Form-based uploads.

Historically, two mechanisms were mainly used for moving data from outside of Model 204 into procedures:

• Placing data into a BATCH2 (or BATCH204) stream between a PROCEDURE and END PROCEDURE statement.
• Using READ IMAGE and $BldProc to create Model 204 procedures.

The latter mechanism can be quite cumbersome, so it is not discussed further. The former mechanism has some problems uploading binary or text data that has long records (lines), which are often created by HTML-generating packages.

The SIRPIPE and SIRPUT utilities were designed for loading binary and text data into Model 204. SIRPIPE uses BATCH2 or BATCH204 to load the data, while SIRPUT uses HTTP PUT. With these utilities, moving data from a workstation to Model 204 is a two-step process:

1. A file upload program moves the data from a workstation to an operating system file. Since any site using Janus Web Server will have a TCP/IP package on the host mainframe, FTP is probably the best of the many ways to transfer files from a workstation to a mainframe (FTP, IND$, 3270 emulator uploads, PCTRANS, Kermit, etc.).
2. SIRPIPE or SIRPUT moves the data from the operating system file into Model 204. For information about using SIRPIPE and SIRPUT, see Loading with the SIRPIPE and SIRPUT utilities.

While an FTP plus SIRPIPE or SIRPUT, batch-based, transfer mechanism is very flexible, it generally requires that any transfer be initiated from the mainframe. This might not be a good solution at sites where it is desirable to allow programmers, web designers, and end-users to upload data directly from their workstations.

To service workstation-initiated file transfers, Janus Web Server provides HTTP PUT support and "form-based file upload" support, as described in the following sections. An additional alternative, comparable or superior to these methods, is the Janus FTP Server, which lets you upload and download directly to and from Model 204 procedure files. Not a Janus Web Server feature, the Janus FTP Server requires that you be licensed for Janus Sockets and Janus TCP/IP Base and be running the Sirius Mods.

HTTP PUT support

HTTP PUT support adds Web Server handling of the PUT method, which calls for an upload to the server of the data in the request. With PUT support, you can use JANUS WEB ON rules to control the uploading of files to Model 204, as described in Defining Web rules. You can programmatically control HTTP PUT file uploads with JANUS WEB ON rules and the $Web_Proc_Receive and $Web_List_Receive functions.

Since the PUT method is part of the HTTP 1.1 standard, it is expected that most vendors of HTML editors and perhaps browsers will ultimately support this method. As of this writing, HTTP PUT is supported on most web servers and by Perl, PHP, Python and the other popular web development languages. but is not widely accessible on web clients.

Form-based uploads

An alternative approach to uploading files to a Janus Web Server is form-based uploads. Form-based uploads work in conjunction with HTML forms. An HTML form can have in input field with a type of "file." The content of such a field is the name of a file (including disk and path information), and when the form is submitted, the contents of the named file, as well as the name of the file, are sent to the web server.

The following is an example of HTML that generates a form that allows the upload of a single file that defaults to a file called capture.dat:

<form method=POST enctype="multipart/form-data"> <input type=FILE name=indata value="C:/upload/capture.dat"> <br> <input type=SUBMIT value="Upload file"> </form>

Note especially the enctype attribute in the form tag. This is required because the default form encoding, application/x-www-form-urlencoded, does not allow file uploads.

Janus Web Server automatically handles the different encoding mechanisms for form data, so other than specifying the appropriate enctype when form-based file uploads are to be used, an application programmer need not be concerned about the encoding mechanism. An enctype of multipart/form-data can be used even if no type=file input fields are present in the form, but this is a slightly less efficient way of representing a form than the default encoding mechanism of application/x-www-form-urlencoded.

The form encoding mechanism can be queried with the $Web_Hdr_Parm function, as in the following:

If $web_hdr_parm('content-type') Ne 'MULTIPART/FORM-DATA' Then %rc = $web_done(400, 'Invalid upload type') Stop End if

Since form-based file uploads are part of both the HTTP and HTML standards, it is quite likely that most browsers already support this mechanism.

A limitation of this technology is that some implementations of the <input type=file> tag by Internet Explorer essentially ignored the default value (file name). This might be viewed as a security feature since it prevents a malicious server from presenting a user with a form with a file upload buried in the middle that uploads a secure file from the browser's machine. If the user doesn't notice that this is being done on the form, the secure file could be blindly uploaded to the malicious server. In theory, this security hole could be dealt with by displaying a pop-up window that warns the user which file is being uploaded and gives the user a chance to cancel the request. It is not currently known if this type of warning system has been or will be implemented in Internet Explorer or other browsers. A form-based upload might always bear the inconvenience of forcing the end-user to pick out the uploaded file at each invocation.

Another weakness of form-based uploads is that each <input type=file> tag can only be associated with a single file. There is no easy way to upload the entire contents of (or several files in) a directory. While multiple <input type=file> tags can be present in a form so that multiple files can be uploaded with a single request, each file must be individually selected by the end-user.

A final weakness of the form-based file upload approach is that since it is part of a form, it must be processed by User Language and cannot be processed with simple JANUS WEB ON ... RECEIVE rules. SOUL can process files being uploaded as part of a form-based upload with some special purpose $functions, which are described in $Web_Proc_Receive and $Web_List_Receive.

Despite these weaknesses, form-based file uploads present the simpler option for allowing untrained end users to upload files to a web server as part of an integrated web application, especially when the upload consists of only a very few files. For relatively frequent uploads of, say, more than five files, the option of choice is the Janus FTP server.

Janus FTP Server

Another option for uploading files to Model 204, for use on the web (or for any other use), is the FTP feature of Janus Sockets. The Janus FTP Server lets you use any standard FTP client to upload one or a set of files in a directory to a Model 204 file, storing the uploaded files as Model 204 procedures.

To implement the FTP feature, you use User Language to define the following, the details for which are provided in Janus FTP Server:

• A Janus Sockets port with the special type FTPSERVER
• Web server rules specific to the upload location

The following sample definition deck creates and starts a Janus FTP Server port called MYFTP:

JANUS FORCE MYFTP JANUS DELETE MYFTP * FTP server on port 3205 JANUS DEFINE MYFTP 3205 FTPSERVER 4 CLIENTSOCKET FTPDCLIENT - AUDTERM PASVPORT 3001 BINDADDR 198.242.244.100 * Client port definition for active transfers JANUS DEFINE FTPDCLIENT * CLSOCK 5 REMOTE SOCKPMAX 5 JANUS CLSOCK FTPDCLIENT ALLOW * FTP access to MYPROC JANUS FTP MYFTP ASSIGN /MY TO FILE MYPROC * Permissions JANUS FTP MYFTP ALLOW /MY WRITE TO USER CARTMAN * Set the home folder JANUS FTP MYFTP HOME /MY TO ALL JANUS START MYFTP JANUS START FTPDCLIENT

In this example:

• The MYFTP port defined above runs on port 3205 on machine 198.242.244.100. The port definition includes a specification (PASVPORT) to support passive FTP transfers for clients that require that mode.
• The FTPDCLIENT port is defined to support active FTP transfers for clients that require that mode.
• The JANUS FTP ASSIGN command maps the directory orientation seen by an FTP client onto the Model 204 file and procedure structure that actually exists in Model 204. In this example, the FTP ASSIGN command maps the folder /MY to the file MYPROC. Any file uploaded to the /MY directory becomes a procedure in file MYPROC.
• The user CARTMAN is the only user allowed to upload to the port, as defined by the JANUS FTP ALLOW command.
• The "home folder" /MY, defined by the JANUS FTP HOME command is the folder into which the user is automatically placed when they connect to the port.

Once these SOUL statements are executed in the Model  204 online, client FTP users anywhere can connect to the Janus server port and execute PUTs or their GUI equivalents for the files they want to upload. FTP clients will need both the host ID (198.242.244.100) and port (3205) of the FTP server to connect. Command line clients, for example, might use:

open 198.242.244.100 3205

Users can upload files containing binary or text information. The FTP Server correctly handles the type of upload that the client FTP package indicates. By default, the server assumes type A (ASCII, with translation to EBCDIC) until notified otherwise.

Users can upload multiple files at a time, executing MPUTs with a command-line client or taking GUI-equivalent measures. And users can upload entire directories and subdirectories (if and only if a matching directory structure is predefined for the server). The dynamic creation of directories (FTP protocol MKD command) is not currently supported by Janus FTP.

For uploading individual files or sets of files, Janus FTP is a fast and convenient utility, suitable for any type of user.

How uploads work with Janus Web Server

Whether HTTP PUT or form-based upload is being used, Janus Web Server copies the file being uploaded into CCATEMP. This way, if a network or browser problem prevents the entire file from being transferred, the request can simply be discarded without a risk of leaving partial data in a procedure.

Once all the data from a request has been received into Model 204, the web rules are scanned to determine the appropriate action. If the upload was the result of an HTTP PUT, and the appropriate action ("RECEIVE") is indicated in a JANUS WEB ON rule, the file is simply copied into the target procedure, perhaps being converted to the special Janus Web Server binary or base64 formats if the input file consists of binary non-text data.

If the JANUS WEB ON rule associated with the upload request indicates that a command should be executed, or if the request is a form-based upload, User Language $functions ($Web_Proc_Receive, $Web_List_Receive) can be used to upload the file(s) into a procedure or into a $list.

When a procedure is being updated either as the result of a JANUS WEB ON rule or a $Web_Proc_Receive function, Janus Web Server attempts to get an exclusive lock on the procedure before doing the update. If it is unable to obtain an exclusive procedure lock, the request fails. The exclusive lock is only held while data is copied from CCATEMP to the procedure. This means two things:

1. Once the copy operation is started, it will complete regardless of the status of the connection with a browser. Put another way, there is no danger of an HTTP PUT or form-based upload leaving around a partial procedure.
2. The exclusive lock on a procedure is generally held very briefly (typically a few milliseconds), so it is not likely to disrupt download operations for the same procedures.

Note: Although the preceding discussion concerns Janus Web Server handling of HTTP PUT and form-based uploads, the Janus FTP Server handling of uploaded files uses CCATEMP and locking in the same way to safeguard your procedures.

Choosing the correct upload method

There are many options for doing file upload to Model 204 with Janus Web Server. None of them is perfect and each has its advantages. The following table summarizes the advantages and disadvantages of each technique to help determine the correct technique for any application.

FTP and SIRPIPE|SIRPUT	Advantages Easy to configure and automate. Can be initiated from mainframe. Not dependent on any browsers. Disadvantages Cannot be initiated from a workstation. BATCH2 is extremely slow, so SIRPUT is preferred to SIRPIPE. Likely users System administrators, operators (running canned procedures), and mainframe-oriented developers.
HTTP PUT	Advantages Can upload entire directories. Can be initiated from a workstation. Can be set up without programming in Model 204. Disadvantages Not widely supported beyond Netscape Composer. Can't be integrated into web application. Some training required. Likely users Web developers and designers.
Form-based upload	Advantages Can be integrated into web applications. Little or no training required. Disadvantages No way to upload directories or groups of files. Some SOUL programming is required to set it up. Likely users End-users.
Janus FTP Server	Advantages Fast transfers, including both uploads and downloads. Can be initiated from a workstation. Easy-to-use interface. Can upload multiple files and directories. Disadvantages Can't be integrated into web application. Must be set up with User Language programming. Might need to acquire an FTP client tool. Requires Janus Sockets. Can't dynamically create subdirectories. Likely users Web developers, designers, and end users.