IMAGINE command

Move Model 204 data to Imagine

This command provides the facilities to extract Model 204 data and load it into Imagine.

Summary

Privileges

File manager

Function

provides the facilities to extract Model 204 data and load it into Imagine.

IMAGINE SYNC and IMAGINE EXTRACT

IMAGINE SYNC | EXTRACT [ASYNC] [MAXERR e] [MAXPENDING m] [RECORDNUMBER r] [NOCOMMIT] [TICK t]

Where:

• SYNC indicates that data is to be extracted and the Model 204 file and Imagine are to be kept in sync after the command.
• EXTRACT indicates that data is to be extracted but the Model 204 file and Imagine are not to be kept in sync after the command.
• ASYNC indicates that the actual extract is to be performed on a daemon. In this mode, the command returns immediately after some preliminary setup. The advantage of this mode is that it frees up the issuing thread for other commands. The disadvantage is that one must look elsewhere to determine when the request completed and whether or not it was successful.
• MAXERR indicates the maximum number of record data errors allowed before the extract is aborted. One might wish to set this to a high value for an extract intended to scan a file for data integrity errors. Because records are sent asynchronously in batches, it's quite possible for the actual number of errors to exceed the MAXERROR value. MAXERROR defaults to 50.
• MAXPENDING indicates the maximum number of uncommitted record sends to be allowed in the extract. Setting this to a low value could result in reduced transfer rates because of network latency. MAXPENDING defaults to 4095.
• NOCOMMIT indicates that the data is to be sent to Imagine and processed, but not committed. This provides a way of validating that a file can be loaded into Imagine without actually doing so. This option makes no sense for a SYNC extract, so is only allowed with IMAGINE EXTRACT.
• RECORDNUMBER indicates the record number of the single record to be sent to Imagine. This might be useful for problem diagnosis or resending fixed up records that failed on an initial extract. This option makes no sense for a SYNC extract so is only allowed with IMAGINE EXTRACT. Other options (except NOCOMMIT) are ignored if RECORDNUMBER is specified.
• TICK indicates the number of records between progress messages (M204.3073) to the journal. This parameter also sets the frequency of progress messages on Imagine. TICK defaults to 100000.

Syntax notes

The IMAGINE command operates on the context file which is either the current file or one explicitly specified before the command as in:

IN FILE FOOBAR EXTRACT NOCOMMIT

Example

To put file FOO into sync mode, one can do:

OPEN FILE FOO IMAGINE SYNC TICK 200000

The TICK keyword indicates that a M204.3073 message should be issued every 200,000 records. If one wishes to see these messages on the operator console, one can do:

MSGCTL M204.3073 OPR

Note that the actual extraction of records is done on a daemon thread created by the IMAGINE command so the thread that issued the IMAGINE command would return fairly quickly. Also note that this means that there is no terminal for M204.3073 (or other) messages issued during the extract to appear, so one needs to look in the journal for errors, should they occur.

Usage notes

To obtain a consistent view of the file on Imagine, the IMAGINE command first quiesces the file; that is, ensures that no updating transactions that had updated the file are active. This will usually take less than a second, but a long running transaction that had updated the file could make this take quite a while. As such, it is recommended that the IMAGINE command be issued at a relatively quiet time in terms of file updates.

Once the extraction process has begun, whether SYNC, EXTRACT, or NOCOMMIT, any other thread that wishes to update a record in the file being extracted, cannot update the record until it has been copied to Imagine and committed. If it has already been done at the time of the commit, no wait is required. However, if the record has not been copied to and committed to Imagine the thread would send a request to the extracting daemon to immediately send the record to Imagine (if not already sent) and then to wake the updating thread when that record is committed on Imagine. This can cause a slight delay in update processing so, again, the recommendation is for extracts to be done at relatively quiet times for a file.

If a file is in SYNC mode, once a record has been sent to Imagine, any updates to the file are sent to Imagine (via a broker connection) and those updates must be committed for the transaction to succeed. That is, if some error occurs on Imagine for a SYNC update, the request would be cancelled and the transaction backed out. Note that this processing occurs even after the file has been fully extracted, and the extracting daemon is no longer active. This ensures that the SYNC'ed file will remain consistent between Model 204 and Imagine making it feasible to switch the file to transparency mode with a minimal outage.

For a plain EXTRACT (as opposed to SYNC), updates to records in the file after they have committed in Imagine are not sent to Imagine, so the file can get out of synch between Model 204 and Imagine. However, by not allowing any records to be updated in the file during the EXTRACT process until they have been committed on Imagine, the file on Imagine will be ensured to be a consistent view of the file as of the point in the IMAGINE EXTRACT command where updates to the file had been quiesced.

IMAGINE ENDSYNC

IMAGINE ENDSYNC

This format indicates that a SYNC or EXTRACT for the file should be terminated.

IMAGINE LISTSYNC

IMAGINE LISTSYNC

This format lists the files currently in SYNC mode. For each such file, a row is printed with the following columns:

• FILE — the Model 204 file name.
• SFX — the broker suffix being used for the file's sync traffic (the value supplied via APSYSUFFIX when the file was put into sync mode), or - if no suffix is in use.
• START — the time at which the IMAGINE SYNC command was issued for the file, formatted as YYYYMMDDHHMISS.
• END — the time at which the extract phase completed, formatted as YYYYMMDDHHMISS. Blank if the extract is still running.
• UPDATES — the number of post-extract sync updates that have been sent to Imagine for the file. The counter is zeroed when the extract phase completes, so this value reflects only ongoing sync traffic, not the records sent during the initial extract.

IMAGINE LISTSYNC takes no parameters and is independent of the current context file.

See also

• Imagine Transparency