IMAGINE command
Move Model 204 data to Imagine
This command provides the facilities to extract Model 204 data and load it into Imagine.
Summary
- Privileges
- File manager
- Function
- provides the facilities to extract Model 204 data and load it into Imagine.
Syntax for IMAGINE
IMAGINE SYNC | EXTRACT [ASYNC] [MAXERR e] [MAXPENDING m] [RECORDNUMBER r] [NOCOMMIT] [TICK t]
Where:
- SYNC indicates that data is to be extracted and the Model 204 file and Imagine are to be kept in sync after the command.
- EXTRACT indicates that data is to be extracted but the Model 204 file and Imagine are not to be kept in sync after the command.
- ASYNC indicates that the actual extract is to be performed on a daemon. In this mode, the command returns immediately after some preliminary setup. The advantage of this mode is that it frees up the issuing thread for other commands. The disadvantage is that one must look elsewhere to determine when the request completed and whether or not it was successful.
- MAXERR indicates the maximum number of record data errors allowed before the extract is aborted. One might wish to set this to a high value for an extract intended to scan a file for data integrity errors. Because records are sent asynchronously in batches, it's quite possible for the actual number of errors to exceed the MAXERROR value. MAXERROR defaults to 50.
- MAXPENDING indicates the maximum number of uncommitted record sends to be allowed in the extract. Setting this to a low value could result in reduced transfer rates because of network latency. MAXPENDING defaults to 4095.
- NOCOMMIT indicates that the data is to be sent to Imagine and processed, but not committed. This provides a way of validating that a file can be loaded into Imagine without actually doing so. This option makes no sense for a SYNC extract, so is only allowed with
IMAGINE EXTRACT. - RECORDNUMBER indicates the record number of the single record to be sent to Imagine. This might be useful for problem diagnosis or resending fixed up records that failed on an initial extract. This option makes no sense for a SYNC extract so is only allowed with
IMAGINE EXTRACT. Other options (except NOCOMMIT) are ignored if RECORDNUMBER is specified. - TICK indicates the number of records between progress messages (M204.3073) to the journal. This parameter also sets the frequency of progress messages on Imagine. TICK defaults to 100000.
There is also a format of the IMAGINE command:
IMAGINE ENDSYNC
That indicates a SYNC or EXTRACT for the file should be terminated.
Syntax notes
The IMAGINE command operates on the context file which is either the current file or one explicitly specified before the command as in:
IN FILE FOOBAR EXTRACT NOCOMMIT
Example
To put file FOO into sync mode, one can do:
OPEN FILE FOO IMAGINE SYNC TICK 200000
The TICK keyword indicates that a M204.3073 message should be issued every 200,000 records. If one wishes to see these messages on the operator console, one can do:
MSGCTL M204.3073 OPR
Note that the actual extraction of records is done on a daemon thread created by the IMAGINE command so the thread that issued the IMAGINE command would return fairly quickly. Also note that this means that there is no terminal for M204.3073 (or other) messages issued during the extract to appear, so one needs to look in the journal for errors, should they occur.
Usage notes
To obtain a consistent view of the file on Imagine, the IMAGINE command first quiesces the file; that is, ensures that no updating transactions that had updated the file are active. This will usually take less than a second, but a long running transaction that had updated the file could make this take quite a while. As such, it is recommended that the IMAGINE command be issued at a relatively quiet time in terms of file updates.
Once the extraction process has begun, whether SYNC, EXTRACT, or NOCOMMIT, any other thread that wishes to update a record in the file being extracted, cannot update the record until it has been copied to Imagine and committed. If it has already been done at the time of the commit, no wait is required. However, if the record has not been copied to and committed to Imagine the thread would send a request to the extracting daemon to immediately send the record to Imagine (if not already sent) and then to wake the updating thread when that record is committed on Imagine. This can cause a slight delay in update processing so, again, the recommendation is for extracts to be done at relatively quiet times for a file.
If a file is in SYNC mode, once a record has been sent to Imagine, any updates to the file are sent to Imagine (via a broker connection) and those updates must be committed for the transaction to succeed. That is, if some error occurs on Imagine for a SYNC update, the request would be cancelled and the transaction backed out. Note that this processing occurs even after the file has been fully extracted, and the extracting daemon is no longer active. This ensures that the SYNC'ed file will remain consistent between Model 204 and Imagine making it feasible to switch the file to transparency mode with a minimal outage.
For a plain EXTRACT (as opposed to SYNC), updates to records in the file after they have
committed in Imagine are not sent to Imagine, so the file can
get out of synch between Model 204 and Imagine. However, by not
allowing any records to be updated in the file during the EXTRACT process until they
have been committed on Imagine, the file on Imagine will be ensured
to be a consistent view of the file as of the point in the IMAGINE EXTRACT
command where updates to the file had been quiesced.