Qorus Integration Engine®
4.0.3.p2_git
|
Workflows are made up a set of interdependent steps that each perform one traced and restartable action. To design a workflow, a series of logical steps and their dependencies must be defined and then represented by a workflow definition file. Once the high-level design for the workflow has been done, then the logic for the steps can be implemented and the workflow definition and the functions can be loaded into the database and executed.
The following table defines the major elements used when designing and implementing an Qorus workflow.
Qorus Workflow Elements
Element | Description |
Step | The lowest element in a workflow, represents one traced and restartable action. Each step is defined by at least primary step code containing the logic for the step, and optionally other code attributes (such as validation code, run when the step is run in error recovery mode, or asynchronous back-end code, required for asynchronous steps) and other option attributes. |
Workflow | The workflow is the highest level element defining the logic as a set of steps and inter-step dependencies, along with other attributes; workflows process workflow order data instances that in turn contain the data and the status of processing (status of all steps). A running workflow is called a workflow execution instance and can be run either in batch mode (OMQ::WM_Normal), batch recovery mode (OMQ::WM_Recovery), or synchronous mode. |
Queue | Asynchronous steps require a queue for linking the associated step data created by the front-end logic with the back-end code. |
Workflow Synchronization Events | Workflow synchronization event steps allow multiple workflow orders to synchronize their processing based on a single event |
Qorus includes a framework for defining error information and raising errors. If a workflow defines an errorfunction in the workflow definition, this function is executed by oload, and the error information is loaded in the system database in the GLOBAL_WORKFLOW_ERRORS
and WORKFLOW_ERRORS
(see Global and Workflow-Specific Error Definitions for more information). The return value of this function is a hash describing how the system should act when certain errors are raised when processing workflow order data.
Workflows raise errors by calling WorkflowApi::stepError() (in Qore code), WorkflowApi.stepError() (in Java code) or by throwing an appropriate exception.
In the case an exception is thrown, for Qore exceptions, the exception err
code is used as the error name, for Java exceptions, the exception full class name (ex: "javax.xml.soap.SOAPException"
) is used as the error name; the system will then use the error name as the hash key to look up error information in the value returned by the errorfunction.
To allow a workflow to recover gracefully from an error, implement validation code for each step. Validation code allows workflows to recover gracefully from errors such as lost request or response messages or temporary communication failures without requiring manual intervention.
By default, error definitions are global. A global definition is a workflow error definition that applies to all workflows. Workflow-specific error definitions apply only to a particular workflow configuration.
There are three ways to create workflow-specific error definitions:
"level"
= OMQ::ErrLevelWorkflow in the errorfunction's return value"level"
key unassigned (or assigned to the default: OMQ::ErrLevelAuto) in the errorfunction's return value but give the error a different definition than an existing global errorThe last point above implies that if two or more workflows define the same error with different attributes (but leave the error's "level"
option either unset or assigned to the default: OMQ::ErrLevelAuto), the first error will be a global error, and each time the other workflows define the error with a different set of attributes, those new errors will be workflow-specific error definitions.
Qorus includes default error definitions for common technical errors that should normally result in a workflow order instance retry and also template errors that can be re-used in workflows as needed.
The following table lists the default errors delivered with Qorus; note that workflows with a severity level of OMQ::ES_Warning ("WARNING"
) only result in a warning error record but do not affect the logic flow of the workflow.
Default Global Workflow Error Definitions
Error | Severity | Status | Business | Description |
SOCKET-SSL-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An SSL error occurred while sending or receiving data on a socket |
SOCKET-SEND-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O error occurred while sending socket data |
SOCKET-RECV-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O error occurred while receiving socket data |
SOCKET-TIMEOUT | OMQ::ES_Major | OMQ::StatRetry | False | The socket transfer exceeded the timeout period |
SOCKET-CONNECT-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O error occurred while attempting to connect to a remote socket |
HTTP-CLIENT-RECEIVE-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | I/O or protocol error occurred while reading HTTP data |
SOCKET-HTTP-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O or protocol error occurred while reading HTTP data on the socket |
FTP-RECEIVE-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O or protocol error occurred while reading FTP data |
FTP-CONNECT-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred connecting to the FTP server |
FTPS-SECURE-DATA-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O or protocol error occurred on the secure FTP data channel |
FTP-DATA-SOCKET-CLOSED | OMQ::ES_Major | OMQ::StatRetry | False | The FTP data channel was closed prematurely |
FTP-DATA-SOCKET-RECV-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An I/O error occurred on the FTP data channel |
FTP-TIMEOUT-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | the FTP transfer exceeded the timeout period |
SSH2-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server |
SFTPCLIENT-TIMEOUT | OMQ::ES_Major | OMQ::StatRetry | False | A timeout occurred communicating with the remote server |
SFTPCLIENT-LIST-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::list() method |
SFTPCLIENT-LISTFULL-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::listFull() method |
SFTPCLIENT-CHMOD-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::chmod() method |
SFTPCLIENT-MKDIR-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::mkdir() method |
SFTPCLIENT-RMDIR-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::rmdir() method |
SFTPCLIENT-RENAME-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::rename() method |
SFTPCLIENT-REMOVEFILE-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::removeFile() method |
SFTPCLIENT-CONNECT-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating while connecting to the remote server |
SFTPCLIENT-GETFILE-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::getFile() method |
SFTPCLIENT-GETTEXTFILE-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::getTextFile() method |
SFTPCLIENT-PUTFILE-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::putFile() method |
SFTPCLIENT-STAT-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::stat() method |
SSH2CLIENT-SCPGET-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SSH2Client::scpGet() method |
SSH2CLIENT-SCPPUT-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the remote server in the Qore::SSH2::SSH2Client::scpPut() method |
SSH2CLIENT-TIMEOUT | OMQ::ES_Major | OMQ::StatRetry | False | A timeout error occurred communicating with the remote server |
TRANSACTION-LOCK-TIMEOUT | OMQ::ES_Major | OMQ::StatRetry | False | a shared database connection could not be acquired within the timeout period |
DATASOURCE-RETRY-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | An error occurred communicating with the database |
RETRY-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | (generic) An error occured that results in a workflow order retry; see info for details |
REST-CONNECTION-ERROR | OMQ::ES_Major | OMQ::StatRetry | False | an error occurred in a REST request |
BUSINESS-ERROR | OMQ::ES_Major | OMQ::StatError | True | (generic) A business error was detected; see info for details |
BUSINESS-RETRY-ERROR | OMQ::ES_Major | OMQ::StatRetry | True | (generic) A business error was detected that results in a workflow order retry; see info for details |
BUSINESS-WARNING | OMQ::ES_Warning | OMQ::StatError | True | (generic) A condition was detected affecting the business logic that results in a warning; see info for details |
TECHNICAL-WARNING | OMQ::ES_Warning | OMQ::StatError | False | (generic) A technical condition was detected that results in a warning; see info for details |
java.io.InterruptedIOException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Signals that an I/O operation has been interrupted. An InterruptedIOException is thrown to indicate that an input or output transfer has been terminated because the thread performing it was interrupted |
java.net.SocketTimeoutException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Signals that a timeout has occurred on a socket read or accept |
java.net.ConnectException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the connection was refused remotely (e.g., no process is listening on the remote address/port) |
java.net.NoRouteToHostException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the remote host cannot be reached because of an intervening firewall, or if an intermediate router is down |
java.net.HttpRetryException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Thrown to indicate that a HTTP request needs to be retried but cannot be retried automatically, due to streaming mode being enabled |
javax.imageio.IIOException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) An exception class used for signaling run-time failure of reading and writing operations |
java.nio.channels.InterruptedByTimeoutException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Checked exception received by a thread when a timeout elapses before an asynchronous operation completes |
java.net.ProtocolException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Thrown to indicate that there is an error in the underlying protocol, such as a TCP error. |
java.rmi.ConnectException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) A ConnectException is thrown if a connection is refused to the remote host for a remote method call |
java.rmi.ConnectIOException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) A ConnectIOException is thrown if an IOException occurs while making a connection to the remote host for a remote method call |
java.net.SocketException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Thrown to indicate that there is an error creating or accessing a Socket |
javax.net.ssl.SSLException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Indicates some kind of error detected by an SSL subsystem |
javax.net.ssl.SSLHandshakeException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Indicates that the client and server could not negotiate the desired level of security. The connection is no longer usable |
javax.net.ssl.SSLProtocolException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) Reports an error in the operation of the SSL protocol |
javax.xml.soap.SOAPException | OMQ::ES_Major | OMQ::StatRetry | False | (Java) An exception that signals that a SOAP exception has occurred |
See the Qorus web UI's "Global Errors" page or the detail page for each workflow to see how error's are configured in Qorus as errors may have been redefined or overridden after the initial installation.
GLOBAL_WORKFLOW_ERRORS
tableWorkflows and steps, along with almost every other object in the Qorus schema, are versioned. When a new version of a workflow is released, either due to a logic upgrade or a bug fix, keep in mind that workflows can only recover data that has been processed by the same version of that workflow.
EXAMPLE-WORKFLOW
1.1
cannot recover data processed by EXAMPLE-WORKFLOW
1.0
. Furthermore, example-step
1.1
cannot recover data processed by example-step
1.0
.When it is necessary to maintain recovery compatibility in a workflow update, then it will be necessary to redefine steps and workflows with the same version name and number. In these cases, the patch attribute of the object should be updated to reflect the change.
Also keep in mind that new steps added to a workflow will be executed for data that is being recovered even with data that was initially processed by a previous version of the workflow that did not include the new step in its definition.
If a step is removed from a workflow and data must be recovered where that step has a OMQ::StatRetry status, then that data cannot be recovered by a definition of the workflow that no longer includes that step.
These points should be considered carefully when planning workflow updates.
Sometimes it may be necessary to release two or more versions of a workflow when changes must be made; for example one version to make a bug fix in an existing version, and a new version with additional or removed steps also including the bug fix.
Because a running workflow execution instance can be working on several different orders at once in different threads, accessing workflow data is performed through API calls in order to guarantee that the workflow's program code accesses only the correct data for the current order being processed at all times.
Accessing and processing data is done using the Qorus API as outlined in this section; these APIs set up the data context for each thread so that the correct data is accessed.
Static data represents the workflow order data being processed. Workflow static order data cannot be updated or deleted by the Qorus workflow API; it is read-only data representing the order data to be processed or fulfilled by the workflow.
APIs:
The information returned by the above function corresponds to the deserialized contents of the field ORDER_INSTANCE.STATICDATA
.
Dynamic data is associated with the workflow order data instance being processed, but it can be updated and is persistent. Any changes made to dynamic data will be committed to the database before the update method returns, therefore any changes will be available in the future, even in the case of errors and later recovery processing.
Dynamic data is appropriate for storing identifiers and references generated during order processing that are needed in subsequent steps, for example.
Qore API support:
Java API support:
Dynamic step data, like dynamic order data is associated with the workflow order data instance and also the current step being processed, additionally it can be updated and is persistent like dynamic order data. Any changes made to dynamic step data will be committed to the database before the update method returns, therefore any changes will be available in the future, even in the case of errors and later recovery processing.
Dynamic step data is appropriate for storing information specific to a particular step, particularly user-driven form data with asynchronous steps with the user-interaction flag enabled.
Qore API support:
Java API support:
REST API support:
Qorus maintains a hash of temporary data associated to the workflow order data instance being processed. This hash can be updated, but it is not persistent, therefore this hash is suitable for temporary data storage only.
This data is lost every time Qorus detaches (i.e. temporarily or permanently stops processing a workflow order data instance, for example, due to an ERROR
status and purges the data from the workflow data cache) from a workflow order data instance.
Because temporary data is deleted every time Qorus detaches from a workflow order data instance, it can only be reliably set in the attach logic.
Qore API support:
Java API support:
Qorus was designed to allow workflow sensitive order data to be processed while avoiding inadvertent disclosure of this data to unauthorized persons.
Workflow sensitive order data must be processed separately for each data subject stored against the workflow order, this is because each data subject's sensitive data can be queried, updated, or deleted separately across all workflow orders in the system (both in the system schema and any archiving schema).
Workflow order sensitive data is stored separately for each data subject against the workflow order using two identifiers as follows:
skey:
the sensitive data key type (not treated as sensitive itself, ex: "tax_id"
, "social_insurance_nr"
, "ssn"
, etc)svalue:
the sensitive data key value, which is also treated as sensitive itselfThe following image provides an overview of a concrete example of sensitive data stored against a workflow order storing sensitive data for at least two data subjects with tax_ids "984.302192.AF"
and "739.323.714.BR"
:
The svalue
value, being sensitive itself, should not be stored in static, dynamic data, or step dynamic data. To reference sensitive data from within non-sensitive data, an alias can be used, which is a unique identifier within a workflow order that can be used to uniquely identify sensitive data for a single data subject. Sensitive data aliases are only usable in internal sensitive data APIs.
For example, if a workflow order consists of non-sensitive order information along with a natural person's name and address (which is sensitive) for each order, then the sensitive data alias could simply be the list index (ex: "0"
, "1"
, ...).
The following internal workflow APIs provide sensitive data support:
Qore APIs:
Java APIs:
Workflow execution instance data is stored in a hash maintained by the system. This data is local to the running workflow execution instance pseudo-process and persists until the workflow execution instance terminates.
Any changes made to this data will persist within the running workflow execution instance (pseudo-process) independently of the workflow order data processed.
Because of this, workflow execution instance data is a substitute for global variables in a workflow program. Workflow programs are shared between all running workflow execution instances of that same type (sharing the same name and version and the same workflowid); global variables are not allowed because it is considered unsafe for workflow execution instances to share any common state. If your workflows do need to share some data between execution instances, implement a Qorus service to provide this functionality instead.
Workflow execution instance data will be set in the onetimeinit code (initializing resources for the workflow execution instance), and read by the rest of the workflow.
Qore methods:
Java methods:
The following properties of the workflow metadata can be returned by the WorkflowApi::getWorkflowMetadata() method (in Qore) or the WorkflowApi.getWorkflowMetadata() method (in Java):
"name"
: name of the workflow"version"
: version of the workflow"patch"
: The patch attribute of the workflow"workflowid"
: ID of the current workflow (metadata ID)"remote"
: a boolean value giving the remote status of the workflow (if it is running as an independent process or not; see the remote flag)"description"
: The description of the workflow"cached"
: The date and time the workflow metadata was read and cached from the database"errorfunction_instanceid"
: The function instance ID of the error function of the workflow"attach_func_instanceid"
: The function instance ID of the attach function for the workflow"detach_func_instanceid"
: The function instance ID of the detach function for the workflow"onetimeinit_func_instanceid"
: The function instance ID of the onetimeinit function for the workflow"errhandler_func_instanceid"
: The function instance ID of the error handler function for the workflow"keylist"
: A list of valid order keys for the workflow"errors"
: The error hash as returned from the error function"options"
: A hash of valid workflow options, key = option, value = descriptionProperties of the current running workflow execution instance can be retrieved via the WorkflowApi::getWorkflowInstanceData() method (in Qore) or the WorkflowApi.getWorkflowInstanceData() method (in Java); the following keys are present in the response:
"dbstatus"
: the status of the workflow order in the database (for the current status, see the status
key, see Workflow, Segment, and Step Status Descriptions for possible values); this will normally be OMQ::StatInProgress, unless called from the Detach Function (first available in Qorus 2.6.2)"external_order_instanceid"
: The external order instance ID saved against the order, if any"execid"
: the execution instance ID (workflow pseudo-process ID, first available in Qorus 2.6.0.3)"initstatus"
: the status of the workflow when it was cached (before it was updated to OMQ::StatInProgress, see Workflow, Segment, and Step Status Descriptions for possible values); note that synchronous workflow orders are created with status OMQ::StatInProgress, so initstatus
should always be OMQ::StatInProgress for synchronous orders (first available in Qorus 2.6.2)"instancemode"
: The mode the workflow execution instance process is running in (OMQ::WM_Normal or OMQ::WM_Recovery) OMQ::StatInProgress, unless called from the Detach Function"mode"
: The mode the current thread is running in (OMQ::WM_Normal or OMQ::WM_Recovery)"remote"
: a boolean value giving the remote status of the workflow (if it is running as an independent process or not; see Workflow Remote Parameter)"sync"
: True if the workflow execution instance is synchronous, False if not"name"
: name of the workflow"parent_workflow_instanceid"
: The workflow order data instance ID of the parent workflow if the current workflow is a subworkflow, NOTHING if not"priority"
: The priority (0 - 999) of the workflow order data instance (first available in Qorus 2.6.0.3)"started"
: The date/time the workflow order data instance was created (first available in Qorus 2.6.0.3)"status"
: The current status of the workflow order data instance (see Workflow, Segment, and Step Status Descriptions for possible values); this will normally be OMQ::StatInProgress; first available in Qorus 2.6.2)"version"
: version of the workflow"workflowid"
: ID of the current workflow (metadata ID)"workflow_instanceid"
: The current workflow order data instance ID being processedWorkflow definition files define workflow metadata including the steps and dependencies between steps.
Workflow definition file names must follow the following pattern: *.qwf
In Qore-language workflow definition files, the actual workflow definitions are made by assigning values to the workflows
variable in the workflow description file. The top-level keys represent the version(s) of the workflow(s) being described, and the member data of each version key defines that version of the workflow. For a description of this variable, see Workflow Definitions.
The format_version
variable in the workflow definition file must be set to "2.6"
to declare compatibility with the workflow definition file format documented in this manual.
In order to describe the metadata to the loader (oload), variables with pre-defined names and a pre-defined structure must be defined in this file.
Qore Workflow Definition File Variables
Variable | Description |
format_version | Set to "2.6" to declare compatibility with the oload format defined in this documentation for Qorus 4.0.3.p2_git |
workflows | Defines workflow metadata |
queues | Defines asynchronous queues |
groups | Defines RBAC access groups |
events | Defines workflow synchronization events |
Any other variables, functions, and classes defined in the file will be ignored unless explicitly executed in the script (for example, when building the workflows
or queues
data structures)
Workflow Definition Example
See Workflow Parameters for details on how to define workflow in the workflow definition file, and see $OMQ_DIR/examples/user/TEST-WORKFLOWS
for some example workflow definition files.
Queues provide the storage and delivery mechanism by which the results of executing an asynchronous event for an asynchronous step are delivered to the right step instance. The queues
variable in the workflow definition file must be set to a hash defining queue metadata. The top-level key gives the queue names, and the member data under the queue is a hash with a the desc key, giving a description of the queue.
queues.
"queuename" = ("desc":
"description string" ));
If a workflow has at least one asynchronous step, a queue must be defined. A single queue can be used for any number of asynchronous steps, however the keys in a queue must be unique. See the step queue attribute for more information.
The following are examples of queue definitions:
In order to allow workflows to declare membership to Interface Groups, the group may be declared in the workflow definition file by assigning a value to the global groups
variable.
groups.
"groupname" = ("desc":
"description string" ));
The following is an example of a group definition:
Workflow synchronization events are used to synchronize processing from multiple workflow orders; each workflow synchronization event has a key that will be bound to the workflow synchronization event step by calling QorusEventStepBase::bindEvent() or QorusEventStepBase::bindEventUnposted() (in Qore) or QorusEventStepBase.bindEvent() or QorusEventStepBase.bindEventUnposted() (in Java) in the step's primary step code. The workflow synchronization event key is unique within it's workflow synchronization event type; as such, the event type can be compared to a queue name.
The workflow synchronization event type is referenced in workflow synchronization event steps in the eventtype tag.
The actual event type definition is made in the workflow definition file by assigning a value to the global events
variable:
events.
"eventname" = ("desc":
"description string" ));
The following are examples of workflow synchronization event type definitions:
In this hash, descriptions for single character keys of custom statuses available for the workflow can be defined.
See
The workflow hash description consists of the step dependencies and other attributes.
The following diagram illustrates a subset of the attributes of a workflow.
The "workflows"
variable in a Qore workflow definition file must declare a hash that will define workflow metadata.
"workflows"
global variable represent workflow names; these keys must also be assigned to a hash, where each key is a version of the workflow being defined. The value assigned to each version key must be a workflow description hash, representing the workflow metadata definition.The workflow description hash has the following keys (optional keys are enclosed in square brackets []):
"my_attach:1.0"
); cannot be supplied if "class"
is used"my_detach:1.0"
); cannot be supplied if "class"
is used"my_workflow_init:1.0"
); cannot be supplied if "class"
is used"my_error_handler:1.0"
); cannot be supplied if "class"
is used"my_error_func:1.0"
); cannot be supplied if "class"
is usedHere step and function designators (except for in the "functions" tag above) define a specific version of the appropriate object. This is done by specifying a string in the following format: "name:version" or by specifying a hash with name
and version
keys set to the appropriate values.
Workflow Parameter Descriptions
Parameter | Description |
author | The author of the workflow |
steps | This is where steps and step dependencies are defined |
desc | A string giving the description for the workflow |
workflow-modules | lists modules providing new step base classes for class-based steps |
remote | flag that indicates if a workflow can run in a remote qwf process or not (see qorus-client.remote for how the default value is assigned by oload) |
autostart | the number of workflow execution instances to start when the system starts |
[sla_threshold] | the amount of time as an integer in seconds in which each workflow order should get a final status; if not present the default value of 30 minutes is assumed (see DefaultWorkflowSlaThreshold) |
max_instances | the maximum number of workflow execution instances that can be started |
classes | If defined, this list of strings designates the Qore classes that will be loaded into the workflow's program object from the CLASSES table when the workflow is cached (represented by the "Shared Classes" object in the diagram above). Note that only the Qorus class object name should be given as it appears in the database without any version number, the latest version of the class will be loaded into the workflow's Program object. |
constants | If defined, this list designates the Qore constant definitions that will be loaded into the workflow's program object from the CONSTANTS table when the workflow is cached (represented by the "Shared Constants" object in the diagram above). Note that only the Qorus constant object name should be given as it appears in the database without any version number, the latest version of the constant will be loaded into the workflow's Program object. |
functions | If defined, this list designates the Qore function definitions that will be loaded into the workflow's program object from the FUNCTIONS table when the workflow is cached (represented by the "Shared Functions" object in the diagram above). Note that only the Qorus function object name should be given as it appears in the database without any version number, the latest version of the function will be loaded into the workflow's Program object |
mappers | If defined, this list designates the mappers that will be registered with the workflow so they are available with the UserApi::getMapper() call in the workflow's code; the mapper name must be given with an explicit version number in each string (ex: "my-mapper-1:1.0" ) |
vmaps | If defined, this list designates the value maps to register with the workflow so they can be used with the UserApi::getValueMap() call in the workflow's code; no version number can be given in the value map names |
attach | If defined, this string identifies a function that will be executed every time Qorus caches and starts working on ("attaches to") a particular workflow order data instance (represented by the "Attach Logic" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "detach_function_name:1.0" |
detach | If defined, this string identifies a function that will be executed every time Qorus stops working on ("detaches from") a particular workflow order data instance (represented by the "Detach Logic" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "detach_function_name:1.0" . |
onetimeinit | If defined, this string identifies a function that will be executed once when each workflow execution instance is started (represented by the "Init Logic" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "onetimeinit_function_name:1.0" |
error_handler | If defined, this string identifies a function that will be executed every time an error is raised by the workflow (represented by the "Error Handler" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "error_handler_function_name:1.0" |
errorfunction | This string identifies a function that is called by Qorus when the workflow definition is loaded by oload (represented by the "Error Definitions" object in the diagram above). It should return a hash of error information. See Workflow Error Handling and Recovery for more information. The function's version may be given by appending a colon and the version identifier to the name as follows: "error_function_name:1.0" |
options | The options key should be assigned to a hash defining valid workflow options |
keylist | This key should be assigned to a list giving valid workflow keys for fast lookups of workflow order instance data |
groups | list of workflow and service groups the workflow will belong to |
statuses | a hash defining extended custom statuses descriptions. |
If the options
key is defined in the workflow definition, the workflow will advertise and accept only the options defined under this key as valid options for the workflow (aside from system options that can be overridden at the workflow level, which are always accepted as workflow options). Valid options are given as a hash assigned to this key, where the hash keys are the option names, and the values assigned to the keys are the descriptions of the options.
Here is an example of am options declaration in a Qore workflow definition file:
Options set on workflows are persistent; the option values are written to the WORKFLOW_OPTIONS
table.
The workflow definition option keylist
defines one or more "order keys" that can be used to quickly look up workflow order data instances. Any number of workflow keys may be given in the workflow definition, in contrast to the single external_order_instanceid
that can be given at the workflow order data instance object level and can be used for the same purpose (quickly looking up workflow order data instances from a external key).
Valid keys are defined as per the following examples:
Keys can be set and retrieved by the WorkflowApi::setOrderKeys() and WorkflowApi::getOrderKeys() methods (in Qore) or the WorkflowApi.setOrderKeys() and WorkflowApi.getOrderKeys() methods (in Java), respectively. Key metadata is saved in the WORKFLOW_KEYS
table, and workflow order data instance keys are saved in the ORDER_INSTANCE_KEYS
table. Note that a single key value can be saved for more than one workflow order data instance; the indexes on the ORDER_INSTANCE_KEYS
only enforce that values may not be repeated for the same workflow_instanceid
and key name.
The REST API provides methods to lookup workflow orders by workflow keys; see the following API for more information:
The workflow definition option groups
defines one or more Interface Groups that the workflow is a member of. RBAC access groups can also be declared in the workflow file using the global groups
variable (see RBAC Access Group Definition).
RBAC access group membership is declared as per the following examples:
The inter-step dependencies are defined in the steps key in the workflow definition. This section describes how to define step dependencies and simple step definitions. For details on how to define complex steps, see Step Definitions.
The basic format of this data structure is a list. At the top level, a list indicates sequential dependencies, as in the following examples:
In this example; the workflow EXAMPLE-WORKFLOW
1.0
is created. The step dependencies are linear: MyStep1
has no dependencies and therefore is the starting step, MyStep2
is dependent only on MyStep1
, and MyTestStep3
is the final step in the workflow and is dependent only on MyStep2
.
In this simple example, 3 steps will be created (or updated if the steps already exist) with the same names and version specifiers as the class names above and placed in simple linear dependency in the steps list.
"MyStep1"
version "1.0"
, "MyStep2"
version "1.0"
, and "MyStep3"
version "1.0"
must also be defined; See array steps, subworkflow steps, asynchronous steps, validation code, function definition file.It is possible to create more complex step dependencies. For example, if one of the entries in the top-level steps list is itself a list, then all entries in the sublist have the same dependency and therefore will be executed in parallel threads. Furthermore, of one of the entries in the sublist is also a list, then they will be assigned sequential dependencies (i.e.: be executed sequentially) within this sublist. Using this simple syntax, it is possible to define a complex multithreaded workflow with minimum effort.
Here are more complex examples:
In this example, MyStep1 will be executed first, then 2 threads will start in parallel. In one thread, MyStep2
and MyStep3
will be executed in sequence. In the other thread, MyStep4
will be executed. Only when all of these steps reach a OMQ::StatComplete status, will MyStep5
be executed as the final step in the workflow.
The execution diagram will look like the following:
Using this simple rule of alternating list levels for sequential and parallel execution, it is possible to create very complex multithreaded workflows with minimum effort.
The "workflow-modules"
option lists modules providing new base classes for class-based steps as in the following example.
Modules declared like this will be loaded into each workflow's Program object, and their classes can be used as base classes for class-based steps.
The "author"
value indicates the author of the workflow and will be returned with the workflow metadata in the REST API and also is displayed in the system UI.
The "remote"
flag indicates if the workflow will run as an independent qwf process communicating with other elements of Qorus Integration Engine with a distributed queue protocol rather than internally in the qorus-core process.
When workflows run in separate qwf processes, it provides a higher level of stability and control to the integration platform as a whole, as a workflow with implementation problems cannot cause the integration platform to fail.
There is a performance cost to running in separate qwf processes; workflow startup and shutdown is slightly slower, and communication with qorus-core also suffers a performance hit as all communication must be serialized and transmitted over the network.
Furthermore memory usage is significantly higher for interfaces running in separate programs, as all the common infrastructure for each interface must be duplicated in each process.
The default for this option depends on the client option qorus-client.remote (if this client option is not set, then the default value is True).
The remote value can be changed at runtime by using the following REST API: PUT /api/latest/workflows/{id_or_name}?action=setRemote
remote
flag is considered to be managed by operations, which means that once an interface has been loaded into Qorus, if its remote
flag is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the interface by oload.The workflow "autostart"
parameter sets the number of workflow execution instances to be started when the system is started; if the system should ensure that this workflow is generally running, then set this key to a value greater than zero.
If no value is provided for this option, the system will not start the workflow automatically; any workflow execution instances for this workflow must be started manually.
If a non-zero value is provided for this workflow, then the system will attempt to start the workflow at all times if all its dependencies are met, and it is not disabled. Additionally, if the workflow cannot be started for any reason (for example, due to an error in the onetimeinit logic or a dependency error), an ongoing system alert will be raised, which is only cleared when the workflow is successfully started (or the autostart parameter is set to zero).
"autostart"
value is considered to be managed by operations, which means that once a workflow has been loaded into Qorus, if its "autostart"
value is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the workflow by oload.The workflow "sla_threshold"
parameter sets the amount of time as an integer in seconds in which each workflow order should get a final status, where a final status is defined as either COMPLETE or CANCELED; if not present the default value of 30 minutes is assumed (see DefaultWorkflowSlaThreshold).
This value is used to do SLA reporting for workflow orders in the REST and WebSocket APIs.
"sla_threshold"
value is considered to be managed by operations, which means that once a workflow has been loaded into Qorus, if its "sla_threshold"
value is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the workflow by oload.The optional "max_instances" key sets the maximum number of workflow execution instances that can be running at one time; note that workflow execution instances are capable of processing up to two orders in parallel at any one time: one in "normal" mode and one in recovery mode.
To serialize workflow processing and ensure that only one order is processed at one time at any particular point in the workflow's logic, set the "max_instances"
value to one and use the synchronized
keyword on a function or method called in both normal and recovery mode or make a service call to a service method that uses threading primitives to ensure atomicity of operation (or has the service method write lock flag set).
Workflows cannot declare configuration items because workflows are just set of steps. Instead it's allowed to set the value of a step configuration item on workflow level at runtime using the operational web UI or the REST API.
onetimeinit
function. Use the WorkflowApi::updateInstanceData() method (in Qore) or the WorkflowApi.updateInstanceData() method (in Java) to save resources acquired, and WorkflowApi::getInstanceData() (in Qore) or the WorkflowApi.getInstanceData() method (in Java) to retrieve the resources during the workflow's execution.nothing
sub functionname() {}
NOTHING
(any return value will be ignored)nothing
sub functionname() {}
NOTHING
(any return value will be ignored)nothing
sub functionname(string status, *string external_order_instanceid) {}
Detach Function Parameters
Type and Name | Description |
string status | The status being set for the workflow order data instance, see Workflow, Segment, and Step Status Descriptions |
__7_ string external_order_instanceid | The external key for the workflow data being processed, if any exists, otherwise NOTHING |
NOTHING
(any return value will be ignored)nothing
sub functionname(string errcode, *hash errinfo, any opt) {}
Error Handler Function Parameters
Type and Name | Description |
string errcode | The error code string (ex: "QORE-EXCEPTION" ) |
__7_ hash errinfo | This parameter will only be present if the error has been defined by the Workflow Error Function; if so, the hash should have at least the following keys (key names in square brackets (i.e. [name]) are optional): - desc: description of the error- severity: Error Severity Codes- status: either OMQ::StatRetry (meaning that the step should be retried) or OMQ::StatError- [retry-delay]: the delay in seconds before the error should be retried (only valid when status is OMQ::StatRetry)- [business]: if True , this error represents a business error (rather than a technical error); usually implying that there is a problem with the consistency of the order data |
any opt | This is an optional parameter that can be supplied by the workflow when an error is raised. Normally it will be a string providing additional information about the error. |
NOTHING
(any return value will be ignored)hash
sub functionname() {}
"PAYMENT-MESSAGE-TIMEOUT"
), and each value is a hash with error information as given in the following description of the keys:desc:
the description of the errorseverity:
the severity code; see Error Severity Codes, default: OMQ::ES_Majorstatus:
the status code for the step due to this error; either OMQ::StatRetry or OMQ::StatError (default if not present)business:
is this a business error? default: False
retry-delay
: the default retry time for the error as an integer giving seconds (ie 1200 for 20 minutes) or a relative date/time value (i.e. 2D
+ 12h or P2D12H
, both meaning 2 days and 12 hours); the relative date/time value format is normally recommended as it's more readablelevel:
the error level value; see Error Level Type Constants for possible values; default: OMQ::ErrLevelAuto ("AUTO"
)desc
key is required. Note:"severity"
key is not present, the error has a default severity of OMQ::ES_Major ("MAJOR"
)"ERROR"
)"AUTO"
)"retry-delay"
key should only be given on errors with status set to OMQ::StatRetry ("RETRY"
)In the previous section we learned how to define simple steps where the step's name and version attributes were the same as that for the primary step code. However this is not enough to enable the development of complex workflows requiring more advanced functionality. Furthermore, as all workflows should be developed to provide comprehensive error recoverability in the case of errors, additional error-handling functionality is required from the system. This section will outline all the options for defining steps supporting the advanced functionality of Qorus.
The following diagram illustrates a subset of the attributes of a step that can be defined.
Note that each logic attribute is defined by either a step class or a function object, and the step ID is not assigned in the workflow definition file, but rather by the loader (oload).
The recommended way to define steps is by defining all the step's logic as a subclass of one of the following step classes:
Step Classes per Step Type
Step Type | Step Class |
Asynchronous Steps | Qore: QorusAsyncStep Java: QorusAsyncStep |
Workflow Synchronization Event Steps | Qore: QorusEventStep Java: QorusEventStep |
Normal Steps | Qore: QorusNormalStep Java: QorusNormalStep |
Subworkflow Step | Qore: QorusSubworkflowStep Java: QorusSubworkflowStep |
Array Asynchronous Steps | Qore: QorusAsyncArrayStep Java: QorusAsyncArrayStep |
Array Workflow Synchronization Event Steps | Qore: QorusEventArrayStep Java: QorusEventArrayStep |
Array Normal Steps | Qore: QorusNormalArrayStep Java: QorusNormalArrayStep |
Array Subworkflow Step | Qore: QorusSubworkflowArrayStep Java: QorusSubworkflowArrayStep |
In order to define class steps, each step declaration in the workflow definition file must be defined as a hash containing the following attributes:
Step Definition Hash Key Descriptions
Step Key | Type | Mand.? | Description |
name | string | Y | The name and optionally the version of the step (i.e.: "step1:1.0" ) |
version | string | N | The version of the step; this key is not required in the step definition if the step's version is given in the step's name (i.e.: "step1:1.0" ) |
desc | string | N | The description of the step; if this key is not present, then the Qorus server will return the description of the primary step code when step information is requested |
classname | string | N | The name and optional version (i.e. "MyStep1:1.0" ) of the step class, providing all the logic for a step. Note that if this key is not given, oload will assume that a function-based step is being defined |
classversion | string | N | The version of the step class if not given in the "classname" parameter |
queue | string | N | The name of the queue associated with this step (corresponding to the "Message Queue" element in the diagram above). This key is required for asynchronous steps |
arraytype | string | N | For array steps, this valid must be "SERIES" , in which case the step will be an array step |
subworkflow | bool | N | If this is set to True , then the step type attribute will be set to OMQ::ExecSubWorkflow making the step a subworkflow step. Steps may not be simultaneously subworkflow and asynchronous or workflow synchronization event steps |
eventtype | string | N | The name of the workflow event type for workflow synchronization event steps; in this case the step may not be a subworkflow or asynchronous step |
user-interaction | bool | N | If this is set to True , then the asynchronous step will support APIs for user interaction; can only be set on asynchronous steps; if not present the default value is False |
Class-based steps can have a constructor and classes can have static initialization, but please note that if the step has configuration items, it must be instantiated by oload in order to create the step's configuration in the system. In such a case, if the constructor or static class initialization requires features that are only available at runtime in Qorus itself, the errors raised will cause step class instantiation or static class instantiation to fail.
The step constructor takes no arguments.
Class-based steps can also declare configuration items to allow for the behavior of the step to be modified by users at runtime using the operational web UI or the REST API.
Step configuration items are:
Step configuration items are designed to allow users to affect the execution of a workflow so that changes can be made by authorized users in the UI without requiring a change to development.
If the strictly_local
flag on a step configuration item is False
, then the step configuration item is not local and the value can also be set on workflow or global level.
If the strictly_local
flag is False
, then the step configuration item is local and hence the value for this item cannot be set on workflow either global level.
strictly_local
flag set to False
; use the following REST API to retrieve step configuration items in the context of its declaring workflow: GET /api/latest/workflows/{id_or_name}/stepinfo/{id_or_name}/config/{name}Step configuration items in Qore are declared by overriding the QorusConfigurationItemProvider::getConfigItemsImpl() method in the step class. The step configuration items are registerted as part of the basic configuration of the step by oload when the workflow is loaded.
Each ConfigItemInfo hash in the return value of this method defines a configuration item for the step. The value of configuration items can then be retrieved and used in the workflow by calling one of the following APIs.
API Type | API | Description |
Qore | WorkflowApi::getConfigItemValue() | retrieves a single configuration item value |
Qore | WorkflowApi::getConfigItemHash() | retrieves all configuration items |
Step configuration items in Java are declared by overriding the QorusStepBase.getConfigItemsImpl() method in the step class. The step configuration items are registerted as part of the basic configuration of the step by oload when the workflow is loaded.
Each ConfigItem object in the return value of this method defines a configuration item for the step. The value of the configuration item can then be retrieved and used in the workflow by calling one of the following APIs.
API Type | API | Description |
Java | WorkflowApi.getConfigItemValue() | retrieves a single configuration item value |
Java | WorkflowApi.getConfigItemHash() | retrieves all configuration items |
Steps can also provide user-defined metadata which is returned as part of the step's description by overriding one of the following methods in the step class:
Note that step metadata is created by oload when the step is loaded, therefore this method should not rely on any runtime features of Qorus that are not available in oload.
For backwards compatibility, Qorus supports defining steps with functions defining each logic or code attribute of a step. In order to define complex function-based steps, each step declaration in the workflow definition file must be defined as a hash containing the following attributes:
Step Definition Hash Key Descriptions
Step Key | Type | Mand.? | Description |
name | string | Y | The name and optionally the version of the step (i.e.: "step1:1.0" ) |
version | string | N | The version of the step; this key is not required in the step definition if the step's version is given in the step's name (i.e.: "step1:1.0" |
desc | string | N | The description of the step; if this key is not present, then the Qorus server will return the description of the primary step code when step information is requested |
funcname | string | N | The name and version of the primary step code (corresponding to the "Step Logic" element in the diagram above). Note that if this key is not given, the step function's name and version will be assumed to be the same as the step's (as defined in the name key) |
valname | string | N | The name and version of the validation code for the step (corresponding to the "Validation Function" element in the diagram above). The validation code is run whenever the step is recovered. Note that validation code may not be defined for subworkflow or workflow synchronization event steps |
endname | string | N | The name and version of the back-end code for asynchronous steps (corresponding to the "Async Logic" element in the diagram above). Note that if this key is defined, a queue must be defined, and the step type attribute will be set to OMQ::ExecAsync. The step may not also be a subworkflow step or a workflow synchronization event step |
queue | string | N | The name of the queue associated with this step (corresponding to the "Message Queue" element in the diagram above). This key is required for asynchronous steps |
arrayname | string | N | The name and version of the array function (corresponding to the "Array Logic" element in the diagram above). If this key is defined, then the step will be an array step |
subworkflow | bool | N | If this is set to True , then the step type attribute will be set to OMQ::ExecSubWorkflow making the step a subworkflow step. Steps may not be simultaneously subworkflow and asynchronous or workflow synchronization event steps |
eventtype | string | N | The name of the workflow event type for workflow synchronization event steps; in this case the step may not be a subworkflow or asynchronous step |
user-interaction | bool | N | If this is set to True , then the asynchronous step will support APIs for user interaction; can only be set on asynchronous steps; if not present the default value is False |
A normal step is a step that is not a subworkflow, asynchronous, or workflow synchronization event step. Normal steps do not have asynchronous back-end code, a queue, or a workflow synchronization event type, and their subworkflow attribute is False
.
A normal step may be an array step, which would make it a normal array step.
The base step classes to be inherited by normal step classes are:
Qore classes:
Java classes:
See the above classes for example step definitions.
A subworkflow step binds a child workflow (called a subworkflow) to a step. The child workflow's status will be bound to the step's status; that is; whatever status the child workflow has will be reflected as the step's status. This is how Qorus supports logical branching in workflows, where one branch of processing is optionally executed based on a logical condition.
The base step classes to be inherited by subworkflow step classes are as follows.
Qore classes:
Java classes:
See the above classes for example step definitions.
Subworkflow steps are defined by setting the subworkflow attribute to True
in the workflow configuration file.
Subworkflow steps are not bound to any particular workflow type; the only rule for a subworkflow step is that one of the following API calls must be made in the primary step code for the steps.
Qore methods:
Java methods:
A subworkflow step may be an array step, which would make it a subworkflow array step.
Asynchronous steps allow Qorus to efficiently process asynchronous actions. In the context of Qorus workflow processing, asynchronous actions are actions that take a significant amount of time to complete. It is not necessary to know in advance how much time the action will take to complete to define an asynchronous step.
The base step classes to be inherited by asynchronous step classes are as follows.
Qore classes:
Java classes:
See the above classes for example step definitions.
To define an asynchronous step, the step definition must define asynchronous back-end code and a queue, and one of the following API calls must be made during the execution of the primary step code.
Qore methods:
Java methods:
QorusAsyncStepBase::submitAsyncKey() (Qore) and submitAsyncKey() (Java) (as well as the old-style Qore wf_submit_async_key() function) save a reference to the step's action as a unique key in the namespace of the queue; all keys submitted to a queue of the same name must be unique (this is enforced by an index in the Qorus database).
The system option qorus.async_delay determines when the system will retry the step if the step's queue has not been updated in time. If there is no validation code, the default behavior for the system will be to delete any queue data for the step, and re-run the step's primary step code. If this is not the desired behavior, then you must implement validation code to control how the system reacts to an asynchronous timeout.
When the result of the asynchronous action is available, code external to the workflow must call the REST API POST /api/latest/async-queues/{queue}?action=update using the name of the queue, the unique key in the queue identifying the step that created the action, and any data representing the result of asynchronous processing. Often this is done by implementing a Qorus service to monitor the result of asynchronous processing and updating the appropriate queue entry.
Within Qorus, the following is an example of how to call the REST API POST /api/latest/async-queues/{queue}?action=update from a Qorus service; in this example, a table in an external database is polled by checking which queue entries are still outstanding by calling GET /api/v3/async-queues/{queue}?action=qinfo and then updated with POST /api/latest/async-queues/{queue}?action=update .
An asynchronous step may be an array step, which would make it an asynchronous array step.
Asynchronous steps define a segment discontinuity; all normal steps leading up to the asynchronous step are in the same segment; and all normal steps executed after the asynchronous step are in another segment. Qorus workflow segments are each processed in their own thread and can process workflow order data instances independently of one another. See:
Workflow synchronization event steps allow many workflow orders to synchronize their processing based on a single event. A workflow synchronization event step may be an array step, which would make it a workflow synchronization event array step.
The base step classes to be inherited for asynchronous step classes are as follows.
Qore classes:
Java classes:
See the above classes for example step definitions.
The primary step code for a workflow synchronization event step must make one of the following API calls:
The step definition must reference a workflow synchronization event type as well; event keys bound or posted are treated unique within their event type. Additionally, workflow synchronization event steps may not have validation code.
Any type of step can be an array step. An array step is any step that may need to repeat its action more than once. To define an array step, first define the normal, asynchronous, subworkflow, or workflow synchronization event step, and then define the arraytype attribute for class-based steps as "SERIES"
(or define an array function for a function-based step).
The recommended way to define steps is by defining all the step's logic as a subclass of one of the following step classes:
Array Step Classes per Step Type
Step Type | Base Step Class |
Array Asynchronous Steps | Qore: QorusAsyncArrayStep Java: QorusAsyncArrayStep |
Array Workflow Synchronization Event Steps | Qore: QorusEventArrayStep Java: QorusEventArrayStep |
Array Normal Steps | Qore: QorusNormalArrayStep Java: QorusNormalArrayStep |
Array Subworkflow Step | Qore: QorusSubworkflowArrayStep Java: QorusSubworkflowArrayStep |
See the above classes for example step definitions.
The return value of the array code will determine how many times the step will execute, and on what data. Please note that array steps have different code signatures, as the array element is always passed to the step logic code (primary step code, validation code, and asynchronous back-end code for asynchronous steps).
The arraytype
attribute may be given in a step definition to ensure that the step is defined as an array step. For class-based steps, this step attribute is mandatory for array steps, and will ensure that one of the above step classes is used as a base class for the step's class.
Example:
funcname
key, or, if this is not present, the name of the step is assumed to be the name of the primary step code as well.Every step base class has an abstract primary()
method where the primary step logic must be defined. See the class documentation for the specific step class for more information on requirements for the primary step method.
Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a primary step function for a step.
nothing
sub functionname() {}
nothing
sub functionname(auto arrayelement) {}
NOTHING
(any return value will be ignored)see $OMQ_DIR/examples/user/TEST-WORKFLOWS/test-functions-v1.0.qfd
for further example function definitions for all function types in this section.
The following table describes how the system reacts depending on the return value of the validation logic.
Validation Code Return Value
Return Value | System Behavior |
OMQ::StatComplete | Do not run the primary step logic; mark the step as "COMPLETE" and continue. For asynchronous steps, back-end code also will not be run |
OMQ::StatError | Do not run the primary step logic; mark the step as "ERROR" and stop running any further dependencies of this step |
OMQ::StatRetry | Run the primary step loggic again immediately. If the step is an asynchronous step with queue data with a OMQ::QS_Waiting status, the queue data will be deleted immediately before the primary step logic is run again |
OMQ::StatAsyncWaiting | For asynchronous steps only, do not run the primary step logic and keep the "ASYNC-WAITING" status. For non-asynchronous steps, raises an error and the return value is treated like OMQ::StatError |
any other status | an error is raised and the return value is treated like OMQ::StatError |
Some step base classes have a validation()
method that can be overridden where the validation logic can be defined. See the class documentation for the specific step class for more information on the signature and requirements for the validation method (if it's supported for the step type, not all step types support validation logic), and see Validation Code Return Value for a description of how the return value of this method affects workflow order processing.
Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a validation function for a step.
string
sub functionname() {}
string
sub functionname(auto array_element) {}
string
sub functionname(*string async_key, auto array_element) {}
string
sub functionname(*string async_key) {}
string:
step status constantAsynchronous steps can define an end()
method to process data submitted for the asynchronous workflow event. See the class documentation for the specific step class for more information on the end() method.
Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a asynchronous back-end function for a step.
nothing
sub functionname(auto queue_data) {}
nothing
sub functionname(auto queue_data, auto array_element) {}
array_element
argument will be passed to the back end function only if the step is an array step. Otherwise only the queue_data
argument will be passed to the function; the queue_data
argument is the data that is posted on the queue when with the POST /api/latest/async-queues/{queue}?action=update call when the queue is updated.NOTHING
(any return value will be ignored)queue
tag must be set to a string giving the name of the queue for asynchronous steps. The queue will link the asynchronous step's primary step code with the back-end code.QUEUE_DATA
) and pass it to the asynchronous back-end code in order to determine the step's status.For class-based steps, the array method is an abstract method defined in all array base step classes and must be defined in the step's class.
Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a asynchronous back-end function for a step.
any
sub functionname() {}
list:
List of elements for array step; if the list is empty, then the array step is skippedany:
A single element to execute a single substep in the array stepnothing:
if NOTHING
is returned, then the array step is skippedTrue
, then the step will be a subworkflow step, and the step's status will be bound to a new workflow (the subworkflow) when the QorusSubworkflowStepBase::bindSubworkflow() method (in Qore class-based steps) or the QorusSubworkflowStepBase.bindSubworkflow() method (in Java) is called. If neither this method nor QorusSubworkflowStepBase::skipSubworkflow() (in Qore class-based steps) nor QorusSubworkflowStepBase.skipSubworkflow() (in Java) is called, then the step will exit with an error. Otherwise, if a subworkflow is bound, the step will receive a WAITING status until the subworkflow is COMPLETE or ERROR, at which time the step will be updated with the subworkflow's status.True
True
, then the asynchronous step will support APIs for user interaction; can only be set on asynchronous steps. If this key is not present in the step definition, the default value is False
.Qorus function definition files should have the extension: *.qfd
Function definition files are used to create function objects in the Qorus schema that can then be used by workflows (or loaded into service program objects as well for GENERIC functions).
Functions are defined by adding metadata tags in the form of special line comments to normal text files containing function definitions. The tags are parsed by the oload program, which will create function objects in the Qorus database according to the definitions in the file.
Function definitions begin with metadata tags as given in as specially-formatted comments as seen in the example below. After the header tags, one or more function definitions then follow, and the END
tag terminates the function definition.
Function Definition File Tags
Tag | Mand.? | Description |
type | Y | must be the first tag in a function definition, value must be one of the Step Function Types |
name | N | If the name tag is not defined, then the name of the function object in the database will be taken from the last function defined in the code block. If the name tag is defined, then a function with this name generally must be defined in the code block when used as a step function, however when used as a library function the name of the function object does not have to correspond to any function in the code block |
version | Y | Version string for the function object |
desc | Y | Description of the function object |
author | N | The optional author of the code object |
patch | N | Optional patch string – should be updated every time the same version of the function object is updated in the database |
TAG | N | Option tag definition in the format "key: value" |
END | Y | This tag must be included to terminate the function definition |
Qorus class definition files should have the extension: *.qclass
or *.qclass.java
(for Java-based classes).
Class definition files are used to create class objects in the Qorus schema that can then be used by workflows (or loaded into service program objects as well).
Like function objects, classes are defined by adding metadata tags in the form of special comments to normal text files containing class definitions. The tags are parsed by the oload program, which will create class objects in the Qorus database according to the definitions in the file.
Class definitions begin with metadata tags as given in as specially-formatted comments as seen in the examples below. After the header tags, one or more class definitions then follow, and the END
tag terminates the class definition.
Class Definition File Tags
Key | Mand.? | Description |
name | N | If the name tag is not defined, then the name of the class object will be taken from the last class defined in the code block. If the name tag is defined, then a class with this name should normally be defined in the code block, but the current version of Qorus does not enforce this |
version | Y | Version string for the class object |
desc | Y | Description of the class object |
author | N | The optional author of the code object |
patch | N | Optional patch string – should be updated every time the same version of the class object is updated in the database |
lang | N | the possible values are "qore" for Qore code (the default) and "java" for code targeting the Java 8 JVM (see Developing in Java) |
requires | N | Optional list of classes required by the current class; causes the required classes to be loaded when the current class is loaded |
TAG | N | Option tag definition in the format "key: value" ; for Java classes, entries in the classpath can be added here by adding a colon-separated list of path names to the "classpath" tag; note also that $OMQ_DIR (or any other server-side environment variable) can be used in classpath entries and will be resolved to the Qorus directory |
END | Y | This tag must be included to terminate the class definition |
classpath
tag can be used to add entries to the classpath for Java classes; $OMQ_DIR
can be used in classpath entries as in the above example and will be resolved to the Qorus directory; see Java Classpath Handling in Qorus for more informationQorus constant definition files should have the extension: *.qconst
Constant definition files are used to create constant objects in the Qorus schema that can then be used by workflows (or loaded into service program objects).
Like function and class objects, constant objects are defined by adding metadata tags in the form of special comments to normal text files containing constant definitions. The tags are parsed by the oload program, which will create constant objects in the Qorus database according to the definitions in the file.
Constant definitions begin with metadata tags as given in as specially-formatted comments as seen in the example below. After the header tags, one or more constant definitions then follow, and the END
tag terminates the constant definition.
Constant Definition File Tags
Key | Mand.? | Description |
name | Y | The name tag must be defined for constant objects |
version | Y | Version string for the constant object |
desc | Y | Description of the constant object |
author | N | The optional author of the code object |
patch | N | Optional patch string – should be updated every time the same version of the constant object is updated in the database |
TAG | N | Option tag definition in the format "key: value" |
END | Y | This tag must be included to terminate the constant definition |